Wednesday, July 25, 2012

Calling a Clojure Function With :keys Destructuring From Java

I had the occasion to do some hardcore Java-on-Clojure interop at my day job today.

Like this: we have a Clojure function with a signature like

(defn clojure-function [& {:keys [host port]
                           :or {host "localhost", port 8080}}]
  (println "host:" host)
  (println "port:" port))
So parameters to clojure-function are expected to take the form (:host "myhost" :port 1234). Via the magic of the :keys directive, the values are bound to local variables named "host" and "port", corresponding to the keywords :host and :port from the input. The :or bit provides default values if one or the other (or both) of the expected keys is not specified.

This is a very useful idiom in Clojure, one which permits named or "keyword" parameters, which I think are easier to deal with than regular positional parameters. But the need to invoke this method from Java poses some ... shall we say "fun" challenges.

Challenge the First: There is nothing in Java that corresponds to the Clojure keyword. Try to invoke a method like LonoCloudClass.clojure-function(:host, "myhost", :port, myport) and the compiler will immediately complain that :host is invalid syntax. Great. So we'll probably want to pass those things in as strings.

Challenge the Second: The :keys directive and behavior implies a key-value mapping. So it would seem reasonable to pass in an associative data structure like, say, an instance of java.util.Map. Or even a Clojure map literal. That's easy enough to do from the Java side, but, perversely, the Clojure side then complains with a message like "java.lang.IllegalArgumentException: No value supplied for key: {:host "somehost", :port 1234}".

Notice what the exception message claims is the key with no value: it's the entire map! So the map is not what gets destructured by the :keys directive; what gets destructured is a sequence of things that can be grouped in pairs. The first item of each pair needs to be a Clojure keyword, which gets a local variable named after it. The second item of each pair becomes the value of that variable.

Armed with this knowledge, we can take a shot at meeting both of these challenges.

On the Java side:

- Build a java.util.Map with java.util.String instances as keys and some Object type as values.

- Invoke -javaFunction (see definition below) with the Map as its parameter

On the Clojure side:

- Write a function (-javaFunction) that takes a java.util.Map as its only parameter

- Convert the map into a Clojure sequence in which every other item is a keyword

- Pass the sequence as the parameters to the actual Clojure function

(defn- keywordize [coll]
  (apply concat (for [[k v] (into {} coll)] [(keyword k) v])))

(defn -javaFunction [args]
  (apply clojure-function (keywordize args)))

Note that in order to complete the cycle you will need to AOT (ahead-of-time compile) the Clojure namespace that defines -javaFunction and clojure-function. You will also need to add a :gen-class directive to your namespace declaring javaFunction as a static method in the generated Java class. Something like this:

 :methods [#^{:static true} [javaFunction [java.util.Map] void]])

Also note that the strings that serve as the keys/keywords need to not be prefaced by a colon on the Java side. The Clojure function (keyword "thing") will return a result of :thing, which is exactly what we want in this case.

Friday, May 18, 2012

Building Modern Clojure-now-with-reducers! with JDK 6

The Clojure times, they are a-changin'. Time was, an enterprising young hacker could just clone the Clojure repo from GitHub, run ant to build it, then hack h[er|is] way to [fame|glory]. Now, with the advent of reducers, those days are sadly gone. Or at least slightly delayed.

You see, reducers require some ultra-modern Java-isms; namely, java.util.concurrent.ForkJoin[Pool|Task]. If you have JDK 7 installed as your default JDK (or can arrange to so install it), you have everything you need and your Clojure build should Just Work™.

If you're stuck with JDK 6, you may have a bit more work to do. Your main symptom will be that your Clojure build does *not* Just Work™, but rather fails with an exception of the form java.lang.ClassNotFoundException: jsr166y.ForkJoinPool.

The Clojure Maven build now declares a dependency on an artifact called jsr166y, which provides an implementation of the needed ForkJoin... classes. Running mvn package will detect the new dependency, download the correct jar to your ~/.m2/ directory, and include that jar on the build classpath.

If you prefer to use ant instead of Maven (as I do), you have yet more work to do.

One thing you could do is download the jsr166y jar, stash it somewhere, and manually add it to your classpath. That seems kind of tedious to me; but I guess your tolerance level for tedium could be higher than your tolerance level for Maven.

Luckily, the Clojure core team has anticipated your preference for ant, and they want to alleviate your tedium. Unluckily, their solution actually uses Maven behind the scenes; you ant purists may want to stop reading at this point and just go download the jar and add it to your classpath.

Right before they committed the changes to implement reducers, the Clojure core team added the Maven dependency on jsr166y. Also, for medium-core ant fans, they added a one-time script at the top level of the project: Run that script once to create a new file: (and download the jsr166y jar from some Maven repo if you don't already have it). The ant build now includes that properties file and adds the classpath it specifies to the ant build classpath and you're good to go.

Oh. Except for the whole having to sit on the sidewalk ringing a bell and chanting "Unclean!" at regular intervals. But that's a small price to pay for Clojure [fame|glory], right?

Monday, March 05, 2012

Starting a Java Process in the Background from Clojure is a beautiful thing. With it, you can execute a shell process from Clojure, get its output, and generally work your will.

Unless, that is, the shell process you want to execute is a long-lived one, and you want to kick it off from Clojure and let it run in the background while you accomplish other great things with Clojure.

You can observe this behavior with a simple experiment. In a Clojure REPL:
user=> (require '
user=> ( "bash" "-c" "sleep 10 &")

Ten seconds should give you time to switch to another terminal window and verify that the process is indeed running in the background. But you will not get your REPL back until that ten seconds is up.

The thing is, sh will hang around and wait until the process finishes, whether or not it's in the background. I'm not sure just why this is. sh uses Java's Runtime.exec() to do its magic; I may update it to use ProcessBuilder.start() instead and see if that helps. I'm not hopeful.

Until then, I've found a workaround for the specific case that's been bugging me. All I have to do is redirect stdout and stderr in the command that I send to the shell. Something like:> (sh "bash" "-c" "sleep 10 1>/tmp/mylog.log 2>&1 &")

There's a downside to this -- normally you would be able to inspect the return value from sh and see what the stdout of the process looks like. For short-running processes, this is probably exactly what you'd want to do. For long-running processes, you probably don't want sh hanging indefinitely just so you can eventually inspect its output. So you can always open the log file and deal with the output there.

For my specific problem, though, I want to kick off the java process in the background *and* record its pid. To do that, I simply append a second command to my command string:
(sh "bash" "-c" "sleep 10 1>/tmp/mylog.log 2>&1 & \n echo $!")

Now when the shell exits, the value of the :out key of the result has the pid of the new background process. The \n is necessary because neither of the other usual ways of combining multiple commands on one line (&& and ;) seems happy when following a single ampersand.

Saturday, February 18, 2012

Installing Xcode 4.3 Confuses Homebrew (Briefly)

I have a lot of catching up to do, iOS-development-wise. Don't get me wrong, I absolutely love the work I get to do at my day job, which is All Clojure All The Time. But that doesn't leave a lot of time for iOS development.

In an effort to not fall too far behind, I installed Xcode 4.3, which promptly trashed my /Developer directory (not a bad thing, really). Homebrew was supposedly updated to cope, but when I tried to brew install graphviz today, it complained about not finding the developer directory there.

I was able to convince graphviz to install without resorting to symlink-ing, using the following incantation:
sudo xcode-select -switch /Applications/

Saturday, December 10, 2011

clj-json breaks when running 'lein javac'

I'm working on a potential patch to Mark McGranaghan's excellent clj-json library for converting Clojure data to JSON. Unfortunately, I can't test my changes right now, because of an incompatibility between Leiningen version 1.6.2 and clj-json version 0.4.3, which causes lein javac to throw an exception.

It looks like clj-json has a dev dependency on an old version of a lein-javac plugin for Leiningen, which itself depends on ant-launcher version 1.6.5. It also looks like modern lein is smart enough to use its built-in javac task in preference to the old plugin, but it gets tripped up on the old version of ant-launcher.jar. The specific exception is:
Exception in thread "main" java.lang.NoSuchMethodError: \;)\
Ljava/lang/String; (NO_SOURCE_FILE:0)

I've submitted an issue on the clj-json project, but for now a simple workaround is to delete the dev dependency from clj-json's project.clj file, remove everything from its lib/ directory, and run lein deps to refresh dependencies.

Sunday, November 13, 2011

Pro Tip: Fullscreen-ness of Apps Seems to be Sticky in Mac OS Lion

I've had a lot of fun the past couple days working with the Terminal app in fullscreen mode. Having entered fullscreen, I can work without distraction, but I can also switch freely back to my desktop or other running apps. Best of all: when I quit Terminal and restart it, it automatically restarts in fullscreen mode. Good job, Apple!

Friday, November 11, 2011

Installing Java on AWS EC2 Instances with Pallet

I have recently been spinning up a lot of t1.micro instances on Amazon's EC2 service. I'm deliberately using minimal / default images, and I need to install Java on them from scratch. There have been reports (and I have experienced myself) that the Java install process hangs indefinitely when installing either Oracle's JRE/JDK or OpenJDK on a t1.micro running Ubuntu. I have been able to install Java successfully by explicitly requiring a 64-bit version of the OS.

Here's how you can do that in your Pallet node-spec: