Building Modern Clojure-now-with-reducers! with JDK 6

The Clojure times, they are a-changin'. Time was, an enterprising young hacker could just clone the Clojure repo from GitHub, run ant to build it, then hack h[er|is] way to [fame|glory]. Now, with the advent of reducers, those days are sadly gone. Or at least slightly delayed.

You see, reducers require some ultra-modern Java-isms; namely, java.util.concurrent.ForkJoin[Pool|Task]. If you have JDK 7 installed as your default JDK (or can arrange to so install it), you have everything you need and your Clojure build should Just Work™.

If you're stuck with JDK 6, you may have a bit more work to do. Your main symptom will be that your Clojure build does *not* Just Work™, but rather fails with an exception of the form java.lang.ClassNotFoundException: jsr166y.ForkJoinPool.

The Clojure Maven build now declares a dependency on an artifact called jsr166y, which provides an implementation of the needed ForkJoin... classes. Running mvn package will detect the new dependency, download the correct jar to your ~/.m2/ directory, and include that jar on the build classpath.

If you prefer to use ant instead of Maven (as I do), you have yet more work to do.

One thing you could do is download the jsr166y jar, stash it somewhere, and manually add it to your classpath. That seems kind of tedious to me; but I guess your tolerance level for tedium could be higher than your tolerance level for Maven.

Luckily, the Clojure core team has anticipated your preference for ant, and they want to alleviate your tedium. Unluckily, their solution actually uses Maven behind the scenes; you ant purists may want to stop reading at this point and just go download the jar and add it to your classpath.

Right before they committed the changes to implement reducers, the Clojure core team added the Maven dependency on jsr166y. Also, for medium-core ant fans, they added a one-time script at the top level of the project: antsetup.sh. Run that script once to create a new file: maven-classpath.properties (and download the jsr166y jar from some Maven repo if you don't already have it). The ant build now includes that properties file and adds the classpath it specifies to the ant build classpath and you're good to go.

Oh. Except for the whole having to sit on the sidewalk ringing a bell and chanting "Unclean!" at regular intervals. But that's a small price to pay for Clojure [fame|glory], right?

Starting a Java Process in the Background from Clojure

clojure.java.shell/sh is a beautiful thing. With it, you can execute a shell process from Clojure, get its output, and generally work your will.

Unless, that is, the shell process you want to execute is a long-lived one, and you want to kick it off from Clojure and let it run in the background while you accomplish other great things with Clojure.

You can observe this behavior with a simple experiment. In a Clojure REPL:

user=> (require 'clojure.java.shell)
user=> (clojure.java.shell/sh "bash" "-c" "sleep 10 &")

Ten seconds should give you time to switch to another terminal window and verify that the process is indeed running in the background. But you will not get your REPL back until that ten seconds is up.

The thing is, sh will hang around and wait until the process finishes, whether or not it's in the background. I'm not sure just why this is. sh uses Java's Runtime.exec() to do its magic; I may update it to use ProcessBuilder.start() instead and see if that helps. I'm not hopeful.

Until then, I've found a workaround for the specific case that's been bugging me. All I have to do is redirect stdout and stderr in the command that I send to the shell. Something like:

clojure.java.shell=> (sh "bash" "-c" "sleep 10 1>/tmp/mylog.log 2>&1 &")

There's a downside to this -- normally you would be able to inspect the return value from sh and see what the stdout of the process looks like. For short-running processes, this is probably exactly what you'd want to do. For long-running processes, you probably don't want sh hanging indefinitely just so you can eventually inspect its output. So you can always open the log file and deal with the output there.

For my specific problem, though, I want to kick off the java process in the background *and* record its pid. To do that, I simply append a second command to my command string:

(sh "bash" "-c" "sleep 10 1>/tmp/mylog.log 2>&1 & \n echo $!")

Now when the shell exits, the value of the :out key of the result has the pid of the new background process. The \n is necessary because neither of the other usual ways of combining multiple commands on one line (&& and ;) seems happy when following a single ampersand.

Installing Xcode 4.3 Confuses Homebrew (Briefly)

I have a lot of catching up to do, iOS-development-wise. Don't get me wrong, I absolutely love the work I get to do at my day job, which is All Clojure All The Time. But that doesn't leave a lot of time for iOS development.

In an effort to not fall too far behind, I installed Xcode 4.3, which promptly trashed my /Developer directory (not a bad thing, really). Homebrew was supposedly updated to cope, but when I tried to brew install graphviz today, it complained about not finding the developer directory there.

I was able to convince graphviz to install without resorting to symlink-ing, using the following incantation:

sudo xcode-select -switch /Applications/Xcode.app/Contents/Developer/

clj-json breaks when running 'lein javac'

I'm working on a potential patch to Mark McGranaghan's excellent clj-json library for converting Clojure data to JSON. Unfortunately, I can't test my changes right now, because of an incompatibility between Leiningen version 1.6.2 and clj-json version 0.4.3, which causes lein javac to throw an exception.

It looks like clj-json has a dev dependency on an old version of a lein-javac plugin for Leiningen, which itself depends on ant-launcher version 1.6.5. It also looks like modern lein is smart enough to use its built-in javac task in preference to the old plugin, but it gets tripped up on the old version of ant-launcher.jar. The specific exception is:

Exception in thread "main" java.lang.NoSuchMethodError: \
org.apache.tools.ant.launch.Locator.fromJarURI(Ljava/lang/String;)\
Ljava/lang/String; (NO_SOURCE_FILE:0)

I've submitted an issue on the clj-json project, but for now a simple workaround is to delete the dev dependency from clj-json's project.clj file, remove everything from its lib/ directory, and run lein deps to refresh dependencies.

Pro Tip: Fullscreen-ness of Apps Seems to be Sticky in Mac OS Lion

I've had a lot of fun the past couple days working with the Terminal app in fullscreen mode. Having entered fullscreen, I can work without distraction, but I can also switch freely back to my desktop or other running apps. Best of all: when I quit Terminal and restart it, it automatically restarts in fullscreen mode. Good job, Apple!

Installing Java on AWS EC2 Instances with Pallet

I have recently been spinning up a lot of t1.micro instances on Amazon's EC2 service. I'm deliberately using minimal / default images, and I need to install Java on them from scratch. There have been reports (and I have experienced myself) that the Java install process hangs indefinitely when installing either Oracle's JRE/JDK or OpenJDK on a t1.micro running Ubuntu. I have been able to install Java successfully by explicitly requiring a 64-bit version of the OS.

Here's how you can do that in your Pallet node-spec:

Starting an EC2 Instance with an Admin User Using Pallet

At my new day job today, I learned the hard way that if you're using Pallet to create an EC2 instance, and you want to use its automated-admin-user crate (which authorizes the user running the crate to login on the new instance via SSH using your RSA key), you really *really* want to run that crate during the :bootstrap phase. Running during the :configure phase does not seem happy using my current configuration of Pallet version 0.6.4 and jclouds version 1.0.0. Here's the relevant Clojure snippet:

(require 'pallet.core)
(require 'pallet.phase)
(require 'pallet.crate.automated-admin-user)

(def server-spec
  (pallet.core/server-spec
   :node-spec node-spec
   :phases {:bootstrap (pallet.phase/phase-fn (pallet.crate.automated-admin-user/automated-admin-user))}))

P.S.: LonoCloud is hiring Clojure developers! Join us!