Starting a Java Process in the Background from Clojure

clojure.java.shell/sh is a beautiful thing. With it, you can execute a shell process from Clojure, get its output, and generally work your will.

Unless, that is, the shell process you want to execute is a long-lived one, and you want to kick it off from Clojure and let it run in the background while you accomplish other great things with Clojure.

You can observe this behavior with a simple experiment. In a Clojure REPL:

user=> (require 'clojure.java.shell)
user=> (clojure.java.shell/sh "bash" "-c" "sleep 10 &")

Ten seconds should give you time to switch to another terminal window and verify that the process is indeed running in the background. But you will not get your REPL back until that ten seconds is up.

The thing is, sh will hang around and wait until the process finishes, whether or not it's in the background. I'm not sure just why this is. sh uses Java's Runtime.exec() to do its magic; I may update it to use ProcessBuilder.start() instead and see if that helps. I'm not hopeful.

Until then, I've found a workaround for the specific case that's been bugging me. All I have to do is redirect stdout and stderr in the command that I send to the shell. Something like:

clojure.java.shell=> (sh "bash" "-c" "sleep 10 1>/tmp/mylog.log 2>&1 &")

There's a downside to this -- normally you would be able to inspect the return value from sh and see what the stdout of the process looks like. For short-running processes, this is probably exactly what you'd want to do. For long-running processes, you probably don't want sh hanging indefinitely just so you can eventually inspect its output. So you can always open the log file and deal with the output there.

For my specific problem, though, I want to kick off the java process in the background *and* record its pid. To do that, I simply append a second command to my command string:

(sh "bash" "-c" "sleep 10 1>/tmp/mylog.log 2>&1 & \n echo $!")

Now when the shell exits, the value of the :out key of the result has the pid of the new background process. The \n is necessary because neither of the other usual ways of combining multiple commands on one line (&& and ;) seems happy when following a single ampersand.

Installing Xcode 4.3 Confuses Homebrew (Briefly)

I have a lot of catching up to do, iOS-development-wise. Don't get me wrong, I absolutely love the work I get to do at my day job, which is All Clojure All The Time. But that doesn't leave a lot of time for iOS development.

In an effort to not fall too far behind, I installed Xcode 4.3, which promptly trashed my /Developer directory (not a bad thing, really). Homebrew was supposedly updated to cope, but when I tried to brew install graphviz today, it complained about not finding the developer directory there.

I was able to convince graphviz to install without resorting to symlink-ing, using the following incantation:

sudo xcode-select -switch /Applications/Xcode.app/Contents/Developer/

clj-json breaks when running 'lein javac'

I'm working on a potential patch to Mark McGranaghan's excellent clj-json library for converting Clojure data to JSON. Unfortunately, I can't test my changes right now, because of an incompatibility between Leiningen version 1.6.2 and clj-json version 0.4.3, which causes lein javac to throw an exception.

It looks like clj-json has a dev dependency on an old version of a lein-javac plugin for Leiningen, which itself depends on ant-launcher version 1.6.5. It also looks like modern lein is smart enough to use its built-in javac task in preference to the old plugin, but it gets tripped up on the old version of ant-launcher.jar. The specific exception is:

Exception in thread "main" java.lang.NoSuchMethodError: \
org.apache.tools.ant.launch.Locator.fromJarURI(Ljava/lang/String;)\
Ljava/lang/String; (NO_SOURCE_FILE:0)

I've submitted an issue on the clj-json project, but for now a simple workaround is to delete the dev dependency from clj-json's project.clj file, remove everything from its lib/ directory, and run lein deps to refresh dependencies.

Pro Tip: Fullscreen-ness of Apps Seems to be Sticky in Mac OS Lion

I've had a lot of fun the past couple days working with the Terminal app in fullscreen mode. Having entered fullscreen, I can work without distraction, but I can also switch freely back to my desktop or other running apps. Best of all: when I quit Terminal and restart it, it automatically restarts in fullscreen mode. Good job, Apple!

Installing Java on AWS EC2 Instances with Pallet

I have recently been spinning up a lot of t1.micro instances on Amazon's EC2 service. I'm deliberately using minimal / default images, and I need to install Java on them from scratch. There have been reports (and I have experienced myself) that the Java install process hangs indefinitely when installing either Oracle's JRE/JDK or OpenJDK on a t1.micro running Ubuntu. I have been able to install Java successfully by explicitly requiring a 64-bit version of the OS.

Here's how you can do that in your Pallet node-spec:

Starting an EC2 Instance with an Admin User Using Pallet

At my new day job today, I learned the hard way that if you're using Pallet to create an EC2 instance, and you want to use its automated-admin-user crate (which authorizes the user running the crate to login on the new instance via SSH using your RSA key), you really *really* want to run that crate during the :bootstrap phase. Running during the :configure phase does not seem happy using my current configuration of Pallet version 0.6.4 and jclouds version 1.0.0. Here's the relevant Clojure snippet:

(require 'pallet.core)
(require 'pallet.phase)
(require 'pallet.crate.automated-admin-user)

(def server-spec
  (pallet.core/server-spec
   :node-spec node-spec
   :phases {:bootstrap (pallet.phase/phase-fn (pallet.crate.automated-admin-user/automated-admin-user))}))

P.S.: LonoCloud is hiring Clojure developers! Join us!

Implementing Continuations in iOS Using Blocks

Came up with a neat use for blocks in iOS at my day job today: continuations!

Like this: I'm working on a native iOS app that occasionally shows data from a Rails website in a UIWebView. When it does so, it has to make sure the user of the iOS app is authenticated to the web app. This way we should never have to see the web app's login page in the native app, which has its own login screen.

Of course, all of this reeks of asynchronicity. Which usually means callbacks. Which usually means implementing one or twelve protocols. We've tried to encapsulate all of the authentication logic in a custom class that's used by the controller that presents the UIWebViews, but we've also recently come up with a need for multiple controllers, controlling multiple web views. And the code is already (ahem) not as attractive as it could be.

Our original workflow looks something like this:

* Render a UIWebView and invoke its loadRequest: method with some URL.
* Have the controller implement the UIWebViewDelegate protocol.
** In particular, implement webView:shouldStartLoadWithRequest:navigationType:.
** Ask the authenticator object if we need to refresh authentication.
*** If not, just load the request (happy path).
*** If so, tell the authenticator object to refresh authentication and return NO.
*** This may require collecting new authentication credentials from the user, which means another controller and view (and asynchronous request cycle) entirely.
*** It certainly requires a separate asynchronous call to the web app, to verify authentication and set the timeout for this session.
*** Oh yeah -- we need to have remembered the original request and restart it when the dust settles.
* Insanity ensues, in the form of delegates with delegates and very hard-to-follow callback trails and state management.

Enter: blocks.

With blocks we can streamline, thusly (starting with webView:shouldStartLoadWithRequest:...):

* Tell the authenticator object to make sure we're authenticated, passing in a block to execute once we are.
** This block closes over all the state we need to "do the next thing". It can refer to methods in the current controller and access the current controller's instance variables, even though it'll be executed by the authenticator object.
* If the authenticator decides we're already authenticated, it simply executes the block (the happy path).
* If we're not authenticated, but we already know our credentials, open a NSURLConnection to the web app, using the usual callbacks. When that connection finishes successfully, execute the block we were given from the original controller.
* If we're not authenticated and we need new credentials, push the login screen controller on the stack, passing *it* a block that does the NSURLConnection stuff, followed by executing the block from the original controller.
* The login controller presents a modal dialog, accepts the user's input for authentication credentials, then calls its block (the one passed from the previous step, which encapsulates "what to do next" at two levels).

When we can implement this strategy pervasively, every entity will simply ask its collaborators to do whatever they need to do, however they need to do it (including asynchronously), and know that once the task of e.g., login, is accomplished, whatever is supposed to happen next actually does.

This idea was inspired by Continuation Passing Style in compiler theory, in which every function is called with the data it needs to perform its own business, *plus* information about what to do next. When programs are expressed in this style, there is no recursion, no nesting of routines; there is only "do what you're supposed to do, then jump to the next thing". It's not a perfect analogy, but it has been surprisingly effective in making some pretty complex and ugly code easier to understand.