Monday, March 05, 2012

Starting a Java Process in the Background from Clojure

clojure.java.shell/sh is a beautiful thing. With it, you can execute a shell process from Clojure, get its output, and generally work your will.

Unless, that is, the shell process you want to execute is a long-lived one, and you want to kick it off from Clojure and let it run in the background while you accomplish other great things with Clojure.

You can observe this behavior with a simple experiment. In a Clojure REPL:
user=> (require 'clojure.java.shell)
user=> (clojure.java.shell/sh "bash" "-c" "sleep 10 &")

Ten seconds should give you time to switch to another terminal window and verify that the process is indeed running in the background. But you will not get your REPL back until that ten seconds is up.

The thing is, sh will hang around and wait until the process finishes, whether or not it's in the background. I'm not sure just why this is. sh uses Java's Runtime.exec() to do its magic; I may update it to use ProcessBuilder.start() instead and see if that helps. I'm not hopeful.

Until then, I've found a workaround for the specific case that's been bugging me. All I have to do is redirect stdout and stderr in the command that I send to the shell. Something like:
clojure.java.shell=> (sh "bash" "-c" "sleep 10 1>/tmp/mylog.log 2>&1 &")

There's a downside to this -- normally you would be able to inspect the return value from sh and see what the stdout of the process looks like. For short-running processes, this is probably exactly what you'd want to do. For long-running processes, you probably don't want sh hanging indefinitely just so you can eventually inspect its output. So you can always open the log file and deal with the output there.

For my specific problem, though, I want to kick off the java process in the background *and* record its pid. To do that, I simply append a second command to my command string:
(sh "bash" "-c" "sleep 10 1>/tmp/mylog.log 2>&1 & \n echo $!")

Now when the shell exits, the value of the :out key of the result has the pid of the new background process. The \n is necessary because neither of the other usual ways of combining multiple commands on one line (&& and ;) seems happy when following a single ampersand.

8 comments:

BCS said...

& can also end a command:

$ sleep 10 & echo $!
[2] 4962
4962
$ pgrep sleep
4962

David Rupp said...

@BCS: Nice; did not know (and would not have guessed) that one. Thanks!

Alex Ott said...

Hi David...
Can you add clojure label to this post, so it will propagated to Planet Clojure

David Rupp said...

Alex: It is done. Thanks for that tip. :-)

Rayne said...

Howdy. Not sure if you knew about it or not, but I wrote a little library (to complement clojure.java.shell) that likely does exactly what you want. https://github.com/Raynes/conch.

David Rupp said...

@Rayne: Very nice. I had not seen this before; thanks for pointing it out!

Carsten said...

Thank you. This is exactly what i was looking for. Great work.

David Rupp said...

@Carsten: I'm glad this was helpful for you, even after all this time. And I apologize for the delay in publishing your comment; I'd forgotten that I'd turned on moderation.