Executing external command from within JVM often causes problems- be it in terms of the code to write and manage or in the ease of implementation. I had similar requirement in my Major project for my Under Graduate Degree, where in I had to launch a C program from the Java code. I ran into different issues like- the Main thread getting blocked, the GUI freezing, or reading the output streams and so on. Finally I had to give up the idea and stick with launching the external command externally 😛 Had I found the Exec library from Apache Commons then, my work would have been lot easier. Anyways better late then never. I will quickly go through how one can use Exec library to launch external programs from JVM- Its a wrapper over Java’s ProcessBuilder, Runtime.getRuntime().exe(). The Javadoc shows lots of classes- but important one among them- which the end user would mostly be using are:
- DefaultExecutor (A default implementation for Executor Interface)
- DefaultExecuteResultHandler (A default implementation for ExecuteResultHandler interface)
- ExecuteWatchdog
- PumpStreamHandler
- CommandLine
- EnvironmentUtils
There are other CommandLauncher classes which are internally called by when one uses DefaultExecutor, though the methods of these classes are available for one to use them directly.
Using Exec helps us to:
- Set environment variables- Current Java implementation provides- System.getEnv() or System.getEnv(String) but no System.setEnv(). Yet the exec() method of Runtime class is overloaded to accept the Environment variables. This can at time lead to confusion. Exec provides an EnvironmentUtils class which takes care of environment variable related tasks.
- Easily set the Stream handlers- There’s a class PumpStreamHandler which can take- Output Stream, Input Stream and error stream. It captures the data in different streams via creating different Threads for each stream. So one need not worry about creating threads for each stream.
- Build the Command using CommandLine class.
- Execute the command- Synchronously or Asynchronously.
- Associate watchdogs for each command- So that these commands dont take too much of time.
- Destroying the process when the JVM exits.
Lets have a look at the implementation of the library.
Building the Command-Uses CommandLine class.
//Just the Executable, No arguments CommandLine command = new CommandLine("ls"); command.addArgument(String) //Adding a String argument command.addArgument("${arg1}"); //Expands the arg1 value from the Map //map used to expand the arguments like above command.setSubstitutionMap(Map); command.addArguments(String[]) //Adding all the arguments at one go
Now we need to set the Stream handler- PumpStreamHandler
//Takes System.out for dumping the output and System.err for Error PumpStreamHandler streamHandler = new PumpStreamHander(); //Create a new FileOutputStream instance //Sends the output and error to the file streamHandler = new PumpStreamHandler(FileOutputStream);
In this way PumpStreamHandler provides overloaded Constructors. Also note that internally it creates different Threads for each Stream viz Output, Input, Error.
Now we should be executing the command- DefaultExecutor– There are lot of variations in this. Lets start with the Simple-
DefaultExecutor executor = new DefaultExecutor(); executor.setStreamHandler(PumpStreamHandler); //Sets the stream handler executor.execute(CommandLine);//Executes the command
Lets apply the above to create a simple “ls” on Linux systems.
CommandLine command = new CommandLine("ls"); PumpStreamHandler streamHandler = new PumpStreamHandler(); DefaultExecutor executor = new DefaultExecutor(); executor.setStreamHandler(streamHandler); executor.execute(command);
Now lets execute the same Asynchronously- uses DefaultExeuteResultHandler class. I would be using the sleep command as that will clearly show the difference
CommandLine command = new CommandLine("sleep"); command.addArgument("10");//Number of seconds to sleep System.out.println("Before Sleep"); DefaultExecuteResultHandler resultHandler = new DefaultExecuteResultHandler(); DefaultExecutor executor = new DefaultExecutor(); PumpStreamHandler streamHandler = new PumpStreamHandler(); executor.setStreamHandler(streamHandler); executor.execute(command,resultHandler); System.out.println("After Sleep"); //Use of resultHandler makes this a Asynch process
You can see that there’s now time interval between the two print statements. If we add this line after Line 09-
//Wait for the command to finish execution. resultHandler.waitFor(); //Obtain the exit value of the command. int exitValue = resultHandler.getExitValue();
Then there’s a delay- which means the main thread waits for the sub process to complete before proceeding- waitFor() internally uses Thread.sleep(time). Also the exitValue is captured which can be used for:
//testing if the exitValue indicates failure if(executor.isFailure(exitValue){ System.out.println("Command execution failed"); }else{ System.out.println("Command execution Successful"); }
ifFailure() checks if the exitValue obtained after executing the command denotes a Success (exitValue=0) or a Failure (any other exitValue). One can also set exitValue for the executor using setExitValue(int) or setExitValues(int[]) in that case the exitValue for success will not be “0” but the value specified. One can use waitFor() if the command has to be executed before the program continues execution. Otherwise in cases where the user wants to execute the command and store the result in a File waitFor() will not be required.
Sometimes the command might continue executing for a long time and we need to keep a check to see that it doesnt exceed the time. In those cases ExecuteWatchdog class can be used. Whenever the process terminates due to the timeout specified in WatchDog, the exit value returned will be for that of a failure.
ExecuteWatchdog watchDog = new ExecuteWatchdog(timeout); exeuctor.setWatchDog(watchDog);//setting the watchdog for the executor //The time out can also be ExecuteWatchdog.INFINITE_TIMEOUT
Summarizing:
- DefaultExecutor– Class for launching/executing the commands. Can set TIME_OUT, StreamHandler, ResultHandler, Working Directory for the command.
- DefaultExecuteResultHandler– Monitors the process when its launched Asynchronously. Can be used to get the Exception, Exit_Value for the process.
- PumpStreamHandler– Manager the Output,Error,Input Stream of the Process- Each of them in a separate thread.
- ExecuteWatchdog– Used for setting the timeout for the command.
- EnvironmentUtils– Utility class to work with the Environment variables
- CommandLine– For setting up the executable, and its arguments. Can take the argument values from a Map.
Note: Most of the classes are default implementations of their interfaces
Let me give a sample code which uses all of the above:
//Command to be executed CommandLine command = new CommandLine("ls"); //Adding its arguments command.addArguments(new String[]{"-l","-t","-R"}); //Infinite timeout ExecuteWatchdog watchDog = new ExecuteWatchdog(ExecuteWatchdog.INFINITE_TIMEOUT); //Result Handler for executing the process in a Asynch way DefaultExecuteResultHandler resultHandler = new DefaultExecuteResultHandler(); //Using Std out for the output/error stream PumpStreamHandler streamHandler = new PumpStreamHandler(); //This is used to end the process when the JVM exits ShutdownHookProcessDestroyer processDestroyer = new ShutdownHookProcessDestroyer(); //Our main command executor DefaultExecutor executor = new DefaultExecutor(); //Setting the properties executor.setStreamHandler(streamHandler); executor.setWatchdog(watchDog); //Setting the working directory //Use of recursion along with the ls makes this a long running process executor.setWorkingDirectory(new File("/home")); executor.setProcessDestroyer(processDestroyer); //Executing the command executor.execute(command,resultHandler); //The below section depends on your need //Anything after this will be executed only when the command completes the execution resultHandler.waitFor(); int exitValue = resultHandler.getExitValue(); System.out.println(exitValue); if(executor.isFailure(exitValue)){ System.out.println("Execution failed"); }else{ System.out.println("Execution Successful"); }
Note: The Exec library has to be in your classpath- Either add it in CLASSPATH env variable or use java -cp
Please drop in any corrections, additional information as comments. I will update the post accordingly
Categories: Java, Open Source
I don’t see any obvious advantages over using the standard Java SE classes.
Moreover, you CAN set environment variables, using the java.lang.ProcessBuilder#environment() method, which returns a Map. Simply add extra variables to this map.
If we explore the Exec API – We can find lot of advantages. There are features like-
>> Associate watchdogs for each command- Kill the process if its taking too much of time or time more than what we would be expecting
>> Easily set the Stream handlers which internally are multi threaded- So we need not worry about handling these stream readers in a multi threaded way.
>> Execute the commands asynchronously
These are just few which I could list from what I have implemented.
And this article: http://www.javaworld.com/jw-12-2000/jw-1229-traps.html explains certain pitfalls associated with Runtime.exec(). So according to me- Exec libraries tries to overcome few of these.
You shouldn’t need Runtime.exec() anymore anyway. Just use Process & ProcessBuilder.
Moreover, in 98% of the cases, you don’t need those few “extras” the library has to offer.
Sorry, but you can’t convince me to try this out 🙂
i think groovy adds similar features to Java. How do you compare them ?
I haven’t tried Groovy. Do add in if you have worked on Groovy. If I can- I will try to learn and see how it compares with Exec library.
Thank you for this really nice breakdown of the apache’s commons-exec libraries. I was wonderign if you know about and wouldn’t mind enhancing your article to cover inputStreamPumper?
Thanks for reading the article.
I dont know about InputStreamPumper, I will look into it and try to enhance the article. Thanks for the suggestion.
How can I get a java.lang.String from the PumpStreamHandler?? or even a InputStream?
the issue is that i cannot directly get a java.io.InputStream object from this. And to set one up, i need to implement the abstract read() method.
I need to get the output of a process to parse it with regex….
int exitValue = resultHandler.waitFor() returns void.
It should be resultHandler.getExitValue()
Thanks for pointing out the change. With the latest release of Apache Exec this change has been update.
I have updated the post with the correct code.
Mohamed:
Great article. Does a great job of explaining this useful library. Thanks for sharing it!