Piping Between Processes

In my last post Executing Operating System Commands I described using java.lang.Process and java.lang.ProcessBuilder to execute operating system processes from Java and other JVM languages.

Another regular task with operating system processes is piping: reading data from one process’ output — the stdout — and writing to another process’ input — the stdin. The following shows grepping a string from a process list:

$ ps ax | grep java
32228   ??  S      0:00.07 <a class="zem_slink" href="http://www.gnu.org/software/bash/" title="Bash (Unix shell)" rel="homepage" target="_blank">/bin/bash</a> /Applications/NetBeans/NetBeans 6.9.1.app/Contents/MacOS/ ...
[...]

Class Piper

We can do this in Java too by setting up a thread which reads and writes a stream as long as data is available. I created a class named Piper:

public class Piper implements java.lang.Runnable {

    private java.io.InputStream input;

    private java.io.OutputStream output;

    public Piper(java.io.InputStream input, java.io.OutputStream output) {
        this.input = input;
        this.output = output;
    }

    public void run() {
        try {
            // Create 512 <a class="zem_slink" href="http://en.wikipedia.org/wiki/Byte" title="Byte" rel="wikipedia" target="_blank">bytes</a> buffer
            byte[] b = new byte[512];
            int read = 1;
            // As long as data is read; -1 means <a class="zem_slink" href="http://en.wikipedia.org/wiki/End-of-file" title="End-of-file" rel="wikipedia" target="_blank">EOF</a>
            while (read &gt; -1) {
                // <a class="zem_slink" href="http://en.wikipedia.org/wiki/Read_%28system_call%29" title="Read (system call)" rel="wikipedia" target="_blank">Read</a> bytes into buffer
                read = input.read(b, 0, b.length);
                //System.out.println("read: " + new String(b));
                if (read &gt; -1) {
                    // Write bytes to output
                    output.write(b, 0, read);
                }
            }
        } catch (Exception e) {
            // Something happened while reading or writing streams; pipe is broken
            throw new RuntimeException("Broken pipe", e);
        } finally {
            try {
                input.close();
            } catch (Exception e) {
            }
            try {
                output.close();
            } catch (Exception e) {
            }
        }
    }

}

To use it, setup two processes, give their streams to an instance of Piper and wait for the second process to finish and grab its output.

A test:

public class PipeTest {

    public static void main(String[] args) throws java.io.IOException, java.lang.InterruptedException {
        java.lang.Runtime rt = java.lang.Runtime.getRuntime();
        // Start two processes: ps ax | grep rbe
        java.lang.Process p1 = rt.exec("ps ax");
        // grep will wait for input on stdin
        java.lang.Process p2 = rt.exec("grep rbe");
        // Create and start Piper
        Piper pipe = new Piper(p1.getInputStream(), p2.getOutputStream());
        new Thread(pipe).start();
        // Wait for second process to finish
        p2.waitFor();
        // Show output of second process
        java.io.BufferedReader r = new java.io.BufferedReader(new java.io.InputStreamReader(p2.getInputStream()));
        String s = null;
        while ((s = r.readLine()) != null) {
            System.out.println(s);
        }
    }
}

Here we go:

$ java PipeTest
10401   ??  S      0:00.01 /bin/sh ./bin/mysqld_safe --datadir=/usr/local/mysql/data --pid-file=/usr/local/mysql/data/rbemac.local.pid
10463   ??  S      1:01.11 /usr/local/mysql/bin/mysqld --basedir=/usr/local/mysql --datadir=/usr/local/mysql/data ...
32228   ??  S      0:00.07 /bin/bash /Applications/NetBeans/NetBeans 6.9.1.app/Contents/MacOS/ ...
32461   ??  S      3:38.98 /System/Library/Frameworks/<a class="zem_slink" href="http://en.wikipedia.org/wiki/Java_virtual_machine" title="Java virtual machine" rel="wikipedia" target="_blank">JavaVM</a>.framework/Versions/1.6/Home/bin/java ...
32624   ??  R      0:00.28 /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/bin/java ...
25078 s000  Ss     0:00.17 login -pf rbe

Piping Between More Than Two Processes

We can pipe between as many processes as needed by creating multiple Pipers. Like in the shell:

$ ps ax | grep java | grep JavaVM

We extend the Piper with a static method pipe(java.lang.Process[]) to setup all needed pipes, wait for the last process in chain to finish and return its java.io.InputStream:

public class Piper implements java.lang.Runnable {

    // Same code as above

    public static java.io.InputStream pipe(java.lang.Process... proc) throws java.lang.InterruptedException {
        // Start Piper between all processes
        java.lang.Process p1;
        java.lang.Process p2;
        for (int i = 0; i &lt; proc.length; i++) {
            p1 = proc[i];
            // If there's one more process
            if (i + 1 &lt; proc.length) {
                p2 = proc[i + 1];
                // Start piper
                new Thread(new Piper(p1.getInputStream(), p2.getOutputStream())).start();
            }
        }
        java.lang.Process last = proc[proc.length - 1];
        // Wait for last process in chain; may throw InterruptedException
        last.waitFor();
        // Return its InputStream
        return last.getInputStream();
    }

}

Give it a try:

public class PipeTest {

    public static void main(String[] args) throws java.io.IOException, java.lang.InterruptedException {
        java.lang.Runtime rt = java.lang.Runtime.getRuntime();
        // Start three processes: ps ax | grep rbe | grep JavaVM
        java.lang.Process p1 = rt.exec("ps ax");
        java.lang.Process p2 = rt.exec("grep rbe");
        java.lang.Process p3 = rt.exec("grep JavaVM");
        // Start piping
        java.io.InputStream in = Piper.pipe(p1, p2, p3);
        // Show output of last process
        java.io.BufferedReader r = new java.io.BufferedReader(new java.io.InputStreamReader(in));
        String s = null;
        while ((s = r.readLine()) != null) {
            System.out.println(s);
        }
    }

}

Result:

32461   ??  R      4:45.33 /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/bin/java ...
32700   ??  R      0:00.28 /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/bin/java ...

Groovy

It’s easy to use piping between processes in Groovy. This reminds me of UNIX shell programming:

// Look for a certain process
def p = "ps -e".execute() | "grep java".execute() | "grep -v grep".execute()
p.waitFor()
// Show PID
println p.text.trim().split(" ")[0]
This entry was posted in Software Development and tagged . Bookmark the permalink.