Kjetil's Information Center: A Blog About My Projects

Process Dispatcher

This is a script I hacked together in Python to be able to run several processes in parallel easily. You specify a program to run as the argument to the script itself, then the script presents a shell where you can input a "real" argument for the program previously specified. Once an argument has been entered on the shell, a sub process is started in the background, and the shell is immediately ready for a new argument. A practical usage of this is to download several files in parallel with "wget". Simply start the script with wget as the argument and enter URLs on the shell to download them. A counter in the shell prompt is used to keep track of how many sub processes are ongoing.

Enjoy:

#!/usr/bin/python

import threading
import subprocess

class Dispatcher(object):
    def __init__(self, command):
        self.command = command

    class _Executor(threading.Thread):
        def __init__(self, command, arg):
            threading.Thread.__init__(self)
            self.command = command
            self.arg = arg

        def run(self):
            null = open("/dev/null")
            subprocess.call([self.command, self.arg], stdout=null, stderr=null)
            null.close()

    def run(self):
        while True:
            arg = raw_input(str(threading.active_count() - 1) + ">")
            if arg == 'quit':
                break
            if len(arg) > 0:
                thread = self._Executor(self.command, arg)
                thread.start()
            else:
                for thread in threading.enumerate():
                    if thread.name != "MainThread":
                        print thread.arg

if __name__ == "__main__":
    import sys
    if len(sys.argv) > 1:
        d = Dispatcher(sys.argv[1])
        d.run()
        sys.exit(0)
    else:
        print "Usage: %s <executable command>" % (sys.argv[0])
        sys.exit(1)
          


Topic: Scripts and Code, by Kjetil @ 01/07-2012, Article Link