wparallelizer

Name

wparallelizer -- run a number of commands in parallel

Synopsis

wparallelizer [--max NUM] [--names DELIMITER NAMES] [--output OUTPUT-MODE] [--commands COMMAND-FILE] [--error-table ERROR-TABLE] [--log LOG] [--0]

DESCRIPTION

This program runs a number of commands, possibly specifying a maximum number to run concurrently. The backend "rendering" engine is designed to allow convenient output mode customization.

Commands are read from scripts specified as parameters to the --commands flag. By default, we read newline separated commands (just like a normal script), or if --0 is specified, the command files will be parsed assuming NUL (character 0) is the delimiter. (This is useful if the commands may contain newlines.)

If multiple --commands or --message lines are given, the will be run in serial, that is, all the commands in a given command block are executed before the next command or message is processed.

A log of all the process output is optionally kept in file specified with the --log flag.

INTERFACE OPTIONS

The following output modes may be specified with the --output flag:

blob

Serialize the output of each process separately, in the same order in which the processes ended.

line

Each output line from each host is preserved on a separate line, with the host/process name prepended. Though the lines from all the hosts are printed without delay, it will not print will always wait for a full line from each host; each line from each host is print intact.

raw

No processing is done at all, all the outputs are scrambled together.

tiled

Make separate windows for the output of each process (using ncurses).

In addition to the output redirected from the subprocesses wparallelizer itself may be directed to print a summary of its operation. By default, it prints the exit statuses of failed processes (those which did not exit with status 0). You may chose a different summary type with --wrapup; here are the currently defined wrapup modes:

none

Do not output anything upon termination.

failed

Print information about processes which did not exit successfully. (This is the default)

all

Print information about all the processes that we ran.

stats

Show the exit statuses of all processes and the amount of data input and output by those processes.

ERROR HANDLING

wparallelizer is built to be general purpose, but it has compiled-in defaults that are only really appropriate for wigwam. We support a compile-time default and a runtime-choosable configuration of error messages, error id strings, and exit codes. The format of this file is rather strict; here is a two entry example:
  {
    0,
    "success",
    "The process completed successfully"
  },
  {
    3,
    "tragedy"
    "You don't have write access to a directory required for publication"
  },
Please note that it is mandatory that the curly braces appear by themselves on a line, and also that it adheres to C-array declaration syntax.

wparallelizer defines some magic for making shell scripts more readable. If you invoke it with the --print-sh-error-table flag it will print a sh-script you can eval to import symbolic names for the variables; the symbolic names are the second field in the error-table, with errcode_ prepended; the error messages are likewise, but with errmessage_ prepended. For example the above error table would cause the following sh-script to be emitted:
errcode_success=0
errcode_success="The process completed successfully"
errcode_tragedy=3
errmessage_tragedy="A tragedy occurred"

To change the compile-time default add CFLAGS=-DERROR_TABLE_FILENAME="filename" to your environment then upgrade -f wigwam-base. You, uh, probably want to use the runtime setting, WPARALLELIZER_ERROR_TABLE_FILE instead.