condor_ dagman

meta scheduler of the jobs submitted as the nodes of a DAG or DAGs

Synopsis

condor_ dagman [-debug level] [-rescue filename] [-maxidle numberOfJobs] [-maxjobs numberOfJobs] [-maxpre NumberOfPREscripts] [-maxpost NumberOfPOSTscripts] [-noeventchecks] [-allowlogerror] [-usedagdir] (-condorlog filename | -storklog filename) -lockfile filename [-waitfordebug] [-autorescue 0|1] [-dorescuefrom number] -dag dag_file [-dag dag_file_2 ... -dag dag_file_n ]

Description

condor_ dagman is a meta scheduler for the Condor jobs within a DAG (directed acyclic graph) (or multiple DAGs). In typical usage, a submitter of jobs that are organized into a DAG submits the DAG using condor_ submit_dag. condor_ submit_dag does error checking on aspects of the DAG and then submits condor_ dagman as a Condor job. condor_ dagman uses log files to coordinate the further submission of the jobs within the DAG.

As part of daemoncore, the set of command-line arguments given in section 3.9.2 work for condor_ dagman.

Arguments to condor_ dagman are either automatically set by condor_ submit_dag or they are specified as command-line arguments to condor_ submit_dag and passed on to condor_ dagman. The method by which the arguments are set is given in their description below.

condor_ dagman can run multiple, independent DAGs. This is done by specifying multiple -dag arguments. Pass multiple DAG input files as command-line arguments to condor_ submit_dag.

Debugging output may be obtained by using the -debug level option. Level values and what they produce is described as

• level = 0; never produce output, except for usage info
• level = 1; very quiet, output severe errors
• level = 2; normal output, errors and warnings
• level = 3; output errors, as well as all warnings
• level = 4; internal debugging output
• level = 5; internal debugging output; outer loop debugging
• level = 6; internal debugging output; inner loop debugging
• level = 7; internal debugging output; rarely used

Options

-debug level

An integer level of debugging output. level is an integer, with values of 0-7 inclusive, where 7 is the most verbose output. This command-line option to condor_ submit_dag is passed to condor_ dagman or defaults to the value 3, as set by condor_ submit_dag.

-rescue filename

Sets the file name of the rescue DAG to write in the case of a failure. As passed by condor_ submit_dag, the name of the file will be the name of the DAG input file concatenated with the string .rescue. This argument is now optional, and in general it is preferred to not specify it. This allows condor_ dagman to automatically generate an appropriate rescue DAG name.

-maxidle NumberOfJobs

Sets the maximum number of idle jobs allowed before condor_ dagman stops submitting more jobs. Once idle jobs start to run, condor_ dagman will resume submitting jobs. NumberOfJobs is a positive integer. This command-line option to condor_ submit_dag is passed to condor_ dagman. If not specified, the number of idle jobs is unlimited.

-maxjobs numberOfJobs

Sets the maximum number of jobs within the DAG that will be submitted to Condor at one time. numberOfJobs is a positive integer. This command-line option to condor_ submit_dag is passed to condor_ dagman. If not specified, the default number of jobs is unlimited.

-maxpre NumberOfPREscripts

Sets the maximum number of PRE scripts within the DAG that may be running at one time. NumberOfPREScripts is a positive integer. This command-line option to condor_ submit_dag is passed to condor_ dagman. If not specified, the default number of PRE scripts is unlimited.

-maxpost NumberOfPOSTscripts

Sets the maximum number of POST scripts within the DAG that may be running at one time. NumberOfPOSTScripts is a positive integer. This command-line option to condor_ submit_dag is passed to condor_ dagman. If not specified, the default number of POST scripts is unlimited.

-noeventchecks

This argument is no longer used; it is now ignored. Its functionality is now implemented by the DAGMAN_ALLOW_EVENTS configuration macro (see section 3.3.23).

-allowlogerror

This optional argument has condor_ dagman try to run the specified DAG, even in the case of detected errors in the user log specification.

-usedagdir

This optional argument has causes condor_ dagman to run each specified DAG as if the directory containing that DAG file was the current working directory. This option is most useful when running multiple DAGs in a single condor_ dagman.

-storklog filename

Sets the file name of the Stork log for data placement jobs.

-condorlog filename

Sets the file name of the file used in conjunction with the -lockfile filename in determining whether to run in recovery mode.

-lockfile filename

Names the file created and used as a lock file. The lock file prevents execution of two of the same DAG, as defined by a DAG input file. A default lock file ending with the suffix .dag.lock is passed to condor_ dagman by condor_ submit_dag.

-waitfordebug

This optional argument causes condor_ dagman to wait at startup until someone attaches to the process with a debugger and sets the wait_for_debug variable in main_init() to false.

-autorescue 0|1

Whether to automatically run the newest rescue DAG for the given DAG file, if one exists (0 = false, 1 = true).

-dorescuefrom number

Forces condor_ dagman to run the specified rescue DAG number for the given DAG. A value of 0 is the same as not specifying this option. Specifying a non-existant rescue DAG is a fatal error.

-dag filename

filename is the name of the DAG input file that is set as an argument to condor_ submit_dag, and passed to condor_ dagman.

Exit Status

condor_ dagman will exit with a status value of 0 (zero) upon success, and it will exit with the value 1 (one) upon failure.

Examples

condor_ dagman is normally not run directly, but submitted as a Condor job by running condor_ submit_dag . See the condor_ submit_dag manual page  for examples.

Author

Condor Team, University of Wisconsin-Madison

Copyright

Copyright © 1990-2008 Condor Team, Computer Sciences Department, University of Wisconsin-Madison, Madison, WI. All Rights Reserved. No use of the Condor Software Program is authorized without the express consent of the Condor Team. For more information contact: Condor Team, Attention: Professor Miron Livny, 7367 Computer Sciences, 1210 W. Dayton St., Madison, WI 53706-1685, (608) 262-0856 or miron@cs.wisc.edu.

U.S. Government Rights Restrictions: Use, duplication, or disclosure by the U.S. Government is subject to restrictions as set forth in subparagraph (c)(1)(ii) of The Rights in Technical Data and Computer Software clause at DFARS 252.227-7013 or subparagraphs (c)(1) and (2) of Commercial Computer Software-Restricted Rights at 48 CFR 52.227-19, as applicable, Condor Team, Attention: Professor Miron Livny, 7367 Computer Sciences, 1210 W. Dayton St., Madison, WI 53706-1685, (608) 262-0856 or miron@cs.wisc.edu.

See the Condor Version 7.1.0 Manual for additional notices.