Condor - High Throughput Computing

DAGMan

DAGMan (Directed Acyclic Graph Manager) is a meta-scheduler for Condor. It manages dependencies between jobs at a higher level than the Condor Scheduler.

A directed acyclic graph (DAG) can be used to represent a set of programs where the input, output, or execution of one or more programs is dependent on one or more other programs. The programs are nodes (vertices) in the graph, and the edges (arcs) identify the dependencies. Condor finds machines for the execution of programs, but it does not schedule programs (jobs) based on dependencies. The Directed Acyclic Graph Manager (DAGMan) is a meta-scheduler for Condor jobs. DAGMan submits jobs to Condor in an order represented by a DAG and processes the results. An input file defined prior to submission describes the DAG, and a Condor submit description file for each program in the DAG is used by Condor.

Each node (program) in the DAG needs its own Condor submit description file. As DAGMan submits jobs to Condor, it uses a single Condor log file to enforce the ordering required for the DAG. The DAG itself is defined by the contents of a DAGMan input file. DAGMan is responsible for scheduling, recovery, and reporting for the set of programs submitted to Condor.

DAGMan Documentation

Condor is fully documented in the Condor manual. The Inter-Job Dependancies section describes DAGMan in great detail.

Downloading DAGMan

DAGMan is part of the Condor software, which can be downloaded here.

DAGMan and Pegasus

DAGMan can also be used in conjunction with the Pegasus Workflow Management System, which provides a resource-independent interface and management technologies that map the workflow specification onto Condor pools, campus clusters and other cyberinfrastrure resources.


condor-admin@cs.wisc.edu