LIGO Support Ticket 15811

Ticket Information
  Number:      admin 15811
  User:        anderson@ligo.caltech.edu
  Email:       espinoza_e__AT__ligo.caltech.edu,duncan__AT__gravity.phys.uwm.edu,skoranda__AT__gravity.phys.uwm.edu
  Status:      resolved
  Assigned To: tannenba

Date: Fri, 6 Jul 2007 10:34:58 -0700
From: Stuart Anderson <anderson__AT__ligo.caltech.edu>
To: condor-admin__AT__cs.wisc.edu, wenger__AT__cs.wisc.edu,         Todd Tannenbaum
 <tannenba__AT__cs.wisc.edu>
CC: Erik Espinoza <espinoza_e__AT__ligo.caltech.edu>,         Brown Duncan
 <duncan__AT__gravity.phys.uwm.edu>,         Scott Koranda
 <skoranda__AT__gravity.phys.uwm.edu>
Subject: LIGO: intra-DAG node prioritizaion and throttling

As discussed in recent LIGO-Condor telecons LIGO would like to be able to
specify the relative priorities of nodes with a single DAG--sometimes referred
to as "coloring the graph". It was decided that this would be more generally
beneficial to the wider Condor community, and of similar cost to implement
than solving the initial and more restrictive problem of just providing true
depth-first evaluation of a DAG.

An additional request is to generalize the condor_submit_dag -maxjobs
functionality to be able to specify the maximum number of jobs at each
priority level (preferably within a dag input file rather than a command line
option). The motivation for this is that we have DAG's where different nodes
have dramatically different resource requirements, e.g., some nodes are
"well behaved" with little I/O and hours of in memory computation, while
others do almost no computation but have large I/O requirements. Since this
information is available a priori, we would like to be able to color the
abusive jobs and restrict the number of parallel instances running.

At first glance it seems that overloading one color per node to represent
both intra-DAG queuing priority and maxjobs setting would be sufficient,
but it would certainly be possible to have two (or more?) colors per
node to keep these distinct if you can think of more general problems
that this would solve.

Thanks.

-- 
Stuart Anderson  anderson__AT__ligo.caltech.edu
http://www.ligo.caltech.edu/~anderson

===========================================================================
Date of creation: Fri Jul  6 12:35:19 2007 (1183743321)

Subject: Actions

Assigned to tannenba by danb
===========================================================================
Date of actions: Fri Jul  6 16:48:28 2007 (1183758508)

Subject: Actions

Ticket resolved by wenger
===========================================================================
Date of actions: Fri Dec 14 13:44:18 2007 (1197661458)