LIGO Support Ticket 18696
Ticket Information
Number: admin 18696
User: nvf@gravity.phys.uwm.edu
Email: dabrown__AT__physics.syr.edu
Status: open
Assigned To: wenger
From: Nickolas Fotopoulos <nvf__AT__gravity.phys.uwm.edu>
To: condor-admin__AT__cs.wisc.edu
Subject: LIGO: noop jobs are queued
Date: Wed, 29 Oct 2008 10:56:11 -0500
CC: Duncan Brown <dabrown__AT__physics.syr.edu>
X-Seen-BY: mailfromd 4.1 gypsum.cs.wisc.edu
--Apple-Mail-2-27952048
Dear Condor team,
I have a large block of jobs for which I often set noop_job=True. I
need these jobs in my pipeline construction so that all the plumbing
(input and output) is hooked up correctly within our framework.
noop_job seems to be working since there is no ULOG_EXECUTE, but it is
definitely queued and waits for something; condor_q shows more than
one cycle worth of /bin/true jobs; the log shows 18 seconds between
submission and termination. With tens of thousands of them and only 5
queued per cycle, this is a noticeable delay. Is there anything that
can be done? Can dagman read the sub file and notice noop_job=True?
Many thanks,
Nick
===================================
Nickolas Fotopoulos
nvf__AT__gravity.phys.uwm.edu
Office: (414) 229-6438
Fax: (414) 229-5589
University of Wisconsin - Milwaukee
Physics Bldg, Rm 471
===================================
--Apple-Mail-2-27952048
MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEH
AQAAoIIILzCCBA0wggL1oAMCAQICAleLMA0GCSqGSIb3DQEBBQUAMGkxEzAR
BgoJkiaJk/IsZAEZFgNvcmcxGDAWBgoJkiaJk/IsZAEZFghET0VHcmlkczEg
MB4GA1UECxMXQ2VydGlmaWNhdGUgQXV0aG9yaXRpZXMxFjAUBgNVBAMTDURP
RUdyaWRzIENBIDEwHhcNMDgwNDAzMjAzODIzWhcNMDkwNDAzMjAzODIzWjBl
MRMwEQYKCZImiZPyLGQBGRYDb3JnMRgwFgYKCZImiZPyLGQBGRYIZG9lZ3Jp
ZHMxDzANBgNVBAsTBlBlb3BsZTEjMCEGA1UEAxMaTmlja29sYXMgRm90b3Bv
dWxvcyAxNjk2ODgwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDo
ixYGskJaFgOWaSsFu+E4nChe9Vun1OdsRplGDBmS8uuJldqt91c71n8gDzF7
70XOmWShki9PunuZtnULkW7k3dljYGs5bRkUn80wn2yVh8sPcezYRHV9jVJn
WjNhgoROK/XhWbjcw/HcRzqMJsFyULZz+WOtUUUntYj79dsq7ERmik+F54dl
LhK2EPhNSFazI3an4neRQ2oRUkp9I4Hk0xj7awTJHDc676iiQhzjkCyzCTtL
IjjvYXU+KHeRZN/vvzEVsVYSZeOFGDEV6G9lx9wrt/gjWhc5/lDOOoA73tuP
ff0AQPI4Zzt/5/y13Fn/Lfnutcpo9MA17OC5etCRAgMBAAGjgcIwgb8wEQYJ
YIZIAYb4QgEBBAQDAgWgMA4GA1UdDwEB/wQEAwIF4DAfBgNVHSMEGDAWgBTK
GR0Sjm6kOF1C1DEOCNvZjRcNXTAYBgNVHSAEETAPMA0GCyqGSIb3TAMHAQIJ
MDoGA1UdHwQzMDEwL6AtoCuGKWh0dHA6Ly9wa2kxLmRvZWdyaWRzLm9yZy9D
UkwvMWMzZjJjYTguY3JsMCMGA1UdEQQcMBqBGG52ZkBncmF2aXR5LnBoeXMu
dXdtLmVkdTANBgkqhkiG9w0BAQUFAAOCAQEAOfx3c26BEJppo7PNfxts5U6v
kMgS/rqCLOrlte1JT9AhXuGZBabpSO1/HJJVZ6AgLOR9ZGGKcJiWE5QXtuN6
lSDhbjH0FcJWTaHAdnV71Mz2CEG4XXmAxZ74lRZB/qLm40sioIzsjaVieFAf
YX6naSgSsqpTGeiXTD4H2mm3A+14Hpt2fyQdbgOdR9XFy0KkfWiM4vgpoGPe
O2I5DIPM5mQOsa7t8tKKEnw+kp1D7IQwW+O2qxR93n8wH0CDL3PF+j15ub9i
Y9yNnkrMi8f5B9745BZaiikddyeRMDGX46WfdXKZc+kPw9iPMQ3441yJG4yx
zPiv1Zxnj+dmBPoYPTCCBBowggMCoAMCAQICAjcbMA0GCSqGSIb3DQEBBQUA
MGkxEzARBgoJkiaJk/IsZAEZFgNvcmcxGDAWBgoJkiaJk/IsZAEZFghET0VH
cmlkczEgMB4GA1UECxMXQ2VydGlmaWNhdGUgQXV0aG9yaXRpZXMxFjAUBgNV
BAMTDURPRUdyaWRzIENBIDEwHhcNMDcwNDA5MjE0NDE3WhcNMDgwNDA4MjE0
NDE3WjBlMRMwEQYKCZImiZPyLGQBGRYDb3JnMRgwFgYKCZImiZPyLGQBGRYI
ZG9lZ3JpZHMxDzANBgNVBAsTBlBlb3BsZTEjMCEGA1UEAxMaTmlja29sYXMg
Rm90b3BvdWxvcyA0NTIwNzkwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK
AoIBAQDAP5sWLsZ3lvIzwMRbq17A6gBFracKYVzaNiOU+7o2/xnlQn27diVn
Bln9S29bm84BBnOU2uBvBvGbIh1PjM/QaehzI+3a+V4SnaSCm8NP8voTtfod
AiXrKUEQ2UaD9h6zVc57MEAp10Fyi2eRn9bjZl7DSxpDDqB/P3I7c6pYVyXA
6DaYZwtNOvjk9FRVHh9efUv9KFLhD9BaWwFjRqFb3a29RRNudYo877Os7PGI
8fGy78ohaLlDt5Yrr9OQT6xrfamrdsE8ifNE1GUrNTfL7biXd/pLXfkhBGOL
PwotNrJMrQVgdNGmzp+R8i6Os0HoYG/kospUWVbX33TGf7xlAgMBAAGjgc8w
gcwwEQYJYIZIAYb4QgEBBAQDAgXgMA4GA1UdDwEB/wQEAwIE8DAYBgNVHSAE
ETAPMA0GCyqGSIb3TAMHAQIHMDoGA1UdHwQzMDEwL6AtoCuGKWh0dHA6Ly9w
a2kxLmRvZWdyaWRzLm9yZy9DUkwvMWMzZjJjYTguY3JsMDAGA1UdEQQpMCeB
C252ZkBtaXQuZWR1gRhudmZAZ3Jhdml0eS5waHlzLnV3bS5lZHUwHwYDVR0j
BBgwFoAUyhkdEo5upDhdQtQxDgjb2Y0XDV0wDQYJKoZIhvcNAQEFBQADggEB
AEQkNxVIlDuZdmsogWbkRIbvUGT1rZ9bqus605xQ0toiIFT0sE79R9PqCAfB
I86HOaI8rONjx4q61Z4Fa3F0B1YpyYBsYnyft9IfK/n5o/CttCDxz0/LvWtL
8YpAYee9nztS3Dv/KzY12M344DYrH67qHMFuUWOcPuNzLYtrxw0Iuj0Dtbwd
67iq6R6eOt5q/aaj/3IspqZFMQO0/X7eWeGkm/jhrn4oETku5Mw+O7z+Fwem
fhdly7vd2f8TRDie+mQDLiXPuY2RJSaDIszQ3CTkkYNwtd8Qy1Y1gcKURA17
aRtibit2tingmAaYRG6uqwVCkKLsH0/pceTYnWOWE9cxggL6MIIC9gIBATBv
MGkxEzARBgoJkiaJk/IsZAEZFgNvcmcxGDAWBgoJkiaJk/IsZAEZFghET0VH
cmlkczEgMB4GA1UECxMXQ2VydGlmaWNhdGUgQXV0aG9yaXRpZXMxFjAUBgNV
BAMTDURPRUdyaWRzIENBIDECAleLMAkGBSsOAwIaBQCgggFgMBgGCSqGSIb3
DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTA4MTAyOTE1NTYx
MVowIwYJKoZIhvcNAQkEMRYEFKVnmdzZTykhiaXjuYlpAz9dsMzEMH4GCSsG
AQQBgjcQBDFxMG8waTETMBEGCgmSJomT8ixkARkWA29yZzEYMBYGCgmSJomT
8ixkARkWCERPRUdyaWRzMSAwHgYDVQQLExdDZXJ0aWZpY2F0ZSBBdXRob3Jp
dGllczEWMBQGA1UEAxMNRE9FR3JpZHMgQ0EgMQICNxswgYAGCyqGSIb3DQEJ
EAILMXGgbzBpMRMwEQYKCZImiZPyLGQBGRYDb3JnMRgwFgYKCZImiZPyLGQB
GRYIRE9FR3JpZHMxIDAeBgNVBAsTF0NlcnRpZmljYXRlIEF1dGhvcml0aWVz
MRYwFAYDVQQDEw1ET0VHcmlkcyBDQSAxAgI3GzANBgkqhkiG9w0BAQEFAASC
AQBjFM6ZwVG0b1TP+0auoyJXudLHBuP1UDz3nUJ6WBy8tsU2bDyCB8QcDiqK
xktDhtlzoF6UVshwF6dippEfY+b1N2HtTZH30QTVONWTgdL5ZQikEaYUBqvI
8FmdxBpo/Z2WbRukzS2FKVbw7EuQHqaHyxpiuo/gnA8AX8GxJFDM/qGpGmtQ
ODY7UvnA9tjYyPc2ericHIMuuIPnjzlgZI84SbMVgW4RDP0r6b0UAGdTZYj5
i6+68y5M0CGdLKZjMlYhKWIXckuOcH8m5+ckrnbLyrIaWdscrVAXfTNLUKqu
vZmWbJBdXEgxbXcQTbl1lSZkAlFhSLJxQfVFADRnpeskAAAAAAAA
--Apple-Mail-2-27952048--
===========================================================================
Date of creation: Wed Oct 29 10:56:18 2008 (1225295781)
Subject: Actions
Assigned to wenger by gthain
===========================================================================
Date of actions: Thu Oct 30 15:33:11 2008 (1225398791)
Date: Fri, 31 Oct 2008 08:56:29 -0500 (CDT)
From: "R. Kent Wenger" <wenger__AT__cs.wisc.edu>
To: gthain <condor-admin__AT__cs.wisc.edu>
CC: "R. Kent Wenger" <wenger__AT__cs.wisc.edu>
Subject: Re: [condor-admin #18696] LIGO: noop jobs are queued
Nick,
> I have a large block of jobs for which I often set noop_job=True. I
> need these jobs in my pipeline construction so that all the plumbing
> (input and output) is hooked up correctly within our framework.
> noop_job seems to be working since there is no ULOG_EXECUTE, but it is
> definitely queued and waits for something; condor_q shows more than
> one cycle worth of /bin/true jobs; the log shows 18 seconds between
> submission and termination. With tens of thousands of them and only 5
> queued per cycle, this is a noticeable delay. Is there anything that
> can be done? Can dagman read the sub file and notice noop_job=True?
Hmm, that's an interesting idea. I've created a gnats PR for this PR
951).
I don't think it would be too hard; the main thing is that we'd have to do
some thinking to make sure there wasn't an obscure case that would get
goofed up by this. And, of course, there are something like 100 other
bugs/feature requests for DAGMan...
Also, you can submit more than 5 jobs per cycle by setting
DAGMAN_MAX_SUBMITS_PER_INTERVAL to a higher value (up to 1000). That
would be a short-term change that should at least improve your
performance.
Kent Wenger
Condor Team
===========================================================================
Date mail was appended: Fri Oct 31 8:56:32 2008 (1225461392)
From: Nickolas Fotopoulos <nvf__AT__gravity.phys.uwm.edu>
To: condor-admin__AT__cs.wisc.edu
Subject: Re: [condor-admin #18696] LIGO: noop jobs are queued
Date: Fri, 31 Oct 2008 09:22:31 -0500
X-Seen-BY: mailfromd 4.1 gypsum.cs.wisc.edu
--Apple-Mail-11-195132440
Kent,
On Oct 31, 2008, at 8:56 AM, condor-admin response tracking system
wrote:
> Nick,
>
>> I have a large block of jobs for which I often set noop_job=True. I
>> need these jobs in my pipeline construction so that all the plumbing
>> (input and output) is hooked up correctly within our framework.
>> noop_job seems to be working since there is no ULOG_EXECUTE, but it
>> is
>> definitely queued and waits for something; condor_q shows more than
>> one cycle worth of /bin/true jobs; the log shows 18 seconds between
>> submission and termination. With tens of thousands of them and
>> only 5
>> queued per cycle, this is a noticeable delay. Is there anything that
>> can be done? Can dagman read the sub file and notice noop_job=True?
>
> Hmm, that's an interesting idea. I've created a gnats PR for this PR
> 951).
>
> I don't think it would be too hard; the main thing is that we'd have
> to do
> some thinking to make sure there wasn't an obscure case that would get
> goofed up by this. And, of course, there are something like 100 other
> bugs/feature requests for DAGMan...
Thanks for accepting the PR. Perhaps one day...
> Also, you can submit more than 5 jobs per cycle by setting
> DAGMAN_MAX_SUBMITS_PER_INTERVAL to a higher value (up to 1000). That
> would be a short-term change that should at least improve your
> performance.
I'll see what the admins think about that. Thanks for the suggestion.
Take care,
Nick
> Kent Wenger
> Condor Team
>
>
> ========================================
> MESSAGE INFORMATION
> ========================================
> * From: "R. Kent Wenger" <wenger__AT__cs.wisc.edu>
> * Ticket Email List: nvf__AT__gravity.phys.uwm.edu, dabrown__AT__physics.syr.edu
===================================
Nickolas Fotopoulos
nvf__AT__gravity.phys.uwm.edu
Office: (414) 229-6438
Fax: (414) 229-5589
University of Wisconsin - Milwaukee
Physics Bldg, Rm 471
===================================
--Apple-Mail-11-195132440
MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEH
AQAAoIIILzCCBA0wggL1oAMCAQICAleLMA0GCSqGSIb3DQEBBQUAMGkxEzAR
BgoJkiaJk/IsZAEZFgNvcmcxGDAWBgoJkiaJk/IsZAEZFghET0VHcmlkczEg
MB4GA1UECxMXQ2VydGlmaWNhdGUgQXV0aG9yaXRpZXMxFjAUBgNVBAMTDURP
RUdyaWRzIENBIDEwHhcNMDgwNDAzMjAzODIzWhcNMDkwNDAzMjAzODIzWjBl
MRMwEQYKCZImiZPyLGQBGRYDb3JnMRgwFgYKCZImiZPyLGQBGRYIZG9lZ3Jp
ZHMxDzANBgNVBAsTBlBlb3BsZTEjMCEGA1UEAxMaTmlja29sYXMgRm90b3Bv
dWxvcyAxNjk2ODgwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDo
ixYGskJaFgOWaSsFu+E4nChe9Vun1OdsRplGDBmS8uuJldqt91c71n8gDzF7
70XOmWShki9PunuZtnULkW7k3dljYGs5bRkUn80wn2yVh8sPcezYRHV9jVJn
WjNhgoROK/XhWbjcw/HcRzqMJsFyULZz+WOtUUUntYj79dsq7ERmik+F54dl
LhK2EPhNSFazI3an4neRQ2oRUkp9I4Hk0xj7awTJHDc676iiQhzjkCyzCTtL
IjjvYXU+KHeRZN/vvzEVsVYSZeOFGDEV6G9lx9wrt/gjWhc5/lDOOoA73tuP
ff0AQPI4Zzt/5/y13Fn/Lfnutcpo9MA17OC5etCRAgMBAAGjgcIwgb8wEQYJ
YIZIAYb4QgEBBAQDAgWgMA4GA1UdDwEB/wQEAwIF4DAfBgNVHSMEGDAWgBTK
GR0Sjm6kOF1C1DEOCNvZjRcNXTAYBgNVHSAEETAPMA0GCyqGSIb3TAMHAQIJ
MDoGA1UdHwQzMDEwL6AtoCuGKWh0dHA6Ly9wa2kxLmRvZWdyaWRzLm9yZy9D
UkwvMWMzZjJjYTguY3JsMCMGA1UdEQQcMBqBGG52ZkBncmF2aXR5LnBoeXMu
dXdtLmVkdTANBgkqhkiG9w0BAQUFAAOCAQEAOfx3c26BEJppo7PNfxts5U6v
kMgS/rqCLOrlte1JT9AhXuGZBabpSO1/HJJVZ6AgLOR9ZGGKcJiWE5QXtuN6
lSDhbjH0FcJWTaHAdnV71Mz2CEG4XXmAxZ74lRZB/qLm40sioIzsjaVieFAf
YX6naSgSsqpTGeiXTD4H2mm3A+14Hpt2fyQdbgOdR9XFy0KkfWiM4vgpoGPe
O2I5DIPM5mQOsa7t8tKKEnw+kp1D7IQwW+O2qxR93n8wH0CDL3PF+j15ub9i
Y9yNnkrMi8f5B9745BZaiikddyeRMDGX46WfdXKZc+kPw9iPMQ3441yJG4yx
zPiv1Zxnj+dmBPoYPTCCBBowggMCoAMCAQICAjcbMA0GCSqGSIb3DQEBBQUA
MGkxEzARBgoJkiaJk/IsZAEZFgNvcmcxGDAWBgoJkiaJk/IsZAEZFghET0VH
cmlkczEgMB4GA1UECxMXQ2VydGlmaWNhdGUgQXV0aG9yaXRpZXMxFjAUBgNV
BAMTDURPRUdyaWRzIENBIDEwHhcNMDcwNDA5MjE0NDE3WhcNMDgwNDA4MjE0
NDE3WjBlMRMwEQYKCZImiZPyLGQBGRYDb3JnMRgwFgYKCZImiZPyLGQBGRYI
ZG9lZ3JpZHMxDzANBgNVBAsTBlBlb3BsZTEjMCEGA1UEAxMaTmlja29sYXMg
Rm90b3BvdWxvcyA0NTIwNzkwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK
AoIBAQDAP5sWLsZ3lvIzwMRbq17A6gBFracKYVzaNiOU+7o2/xnlQn27diVn
Bln9S29bm84BBnOU2uBvBvGbIh1PjM/QaehzI+3a+V4SnaSCm8NP8voTtfod
AiXrKUEQ2UaD9h6zVc57MEAp10Fyi2eRn9bjZl7DSxpDDqB/P3I7c6pYVyXA
6DaYZwtNOvjk9FRVHh9efUv9KFLhD9BaWwFjRqFb3a29RRNudYo877Os7PGI
8fGy78ohaLlDt5Yrr9OQT6xrfamrdsE8ifNE1GUrNTfL7biXd/pLXfkhBGOL
PwotNrJMrQVgdNGmzp+R8i6Os0HoYG/kospUWVbX33TGf7xlAgMBAAGjgc8w
gcwwEQYJYIZIAYb4QgEBBAQDAgXgMA4GA1UdDwEB/wQEAwIE8DAYBgNVHSAE
ETAPMA0GCyqGSIb3TAMHAQIHMDoGA1UdHwQzMDEwL6AtoCuGKWh0dHA6Ly9w
a2kxLmRvZWdyaWRzLm9yZy9DUkwvMWMzZjJjYTguY3JsMDAGA1UdEQQpMCeB
C252ZkBtaXQuZWR1gRhudmZAZ3Jhdml0eS5waHlzLnV3bS5lZHUwHwYDVR0j
BBgwFoAUyhkdEo5upDhdQtQxDgjb2Y0XDV0wDQYJKoZIhvcNAQEFBQADggEB
AEQkNxVIlDuZdmsogWbkRIbvUGT1rZ9bqus605xQ0toiIFT0sE79R9PqCAfB
I86HOaI8rONjx4q61Z4Fa3F0B1YpyYBsYnyft9IfK/n5o/CttCDxz0/LvWtL
8YpAYee9nztS3Dv/KzY12M344DYrH67qHMFuUWOcPuNzLYtrxw0Iuj0Dtbwd
67iq6R6eOt5q/aaj/3IspqZFMQO0/X7eWeGkm/jhrn4oETku5Mw+O7z+Fwem
fhdly7vd2f8TRDie+mQDLiXPuY2RJSaDIszQ3CTkkYNwtd8Qy1Y1gcKURA17
aRtibit2tingmAaYRG6uqwVCkKLsH0/pceTYnWOWE9cxggL6MIIC9gIBATBv
MGkxEzARBgoJkiaJk/IsZAEZFgNvcmcxGDAWBgoJkiaJk/IsZAEZFghET0VH
cmlkczEgMB4GA1UECxMXQ2VydGlmaWNhdGUgQXV0aG9yaXRpZXMxFjAUBgNV
BAMTDURPRUdyaWRzIENBIDECAleLMAkGBSsOAwIaBQCgggFgMBgGCSqGSIb3
DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTA4MTAzMTE0MjIz
MlowIwYJKoZIhvcNAQkEMRYEFC9RZrwsqlpPeB5+5gipPacZVgWuMH4GCSsG
AQQBgjcQBDFxMG8waTETMBEGCgmSJomT8ixkARkWA29yZzEYMBYGCgmSJomT
8ixkARkWCERPRUdyaWRzMSAwHgYDVQQLExdDZXJ0aWZpY2F0ZSBBdXRob3Jp
dGllczEWMBQGA1UEAxMNRE9FR3JpZHMgQ0EgMQICNxswgYAGCyqGSIb3DQEJ
EAILMXGgbzBpMRMwEQYKCZImiZPyLGQBGRYDb3JnMRgwFgYKCZImiZPyLGQB
GRYIRE9FR3JpZHMxIDAeBgNVBAsTF0NlcnRpZmljYXRlIEF1dGhvcml0aWVz
MRYwFAYDVQQDEw1ET0VHcmlkcyBDQSAxAgI3GzANBgkqhkiG9w0BAQEFAASC
AQCOnS0osl1OvdcnPzn2RoPSW7hjuL96yegXoFuf4G32YLtNwJKFVcuKtrXj
cjpQ94OQb0LhE8jD5WqzhV0LOp1T95u8DT47O+jeQkoPIAt+y2JoUREi9hIT
h4ASLggKUWxdL5K8wDn9AhFnN/90s3dCZ+81zqj5Xboy2z4xy2qc7oQfMU+C
LOHzRlpGC+26+Uf1uknmmZeMONmeMSB1i+mMLCP4G//6Vc+9wzx3sVr8KUxV
o1VY2DbVVqpGgVbcY02xdqfp29XdKVfPWN7cjOuMuersV0RJAwRLNIDy417I
bsFXxLcFrFC2fFXw0E6muMJRZ34XJz514joHBIZO9PVQAAAAAAAA
--Apple-Mail-11-195132440--
===========================================================================
Date mail was appended: Fri Oct 31 9:22:36 2008 (1225462956)
Date: Fri, 31 Oct 2008 09:26:29 -0500 (CDT)
From: "R. Kent Wenger" <wenger__AT__cs.wisc.edu>
To: condor-admin response tracking system <condor-admin__AT__cs.wisc.edu>
CC: "R. Kent Wenger" <wenger__AT__cs.wisc.edu>
Subject: Re: [condor-admin #18696] LIGO: noop jobs are queued
Nick,
>> Also, you can submit more than 5 jobs per cycle by setting
>> DAGMAN_MAX_SUBMITS_PER_INTERVAL to a higher value (up to 1000). That
>> would be a short-term change that should at least improve your
>> performance.
>
> I'll see what the admins think about that. Thanks for the suggestion.
That doesn't have to be done globally (in fact, it probably shouldn't be).
You can do it on a per-DAG basis by using a DAG configuration file (see
http://www.cs.wisc.edu/condor/manual/v7.1/2_10DAGMan_Applications.html#SECTION0031021100000000000000)
or by just setting the environment variable
_CONDOR_DAGMAN_MAX_SUBMITS_PER_INTERVAL when you run condor_submit_dag.
Kent Wenger
Condor Team
===========================================================================
Date mail was appended: Fri Oct 31 9:26:32 2008 (1225463192)
Date: Fri, 7 Nov 2008 14:44:24 -0600 (CST)
From: "R. Kent Wenger" <wenger__AT__cs.wisc.edu>
To: gthain <condor-admin__AT__cs.wisc.edu>
CC: "R. Kent Wenger" <wenger__AT__cs.wisc.edu>
Subject: Re: [condor-admin #18696] LIGO: noop jobs are queued
Nick,
> I have a large block of jobs for which I often set noop_job=True. I
> need these jobs in my pipeline construction so that all the plumbing
> (input and output) is hooked up correctly within our framework.
> noop_job seems to be working since there is no ULOG_EXECUTE, but it is
> definitely queued and waits for something; condor_q shows more than
> one cycle worth of /bin/true jobs; the log shows 18 seconds between
> submission and termination. With tens of thousands of them and only 5
> queued per cycle, this is a noticeable delay. Is there anything that
> can be done? Can dagman read the sub file and notice noop_job=True?
We just had a discussion of this during the phone conference, and I got
some feedback as to why just increasing the number of submits per interval
probably isn't really the solution. So I'll move this up the priorities a
bit, although I don't know exactly how soon we'll get to it.
Gnats PR 951 also refers to this.
Kent Wenger
Condor Team
===========================================================================
Date mail was appended: Fri Nov 7 14:44:26 2008 (1226090666)