LIGO Support Ticket 1739
Ticket Information
Number: support 1739
User: espinoza@ligo.caltech.edu
Email: anderson__AT__ligo.caltech.edu
Status: resolved
Assigned To: gthain
Date: Wed, 15 Nov 2006 13:53:20 -0800
From: "Erik A. Espinoza" <espinoza__AT__ligo.caltech.edu>
To: condor-support response tracking system <condor-support__AT__cs.wisc.edu>
CC: Stuart Anderson <anderson__AT__ligo.caltech.edu>
Subject: LIGO: DedicatedScheduler schedd crashed on condor_rm
X-Enigmail-Version: 0.94.1.0
Openpgp: url=http://pgp.mit.edu/
X-No-Archive: Yes
X-Archive: No
We have two schedd's running on our cluster.
1) Standard schedd
2) DedicatedScheduler schedd
This problem report is not about our primary "Standard" scheduler, where
the majority of our users are operating. The dedicated scheduler is used
to run mpi jobs, and is only used by one user. This user executed a
condor_rm around the time of this crash.
A few minutes before the schedd crashed, and a few minutes after, we had
the FRGN_KILL startd crash on two separate nodes. Refer to ticket #1694
for more info on the FRGN_KILL crashes. I am including info on the
FRGN_KILL crashes as they may hold clues. Below is the obituary logs.
Please let me know if there is any further information you require.
===DedicatedScheduler schedd crash===
-------- Original Message --------
Subject: [Condor] Problem
Date: Tue, 14 Nov 2006 11:03:48 -0800
From: condor__AT__ldas-pcdev1.ligo.caltech.edu
To: ldas_admin_cit__AT__ligo.caltech.edu
This is an automated email from the Condor system
on machine "ldas-pcdev1.ligo.caltech.edu". Do not reply.
"/ldcg/condor/sbin/condor_schedd" on "ldas-pcdev1.ligo.caltech.edu" died
due to signal 11.
Condor will automatically restart this process in 10 seconds.
*** Last 20 line(s) of file SchedLog:
11/14 11:01:24 (pid:31560) DaemonCore: received command 443
(VACATE_SERVICE), calling handler (vacate_service)
11/14 11:01:24 (pid:31560) Got VACATE_SERVICE from <10.14.1.229:42254>
11/14 11:01:27 (pid:31560) DaemonCore: Command received via TCP from
host <10.14.1.186:43953>
11/14 11:01:27 (pid:31560) DaemonCore: received command 443
(VACATE_SERVICE), calling handler (vacate_service)
11/14 11:01:27 (pid:31560) Got VACATE_SERVICE from <10.14.1.186:43953>
11/14 11:01:27 (pid:31560) DaemonCore: Command received via TCP from
host <10.14.1.229:50848>
11/14 11:01:27 (pid:31560) DaemonCore: received command 443
(VACATE_SERVICE), calling handler (vacate_service)
11/14 11:01:27 (pid:31560) Got VACATE_SERVICE from <10.14.1.229:50848>
11/14 11:01:27 (pid:31560) DaemonCore: Command received via TCP from
host <10.14.1.43:51772>
11/14 11:01:27 (pid:31560) DaemonCore: received command 443
(VACATE_SERVICE), calling handler (vacate_service)
11/14 11:01:27 (pid:31560) Got VACATE_SERVICE from <10.14.1.43:51772>
11/14 11:01:29 (pid:31560) DaemonCore: Command received via TCP from
host <10.14.1.178:54077>
11/14 11:01:29 (pid:31560) DaemonCore: received command 443
(VACATE_SERVICE), calling handler (vacate_service)
11/14 11:01:29 (pid:31560) Got VACATE_SERVICE from <10.14.1.178:54077>
11/14 11:01:30 (pid:31560) DaemonCore: Command received via TCP from
host <10.14.1.204:55764>
11/14 11:01:30 (pid:31560) DaemonCore: received command 443
(VACATE_SERVICE), calling handler (vacate_service)
11/14 11:01:30 (pid:31560) Got VACATE_SERVICE from <10.14.1.204:55764>
11/14 11:03:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/14 11:03:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/14 11:03:41 (pid:31560) Inserting new attribute Scheduler into
non-active cluster cid=22338 acid=-1
*** End of file SchedLog
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Questions about this message or Condor in general?
Email address of the local Condor administrator:
ldas_admin_cit__AT__ligo.caltech.edu
The Official Condor Homepage is http://www.cs.wisc.edu/condor
===DedicatedScheduler schedd crash===
===startd crash #1===
-------- Original Message --------
Subject: [Condor] Problem
Date: Tue, 14 Nov 2006 10:57:14 -0800
From: condor__AT__node8.ldas-cit.ligo.caltech.edu
To: ldas_admin_cit__AT__ligo.caltech.edu
This is an automated email from the Condor system
on machine "node8.ldas-cit.ligo.caltech.edu". Do not reply.
"/ldcg/condor/sbin/condor_startd" on "node8.ldas-cit.ligo.caltech.edu"
died due to signal 6.
Condor will automatically restart this process in 10 seconds.
*** Last 200 line(s) of file StartLog:
11/14 10:55:47 vm3: New claim has sufficient rank, preempting current claim.
11/14 10:55:47 vm3: Got RemoteUser (desai@ligo) from request classad
11/14 10:55:47 vm3: State change: preempting claim based on user priority
11/14 10:55:47 vm3: State change: retiring due to preempting claim
11/14 10:55:47 vm3: Changing activity: Busy -> Retiring
11/14 10:55:47 vm3: State change: retirement ended/expired
11/14 10:55:47 vm3: Canceled ClaimLease timer (32715)
11/14 10:55:47 vm3: Changing state and activity: Claimed/Retiring ->
Preempting/Vacating
11/14 10:55:47 vm3: In Starter::kill() with pid 7684, sig 15 (SIGTERM)
11/14 10:55:47 vm3: Got KILL_FRGN_JOB while in Preempting state, ignoring.
11/14 10:55:47 DaemonCore: No more children processes to reap.
11/14 10:55:47 Starter pid 7684 exited with status 0
11/14 10:55:47 ProcAPI::buildFamily failed: parent 7684 not found on system.
11/14 10:55:47 ProcAPI::getProcInfo() pid 7684 does not exist.
11/14 10:55:47 ProcAPI::getProcInfo() pid 7684 does not exist.
11/14 10:55:47 ProcAPI::getProcInfo() pid 7684 does not exist.
11/14 10:55:47 ProcAPI::getProcInfo() pid 7684 does not exist.
11/14 10:55:47 ProcAPI::getProcInfo() pid 7684 does not exist.
11/14 10:55:47 ProcAPI::getProcInfo() pid 7690 does not exist.
11/14 10:55:47 ProcAPI::getProcInfo() pid 7690 does not exist.
11/14 10:55:47 ProcAPI::getProcInfo() pid 7690 does not exist.
11/14 10:55:47 ProcAPI::getProcInfo() pid 7690 does not exist.
11/14 10:55:47 ProcAPI::getProcInfo() pid 7690 does not exist.
11/14 10:55:47 ProcAPI::buildFamily() Found daddypid on the system: 7961
11/14 10:55:47 vm3: State change: starter exited
11/14 10:55:47 Entered vacate_client <10.14.0.12:44107>
ldas-grid.ligo.caltech.edu...
11/14 10:55:47 vm3: State change: preempting claim exists - START is
true or undefined
11/14 10:55:47 vm3: Remote owner is desai@ligo
11/14 10:55:47 vm3: State change: claiming protocol successful
11/14 10:55:47 vm3: Changing state and activity: Preempting/Vacating ->
Claimed/Idle
11/14 10:55:47 vm3: Started ClaimLease timer (33268) w/ 1800 second
lease duration
11/14 10:55:47 vm3: Computing claimWorklifeExpired(); ClaimAge=0,
ClaimWorklife=1800
11/14 10:55:49 Trying to update collector <10.14.0.12:9618>
11/14 10:55:49 Attempting to send update via UDP to collector
ldas-grid.ligo.caltech.edu <10.14.0.12:9618>
11/14 10:55:49 vm3: Sent update to 1 collector(s)
11/14 10:55:49 Warning: can't find resource with ClaimId
(<10.14.1.8:44323>#1162511728#579)
11/14 10:55:51 vm3: Got activate_claim request from shadow
(<10.14.0.12:35453>)
11/14 10:55:51 vm3: Read request ad and starter from shadow.
11/14 10:55:51 Swap space: 10481940
11/14 10:55:51 178084452 kbytes available for "/usr1/condor/execute"
11/14 10:55:51 Looking up RESERVED_DISK parameter
11/14 10:55:51 Reserving 5120 kbytes for file system
11/14 10:55:51 Disk space: 178079332
11/14 10:55:51 vm3: Remote job ID is 9001401.0
11/14 10:55:51 vm3: Remote global job ID is
ldas-grid.ligo.caltech.edu#1163530512#9001401.0
11/14 10:55:51 vm3: JobLeaseDuration defined in job ClassAd: 1200
11/14 10:55:51 vm3: Resetting ClaimLease timer (33268) with new duration
11/14 10:55:51 vm3: About to Create_Process "condor_starter -f -a vm3
ldas-grid.ligo.caltech.edu"
11/14 10:55:51 ProcAPI::buildFamily() Found daddypid on the system: 12594
11/14 10:55:51 vm3: Got RemoteUser (desai@ligo) from request classad
11/14 10:55:51 vm3: Got universe "VANILLA" (5) from request classad
11/14 10:55:51 vm3: State change: claim-activation protocol successful
11/14 10:55:51 vm3: Changing activity: Idle -> Busy
11/14 10:55:52 DaemonCore: Command received via UDP from host
<10.14.1.8:53304>
11/14 10:55:52 DaemonCore: received command 60008 (DC_CHILDALIVE),
calling handler (HandleChildAliveCommand)
11/14 10:55:57 Getting monitoring info for pid 1923
11/14 10:55:57 Trying to update collector <10.14.0.12:9618>
11/14 10:55:57 Attempting to send update via UDP to collector
ldas-grid.ligo.caltech.edu <10.14.0.12:9618>
11/14 10:55:57 vm3: Sent update to 1 collector(s)
11/14 10:56:07 ProcAPI::buildFamily() Found daddypid on the system: 12594
11/14 10:56:28 vm1: match_info called
11/14 10:56:28 vm1: Started match timer (33273) for 120 seconds.
11/14 10:56:30 vm1: Canceled match timer (33273)
11/14 10:56:30 vm1: Schedd addr = <10.14.0.12:44107>
11/14 10:56:30 vm1: Alive interval = 300
11/14 10:56:30 vm1: Received ClaimId from schedd
(<10.14.1.8:44323>#1162511728#582)
11/14 10:56:30 vm1: Rank of this claim is: 0.000000
11/14 10:56:30 vm1: Preempting claim has correct ClaimId.
11/14 10:56:30 vm1: New claim has sufficient rank, preempting current claim.
11/14 10:56:30 vm1: Got RemoteUser (cokelaer@ligo) from request classad
11/14 10:56:30 vm1: State change: preempting claim based on user priority
11/14 10:56:30 vm1: State change: retiring due to preempting claim
11/14 10:56:30 vm1: Changing activity: Busy -> Retiring
11/14 10:56:30 vm1: State change: retirement ended/expired
11/14 10:56:30 vm1: Canceled ClaimLease timer (32713)
11/14 10:56:30 vm1: Changing state and activity: Claimed/Retiring ->
Preempting/Vacating
11/14 10:56:30 vm1: In Starter::kill() with pid 7688, sig 15 (SIGTERM)
11/14 10:56:30 vm1: Got KILL_FRGN_JOB while in Preempting state, ignoring.
11/14 10:56:30 DaemonCore: No more children processes to reap.
11/14 10:56:30 Starter pid 7688 exited with status 0
11/14 10:56:30 ProcAPI::buildFamily failed: parent 7688 not found on system.
11/14 10:56:30 ProcAPI::getProcInfo() pid 7688 does not exist.
11/14 10:56:30 ProcAPI::getProcInfo() pid 7688 does not exist.
11/14 10:56:30 ProcAPI::getProcInfo() pid 7688 does not exist.
11/14 10:56:30 ProcAPI::getProcInfo() pid 7688 does not exist.
11/14 10:56:30 ProcAPI::getProcInfo() pid 7688 does not exist.
11/14 10:56:30 ProcAPI::getProcInfo() pid 7757 does not exist.
11/14 10:56:30 ProcAPI::getProcInfo() pid 7757 does not exist.
11/14 10:56:30 ProcAPI::getProcInfo() pid 7757 does not exist.
11/14 10:56:30 ProcAPI::getProcInfo() pid 7757 does not exist.
11/14 10:56:30 ProcAPI::getProcInfo() pid 7757 does not exist.
11/14 10:56:30 ProcAPI::buildFamily() Found daddypid on the system: 7988
11/14 10:56:30 vm1: State change: starter exited
11/14 10:56:30 Entered vacate_client <10.14.0.12:44107>
ldas-grid.ligo.caltech.edu...
11/14 10:56:30 vm1: State change: preempting claim exists - START is
true or undefined
11/14 10:56:30 vm1: Remote owner is cokelaer@ligo
11/14 10:56:30 vm1: State change: claiming protocol successful
11/14 10:56:30 vm1: Changing state and activity: Preempting/Vacating ->
Claimed/Idle
11/14 10:56:30 vm1: Started ClaimLease timer (33274) w/ 1800 second
lease duration
11/14 10:56:30 vm1: Computing claimWorklifeExpired(); ClaimAge=0,
ClaimWorklife=1800
11/14 10:56:31 Warning: can't find resource with ClaimId
(<10.14.1.8:44323>#1162511728#581)
11/14 10:56:31 vm1: State change: received RELEASE_CLAIM command
11/14 10:56:31 vm1: Canceled ClaimLease timer (33274)
11/14 10:56:31 vm1: Changing state and activity: Claimed/Idle ->
Preempting/Vacating
11/14 10:56:31 Entered vacate_client <10.14.0.12:44107>
ldas-grid.ligo.caltech.edu...
11/14 10:56:31 vm1: State change: No preempting claim, returning to owner
11/14 10:56:31 vm1: Changing state and activity: Preempting/Vacating ->
Owner/Idle
11/14 10:56:31 vm1: State change: IS_OWNER is false
11/14 10:56:31 vm1: Changing state: Owner -> Unclaimed
11/14 10:56:31 vm1: State change: START_BACKFILL is TRUE
11/14 10:56:31 vm1: Changing state: Unclaimed -> Backfill
11/14 10:56:31 Warning: can't find resource with ClaimId
(<10.14.1.8:44323>#1162511728#582)
11/14 10:56:32 ProcAPI::buildFamily() Found daddypid on the system: 7686
11/14 10:56:32 ProcAPI::buildFamily() Found daddypid on the system: 12521
11/14 10:56:32 Entering BOINC_BackfillMgr::spawnClient()
11/14 10:56:32 About to Create_Process "condor_starter -f -append boinc
-job-keyword boinc"
11/14 10:56:32 ProcAPI::buildFamily() Found daddypid on the system: 12688
11/14 10:56:32 Spawned BOINC starter: (pid 12688)
11/14 10:56:32 New BOINC_BackfillVM (id 1) created
11/14 10:56:32 State change: BOINC client running for vm1
11/14 10:56:32 vm1: Changing activity: Idle -> Busy
11/14 10:56:32 Trying to update collector <10.14.0.12:9618>
11/14 10:56:32 Attempting to send update via UDP to collector
ldas-grid.ligo.caltech.edu <10.14.0.12:9618>
11/14 10:56:32 vm1: Sent update to 1 collector(s)
11/14 10:56:33 DaemonCore: Command received via UDP from host
<10.14.1.8:53307>
11/14 10:56:33 DaemonCore: received command 60008 (DC_CHILDALIVE),
calling handler (HandleChildAliveCommand)
11/14 10:56:57 ProcAPI::buildFamily() Found daddypid on the system: 12594
11/14 10:57:02 vm1: match_info called
11/14 10:57:02 vm1: Received match <10.14.1.8:44323>#1162511728#605
11/14 10:57:02 vm1: Started match timer (33277) for 120 seconds.
11/14 10:57:02 vm1: State change: match notification protocol successful
11/14 10:57:02 vm1: Set destination state to Matched
11/14 10:57:02 vm1: Changing activity: Busy -> Killing
11/14 10:57:02 In Starter::kill() with pid 12688, sig 3 (SIGQUIT)
11/14 10:57:02 BOINC_BackfillMgr::killClient(): told BOINC starter (pid
12688) to hardkill
11/14 10:57:02 vm2: match_info called
11/14 10:57:02 vm2: Started match timer (33281) for 120 seconds.
11/14 10:57:02 DaemonCore: No more children processes to reap.
11/14 10:57:02 BOINC client (pid 12688) exited with status 0
11/14 10:57:02 Canceled hardkill-starter timer (33278)
11/14 10:57:02 ProcAPI::buildFamily() Parent pid 12688 is gone. Found
descendant 12690 via ancestor environment tracking and assigning as new
"parent".
11/14 10:57:02 ProcAPI::getProcInfo() pid 12688 does not exist.
11/14 10:57:02 ProcAPI::getProcInfo() pid 12688 does not exist.
11/14 10:57:02 ProcAPI::getProcInfo() pid 12688 does not exist.
11/14 10:57:02 ProcAPI::getProcInfo() pid 12688 does not exist.
11/14 10:57:02 ProcAPI::getProcInfo() pid 12688 does not exist.
11/14 10:57:02 BOINC_BackfillVM (id 1) deleted
11/14 10:57:02 vm1: State change: starter exited
11/14 10:57:02 vm1: Changing state and activity: Backfill/Killing ->
Matched/Idle
11/14 10:57:02 vm1: Canceled match timer (33277)
11/14 10:57:02 vm1: Schedd addr = <10.14.0.12:44107>
11/14 10:57:02 vm1: Alive interval = 300
11/14 10:57:02 vm1: Received ClaimId from schedd
(<10.14.1.8:44323>#1162511728#605)
11/14 10:57:02 vm1: Rank of this claim is: 0.000000
11/14 10:57:02 vm1: Request accepted.
11/14 10:57:02 vm1: Remote owner is cokelaer@ligo
11/14 10:57:02 vm1: State change: claiming protocol successful
11/14 10:57:02 vm1: Changing state: Matched -> Claimed
11/14 10:57:02 vm1: Started ClaimLease timer (33282) w/ 1800 second
lease duration
11/14 10:57:02 vm1: Computing claimWorklifeExpired(); ClaimAge=0,
ClaimWorklife=1800
11/14 10:57:02 vm2: Canceled match timer (33281)
11/14 10:57:02 vm2: Schedd addr = <10.14.0.12:44107>
11/14 10:57:02 vm2: Alive interval = 300
11/14 10:57:02 vm2: Received ClaimId from schedd
(<10.14.1.8:44323>#1162511728#583)
11/14 10:57:02 vm2: Rank of this claim is: 0.000000
11/14 10:57:02 vm2: Preempting claim has correct ClaimId.
11/14 10:57:02 vm2: New claim has sufficient rank, preempting current claim.
11/14 10:57:02 vm2: Got RemoteUser (cokelaer@ligo) from request classad
11/14 10:57:02 vm2: State change: preempting claim based on user priority
11/14 10:57:02 vm2: State change: retiring due to preempting claim
11/14 10:57:02 vm2: Changing activity: Busy -> Retiring
11/14 10:57:02 vm2: State change: retirement ended/expired
11/14 10:57:02 vm2: Canceled ClaimLease timer (32714)
11/14 10:57:02 vm2: Changing state and activity: Claimed/Retiring ->
Preempting/Vacating
11/14 10:57:02 vm2: In Starter::kill() with pid 7686, sig 15 (SIGTERM)
11/14 10:57:02 vm2: Got KILL_FRGN_JOB while in Preempting state, ignoring.
11/14 10:57:02 DaemonCore: No more children processes to reap.
11/14 10:57:02 Starter pid 7686 exited with status 0
11/14 10:57:02 ProcAPI::buildFamily failed: parent 7686 not found on system.
11/14 10:57:02 ProcAPI::getProcInfo() pid 7686 does not exist.
11/14 10:57:02 ProcAPI::getProcInfo() pid 7686 does not exist.
11/14 10:57:02 ProcAPI::getProcInfo() pid 7686 does not exist.
11/14 10:57:02 ProcAPI::getProcInfo() pid 7686 does not exist.
11/14 10:57:02 ProcAPI::getProcInfo() pid 7686 does not exist.
11/14 10:57:02 ProcAPI::getProcInfo() pid 7726 does not exist.
11/14 10:57:02 ProcAPI::getProcInfo() pid 7726 does not exist.
11/14 10:57:02 ProcAPI::getProcInfo() pid 7726 does not exist.
11/14 10:57:02 ProcAPI::getProcInfo() pid 7726 does not exist.
11/14 10:57:02 ProcAPI::getProcInfo() pid 7726 does not exist.
11/14 10:57:02 ProcAPI::getProcInfo() pid 7972 does not exist.
11/14 10:57:02 ProcAPI::getProcInfo() pid 7972 does not exist.
11/14 10:57:02 ProcAPI::getProcInfo() pid 7972 does not exist.
11/14 10:57:02 ProcAPI::getProcInfo() pid 7972 does not exist.
11/14 10:57:02 ProcAPI::getProcInfo() pid 7972 does not exist.
11/14 10:57:02 ProcAPI::getProcInfo() pid 8071 does not exist.
11/14 10:57:02 ProcAPI::getProcInfo() pid 8071 does not exist.
11/14 10:57:02 ProcAPI::getProcInfo() pid 8071 does not exist.
11/14 10:57:02 ProcAPI::getProcInfo() pid 8071 does not exist.
11/14 10:57:02 ProcAPI::getProcInfo() pid 8071 does not exist.
11/14 10:57:02 ProcAPI::buildFamily failed: parent 7972 not found on system.
*** End of file StartLog
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Questions about this message or Condor in general?
Email address of the local Condor administrator:
ldas_admin_cit__AT__ligo.caltech.edu
The Official Condor Homepage is http://www.cs.wisc.edu/condor
===startd crash #1===
===startd crash #2===
-------- Original Message --------
Subject: [Condor] Problem
Date: Tue, 14 Nov 2006 11:05:42 -0800
From: condor__AT__node14.ldas-cit.ligo.caltech.edu
To: ldas_admin_cit__AT__ligo.caltech.edu
This is an automated email from the Condor system
on machine "node14.ldas-cit.ligo.caltech.edu". Do not reply.
"/ldcg/condor/sbin/condor_startd" on "node14.ldas-cit.ligo.caltech.edu"
died due to signal 6.
Condor will automatically restart this process in 10 seconds.
*** Last 200 line(s) of file StartLog:
11/14 11:02:16 ProcAPI::getProcInfo() pid 13445 does not exist.
11/14 11:02:16 ProcAPI::getProcInfo() pid 13445 does not exist.
11/14 11:02:16 vm1: State change: starter exited
11/14 11:02:16 vm1: Changing activity: Busy -> Idle
11/14 11:02:16 vm1: Computing claimWorklifeExpired(); ClaimAge=274,
ClaimWorklife=1800
11/14 11:02:18 vm1: Computing claimWorklifeExpired(); ClaimAge=276,
ClaimWorklife=1800
11/14 11:02:19 vm1: State change: received RELEASE_CLAIM command
11/14 11:02:19 vm1: Canceled ClaimLease timer (35527)
11/14 11:02:19 vm1: Changing state and activity: Claimed/Idle ->
Preempting/Vacating
11/14 11:02:19 Entered vacate_client <10.14.0.12:44107>
ldas-grid.ligo.caltech.edu...
11/14 11:02:19 vm1: State change: No preempting claim, returning to owner
11/14 11:02:19 vm1: Changing state and activity: Preempting/Vacating ->
Owner/Idle
11/14 11:02:19 vm1: State change: IS_OWNER is false
11/14 11:02:19 vm1: Changing state: Owner -> Unclaimed
11/14 11:02:19 vm1: State change: START_BACKFILL is TRUE
11/14 11:02:19 vm1: Changing state: Unclaimed -> Backfill
11/14 11:02:19 Warning: can't find resource with ClaimId
(<10.14.1.14:35408>#1162513127#1078)
11/14 11:02:20 Trying to update collector <10.14.0.12:9618>
11/14 11:02:20 Attempting to send update via UDP to collector
ldas-grid.ligo.caltech.edu <10.14.0.12:9618>
11/14 11:02:20 vm1: Sent update to 1 collector(s)
11/14 11:02:24 Entering BOINC_BackfillMgr::spawnClient()
11/14 11:02:24 About to Create_Process "condor_starter -f -append boinc
-job-keyword boinc"
11/14 11:02:24 ProcAPI::buildFamily() Found daddypid on the system: 13453
11/14 11:02:24 Spawned BOINC starter: (pid 13453)
11/14 11:02:24 New BOINC_BackfillVM (id 1) created
11/14 11:02:24 State change: BOINC client running for vm1
11/14 11:02:24 vm1: Changing activity: Idle -> Busy
11/14 11:02:25 DaemonCore: Command received via UDP from host
<10.14.1.14:54349>
11/14 11:02:25 DaemonCore: received command 60008 (DC_CHILDALIVE),
calling handler (HandleChildAliveCommand)
11/14 11:02:28 Trying to update collector <10.14.0.12:9618>
11/14 11:02:28 Attempting to send update via UDP to collector
ldas-grid.ligo.caltech.edu <10.14.0.12:9618>
11/14 11:02:28 vm1: Sent update to 1 collector(s)
11/14 11:02:42 Swap space: 10482320
11/14 11:02:42 125487080 kbytes available for "/usr1/condor/execute"
11/14 11:02:42 Looking up RESERVED_DISK parameter
11/14 11:02:42 Reserving 5120 kbytes for file system
11/14 11:02:42 Disk space: 125481960
11/14 11:02:46 Trying to update collector <10.14.0.12:9618>
11/14 11:02:46 Attempting to send update via UDP to collector
ldas-grid.ligo.caltech.edu <10.14.0.12:9618>
11/14 11:02:46 vm1: Sent update to 1 collector(s)
11/14 11:02:47 Trying to update collector <10.14.0.12:9618>
11/14 11:02:47 Attempting to send update via UDP to collector
ldas-grid.ligo.caltech.edu <10.14.0.12:9618>
11/14 11:02:47 vm2: Sent update to 1 collector(s)
11/14 11:02:48 Trying to update collector <10.14.0.12:9618>
11/14 11:02:48 Attempting to send update via UDP to collector
ldas-grid.ligo.caltech.edu <10.14.0.12:9618>
11/14 11:02:48 vm3: Sent update to 1 collector(s)
11/14 11:02:49 Trying to update collector <10.14.0.12:9618>
11/14 11:02:49 Attempting to send update via UDP to collector
ldas-grid.ligo.caltech.edu <10.14.0.12:9618>
11/14 11:02:49 vm4: Sent update to 1 collector(s)
11/14 11:02:54 ProcAPI::buildFamily() Found daddypid on the system: 10524
11/14 11:02:59 ProcAPI::buildFamily() Found daddypid on the system: 10634
11/14 11:03:04 ProcAPI::buildFamily() Found daddypid on the system: 12422
11/14 11:03:17 Getting monitoring info for pid 1889
11/14 11:03:44 ProcAPI::buildFamily() Found daddypid on the system: 10524
11/14 11:03:49 ProcAPI::buildFamily() Found daddypid on the system: 10634
11/14 11:03:54 ProcAPI::buildFamily() Found daddypid on the system: 12422
11/14 11:04:34 ProcAPI::buildFamily() Found daddypid on the system: 10524
11/14 11:04:36 DaemonCore: Command received via UDP from host
<10.14.0.12:44264>
11/14 11:04:36 DaemonCore: received command 441 (ALIVE), calling handler
(command_handler)
11/14 11:04:36 DaemonCore: Command received via UDP from host
<10.14.0.12:44264>
11/14 11:04:36 DaemonCore: received command 441 (ALIVE), calling handler
(command_handler)
11/14 11:04:36 DaemonCore: Command received via UDP from host
<10.14.0.12:44264>
11/14 11:04:36 DaemonCore: received command 441 (ALIVE), calling handler
(command_handler)
11/14 11:04:39 ProcAPI::buildFamily() Found daddypid on the system: 10634
11/14 11:04:44 ProcAPI::buildFamily() Found daddypid on the system: 12422
11/14 11:04:59 vm1: match_info called
11/14 11:04:59 vm1: Received match <10.14.1.14:35408>#1162513127#1080
11/14 11:04:59 vm1: Started match timer (35549) for 120 seconds.
11/14 11:04:59 vm1: State change: match notification protocol successful
11/14 11:04:59 vm1: Set destination state to Matched
11/14 11:04:59 vm1: Changing activity: Busy -> Killing
11/14 11:04:59 In Starter::kill() with pid 13453, sig 3 (SIGQUIT)
11/14 11:04:59 BOINC_BackfillMgr::killClient(): told BOINC starter (pid
13453) to hardkill
11/14 11:04:59 DaemonCore: No more children processes to reap.
11/14 11:04:59 BOINC client (pid 13453) exited with status 0
11/14 11:04:59 Canceled hardkill-starter timer (35550)
11/14 11:04:59 ProcAPI::buildFamily() Parent pid 13453 is gone. Found
descendant 13455 via ancestor environment tracking and assigning as new
"parent".
11/14 11:04:59 ProcAPI::getProcInfo() pid 13453 does not exist.
11/14 11:04:59 ProcAPI::getProcInfo() pid 13453 does not exist.
11/14 11:04:59 ProcAPI::getProcInfo() pid 13453 does not exist.
11/14 11:04:59 ProcAPI::getProcInfo() pid 13453 does not exist.
11/14 11:04:59 ProcAPI::getProcInfo() pid 13453 does not exist.
11/14 11:04:59 BOINC_BackfillVM (id 1) deleted
11/14 11:04:59 vm1: State change: starter exited
11/14 11:04:59 vm1: Changing state and activity: Backfill/Killing ->
Matched/Idle
11/14 11:05:03 Trying to update collector <10.14.0.12:9618>
11/14 11:05:03 Attempting to send update via UDP to collector
ldas-grid.ligo.caltech.edu <10.14.0.12:9618>
11/14 11:05:03 vm1: Sent update to 1 collector(s)
11/14 11:05:15 vm1: Canceled match timer (35549)
11/14 11:05:15 vm1: Schedd addr = <10.14.0.18:51007>
11/14 11:05:15 vm1: Alive interval = 300
11/14 11:05:15 vm1: Received ClaimId from schedd
(<10.14.1.14:35408>#1162513127#1080)
11/14 11:05:15 vm1: Rank of this claim is: 0.000000
11/14 11:05:15 vm1: Request accepted.
11/14 11:05:15 vm1: Remote owner is
DedicatedScheduler__AT__ldas-pcdev1.ligo.caltech.edu
11/14 11:05:15 vm1: State change: claiming protocol successful
11/14 11:05:15 vm1: Changing state: Matched -> Claimed
11/14 11:05:15 vm1: Started ClaimLease timer (35552) w/ 1800 second
lease duration
11/14 11:05:15 vm1: Computing claimWorklifeExpired(); ClaimAge=0,
ClaimWorklife=1800
11/14 11:05:19 vm1: Computing claimWorklifeExpired(); ClaimAge=4,
ClaimWorklife=1800
11/14 11:05:19 Trying to update collector <10.14.0.12:9618>
11/14 11:05:19 Attempting to send update via UDP to collector
ldas-grid.ligo.caltech.edu <10.14.0.12:9618>
11/14 11:05:19 vm1: Sent update to 1 collector(s)
11/14 11:05:19 vm1: Got activate_claim request from shadow
(<10.14.0.18:57292>)
11/14 11:05:19 vm1: Read request ad and starter from shadow.
11/14 11:05:19 Swap space: 10482320
11/14 11:05:19 125487052 kbytes available for "/usr1/condor/execute"
11/14 11:05:19 Looking up RESERVED_DISK parameter
11/14 11:05:19 Reserving 5120 kbytes for file system
11/14 11:05:19 Disk space: 125481932
11/14 11:05:19 vm1: Remote job ID is 22759.0
11/14 11:05:19 vm1: Remote global job ID is
ldas-pcdev1.ligo.caltech.edu#1163443634#22759.0
11/14 11:05:19 vm1: JobLeaseDuration defined in job ClassAd: 1200
11/14 11:05:19 vm1: Resetting ClaimLease timer (35552) with new duration
11/14 11:05:19 vm1: About to Create_Process "condor_starter -f -a vm1
ldas-pcdev1i.ligo"
11/14 11:05:19 ProcAPI::buildFamily() Found daddypid on the system: 13484
11/14 11:05:19 vm1: Got RemoteUser (dbrown@ligo) from request classad
11/14 11:05:19 vm1: Got universe "PARALLEL" (11) from request classad
11/14 11:05:19 vm1: State change: claim-activation protocol successful
11/14 11:05:19 vm1: Changing activity: Idle -> Busy
11/14 11:05:20 DaemonCore: Command received via UDP from host
<10.14.1.14:54358>
11/14 11:05:20 DaemonCore: received command 60008 (DC_CHILDALIVE),
calling handler (HandleChildAliveCommand)
11/14 11:05:22 vm1: Called deactivate_claim_forcibly()
11/14 11:05:22 vm1: In Starter::kill() with pid 13484, sig 3 (SIGQUIT)
11/14 11:05:23 Trying to update collector <10.14.0.12:9618>
11/14 11:05:23 Attempting to send update via UDP to collector
ldas-grid.ligo.caltech.edu <10.14.0.12:9618>
11/14 11:05:23 vm1: Sent update to 1 collector(s)
11/14 11:05:24 ProcAPI::buildFamily() Found daddypid on the system: 10524
11/14 11:05:29 ProcAPI::buildFamily() Found daddypid on the system: 10634
11/14 11:05:29 vm2: match_info called
11/14 11:05:29 vm2: Started match timer (35559) for 120 seconds.
11/14 11:05:29 vm4: match_info called
11/14 11:05:29 vm4: Started match timer (35561) for 120 seconds.
11/14 11:05:30 vm2: Canceled match timer (35559)
11/14 11:05:30 vm2: Schedd addr = <10.14.0.12:44107>
11/14 11:05:30 vm2: Alive interval = 300
11/14 11:05:30 vm2: Received ClaimId from schedd
(<10.14.1.14:35408>#1162513127#1068)
11/14 11:05:30 vm2: Rank of this claim is: 0.000000
11/14 11:05:30 vm2: Preempting claim has correct ClaimId.
11/14 11:05:30 vm2: New claim has sufficient rank, preempting current claim.
11/14 11:05:30 vm2: Got RemoteUser (cokelaer@ligo) from request classad
11/14 11:05:30 vm2: State change: preempting claim based on user priority
11/14 11:05:30 vm2: State change: retiring due to preempting claim
11/14 11:05:30 vm2: Changing activity: Busy -> Retiring
11/14 11:05:30 vm2: State change: retirement ended/expired
11/14 11:05:30 vm2: Canceled ClaimLease timer (35007)
11/14 11:05:30 vm2: Changing state and activity: Claimed/Retiring ->
Preempting/Vacating
11/14 11:05:30 vm2: In Starter::kill() with pid 10634, sig 15 (SIGTERM)
11/14 11:05:30 vm4: Canceled match timer (35561)
11/14 11:05:30 vm4: Schedd addr = <10.14.0.12:44107>
11/14 11:05:30 vm4: Alive interval = 300
11/14 11:05:30 vm4: Received ClaimId from schedd
(<10.14.1.14:35408>#1162513127#1069)
11/14 11:05:30 vm4: Rank of this claim is: 0.000000
11/14 11:05:30 vm4: Preempting claim has correct ClaimId.
11/14 11:05:30 vm4: New claim has sufficient rank, preempting current claim.
11/14 11:05:30 vm4: Got RemoteUser (cokelaer@ligo) from request classad
11/14 11:05:30 vm4: State change: preempting claim based on user priority
11/14 11:05:30 vm4: State change: retiring due to preempting claim
11/14 11:05:30 vm4: Changing activity: Busy -> Retiring
11/14 11:05:30 vm4: State change: retirement ended/expired
11/14 11:05:30 vm4: Canceled ClaimLease timer (35008)
11/14 11:05:30 vm4: Changing state and activity: Claimed/Retiring ->
Preempting/Vacating
11/14 11:05:30 vm4: In Starter::kill() with pid 10524, sig 15 (SIGTERM)
11/14 11:05:30 DaemonCore: No more children processes to reap.
11/14 11:05:30 Starter pid 13484 exited with status 0
11/14 11:05:30 vm1: Canceled hardkill-starter timer (35557)
11/14 11:05:30 ProcAPI::buildFamily failed: parent 13484 not found on
system.
11/14 11:05:30 ProcAPI::getProcInfo() pid 13484 does not exist.
11/14 11:05:30 ProcAPI::getProcInfo() pid 13484 does not exist.
11/14 11:05:30 ProcAPI::getProcInfo() pid 13484 does not exist.
11/14 11:05:30 ProcAPI::getProcInfo() pid 13484 does not exist.
11/14 11:05:30 ProcAPI::getProcInfo() pid 13484 does not exist.
11/14 11:05:30 vm1: State change: starter exited
11/14 11:05:30 vm1: Changing activity: Busy -> Idle
11/14 11:05:30 vm1: Computing claimWorklifeExpired(); ClaimAge=15,
ClaimWorklife=1800
11/14 11:05:30 vm4: Got KILL_FRGN_JOB while in Preempting state, ignoring.
11/14 11:05:30 DaemonCore: No more children processes to reap.
11/14 11:05:30 Starter pid 10524 exited with status 0
11/14 11:05:30 ProcAPI::buildFamily failed: parent 10524 not found on
system.
11/14 11:05:30 ProcAPI::getProcInfo() pid 10524 does not exist.
11/14 11:05:30 ProcAPI::getProcInfo() pid 10524 does not exist.
11/14 11:05:30 ProcAPI::getProcInfo() pid 10524 does not exist.
11/14 11:05:30 ProcAPI::getProcInfo() pid 10524 does not exist.
11/14 11:05:30 ProcAPI::getProcInfo() pid 10524 does not exist.
11/14 11:05:30 ProcAPI::getProcInfo() pid 10542 does not exist.
11/14 11:05:30 ProcAPI::getProcInfo() pid 10542 does not exist.
11/14 11:05:30 ProcAPI::getProcInfo() pid 10542 does not exist.
11/14 11:05:30 ProcAPI::getProcInfo() pid 10542 does not exist.
11/14 11:05:30 ProcAPI::getProcInfo() pid 10542 does not exist.
11/14 11:05:30 ProcAPI::getProcInfo() pid 10646 does not exist.
11/14 11:05:30 ProcAPI::getProcInfo() pid 10646 does not exist.
11/14 11:05:30 ProcAPI::getProcInfo() pid 10646 does not exist.
11/14 11:05:30 ProcAPI::getProcInfo() pid 10646 does not exist.
11/14 11:05:30 ProcAPI::getProcInfo() pid 10646 does not exist.
11/14 11:05:30 ProcAPI::getProcInfo() pid 10812 does not exist.
11/14 11:05:30 ProcAPI::getProcInfo() pid 10812 does not exist.
11/14 11:05:30 ProcAPI::getProcInfo() pid 10812 does not exist.
11/14 11:05:30 ProcAPI::getProcInfo() pid 10812 does not exist.
11/14 11:05:30 ProcAPI::getProcInfo() pid 10812 does not exist.
11/14 11:05:30 ProcAPI::buildFamily failed: parent 10646 not found on
system.
*** End of file StartLog
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Questions about this message or Condor in general?
Email address of the local Condor administrator:
ldas_admin_cit__AT__ligo.caltech.edu
The Official Condor Homepage is http://www.cs.wisc.edu/condor
===startd crash #2===
--
Erik A. Espinoza
Systems Administrator
LIGO/Caltech - MS 18-34
Pasadena, CA 91125
Ph: 626-395-8517
===========================================================================
Date of creation: Wed Nov 15 15:54:14 2006 (1163627657)
Subject: Actions
Assigned to gthain by bgietzel
===========================================================================
Date of actions: Thu Nov 16 14:17:27 2006 (1163708247)
Date: Thu, 16 Nov 2006 14:26:06 -0600
From: Greg Thain <gthain__AT__cs.wisc.edu>
To: condor-support__AT__cs.wisc.edu
Subject: Re: [condor-support #1739] LIGO: DedicatedScheduler schedd
crashed on condor_rm
Erik & Stuart:
Is there a core file from the schedd?
-Greg
===========================================================================
Date mail was appended: Thu Nov 16 14:26:12 2006 (1163708772)
Date: Thu, 16 Nov 2006 13:09:23 -0800
From: Stuart Anderson <anderson__AT__ligo.caltech.edu>
To: condor-support response tracking system <condor-support__AT__cs.wisc.edu>
CC: espinoza__AT__ligo.caltech.edu
Subject: Re: [condor-support #1739] LIGO: DedicatedScheduler schedd
crashed on condor_rm
On Thu, Nov 16, 2006 at 02:26:12PM -0600, condor-support response tracking system wrote:
>
> Erik & Stuart:
>
> Is there a core file from the schedd?
No, but there is the following syslog message:
Nov 14 11:03:42 ldas-pcdev1 kernel: condor_schedd[31560]: segfault at ffffffffff
ffffff rip 00000000005038c7 rsp 00007fffe529df50 error 4
--
Stuart Anderson anderson__AT__ligo.caltech.edu
http://www.ligo.caltech.edu/~anderson
===========================================================================
Date mail was appended: Thu Nov 16 15:09:50 2006 (1163711391)
Date: Thu, 16 Nov 2006 15:14:17 -0600
From: Greg Thain <gthain__AT__cs.wisc.edu>
To: condor-support__AT__cs.wisc.edu
Subject: Re: [condor-support #1739] LIGO: DedicatedScheduler schedd
crashed on condor_rm
condor-support response tracking system wrote:
> On Thu, Nov 16, 2006 at 02:26:12PM -0600, condor-support response tracking system wrote:
>> Erik & Stuart:
>>
>> Is there a core file from the schedd?
>
> No, but there is the following syslog message:
>
> Nov 14 11:03:42 ldas-pcdev1 kernel: condor_schedd[31560]: segfault at ffffffffff
> ffffff rip 00000000005038c7 rsp 00007fffe529df50 error 4
>
Can I get, say, the last 10k lines from the SchedLog?
I assume the Dedicated Scheduler crash is relatively rare?
-Greg
===========================================================================
Date mail was appended: Thu Nov 16 15:14:23 2006 (1163711663)
Date: Thu, 16 Nov 2006 15:46:10 -0600
From: Greg Thain <gthain__AT__cs.wisc.edu>
To: condor-support__AT__cs.wisc.edu
Subject: Re: [condor-support #1739] LIGO: DedicatedScheduler schedd
crashed on condor_rm
Erik:
If you have
/proc/sys/fs/suid_dumpable
or
/proc/sys/kernel/suid_dumpable
on this machine, can you do
echo 2 > /proc/sys/kernel/suid_dumpable
This will enable setuid programs (like the schedd) to generate core
files. The core files will be only readable by root, so it should be
secure.
-Greg
===========================================================================
Date mail was appended: Thu Nov 16 15:46:14 2006 (1163713575)
Date: Thu, 16 Nov 2006 13:58:57 -0800
From: Stuart Anderson <anderson__AT__ligo.caltech.edu>
To: condor-support response tracking system <condor-support__AT__cs.wisc.edu>
CC: espinoza__AT__ligo.caltech.edu
Subject: Re: [condor-support #1739] LIGO: DedicatedScheduler schedd
crashed on condor_rm
--CE+1k2dSO48ffgeK
On Thu, Nov 16, 2006 at 03:14:23PM -0600, condor-support response tracking system wrote:
> condor-support response tracking system wrote:
> > On Thu, Nov 16, 2006 at 02:26:12PM -0600, condor-support response tracking system wrote:
> >> Erik & Stuart:
> >>
> >> Is there a core file from the schedd?
> >
> > No, but there is the following syslog message:
> >
> > Nov 14 11:03:42 ldas-pcdev1 kernel: condor_schedd[31560]: segfault at ffffffffff
> > ffffff rip 00000000005038c7 rsp 00007fffe529df50 error 4
> >
>
> Can I get, say, the last 10k lines from the SchedLog?
Attached.
>
> I assume the Dedicated Scheduler crash is relatively rare?
Yes. This is the first crash of the schedd we have had since upgrading
to 6.8.2 on Oct 17.
--
Stuart Anderson anderson__AT__ligo.caltech.edu
http://www.ligo.caltech.edu/~anderson
--CE+1k2dSO48ffgeK
11/11 15:32:14 (pid:31560) Dedicated job abnormally ended, releasing claim
11/11 15:32:14 (pid:31560) ERROR in releaseClaim(): NULL m_rec
11/11 15:32:14 (pid:31560) Dedicated job abnormally ended, releasing claim
11/11 15:32:14 (pid:31560) ERROR in releaseClaim(): NULL m_rec
11/11 15:32:14 (pid:31560) Dedicated job abnormally ended, releasing claim
11/11 15:32:14 (pid:31560) ERROR in releaseClaim(): NULL m_rec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:32:14 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.54:34966>
11/11 15:32:14 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:32:14 (pid:31560) Got VACATE_SERVICE from <10.14.1.54:34966>
11/11 15:32:23 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.73:59652>
11/11 15:32:23 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:32:23 (pid:31560) Got VACATE_SERVICE from <10.14.1.73:59652>
11/11 15:32:23 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.36:48375>
11/11 15:32:23 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:32:23 (pid:31560) Got VACATE_SERVICE from <10.14.1.36:48375>
11/11 15:32:23 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.209:59735>
11/11 15:32:23 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:32:23 (pid:31560) Got VACATE_SERVICE from <10.14.1.209:59735>
11/11 15:32:23 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.194:39437>
11/11 15:32:23 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:32:23 (pid:31560) Got VACATE_SERVICE from <10.14.1.194:39437>
11/11 15:32:23 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.158:45291>
11/11 15:32:23 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:32:23 (pid:31560) Got VACATE_SERVICE from <10.14.1.158:45291>
11/11 15:32:23 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.2.68:40554>
11/11 15:32:23 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:32:23 (pid:31560) Got VACATE_SERVICE from <10.14.2.68:40554>
11/11 15:32:23 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.120:45263>
11/11 15:32:23 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:32:23 (pid:31560) Got VACATE_SERVICE from <10.14.1.120:45263>
11/11 15:32:23 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.74:57072>
11/11 15:32:23 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:32:23 (pid:31560) Got VACATE_SERVICE from <10.14.1.74:57072>
11/11 15:32:23 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.148:60779>
11/11 15:32:23 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:32:23 (pid:31560) Got VACATE_SERVICE from <10.14.1.148:60779>
11/11 15:32:23 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.116:45810>
11/11 15:32:23 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:32:23 (pid:31560) Got VACATE_SERVICE from <10.14.1.116:45810>
11/11 15:32:23 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.113:54079>
11/11 15:32:23 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:32:23 (pid:31560) Got VACATE_SERVICE from <10.14.1.113:54079>
11/11 15:32:23 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.177:42287>
11/11 15:32:23 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:32:23 (pid:31560) Got VACATE_SERVICE from <10.14.1.177:42287>
11/11 15:32:23 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.215:47712>
11/11 15:32:23 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:32:23 (pid:31560) Got VACATE_SERVICE from <10.14.1.215:47712>
11/11 15:32:23 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.175:35553>
11/11 15:32:23 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:32:23 (pid:31560) Got VACATE_SERVICE from <10.14.1.175:35553>
11/11 15:32:31 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.215:42493>
11/11 15:32:31 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:32:31 (pid:31560) Got VACATE_SERVICE from <10.14.1.215:42493>
11/11 15:37:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 15:37:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 15:37:13 (pid:31560) Inserting new attribute Scheduler into non-activ=
e cluster cid=3D22319 acid=3D-1
11/11 15:38:01 (pid:31560) Activity on stashed negotiator socket
11/11 15:38:01 (pid:31560) Negotiating for owner: DedicatedScheduler@ldas-p=
cdev1.ligo.caltech.edu
11/11 15:38:02 (pid:31560) Out of servers - 0 reqs matched, 16 reqs idle, 1=
6 reqs rejected
11/11 15:38:05 (pid:31560) Activity on stashed negotiator socket
11/11 15:38:05 (pid:31560) Negotiating for owner: DedicatedScheduler@ldas-p=
cdev1.ligo.caltech.edu
11/11 15:38:06 (pid:31560) Out of requests - 16 reqs matched, 0 reqs idle
11/11 15:38:08 (pid:31560) Inserting new attribute Scheduler into non-activ=
e cluster cid=3D22319 acid=3D-1
11/11 15:38:09 (pid:31560) Starting add_shadow_birthdate(22319.0)
11/11 15:38:09 (pid:31560) Started shadow for job 22319.0 on "<10.14.2.8:33=
080>", (shadow pid =3D 7900)
11/11 15:38:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 15:38:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 15:38:10 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.0.18:45628>
11/11 15:38:10 (pid:31560) DaemonCore: received command 71003 (GIVE_MATCHES=
), calling handler (DedicatedScheduler::giveMatches)
11/11 15:43:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 15:43:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 15:46:15 (pid:31560) In DedicatedScheduler::reaper pid 7900 has statu=
s 25600
11/11 15:46:15 (pid:31560) Shadow pid 7900 exited with status 100
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) DedicatedScheduler::deallocMatchRec
11/11 15:46:15 (pid:31560) statfs() failed: 13/Permission denied
11/11 15:48:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 15:48:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 15:53:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 15:53:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 15:56:15 (pid:31560) Resource vm2__AT__node272.ldas-cit.ligo.caltech.edu h=
as been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm3__AT__node235.ldas-cit.ligo.caltech.edu h=
as been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm3__AT__node221.ldas-cit.ligo.caltech.edu h=
as been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm1__AT__node210.ldas-cit.ligo.caltech.edu h=
as been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm1__AT__node258.ldas-cit.ligo.caltech.edu h=
as been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm2__AT__node168.ldas-cit.ligo.caltech.edu h=
as been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm1__AT__node191.ldas-cit.ligo.caltech.edu h=
as been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm2__AT__node154.ldas-cit.ligo.caltech.edu h=
as been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm4__AT__node107.ldas-cit.ligo.caltech.edu h=
as been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm4__AT__node125.ldas-cit.ligo.caltech.edu h=
as been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm1__AT__node272.ldas-cit.ligo.caltech.edu h=
as been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm1__AT__node45.ldas-cit.ligo.caltech.edu ha=
s been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm1__AT__node284.ldas-cit.ligo.caltech.edu h=
as been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm1__AT__node185.ldas-cit.ligo.caltech.edu h=
as been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm1__AT__node123.ldas-cit.ligo.caltech.edu h=
as been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm2__AT__node100.ldas-cit.ligo.caltech.edu h=
as been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm1__AT__node262.ldas-cit.ligo.caltech.edu h=
as been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm4__AT__node313.ldas-cit.ligo.caltech.edu h=
as been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm1__AT__node208.ldas-cit.ligo.caltech.edu h=
as been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm3__AT__node123.ldas-cit.ligo.caltech.edu h=
as been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm2__AT__node142.ldas-cit.ligo.caltech.edu h=
as been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm2__AT__node56.ldas-cit.ligo.caltech.edu ha=
s been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm3__AT__node38.ldas-cit.ligo.caltech.edu ha=
s been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm1__AT__node69.ldas-cit.ligo.caltech.edu ha=
s been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm4__AT__node279.ldas-cit.ligo.caltech.edu h=
as been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm3__AT__node289.ldas-cit.ligo.caltech.edu h=
as been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm2__AT__node22.ldas-cit.ligo.caltech.edu ha=
s been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm1__AT__node33.ldas-cit.ligo.caltech.edu ha=
s been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm2__AT__node14.ldas-cit.ligo.caltech.edu ha=
s been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm4__AT__node35.ldas-cit.ligo.caltech.edu ha=
s been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm1__AT__node56.ldas-cit.ligo.caltech.edu ha=
s been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) Resource vm2__AT__node11.ldas-cit.ligo.caltech.edu ha=
s been unused for 600 seconds, limit is 600, releasing
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.2.22:35248>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.2.22:35248>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.235:51007>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.235:51007>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.221:50329>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.221:50329>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.210:53085>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.210:53085>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.2.8:50504>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.2.8:50504>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.168:32988>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.168:32988>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.191:58130>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.191:58130>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.154:59724>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.154:59724>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.107:46547>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.107:46547>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.125:55646>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.125:55646>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.2.22:41634>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.2.22:41634>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.45:52477>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.45:52477>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.2.34:38433>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.2.34:38433>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.185:48688>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.185:48688>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.123:60226>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.123:60226>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.100:41027>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.100:41027>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.2.12:43338>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.2.12:43338>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.2.63:35719>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.2.63:35719>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.208:59081>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.208:59081>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.123:40479>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.123:40479>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.142:44787>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.142:44787>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.56:44634>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.56:44634>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.38:50400>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.38:50400>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.69:42266>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.69:42266>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.2.29:52651>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.2.29:52651>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.2.39:53608>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.2.39:53608>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.22:40822>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.22:40822>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.33:49378>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.33:49378>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.14:60777>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.14:60777>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.35:38093>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.35:38093>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.56:37435>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.56:37435>
11/11 15:56:15 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.1.11:53578>
11/11 15:56:15 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/11 15:56:15 (pid:31560) Got VACATE_SERVICE from <10.14.1.11:53578>
11/11 15:58:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 15:58:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 16:03:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 16:03:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 16:08:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 16:08:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 16:13:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 16:13:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 16:18:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 16:18:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 16:23:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 16:23:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 16:28:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 16:28:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 16:33:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 16:33:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 16:38:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 16:38:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 16:43:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 16:43:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 16:48:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 16:48:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 16:53:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 16:53:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 16:58:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 16:58:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 17:03:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 17:03:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 17:08:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 17:08:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 17:13:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 17:13:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 17:18:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 17:18:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 17:23:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 17:23:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 17:28:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 17:28:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 17:33:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 17:33:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 17:38:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 17:38:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 17:43:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 17:43:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 17:48:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 17:48:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 17:53:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 17:53:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 17:58:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 17:58:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 18:03:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 18:03:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 18:08:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 18:08:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 18:13:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 18:13:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 18:18:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 18:18:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 18:23:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 18:23:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 18:28:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 18:28:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 18:33:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 18:33:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 18:38:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 18:38:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 18:43:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 18:43:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 18:48:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 18:48:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 18:53:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 18:53:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 18:58:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 18:58:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 19:03:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 19:03:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 19:08:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 19:08:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 19:13:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 19:13:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 19:18:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 19:18:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 19:23:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 19:23:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 19:28:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 19:28:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 19:33:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 19:33:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 19:38:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 19:38:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 19:43:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 19:43:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 19:48:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 19:48:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 19:53:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 19:53:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 19:58:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 19:58:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 20:03:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 20:03:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 20:08:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 20:08:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 20:13:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 20:13:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 20:18:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 20:18:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 20:23:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 20:23:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 20:28:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 20:28:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 20:33:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 20:33:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 20:38:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 20:38:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 20:43:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 20:43:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 20:48:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 20:48:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 20:53:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 20:53:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 20:58:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 20:58:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 21:03:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 21:03:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 21:08:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 21:08:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 21:13:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 21:13:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 21:18:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 21:18:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 21:23:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 21:23:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 21:28:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 21:28:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 21:33:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 21:33:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 21:38:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 21:38:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 21:43:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 21:43:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 21:48:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 21:48:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 21:53:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 21:53:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 21:58:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 21:58:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 22:03:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 22:03:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 22:08:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 22:08:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 22:13:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 22:13:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 22:18:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 22:18:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 22:23:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 22:23:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 22:28:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 22:28:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 22:33:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 22:33:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 22:38:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 22:38:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 22:43:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 22:43:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 22:48:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 22:48:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 22:53:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 22:53:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 22:58:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 22:58:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 23:03:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 23:03:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 23:08:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 23:08:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 23:13:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 23:13:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 23:18:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 23:18:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 23:23:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 23:23:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 23:28:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 23:28:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 23:33:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 23:33:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 23:38:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 23:38:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 23:43:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 23:43:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 23:48:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 23:48:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 23:53:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 23:53:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/11 23:58:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/11 23:58:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 00:03:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 00:03:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 00:08:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 00:08:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 00:13:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 00:13:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 00:18:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 00:18:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 00:23:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 00:23:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 00:28:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 00:28:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 00:33:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 00:33:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 00:38:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 00:38:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 00:43:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 00:43:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 00:48:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 00:48:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 00:53:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 00:53:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 00:58:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 00:58:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 01:03:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 01:03:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 01:08:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 01:08:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 01:13:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 01:13:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 01:18:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 01:18:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 01:23:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 01:23:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 01:28:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 01:28:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 01:33:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 01:33:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 01:38:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 01:38:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 01:43:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 01:43:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 01:48:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 01:48:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 01:53:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 01:53:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 01:58:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 01:58:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 02:03:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 02:03:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 02:08:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 02:08:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 02:13:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 02:13:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 02:18:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 02:18:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 02:23:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 02:23:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 02:28:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 02:28:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 02:33:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 02:33:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 02:38:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 02:38:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 02:43:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 02:43:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 02:48:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 02:48:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 02:53:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 02:53:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 02:58:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 02:58:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 03:03:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 03:03:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 03:08:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 03:08:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 03:13:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 03:13:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 03:18:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 03:18:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 03:23:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 03:23:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 03:28:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 03:28:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 03:33:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 03:33:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 03:38:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 03:38:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 03:43:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 03:43:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 03:48:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 03:48:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 03:53:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 03:53:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 03:58:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 03:58:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 04:03:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 04:03:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 04:08:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 04:08:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 04:13:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 04:13:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 04:18:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 04:18:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 04:23:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 04:23:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 04:28:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 04:28:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 04:33:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 04:33:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 04:38:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 04:38:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 04:43:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 04:43:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 04:48:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 04:48:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 04:53:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 04:53:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 04:58:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 04:58:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 05:03:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 05:03:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 05:08:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 05:08:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 05:13:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 05:13:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 05:18:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 05:18:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 05:23:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 05:23:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 05:28:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 05:28:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 05:33:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 05:33:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 05:38:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 05:38:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 05:43:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 05:43:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 05:48:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 05:48:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 05:53:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 05:53:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 05:58:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 05:58:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 06:03:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 06:03:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 06:08:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 06:08:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 06:13:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 06:13:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 06:18:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 06:18:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 06:23:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 06:23:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 06:28:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 06:28:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 06:33:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 06:33:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 06:38:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 06:38:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 06:43:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 06:43:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 06:48:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 06:48:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 06:53:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 06:53:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 06:58:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 06:58:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 07:03:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 07:03:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 07:08:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 07:08:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 07:13:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 07:13:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 07:18:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 07:18:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 07:23:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 07:23:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 07:28:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 07:28:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 07:33:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 07:33:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 07:38:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 07:38:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 07:43:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 07:43:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 07:48:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 07:48:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 07:53:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 07:53:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 07:58:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 07:58:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 08:03:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 08:03:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 08:08:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 08:08:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 08:13:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 08:13:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 08:18:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 08:18:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 08:23:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 08:23:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 08:28:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 08:28:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 08:33:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 08:33:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 08:38:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 08:38:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 08:43:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 08:43:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 08:48:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 08:48:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 08:53:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 08:53:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 08:58:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 08:58:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 09:03:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 09:03:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 09:08:10 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 09:08:10 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 09:10:42 (pid:31560) Cleaning job queue...
11/12 09:13:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 09:13:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 09:18:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 09:18:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 09:23:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 09:23:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 09:28:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 09:28:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 09:33:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 09:33:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 09:38:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 09:38:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 09:43:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 09:43:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 09:48:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 09:48:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 09:53:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 09:53:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 09:58:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 09:58:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 10:03:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 10:03:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 10:08:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 10:08:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 10:13:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 10:13:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 10:18:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 10:18:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 10:23:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 10:23:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 10:28:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 10:28:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 10:33:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 10:33:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 10:38:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 10:38:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 10:43:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 10:43:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 10:48:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 10:48:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 10:53:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 10:53:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 10:58:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 10:58:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 11:03:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 11:03:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 11:08:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 11:08:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 11:13:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 11:13:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 11:18:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 11:18:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 11:23:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 11:23:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 11:28:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 11:28:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 11:33:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 11:33:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 11:38:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 11:38:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 11:43:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 11:43:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 11:48:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 11:48:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 11:53:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 11:53:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 11:58:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 11:58:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 12:03:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 12:03:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 12:08:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 12:08:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 12:13:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 12:13:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 12:18:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 12:18:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 12:23:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 12:23:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 12:28:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 12:28:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 12:33:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 12:33:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 12:38:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 12:38:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 12:43:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 12:43:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 12:48:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 12:48:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 12:53:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 12:53:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 12:58:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 12:58:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 13:03:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 13:03:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 13:08:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 13:08:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 13:13:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 13:13:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 13:18:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 13:18:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 13:23:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 13:23:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 13:28:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 13:28:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 13:33:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 13:33:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 13:38:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 13:38:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 13:43:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 13:43:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 13:48:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 13:48:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 13:53:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 13:53:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 13:58:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 13:58:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 14:03:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 14:03:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 14:08:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 14:08:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 14:13:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 14:13:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 14:18:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 14:18:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 14:23:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 14:23:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 14:28:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 14:28:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 14:33:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 14:33:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 14:38:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 14:38:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 14:43:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 14:43:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 14:48:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 14:48:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 14:53:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 14:53:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 14:58:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 14:58:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 15:03:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 15:03:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 15:08:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 15:08:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 15:13:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 15:13:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 15:18:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 15:18:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 15:23:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 15:23:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 15:28:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 15:28:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 15:33:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 15:33:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 15:38:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 15:38:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 15:43:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 15:43:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 15:48:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 15:48:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 15:53:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 15:53:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 15:58:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 15:58:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 16:03:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 16:03:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 16:08:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 16:08:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 16:13:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 16:13:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 16:18:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 16:18:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 16:23:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 16:23:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 16:28:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 16:28:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 16:33:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 16:33:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 16:38:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 16:38:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 16:43:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 16:43:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 16:48:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 16:48:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 16:53:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 16:53:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 16:58:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 16:58:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 17:03:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 17:03:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 17:08:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 17:08:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 17:13:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 17:13:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 17:18:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 17:18:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 17:23:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 17:23:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 17:28:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 17:28:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 17:33:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 17:33:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 17:38:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 17:38:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 17:43:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 17:43:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 17:48:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 17:48:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 17:53:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 17:53:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 17:58:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 17:58:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 18:03:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 18:03:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 18:08:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 18:08:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 18:13:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 18:13:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 18:18:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 18:18:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 18:23:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 18:23:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 18:28:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 18:28:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 18:33:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 18:33:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 18:38:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 18:38:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 18:43:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 18:43:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 18:48:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 18:48:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 18:53:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 18:53:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 18:58:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 18:58:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 19:03:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 19:03:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 19:08:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 19:08:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 19:13:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 19:13:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 19:18:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 19:18:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 19:23:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 19:23:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 19:28:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 19:28:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 19:33:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 19:33:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 19:38:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 19:38:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 19:43:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 19:43:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 19:48:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 19:48:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 19:53:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 19:53:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 19:58:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 19:58:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 20:03:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 20:03:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 20:08:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 20:08:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 20:13:11 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 20:13:11 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 20:14:58 (pid:31560) warning: setting UserUid to 5053, was 620 previo=
sly
11/12 20:14:58 (pid:31560) DaemonCore: Command received via UDP from host <=
10.14.0.18:59493>
11/12 20:14:58 (pid:31560) DaemonCore: received command 421 (RESCHEDULE), c=
alling handler (reschedule_negotiator)
11/12 20:14:58 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 20:14:58 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 20:14:58 (pid:31560) Sent ad to central manager for hans@ligo
11/12 20:14:58 (pid:31560) Sent ad to 1 collectors for hans@ligo
11/12 20:14:58 (pid:31560) Called reschedule_negotiator()
11/12 20:15:03 (pid:31560) Activity on stashed negotiator socket
11/12 20:15:03 (pid:31560) Negotiating for owner: hans@ligo
11/12 20:15:03 (pid:31560) Checking consistency running and runnable jobs
11/12 20:15:03 (pid:31560) Tables are consistent
11/12 20:15:03 (pid:31560) Out of jobs - 1 jobs matched, 0 jobs idle, flock=
level =3D 0
11/12 20:15:03 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 20:15:03 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 20:15:03 (pid:31560) Sent ad to central manager for hans@ligo
11/12 20:15:03 (pid:31560) Sent ad to 1 collectors for hans@ligo
11/12 20:15:05 (pid:31560) Starting add_shadow_birthdate(22651.0)
11/12 20:15:05 (pid:31560) Started shadow for job 22651.0 on "<10.14.2.13:3=
6542>", (shadow pid =3D 6190)
11/12 20:15:08 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 20:15:08 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 20:15:08 (pid:31560) Sent ad to central manager for hans@ligo
11/12 20:15:08 (pid:31560) Sent ad to 1 collectors for hans@ligo
11/12 20:15:08 (pid:31560) Shadow pid 6190 for job 22651.0 exited with stat=
us 100
11/12 20:15:08 (pid:31560) match (<10.14.2.13:36542>#1161969239#2256) out o=
f jobs (cluster id 22651); relinquishing
11/12 20:15:08 (pid:31560) Sent RELEASE_CLAIM to startd on <10.14.2.13:3654=
2>
11/12 20:15:08 (pid:31560) Match record (<10.14.2.13:36542>, 22651, -1) del=
eted
11/12 20:15:08 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.2.13:38944>
11/12 20:15:08 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/12 20:15:08 (pid:31560) Got VACATE_SERVICE from <10.14.2.13:38944>
11/12 20:16:09 (pid:31560) DaemonCore: Command received via UDP from host <=
10.14.0.18:59508>
11/12 20:16:09 (pid:31560) DaemonCore: received command 421 (RESCHEDULE), c=
alling handler (reschedule_negotiator)
11/12 20:16:09 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 20:16:09 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 20:16:09 (pid:31560) Sent ad to central manager for hans@ligo
11/12 20:16:09 (pid:31560) Sent ad to 1 collectors for hans@ligo
11/12 20:16:09 (pid:31560) Called reschedule_negotiator()
11/12 20:16:16 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.0.18:34602>
11/12 20:16:16 (pid:31560) DaemonCore: received command 478 (ACT_ON_JOBS), =
calling handler (actOnJobs)
11/12 20:16:16 (pid:31560) Attempting to chown '/usr1/condor/spool/cluster2=
2652.proc0.subproc0', but it doesn't appear to exist.
11/12 20:16:16 (pid:31560) Error: Unable to chown '/usr1/condor/spool/clust=
er22652.proc0.subproc0' from 5053 to 4135.4135
11/12 20:16:16 (pid:31560) (22652.0) Failed to chown /usr1/condor/spool/clu=
ster22652.proc0.subproc0 from 5053 to 4135.4135. User may run into permiss=
ions problems when fetching sandbox.
11/12 20:16:28 (pid:31560) DaemonCore: Command received via UDP from host <=
10.14.0.18:59512>
11/12 20:16:28 (pid:31560) DaemonCore: received command 421 (RESCHEDULE), c=
alling handler (reschedule_negotiator)
11/12 20:16:28 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 20:16:28 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 20:16:28 (pid:31560) Sent ad to central manager for hans@ligo
11/12 20:16:28 (pid:31560) Sent ad to 1 collectors for hans@ligo
11/12 20:16:28 (pid:31560) Called reschedule_negotiator()
11/12 20:16:34 (pid:31560) Activity on stashed negotiator socket
11/12 20:16:34 (pid:31560) Negotiating for owner: hans@ligo
11/12 20:16:34 (pid:31560) Checking consistency running and runnable jobs
11/12 20:16:34 (pid:31560) Tables are consistent
11/12 20:16:34 (pid:31560) Out of jobs - 1 jobs matched, 0 jobs idle, flock=
level =3D 0
11/12 20:16:34 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 20:16:34 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 20:16:34 (pid:31560) Sent ad to central manager for hans@ligo
11/12 20:16:34 (pid:31560) Sent ad to 1 collectors for hans@ligo
11/12 20:16:36 (pid:31560) Starting add_shadow_birthdate(22653.0)
11/12 20:16:36 (pid:31560) Started shadow for job 22653.0 on "<10.14.2.13:3=
6542>", (shadow pid =3D 6223)
11/12 20:16:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 20:16:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 20:16:39 (pid:31560) Sent ad to central manager for hans@ligo
11/12 20:16:39 (pid:31560) Sent ad to 1 collectors for hans@ligo
11/12 20:17:04 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.0.18:51498>
11/12 20:17:04 (pid:31560) DaemonCore: received command 478 (ACT_ON_JOBS), =
calling handler (actOnJobs)
11/12 20:17:05 (pid:31560) Shadow pid 6223 for job 22653.0 exited with stat=
us 102
11/12 20:17:05 (pid:31560) match (<10.14.2.13:36542>#1161969239#2258) out o=
f jobs (cluster id 22653); relinquishing
11/12 20:17:05 (pid:31560) Sent RELEASE_CLAIM to startd on <10.14.2.13:3654=
2>
11/12 20:17:05 (pid:31560) Match record (<10.14.2.13:36542>, 22653, -1) del=
eted
11/12 20:17:05 (pid:31560) DaemonCore: Command received via TCP from host <=
10.14.2.13:52882>
11/12 20:17:05 (pid:31560) DaemonCore: received command 443 (VACATE_SERVICE=
), calling handler (vacate_service)
11/12 20:17:05 (pid:31560) Got VACATE_SERVICE from <10.14.2.13:52882>
11/12 20:21:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 20:21:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 20:21:39 (pid:31560) Sent owner (0 jobs) ad to 1 collectors
11/12 20:26:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 20:26:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 20:31:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 20:31:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 20:36:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 20:36:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 20:41:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 20:41:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 20:46:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 20:46:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 20:51:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 20:51:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 20:56:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 20:56:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 21:01:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 21:01:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 21:06:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 21:06:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 21:11:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 21:11:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 21:16:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 21:16:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 21:21:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 21:21:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 21:26:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 21:26:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 21:31:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 21:31:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 21:36:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 21:36:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 21:41:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 21:41:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 21:46:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 21:46:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 21:51:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 21:51:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 21:56:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 21:56:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 22:01:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 22:01:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 22:06:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 22:06:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 22:11:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 22:11:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 22:16:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 22:16:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 22:21:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 22:21:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 22:26:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 22:26:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 22:31:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 22:31:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 22:36:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 22:36:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 22:41:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 22:41:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 22:46:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 22:46:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 22:51:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 22:51:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 22:56:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 22:56:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 23:01:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 23:01:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 23:06:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 23:06:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 23:11:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 23:11:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 23:16:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 23:16:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 23:21:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 23:21:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 23:26:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 23:26:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 23:31:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 23:31:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 23:36:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 23:36:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 23:41:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 23:41:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 23:46:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 23:46:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 23:51:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 23:51:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/12 23:56:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/12 23:56:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 00:01:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 00:01:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 00:06:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 00:06:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 00:11:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 00:11:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 00:16:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 00:16:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 00:21:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 00:21:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 00:26:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 00:26:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 00:31:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 00:31:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 00:36:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 00:36:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 00:41:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 00:41:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 00:46:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 00:46:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 00:51:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 00:51:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 00:56:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 00:56:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 01:01:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 01:01:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 01:06:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 01:06:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 01:11:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 01:11:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 01:16:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 01:16:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 01:21:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 01:21:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 01:26:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 01:26:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 01:31:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 01:31:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 01:36:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 01:36:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 01:41:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 01:41:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 01:46:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 01:46:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 01:51:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 01:51:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 01:56:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 01:56:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 02:01:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 02:01:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 02:06:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 02:06:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 02:11:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 02:11:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 02:16:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 02:16:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 02:21:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 02:21:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 02:26:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 02:26:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 02:31:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 02:31:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 02:36:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 02:36:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 02:41:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 02:41:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 02:46:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 02:46:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 02:51:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 02:51:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 02:56:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 02:56:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 03:01:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 03:01:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 03:06:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 03:06:39 (pid:31560) Sent ad to 1 collectors for dbrown@ligo
11/13 03:11:39 (pid:31560) Sent ad to central manager for dbrown@ligo
11/13 03:11:39 (pid:31560)