Grid Exerciser

summarize_condor_log

summarize_condor_log generates reports based on a grid exerciser run started by site_submit.

Execution

summarize_condor_log generates a report on a running exercise. summarize_condor_log takes the path to one or more directories created by site_submit, merging the results if there are more than one directory. By merging directories multiple runs can be treated as a single run.

By default summarize_condor_log will generate results for the last 24 hours. Results are written to standard output. --start-time and --end-time can be used to control the specific time range. --email can be used to email a copy of the report somewhere.

summarize_condor_log does not yet support automatic generation, for ongoing grid exerciser runs you will want to use a system like cron to automatically run summarize_condor_log.

Optional arguments

--start-time 'Date/Time'

First time to consider jobs in. Jobs the finish prior to this time will be ignored in the report. The format is parsed by Perl's Date::Manip and thus can be almost anything human readable. Defaults to the current time.

--end-time 'Date/Time'

Last time to consider jobs in. Jobs the begin after to this time will be ignored in the report. The format is parsed by Perl's Date::Manip and thus can be almost anything human readable. Defaults to 24 hours ago.

--email emailaddress@example.com

A copy of the report will be sent to this address.

--write-txt /path/to/file

A copy of the report will be written to this file.

--silent

Do not write a copy of the report to standard output.

Reading the report

Information on the generated report has been moved to this page.

Implementation

summarize_condor_log searches for all job logs it can find, scans then, and sorts the results by site. It also uses resourcename.info file so that it can report how many jobs were sent to each site.

summarize_condor_log does not automatically run. At the moment I just run it from cron. Since the directory I need to run it on changes whenever I re-run site_submit, I use symlinks (latest and previous) as the directories to process. My crontab entry looks like:

0 0 * * * /s/std/bin/runauth \
  /p/condor/workspaces/adesmet/tools/grid-exerciser/summarize_condor_log \
  /scratch/adesmet/gridexerciser/run/latest \
  /scratch/adesmet/gridexerciser/run/previous \
  --silent \
  --email foo@example.com \
  --write-txt /afs/cs.wisc.edu/u/a/d/adesmet/public/html/ge/results.txt

summarize_condor_log contains a basic Condor job log parser. Scanning all of the logs takes a fair amount of time. Scanning about 50MB of logs takes about 15 minutes. I don't yet know where the time is spent.