LIGO Support Ticket 19218

Ticket Information
  Number:      admin 19218
  User:        ram@gravity.phys.uwm.edu
  Email:       anderson__AT__ligo.caltech.edu,carsten.aulbert__AT__aei.mpg.de
  Status:      resolved
  Assigned To: psilord

Domainkey-Signature: a=rsa-sha1; c=nofws;        d=gmail.com; s=gamma;    
    h=mime-version:sender:from:date:x-google-sender-auth:message-id        
 :subject:to:cc:content-type:content-transfer-encoding;       
 b=i6GUP+WYdtyTSYwQ3/XdRdUZ1dMPXx6upV2u+texRi7V6x3UJitIo0QszSLkW+StFb      
   ANWLcbJ2c3Vkt7iSffKhmC5ahcAGl3VKwztyvOrZV2oaJyIbXbHCNuZksbtYjTYKaJGE    
     G6z55bGM3wVlhovJJKHj3wrwsMeMpWc8oycEU=
From: Adam Mercer <ram__AT__gravity.phys.uwm.edu>
Date: Thu, 16 Apr 2009 11:15:05 -0500
Subject: LIGO condor_compile causing strptime() to fail
To: condor-admin__AT__cs.wisc.edu
CC: =?UTF-8?B?U3R1YXJ077u/77u/IEFuZGVyc29u77u/77u/?=
 <anderson__AT__ligo.caltech.edu>,
 =?UTF-8?B?Q2Fyc3Rlbu+7v++7vyBBdWxiZXJ077u/77u/?=
 <carsten.aulbert__AT__aei.mpg.de>

Hi

In tracking down an issue where some of our condor compiled code was
crashing, due to the inability to parse a date string using
strptime(). What makes this issue strange is that the condor compiled
version of the code fails whereas the standard compiled code succeeds.
I have managed to write a small example program, that exhibits this
behaviour:

ram@h2:~$ cat parse_date.c
#include <stdio.h>
#include <stdlib.h>
#include <time.h>

int main(void)
{
  char *date = "2009-04-14 10:53:05 -0500";
  struct tm utc;

  if (!strptime(date, "%F %T %z", &utc))
  {
    fprintf(stderr, "cannot parse \"%s\"\n", date);
    exit(1);
  }
  else
  {
    fprintf(stdout, "date parsed\n");
  }

  return 0;
}
ram@h2:~$

If I compile and run the code it correctly parses the date string:

ram@h2:~$ gcc parse_date.c -o parse_date
ram@h2:~$ ./parse_date
date parsed
ram@h2:~$

whereas if I condor compile, the the code fails:

ram@h2:~$ condor_compile gcc parse_date.c -o parse_date_condor
LINKING FOR CONDOR : /usr/bin/ld -L/opt/condor/lib -Bstatic
--eh-frame-hdr -m elf_x86_64 -dynamic-linker
/lib64/ld-linux-x86-64.so.2 -o parse_date_condor
/opt/condor/lib/condor_rt0.o
/usr/lib/gcc/x86_64-linux-gnu/3.4.6/../../../../lib/crti.o
/usr/lib/gcc/x86_64-linux-gnu/3.4.6/crtbeginT.o -L/opt/condor/lib
-L/usr/lib/gcc/x86_64-linux-gnu/3.4.6
-L/usr/lib/gcc/x86_64-linux-gnu/3.4.6
-L/usr/lib/gcc/x86_64-linux-gnu/3.4.6/../../../../lib
-L/usr/lib/gcc/x86_64-linux-gnu/3.4.6/../../.. -L/lib/../lib
-L/usr/lib/../lib /tmp/ccOVtObr.o /opt/condor/lib/libcondorsyscall.a
/opt/condor/lib/libcondor_z.a /opt/condor/lib/libcomp_libstdc++.a
/opt/condor/lib/libcomp_libgcc.a /opt/condor/lib/libcomp_libgcc_eh.a
--as-needed --no-as-needed -lcondor_c -lcondor_nss_files
-lcondor_nss_dns -lcondor_resolv -lcondor_c -lcondor_nss_files
-lcondor_nss_dns -lcondor_resolv -lcondor_c
/opt/condor/lib/libcomp_libgcc.a /opt/condor/lib/libcomp_libgcc_eh.a
--as-needed --no-as-needed
/usr/lib/gcc/x86_64-linux-gnu/3.4.6/crtend.o
/usr/lib/gcc/x86_64-linux-gnu/3.4.6/../../../../lib/crtn.o
/opt/condor/lib/libcondorsyscall.a(condor_file_agent.o): In function
`CondorFileAgent::open(char const*, int, int)':
/local/build_condor/condor-7.2.0/src/condor_ckpt/condor_file_agent.cpp:106:
warning:
/opt/condor/lib/libcondorsyscall.a(switches.o): In function `__gets_chk':
/local/build_condor/condor-7.2.0/src/condor_syscall_lib/switches.remap-LINUX.h:430:
warning:
ram@h2:~$ ./parse_date_condor
Condor: Notice: Will checkpoint to ./parse_date_condor.ckpt
Condor: Notice: Remote system calls disabled.
cannot parse "2009-04-14 10:53:05 -0500"
ram@h2:~$

This is running on a 64bit Debian Etch (4.0) system running the
following condor version:

ram@h2:~$ condor_version
$CondorVersion: 7.2.0 Dec 25 2008 BuildID: X86_64-LINUX_DEBIAN40_ATLAS $
$CondorPlatform: X86_64-LINUX_DEBIAN40 $
ram@h2:~$

I have also reproduced this same problem on a 64bit Fedora Core 4
system running the following condor version:

[ram__AT__ldas-pcdev1.ligo ~]$ condor_version
$CondorVersion: 7.2.2 Apr  9 2009 BuildID: 145189 $
$CondorPlatform: X86_64-LINUX_RHEL3 $
[ram__AT__ldas-pcdev1.ligo ~]$

However I am not able to reproduce this on a 64bit CentOS 5.2 system
running the following condor version:

[ram@trout ~]$ condor_version
$CondorVersion: 7.2.0 Dec 19 2008 BuildID: 121001 $
$CondorPlatform: X86_64-LINUX_RHEL5 $
[ram@trout ~]$

In this case both the standard compiled and condor compiled version
are able to parse the date string:

[ram@trout ~]$ ./parse_date
date parsed
[ram@trout ~]$ ./parse_date_condor
Condor: Notice: Will checkpoint to ./parse_date_condor.ckpt
Condor: Notice: Remote system calls disabled.
date parsed
[ram@trout ~]$

Please let me know if you need any further details.

Cheers

Adam

===========================================================================
Date of creation: Thu Apr 16 11:16:40 2009 (1239898603)

Subject: Actions

Assigned to psilord by psilord
===========================================================================
Date of actions: Thu Apr 16 14:59:37 2009 (1239911977)

Domainkey-Signature: a=rsa-sha1; c=nofws;        d=gmail.com; s=gamma;    
    h=mime-version:sender:in-reply-to:references:from:date        
 :x-google-sender-auth:message-id:subject:to:content-type        
 :content-transfer-encoding;        b=Rj2drphBEPp+t4La9YHQr0fuM9S6xKfjNtdqysLel9cWIf4VunCggQD+kZuNdb0CBy
         EMnHtMwG1eh7e5Kp/+oHjn0zcaonmoIPjLCTuew3XGVQSer6gh0xb4CgJHuPzIuST/UW
         m7HZGZ+vkaWH6p9QfsZRiETlo1fY41Rz+MqNE=
From: Adam Mercer <ram__AT__gravity.phys.uwm.edu>
Date: Wed, 22 Apr 2009 15:51:24 -0500
Subject: Re: [condor-admin #19218] LIGO condor_compile causing strptime()
 to 	fail
To: condor-admin__AT__cs.wisc.edu

On Thu, Apr 16, 2009 at 11:16,  <condor-admin__AT__cs.wisc.edu> wrote:

> In order to help us track the progress of your request, we ask that you
> include the string:
> =A0"[condor-admin #19218] LIGO condor_compile causing strptime() to fail"
> in the subject line of any further mail about this particular request.

I have done more investigation into this and it appears that the
problem is caused by the GNU extensions to strptime() I was using, if
I remove the use of the extensions then the condor compiled code
behaves the same as the regulary compiled version.

This unfortunately leads to the inability to parse date strings with
time zones, whilst not that major a problem it would be really helpful
if the condor compiled code had access to these GNU extensions.

Cheers

Adam

===========================================================================
Date mail was appended: Wed Apr 22 15:51:51 2009 (1240433511)

Date: Fri, 24 Apr 2009 11:35:39 -0500
From: Peter Keller <psilord__AT__cs.wisc.edu>
To: condor-admin response tracking system <condor-admin__AT__cs.wisc.edu>
Subject: Re: [condor-admin #19218] LIGO condor_compile causing strptime()
 to fail

On Wed, Apr 22, 2009 at 03:51:51PM -0500, condor-admin response tracking system wrote:
> On Thu, Apr 16, 2009 at 11:16,  <condor-admin__AT__cs.wisc.edu> wrote:
> 
> I have done more investigation into this and it appears that the
> problem is caused by the GNU extensions to strptime() I was using, if
> I remove the use of the extensions then the condor compiled code
> behaves the same as the regulary compiled version.

Can you supply a very small piece of code that has a compile time flag to
turn on and off the gnu extension for this function? Then I'll be able
to figure out how I can go about dealing with this issue.

Thank you.

Condor Admin

===========================================================================
Date mail was appended: Fri Apr 24 11:35:44 2009 (1240590945)

Domainkey-Signature: a=rsa-sha1; c=nofws;        d=gmail.com; s=gamma;    
    h=mime-version:sender:in-reply-to:references:from:date        
 :x-google-sender-auth:message-id:subject:to:cc:content-type;       
 b=RJYKmuGlC6nqbPKQsBXfFsXovRSmjGuhcTUf2Pr9NOGN21H67EX4xO3Ko7tOFV+tmM      
   RAH3SMUEKhUayGebKbZou0Q/bbfj9i5Cix7rlR9WmAtoPd1GL4cs0luP7xcj3UCbxV3/    
     +6EKVfxOaGy3XsaZrTVaNqTXJk4xUwH1MbrqY=
From: Adam Mercer <ram__AT__gravity.phys.uwm.edu>
Date: Fri, 24 Apr 2009 12:08:10 -0500
Subject: Re: [condor-admin #19218] LIGO condor_compile causing strptime()
 to 	fail
To: condor-admin__AT__cs.wisc.edu
CC: anderson__AT__ligo.caltech.edu, carsten.aulbert__AT__aei.mpg.de

--0016364272b5daa7130468500b5c

On Fri, Apr 24, 2009 at 11:35, condor-admin response tracking system
<condor-admin__AT__cs.wisc.edu> wrote:

>> I have done more investigation into this and it appears that the
>> problem is caused by the GNU extensions to strptime() I was using, if
>> I remove the use of the extensions then the condor compiled code
>> behaves the same as the regulary compiled version.
>
> Can you supply a very small piece of code that has a compile time flag to
> turn on and off the gnu extension for this function? Then I'll be able
> to figure out how I can go about dealing with this issue.

Hows the attached? This results in the following behaviour:

[ram__AT__ldas-pcdev1.ligo condor]$ ./parse_date
date parsed
[ram__AT__ldas-pcdev1.ligo condor]$ ./parse_date_GNUEXT
date parsed
[ram__AT__ldas-pcdev1.ligo condor]$ ./parse_date_condor
Condor: Notice: Will checkpoint to ./parse_date_condor.ckpt
Condor: Notice: Remote system calls disabled.
date parsed
[ram__AT__ldas-pcdev1.ligo condor]$ ./parse_date_condor_GNUEXT
Condor: Notice: Will checkpoint to ./parse_date_condor_GNUEXT.ckpt
Condor: Notice: Remote system calls disabled.
cannot parse "2009-04-14 10:53:05 +0000"
[ram__AT__ldas-pcdev1.ligo condor]$

on 64bit  FC4 with condor:

[ram__AT__ldas-pcdev1.ligo condor]$ condor_version
$CondorVersion: 7.2.2 Apr  9 2009 BuildID: 145189 $
$CondorPlatform: X86_64-LINUX_RHEL3 $
[ram__AT__ldas-pcdev1.ligo condor]$

Cheers

Adam

--0016364272b5daa7130468500b5c
X-Attachment-Id: f_ftx4xa3e0

I2luY2x1ZGUgPHN0ZGlvLmg+CiNpbmNsdWRlIDxzdGRsaWIuaD4KI2luY2x1
ZGUgPHRpbWUuaD4KCmludCBtYWluKHZvaWQpCnsKCWNoYXIgKmRhdGUgPSAi
MjAwOS0wNC0xNCAxMDo1MzowNSArMDAwMCI7CglzdHJ1Y3QgdG0gdXRjOwoj
aWZuZGVmIEdOVUVYVAogIGlmICghc3RycHRpbWUoZGF0ZSwgIiVZLSVtLSVk
ICVUICswMDAwIiwgJnV0YykpCiNlbHNlCglpZiAoIXN0cnB0aW1lKGRhdGUs
ICIlRiAlVCAleiIsICZ1dGMpKQojZW5kaWYgLyogR05VRVhUICovCiAgewog
ICAgZnByaW50ZihzdGRlcnIsICJjYW5ub3QgcGFyc2UgXCIlc1wiXG4iLCBk
YXRlKTsKICAgIGV4aXQoMSk7CiAgfQogIGVsc2UKICB7CiAgICBmcHJpbnRm
KHN0ZG91dCwgImRhdGUgcGFyc2VkXG4iKTsKICB9CgogIHJldHVybiAwOwp9
Cg==

--0016364272b5daa7130468500b5c
X-Attachment-Id: f_ftx4xfk31

YWxsOiBwYXJzZV9kYXRlX0dOVUVYVCBwYXJzZV9kYXRlIHBhcnNlX2RhdGVf
Y29uZG9yX0dOVUVYVCBwYXJzZV9kYXRlX2NvbmRvcgoKcGFyc2VfZGF0ZV9H
TlVFWFQ6IHBhcnNlX2RhdGUuYwoJZ2NjIC1ER05VRVhUIHBhcnNlX2RhdGUu
YyAtbyBwYXJzZV9kYXRlX0dOVUVYVAoKcGFyc2VfZGF0ZTogcGFyc2VfZGF0
ZS5jCglnY2MgcGFyc2VfZGF0ZS5jIC1vIHBhcnNlX2RhdGUKCQpwYXJzZV9k
YXRlX2NvbmRvcl9HTlVFWFQ6IHBhcnNlX2RhdGUuYwoJY29uZG9yX2NvbXBp
bGUgZ2NjIC1ER05VRVhUIHBhcnNlX2RhdGUuYyAtbyBwYXJzZV9kYXRlX2Nv
bmRvcl9HTlVFWFQKCnBhcnNlX2RhdGVfY29uZG9yOiBwYXJzZV9kYXRlLmMK
CWNvbmRvcl9jb21waWxlIGdjYyBwYXJzZV9kYXRlLmMgLW8gcGFyc2VfZGF0
ZV9jb25kb3IKCi5QSE9OWTogY2xlYW4KY2xlYW46Cgktcm0gLXJmIHBhcnNl
X2RhdGVfR05VRVhUCgktcm0gLXJmIHBhcnNlX2RhdGUKCS1ybSAtcmYgcGFy
c2VfZGF0ZV9jb25kb3JfR05VRVhUCgktcm0gLXJmIHBhcnNlX2RhdGVfY29u
ZG9yCg==

--0016364272b5daa7130468500b5c--

===========================================================================
Date mail was appended: Fri Apr 24 12:08:41 2009 (1240592921)

Date: Wed, 30 Sep 2009 15:47:35 -0500
From: Peter Keller <psilord__AT__cs.wisc.edu>
To: condor-admin response tracking system <condor-admin__AT__cs.wisc.edu>
Subject: Re: [condor-admin #19218] LIGO condor_compile causing strptime()
 to fail

Hello,

> On Fri, Apr 24, 2009 at 11:35, condor-admin response tracking system
> <condor-admin__AT__cs.wisc.edu> wrote:
> >> I have done more investigation into this and it appears that the
> >> problem is caused by the GNU extensions to strptime() I was using, if
> >> I remove the use of the extensions then the condor compiled code
> >> behaves the same as the regulary compiled version.

Ok, I figured it out.

The bug is that the %z format option, which is a GNU Extension to the
POSIX interface to strptime() in order to support strftime(), isn't
implemented in the revision of glibc (2.3.2, actually 2.3.2.27.9.7) used
in RHEL3. I looked at the glibc code itself, and %z isn't in the cases
of the switch statement.  I have in fact reproduced this problem outside
of condor_compile all together, removing Condor totally from the equation.

Since the RHEL3 full port of Condor uses that exact revision of glibc,
any program condor_compiled on a RHEL3 port of Condor using strptime()
and the %z format option is doomed to fail.

It appears in my testing that the RHEL5 full port of Condor doesn't
suffer from this problem and the glibc version of strptime() on that
machine implements %z.

My advice is to continue to use the POSIX compliant formatting specifiers
for the most portable code. If you are unable, then use a RHEL5 full
port of Condor.

Thank you.

Condor Admin

===========================================================================
Date mail was appended: Wed Sep 30 15:47:41 2009 (1254343661)

Subject: Actions

Ticket resolved by psilord
===========================================================================
Date of actions: Wed Sep 30 15:49:23 2009 (1254343763)