ClassAds Programming Tutorial (C++)

Edited by Alain Roy (Condor Team), January 2005
Originally by Rajesh Raman (Condor Team), June 2000
This document is a tutorial for programming the C++ ClassAds library developed for the Condor system. Although classads were developed for Condor, they are applicable in many general contexts. The classad libraries are available as standalone packages independent of Condor. For a general introduction to the concept of classads, please read this paper .

This document is an informal tutorial. For an in-depth description of the ClassAd language, see the reference manual. If there is a discrepancy between this tutorial and the reference manual, then the reference manual is correct. If there is a discrepancy between the code and the reference manual, the reference manual is correct and there is a bug in the code.

For detailed documentation on the C++ ClassAd API, see the API documentation.

Table of Contents

  1. Informal Language Description
    1. Syntax and Evaluation Semantics
      1. Literals
      2. ClassAds
      3. Attribute References
      4. Operators
      5. Lists
      6. Function Calls
    2. Examples
  1. The C++ ClassAd API
    1. Preliminary
    2. Creating ClassAds
    3. Creating Expressions
    4. Inserting Expressions into ClassAds
    5. Removing Expressions from ClassAds
    6. Looking up Expressions in ClassAds
    7. Evaluating Expressions
    8. Testing Matches
    9. Displaying and Transporting Expressions and Values
    10. XML ClassAds

1. Informal Language Description

A classad is a structure that contains a set of named expressions. Think of it as an attribute-value list that does not have a scheme constraining the list of attributes. The expressions are very similar to their counterparts in C, but are more general:

1.1. Syntax and Evaluation Semantics

1.1.1. Literals

Literals are atomic expressions which represent single scalar values.
Integers
7, 0x1a, 0666, -33
Reals
3.14, -2.718, 1e+5, 1M, 2.3G.
(Note: The suffixes B, K, M, G and T indicate bytes, kilobytes, megabytes, gigabytes and terabytes respectively. The numeric prefix is converted to a real and multiplied by the appropriate power of 2.)
Strings
"aaa", "BBB", "a\t\"" (escape characters are supported; see K&R p.38)
Booleans
false, true
Absolute Time
absTime("2005-01-13T08:17:00-06:00")
Relative Time
relTime("-5:00")
Undefined
undefined (single value)
Error
error (single value)

1.1.2 ClassAds

ClassAds have the following syntax: 
  [ id0 = expr0 ; id1 = expr1 ; ... ; idn = exprn ]
where each id is a valid identifier and each expr is an expression. Identifiers begin with an alphabetic character or an underscore, and are followed with any number of alphabetic, numeric, or underscore characters. Identifiers are case-insensitive, but case-preserving. That is, if you refer to "Rank" or "rANK", it refers to the same attribute, but it will print with the same case that it was created with.

Since classads are themselves expressions, classads can be hierarchically nested, as illustrated in the following example. 

  [ a = 7 ; b = "foo" ; c = [ a = -1 ; b = "bar" ] ]
The outermost classad is referred to the root scope. Every other classad is contained in its parent scope, which is the unique classad that immediately contains the classad in question.

Classads are self-evaluating structures: a classad "expression" evaluates to a classad "value."

1.1.3. Attribute References

Attribute references are analogous to variable references in C++ expressions, but are more sophisticated. Attribute references have three forms:
  1. The first variant attr evaluates to the value of the bound expression in the attribute named attr. If no such attribute exists in the current scope, succeeding parent scopes are searched. If no attribute is found, the result of the evaluation is the undefined value.
  2. The second variant .attr is similar to the first variant, but looks up only the root scope. If the named attribute does not exist in the root scope, the result is the undefined value.
  3. The third variant expr.attr first evaluates the expression expr, which must evaluate to a classad value. The search for the expression begins in the scope of the evaluated classad value, and then proceeds as in the first variant. If the expression itself evaluated to undefined, the result is undefined. Otherwise, if the expression neither evaluated to a classad or undefined, the result is error.
The following references are special built-ins:
  1. The attribute self evaluates to the closest classad that encloses the attribute reference.
  2. The reference parent evaluates to the parent scope of the classad containing the reference. (This evaluates to undefined for the root scope.)
  3. The reference root evaluates to the root scope of the evaluation.

1.1.4. Operators

The classad language provides almost all the operators of C: the only operators not supported are assignment and pointer operators. In general, operators are strict with respect to undefined and error, so if any operand is undefined (error), the result is also undefined (error).

The following important points must be noted:

  1. The operators is and isnt are like == and !=, but they are not strict. I.e., they test the "is identical to" predicate, and always return either true or false.
  2. String comparison is case-sensitive only when comparing with the is and isnt operators, and case-insensitive in all other contexts.
  3. The ternary conditional operator ?: is strict only in the selector expression.
  4. The logical operators || (OR) and && (AND) are non-strict. Their truth tables are supplied below.
    & & (AND) F T U E
    F F F F F
    T F T U E
    U F U U E
    E E E E E
    || (OR) F T U E
    F F T U E
    T T T T T
    U U T U E
    E E E E E
    Operations with values not in the above table evaluate to error.

1.1.5. Lists

The classad language allows lists of expressions to be constructed. The syntax for list specification is: 
 { e1, e2 ... , en }
where each 'e' is an expression. Components of lists may be accessed via the subscript operator as in:
{ 10, "foo", -3.14 }[0]
which evaluates to the integer value 10.

1.1.6. Function Calls

The classad language provides a rich set of built-in functions. User-defined functions may not be defined. However, additional functions may easily be added if access to the library source code is available.

The syntax of a function call is 

name( arg0, arg1, ... , argn )
As with operators, most functions are strict with respect to undefined and error on all arguments. However, some functions are non-strict. The name of the function is case-insensitive. The supported set of functions are:
Type predicates (Non-Strict)
isUndefined(V)True if and only if V is the undefined value.
isError(V)True if and only if V is the error value.
isString(V)True if and only if V is a string value.
isList(V)True if and only if V is a list value.
isClassad(V)True if and only if V is a classad value.
isBoolean(V)True if and only if V is a boolean value.
isAbsTime(V)True if and only if V is an absolute time value.
isRelTime(V)True if and only if V is a relative time value.
List Membership
member(V,L)True if and only if scalar value V is a member of the list L
identicalMember(V,L)Like Member, but uses is for comparison instead of ==. Not strict on first argument.
Time
time()Returns the current Coordinated Universal Time, in seconds since midnight January 1, 1970.
currentTime()Get current time as an absolute time
timeZoneOffset()Get time zone offset as a relative time
dayTime()Get current time as relative time since midnight.
splitTime(T)Creates a ClassAd with each component of the time (absolute or relative) as an element of the ClassAd
formatTime(T, S)Formats an absolute time according the the strftime-style format. See the reference manual for details.
String Functions
strcat(V1, ... , Vn)Concatenates string representations of values V1 through Vn
toUpper(S)Converts the string S to uppercase
toLower(S)Converts the string S to lowercase
substr(S,offset [,len])Returns substring of S. Negative offsets and lengths count from the end of the string.
regexp(P,S) Checks if S matches pattern P (both args must be strings). For details on the options other than "i" for case-insensitive matching, see the reference manual.
Type Conversion Functions
int(V) Converts V to an integer. Time values are converted to number of seconds, strings are parsed, bools are mapped to 0 or 1. Other values result in error.
real(V)Similar to int(V), but to a real value.
string(V)Converts V to its string representation
bool(V)Converts V to a boolean value. Empty strings, and zero values converted to false; non-empty strings and non-zero values converted to true.
absTime(V)Converts V to an absolute time. Numeric values treated as seconds past UNIX epoch, strings parsed as necessary. See the reference manual for details.
relTime(V)Converts V to an relative time. Numeric values treated as number of seconds, strings parsed as necessary. See the reference manual for details.
Mathematical Functions
floor(N)Floor of numeric value N
ceiling(N)Ceiling of numeric value N
round(N)Rounded value of numeric value N
random(N)>If N is an integer, the result is an integer random number R in the range 0 <= R < N. If N is a real number, the result is a real random number in the same range.If N is anything else, the result is an error.

1.2. Examples

Consider the following classad 
    [
       a = 17;
       b = "foo";
       c = { "x", "y", 3*a };
       d = [
              a = 23;
              b = 15;
              c = [];
              d = .a;
           ];
       e = '00:15:00';
    ]
We now provide a few example expressions, and their resulting values when evaluated in the context of the above classad.
Expression Result
a 17
strcat(b, "bar", a)"foobar17"
x undefined
x+10 undefined
x || true true
d.a 23
d.b 15
d.self [ a = 23 ; b = 15 ; c = [] ; d = .a ]
d.c []
d.c.parent [ a = 23 ; b = 15 ; c = [] ; d = .a ]
d.c.a 23
d.parent.c { "x", "y", 3*a }
d.parent.c[2] 51
d.d 17
e*4 '01:00:00'

2. The C++ ClassAd API

The primary activities associated with ``programming'' classads are constructing, evaluating and displaying expressions.

2.1. Preliminary

2.2 Creating ClassAds

ClassAds are really a special case of expressions, so the next section will explain how to create ClassAds. However, since ClassAds are often the primary concern of programmers that use the ClassAd library, we will explain them separately here.

2.2.1 Parsing ClassAds

The most common method to create ClassAds is with the ClassAd parser. The parser can take text input (a string, a file, or a stream) and convert it to a ClassAd. The parser is encapsulated in the ClassAdParser class, which can be found in source.h. Normally you won't need to include source.h directly, since it is included by classad_distribution.h.

The ClassAdParser has two prototypical ways of creating a ClassAd, though there are several variants with each method. For this tutorial, we'll create ClassAds from strings: the variants merely let you construct the ClassAds from different sources.

Method One: Return a string

string         classad_string = "[a = 1; b = \"Cardini\"]";
ClassAd        *classad;
ClassAdParser  parser;

classad = parser.ParseClassAd(classad_string, true);
In this example, true is passed to ParseClassAd to indicate that no extra text follows the ClassAd. If it does, it will return an error. If you legitimately may have extra text after a ClassAd, pass false instead. If the parsing fails, NULL will be returned. You own the ClassAd, so it is up to you to delete it when you are finished with it.

Method Two: Fill in a ClassAd

string         classad_string = "[a = 1; b = \"Cardini\"]";
ClassAd        classad;
ClassAdParser  parser;
bool           success.

success = parser.ParseClassAd(classad_string, classad, true);
In this example, the classad is filled in directly. If the ClassAd already contains attributes, they are cleared. You could also allocated the ClassAd with new and pass it as *classad, if you prefer.

2.2.1 Creating ClassAds manually

If you prefer, you can create and empty ClassAd, then fill it up with expressions that you insert on the fly.. Here is an example: 

ClassAd  *classad;

classad = new ClassAd;

// Insert a = 1
classad->InsertAttr("a", 1);
// Insert b = "Cardini"
classad->InsertAttr("b", "Cardini");

2.3 Creating expressions

All ClassAd expressions are sub-classed from the ExprTree abstract class, which defines a standard interface for all expressions. The specific kinds of expressions (e.g., Literals, FunctionCalls, etc.) are derived from ExprTree, and have protected constructors. With the exception of ClassAd, all other expressions can only be explicitly constructed via their static factory methods.

However, users rarely directly invoke factory methods since expressions may usually be constructed through more convenient means. Expressions can be created in three ways.

  1. As previously mentioned, they can be explicitly created via their factory methods. This is usually only convenient for creating literals, especially time values. For example: 
    ExprTree *now        = Literal::MakeAbsTime(); // current time value
ExprTree *past       = Literal::MakeAbsTime("1999-01-13T09:00:00-0600");
ExprTree *tenMinutes = Literal::MakeRelTime(600); // in seconds
    The Literal::MakeRelTime() factory also has a variant which accepts two time_t values and constructs the interval between their values. If -1 is passed for either argument, the current time is used in that place.
  2. The same ClassAdParser that can create ClassAds from strings can also create any other kind of expression from a string. There are fewer variations of ParseExpression than ParseClassAd. An example: 
    ClassAdParser	parser;
ExprTree	*tree;
ClassAd		*ad;

if( !( tree = parser.ParseExpression( "10 * 17 / foo.bar" ) ) ||
    !( ad = parser.ParseClassAd( "[ a=3 ; b=5 ]" ) ) ) {
    // error
    ...
} else {
    ad->Insert("c", tree);
}
  3. Most classad expressions are simple literals which are used as attributes in classads. These expressions may be implicitly created by simply inserting values into a classad. 
    	ClassAd		ad;

	if( !ad.InsertAttr( "Memory", 128, Value::M_FACTOR )|| // Memory=128M
		!ad.InsertAttr( "Name", "foo.cs.wisc.edu" ) ||     // Name="foo.cs.wisc.edu"
		!ad.InsertAttr( "IsBlue", false ) ) {	           // IsBlue=false
			// error
			...
		}
	}

2.4. Inserting Expressions into ClassAds

Expressions are inserted into a classad via the Insert method. 
ExprTree	*now;
ClassAd		ad;

if( !( now = Literal::MakeAbsTime() ) ||
    !(ad.Insert( "CurrentTime", now ) ) ) {
    // error
    ...
}
If a similarly named expression (case-insensitive) already exists in the ad, the old expression is deleted, and the new expression takes its place. Once inserted, the storage associated with the expression is adopted by the classad---do not deallocate the storage of the expression yourself. If you will need direct access to the expression after you insert it, use the Copy() method first to make a private copy as follows: 
ExprTree *original_expression, *private_copy;

...
// Assume original_expression has been created
private_copy = original_expression->Copy( );

// Now we insert the original_copy
ad.Insert( "foo", original_copy);

// At this point, we shouldn't delete the original_copy
// because it is owned by the ClassAd, but we can manipulate
// the private_copy as much as we like.

2.5 Removing expressions from ClassAds

Expressions may be removed from a Classad in two ways:
  1. The Delete method removes an expression from the classad, and deallocates the storage associated with the expression.
  2. The Remove method removes an expression from the classad, and returns the expression to the user without deallocating its storage. 
ClassAd		ad;

if( !ad.InsertAttr( "foo", 10 ) || !ad.InsertAttr( "bar", 20 ) ) {
    // error
    ...
}

if( !ad.Delete( "foo" ) ) {
    // attr not found in ad
}

ExprTree *tree = ad.Remove( "bar" );

// "tree" is now yours (NULL if ad had no "bar")
...

delete tree;

2.6 Looking Up Expressions in ClassAds

Once expressions have been inserted into classads they may be looked up with the Lookup() method. The method returns the expression (or NULL if no such attribute exists in the classad), but the expression still belongs to the classad. Thus, the expression must not be deallocated, or its parent scope reset. 

ClassAd	ad;
ExprTree	*tree;

ad.InsertAttr( "foo", 10 );
tree = ad.Lookup( "foo" );
If the classad being used is nested in another classad, the LookupInScope() method may be used to test if the expression exists in the classad, or any of its parent scopes. Often you will prefer to evaluate expressions instead of looking them up. See below for more details.

Expressions in the classad may also be enumerated by means of an iterator as illustrated below. 

ClassAd	ad;
	
...
// assume the ad is filled with attributes
ClassAdUnParser unp;
string          buffer;

ClassAdIterator	itr;
string          attrName;
const ExprTree  *attrExpr;


itr.Initialize( ad );
while( !itr.IsAfterLast( ) ) {
	itr.CurrentAttribute( attrName, attrExpr );
	buffer += attrName + " = ";
	unp.Unparse( buffer, attrExpr );
	buffer += "  ";

	itr.NextAttribute( attrName, attrExpr );
}

You can also use STL-style iterators to iterate through a ClassAd. 

ClassAd	ad;
	
...
// assume the ad is filled with attributes

ClassAd::iterator iter;

iter = ad->begin();
while (iter != ad->end()) {
    string    name = iter->first;
    ExprTree *tree = iter->second;
    ...
    iter++;
}

2.7 Evaluating Expressions

Expressions can only be evaluated in the context of a classad. The result of an evaluation is a Value object, which has several methods defined to obtain the type and result of the evaluation.

Expressions already in the classad may be evaluated via the EvaluateAttr() method. 

ClassAd		ad;
Value		val;
int		fooVal;

if( !ad.InsertAttr( "foo" ,10 ) || !ad.EvaluateAttr( "foo", val ) ) {
    // error
    ...
}


if( !val.IsIntegerValue( fooVal ) ) {
    // enh???
}
In the above example, the result of the evaluation was already expected to be an integer. Thus, the following shortcut may be used. 
if( !ad.EvaluateAttrInt( "foo", fooVal ) ) {
    // eval error, or not an int
    ...
}
Similar convenience methods exist for all the other types.

Expressions which have not been inserted into the classad may be evaluated in the context of the classad via the EvaluateExpr() method. However, the expression must be notified of the scope that it is being evaluated in, as illustrated in the following example. (Return values are not checked to keep the example brief.) 

ClassAd       ad;
ClassAdParser parser;
ExprTree      *tree;
Value         val;

ad.InsertAttr( "foo", 10 );
ad.InsertAttr( "bar", 20 );
tree = parser.ParseExpression( "foo - bar" );

tree->SetParentScope( &ad );	// to be evaluated in the context of ad
ad.EvaluateExpr( tree, val );		// now evaluate the expression

2.8 Testing Matches

Testing for a match between two classads is performed by evaluating the "Requirements" expressions of the two classads in an evaluation environment that maps the "other" attribute to the match candidate. The classad language is powerful enough to represent this evaluation environment as a classad. In the API, the MatchClassAd object provides this functionality.

The MatchClassAd object extends a classad by providing two additional functionalities:

  1. It defines two scopes (left and right), which may be filled with the ReplaceLeftAd() and ReplaceRightAd() calls. Since these calls will delete the classad previously occupying that position, you may wish to first RemoveLeftAd() and/or RemoveRightAd().
  2. The MatchClassAd object also defines the following convenience attributes: symmetricMatch, leftMatchesRight, rightMatchesLeft, leftRankValue and rightRankValue.
Thus, to test two classads for a match, proceed as follows: 
MatchClassAd	mad;
ClassAd		*mach, *job;

...
// assume mach and job are two valid classads

// the following is not necessary if no ads were inserted
mad.RemoveLeftAd( );
mad.RemoveRightAd( );

// insert the two ads into the context
mad.ReplaceLeftAd( mach );
mad.ReplaceRightAd( job );

// test 
bool 	match
if( !mad.EvaluateAttrBool( "symmetricMatch" match ) || !match ) {
    // they don't like each other
    ...
} else {
    // they like each other
    ...
}
For your information, the MatchClassAd sets itself up as follows: 
[
   symmetricMatch   = leftMatchesRight && rightMatchesLeft;
   leftMatchesRight = adcr.ad.requirements;
   rightMatchesLeft = adcl.ad.requirements;
   leftRankValue    = adcl.ad.rank;
   rightRankValue   = adcr.ad.rank;
   adcl             =
           [
               other    = .adcr.ad;
               my       = ad;       // for condor backwards compatibility
               target   = other;    // for condor backwards compatibility
               ad       = 
                  [
                      // the ``left'' match candidate goes here
                  ]
           ];
       adcr             =
           [
               other    = .adcl.ad;
               my       = ad;       // for condor backwards compatibility
               target   = other;    // for condor backwards compatibility
               ad       = 
                  [
                      // the ``right'' match candidate goes here
                  ]
           ];
]

2.9 Displaying and Transporting Expressions and Values

ClassAds, expressions and values are displayed by first "unparsing" them into STL strings and then printing them to the screen. The same mechanism is used to transport classads over the network: they are first unparsed into a string which is transported over the network, and then parsed back by the receiver.

Unparsing is performed by the ClassAdUnParser object. The interface is straightforward. 

ClassAdUnParser	unp;
ExprTree	*tree;
ClassAd		ad;
Value		val;

...
// assume tree, ad, val need to be unparsed

string	buffer1, buffer2, buffer3;

unp.Unparse( buffer1, tree );
unp.Unparse( buffer2, &ad );
unp.Unparse( buffer3, val );
A more sophisticated unparser called PrettyPrint is also available. The pretty printer can indent classads and lists, and display parenthesized expressions with the minimal number of required parentheses. 
PrettyPrint	pp;
ExprTree	*tree;
ClassAd		ad;
Value		val;

...
// assume tree, ad, val need to be unparsed

string	buffer1, buffer2, buffer3;

pp.Unparse( buffer1, tree );
pp.Unparse( buffer2, &ad );
pp.Unparse( buffer3, val );

XML Classads

ClassAds can be displayed and parsed in XML as well as the native format used above. For example, a ClassAd in XML format might look like this: 

<?xml version="1.0"?>
<c>
<a n=\"A\"><s>Alain Aslag Roy</s></a>
<a n=\"B\"><i>3</i></a>
</c>

They are parsed and printed similarly to the native format described above. To parse an XML ClassAd you use the ClassAdXMLParser class: 

ClassAdXMLParser  parser;
ClassAd           *classad;
string            xml = ... // The ClassAd

classad = parser.ParseClassAd(xml);

To print out a ClassAd in XML format you use the ClassAdXMLUnParser class: 

ClassAd             *classad;
ClassAdXMLUnParser  unparser;
string              printed_classad;

// Assume classad is already created

unparser.SetCompactSpacing(false);
unparser.Unparse(printed_classad, classad);