By John Mak
Before 1988, a study involving tens of thousands of pieces of data would have been a scientist’s life’s work. Years spent meticulously measuring, tabulating, and hypothesizing would mean that only the most groundbreaking hypotheses could afford to be researched. Then, in 1988, UW-Madison revealed and launched Condor. Condor (now HTCondor) is a workload management system that specializes in high-throughput computing, seamlessly combining an entire organization’s computing power into one super-processor. This means that millions of calculations and measurements can be completed within a single day, turning what would have been a life’s work into a project that could be completed within a year. With the launch of Condor, the world of research was vastly expanded, and the possibilities for research became endless.
Renamed in 2012, HTCondor now helps researchers (and animators) achieve their dreams all over the globe, including at such well-known organizations as NASA, DreamWorks Animation, Boeing, CERN, and the Hubble Space Telescope operations center. Back home at UW-Madison with the support of the Center for High Throughput Computing, HTCondor has delivered in the past year alone over 350 million CPU hours to UW-Madison researchers from over 50 departments across campus in support of nearly 300 different research projects. In the College of Letters and Science, many researchers are using HTCondor in innovative and brilliant ways, from botany to psychology to space exploration.
“I can’t imagine doing the work we’ve been doing for the last ten years without it,” says Edgar Spalding, a faculty member and researcher in the Department of Botany. Spalding and his team used HTCondor for two projects: one to study the genome of corn, and another the effects of gravity on plant root growth. With the corn genome study, Spalding and his team wanted to identify specific sequences in the genome of corn that influenced specific physical traits. The only issue was that there were thousands of ears of corn to scan and examine, a process that would have ordinarily taken years. “If we want to study a population of plants, each one genetically different from the other, and we have to study each genetic version of the plants
multiple times, then the number of images grows really large,” Spalding remarked. With HTCondor, Spalding and his Agronomy collaborators could scan pictures of the corn, develop algorithms to measure specific traits such as ear size and kernel shape, and pass the tool off to HTCondor, along with their scanned data, to perform the calculations. As a result, what would have been years of tedious manual measurement was reduced to a couple months.
When asked about his upcoming projects involving HTCondor, Spalding revealed that his lab has been flying a drone over fields of corn for the past four years and taking pictures, totaling up to over 40,000 images. Spalding plans to use these images to identify the portion of the corn genome that influences kernel spacing, another project that would be impossible without HTCondor.
With his plant-root gravitropism experiment, Spalding’s lab had a dozen petri dishes with a single tiny plant growing at an angle, each hooked up to an incredibly powerful camera, which took pictures of the plant’s roots every five minutes for several hours. Spalding’s goal was to identify which portions of the root actually extended to redirect the root towards gravity, meaning that the changes they were trying to identify were extremely miniscule, arguably unidentifiable to the human eye. As such, Spalding’s lab is currently training an AI to detect these tiny changes between pictures and is going to run the AI through HTCondor to perform their experiment. They hope to find out exactly which sections of the root actually lengthened (and whether this was a consistent section between plants), in order for the roots as a whole to grow downwards.
In the Department of Psychology, graduate student Sarah Sant’Ana, PI John Curtin, and graduate student Gaylen Fronk are working with Hannah Moshontz de la Rocha of Duke University to “help psychological scientists understand and appropriately use a technique called k-fold cross validation (CV),” explains Moshontz. She continues, “Psychological scientists are new to machine learning approaches which involve using k-fold CV (or another re-sampling technique) to select among different statistical learning algorithms.”
” . . . this kind of project simply wouldn’t be feasible without HTCondor,” says Moshontz.
In their study, Moshontz and her team illustrate that a commonly used form of k-fold CV, single loop k-fold CV, can overestimate how well an algorithm will perform in new data. This is especially likely to happen in data sets with small samples, which are common in psychology. Other forms of k-fold CV, like nested CV, provide accurate estimates of model performance even in small samples. “To demonstrate the limitations of single loop k-fold CV, we compare it to two other forms of k-fold CV, applying all three forms to more than 20,000 simulated (i.e., manufactured) data sets with different sample sizes, numbers of variables, and outcome variables,” says Moshontz.
The project has consumed 1.5 million computing hours so far and is projected to consume 2.5 million more, the total equivalent of 456 years if it were run on a single computer. “Because of how many data sets we test and how much time testing just one takes, this kind of project simply wouldn’t be feasible without HTCondor,” says Moshontz.
In the Department of Physics, Joel Siegel’s lab used HTCondor to design a self-stabilizing laser sail space explorer, a revolutionary new space probe that uses the reflection of lasers to propel itself across space at a fifth of the speed of light. This makes Siegel’s laser sail the fastest human-made object in history. Instead of a rocket and conventional rocket fuel, a laser sail traverses space via a 10 gram sail with a 100 gigawatt laser (the equivalent of 10 nuclear power plants) firing at it. As the laser reflects off the sail, it generates a force, propelling the sail in the desired direction. Siegel’s sail is designed in such a way that it not only maximizes the force generated but also stabilizes itself against outside forces and perturbations, allowing it to stay its course. “It’s a new way to explore the stars,” Siegel says. Siegel’s lab ran one million simulations through HTCondor, slowly perfecting the sail’s design. These simulations required a huge amount of power on the scale of 80 CPUs, 500 GB of RAM, and 5GB of disk space, with each simulation outputting data of about 5GB in size, making HTCondor the only way this project would be possible.
Regardless of field, HTCondor has helped countless researchers in their work, allowing them to conduct studies and experiments on a scale previously unheard of. Whether it’s localizing a corn genome, improving the accuracy of psychological studies, or finding a new way to explore outer space, HTCondor has bridged the gap between imagination and reality.