By Rachel Robey
Josiah Hanna’s soccer-playing robots train with reinforcement learning, a branch of artificial intelligence that mimics one of the ways humans learn.
“The world is a fundamentally uncertain place,” says Josiah Hanna, Assistant Professor of Computer Sciences. “It’s not like a game of chess, where all the information you need is right in front of you.”
To Hanna, a researcher in robotics and artificial intelligence, the world is like soccer: It’s dynamic, it’s fast paced, and there’s too much going on to process everything at once. That’s what makes “the beautiful game” the ideal forum for training his intelligent robots of the future — ones capable of reasoning, decision-making, and bending it like Beckham.
Designing intelligent robots of the future
In Hanna’s lab, one of the more popular research projects among undergraduates is the RoboCup team. The small team competes annually at RoboCup, an annual robot soccer tournament that brings international researchers together to promote and foster robotics research. At its founding in 1997, RoboCup pledged a “publicly appealing, but formidable” goal: create soccer-playing robots capable of beating the World Cup champions by 2050. Equal parts ambitious and entertaining, it’s been an effective driver for public interest in robotics.
Guided by Hanna, student researchers program and train autonomous soccer-playing robots — until recently, they worked with Aldebaran NAO robots — using cutting-edge AI techniques. Each summer, Hanna, the students, and their robots travel to RoboCup to play other teams in five-on-five and seven-on-seven matches. In France in 2023, Wisconsin’s first year participating, the team came in third in the Challenge Shield League. In the Netherlands in 2024, they formed WisTex United with University of Texas at Austin and came in first place. In Brazil in 2025, they jumped up to the Champions Cup League and came in third.
Yet the real reward isn’t ending up on the podium — it’s inching both researchers and robots closer to the future of robotics. Hanna and his students have also been steadily publishing papers at robotics and AI conferences to further expand on the learnings from their RoboCup progress. “Games are one of the first things humans learn to do. They’re where we first develop skills that will serve us for the rest of our lives,” says Hanna. “As we push the boundaries of artificial intelligence, it makes sense to use games as a test bed.”
Teaching robots through play
Plenty of intelligent animals learn through play, from human children to orca calves to fledgling crows. Across species, it’s a seemingly fundamental way of engaging with the world. Hanna’s lab borrows this approach for training intelligent non-life, hypothesizing that teaching robots to play soccer is a gateway to teaching them other things: First they beat the World Cup Champions, then they revolutionize disaster response and rescue operations. Or fight wildfires. Or any number of scenarios, says Hanna, in which the “robot is faced with uncertainty about the true state of the world and must make decisions quickly.”
While many RoboCup teams manually program their robots for a range of gameplay scenarios, his team relies on reinforcement learning, a branch of machine learning. Using this method, autonomous agents learn how to achieve (in this case, score) goals through trial-and-error — not unlike how humans learn. Hanna’s students train robots using a simulated environment, allowing them to run different scenarios and figure out how to handle them faster than real time. So far, this has given Wisconsin’s robots an edge over many other teams’ hand-designed robots.
Regardless of how the bots are programmed, two things are always true on the soccer pitch: They must perform autonomously, and the unexpected will always occur. (“Sometimes during a game they just end up in a pile,” says Will Cong, a third-year PhD student in Hanna’s lab.) This is when the competition boils down to which team has the winning approach. If RoboCup results are any indication, Wisconsin’s reinforcement learning approach has potential.
Dull, dirty, dangerous
“Robots have tremendous promise for our society,” says Hanna. “There are various industries where there aren’t enough people to fill all open jobs.” Among robotics researchers, the “Three D’s of Robotics,” operates as a catch-all for the three kinds of jobs (dull, dirty, and dangerous) that robots are best suited for. In these situations, Hanna says, embodied artificial intelligence could improve and support human life.
But before robots can do that, Hanna says researchers must address the gulf between human and artificial intelligence. RoboCup advances this goal through friendly international competition, pushing researchers to continually improve. Today’s robots are already unrecognizable from when RoboCup was founded — in the coming year, Hanna will begin working with Booster K1 robots. “We’re looking at more advanced humanoid robots that support faster movement and have more computational power for modern AI models,” he says.
There’s still a long way before we have World Cup champion-level robots, however. Hanna cites Moravec’s paradox, which observes that the perception and mobility skills of a young child are very hard for a computer to reproduce, yet the things humans most associate with intelligence — like playing chess or complicated calculations — are effortless for computers. So how does one bridge the gap between such different forms of intelligence?
“The fun part about research is solving problems that no one has ever solved before,” he says by way of explanation. “Even failed experiments are still a learning signal. We’ve discovered something that didn’t work.” Eventually, Hanna knows the technology and methods will click — and then we’ll be in the next era of intelligent robotics. For now, Wisconsin researchers are enthusiastically working toward it.
This is part one of a series covering robotics at Wisconsin. Subscribe to WisComputes or follow along on LinkedIn and Instagram for parts two and three.
