UW-Madison Computer Sciences professor Stephen Wright and three colleagues formerly from the department were announced winners of the Test of Time Award at the 2020 Conference on Neural Information Processing Systems (NeurIPS). Wright, alongside former UW-Madison Computer Sciences professors Ben Recht and Chris Ré and former UW-Madison Ph.D. student Feng Niu, received this award for their 2011 paper’s lasting impact on the field of Machine Learning. The authors were also members of the optimization group at the Wisconsin Institute for Discovery (WID), which was in its first months of operation when the paper was written.
NeurIPS is one of the most prestigious conferences in artificial intelligence (AI). The Test of Time Award is given to a paper presented at the conference approximately 10 years ago that has had the most significant impact in the field of AI. This is only the fourth time this award has been presented.
Wright and his colleagues received this award for their 2011 paper “Hogwild!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent.” Their paper proposed an alternative way to implement Stochastic Gradient Descent (SGD) without any locking of memory access, that “outperforms alternative schemes that use locking by an order of magnitude.” SGD is the algorithm that drives many machine learning systems.
“Our NeurIPS 2011 paper contains both analysis and computational results, both of which verify that the Hogwild! approach is effective,” said Wright in a press statement read at the award press conference Monday morning. “Nowadays, asynchronous parallel approaches are ubiquitous, and our analysis has subsequently been extended in many ways, by us and many others,” said Wright.
Wright emphasized that collaboration between the fields of computer systems, optimization, and machine learning was crucial in developing the approach. “Going forward, we believe that the systems and optimization perspectives will continue to have a lot of value for machine learning,” said Wright. “Conversely, we note that machine learning has contributed – and will continue to contribute – to developments in computer systems and optimization.”
To view the acceptance talk for the award, visit this link.