Tibshirani Receives Department of Defense MURI Award

Ryan Tibshirani, associate professor in the Department of Statistics as well as the Machine Learning Department at Carnegie Mellon University, has received a 2020 Department of Defense (DOD) Office of Naval Research Multidisciplinary University Research Initiative (MURI) award.

Tibshirani joins a multi-institutional team on a project titled "Theoretical Foundations of Deep Learning.” The team will receive $6M over five years to develop a principled theory of deep learning that is based on rigorous mathematical principles. This project will emphasize mathematical foundation that quantifies the advantages of deep networks over more traditional approaches.

“This a truly unique opportunity to work with leaders in fields [other] than my own,” said Tibshirani. “I can already tell from the work we did in putting the proposal together that we will all learn a lot from each other.”

The team includes members from University of Maryland, Texas A&M University, University of Wisconsin, University of California, Los Angeles and Rice University. The participants bring their expertise in nonlinear approximation theory, nonlinear differential equations, large-scale optimization, high-dimensional statistics, formal methods and computer vision.

Tibshirani will work with his colleagues to try to figure out what types of "trends" deep neural network architectures can learn, as well as understand the "implicit regularization" properties of various algorithms that are used to train neural networks.

According to Tibshirani, the field still does not understand what is going on statistically inside a deep net. The complexity and sheer size of network structures will continue to elude our understanding unless new theory is developed. At this time, a lack of understanding can lead to unreliable results.

“The hypothesis is that, in training [the deep neural network architectures], we are "implicitly" imposing some kind of regularization which is controlling their effective complexity,” said Tibshirani. “Understanding this precisely is one of the bigger open questions in the field right now.”

The MURI program is a tri-service DOD program that supports research teams whose research efforts intersect more than one traditional science and engineering discipline. A multidisciplinary team effort can accelerate research progress in areas particularly suited to this approach by cross-fertilization of ideas, can hasten the transition of basic research findings to practical applications and can help to train students in science and/or engineering.