Carmella Li Brings Physics Knowledge to AI Research
Heidi Opdyke
- Associate Dean of Marketing and Communications, MCS
- Email opdyke@andrew.cmu.edu
- Phone 412-268-9982
Inside today’s most advanced artificial intelligence systems, billions of invisible interactions unfold. Recent Carnegie Mellon University graduate Carmella Li is using physics to understand what’s happening.
Modern AI models can generate text, images and predictions, but researchers are unsure how models arrive at those results. Li, who graduated with a major in physics on the quantum physics track with an additional major in artificial intelligence, saw an opportunity.
“We do not have a fundamental theoretical understanding of how large language models work,” Li said. “So we wanted to see if we could interpret neural networks using quantum field theory.”
Quantum field theory is a framework typically used to model subatomic particles and their interactions. Li and Riccardo Penco, associate professor of physics, wanted to use that same framework to describe how the neural networks behind artificial intelligence work.
Each neural network contains a series of nodes, which simulate neurons in the brain. Not all nodes connect with each other, and when nodes do connect, some are more likely to interact than others. Previous research done outside of Carnegie Mellon attempted to describe how nodes work in transformer architectures, structures that make up the basis of some large language models. Penco and Li wanted to see if quantum field theory could be applied to simpler AI architectures.
Li used scalar field theory, a subset of quantum field theory that describes where different points are in space, usually in relation to excitation of particles. By modeling neural network nodes within this framework, Li could create a plot of how she expected the nodes to behave. She then compared the plot to multiple AI programs that she and Penco had access to. She found that the method she used applied multiple types of neural network structures, meaning the framework could be used to gain a better understanding of how AI programs function.
“This idea is more or less universal,” Li said. “We could probably use physics methods to describe AI models.”
Li conducted this work partially through Carnegie Mellon’s Summer Undergraduate Research Fellowship, which supports independent student research. Penco said her ability to bridge physics theory and computational practice stood out.
“I was extremely impressed with the rapid pace at which she made progress and her ability to combine understanding of quantum field theory concepts with coding proficiency,” Penco said. “By the end of the summer, she had already made two novel contributions.” Those contributions included correcting errors in an existing codebase and deriving a new analytic expression for neural network behavior, work that required mathematical rigor and programming skill.
Li said she hopes that her findings will lead to both better understanding of how AI models work as well as more targeted training.
“Large language models work because they have very complicated structures with a lot of parameters,” Li said. “Having a theoretical understanding could accelerate the process of training and perhaps save energy."
For her efforts, Li earned the Judith A. Resnik Award. Named for Carnegie Mellon alumna and astronaut Judith Resnik, the award honors an exceptional senior graduating from a technical course of study and pursuing a graduate degree. She was also the inaugural recipient of the Gilman Award in Physics, a newly established award recognizing exceptional research excellence by a graduating senior. Gillian Ryan, teaching professor and director of undergraduate affairs for the Department of Physics, said Li’s contributions were notable.
“Carmella is a versatile, meticulous and driven physicist,” said Ryan, who was Li’s advisor. “She is uniquely positioned to become a transformative figure at the intersection of physics and machine learning.”
This fall, Li will begin a Ph.D. in particle physics at Columbia University, where she hopes to continue exploring deep theoretical questions.
“Theoretical physics lets you work on interesting problems in many aspects of physics,” Li said. “I’m really excited to look at whatever interests me.”
