Skip to main content
Zico Kolter -

Zico Kolter

Associate Professor and Director of Machine Learning

Zico Kolter researches how to make deep learning algorithms more robust, safer, and understand how data impacts how models function.


Expertise

Topics:  Elections, Large Language Models, Generative AI, Neural Networks, Deep Learning, Machine Learning, AI Models

Zico Kolter is a Professor of Computer Science and the head of the Machine Learning Department at Carnegie Mellon University, where he has been a key figure for 12 years. Zico completed his Ph.D. in computer science at Stanford University in 2010, followed by a postdoctoral fellowship at MIT from 2010 to 2012. Throughout his career, he has made significant contributions to the field of machine learning, authoring numerous award-winning papers at prestigious conferences such as NeurIPS, ICML, and AISTATS.

Zico's research includes developing the first methods for creating deep learning models with guaranteed robustness. He pioneered techniques for embedding hard constraints into AI models using classical optimization within neural network layers. More recently, in 2023, his team developed innovative methods for automatically assessing the safety of large language models (LLMs), demonstrating the potential to bypass existing model safeguards through automated optimization techniques. Alongside his academic pursuits, Zico has worked closely within the industry throughout his career, formerly as Chief Data Scientist at C3.ai, and currently as Chief Expert at Bosch and Chief Technical Advisor at Gray Swan, a startup specializing in AI safety and security.

Media Experience

Small Language Models Are the New Rage, Researchers Say  — Wired
Small Language Models (SMLs) are capturing the attention of researchers. Using less power than LLMs, they are not used as general purpose tools, instead they focus on narrowly defined tasks like summarizing conversations. "The reason [SLMs] get so good with such small models and such little data is that they use high-quality data instead of the messy stuff,” said Zico Kolter (School of Computer Science).

The AI Agent Era Requires a New Kind of Game Theory  — Wired
Zico Kolter, a Carnegie Mellon professor and board member at OpenAI, tells WIRED about the dangers of AI agents interacting with one another—and why models need to be more resistant to attacks.

Meet Alphalab’s 2025 cohort of innovative Pittsburgh startups  — Technical.ly
Zico Kolter (School of Computer Science) will join ex-Google chief Eric Schmidt's AI Safety Science program. Schmidt is spending $10M on fundamental research into safety problems in AI. Kolter's role will focus on AI attacks.

From emotional bonds with chatbots to the impact of AI on government jobs  — Pittsburgh Post-Gazette
Recapping the K&L Gates conference, this piece highlights Zico Kolter (School of Computer Science) and Carol Smith (Human-Computer Interaction Institute) debate of the risks of unregulated AI development. The discussion reflected the conference's broader theme about AI governance, safety and global competition.

Pittsburgh’s AI-Powered Renaissance  — CMU News
"Pittsburgh has positioned itself as a worldwide leader in AI, led of course by Carnegie Mellon's long-time leadership and dedication to the field. Starting with Allen Newell and Herb Simon's inspiration and initiative, to the founding of departments dedicated to AI like the Machine Learning Department and Robotics Institute, and continued with today's influence on Generative AI and creation of AI startups, CMU has been a driving force in AI since the field's inception. With the recent continued expansion and public awareness of AI, in addition to continually welcoming numerous AI focused businesses, startups and research facilities to the city, Pittsburgh itself is well-positioned to capitalize on our lasting contributions."

Zico Kolter Joins OpenAI’s Board of Directors  — Bloomberg
“I think part of my value is being deeply involved and integrated in research, and at the forefront of what’s happening in the field of not just the deployment of AI but the academic research into AI,” he said.

Can smart solutions be artificial? They sure can!  — Kosch
In our interview series “Thought leaders in AI”, we had the opportunity to talk to Zico Kolter, Chief Scientist for AI at Bosch, about his personal view on various topics in the field of artificial intelligence. An AI system played the moderator and asked him questions on various exciting topics: How does he see the differences in AI development between Europe and the USA? Which celebrities would he like to meet one day? And finally: Which Bosch product does he particularly like?

LLMs Pose Major Security Risks, Serving As ‘Attack Vectors’  — C3.ai
Zico Kolter, an associate professor of Computer Science at Carnegie Mellon and author of the report, Universal and Transferable Adversarial Attacks on Aligned Language Models, put it bluntly: “These tools are attack vectors,” he said.

How researchers broke ChatGPT and what it could mean for future AI development  — ZDNET
"There is no obvious solution," Zico Kolter, a professor at Carnegie Mellon and author of the report, told the Times. "You can create as many of these attacks as you want in a short amount of time."

Education

Ph.D., Computer Science, Stanford University
B.S., Computer Science, Georgetown University

Spotlights

Pittsburgh’s AI-Powered Renaissance
(October 14, 2024)

Links

Event Appearances

Moderator: AI in Financial Services: Transforming the Sector for a Better World
AI Horizons Pittsburgh Summit, Pittsburgh, PA
October 10, 2024

Speaker: AI Horizons Keynote: AI for a Better World – Navigating Truth in the AI Era
AI Horizons Pittsburgh Summit, Pittsburgh, PA
October 10, 2024

Articles

Rethinking LLM Memorization through the Lens of Adversarial Compression  —  arXiv preprint

Forcing Diffuse Distributions out of Language Models  —  arXiv preprint

Massive Activations in Large Language Models  —  arXiv preprint

Tofu: A task of fictitious unlearning for llms  —  arXiv preprint

Scaling Laws for Data Filtering–Data Curation cannot be Compute Agnostic  —  Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

Photos

Videos