Supporting NIST’s Development of Guidelines on Red-teaming for Generative AI
Past Event: February 23, 2025
CMU expert convening on “Supporting NIST’s Development of Guidelines on Red-teaming for Generative AI”.
Sponsored by the Responsible AI Initiative at the Block Center and the K&L Gates Initiative on Ethics and Computational Technologies at Carnegie Mellon.
Context: The recent presidential order on “Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence” requires NIST to produce a set of guidelines around red-teaming of generative AI models by July 26th, 2024. In February 2024, CMU hosted an expert conevening that sought to support NIST’s effort in this area by bringing prominent voices from academia, government, industry, and civil society, to discuss and shape these guidelines.
Speakers
Theresa Mayer - Vice President for Research - Carnegie Mellon University
Theresa S. Mayer is Carnegie Mellon University’s Vice President for Research, providing leadership for the University’s research enterprise and advocating for the role that science, technology, and innovation play nationally and globally.
Theresa holds joint faculty appointments in the Department of Electrical and Computer Engineering and the Department of Materials Science and Engineering in the College of Engineering.
Ramayya Krishnan - William W. And Ruth F. Cooper Professor Of Management Science And Information Systems
Ramayya Krishnan is the W. W. Cooper and Ruth F. Cooper Professor of Management Science and Information Systems at Heinz College and the Department of Engineering and Public Policy at Carnegie Mellon University. A faculty member at CMU since 1988, Krishnan was also the Dean of Heinz College from 2009 - 2025.
Hoda Heidari - K&L Gates Career Development Assistant Professor in Ethics and Computational Technologies - Carnegie Mellon University
Hoda Heidari is the K&L Gates Career Development Assistant Professor in Ethics and Computational Technologies at Carnegie Mellon University with joint appointments in the Machine Learning Department and the Institute for Software, Systems, and Society. She is also affiliated with the Human-Computer Interaction Institute and the Heinz College of Information Systems and Public Policy at CMU, and co-leads the university-wide Responsible AI Initiative.
Zico Kolter - Associate Professor, School of Computer Science - Carnegie Mellon University
Zico's research focuses on computational approaches to sustainability and energy domains, focusing on core challenges arising in machine learning, optimization, and control in these areas. On the application side, his interests range from improving the efficiency of generation, controlling power in smart grids, and analyzing energy consumption in homes and buildings. To attack these problems, Zico focuses on techniques from machine learning, reinforcement learning, time series prediction, approximate inference, and convex optimization, amongst others.
Elham Tabassi - Associate Director for Emerging Technologies, Information Technology Laboratory - National Institute for Standards and Technology
Elham Tabassi is a Senior Scientist at the National Institute of Standards and Technology (NIST) and the Associate Director for Emerging Technologies in the Information Technology Laboratory (ITL). She also leads NIST’s Trustworthy and Responsible AI program that aims to cultivate trust in the design, development, and use of AI technologies.
Graham Neubig - Associate Professor, School of Computer Science - Carnegie Mellon University
Graham's researched is concerned with language and its role in human communication. In particular, his long-term research goal is to break down barriers in human-human or human-machine communication through the development of natural language processing (NLP) technologies. This includes the development of technology for machine translation, which helps break down barriers in communication for people who speak different languages, and natural language understanding, which helps computers understand and respond to human language.
Sanmi Koyejo - Assistant Professor, Computer Science - Stanford University
Sanmi leads the Stanford Trustworthy AI Research (STAIR) group. STAIR works to develop the principles and practice of trustworthy machine learning, including fairness and robustness. Their applied research includes applications to healthcare and neuroimaging.
Matt Fredrikson - Associate Professor, School of Computer Science - Carnegie Mellon University
Matt's research aims to enable systems that make secure, fair, and reliable use of machine learning. His group focuses on finding ways to understand the unique risks and vulnerabilities that arise from learned components, and on developing methods to mitigate them, often with provable guarantees.
Margaret Mitchell - Researcher and Chief Ethics Scientist - Hugging Face
Margaret Mitchell is a computer scientist who works on algorithmic bias and fairness in machine learning. She is most well known for her work on automatically removing undesired biases concerning demographic groups from machine learning models, as well as more transparent reporting of their intended use.
Zach Lipton - Chief Technology Officer / Chief Science Officer - Abridge
Zachary Lipton is an Assistant Professor of Machine Learning at Carnegie Mellon University and the Chief Scientific Officer of Abridge. He directs the Approximately Correct Machine Intelligence (ACMI) lab, whose research focuses include the theoretical and engineering foundations of robust and adaptive machine learning algorithms, applications to both prediction and decision-making problems in clinical medicine, natural language processing, and the impact of machine learning systems on society.
Katherine Lee - Senior Research Scientist - Google DeepMind
Katherine is currently a senior research scientist at Google DeepMind and runs the GenLaw Center. She studies security and privacy in generative AI models and the legal implications those have. Specifically, Katherine evaluates data extraction (memorization) in generative AI models and attacks (mis-aligning) for generative AI models.
Ece Kamar - Managing Director of AI Frontiers - Microsoft
Ece Kamar is the Managing Director of AI Frontiers, leading research and development towards pushing the frontiers of AI capabilities. AI Frontiers is a non-geographical, mission-focused lab inside Microsoft Research that explores innovations in foundation models and platform capabilities to push the frontier of AI capabilities, efficiency and control.
Lama Ahmad - Technical Program Manager, Policy Research - OpenAI
As a part of the Policy Research team at OpenAI, Lama works on the safe development and deployment of AI models. Lama leads the Researcher Access Program to enable the study of key areas related to the responsible deployment of AI and mitigating risks associated with such systems.
Katie Robinson - Associate Design Researcher - Software Engineering Institute
Katherine-Marie Robinson is an assistant design researcher in the SEI’s AI Division. Since joining the SEI in September 2022, Robinson has worked on a wide variety of projects where she aims to bring a responsible AI (RAI) lens to the work at hand including researching and developing tools, curriculums, and frameworks.
Daphne Ippolito - Assistant Professor, School of Computer Science - Carnegie Mellon University
Daphne is an assistant professor in the Language Technologies Institute at Carnegie Mellon University. She studies the tradeoffs and limitations of generating text with neural language models, as well as strategies for evaluating natural language generation systems. She also researches how to incorporate AI-in-the-loop language generation into assistive tools for writers.
Param Singh -Carnegie Bosch Professor of Business Technologies, Tepper School of Business - Carnegie Mellon University
Param's ongoing research attempts to bring insights from economics and machine learning to bear on topics in artificial intelligence and its implications to businesses and society. He studies algorithmic bias, transparency and interpretability from an economic perspective and investigate the impact of artificial intelligence based solutions in mitigating social inequality.
Aaditya Ramdas - Assistant Professor, Dietrich College / School of Computer Science - Carnegie Mellon University
Aaditya's research is in mathematical statistics and learning, with an eye towards designing algorithms that both have strong theoretical guarantees and also work well in practice. His main research interests include selective and simultaneous inference (interactive, structured, online, post-hoc control of false decision rates, etc), game-theoretic statistics (sequential uncertainty quantification, confidence sequences, always-valid p-values, safe anytime-valid inference, e-processes, supermartingales, etc), and distribution-free black-box predictive inference (conformal prediction, post-hoc calibration, etc). His areas of applied interest include privacy, neuroscience, genetics and auditing (elections, real-estate, financial, fairness), and his group's work has received multiple best paper awards.
Jason Hong - Professor, Human-Computer Interaction Institute - Carnegie Mellon University
Jason Hong is a professor in the Human Computer Interaction Institute, part of the School of Computer Science at Carnegie Mellon University. He works in the areas of usability, mobility, privacy, and security, and his research has been featured in the New York Times, MIT Tech Review, CBS, CNN, Slate, the World Economic Forum, and more. Jason is currently on the editorial board for Communications of the ACM, and previously for ACM Transactions on Human Computer Interaction, IEEE Pervasive Computing, and Foundations and Trends. He is an author of the book The Design of Sites, a popular book on web design using web design patterns. Jason is also a co-founder of Wombat Security Technologies, which was acquired by Proofpoint in March 2018.
Peter Zhang -Assistant Professor, Heinz College - Carnegie Mellon University
Peter received his Bachelor of Applied Science in Engineering Science and Master of Applied Science from the University of Toronto, and Ph.D. in Engineering Systems from Massachusetts Institute of Technology. His research interests are in optimization theory, with applications in supply chain, transportation, health, and other socio-technical systems.
Yonatan Bisk - Assistant Professor, School of Computer Science - Carnegie Mellon University
Yonatan Bisk is an assistant professor of computer science in Carnegie Mellon's Language Technologies Institute. His group works on grounded and embodied natural language processing, placing perception and interaction as central to how language is learned and understood. Previously, he received his PhD from the University of Illinois at Urbana-Champaign working on unsupervised Bayesian models of syntax, before spending time at USC's ISI (working on grounding), the University of Washington (for commonsense research), and Microsoft Research (for vision+language).