Advancing the Science of AI Agent Evaluation Workshop

A Cross-Sector Workshop hosted by CMU AIMSEC and Microsoft Research

The Evaluation Gap for AI Agents

As AI agents move from research prototypes into systems that act on people’s behalf, our ability to evaluate them has not kept pace. Rigorous, shared measurement science is the foundation of accountability, and it is currently one of the field’s most significant gaps.

On October 22–23, 2026, Microsoft Research and Carnegie Mellon University’s AI Measurement Science & Engineering Center (AIMSEC) will convene 150+ key leaders across academia, industry, civil society, and government to define shared standards and measurement science for evaluating and governing AI agents.

The program combines keynote presentations, lightning talks, panel discussions, and hands-on working groups addressing AI agent evaluation frameworks, tooling, and domain-specific challenges. Attendees will participate directly in breakout discussions that will inform open public synthesis reports for the field.

Resources 

Learn More

AIMSEC Speakers

KEYNOTE SPEAKER


   Zico Kolter
  Professor, Carnegie Mellon University; Board of Directors, OpenAI

PANELISTS


   Ramayya Krishnan
  Director, AI Measurement Science and Engineering Center (AIMSEC) - Carnegie Mellon University

   Zico Kolter
  Professor, Carnegie Mellon University; Board of Directors, OpenAI

LIGHTNING TALK SPEAKERS


   Rema Padman
  Carnegie Mellon University

   Aditi Raghunathan
  Carnegie Mellon University

   Lujo Bauer
  Carnegie Mellon University

   Hong Shen
  Carnegie Mellon University

   Vyas Sekar
  Carnegie Mellon University

   Fernando Diaz
  Carnegie Mellon University

   Daniel Fried
  Carnegie Mellon University