Carnegie Mellon University

CMU Foundation and Language Model (FLAME) Center 

Foundation models – machine learning models that are pre-trained on open-world data and then specialized for specific tasks – are fast becoming a dominant paradigm in AI. This new approach raises significant questions for the field. What sorts of algorithmic and architectural techniques should be used for such models? How do we evaluate the quality and performance of foundation models, especially in settings where there does not exist much downstream tasks data? And how do we grapple with the larger implications of these models and their ongoing adoption in industry?

CMU FLAME Center provides a home and computational resources at Carnegie Mellon for members working on different aspects of foundation models. We foster fundamental research into powerful, open, and responsible foundation models applicable to all sectors of science and technology.

Interested in learning more?

Contact us

Our Mission

To encourage faculty and students at Carnegie Mellon to pursue collaborative, fundamental research related to foundation models and to enable the most promising projects to scale up experimentation and final downstream impact.

Areas of Expertise

Rigorous Benchmarking of Language Models
We have developed some of the most widely used benchmarks for language models, such as the HotpotQA multi-hop reasoning dataset and the CheckList method for foundation model debugging.

Building New Foundation Models
We have built state-of-the-art foundation models for math (Llemma), speech (OWSM), long-document modeling (XL-Net and Mamba), and code generation (Polycoder) among many others

New Application Areas of Language Models
We are applying language models to novel application domains, such as automated science, creative writing, music, and many more.

Language Model Systems
We are at the forefront of efficient methods for language model deployment, including the widely-used MLC LLM project, and efficient methods for LLM inference.

Responsible Development Practices
We have highly impactful research on the responsible development of AI systems, including examining environmental impacts and bias in language models. They have also examined language model security, demonstrating that language models can be caused to generate unintended outputs, or prompts can be exfiltrated.

Techniques for Better Utilizing Foundation Models
We have invented methods that are now used widely in LLM-based applications such as prompt engineering, or program-aided language models.