AI Gateway

The AI Gateway is a centralized, CMU-supported service, built using LiteLLM. This tool provides developers with streamlined access to multiple AI models using a consistent, OpenAI-compatible API. It enables experimentation and production use of large language models (LLMs) through a unified interface that simplifies model selection, team management, and budget tracking.

Benefits

• Unified Access and Billing: Single API key for multiple top-tier AI models, streamlined account management, and one consolidated pay-as-you-go invoice based only on token usage.
• Financial Control: Transparently set and manage team/individual budgets, consumption limits, and rates using detailed usage dashboards and reporting.
• Data Security: Models are fully vetted and covered by CMU contract terms with Public Cloud providers to guarantee that your data is not used for training public models.
• Broad Compatibility: Supports all academic, research, and development use cases with compatibility for both on-premise and cloud-hosted models.

Eligibility

• Faculty and staff (may request and manage teams).

Best For

• Instructors who need students to access models using API keys.
• Staff writing application code that needs to call APIs programmatically.
• Researchers who want to integrate AI into their Data Analysis.

Fees

• Pay-as-you-go model, with costs depending on the model and volume used. You only pay for the input and output tokens that you generate.
• Teams must provide an Oracle string for cost recovery.

Request a Team (Team Owners)