AI Gateway
The AI Gateway is a centralized, CMU-supported service, built using LiteLLM, that provides developers with streamlined access to multiple AI models using a consistent, OpenAI-compatible API. It enables experimentation and production use of large language models (LLMs) through a unified interface that simplifies model selection, team management, and budget tracking.
Benefits
- Unified API key for access to top-tier AI models.
- Team and individual level usage tracking and budget control.
- Simplified account and provisioning management.
- Compatible with both on-premise and cloud-hosted models.
- Supports academic, research, and development use cases.
- Transparent usage reporting with enterprise support.
- Usage dashboards and consumption limits to prevent overuse.
Eligibility
- Faculty and staff (may request and manage teams).
Fees
- Pay-as-you-go model, with costs depending on the model and volume used.
- Teams must provide an Oracle string for cost recovery.