How to Use the AI Gateway
The AI Gateway is designed for convenience. It allows you to access many popular AI Models using a standardized API contract defined by the OpenAI API platform. We use LiteLLM to manage this.
Benefits
- Pay-as-you-go. You only pay for the input and output tokens that you generate.
- Consolidated invoice. All vendor expenses are combined into a single bill for simplified billing management.
- You’re in control. Send us a simple request and we’ll update your budget, rate, and token limits as well as your team’s membership.
- Your data is always protected. All models have been fully vetted and are covered by CMU contract terms with Public Cloud providers. So rest assured that your data will not be used for training public models.
- Robust reporting. Track spending over time for your usage or your teams.
Best for
- Instructors who need students to access models using API keys.
- Staff writing application code that needs to call APIs programmatically.
- Researchers who want to integrate AI into their Data Analysis.
Request a Team (Team Owners)
AI Gateway access is team-based. Create multiple teams to control access, track spending, and process invoices.
To request up a LiteLLM team:
- Faculty and Staff should complete the AI Gateway Request and provide:
- An Oracle string
- Team Owners
- Initial Team Members
Note: To change the membership, contact the Computing Services Help Center.
- You’ll receive:
- An API key for your Andrew userID.
- The base URL to use in your requests:
https://ai-gateway.andrew.cmu.edu - A list of supported models.