AI Developer API Gateway (LiteLLM)
Computing Services is providing an AI Developer API Gateway to make AI models easier to access. This gateway will enable developers to access CMU-curated AI API endpoints, including cloud-hosted and on-premises models. The requestor will still need to pay any model costs.
Key Features
LiteLLM is the product behind the API Gateway for CMU. Designed to remove friction from staff, faculty, and student developers when building an AI application, LiteLLM will:
- Provide access management to models via API keys.
- Establish guardrails for model usage and cost.
- Streamline access to and payments for Cloud AI endpoints.
- Address other issues that complicate the management of AI model endpoints.
Who should use LiteLLM?
The primary use cases that will benefit from LiteLLM are:
- Faculty or Staff members who need to provide API keys to a team or class.
- Faculty or Staff who would like access to models but don’t want the overhead of setting up and managing a cloud environment.
- Faculty or staff who have fixed budgets or need to implement guardrails.
How can I start using LiteLLM?
Are you interested in using LiteLLM for your research or projects at CMU? Please complete the LiteLLM Interest Form below to request access and receive updates.
Note: An Oracle String is required to pass on any costs incurred from using a model endpoint. There is no cost associated with the use of the LiteLLM app itself.