Project
CSU AI Gateway
A self-service portal that gets university researchers API access to frontier AI models, with hard budget caps and automatic chargeback billing. A vending machine for API keys, minus the surprise invoices.
The problem
Researchers at Colorado State wanted API access to models like Claude and GPT-4 for their work. The official path ran through Azure AI Foundry, which is powerful and also about as approachable as a tax form. The unofficial path was personal API keys on personal cards, which meant unmanaged spend and research data flowing outside the university’s data agreements.
Neither of those is a plan. So we built one.
What I built
From a researcher’s point of view, four steps:
- Request access on a web portal, using your normal university login.
- An admin approves it.
- You get an OpenAI-compatible API key with a hard monthly dollar cap.
- Your usage is metered and billed back to your grant account automatically.
That’s it. No cloud console, no procurement adventure, no P-card.
Under the hood: a FastAPI backend drives LiteLLM as the metering and routing engine, sitting in front of Azure AI Foundry and Azure OpenAI. LiteLLM owns token counting, budget enforcement, and key management. The portal owns approvals, the model catalog, and billing export.
Design decisions I’d defend
- Hard caps, not alerts. A runaway script hits its cap and stops. Nobody discovers a horrifying invoice at the end of the month, which in a university is the difference between a program and a cautionary tale.
- No raw key storage. Keys are shown once at creation. We keep an alias and a masked value, nothing that can leak.
- Enterprise data terms by default. Prompts through the gateway are covered by the university’s agreement with Microsoft, so research data isn’t used to train models. Researchers get that protection without having to read a single terms-of-service page.
- Runtime model management. A new model shows up, an admin prices it, it’s live. No redeploy, no maintenance window.
Why you might care
The real lesson isn’t the plumbing, it’s the pattern: make the sanctioned path easier than the workaround, and compliance happens on its own. People weren’t using personal API keys to be rebels; they were using them because the official route was painful. Fix the pain and governance mostly takes care of itself. Nearly every institutional AI question I get turns out to be some version of this problem.