LiteLLM
LLM gateway (OpenAI proxy) to manage authentication, load balancing, and spend tracking across 100+ LLMs
Overview
LiteLLM has become one of the default choices for teams that need to standardize LLM access internally without locking into a single provider. Its scale numbers (240M+ Docker pulls, 1B+ requests served) reflect genuine adoption, and being fully open source means small teams can run it for free and get enterprise-grade routing, budgeting, and observability without a subscription.
The tradeoff is that self-hosting means you own the operational burden — upgrades, scaling, and uptime are on you unless you pay for the enterprise plan with support and SLAs. Compared to hosted gateways like Portkey, LiteLLM trades convenience for cost and control; if your team already has infrastructure capacity, that's an easy win, but teams wanting a fully managed experience may prefer a hosted alternative.
Key Features
Unified proxy: OpenAI-compatible interface for 100+ LLM providers behind one API.
Spend tracking: Automatic cost attribution to keys, users, and teams.
Virtual keys and budgets: Issue scoped API keys with per-team or per-user budget limits.
Load balancing and fallbacks: Route traffic across providers and models with automatic failover.
Rate limiting: Control request volume per key or team to prevent runaway costs.
Enterprise deployment: Self-hosted or air-gapped options with SSO, JWT auth, and audit logs.
Pricing
Starting price
Free and open source (self-hosted); Enterprise custom pricing
Open Source (Free): 100+ LLM provider integrations, virtual keys, budgets, teams, load balancing, guardrails, self-hosted via GitHub.
Enterprise: Custom pricing with self-hosted/air-gapped deployment, enterprise support and SLAs, JWT auth, SSO, audit logs, plus a 30-day trial available on request.
Disclaimer: pricing may change, confirm on LiteLLM's own pricing page before buying.
Pros
Genuinely free and open source: Full feature set available with no cost for self-hosted use.
Massive provider coverage: Supports 100+ LLM providers through one consistent API.
Proven at scale: Hundreds of millions of downloads and billions of requests served in production.
OpenAI-compatible: Drop-in replacement that minimizes code changes for teams already using the OpenAI SDK format.
Cons
Self-hosting burden: Free tier requires you to manage deployment, scaling, and uptime yourself.
Enterprise pricing not public: Custom pricing requires a sales conversation, no self-serve enterprise checkout.
Less polish than hosted competitors: UI and onboarding are less refined than fully managed commercial gateways.
What Makes It Unique
Open-source-first gateway: One of the few LLM gateways where the full routing, budgeting, and load-balancing feature set is free and self-hostable, not gated behind a paid tier.
Kay Score
7.4
/ 10
Tool Information
Pricing
Free and open source (self-hosted); Enterprise custom pricing
Category
Coding & Development
Platform
Web / iOS / Android
Last Updated
