A recent privilege-escalation vulnerability in LiteLLM has prompted a wave of teams to reevaluate whether running their own LLM gateway is worth the operational cost. CVE-2026-35029, disclosed in early September 2026, allowed a read-only viewer role to escalate to config-modifier privileges. The patch is quick if you already run LiteLLM, but the vulnerability landed at a moment when many teams were already questioning the maintenance burden of self-hosted gateway infrastructure.
Why the Conversation Shifted
LiteLLM was the default open-source LLM gateway from 2024 through mid-2026. It solved the original problem well: translating between different provider APIs behind a single OpenAI-compatible endpoint. But the category has moved. In 2026, teams need MCP passthrough, agent-to-agent authentication, spend caps enforced at the gateway edge rather than after the fact, and a security model that treats the gateway as a trust boundary rather than an internal utility.
The CVE crystallized a latent concern. Running a self-hosted gateway means owning Postgres, Redis, monitoring, patching, secret rotation, and on-call coverage. Estimates for a mature deployment carrying real production traffic land around 15 to 25 engineer-hours per week. That is a real cost that most teams do not budget for when they choose "free" software.
None of this means LiteLLM is poorly built. The maintainers have patched diligently, the open-source model is legitimate, and the community is active. It means the space has widened and the right choice depends on what you actually need.
Five Alternatives,各有不同
Portkey ships both a self-hosted OSS gateway and a managed cloud tier. Both speak OpenAI-compatible wire format. The managed tier adds observability, prompt management, guardrails, and semantic caching. Migration from LiteLLM is straightforward because both use the same request format. The tradeoff is pricing: Portkey's managed tier is priced on request volume, not flat.
Kong AI Gateway is an AI layer added to Kong's existing API gateway. If your team already runs Kong for other services, adding the AI gateway is marginal cost: same admin plane, same routes, same operational model. It is fully self-hosted. The AI-specific features like spend caps and model routing are less mature than the core web-API-gateway functionality, and the setup surface is significant for teams without an existing Kong footprint.
Helicone is primarily observability: logs, cost tracking, and request replay, with proxy-mode routing as a secondary function. Change your base URL and you get a dashboard. It is a clean fit for teams that want visibility into LLM spending without owning the routing layer. It does not provide failover, BYOK to multiple providers behind a single endpoint, or MCP passthrough.
OpenRouter is a managed multi-provider endpoint. One API key gets you access to hundreds of models across all major providers. They hold the keys and handle billing; you pay per token with a small markup on top of provider pricing. It works well for prototyping and individual developers exploring models. The markup adds up on production workloads, and spend caps are alert-based rather than enforced at the edge.
Leanroute takes a different approach: flat monthly pricing ($15 or $25 per account), BYOK to 14 providers, MCP passthrough on the same endpoint, and hard spend caps enforced before the provider invoice grows. It is positioned for startup and mid-market teams running production LLM traffic who want managed infrastructure without the ops burden of self-hosting. The limitation is data residency: teams with strict requirements that no managed vendor can meet need to self-host.
Migration Is Mostly a URL Change
Moving off LiteLLM to any OpenAI-compatible alternative comes down to four steps. Sign up for the alternative and configure provider keys. Change the base URL in your application from the LiteLLM instance to the new endpoint. Send a canary request and verify the response headers confirm routing. Cut over the remaining traffic.
For a straightforward setup, this takes an afternoon. Teams with heavy per-org configuration, custom guardrails, or unusual authentication requirements should budget one to two days.
When Staying on LiteLLM Makes Sense
Two scenarios where the honest answer is to keep what you have. First, if you have a platform engineer who already owns the deployment, patches within hours of CVE announcements, has secret rotation configured, and monitors upstream provider health, you are running it correctly and switching gains you nothing. Second, if you have built custom auth plugins, routing rules, or logging pipelines that would be awkward to reproduce on a managed gateway, you are probably better off staying on a codebase you control.
For everyone else, the question is whether your team actually has the hours to keep owning the operational surface. The teams that get hurt by CVEs like this one are the teams that installed the gateway eighteen months ago, moved on, and never revisited whether they had the capacity to keep it patched. That pattern is not unique to LiteLLM. It is true of any middleware, and the stakes on both sides are quietly getting higher.