LLM API calls fail differently from typical HTTP requests. They time out, hit rate limits, get caught in provider outages, or hang indefinitely. Most developers handle these cases by writing retry logic, timeout handling, and fallback paths once and then rewriting them for every new project. VernLLM is a TypeScript framework that wraps existing provider clients with configurable resilience, observability, and control as defaults rather than afterthoughts.
Why LLM calls need different handling
Standard SDK calls follow predictable failure patterns. An LLM call can stall for seconds before returning a partial response. A provider can rate-limit one request while accepting the next. An outage can affect one region while leaving others functional. These scenarios require retries with backoff, circuit breaking to stop cascading failures, provider fallback to route around outages, and rate limiting to stay within per-minute ceilings.
VernLLM handles all of these as configurable options on a single class. You keep your existing provider client, OpenAI, Anthropic, or whatever you use, and wrap it rather than reimplementing the resilience layer. The framework runs in your own process, which means no extra network hop, no proxy to maintain, and no latency added by routing traffic through an external gateway.
Small interfaces over monolithic configuration
The framework uses small, swappable interfaces rather than one large configuration object. Caching, rate limiting, and circuit breaking each have their own adapter pattern. For caching, VernLLM accepts any adapter implementing get and delete is optional, so Redis, a database, or a custom store can replace the built-in in-memory cache without changing how you call the LLM.
Middleware provides two hooks. A transform function edits or redacts an outgoing request before it is sent, useful for sanitizing inputs or injecting metadata. A wrap function runs around an entire logical call, including retries and fallback attempts, for logging, tracing, or cost tracking. These are composable, not mutually exclusive.
TypeScript-first with zero runtime dependencies
VernLLM is written in TypeScript from the ground up. Structured output schemas, call parameters, and errors are all typed, so mistakes surface at compile time. The framework does not bundle provider SDKs or schema validators like Zod. It relies on compatible interfaces, so you bring your own clients and validators while keeping the dependency tree minimal.
Retry budgets cap how much of recent traffic can be retries, preventing retry storms from overwhelming a provider. Timeout settings prevent individual attempts from hanging indefinitely. Circuit breakers stop repeated failures from cascading across your application. Default values for parameters like max tokens and reasoning effort apply to any call that omits its own, reducing boilerplate across your codebase.
In-process versus gateway
VernLLM runs inside your application process. That gives it access to your application logic, not just the request and response flowing over the wire. It can react to internal state, catch failures that a gateway watching traffic from outside would miss, and make routing decisions based on context that never leaves your process.
A shared gateway remains the better choice when you need one configuration across many services or languages. VernLLM targets the case where resilience logic lives close to the call itself, where the team maintaining the code is the same team that owns the application, and where adding another infrastructure component is not worth the overhead.
The framework installs through npm, pnpm, yarn, or bun. It wraps existing clients instead of replacing them, which means adoption does not require ripping out your current provider integrations. The value proposition is straightforward: configurable resilience and observability around calls you are already making, with the control living in your codebase rather than in external infrastructure.