Reverse Proxies and API Gateways

The Front Door to Your Infrastructure#

A reverse proxy sits in front of your servers and accepts requests on their behalf. To clients it is the server; behind it there may be one application or fifty services. It is where TLS is terminated, where routing decisions are made, where abusive traffic is stopped, and where the one access log that sees every request is written. An API gateway is a reverse proxy specialized for APIs.

Forward Proxy vs. Reverse Proxy#

The difference is whom the proxy represents:

• A forward proxy acts for clients. Clients are configured to use it, and servers see the proxy's address. Corporate egress filters, caching proxies, and the proxies used in scraping are forward proxies.
• A reverse proxy acts for servers. Clients do not know it exists; DNS simply points at it. NGINX, HAProxy, Envoy, Traefik, Caddy and cloud load balancers all play this role.

A load balancer at Layer 7 is a reverse proxy whose main job is spreading traffic; the terms overlap heavily in practice.

What a Reverse Proxy Does#

1. TLS termination: holds certificates, performs the TLS handshake, and forwards plain HTTP (or re-encrypts, for zero-trust networks) to backends. Certificates live in one place and can be renewed automatically with ACME.
2. Routing: maps hostnames and paths to services: /api/ to the API, /static/ to object storage, everything else to the web app.
3. Protocol translation: speaks HTTP/2 or HTTP/3 to browsers while backends use HTTP/1.1; upgrades connections to WebSockets.
4. Compression and caching: gzip or brotli responses, cache static assets (see HTTP Caching).
5. Protection: rate limits, request size limits, header validation, WAF rules, and buffering that shields slow backends from slow clients.
6. Observability: the canonical access log, request IDs, and latency metrics per route.

A Realistic NGINX Configuration#

# http { } context
limit_req_zone $binary_remote_addr zone=perip:10m rate=10r/s;

upstream api_backend { server 10.0.2.10:8000; server 10.0.2.11:8000; keepalive 16; }
upstream web_backend { server 10.0.3.10:3000; }

server {
    listen 443 ssl;
    http2 on;                                   # older NGINX: "listen 443 ssl http2;"
    server_name example.com;
    ssl_certificate     /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
    ssl_protocols TLSv1.2 TLSv1.3;

    client_max_body_size 10m;                   # reject huge uploads early (413)

    location /api/ {
        limit_req zone=perip burst=20 nodelay;
        limit_req_status 429;
        proxy_pass http://api_backend/;         # trailing slash: /api/users -> /users
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_set_header Host              $host;
        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Request-ID      $request_id;
        proxy_connect_timeout 3s;
        proxy_read_timeout    30s;
    }

    location /ws/ {
        proxy_pass http://web_backend;
        proxy_http_version 1.1;
        proxy_set_header Upgrade    $http_upgrade;   # WebSocket handshake
        proxy_set_header Connection "upgrade";
        proxy_read_timeout 1h;                       # long-lived connections
    }

    location / {
        proxy_pass http://web_backend;          # no URI part: path passed through unchanged
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

The non-obvious lines:

• proxy_pass with or without a URI. proxy_pass http://api_backend/; (with a path, even just /) replaces the matched location prefix, so /api/users reaches the backend as /users. proxy_pass http://web_backend; (no path) forwards the original URI untouched. Mixing these up produces 404s that exist only behind the proxy.
• $proxy_add_x_forwarded_for appends the connecting client's address to any existing X-Forwarded-For header, building the chain discussed below.
• Upgrade and Connection headers are hop-by-hop, so proxies drop them by default. WebSockets need them passed explicitly, plus a long read timeout, or connections die after 60 seconds of silence.
• limit_req implements a leaky-bucket limit per client IP; burst=20 nodelay absorbs short spikes. NGINX returns 503 on rejection unless you set limit_req_status 429.
• Timeouts are explicit because the defaults rarely match your application: a short connect timeout fails fast on dead backends, and the read timeout must exceed your slowest legitimate endpoint.

Validate before reloading: nginx -t && nginx -s reload. A reload keeps existing connections alive, unlike a restart.

X-Forwarded Headers and the Real Client IP#

Behind a proxy, the application's socket sees the proxy's address. Proxies preserve client information in headers:

• X-Forwarded-For: the client address and every proxy hop, comma-separated, leftmost oldest.
• X-Forwarded-Proto: http or https, so the app builds correct redirect URLs and marks cookies Secure.
• X-Forwarded-Host: the original Host.
• Forwarded: the standardized single-header version (RFC 7239), less widely used.

These headers are client-controlled input. Anyone can send X-Forwarded-For: 1.2.3.4, and a proxy that appends to it leaves the forged value in place. An application that takes the leftmost value lets attackers spoof their IP, bypassing IP-based rate limits, lockouts and audit logs. The correct algorithm walks the chain from the right, skipping only proxies you operate, and ignores the header entirely when the request did not come from one of them:

import ipaddress

TRUSTED_PROXIES = [ipaddress.ip_network(n) for n in ("10.0.0.0/8", "127.0.0.1/32")]

def is_trusted(addr: str) -> bool:
    try:
        ip = ipaddress.ip_address(addr.strip())
    except ValueError:
        return False
    return any(ip in net for net in TRUSTED_PROXIES)

def client_ip(remote_addr: str, xff_header: str | None) -> str:
    '''Walk the chain right-to-left, skipping our own proxies.'''
    if not is_trusted(remote_addr):
        return remote_addr                    # not via our proxy: ignore the header
    hops = [h.strip() for h in (xff_header or "").split(",") if h.strip()]
    for hop in reversed(hops):
        if not is_trusted(hop):
            return hop                        # first address our proxies did not add
    return remote_addr

print(client_ip("10.0.0.5", "1.2.3.4, 198.51.100.23"))    # spoofed value + real one
print(client_ip("198.51.100.23", "1.2.3.4"))              # bypassed the proxy
print(client_ip("10.0.0.5", "198.51.100.23, 10.0.3.9"))   # two of our proxies
print("naive:", "1.2.3.4, 198.51.100.23".split(",")[0])

198.51.100.23
198.51.100.23
198.51.100.23
naive: 1.2.3.4

Frameworks ship this logic: Werkzeug's ProxyFix(app.wsgi_app, x_for=1, x_proto=1) trusts exactly the number of hops you configure, and NGINX's realip module does it with set_real_ip_from 10.0.0.0/8; real_ip_header X-Forwarded-For; real_ip_recursive on;. Configure the hop count to match your real topology, and make backends unreachable except through the proxy (security groups, private subnets), otherwise the whole scheme is bypassable.

When Proxies and Backends Disagree#

Two HTTP parsers in a row create a class of bugs that do not exist with one. If the proxy and the backend disagree about where a request ends (conflicting Content-Length and Transfer-Encoding headers, for instance), an attacker can hide a second request inside the first: HTTP request smuggling. Defences: keep proxies updated, have them normalize or reject ambiguous requests, prefer HTTP/2 end to end where possible, and test in a lab such as the PortSwigger Web Security Academy. Similarly, path normalization differences (/api/../admin, encoded slashes) can let requests reach routes the proxy meant to block, so enforce authorization in the application too, not only at the edge.

API Gateways#

An API gateway adds API-specific policy on top of reverse-proxy basics:

• Authentication: validate JWTs, API keys or OAuth tokens once at the edge, and forward a trusted identity header to services.
• Rate limiting and quotas: per API key, per user, per endpoint, often with plan tiers.
• Request validation and transformation: schema checks, size limits, header rewrites, version routing.
• Metering and analytics: usage per consumer, for billing and capacity planning.
• Service discovery: route to instances registered in a registry or Kubernetes.

Rate limiting at a gateway is usually a token bucket: each key earns tokens at a steady rate up to a maximum burst, and each request spends one.

import time

class TokenBucket:
    def __init__(self, rate: float, burst: int):
        self.rate, self.burst = rate, burst
        self.buckets = {}                              # key -> (tokens, last_refill)

    def allow(self, key: str) -> bool:
        now = time.monotonic()
        tokens, last = self.buckets.get(key, (self.burst, now))
        tokens = min(self.burst, tokens + (now - last) * self.rate)   # refill
        allowed = tokens >= 1
        self.buckets[key] = (tokens - 1 if allowed else tokens, now)
        return allowed

limiter = TokenBucket(rate=5, burst=10)               # 5 req/s sustained, bursts of 10
results = [limiter.allow("api-key-123") for _ in range(15)]
print(results.count(True), "allowed,", results.count(False), "rejected with 429")

10 allowed, 5 rejected with 429

With several gateway instances, this state must live in a shared store such as Redis (or be approximated per instance), otherwise each instance grants the full quota. Rejections should return 429 with a Retry-After header so well-behaved clients back off. Kong, Tyk, AWS API Gateway, Apigee and Envoy-based gateways package these features; see API Security for which controls matter most.

Gateway, Load Balancer or Service Mesh?#

Many deployments use all of them: a cloud L4 balancer in front of an ingress proxy that doubles as a gateway, with a mesh inside the cluster. Avoid putting business logic in the gateway; it should enforce policy, not implement features.

Debugging Through a Proxy#

• 502 Bad Gateway: the proxy could not get a valid response: backend down, wrong port, TLS mismatch between proxy and backend, or the backend closed a kept-alive connection. The proxy's error log names the cause (connect() failed (111: Connection refused), upstream prematurely closed connection).
• 504 Gateway Timeout: the backend accepted the request but did not answer within proxy_read_timeout.
• 413 / 431: body or headers exceeded the proxy's limits, often an oversized cookie.
• Redirect loops to http://: the app does not see X-Forwarded-Proto: https.
• Correlate logs with a request ID generated at the edge and passed to every service.

Bypass the proxy to isolate the fault: curl -v http://10.0.2.10:8000/users from the proxy host tells you immediately whether the backend or the proxy is misbehaving.

Practice#

Run a small Flask or FastAPI app on port 8000 that returns request.remote_addr and the X-Forwarded-For header, put NGINX in front with the /api/ block above, and send curl -k -H 'X-Forwarded-For: 1.2.3.4' https://localhost/api/. Observe the spoofed value in the chain, then apply ProxyFix or the client_ip function and confirm the app reports your real address.