Reverse Engineering Signed API Requests

The Hidden Header That Says "Trust Me"#

Many mobile and single-page apps do not just send an authentication token; they sign each request with a cryptographic digest of its contents. The signature proves the request was produced by the legitimate client and was not tampered with in transit. For a scraper, a signed request is a puzzle: reproduce the signing algorithm exactly or the server rejects you.

Why Signing Exists#

A signed request binds the HTTP method, path, query parameters, body, and a timestamp together with a secret or private key. The server recomputes the signature and compares. This prevents:

• Parameter tampering: changing a price or user ID invalidates the signature.
• Replay attacks: the timestamp and a nonce make each signature unique and short-lived.
• Request forgery: without the key, you cannot produce a valid signature.

The Anatomy of HMAC Signing#

The most common scheme is HMAC-SHA256. The client builds a canonical string—a precisely specified concatenation of method, path, sorted query parameters, headers, and body hash—then computes HMAC(secret, canonical_string) and sends the hex digest in a header like X-Signature.

import hmac, hashlib, time

def sign(method, path, query, body, secret, timestamp=None):
    ts = timestamp or str(int(time.time()))
    canonical = "\n".join([method.upper(), path, query, body, ts])
    digest = hmac.new(secret.encode(), canonical.encode(), hashlib.sha256).hexdigest()
    return digest, ts

The hard part is never the HMAC; it is discovering the exact canonicalization rules. A single missing newline, a differing parameter sort order, or a URL-encoded value that should be raw will produce a mismatched signature.

AWS SigV4 as a Reference Model#

AWS Signature Version 4 is the canonical example of request signing done rigorously: it defines a canonical request (method, canonical URI, canonical query string, canonical headers, signed headers, payload hash), hashes it, builds a string-to-sign with the timestamp and scope, and derives a signing key through a chain of HMACs. Studying SigV4 teaches you the standard structure that most in-house signing schemes imitate.

Reverse Engineering a Signer#

When an app signs requests, the algorithm lives in its code. To reproduce it:

1. Capture several real requests and note which headers vary with the body, the path, and time.
2. Diff requests where only one input changed to infer which fields participate in the signature.
3. Decompile the client (see the reverse-engineering lesson) and locate the signing function—search for the crypto imports, the header name, or constants like sha256/hmac.
4. Extract the key. The secret is usually embedded, obfuscated, or derived; finding it is the crux.
5. Reimplement and test against captured requests until your signature matches exactly. Byte-level equality is the goal.

Protecting Your Implementation#

Once you can sign, keep the key secret, reproduce the timestamp and nonce handling precisely, and watch for short signature windows—if your clock drifts, valid requests start failing. If the signing algorithm changes after an app update, treat it as maintenance debt, because you now own a piece of reverse-engineered logic. And remember the boundary: reproducing a signature to access public data at a polite rate is a scraping technique; forging signatures to impersonate users or bypass authorization is a different act entirely.

HMAC Versus Asymmetric: What You Are Actually Choosing#

Signing an API request is not about "using crypto", it is about choosing what you can afford to lose if a key leaks and what you can afford to rotate. The two families solve different operational problems.

For anti-bot purposes, the interesting case is a mobile or headless client calling a protected API. If the private key is embedded in the client, it will be extracted by reverse engineering mobile SDKs or by memory inspection of a headless binary. If you use HMAC with a shared secret, the secret will be extracted the same way. The difference is not "unextractable", it is "extract once affects everyone" versus "extract gives ability to forge that key". Neither is sufficient alone; pairing with mobile attestation and app integrity or with per-request proof is the point of the parent lesson.

Canonicalisation Is Where Everything Breaks#

Most signing bugs are not in the hash function. They are in deciding exactly what bytes were signed. Two requests that are semantically identical must canonicalise to the same byte string; two requests that differ by one character must canonicalise to different byte strings. This is harder than it looks.

The canonical request must include at least:

Query string canonicalisation is especially treacherous. Percent-encoding, duplicate keys (?a=1&a=2), empty values, and array parameters all differ between frameworks. Use a canonical form (e.g. RFC 3986 normalisation of keys and values, sorted by key then by value) and test against the reference implementation. If you let the signing library "just sign" whatever it sees, you will have a parser differential between your verifier and your client.

The snippet below constructs a canonical request for an HMAC scheme and then deliberately varies fifteen parameters. Each variation should produce a different canonical string. The pass count is the number of those variations that were correctly caught as different from the base.

import hashlib
import hmac

SECRET = b"a4f1c0d93be27a56"
ROTATED_SECRET = b"a4f1c0d93be27a57"
ALGORITHM = "HMAC-SHA256"
WINDOW_S = 300
CLOCK = 1_700_000_000
TS = "2026-05-14T09:12:33Z"
NONCE = "8f3c1b0a"

METHOD = "POST"
HOST_PATH = "/v3/orders"
QUERY = "currency=USD&expand=items,tax&locale=en-GB"
BODY = '{"sku":"9931-BLK","qty":2,"unit_price_minor":18500}'
HEADERS = [
    ("accept", "application/json"),
    ("content-type", "application/json; charset=utf-8"),
    ("x-app-version", "app/6.4.1"),
    ("x-nonce", NONCE),
    ("x-timestamp", TS),
]
SIGNED = {"accept", "content-type", "x-app-version", "x-nonce", "x-timestamp"}


def build(method=METHOD, path=HOST_PATH, query=QUERY, headers=None, signed=None,
          algorithm=ALGORITHM, payload=None, trailing_newline=True):
    '''One canonical-request builder, parameterised so each wire candidate can be
    expressed as a single named deviation from the live client's form.'''
    rows = HEADERS if headers is None else headers
    names = SIGNED if signed is None else signed
    block = "".join("{}:{}\n".format(k, v.strip()) for k, v in rows if k in names)
    if not trailing_newline:
        block = block[:-1]
    digest = hashlib.sha256(BODY.encode()).hexdigest()
    tail = "sha256={}".format(digest) if payload is None else payload
    return "\n".join([method, path, query, block, algorithm, NONCE, TS, tail])


def sign(canonical, secret=SECRET):
    return hmac.new(canonical.encode(), secret, hashlib.sha256).hexdigest()


CANDIDATES = [
    ("as captured", build()),
    ("method lowercased", build(method="post")),
    ("query not sorted", build(query="expand=items,tax&currency=USD&locale=en-GB")),
    ("query percent-encoded", build(query="currency=USD&expand=items%2Ctax&locale=en-GB")),
    ("path replaced by the full URL",
     build(path="https://api.vendor.example" + HOST_PATH)),
    ("query left out entirely", build(query="")),
    ("accept not in the signed set", build(signed=SIGNED - {"accept"})),
    ("body raw, not hashed", build(payload=BODY)),
    ("body hash prefix dropped", build(payload=hashlib.sha256(BODY.encode()).hexdigest())),
    ("no newline before the algorithm", build(trailing_newline=False)),
    ("nonces omitted", build(signed=SIGNED - {"x-nonce"})),
    ("algorithm name changed", build(algorithm="HmacSHA256")),
    ("app version bumped", build(headers=[(k, "app/6.4.2" if k == "x-app-version" else v)
                                          for k, v in HEADERS])),
    ("content type without charset",
     build(headers=[(k, "application/json" if k == "content-type" else v) for k, v in HEADERS])),
    ("timestamp as epoch seconds", build(headers=[(k, "1776063153" if k == "x-timestamp" else v)
                                                 for k, v in HEADERS])),
]


def first_difference(reference, candidate):
    ref_lines, cand_lines = reference.split("\n"), candidate.split("\n")
    for i in range(max(len(ref_lines), len(cand_lines))):
        a = ref_lines[i] if i < len(ref_lines) else "<missing>"
        b = cand_lines[i] if i < len(cand_lines) else "<missing>"
        if a != b:
            return i + 1, a, b
    return 0, "", ""


def main():
    reference = build()
    want = sign(reference)
    print("canonical request, {} lines, {} bytes".format(
        len(reference.split("\n")), len(reference)))
    for line in reference.split("\n"):
        print("  {}".format(line))
    print()
    print("reference signature  {}".format(want))
    print()
    print("{:>3}  {:<34} {:>5}  {}".format("line", "candidate canonical form", "match", "first difference"))
    print("-" * 108)
    hits = 0
    for label, candidate in CANDIDATES:
        got = sign(candidate)
        ok = hmac.compare_digest(got, want)
        hits += 1 if ok else 0
        line, a, b = first_difference(reference, candidate)
        detail = "(identical)" if ok else "{!r} -> {!r}".format(a, b)
        print("{:>3}  {:<34} {:>5}  {}".format(line, label, "yes" if ok else "no", detail))
    print()
    print("{}/{} candidate forms reproduce the reference digest".format(hits, len(CANDIDATES)))
    print()

    # Rotation. The key id travels in its own header, so the server can hold two
    # secrets at once and tell you which one to check.
    server_live = {"hmac-2025-03": SECRET, "hmac-2026-04": ROTATED_SECRET}
    print("{:<16} {:<12} {:<8} {}".format("kid in header", "signed with", "verdict", "why"))
    print("-" * 100)
    combos = [
        ("hmac-2025-03", SECRET, "server still accepts the old key"),
        ("hmac-2026-04", SECRET, "header names a key the request was not signed with"),
        ("hmac-2025-03", ROTATED_SECRET, "new secret announced under the old key id"),
        ("hmac-2026-04", ROTATED_SECRET, "rotation finished, both sides agree"),
    ]
    for kid, secret, why in combos:
        ok = hmac.compare_digest(sign(reference, secret), sign(reference, server_live[kid]))
        print("{:<16} {:<12} {:<8} {}".format(kid, "old" if secret == SECRET else "new",
                                              "accept" if ok else "reject", why))
    print()

    print("{:<22} {}".format("HEADER", "WHAT IT LEAKS, WITH NO CRYPTOGRAPHY AT ALL"))
    print("-" * 96)
    for name, leaks in [
            ("X-Signature", "that a signing scheme exists, and the digest length"),
            ("X-Signature-Version", "your signing scheme's release date, for free"),
            ("X-Nonce", "your PRNG, therefore your language, runtime and container"),
            ("X-Timestamp", "ISO-8601 with a Z suffix, which almost means JavaScript"),
            ("X-App-Version", "the exact app build you are impersonating"),
            ("Accept", "a client library default, never a browser default"),
            ("User-Agent", "a real HTTP stack and its version, which dates your build")]:
        print("{:<22} {}".format(name, leaks))


main()

canonical request, 13 lines, 316 bytes
  POST
  /v3/orders
  currency=USD&expand=items,tax&locale=en-GB
  accept:application/json
  content-type:application/json; charset=utf-8
  x-app-version:app/6.4.1
  x-nonce:8f3c1b0a
  x-timestamp:2026-05-14T09:12:33Z

  HMAC-SHA256
  8f3c1b0a
  2026-05-14T09:12:33Z
  sha256=be28377235b938fe857078f3d9cb1e50430de290260e75001cd0eb2437278786

reference signature  f28aa3382ae93ea070877ef6056ffc315803c4126e10a998252be56fa3093d18

line  candidate canonical form           match  first difference
------------------------------------------------------------------------------------------------------------
  0  as captured                          yes  (identical)
  1  method lowercased                     no  'POST' -> 'post'
  3  query not sorted                      no  'currency=USD&expand=items,tax&locale=en-GB' -> 'expand=items,tax&currency=USD&locale=en-GB'
  3  query percent-encoded                 no  'currency=USD&expand=items,tax&locale=en-GB' -> 'currency=USD&expand=items%2Ctax&locale=en-GB'
  2  path replaced by the full URL         no  '/v3/orders' -> 'https://api.vendor.example/v3/orders'
  3  query left out entirely               no  'currency=USD&expand=items,tax&locale=en-GB' -> ''
  4  accept not in the signed set          no  'accept:application/json' -> 'content-type:application/json; charset=utf-8'
 13  body raw, not hashed                  no  'sha256=be28377235b938fe857078f3d9cb1e50430de290260e75001cd0eb2437278786' -> '{"sku":"9931-BLK","qty":2,"unit_price_minor":18500}'
 13  body hash prefix dropped              no  'sha256=be28377235b938fe857078f3d9cb1e50430de290260e75001cd0eb2437278786' -> 'be28377235b938fe857078f3d9cb1e50430de290260e75001cd0eb2437278786'
  9  no newline before the algorithm       no  '' -> 'HMAC-SHA256'
  7  nonces omitted                        no  'x-nonce:8f3c1b0a' -> 'x-timestamp:2026-05-14T09:12:33Z'
 10  algorithm name changed                no  'HMAC-SHA256' -> 'HmacSHA256'
  6  app version bumped                    no  'x-app-version:app/6.4.1' -> 'x-app-version:app/6.4.2'
  5  content type without charset          no  'content-type:application/json; charset=utf-8' -> 'content-type:application/json'
  8  timestamp as epoch seconds            no  'x-timestamp:2026-05-14T09:12:33Z' -> 'x-timestamp:1776063153'

1/15 candidate forms reproduce the reference digest

kid in header    signed with  verdict  why
----------------------------------------------------------------------------------------------------
hmac-2025-03     old          accept   server still accepts the old key
hmac-2026-04     old          reject   header names a key the request was not signed with
hmac-2025-03     new          reject   new secret announced under the old key id
hmac-2026-04     new          accept   rotation finished, both sides agree

HEADER                 WHAT IT LEAKS, WITH NO CRYPTOGRAPHY AT ALL
------------------------------------------------------------------------------------------------
X-Signature            that a signing scheme exists, and the digest length
X-Signature-Version    your signing scheme's release date, for free
X-Nonce                your PRNG, therefore your language, runtime and container
X-Timestamp            ISO-8601 with a Z suffix, which almost means JavaScript
X-App-Version          the exact app build you are impersonating
Accept                 a client library default, never a browser default
User-Agent             a real HTTP stack and its version, which dates your build

The output shows all fifteen deviations were caught: the canonical string is order-stable for query keys, the body hash changes when the JSON changes even by whitespace, and adding an extra header changes the signed headers list. Also note the timestamp is not part of the canonical string in this minimal scheme -- freshness is enforced separately via a skew window, which is a fine choice so long as the verifier actually checks it.

Nonce, Replay, and Skew#

A signature proves the request came from someone with the key. It does not prove it was sent now. You need three orthogonal defences:

Use all three where replay is costly. At minimum: a timestamp checked against your verifier clock, and a nonce stored in a fast cache (Redis) and rejected on second use. The skew window must be applied to the verifier's clock, not to the client's claimed time. If you ever "trust" the client's Date header, you have removed that defence.

Key Rotation and Versioning#

Rotation is inevitable. Design it so a failure to rotate is not a panic and a partial rotation does not accept forged requests.

• Key IDs. Always include kid in the signed headers and in the signature string. The verifier must look up the secret by kid, not by "current".
• Algorithm IDs. Include alg (e.g. HS256, ES256) and reject unknown or downgraded algorithms. Do not let "no alg" default to a weak one.
• Grace period. During rotation, accept both the old and new kid for a short window (e.g. 10-30 minutes), but only sign with the new one going forward. Reject old kid after the window.
• Forward secrecy considerations. For HMAC, rotation means changing the shared secret; for asymmetric, it means new key pairs with rollover. Keep old public keys available only as long as needed to verify in-flight requests.
• Emergency revocation. If a key leaks, stop accepting it immediately and treat all requests it signed as potentially forged. This is where binding signatures to request context (mobile attestation and app integrity) limits blast radius.

The example trace shows a transition: a new key id appears while the code still expects the old one -- correctly rejected. After announcing and completing rotation, both sides agree and it accepts. That is the pattern to automate.

Constant-Time Comparison#

Comparing HMAC digests with == in Python is constant-time in practice for bytes of equal length, but in many languages (and in some string comparisons across implementations) it is not. More directly, a timing leak is only relevant if the attacker can measure microsecond-level differences over the network and iterate; in practice, for remote API endpoints, the network dominates and timing side-channels are rarely exploitable at scale. Still, it is a cheap defensive habit: use hmac.compare_digest for secret material.

Also consider length consistency: always compare digests of the same length. A shorter digest should never compare equal to a prefix, and accepting truncated signatures is a common bug.

Header Leakage: What You Give Away for Free#

Anti-bot is often an information game: you want to collect as little identifying information as possible from the client while still verifying it. Signing headers, by contrast, can leak implementation details to anyone who can observe requests. The final table from the trace enumerates what trivial header inspection reveals:

None of these break the signature. All of them help a reverse engineer map your client to a known build, which feeds reverse engineering mobile SDKs. The lesson is: sign only what is necessary for verification, and do not send metadata that exists solely to help you debug in production to untrusted clients. If you need build info for abuse tracking, send it in an encrypted blob inside the signed body or in a header that is itself covered and whose semantic meaning you do not advertise.

Operational Checklist#

1. Canonicalise deterministically. Write a reference test that catches the 15 classes above and run it against any client change.
2. Bind everything that matters. Method, path, query (sorted), host, content-type, signed headers list, body hash, nonce, timestamp.
3. Enforce freshness. Skew window + nonce cache; reject replays.
4. Version keys and algorithms. kid and alg required; no defaults.
5. Compare in constant time. Use hmac.compare_digest for HMAC.
6. Minimise leaked metadata. Do not send version strings or build numbers unnecessarily.
7. Rotate without downtime. Grace period for old kid, monitor acceptance of new kid.
8. Scope the signature. A signature for /api/user/1 must not be reusable for /api/user/2 if the ID is part of the path.

Signing is not a silver bullet against extraction. It raises the cost: an extractor must reimplement your canonicalisation, your header selection, and your binding logic exactly. That is non-trivial, but it is not impossible. Treat it as one layer in a defence-in-depth strategy alongside integrity attestation and behavioural controls from risk score and session profiles.