The Cryptographic Giveaway#

When you write a scraper using requests or axios, you likely spoof the User-Agent header to pretend you are Google Chrome. However, before that User-Agent header is ever transmitted, your script must negotiate a secure HTTPS connection via the TLS Handshake.

This handshake is where 90% of basic bots are caught, thanks to JA3 Fingerprinting.

What is JA3?#

During the TLS ClientHello packet, the client tells the server which cryptographic protocols it supports. Specifically, it sends: - The TLS Version - A list of supported Cipher Suites - A list of TLS Extensions - Supported Elliptic Curves

A standard Python environment (using OpenSSL) sends a very specific list of ciphers in a very specific order. A real Chrome browser sends a completely different list of ciphers, in a different order, utilizing modern extensions like GREASE (random reserved values designed to break middleboxes).

Cloudflare takes these handshake parameters and creates an MD5 hash—the JA3 Fingerprint. If your HTTP headers claim you are Chrome, but your JA3 fingerprint matches the known signature for Python urllib, the WAF instantly drops your connection with a 403 Forbidden.

The Successor: JA4#

JA3 concatenates the raw TLS parameters and hashes them, which made it trivially spoofable by reordering ciphers. The newer JA4 fingerprint captures more discriminating information: it hashes the full ClientHello structure, including the order of extensions, the ALPN protocol lists, and the supported groups, broken into a human-readable string like t13d1516h2_8daaf6152771.... JA4 is significantly harder to forge by accident because there are far more independent ordering dimensions that must all match a real browser.

Case Study: Why the Order Matters#

OpenSSL and BoringSSL (the library inside Chrome) negotiate connections differently. Even when you list identical cipher suites, the order in which they are presented, the padding, the presence or absence of the padding extension, and the supported signature algorithms all differ. Breaking any one of those details produces a unique JA4 that does not match any shipped browser, and a WAF with a modern TLS fingerprint database will spot it immediately.

HTTP/2 Pseudo-Headers#

If you pass the TLS layer, modern WAFs inspect your HTTP/2 frames. HTTP/2 uses "pseudo-headers" (:method, :authority, :scheme, :path). - Chrome always sends them in the exact order: :method, :authority, :scheme, :path. - Firefox sends them in a slightly different order. - Standard HTTP libraries often scramble the order entirely.

Again, a mismatch between your stated User-Agent and your HTTP/2 frame structure results in an instant ban.

How to Spoof TLS in Python#

You cannot fix this by simply changing headers in requests. You must fundamentally alter how the underlying C/Rust socket libraries negotiate the connection.

The modern standard for Python scraping is curl_cffi. It uses a compiled backend (borrowed from the Chrome source code) to flawlessly imitate the exact TLS handshakes and HTTP/2 behaviors of real browsers.

from curl_cffi import requests

# Notice we don't just pass a User-Agent string.
# We explicitly tell the library to impersonate the entire network stack of Chrome 120.
response = requests.get(
    "https://nowsecure.nl", # A Cloudflare test site
    impersonate="chrome120"
)

print(response.status_code) # 200 OK

curl_cffi ships a small catalogue of impersonate targets (Chrome, Safari, Firefox, and the current Edge revisions). Picking the newest browser version in the catalogue and re-pinning it when the WAF vendors update their detection is a maintenance task of its own.

The Limits of Faking#

Impersonating a browser's TLS stack only gets you past the network layer. Every physical Chrome version has a fixed set of TLS behaviors; the WAF knows exactly what each version should look like. If you impersonate Chrome 120 while your User-Agent says Chrome 122, the version mismatch is detectable. Always keep your advertised browser version synchronized with the TLS stack you are actually impersonating.

By perfectly aligning your TLS signature, HTTP/2 frames, and User-Agent, you become statistically invisible to network-layer WAFs.

What the ClientHello Commits To#

Before any certificate is validated, the client sends one message that fixes almost everything a server can later complain about: which protocol versions it will speak, which cipher suites it will accept, which extensions it will honour and in what order, which groups it will negotiate key shares from, and which application protocols it wants. That message is the ClientHello, and it is the entire surface area for TLS-layer detection.

The important structural detail is that the extension list is ordered, and order is observable in the raw bytes. A server does not need to interpret any of the extensions to know that yours arrived in a sequence no shipping browser produces. This is why patching the user agent string is worth almost nothing here: the ClientHello is emitted by the TLS stack, long before and independently of anything HTTP-layer code touches.

# JA3 and JA4 both hash what the client offered, and both are forgeable.
# What is not forgeable is being plausible: a linter for ClientHello
# descriptions, so you can check a stack before a vendor checks it for you.

import hashlib

GREASE = {0x0a0a, 0x1a1a, 0x2a2a, 0x3a3a, 0x4a4a, 0x5a5a, 0x6a6a, 0x7a7a,
          0x8a8a, 0x9a9a, 0xaaaa, 0xbaba, 0xcaca, 0xdada, 0xeaea, 0xfafa}

NAME = {
    0x0000: "server_name", 0x0004: "compress_certificate",
    0x0005: "status_request", 0x000a: "supported_groups",
    0x000b: "ec_point_formats", 0x000d: "signature_algorithms",
    0x000f: "psk_key_exchange_modes",
    0x0010: "application_layer_protocol_negotiation",
    0x0012: "signed_certificate_timestamp", 0x0015: "padding",
    0x0016: "encrypt_then_mac", 0x0017: "extended_master_secret",
    0x001d: "key_share", 0x0023: "session_ticket", 0x002b: "supported_versions",
    0x002d: "psk_key_exchange_modes", 0x002f: "pre_shared_key",
    0x0033: "key_share", 0xff01: "renegotiation_info",
}

# plausible Chrome 131 offered order, with one GREASE value in each list
CHROME = {
    "version": 771, "sni": True, "alpn": ["h2", "http/1.1"],
    "ciphers": [0x0a0a, 0x1301, 0x1302, 0x1303, 0xc02b, 0xc02f, 0xc02c,
                0xc030, 0xcca9, 0xcca8, 0xc013, 0xc014, 0x009c, 0x009d,
                0x002f, 0x0035],
    "extensions": [0x0a0a, 0x0000, 0x0017, 0xff01, 0x000a, 0x000b, 0x0010,
                   0x000d, 0x0012, 0x002b, 0x0023, 0x0033, 0x0005, 0x000f],
    "groups": [0x0a0a, 0x001d, 0x0017, 0x0018, 0x0019, 0x001a, 0x001b,
               0x001c, 0x001e, 0x001f, 0x0020, 0x0021, 0x0022, 0x0023,
               0x0024, 0x0025, 0x0026, 0x0029, 0x002a, 0x002b, 0x002c],
    "formats": [0x0a0a, 0x0000],
    "sigalgs": [0x0804, 0x0403, 0x0503, 0x0603, 0x0805, 0x0806, 0x0401],
}

FIREFOX = {
    "version": 771, "sni": True, "alpn": ["h2", "http/1.1"],
    "ciphers": [0x0a0a, 0x1301, 0x1303, 0x1302, 0xc02b, 0xc02f, 0xcca9,
                0xcca8, 0xc02c, 0xc030, 0xc00a, 0xc009, 0xc013, 0xc014,
                0x009c, 0x009d, 0x002f, 0x0035],
    "extensions": [0x0a0a, 0x0000, 0x0017, 0xff01, 0x000a, 0x000b, 0x0010,
                   0x000d, 0x0012, 0x002b, 0x0033, 0x0016, 0x0005, 0x0023,
                   0x000f],
    "groups": [0x0a0a, 0x001d, 0x0017, 0x0018, 0x0019, 0x001a, 0x001b,
               0x001e, 0x001f, 0x0021, 0x0022, 0x0023, 0x0024, 0x0025,
               0x0026],
    "formats": [0x0a0a, 0x0000],
    "sigalgs": [0x0503, 0x0603, 0x0804, 0x0805, 0x0806, 0x0401, 0x0403],
}

OPENSSL = {
    "version": 771, "sni": True, "alpn": ["http/1.1"],
    "ciphers": [0x002f, 0x0035, 0x009c, 0x009d, 0x1301, 0x1302, 0x1303,
                0xc009, 0xc00a, 0xc013, 0xc014, 0xc02b, 0xc02c, 0xc02f,
                0xc030, 0xcca8, 0xcca9],
    "extensions": [0x0000, 0x000a, 0x000b, 0x000d, 0x0010, 0x0012, 0x0017,
                   0x0023, 0x002b, 0x002f, 0x0033, 0xff01, 0x0015, 0x0016],
    "groups": [0x001d, 0x0017, 0x0018, 0x0019, 0x001a, 0x001b],
    "formats": [0x0000],
    "sigalgs": [0x0403, 0x0503, 0x0603, 0x0804, 0x0201],
}

GO = {
    "version": 771, "sni": True, "alpn": ["h2", "http/1.1"],
    "ciphers": [0x1301, 0x1302, 0x1303, 0xc02c, 0xc02b, 0xc030, 0xc02f,
                0xcca9, 0xcca8, 0x009c, 0x009d, 0x002f, 0x0035],
    "extensions": [0x0000, 0x000a, 0x000b, 0x000d, 0x0010, 0x0012, 0x0017,
                   0x002b, 0x0033, 0x0023, 0x0016, 0xff01, 0x0005],
    "groups": [0x001d, 0x0017, 0x0018, 0x0019, 0x001a, 0x001b, 0x001e,
               0x001f, 0x0020, 0x0021, 0x0022, 0x0023, 0x0024, 0x0025],
    "formats": [0x0000],
    "sigalgs": [0x0403, 0x0804, 0x0401, 0x0503, 0x0603, 0x0203],
}

PLAUSIBLE_GROUPS = {"chrome-131": 20, "firefox-133": 14}
NEEDS_GREASE = {"chrome-131", "firefox-133"}


def strip_grease(values):
    return [v for v in values if v not in GREASE]


def has_grease(values):
    return any(v in GREASE for v in values)


def ja3(p):
    blob = ",".join([
        str(p["version"]),
        ",".join(str(c) for c in strip_grease(p["ciphers"])),
        ",".join(str(e) for e in strip_grease(p["extensions"])),
        ",".join(str(g) for g in strip_grease(p["groups"])),
        ",".join(str(f) for f in strip_grease(p["formats"])),
    ])
    return hashlib.md5(blob.encode()).hexdigest()


def ja4_parts(p):
    clean_c = strip_grease(p["ciphers"])
    clean_e = strip_grease(p["extensions"])
    ciphers = "".join("%04x" % c for c in clean_c)
    a = "t%02x%02x%02x%02x%02x" % (
        p["version"], 1 if p["sni"] else 0,
        len(clean_c), len(clean_e), len(p["alpn"]))
    b = ciphers + "_" + ciphers if len(clean_c) == 1 else ciphers[:4] + "_" + ciphers[-4:]
    # order matters, so extension order is preserved; SNI and ALPN are excluded
    kept = [e for e in clean_e if e not in (0x0000, 0x0010)]
    ext_blob = ",".join("%04x" % e for e in kept)
    sig_blob = ",".join("%04x" % s for s in p["sigalgs"])
    return {"a": a, "b": b,
            "c_ext": hashlib.sha256(ext_blob.encode()).hexdigest()[:12],
            "c_sig": hashlib.sha256(sig_blob.encode()).hexdigest()[:12]}


def ja4(p):
    parts = ja4_parts(p)
    return "_".join([parts["a"], parts["b"], parts["c_ext"], parts["c_sig"]])


def lint(name, p, claims=None):
    # claims is the engine the client says it is, which is not always the
    # name of the descriptor being linted: a forged chrome is linted as chrome.
    claims = claims or name
    out = []
    clean_c = strip_grease(p["ciphers"])
    clean_e = strip_grease(p["extensions"])
    clean_g = strip_grease(p["groups"])
    if claims in NEEDS_GREASE and not has_grease(p["ciphers"]):
        out.append("no GREASE in a client claiming a browser engine")
    if claims in NEEDS_GREASE and not has_grease(p["extensions"]):
        out.append("no GREASE in extensions either")
    if claims in PLAUSIBLE_GROUPS and len(clean_g) != PLAUSIBLE_GROUPS[claims]:
        out.append("group count %d, %s sends about %d"
                   % (len(clean_g), claims, PLAUSIBLE_GROUPS[claims]))
    if "h2" not in p["alpn"]:
        out.append("no h2 in ALPN: not a browser, whatever the UA claims")
    if len(clean_c) > 4 and clean_c == sorted(clean_c):
        out.append("cipher list sorted ascending: an artefact of forging")
    if len(set(clean_e)) != len(clean_e):
        out.append("duplicate extension offered")
    return out


def main():
    clients = [("chrome-131", CHROME), ("firefox-133", FIREFOX),
               ("openssl-3.3", OPENSSL), ("go-1.23", GO)]

    print("chrome-131 extension sequence, in the order it is offered")
    names = [NAME.get(e, "grease") for e in CHROME["extensions"]]
    print("  " + " -> ".join(names[:7]))
    print("  " + " -> ".join(names[7:]))
    print("  order is signal: ja3 ignores it, ja4 hashes it, bytes keep it")
    print()

    print("ja3 and ja4 for four honest client descriptions")
    print(f"{'client':<14} {'ja3':<34} {'ja4'}")
    hashes = {}
    for name, p in clients:
        hashes[name] = ja3(p)
        print(f"{name:<14} {hashes[name]:<34} {ja4(p)}")
    print(f"  {len(set(hashes.values()))} distinct ja3 values from 4 clients")
    print()

    forged = {"version": CHROME["version"], "sni": CHROME["sni"],
              "alpn": CHROME["alpn"], "sigalgs": CHROME["sigalgs"],
              "ciphers": sorted(strip_grease(CHROME["ciphers"])),
              "extensions": sorted(strip_grease(CHROME["extensions"])),
              "groups": sorted(strip_grease(CHROME["groups"])),
              "formats": sorted(strip_grease(CHROME["formats"]))}

    print("the forgery everybody writes: chrome's list, sorted, GREASE stripped")
    print(f"{'client':<14} {'ja3':<34} {'ja4'}")
    print(f"{'chrome-131':<14} {ja3(CHROME):<34} {ja4(CHROME)}")
    print(f"{'chrome-forged':<14} {ja3(forged):<34} {ja4(forged)}")
    print()
    print("  ja3 changed, so a list of known hashes stops matching, which is")
    print("  the point of the exercise and also the limit of it.")
    print()
    print("  ja4, component by component:")
    real_parts = ja4_parts(CHROME)
    fake_parts = ja4_parts(forged)
    for key in ["a", "b", "c_ext", "c_sig"]:
        same = real_parts[key] == fake_parts[key]
        print(f"    {key:<6} {'unchanged' if same else 'changed  '} "
              f"{real_parts[key]} -> {fake_parts[key]}")
    print()
    print("  a is identical: counts still agree, so the forgery survives a")
    print("  component that only counts. b changed, because sorting moves the")
    print("  first and last cipher. c_ext changed, because order is hashed.")
    print("  c_sig did not change, because the signature_algorithms list was")
    print("  copied verbatim: three of the four components fire, and the one")
    print("  that stays clean is the one a lazy patch forgets to touch.")
    print()

    print("linting the ClientHello before a vendor does")
    for name, p, claims in [(n, p, None) for n, p in clients] + \
            [("chrome-forged", forged, "chrome-131")]:
        problems = lint(name, p, claims)
        print(f"  {name:<14} {'clean' if not problems else str(len(problems)) + ' finding(s)'}")
        for problem in problems:
            print(f"      - {problem}")
    print()

    print("what no hash reconstructs, because only the bytes hold it")
    for item in ["how many padding bytes, and whether padding is present at all",
                 "the byte length of the whole ClientHello",
                 "the order of supported_groups against key_share groups",
                 "duplicate extensions, and whether any appear twice",
                 "extension_data lengths inside key_share and status_request",
                 "the certificate message shape, which is a separate hash"]:
        print(f"    - {item}")
    print("  ja4 exists to make a cheap join possible. a vendor that wants")
    print("  depth reads the ClientHello directly and ignores all of it.")


main()

chrome-131 extension sequence, in the order it is offered
  grease -> server_name -> extended_master_secret -> renegotiation_info -> supported_groups -> ec_point_formats -> application_layer_protocol_negotiation
  signature_algorithms -> signed_certificate_timestamp -> supported_versions -> session_ticket -> key_share -> status_request -> psk_key_exchange_modes
  order is signal: ja3 ignores it, ja4 hashes it, bytes keep it

ja3 and ja4 for four honest client descriptions
client         ja3                                ja4
chrome-131     4a80cece9cbf159021c25048d9bccce5   t303010f0d02_1301_0035_77343e1926c5_a780aaa4a51f
firefox-133    402b6463becd5ebd5390b41cb227fdf9   t30301110e02_1301_0035_464881dcc049_c3676392837c
openssl-3.3    7ab119f3438c72a814e046c6a361c90a   t30301110e01_002f_cca9_906560e58ac2_28ea3bad116d
go-1.23        b7f89a6fdc5e033358f2fbc0dde841d5   t303010d0d02_1301_0035_50b7453da5e5_d90a66b30138
  4 distinct ja3 values from 4 clients

the forgery everybody writes: chrome's list, sorted, GREASE stripped
client         ja3                                ja4
chrome-131     4a80cece9cbf159021c25048d9bccce5   t303010f0d02_1301_0035_77343e1926c5_a780aaa4a51f
chrome-forged  261ddff588b1a2d9a6e1bfb38fa405dc   t303010f0d02_002f_cca9_f0e722859c02_a780aaa4a51f

  ja3 changed, so a list of known hashes stops matching, which is
  the point of the exercise and also the limit of it.

  ja4, component by component:
    a      unchanged t303010f0d02 -> t303010f0d02
    b      changed   1301_0035 -> 002f_cca9
    c_ext  changed   77343e1926c5 -> f0e722859c02
    c_sig  unchanged a780aaa4a51f -> a780aaa4a51f

  a is identical: counts still agree, so the forgery survives a
  component that only counts. b changed, because sorting moves the
  first and last cipher. c_ext changed, because order is hashed.
  c_sig did not change, because the signature_algorithms list was
  copied verbatim: three of the four components fire, and the one
  that stays clean is the one a lazy patch forgets to touch.

linting the ClientHello before a vendor does
  chrome-131     clean
  firefox-133    clean
  openssl-3.3    2 finding(s)
      - no h2 in ALPN: not a browser, whatever the UA claims
      - cipher list sorted ascending: an artefact of forging
  go-1.23        clean
  chrome-forged  3 finding(s)
      - no GREASE in a client claiming a browser engine
      - no GREASE in extensions either
      - cipher list sorted ascending: an artefact of forging

what no hash reconstructs, because only the bytes hold it
    - how many padding bytes, and whether padding is present at all
    - the byte length of the whole ClientHello
    - the order of supported_groups against key_share groups
    - duplicate extensions, and whether any appear twice
    - extension_data lengths inside key_share and status_request
    - the certificate message shape, which is a separate hash
  ja4 exists to make a cheap join possible. a vendor that wants
  depth reads the ClientHello directly and ignores all of it.

JA3 and the Limits of Rotating a Hash#

JA3 takes five fields from the ClientHello — version, cipher list, extension list, supported groups, point formats — strips GREASE, joins them with commas, and takes an MD5. The output is 32 hex characters, which is why it ended up in blocklists: joining on a hash is trivial at the edge, cheap to store, and trivially enumerable from a crawler.

The failure mode is obvious and universal. Strip a GREASE value, sort the lists, and the hash becomes one that appears nowhere in the wild, which defeats the exact list you were hiding from. The chrome-forged row shows the arithmetic: the JA3 moved from 4a80cece... to 261ddff5..., and the new value is unique to your fleet. You have not become a different browser. You have become the only client in the world with that hash, which is a stronger signal than a hash the vendor already knows belongs to Chrome.

The deeper problem is that JA3 has no notion of plausibility. It cannot tell you that a client claiming Chrome 131 offered eleven supported groups when the engine offers twenty, or that it omitted h2 from ALPN and is therefore not a browser at all. Those are visible in the input data, and JA3 discards them by hashing.

JA4 Was Built to Fix Exactly That#

JA4 is a different shape, not a better hash. Its first section, a, encodes structure as counts rather than content: transport (t for TCP, q for QUIC), the negotiated version, whether SNI was sent, and the cipher, extension and ALPN counts. Section b is the first and last cipher in the offered order. Section c is a hash of the extension list in its original order, excluding SNI and ALPN, plus a separate hash of the signature algorithms.

The effect is visible in the four honest rows. Chrome and Firefox have different extension counts and therefore different a, and so do OpenSSL and Go, but they now share an a prefix with themselves only. A vendor storing JA4 gets a field they can bucket, threshold and alert on, and a client that forges one section still has to agree with the other three.

None of this makes the ClientHello unforgeable. It makes forgery require agreement rather than resemblance, which is a much higher bar and, more usefully, a much lower cost to verify.

Sorting Is the Tell#

The most instructive output above is the component-by-component diff. Sorting Chrome's cipher list and stripping GREASE leaves a completely untouched, because counts do not move when you sort. It changes b, because the first and last cipher in the list move to the extremes. It changes c_ext, because order is hashed. It leaves c_sig untouched, because the signature algorithms list was copied verbatim.

Two lessons. First, a linter that only checks structure will pass a sorted forgery cleanly, so your checks have to compare against observed sequences, not just counts and lengths. Second, a fleet that has sorted its ciphers has a characteristic: a long, stable c_sig value shared across every forged client, with a b that no engine produces. That is a join key.

The linter section shows the honest baselines are clean, so the tool is not simply flagging everything. OpenSSL draws two findings that are correct as browser findings and wrong as client findings: no h2, and an ascending cipher list, which is exactly what OpenSSL does. Lint against the engine you claim to be, which is why lint() takes the claim separately from the descriptor name.

GREASE Exists to Break This#

GREASE is the mechanism by which the TLS working group made this problem tractable for itself. The client reserves reserved-looking values and deliberately varies them per connection, so a server cannot key on them and a middlebox cannot reject them. Chrome sends one GREASE value in its cipher list, one in its extension list, and one in its group list, and it varies them across connections.

The consequence for anyone holding a static description is severe. If your ClientHello carries the same extension order on every connection for weeks, the absence of GREASE variation is observable without computing anything clever: the values are identical across sessions where a browser's differ. This is the same shape of problem as ALPN and session behaviour, where consistency across a long window is itself the signal.

Which means a static cipher list is the wrong artefact. Whatever you do at this layer has to vary per connection the way the engine does, and ALPN and session resumption covers the connection-level behaviour that has to vary alongside it.

What No Hash Can Reconstruct#

The last section of the output is the honest boundary. Padding length and whether padding is present at all, the total byte length of the ClientHello, the relative order of supported_groups against the groups actually offered in key_share, duplicate extensions, extension data lengths inside key_share and status_request, and the shape of the certificate message: none of these are recoverable from JA3 or JA4, because a hash destroys exactly the structure that holds them.

They are also the cheapest things for a vendor to compute, since they are counts and offsets over bytes already in memory. If you are going to be deep-fingerprinted, this is where it happens, and no amount of cipher-list tuning reaches it.

There is a second category that is behavioural rather than structural and much harder to fake: whether a session ticket is accepted, whether a full handshake happens on the first request to a new host, how many connections are opened before one is reused, and whether the session ticket lifetime behaves like a browser's. A client that resumes perfectly on one connection and never resumes on the next is not a description problem at all.

Rotating a Pin Is Not the Same as Shaping a Stack#

The practical conclusion is uncomfortable: at the TLS layer, static configuration does not hold up, and the tooling that claims otherwise is usually rotating values it does not understand.

Three approaches, in increasing order of durability. Pinning a fixed cipher and extension list per profile is the weakest, because it produces a hash nobody has and an unvarying GREASE profile. Fetching a fresh description from a remote endpoint at session start is better, since the values rotate and can be sourced from a real capture, but it introduces a network dependency in the hot path and a supply-chain question. Actually patching the stack so it emits what the engine emits is the only approach that produces GREASE variation, plausible ordering and correct extension data lengths, because at that point you are no longer describing a browser, you are running one.

That is the same conclusion the TLS session resumption lesson reaches from the connection side, and the reason this material sits next to QUIC and transport fingerprints: if your client negotiates HTTP/3, this entire discussion applies to the TLS-in-QUIC handshake instead, with the same GREASE rules and none of the same library support for controlling them. Automate the check by running a linter like the one above against every profile you ship, in CI, on every browser version bump. Pinning a Chrome 131 description into a fleet in 2026 does not produce a Chrome 131 client; it produces a client that stopped being updated eighteen months ago.