Hiding in Plain Sight#

When a website is heavily protected by Datadome or Akamai, simple HTTP requests (curl_cffi) are often insufficient. The site will force you to execute complex JavaScript challenges or solve invisible Turnstile CAPTCHAs that require a full DOM environment to evaluate.

To proceed, you must use a Headless Browser (Playwright, Puppeteer, Selenium). However, out-of-the-box headless browsers are incredibly easy for WAFs to detect.

The Leaks of Standard Headless Browsers#

When you launch playwright or selenium in headless mode, the Chromium engine leaves dozens of blatant "I am a bot" flags scattered throughout the JavaScript environment: 1. navigator.webdriver === true (The most obvious flag). 2. The window.chrome object is missing or malformed. 3. The Permissions API behaves abnormally in headless mode. 4. The Plugins array is empty (real browsers always have default PDF plugins). 5. The User-Agent string contains the literal substring HeadlessChrome. 6. navigator.languages comes back as a single default instead of a realistic ordered list. 7. document.hidden never fires and Page Visibility always reports visible, because headless reverting tabs never actually happens.

The Solution: Stealth Plugins#

To survive, you must patch the browser binary in real-time, injecting JavaScript before the page loads to overwrite and mock these exposing variables.

1. Puppeteer Extra Stealth (Node.js) The industry standard in the JavaScript ecosystem. It applies a suite of evasion techniques automatically.

const puppeteer = require('puppeteer-extra');
const StealthPlugin = require('puppeteer-extra-plugin-stealth');
puppeteer.use(StealthPlugin());

puppeteer.launch({ headless: true }).then(async browser => {
  const page = await browser.newPage();
  await page.goto('https://bot.sannysoft.com'); // Bot detection test site
  await page.screenshot({ path: 'stealth.png' });
  await browser.close();
});

2. Undetected Chromedriver (Python) For Python developers, undetected-chromedriver is an optimized Selenium wrapper that aggressively patches the ChromeDriver executable binary at the hex level, preventing it from passing the cdc_ string variables that WAFs actively hunt for.

import undetected_chromedriver as uc
import time

options = uc.ChromeOptions()
# uc automatically handles binary patching and JS evasion
driver = uc.Chrome(options=options)

driver.get("https://nowsecure.nl")
time.sleep(5)
driver.quit()

The List Is Longer Than You Think#

Complete stealth means patching far more than navigator.webdriver. A hardened setup also fakes: the window.chrome API surface, a non-empty navigator.plugins array, a realistic navigator.languages, the HTMLMediaElement permission behavior, the chrome://media-engagement ID, WebGL renderer strings, and the crypto.getRandomValues entropy behavior. Many WAFs probe a dozen or more properties at once and compare the combination rather than any single value.

The Sandboxing Trap#

Default Chromium sandboxing and GPU flags are another giveaway. Real browsers enable hardware acceleration, WebGL, and audio output. Launching a headless browser takes 30–60 seconds to warm up the renderer, initialize the media stack, and fetch nothing. Loading the first page cold—a famously detectable signature—can be hidden by warming the browser (loading about:blank, then a neutral page) before touching the target.

The Future: Anti-Detect Browsers#

For enterprise-scale scraping, managing Playwright stealth scripts becomes fragile. Companies now use Anti-Detect Browsers (like AdsPower, Multilogin, or Dolphin Anty). These are custom-compiled forks of Chromium where the fingerprinting APIs (Canvas, WebGL, Audio) have been rewritten in C++ to allow perfect, mathematical spoofing directly from an API payload, completely bypassing JavaScript detection vectors.

The Signal Is Distributional, Not Per-Client#

Everything in signal inventory is a per-request measurement: this client sent this value, and the value is plausible or not. The detection that catches well-built stealth forks does not work that way. It groups your requests into a cohort and asks whether that cohort's distribution over each field looks like it was drawn from real devices.

That distinction is the whole subject of this lesson. A single request can be impeccable and still belong to a fleet that is trivially separable, because fifty thousand requests from one template produce a distribution no population of real devices ever produces. The vendor is not asking "is this a browser". It is asking "is this a sample from my population of browsers".

# A stealth fork that spoofs perfectly is still detectable, because the
# vendor does not compare your fingerprint to a target. It compares your
# fleet to the population. The signal is distributional, not per-client.

import random

SEED = 20260803
N = 1200

FIELDS = ["platform", "screen", "dpr", "cores", "memory", "fonts", "gpu",
          "timezone", "language"]

PLATFORMS = [("Win32", 58), ("MacIntel", 17), ("Linux x86_64", 19),
             ("iPhone", 6)]
SCREENS = [("1920x1080", 34), ("1536x864", 21), ("1440x900", 15),
           ("1366x768", 12), ("2560x1440", 8), ("1280x720", 6),
           ("3840x2160", 4)]
DPR = [("1", 26), ("1.25", 18), ("1.5", 24), ("2", 32)]
CORES = [("2", 9), ("4", 31), ("6", 12), ("8", 27), ("12", 13), ("16", 8)]
MEMORY = [("2", 8), ("4", 30), ("8", 38), ("16", 24)]
GPUS = [("ANGLE (NVIDIA, RTX 3060 Direct3D11)", 14),
        ("ANGLE (Intel, UHD Graphics 630)", 22),
        ("ANGLE (AMD, Radeon RX 6600)", 9),
        ("ANGLE (Apple, M1)", 11),
        ("ANGLE (NVIDIA, GTX 1650 Direct3D11)", 13),
        ("ANGLE (Intel, Iris Xe Graphics)", 18),
        ("ANGLE (Apple M2, M2)", 6),
        ("ANGLE (AMD, Radeon Graphics)", 7)]
ZONES = [("Europe/Berlin", 12), ("America/New_York", 14),
         ("America/Los_Angeles", 11), ("Europe/London", 9),
         ("Europe/Paris", 7), ("Asia/Tokyo", 8), ("Asia/Kolkata", 10),
         ("Australia/Sydney", 5), ("America/Sao_Paulo", 8),
         ("Europe/Madrid", 5), ("Asia/Singapore", 6), ("UTC", 5)]
LANGS = [("en-US", 30), ("en-GB", 13), ("de-DE", 10), ("fr-FR", 9),
         ("es-ES", 8), ("pt-BR", 6), ("ja-JP", 6), ("nl-NL", 4),
         ("it-IT", 5), ("pl-PL", 4), ("en-CA", 5)]

TEMPLATE = {"platform": "Win32", "screen": "1920x1080", "dpr": "1",
            "cores": "8", "memory": "8", "fonts": "210",
            "gpu": "ANGLE (NVIDIA, GTX 1650 Direct3D11)",
            "timezone": "Europe/Berlin", "language": "de-DE"}

TEMPLATES = [
    {"platform": "Win32", "screen": "1920x1080", "dpr": "1", "cores": "8",
     "memory": "8", "fonts": "210",
     "gpu": "ANGLE (NVIDIA, GTX 1650 Direct3D11)",
     "timezone": "Europe/Berlin", "language": "de-DE"},
    {"platform": "MacIntel", "screen": "1512x982", "dpr": "2", "cores": "8",
     "memory": "8", "fonts": "268", "gpu": "ANGLE (Apple, M1)",
     "timezone": "America/New_York", "language": "en-US"},
    {"platform": "Win32", "screen": "1366x768", "dpr": "1.25", "cores": "4",
     "memory": "8", "fonts": "184", "gpu": "ANGLE (Intel, UHD Graphics 630)",
     "timezone": "Europe/London", "language": "en-GB"},
    {"platform": "Linux x86_64", "screen": "1920x1080", "dpr": "1", "cores": "8",
     "memory": "16", "fonts": "142", "gpu": "ANGLE (AMD, Radeon Graphics)",
     "timezone": "Asia/Tokyo", "language": "ja-JP"},
]

# the only two fields the packaged fork is allowed to vary
JITTER_FIELDS = ["screen", "language"]
JITTER_VALUES = {"screen": ["1920x1080", "1536x864", "1440x900", "1366x768"],
                 "language": ["de-DE", "en-US", "fr-FR", "nl-NL"]}

THRESHOLD = 0.20


def pick(rng, table):
    total = sum(w for _v, w in table)
    roll = rng.randrange(total)
    for value, weight in table:
        if roll < weight:
            return value
        roll -= weight
    return table[-1][0]


def real_device(rng):
    return {
        "platform": pick(rng, PLATFORMS),
        "screen": pick(rng, SCREENS),
        "dpr": pick(rng, DPR),
        "cores": pick(rng, CORES),
        "memory": pick(rng, MEMORY),
        "fonts": str(rng.randrange(88, 331)),
        "gpu": pick(rng, GPUS),
        "timezone": pick(rng, ZONES),
        "language": pick(rng, LANGS),
    }


def forked(rng, template, jitter):
    device = dict(template)
    # a font count is never exactly the template value on a real machine
    device["fonts"] = str(int(template["fonts"]) + rng.randrange(-6, 7))
    if jitter:
        for field in JITTER_FIELDS:
            values = JITTER_VALUES[field]
            device[field] = values[rng.randrange(len(values))]
    return device


def counts(devices, field):
    tally = {}
    for device in devices:
        tally[device[field]] = tally.get(device[field], 0) + 1
    return tally


def top_share(tally):
    return max(tally.values()) / float(N)


def naive_terms(devices, field):
    tally = counts(devices, field)
    distinct = len(tally)
    share = top_share(tally)
    term = 2 if share > 0.5 else (-1 if distinct > 2 else 0)
    return distinct, share, term


def tv_distance(a, b):
    # total variation between two field distributions. keys are sorted so the
    # float sum is in a fixed order and the result does not depend on hashing.
    keys = sorted(set(a) | set(b))
    total = float(sum(a.values()))
    other = float(sum(b.values()))
    return 0.5 * sum(abs(a.get(k, 0) / total - b.get(k, 0) / other)
                     for k in keys)


def main():
    rng = random.Random(SEED)
    real = [real_device(rng) for _ in range(N)]
    one = [forked(random.Random(SEED + i), TEMPLATE, True) for i in range(N)]
    four = [forked(random.Random(SEED + i), TEMPLATES[i % 4], False)
            for i in range(N)]

    reference = dict((f, counts(real, f)) for f in FIELDS)
    cohorts = [("real population", real),
               ("one template, jitter on 2 fields", one),
               ("four templates", four)]

    print("three cohorts of %d clients, one seed, no wall clock" % N)
    print()
    print("per-field detail: distinct values, largest single share, naive term")
    print("the naive rule is +2 if one value covers more than half the cohort,")
    print("-1 if the field has more than two values, 0 otherwise.")
    print()
    header = "%-10s %14s %16s %14s" % ("field", "real", "one template",
                                       "four templates")
    print(header)
    for field in FIELDS:
        cells = []
        for _label, cohort in cohorts:
            distinct, share, _term = naive_terms(cohort, field)
            cells.append("%d vals, %3.0f%%" % (distinct, 100 * share))
        print("%-10s %14s %16s %14s" % (field, cells[0], cells[1], cells[2]))
    print()

    print("the same fields scored by the naive rule")
    print("%-34s " % "cohort" + " ".join("%9s" % f[:9] for f in FIELDS)
          + "   total")
    totals = {}
    for label, cohort in cohorts:
        terms = [naive_terms(cohort, f)[2] for f in FIELDS]
        totals[label] = sum(terms)
        print("%-34s " % label
              + " ".join("%9d" % t for t in terms)
              + "  %+5d" % sum(terms))
    print()
    print("  the real population scores negative on font count alone, because")
    print("  243 distinct font lists is more than two. the rule is not")
    print("  measuring whether a cohort is a fleet; it is measuring which")
    print("  fields are uniform, and a single-country deployment scores")
    print("  positive on platform for the same reason a fleet does.")
    print()

    print("total variation against the real population, per field")
    print(f"{'field':<10} {'one template':>14} {'four templates':>16}")
    d_one = dict((f, tv_distance(counts(one, f), reference[f])) for f in FIELDS)
    d_four = dict((f, tv_distance(counts(four, f), reference[f])) for f in FIELDS)
    for field in FIELDS:
        print("%-10s %14.2f %16.2f" % (field, d_one[field], d_four[field]))
    print()

    def summary(label, distances):
        flagged = [f for f in FIELDS if distances[f] > THRESHOLD]
        mean = sum(distances[f] for f in FIELDS) / float(len(FIELDS))
        print("  %-34s fields over %.2f: %2d/%d, mean distance %.2f"
              % (label, THRESHOLD, len(flagged), len(FIELDS), mean))
        if flagged:
            print("      " + ", ".join(flagged))
        else:
            print("      no field exceeds the threshold")

    print("summary at a fixed threshold, defined against the population")
    summary("real population", dict((f, 0.0) for f in FIELDS))
    summary("one template, jitter on 2 fields", d_one)
    summary("four templates", d_four)
    print()
    print("  this rule scores zero for the population by construction, which")
    print("  is the property the naive rule lacked: it cannot fire on")
    print("  honest traffic no matter how concentrated that traffic is.")
    print()

    worst_one = sorted(FIELDS, key=lambda f: -d_one[f])[:3]
    worst_four = sorted(FIELDS, key=lambda f: -d_four[f])[:3]
    print("  worst fields, one template: "
          + ", ".join("%s %.2f" % (f, d_one[f]) for f in worst_one))
    print("  worst fields, four templates: "
          + ", ".join("%s %.2f" % (f, d_four[f]) for f in worst_four))
    print()
    print("  screen gets worse in the four-template cohort: two of the four")
    print("  templates share 1920x1080, so half the fleet reports one value.")
    print("  adding templates without matching their weights to the")
    print("  population is how a team makes itself more detectable.")


main()

three cohorts of 1200 clients, one seed, no wall clock

per-field detail: distinct values, largest single share, naive term
the naive rule is +2 if one value covers more than half the cohort,
-1 if the field has more than two values, 0 otherwise.

field                real     one template four templates
platform     4 vals,  60%     1 vals, 100%   3 vals,  50%
screen       7 vals,  33%     4 vals,  26%   3 vals,  50%
dpr          4 vals,  32%     1 vals, 100%   3 vals,  50%
cores        6 vals,  31%     1 vals, 100%   2 vals,  75%
memory       4 vals,  37%     1 vals, 100%   2 vals,  75%
fonts      243 vals,   1%    13 vals,   9%  52 vals,   3%
gpu          8 vals,  22%     1 vals, 100%   4 vals,  25%
timezone    12 vals,  14%     1 vals, 100%   4 vals,  25%
language    11 vals,  31%     4 vals,  28%   4 vals,  25%

the same fields scored by the naive rule
cohort                              platform    screen       dpr     cores    memory     fonts       gpu  timezone  language   total
real population                            2        -1        -1        -1        -1        -1        -1        -1        -1     -6
one template, jitter on 2 fields           2        -1         2         2         2        -1         2         2        -1     +9
four templates                            -1        -1        -1         2         2        -1        -1        -1        -1     -3

  the real population scores negative on font count alone, because
  243 distinct font lists is more than two. the rule is not
  measuring whether a cohort is a fleet; it is measuring which
  fields are uniform, and a single-country deployment scores
  positive on platform for the same reason a fleet does.

total variation against the real population, per field
field        one template   four templates
platform             0.40             0.16
screen               0.27             0.55
dpr                  0.75             0.31
cores                0.72             0.47
memory               0.63             0.38
fonts                0.94             0.78
gpu                  0.87             0.50
timezone             0.88             0.58
language             0.53             0.46

summary at a fixed threshold, defined against the population
  real population                    fields over 0.20:  0/9, mean distance 0.00
      no field exceeds the threshold
  one template, jitter on 2 fields   fields over 0.20:  9/9, mean distance 0.67
      platform, screen, dpr, cores, memory, fonts, gpu, timezone, language
  four templates                     fields over 0.20:  8/9, mean distance 0.47
      screen, dpr, cores, memory, fonts, gpu, timezone, language

  this rule scores zero for the population by construction, which
  is the property the naive rule lacked: it cannot fire on
  honest traffic no matter how concentrated that traffic is.

  worst fields, one template: fonts 0.94, timezone 0.88, gpu 0.87
  worst fields, four templates: fonts 0.78, timezone 0.58, screen 0.55

  screen gets worse in the four-template cohort: two of the four
  templates share 1920x1080, so half the fleet reports one value.
  adding templates without matching their weights to the
  population is how a team makes itself more detectable.

What jitter Actually Changes#

The three cohorts above differ in exactly one design decision, and the numbers make the cost of that decision visible. The real population samples platform, screen, DPR, core count, memory, font count, GPU, timezone and language from weighted distributions. The one-template fork pins all nine to a single machine, then varies two of them: screen and language.

Read the per-field table. Jittering screen produces four values with a largest share of 26%, and jittering language produces four values at 28%. Both fields now look more like the population than they did, and the naive rule even scores screen favourably. The other seven fields are pinned at 100%: every client reports dpr=1, cores=8, memory=8, timezone=Europe/Berlin, and one GPU string. Jittering the two fields that are cheap to jitter left the seven fields that identify the fleet untouched.

That is the characteristic failure of packaged stealth: it jitters what is easy to reach from a JS context and ignores what is determined by the template's machine description. Screen and locale are two numbers in a config file. GPU, DPR and core count are tied to the hardware story the template is telling, and changing them means changing the story.

The Naive Rule Fails in Both Directions#

The tempting detector is a per-field uniformity threshold: if more than half the cohort shares one value, flag it. The naive rule table shows why that fails.

On the one-template fork it scores +9, correctly. On the real population it scores -6, but look at where the -6 comes from: fonts contributes -1 because the population has 243 distinct font counts. Every other field is either flagged (+2 for platform's 60% share) or varied (-1). The rule is not detecting fleet-ness. It is detecting font-list diversity, and platform concentration, and whatever else happens to cross an arbitrary line.

The four-template fork is the case that kills it: the naive rule scores -3, lower than the real population's -6 is not, but lower than the fleet's +9, and on that basis a naive operator concludes the fork is fine. The distributional test flags 8 of 9 fields.

Two lessons. A per-client rule cannot see a fleet, because the evidence is not in any one client. And a threshold rule fires on honest large populations, because real populations are concentrated: a single-country deployment, an enterprise NAT, or any real traffic from one region looks uniform on the fields that region shares.

A Test That Cannot Fire on Honest Traffic#

The refinement is to stop asking "is this value too common" and start asking "is this distribution the population's distribution". Total variation distance does that. For each field, compare the cohort's value distribution against the reference population's, and flag the field when the distance exceeds a threshold.

The reference cohort scores 0 on every field by construction, so the test has no false positives against the population it was defined on. That is not a coincidence; it is the definition. The one-template fork scores a mean distance of 0.67 with all nine fields over threshold. The four-template fork scores 0.47 with eight over threshold. The expanded fork is closer, and still separable.

This is also why the defence cannot be "be more random". The four-template fork is more random than the one-template fork on most fields, and it is worse on screen, because two of the four templates happen to share 1920x1080. Randomness that is not matched to the population's weights makes the distribution lumpier, not smoother. A template set is a sampling design, and sampling designs are wrong in ways that are visible.

Sampling Weights Are the Product#

The practical implication is that the unit of work is not a stealth patch. It is a template set with weights, sourced from real captures and refreshed as hardware populations change. A template is a description of one machine: platform, renderer, screen geometry, DPR, core count, memory, font list, timezone, locale, and the browser build that machine runs. A weighted set of templates is a population model, and the acceptance test is the distance table.

That is expensive in a way that a stealth package is not, and it explains the commercial landscape. Vendors sell templates because templates are the hard, ongoing part; the patching is a packaging problem that has been solved several times. A team that buys a stealth fork and writes its own profiles has bought the easy half and built the half that fails, which is why population-scale detection and device profile consistency are the lessons to read next.

There is a second-order cost worth naming. Template weights are a function of time. The share of clients on a given GPU, a given Windows build and a given screen size moves every quarter, so a template set that was matched to the population last year is drifting away from it. Keeping the reference distribution current is a maintenance task, not a one-time import.

What a Fork Can Honestly Fix#

Not every field needs a template set. The distance table tells you which fields are cheap and which are structural.

Fields determined by a config value, such as screen, DPR, timezone, language and memory, can be made plausible with a weighted choice at profile creation, provided the weights come from somewhere real. Fields determined by the rendering stack, such as the canvas hash, the WebGL parameter hash and the client-rects pattern, cannot be chosen independently without breaking the agreement that cross-validation checks. fonts is the telling one in every table here: it has the largest distance in both forks, because a font list is an artefact of an operating system image and a template that hard-codes a count is not describing a machine that exists.

So the honest split is: choose per profile the fields that are genuinely independent, derive the rest from a coherent machine description, and accept that the derived fields will only be as good as the description. A fork with one template cannot be fixed by jittering it. It can only be replaced by a population, and automation framework comparison is where the tooling that supports populations, rather than single contexts, is compared.

The uncomfortable conclusion is the one the numbers keep pointing at. Detection at this scale is a statistics problem, and the defence is a sampling problem. Neither is addressed by making any single client look better, which is why the rest of this material spends its time on coherence rather than on evasion.