Hardening Playwright with CDP

Order Is the Whole Discipline

A hardening pipeline fails in a specific way: not because the patches are wrong, but because one of them ran too late. An override installed after the document's first script has already read navigator.webdriver is worth nothing, and a script that patches navigator.userAgent before the UA string is used to build the request headers produces a request the UA never claimed to send.

So this lesson is mostly sequencing: what must be installed before what, which of those things can only be done in a patched binary, and how to audit the result once the ordering is right.

The Init-Script Mechanism

Playwright exposes the same underlying capability at two levels. context.add_init_script() installs a script that runs in every page and every frame of that context, before any of the page's own scripts. page.add_init_script() is the same thing scoped to one page. Underneath, both compile to a CDP call to Page.addScriptToEvaluateOnNewDocument, which schedules the script to execute in the context of every new document, after the document object exists and before any of its scripts run.

Three consequences that matter in practice:

• It runs in the main world, on the same window as the page. Your overrides are visible to any script that inspects them, and to any script that snapshots Object.getOwnPropertyNames(window) early. That is why descriptor shape is part of the work, not a nicety.
• It runs per document, including every iframe. A patch that assumes a single global will be re-run in each frame, so keep the payload idempotent or guard it with a sentinel on window.
• It is not the only way to run early. Page.setBypassCSP, Fetch.enable and Emulation.setUserAgentOverride act at the network and protocol layers and are strictly earlier. When a value must be right before the network stack reads it, use the protocol layer; see the CDP instrumentation lesson for what that instrumentation costs you in return.

Install Order: What Goes In and in What Sequence

A working order, and the reason for each step:

1. Protocol layer first. Emulation.setUserAgentOverride with both the userAgent string and the userAgentMetadata struct, so the User-Agent header, the Sec-CH-UA trio and navigator.userAgent are set from one source before any request is built.
2. Context options at construction. locale, timezone_id, geolocation, permissions, color_scheme, viewport, device_scale_factor, has_touch, reduced_motion, forced_colors, extra_http_headers, proxy. These are not overrides; they change what the browser believes, which is why they belong on the context and not in a script.
3. Init script, in this internal order: - navigator.webdriver removed, and the Navigator.prototype accessor restored to a shape the stock browser has. - Function.prototype.toString patched first, so every function you install afterwards can be made to read as native. - The chrome surface: chrome.runtime, chrome.csi, chrome.loadTimes, chrome.app. - navigator values: platform, vendor, hardwareConcurrency, deviceMemory, maxTouchPoints, languages, language, plugins, mimeTypes. - screen and window geometry: width, height, availWidth, availHeight, colorDepth, devicePixelRatio. - Intl formatting: timezone, locale, number and date formats. - WebGL getParameter for UNMASKED_VENDOR_WEBGL and UNMASKED_RENDERER_WEBGL. - Permissions and capability answers last, because they often depend on values set above.
4. Warm-up navigation, so the browser's own internal reads of these values have already happened.
5. The target navigation. Not before.

The rule behind the sequence: the further down the stack a value is produced, the earlier it must be fixed. Anything produced natively -- the canvas hash, the audio fingerprint, the real GPU string inside the process -- does not belong in a script at all; it belongs in the patched binary.

The JavaScript Payload

This is the actual text that goes into the browser, a compact subset of the list above. The sentinel makes it safe to run in every frame, and the toString patch is installed before anything that needs it.

(() => {
  if (window.__stealthInit) return;
  window.__stealthInit = true;

  const nativeToString = Function.prototype.toString;
  const nativeFns = new WeakSet();
  const markNative = (fn) => { nativeFns.add(fn); return fn; };
  Function.prototype.toString = function () {
    if (nativeFns.has(this)) return 'function ' + this.name + '() { [native code] }';
    return nativeToString.call(this);
  };

  // 1. webdriver: remove the property, do not set it to false
  const wd = Object.getOwnPropertyDescriptor(Navigator.prototype, 'webdriver');
  if (wd) {
    Object.defineProperty(Navigator.prototype, 'webdriver', { get: () => false });
    Object.defineProperty(Navigator.prototype, 'webdriver', wd);
    delete Navigator.prototype.webdriver;
  }

  const defineValue = (obj, prop, value) => {
    const target = Object.getPrototypeOf(obj) || obj;
    Object.defineProperty(target, prop, {
      get: markNative(function () { return value; }),
      set: () => {}, enumerable: true, configurable: true,
    });
  };

  defineValue(navigator, 'platform', 'Win32');
  defineValue(navigator, 'vendor', 'Google Inc.');
  defineValue(navigator, 'hardwareConcurrency', 12);
  defineValue(navigator, 'deviceMemory', 8);
  defineValue(navigator, 'maxTouchPoints', 0);
  defineValue(navigator, 'language', 'en-US');

  // 2. the chrome surface
  if (!window.chrome) window.chrome = {};
  window.chrome.runtime = markNative(function runtime() {});
  window.chrome.runtime.connect = markNative(function connect() {});
  window.chrome.csi = markNative(function csi() { return {}; });
  window.chrome.loadTimes = markNative(function loadTimes() { return {}; });
  window.chrome.app = { isInstalled: false, InstallState: { DISABLED: 'disabled', INSTALLED: 'installed' } };

  // 3. WebGL renderer strings
  const patchGL = (proto) => {
    if (!proto) return;
    const raw = proto.getParameter;
    proto.getParameter = markNative(function getParameter(pname) {
      if (pname === 0x9245) return 'Google Inc. (NVIDIA)';       // UNMASKED_VENDOR_WEBGL
      if (pname === 0x9246) return 'ANGLE (NVIDIA, GeForce RTX 3060 Direct3D11)';
      return raw.call(this, pname);
    });
  };
  patchGL(window.WebGLRenderingContext && WebGLRenderingContext.prototype);
  patchGL(window.WebGL2RenderingContext && WebGL2RenderingContext.prototype);
})();

Note what this payload does not do: it does not touch canvas, it does not touch AudioContext, and it does not fake userAgentData. Those are either native-side or need the protocol layer, and pretending otherwise is how a pipeline ends up self-contradictory.

Self-Consistency Is the Real Bar

The bar is not "does the getter return the right value". It is "does the property look like it was never touched". Three checks catch most mistakes:

• Descriptor shape. Object.getOwnPropertyDescriptor(Navigator.prototype, 'platform') in a stock Chrome reports an accessor with configurable: true and enumerable: true; yours must match. An override installed on the instance instead of the prototype, or left non-configurable, is a one-line detection.
• Native-ness of functions. Anything you installed must return [native code] from Function.prototype.toString when the stock browser would. The patch above registers each function in a WeakSet and consults it in the replacement toString.
• Cross-field agreement. A page that reads Notification.permission, Notification.queryPermission({name: 'notifications'}), and the permissions API result is comparing three answers, and they must be the same string. The same applies to navigator.plugins.length versus navigator.mimeTypes.length versus the plugin entries' length properties, and to screen.width versus window.innerWidth plus chrome.

This is where fingerprint cross-validation stops being theoretical. The detectors do not test whether your values are plausible; they test whether your values are possible together, and a half-finished override is a value that is impossible together.

A Model of the Init Script

The logic is easier to test outside the browser than inside it. The model below is the same table of overrides as the payload above, applied to a mock global in Python, followed by eighteen consistency checks derived from the UA. It is worth having as a Python module precisely because it can run in a unit test: change one field, run the checks, see which pairs break, before you ever restart a browser.

import re

INIT_SCRIPT = {
    "navigator.userAgent": ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
                            "(KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36"),
    "navigator.platform": "Win32",
    "navigator.vendor": "Google Inc.",
    "navigator.webdriver": "DELETE",
    "navigator.hardwareConcurrency": 12,
    "navigator.deviceMemory": 8,
    "navigator.maxTouchPoints": 0,
    "navigator.language": "en-US",
    "navigator.languages": ["en-US", "en"],
    "navigator.plugins": ["PDF Viewer", "Chrome PDF Viewer", "Chromium PDF Viewer"],
    "screen.width": 1920,
    "screen.height": 1080,
    "screen.availWidth": 1920,
    "screen.availHeight": 1040,
    "screen.colorDepth": 24,
    "window.devicePixelRatio": 1,
    "chrome.runtime.connect": "native fn",
    "chrome.csi": "native fn",
    "chrome.loadTimes": "native fn",
    "chrome.app": {"isInstalled": False, "InstallState": {"DISABLED": "disabled"}},
    "Intl.timeZone": "America/Chicago",
    "Intl.locale": "en-US",
    "Intl.numberFormat": "1,234.56",
    "Intl.dateFormat": "30/09/2026",
    "webgl.vendor": "Google Inc. (NVIDIA)",
    "webgl.renderer": "ANGLE (NVIDIA, NVIDIA GeForce RTX 3060 Direct3D11 vs_5_0 ps_5_0, D3D11)",
    "permissions.notifications": "default",
    "permissions.queryPermission": "prompt",
    "headers.acceptLanguage": "en-US,en;q=0.9",
    "headers.secChUa": '"Chromium";v="130", "Google Chrome";v="130", "Not_A Brand";v="24"',
    "headers.secChUaPlatform": '"Windows"',
    "uaData.platform": "Windows",
    "uaData.mobile": False,
    "uaData.brands": '"Chromium";v="130", "Google Chrome";v="130", "Not_A Brand";v="24"',
}

NATIVE_REGISTRY = {"chrome.runtime.connect", "chrome.csi", "chrome.loadTimes"}


def grab(text, pattern):
    m = re.search(pattern, text)
    return m.group(1) if m else None


def apply_init_script(spec, target):
    installed, deleted = [], []
    for path, value in spec.items():
        if value == "DELETE":
            deleted.append(path)
            continue
        node = target
        for part in path.split(".")[:-1]:
            node = node.setdefault(part, {})
        node[path.split(".")[-1]] = value
        installed.append(path)
    return installed, deleted


GLOBAL = {}
INSTALLED, DELETED = apply_init_script(INIT_SCRIPT, GLOBAL)


def get(path, default=None):
    node = GLOBAL
    for part in path.split("."):
        if not isinstance(node, dict) or part not in node:
            return default
        node = node[part]
    return node


UA = get("navigator.userAgent")
OS = grab(UA, r"\(([^;]+);")
CHROME = grab(UA, r"Chrome/(\d+)")
GL_API = "D3D11" if "D3D11" in get("webgl.renderer") else "OpenGL"

CHECKS = [
    ("platform string", get("navigator.platform"), "Win32" if OS == "Windows NT 10.0" else "?"),
    ("client-hint platform", get("headers.secChUaPlatform"), '"Windows"'),
    ("uaData platform", get("uaData.platform"), "Windows"),
    ("uaData brands match sec-ch-ua", get("uaData.brands"), get("headers.secChUa")),
    ("chrome major in brand", grab(get("headers.secChUa"), r'Chrome";v="(\d+)'), CHROME),
    ("webgl api vs OS", GL_API, "D3D11"),
    ("touch points vs mobile", get("navigator.maxTouchPoints"), 0 if not get("uaData.mobile") else 5),
    ("deviceMemory is a power of two", get("navigator.deviceMemory"), 8),
    ("hardwareConcurrency in range", get("navigator.hardwareConcurrency"), 12),
    ("accept-language vs navigator.language",
     get("headers.acceptLanguage").split(",")[0], get("navigator.language")),
    ("Intl locale vs navigator.language", get("Intl.locale"), get("navigator.language")),
    ("Intl date format vs locale", get("Intl.dateFormat"), "9/30/2026"),
    ("Intl number format", get("Intl.numberFormat"), "1,234.56"),
    ("Notification.permission", get("permissions.notifications"), "default"),
    ("Notification.queryPermission", get("permissions.queryPermission"), "default"),
    ("timezone", get("Intl.timeZone"), "America/Chicago"),
    ("faked functions read as native",
     all(p in NATIVE_REGISTRY for p in INSTALLED if get(p) == "native fn"), True),
    ("webdriver deleted, not faked", DELETED, ["navigator.webdriver"]),
]

print("addInitScript payload: {} properties set".format(len(INSTALLED)))
print("properties deleted outright: {}".format(len(DELETED)))
print()
print("{:<36} {}".format("FIELD", "VALUE AFTER SCRIPT"))
print("-" * 82)
for path in INSTALLED:
    value = get(path)
    if isinstance(value, dict):
        value = "{" + ", ".join(sorted(value)) + "}"
    elif isinstance(value, list):
        value = "[{} entries]".format(len(value))
    print("{:<36} {}".format(path, str(value)[:44]))
print()
print("{:<34} {:<26} {:<14} {}".format("CONSISTENCY CHECK", "VALUE", "EXPECTED", "STATUS"))
print("-" * 92)
mismatches = 0
for label, actual, expected in CHECKS:
    ok = actual == expected
    mismatches += 0 if ok else 1
    print("{:<34} {:<26} {:<14} {}".format(
        label, str(actual)[:24], str(expected)[:12], "ok" if ok else "MISMATCH"))
print()
print("{} checks, {} mismatch{}".format(len(CHECKS), mismatches, "" if mismatches == 1 else "es"))
print("verdict: {}".format("consistent, ship it" if not mismatches else
                           "self-contradictory; a page reads these pairs together"))

addInitScript payload: 33 properties set
properties deleted outright: 1

FIELD                                VALUE AFTER SCRIPT
----------------------------------------------------------------------------------
navigator.userAgent                  Mozilla/5.0 (Windows NT 10.0; Win64; x64) Ap
navigator.platform                   Win32
navigator.vendor                     Google Inc.
navigator.hardwareConcurrency        12
navigator.deviceMemory               8
navigator.maxTouchPoints             0
navigator.language                   en-US
navigator.languages                  [2 entries]
navigator.plugins                    [3 entries]
screen.width                         1920
screen.height                        1080
screen.availWidth                    1920
screen.availHeight                   1040
screen.colorDepth                    24
window.devicePixelRatio              1
chrome.runtime.connect               native fn
chrome.csi                           native fn
chrome.loadTimes                     native fn
chrome.app                           {InstallState, isInstalled}
Intl.timeZone                        America/Chicago
Intl.locale                          en-US
Intl.numberFormat                    1,234.56
Intl.dateFormat                      30/09/2026
webgl.vendor                         Google Inc. (NVIDIA)
webgl.renderer                       ANGLE (NVIDIA, NVIDIA GeForce RTX 3060 Direc
permissions.notifications            default
permissions.queryPermission          prompt
headers.acceptLanguage               en-US,en;q=0.9
headers.secChUa                      "Chromium";v="130", "Google Chrome";v="130",
headers.secChUaPlatform              "Windows"
uaData.platform                      Windows
uaData.mobile                        False
uaData.brands                        "Chromium";v="130", "Google Chrome";v="130",

CONSISTENCY CHECK                  VALUE                      EXPECTED       STATUS
--------------------------------------------------------------------------------------------
platform string                    Win32                      Win32          ok
client-hint platform               "Windows"                  "Windows"      ok
uaData platform                    Windows                    Windows        ok
uaData brands match sec-ch-ua      "Chromium";v="130", "Goo   "Chromium";v   ok
chrome major in brand              130                        130            ok
webgl api vs OS                    D3D11                      D3D11          ok
touch points vs mobile             0                          0              ok
deviceMemory is a power of two     8                          8              ok
hardwareConcurrency in range       12                         12             ok
accept-language vs navigator.language en-US                      en-US          ok
Intl locale vs navigator.language  en-US                      en-US          ok
Intl date format vs locale         30/09/2026                 9/30/2026      MISMATCH
Intl number format                 1,234.56                   1,234.56       ok
Notification.permission            default                    default        ok
Notification.queryPermission       prompt                     default        MISMATCH
timezone                           America/Chicago            America/Chic   ok
faked functions read as native     True                       True           ok
webdriver deleted, not faked       ['navigator.webdriver']    ['navigator.   ok

18 checks, 2 mismatches
verdict: self-contradictory; a page reads these pairs together

The two mismatches are the two failure modes worth memorising. Intl.dateFormat returning 30/09/2026 while Intl.locale is en-US is a browser that formats dates in British order while claiming to be an American: individually each value is common, jointly they are a contradiction a single Date().toLocaleDateString() call exposes. Notification.permission returning default while queryPermission returns prompt is a state that no real permission state machine produces, and it costs two lines to get right. The first real run of any new payload should be this table, not a live target.

userAgent and userAgentMetadata

user_agent alone is a half-measure. Playwright's user_agent context option sets the string, and the separate user_agent_metadata option fills the client-hints struct that Chromium actually sends: brands, fullVersionList, fullVersion, platform, platformVersion, architecture, model, mobile, bitness, wow64. Set one without the other and you have a request whose User-Agent says one thing and whose Sec-CH-UA and Sec-CH-UA-Platform say another, which is precisely the contradiction from header forging.

The rule is one object per profile, used for both the header and the JS-visible value, and a unit test that asserts brands contains a "Google Chrome";v="MAJOR" entry whose major matches the UA's Chrome/ token, platform matches the UA's OS token, mobile is false for a desktop profile, and platformVersion is a plausible build number for that OS release. The two things most often missed are platformVersion (a frozen or obviously fake number is rare) and the GREASE brand entry, which real Chromium adds and which a hand-written brand list omits -- a missing GREASE brand is a small tell that costs nothing to fix.

Context Options That Are Not Overrides

Most of what makes a profile coherent is configured at context construction rather than patched in JavaScript, and configuring it is both more honest and more durable:

The pattern is the same every time: a value set at the protocol or context layer is internally consistent with the rest of the browser for free, while the same value forced in JavaScript is consistent only if you remember every other field that depends on it. Set it where it is produced, and patch only what has no context option.

The Fetch Domain: Blocking and Recording

Two jobs share one mechanism. context.route() and the underlying Fetch.enable domain let you intercept every request the page makes, and you can either let it through (continue) or drop it (abort). The operational win is real: a typical retail page pulls 2-4 MB of images, fonts and video that a data extraction never looks at, and dropping them shortens the load, cuts the bandwidth bill and reduces the CPU the renderer spends on rasterisation.

The detection risk is equally real. A page load that issues four requests and no analytics beacons is not a fast page, it is an obviously synthetic one, and third-party noise is part of the request pattern a risk model scores. So the sane policy is to block by resource type and by host, keep a small number of third-party hosts allowed so the log looks lived-in, and measure what you actually cut. The model below is the decision function a route() handler would use, run against a recorded request log:

from urllib.parse import urlsplit

POLICY = {
    "block_types": {"image", "media", "font"},
    "block_third_party": True,
    "third_party": {"metrics.example", "tags.example", "ads.example", "cdn.assets.example"},
    "first_party": "shop.example",
}

REQUEST_LOG = [
    ("https://shop.example/", "document", 41_200),
    ("https://shop.example/assets/app.css", "stylesheet", 18_400),
    ("https://cdn.assets.example/fonts/inter.woff2", "font", 96_300),
    ("https://shop.example/hero.jpg", "image", 412_000),
    ("https://tags.example/pixel.gif", "image", 1_100),
    ("https://metrics.example/collect", "xhr", 640),
    ("https://shop.example/api/cart", "xhr", 2_300),
    ("https://cdn.assets.example/hero.avif", "image", 288_000),
    ("https://shop.example/preview.mp4", "media", 1_940_000),
    ("https://shop.example/api/checkout", "xhr", 3_100),
]


def decide(url, rtype):
    host = urlsplit(url).netloc
    if rtype in POLICY["block_types"]:
        return "abort", "blocked type: " + rtype
    if host in POLICY["third_party"] and POLICY["block_third_party"]:
        return "abort", "third-party host, unneeded"
    where = "first party" if host == POLICY["first_party"] else "third party, allowed"
    return "continue", where


print("route() policy: block_types={} block_third_party={}".format(
    sorted(POLICY["block_types"]), POLICY["block_third_party"]))
print()
print("{:<38} {:<11} {:<9} {}".format("URL", "TYPE", "ACTION", "REASON"))
print("-" * 96)
sent = blocked = kept_bytes = cut_bytes = 0
for url, rtype, size in REQUEST_LOG:
    action, reason = decide(url, rtype)
    if action == "continue":
        sent += 1
        kept_bytes += size
    else:
        blocked += 1
        cut_bytes += size
    print("{:<38} {:<11} {:<9} {}".format(url.replace("https://", ""), rtype, action, reason))
print()
total = kept_bytes + cut_bytes
print("requests: {} sent, {} blocked of {}".format(sent, blocked, len(REQUEST_LOG)))
print("bytes:    {} transferred of {} ({:.0f}% cut)".format(
    "{:,}".format(kept_bytes), "{:,}".format(total), 100.0 * cut_bytes / total))
print()
if sent < 6:
    print("warning: a load this short is a shape real pages do not have;")
    print("         re-allow one analytics host so the request log looks lived-in")

route() policy: block_types=['font', 'image', 'media'] block_third_party=True

URL                                    TYPE        ACTION    REASON
------------------------------------------------------------------------------------------------
shop.example/                          document    continue  first party
shop.example/assets/app.css            stylesheet  continue  first party
cdn.assets.example/fonts/inter.woff2   font        abort     blocked type: font
shop.example/hero.jpg                  image       abort     blocked type: image
tags.example/pixel.gif                 image       abort     blocked type: image
metrics.example/collect                xhr         abort     third-party host, unneeded
shop.example/api/cart                  xhr         continue  first party
cdn.assets.example/hero.avif           image       abort     blocked type: image
shop.example/preview.mp4               media       abort     blocked type: media
shop.example/api/checkout              xhr         continue  first party

requests: 4 sent, 6 blocked of 10
bytes:    65,000 transferred of 2,803,040 (98% cut)

warning: a load this short is a shape real pages do not have;
         re-allow one analytics host so the request log looks lived-in

Two facts worth taking from that. Cutting 98 percent of the bytes is a large bandwidth and CPU win, and it is why the pipeline is faster with a route handler than without. And four requests is a suspiciously tidy page load, which is why the policy keeps the third-party allow list non-empty and why the analyser prints a warning when the surviving count falls below a threshold. Tune the policy per target rather than globally -- a page whose challenge script lives on a third-party CDN cannot have that host blocked.

The same interception is the recording path. Logging every request and response as a HAR turns a challenge solve into an artefact you can diff between runs: which endpoints the sensor posted to, in what order, with what payload sizes, and which cookie appeared after which response. That is the input to the debugging challenges with recorded traffic lesson, and it is the only reliable way to tell a solve that genuinely worked from one that returned an empty page.

The Audit Routine

Hardening is finished when a measurement says so. Run this after every pipeline change, and keep the output:

1. Dump every client-visible field from the warmed page into a JSON file: navigator, screen, Intl resolved options, WebGL vendor and renderer, the chrome surface's key names, the plugin and mimeType lists, and the performance.memory shape if present.
2. Diff against the profile bundle that was supposed to produce it. A field that is not in the bundle is a field the browser invented, and an invented field is a leak.
3. Run the public detectors and store the JSON, not a screenshot: bot.sannysoft.com, creepjs.com, arh.antoinevastel.com, browserleaks.com. A regression becomes a diff.
4. Check the descriptors for every value you patched, comparing against the stock browser's descriptor for the same property.
5. Check the second-order sources: a worker-thread OffscreenCanvas hash against the main-world one, an AudioWorklet fingerprint against the AudioContext one, and a fresh WebGL context against the patched prototype.
6. Exercise the real flow once, end to end, and assert on the extracted data rather than on a 200.

Step two is the one teams skip and the one that catches the most. Most hardened pipelines are not missing a patch; they are carrying a field that no one decided on, which the browser filled in from the machine it is actually running on.

Checklist

• Every override is installed through add_init_script at context level, before any target navigation, and the payload is idempotent so it is safe in every frame.
• Function.prototype.toString is patched before any function is installed, and every installed function is registered as native.
• navigator.webdriver is removed rather than set to false, and the Navigator.prototype descriptor is restored.
• user_agent and user_agent_metadata come from one object, and a unit test asserts the brands, platform, mobile and platformVersion agree with the UA.
• Locale, timezone, geolocation, permissions, viewport, device scale, touch, colour scheme and reduced motion are context options, not script overrides.
• route() blocks by resource type and host, keeps some third-party noise allowed, and its decisions are logged.
• A post-change audit dumps the full field set, diffs it against the profile bundle, and stores the detector JSON for regression diffing.
• Anything still wrong after the script runs is routed to a patched binary rather than patched harder.

Staying Inside the Line

This is a pipeline for testing systems you own or have written permission to test, and for collecting data you have a legitimate right to collect; the ordering discipline above is identical to what a defender uses to harden their own client-side instrumentation. What it is not for is defeating a paywall, forging another user's credentials, or re-adding traffic from an address you have been blocked from, and none of the consistency work above changes the legality of the target. When a site offers an official API, a partner feed, a licence or a documented rate limit, that route is both lawful and cheaper than maintaining this pipeline; see official APIs and alternative data before you ship another init script.