The API Behind the App

Many platforms expose far more data to their mobile apps than to their websites. The app talks to a private JSON or gRPC API that returns exactly the structured data you want, with no HTML parsing and often no bot protection—because the developers assumed only their own app would call it. Reverse engineering that API is one of the highest-leverage skills in modern scraping.

Set Up an Interception Proxy

The app lives on a phone or emulator, so you need to see its traffic. The standard tool is mitmproxy (or Charles Proxy / Proxyman), which acts as a man-in-the-middle:

  1. Run the proxy on your machine and configure the device's Wi-Fi to use it.
  2. Install the proxy's CA certificate on the device so TLS can be decrypted.
  3. On Android 7+, user-installed CAs are ignored by apps; either install the CA as a system certificate (requires a rooted device or a modified emulator image) or use an older Android version for testing.
  4. Open the app and watch the requests stream in.

Certificate Pinning

Hardened apps pin their server certificates, meaning they reject anything signed by an unexpected CA—including your proxy. You will see the app fail with a TLS error. Bypassing pinning on Android typically uses Frida to hook the certificate validation functions and force them to accept your certificate; tools like objection automate the common hooks (objection android sslpinning disable). On iOS, pinned apps may require a jailbroken device or a Frida gadget injected into a repackaged IPA.

Reading the Traffic

Once you can see the requests, catalog them:

  • Base URLs and versioning: many apps version APIs in the path (/v3/...).
  • Auth headers: look for bearer tokens, API keys, or custom signing headers.
  • Request bodies: JSON is easy; protobuf (used by gRPC and many modern APIs) is binary. Capture the .proto schema if the app ships one, or decode the bytes field by field using a tool like protobuf-decoder.
  • GraphQL: some apps use a single endpoint with a query body; extract and replay queries directly.

Authentication Flows

Mobile auth often uses refresh tokens with long lifetimes, device registration, and certificate-based client authentication. Watch the login sequence to learn how tokens are obtained and refreshed. Reproducing that flow (or extracting a long-lived token) is usually the crux of building a usable client. Treat tokens as secrets: store them in a vault, rotate them, and never commit them.

Replaying Without the App

With the endpoint, headers, and body known, you can replay requests from any HTTP client. Because you are mimicking a trusted app's network patterns, you often avoid the browser-focused anti-bot layers entirely—but not always, since modern apps increasingly attach attestation tokens (see the anti-bot course). Start with a plain client, observe what the server rejects, and add signing or attestation as needed.

Legal and Ethical Boundaries

Intercepting an app's API is technically the same act as scraping a website, and the same rules apply: public data, reasonable rates, respect for terms and legal boundaries. Do not bypass authentication you do not own, do not extract other users' private data, and keep your request rates polite. The technique is neutral; the use decides whether it is legitimate research or abuse.