simpleprog
roadmaps topics blogs news papers
web scraping
HTTP Basics for Scraping Understanding HTML Structure requests, Sessions & Headers CSS Selectors with BeautifulSoup XPath Selectors Extracting with Regular Expressions Parsing Engines and Text Encoding Handling Pagination Sitemaps and Feeds Form Submission and Multi-Step Flows Crawl Strategy and the URL Frontier Scraping JavaScript-Rendered Pages Infinite Scroll and Lazy Loading Real Browser Automation with Playwright Finding the Hidden JSON APIs Debugging with HAR and Mitmproxy robots.txt & Scraping Ethics Legal Considerations Retries, Errors & Timeouts Rate Limiting and Polite Crawling Proxies and IP Rotation Caching and Crash-Resume Parallel Scraping with asyncio Storing Scraped Data Cleaning and Validating Scraped Data Incremental Scraping and Change Detection Downloading Files and Streaming PDF and Document Extraction Logins, Tokens & Long-Lived Sessions Scraping GraphQL APIs TLS & HTTP/2 Fingerprinting When a Site Fights Back Reversing Client-Side Request Signing WebSockets and Real-Time Data The Scrapy Framework Scaling a Crawl Across Workers Scheduling and Monitoring Capstone: Build a Production Scraper
Loading...
simpleprog © 2026
about privacy