Contribute Media
A thank you to everyone who makes this possible: Read More

Web scraping: Reliably and efficiently pull data from pages that don't expect it


Exciting information is trapped in web pages and behind HTML forms. In this tutorial, you'll learn how to parse those pages and when to apply advanced techniques that make scraping faster and more stable. We'll cover parallel downloading with Twisted, gevent, and others; analyzing sites behind SSL; driving JavaScript-y sites with Selenium; and evading common anti-scraping techniques.


Improve this page