Description
Web scraping is easy to do in Python, but it quickly becomes tedious when routinely running large batch scraping jobs. This talk looks at how to build a distributed web scraper to reduce batch scraping job times and improve durability of your code as well as lessons learned & stories along the way.