Summary
Ever wondered how python web-scraping libraries compare in terms of speed and accuracy? I'll review lxml, html5lib, BeautifulSoup and scrapy with a series of sites evaluating how quickly they can parse pages and how accurately they can find data, particularly pieces of data that render after DOM loading and other pesky bits like hidden form data, internationalized data and mobile-compliant sites.