scrapy

Scrapy framework for web scraping and crawling at scale. Build spiders, extract data, and manage crawl pipelines. Use for large-scale scraping, data extraction, or building web crawlers.