Juan Riaza - Dive into Scrapy
[EuroPython 2015]
[21 July 2015]
[Bilbao, Euskadi, Spain]

Scrapy is a fast high-level screen scraping and web crawling
framework, used to crawl websites and extract structured data from
their pages. It can be used for a wide range of purposes, from data
mining to monitoring and automated testing.

In this talk some advanced techniques will be shown based on how
Scrapy is used at Scrapinghub.

Goals:

- Understand why its necessary to _Scrapy-ify_ early on.
- Anatomy of a Scrapy Spider.
- Using the interactive shell.
- What are items and how to use item loaders.
- Examples of pipelines and middlewares.
- Techniques to avoid getting banned.
- How to deploy Scrapy projects.

свернуть