Crawlee for Python: Build reliable crawlers. Fast.
Crawlee is an open-source web scraping and browser automation library developed by Apify. This modern library helps you build reliable crawlers. After the Node.js version, now Python version is also available! Learn how to use Crawlee for Python in your own web scraping projects.
💻 Get started with Crawlee for Python: https://apify.it/4dmAzCd
👨🏻💻 Check it out on GitHub: https://apify.it/3WBIPHq
📚 Read more about the library on our blog: https://apify.it/3Anlyl1
👨💻 Check out the demo repository: https://apify.it/3yGgaZN
🏗️ Reliable crawling
Crawlee for Python won't fix broken selectors for you (yet), but it helps you build and maintain your crawlers faster. When a website adds JavaScript rendering, you don't have to rewrite everything, only switch to a browser crawler. When you later find a great API to speed up your crawls, flip the switch back.
🐍 Python with type hints
Crawlee for Python is written in a modern way using type hints, providing code completion in your IDE and helping you catch bugs early on build time.
🌐 Headless browsers
Switch your crawlers from HTTP to a headless browser in 3 lines of code. Crawlee builds on top of Playwright and adds its own features. Chrome, Firefox and more.
🔁 Automatic scaling and proxy management
Crawlee automatically manages concurrency based on available system resources and smartly rotates proxies. Proxies that often time-out, return network errors or bad HTTP codes like 401 or 403 are discarded.
Follow us 🤳
/ apifytech
/ apify
/ apifytech
/ discord
Timestamps ⌛️
00:00 Introduction
00:19 Getting started
01:05 Proxy Configuration
01:45 Using Playwright
03:14 Give Crawlee a go!
#crawlee #python #webscraping