Scraping Indeed.com With Python Scrapy (2022)
In this video, we look at how you can scrape data from Indeed using Python Scrapy. We look at how to search for a specific job name & location, crawl through the search results and then parse the associated job data.
The article that goes along with this video:
https://scrapeops.io/python-scrapy-pl...
Basic Scrapy project used in this tutorial:
https://github.com/python-scrapy-play...
Completed Scapy Project which you can just download and run (after adding your account details!!): https://github.com/python-scrapy-play...
00:00 - Intro
00:39 - Download & setup the Scrapy project
02:10 - Setup the proxy to by-pass indeed anti-bots
04:20 - Looking at how/where the data is rendered on the page
07:55 - Creating our spider to scrape Indeed
15:12 - Running the full spider & viewing the results
18:15 - Outro