Learning to scrape in python and bs4 | A course on scraping websites in python from scratch
Video - a training course on site scraping from scratch in the python programming language.
Let's start with simple things, write a parser using the example of collecting data from an online store and get to the magic of scraping with post authorization, writing parsing data to a table and optimizing the scraper code according to the generator principle.
00:00 python site scraping - introduction
02:37 install libraries pip install requests, beautifulsoup4, lxml
04:04 import libraries into the project
04:57 review of the HTML code of the online store page for scraping
09:15 inspecting the HTML element of the product code
10:45 write code for product scraping
16:28 BeautifulSoup find method
25:07 BeautifulSoup get method
28:00 find_all method scrape all elements of the product catalog page
31:26 scraping all pages of the product catalog
33:38 sleep - pauses between get requests
34:54 masking request headers - headers
37:32 scraping into each product card
55:27 scraper - generator, optimize the code
01:01:02 write scraping result data to exсel table
01:11:42 scraping product pictures (download)
01:12:58 requests get download file
01:22:32 scraping with POST authorization on the site
Links to websites from the video:
Site with product catalog:
https://scrapingclub.com/exercise/lis...
Website with quotes and authorization:
https://quotes.toscrape.com/
Video on working with the requests library:
• Python requests http, https запросы, reque...
Video course on the python language:
• Экспресс курс по python за 7 часов!
Like, subscribe, comment) If you liked the video,
and I will know that such content is of interest to you!