Multithreaded Web Scraping With Pagination Using Java and JSoup With CSV as Input and Output
For this demo, we'll be scraping the Top 100 books from the Amazon Web Store with pagination for the following categories
Best Sellers
New Releases
Movers and Shakers
Input is from a CSV file listing the URLs to be scraped.
Scraping is multithreaded. Each URL from the CSV input will be independently processed.
Output is also a CSV file for each category containing the scraped data formatted into the desired columns.
i.e.
| RANK | TITLE | AUTHOR | PRICE |