Hello Everyone! My name is Andrew Fung, in this video, I will be showing you how to automate your everyday web scraping tasks with beautiful soup and schedule it to run on a regular basis with cron jobs on Mac. In this second episode, I will be firstly using the web scraping script that we have written in the first episode, then create a function for sending email notifications to our mailboxes, and finally, setting up cron jobs to run this task periodically and on a regular basis.
Problem statement: I am currently on a waitlist in my uni's hall application, and my waitlist can be found from the university webpage's pdf link, which will be updated at the same time every week. Thus, I will run a cron job to fetch my waitlist position one hour after the weekly update from the web page and send me an email of my new position.
#taskautomation #cronjob #python #webscraping #beautifulsoup
How I make my YouTube videos:
⌨️ Keyboard - Nuphy Air75 Mechanical Keyboard - https://amzn.to/3Xu4PD3
🎙 Microphone - MAONO A04 Professional Podcaster USB Microphone - https://amzn.to/3k8ocD5
🖱 Mouse - Microsoft Bluetooth Ergonomic Mouse - https://amzn.to/3CHhdHJ
🔌 Accessories - Laptop Docking Station for MacBook Pro - https://amzn.to/3CHi5Mv
Installation and Setup!
Replit: https://replit.com
GMail App passwords:
https://myaccount.google.com/
Potential error and solution:
https://stackoverflow.com/questions/2...
Crontab scheduling and naming conventions:
https://crontab.guru
Source code for this project:
https://github.com/Andrew-FungKinHo/Y...
Check out my Github!
https://github.com/Andrew-FungKinHo
Timestamps
-------------------------------
0:00 | Intro
2:58 | Run Part 1 Locally
4:41 | Creating a Virtual Environment
7:50 | Send Email with smtp
15:11 | Setting Up Cron Jobs
24:55 | Out tro
-------------------------------
Feel free to drop a like and comment if you enjoy and video and let me know if you want me to do other types of programming videos ;) !!!