How to Speed up HTTP Requests Using the Requests Session Object
In this video we demonstrate how to retrieve links from an index of Pybites articles.
Initially, the script, which makes synchronous requests calls, takes around 16 seconds to fetch ten articles.
To address this, we refactor the code to use the Requests Session object. This adjustment cuts down the retrieval time from 16 seconds to just 6 seconds 😎
The beauty of this tip lies in the fact that it significantly boosts performance while keeping the code simple and without the need to switch libraries 💡 😍 (although we would love to dive into httpx at some point as well ...)
This video highlights the benefits of the Session context manager for speedier web requests. Stay tuned for more insightful tips ...
Chapters:
00:00 Cache file with blog article links
00:16 The need for headers
00:42 Running regular requests.get + Unix time utility
01:10 Making it faster with requests.Session() object
01:28 It's a context manager = we use it with "with"
01:43 Running changed code + more than 2x faster
01:53 Performance gain with little extra code
Requests docs:
https://requests.readthedocs.io/en/la...
---
Learn more about Pybites / what we do: https://pybit.es
Get our weekly developer / mindset emails: https://pybit.es/friends/
Join our Slack community: https://pybit.es/community/
And last but not least, we appreciate any feedback to make our YouTube content better 💡
https://form.jotform.com/230941798741062 🙏