Real life data science | accessing prod database, security and project management
Ever wonder how to get insight from a production databases and not just .csv files? In this video I'll show you a simple setup that can get you productive very fast.
Table of Content
- Introduction: 0:00
- Setup Read-Replica: 0:47
- Secure Version Control: 1:48
- Investigate the DB Visually: 3:52
- Ask Questions: 4:59
- Start SIMPLE: 6:01
- Get Partial Results Out: 8:45
- Write a Report: 10:53
- Conclusion: 13:03
Setting up a Read-Replica on cloud providers:
- AWS: • How do I create a read replica for an...
- Google Cloud: https://cloud.google.com/sql/docs/mys...
- Azure: https://docs.microsoft.com/en-us/azur...
Make sure you first secure access to the data. Nothing will get done if you have to wait weeks before getting access to what you need to work.
Then make sure that your version control setup is secure so that you don't leak the read replica database secret into the git repo.
Look at your data. Visualize it using a SQL client and takes lots of notes.
Use your notes from the previous step to ask the relevant people lots of questions. You have to understand as much the business objective as the way the data was collected in a particular way.
Stay lean and iterate. Doesn't matter if your analysis aren't super optimized at the beginning! It's much better to have something working and then refactor when needed than overengineer your data science project. Analysis paralysis is a real thing.
Overcommunicate and iterate rapidly with the stakeholders to orient your analysis. I've never worked on a project that smooth sailed from beginning to the end. There was always course correction along the way and this has to be expected. Data science project have high uncertainty so you need to account for that by reducing the feedback loop.
Write the report from the beginning. Like as soon as you get the assignment or task. Write your assumptions, write your questions, write who to contact for what, write the step you want through and the conclusion you see. Share this report from the get-go with the relevant people and have them interact with it through comments to iteratively make it better.
----
Join the Discord for general discussion: / discord
----
Follow Me Online Here:
Twitter: / codethiscodeth1
GitHub: https://github.com/yacineMahdid
LinkedIn: / yacine-mahdid-809425163
Instagram: / yacine_mahdid
___
Have a great week! 👋