Full Machine Learning Project — Processing Raw Data (Part 2)
Want to get started with freelancing? Let me help: https://www.datalumina.com/data-freel...
Need help with a project? Work with me: https://www.datalumina.com/solutions
In this video, we are going to read all the separate raw CSV files, process them, and merge them into a single DataFrame.
👉🏻 Source material for this week: https://docs.datalumina.io/s1eFpcdtbv...
⏱️ Timestamps
00:00 Introduction
00:58 What we will cover in this video
02:00 Downloading the dataset
03:05 Open Visual Studio Code workspace
04:00 Creating the conda environment
08:05 Downloading the Python script
09:10 Understanding the data
14:46 Understanding the data transformation
19:12 Reading a single CSV file
23:29 List all files in directory
26:10 Extract features from file name
35:40 Read all CSV files
47:01 Working with datetimes
55:58 Turn code blocks into function
59:55 Merging DataFrames
01:06:00 Resample data (frequency conversion)
01:18:38 Final check
01:20:06 Exporting the dataset
01:22:04 Conclusion and summary
Project overview (what you will learn)
Part 1 — Introduction, goal, quantified self, MetaMotion sensor, dataset
Part 2 — Converting raw data, reading CSV files, splitting data, cleaning
Part 3 — Visualizing data, plotting time series data
Part 4 — Outlier detection, Chauvenet’s criterion, local outlier factor
Part 5 — Feature engineering, frequency, low pass filter, PCA, clustering
Part 6 — Predictive modelling, Naive Bayes, SVMs, random forest, neural network
Part 7 — Counting repetitions, creating a custom algorithm
Link to playlist: • Full Machine Learning Project: Coding a Fi...
If you find these videos helpful, consider subscribing @daveebbelaar