CleanStream OBS plugin code walkthrough
In this video, I share a walkthrough of the code for the CleanStream OBS plugin, which cleans live audio streams from unwanted words, utterances, filler words, and profanities. Built in C++, the plugin is an audio filter that uses the Whisper C++ project to run the open AI Whisper speech recognition model. The video covers how the plugin is built, from loading and initializing the whisper context to processing audio from the circular buffer. The video is aimed at those interested in learning more about circular buffers, multi-threading, and using Whisper CPP in their own projects.
Best results with the best audio quality! https://amzn.to/3OocVvj (support my work with this affiliate link :) I personally use the Samson Q2U in my videos and I recommend it. Sponsor my work https://github.com/sponsors/royshil - get value for your team and projects!
Github: https://github.com/obs-ai/obs-cleanst...
Whisper.cpp: https://github.com/ggerganov/whisper.cpp
00:00 Introduction
00:48 Audio filter
01:11 Entry point function
02:20 Introduction to Whisper C++
03:24 Setting up thread for audio buffering
04:10 Input buffer and info buffer
06:12 Cleaning up audio rendering
07:28 Set up and initialization functions
08:48 Resample mechanism for different sample rates
10:17 Update function for parameters
10:39 Destroy function
10:55 Whisper loop thread
13:49 Setting up overlap region
16:07 Implementing VAD
16:56 Running Whisper inference
18:10 Info for frames and timestamps
18:44 Reducing overlap region
20:08 Processing Whisper results
21:25 Detecting fillers and profanities
22:24 Returning detection result
22:46 GitHub repo and conclusion
#obs #obsstudio #obsplugin #cpp #code #walkthrough #audioprocessing #programming #whisper #speechtotext #realtime