Scalable Load Balancing Having Cloud GPU Service Salad Tutorial With Whisper Transcriber Gradio APP
In this tutorial I am introducing a new cloud GPU provider Salad. They have a very unique system that allows you to both make money by renting your GPU and use their GPU orchestrator system to deploy your own docker image or their pre-configured docker images to do AI tasks such as Stable Diffusion image generation, using LLMs or transcribing audio with Whisper. Since they utilize such orchestrator GPU management system, they are able to provide fully scalable, loading balancing have and with almost 100% uptime cloud GPU inference service.
The GitHub Readme File ⤵️
https://github.com/FurkanGozukara/Sta...
Join me on Salad and use code Y3MEDP for a 2x earning rate bonus! https://salad.com
The Self Developed Free Whisper Transcriber Gradio APP ⤵️
https://github.com/FurkanGozukara/Sta...
Salad Main Site : https://salad.com/
Salad Portal : https://portal.salad.com
Your Salad API Key : https://portal.salad.com/api-key
Salad Documentation : https://docs.salad.com/
Rent Your GPU To Salad To Eearn Revenue : https://salad.com/download
Salad Blog : https://blog.salad.com/
Their on site descriptions are as below:
The Most Affordable Cloud for AI/ML Inference at Scale
Deploy AI/ML production models without headaches on the lowest priced GPUs (starting from $0.02/hr) in the market. Get 10X-100X more inferences per dollar compared to managed services and hyperscalers.
Scale without overspending on cloud
Struggling with high cloud costs, AI-focused GPU shortages & infrastructure management? SaladCloud offers a fully-managed container service opening up access to thousands of consumer GPUs on the world’s largest distributed network.
Built for Inference at Scale
Scale easily to thousands of GPU instances worldwide without the need to manage VMs or individual instances, all with a simple usage-based price structure.
Reduce budget bloat by lowering your AI/ML cloud costs
Reduce Budget Bloat
Save up to 50% on orchestration services from big box providers, plus discounts on recurring plans.
GPU-Driven Processing
Distribute data batch jobs, HPC workloads, and rendering queues to thousands of 3D accelerated GPUS.
Global Edge Network
Bring workloads to the brink on low-latency edge nodes located in nearly every corner on the planet.
Distributed network
Multi-cloud Compatible
Deploy Salad Container Engine workloads alongside your existing hybrid or multicloud configurations.
On-Demand Elasticity
Distribute data batch jobs, HPC workloads, and rendering queues to thousands of 3D accelerated GPUS.
Optimized Usage Fees
Bring workloads to the brink on low-latency edge nodes located in nearly every corner on the planet.
Perfect for GPU-heavy workloads of any type
Scale easily to thousands of GPU instances worldwide without the need to manage VMs or individual instances, all with a simple usage-based price structure.
Language Models
Running Large Language Models (LLM) on Salad is a convenient, cost-effective solution to deploy various applications without managing infrastructure or sharing compute.
$0.12 per Million tokens avg. for TGI
$0.04/hr starting price to deploy own LLM
$0.22/hr to run 7 Billion parameter models
0:00 Introduction to the Salad tutorial
0:30 How Salad provides cheap GPU and scalability and load balancing
1:17 Rent your GPU to Salad to earn revenue
2:14 How to start using Salad cloud GPUs
3:05 How to deploy containers / docker images on Salad
3:15 How to use pre-configured docker images / containers / recipes on Salad
3:46 How to deploy Whisper container / recipe on Salad
4:03 What replica count does and what it means : do automated load balancing and scalability
5:04 How to check status of your deployed containers
5:35 How to deploy very best and most powerful Whisper version to get best and most accurate transcription
5:58 Where to find explanations of pre-configured docker images / recipes / containers
6:34 The free Gradio video and audio transcription software that I have prepared
7:41 How to use the Gradio transcriber app
10:13 The time Whisper Distil takes to transcribe 26 minutes audio speaking file
11:15 How to use Whisper Large V3 on Salad