eGPU on Mac feat. TinyGPU using UV
#cuda #usb4 #rtx #rtx4070super #applesilicon #arm64 #adobe #premiere
In this video I'll show you how to install tinygrad library using uv package manager. Tinygrad allows you to use NVIDIA CUDA through eGPU connected to Thunderbolt 4/USB 4 40Gbps port using UT3G board.
I used uv with Python 3.14 arm64 version..
@MacSoundSolutions Spoiler Alert: NVIDIA eGPU isn't detected outside of tinygrad library. The doc has varied slightly after Alex Ziskind video went up. Basically you need to copy or make directories with tinygrad library imported based on GPU make and it isn't easy as it sounds because there's a huge delay in running CUDA kernel or LLMs and pretty slow than bare- metal linux docker setup or Apple MLX which is wicked fast comparatively.
I spent literally 4 days to debug an easy way but there's none at the moment.
Links:
NVIDIA eGPU on Mac: https://docs.tinygrad.org/tinygpu/
Astral UV: https://docs.astral.sh/uv/
Install UV: https://docs.astral.sh/uv/getting-sta...
Brew Package Manager: https://brew.sh/
Minimal Docker Install without GUI: https://dev.to/mochafreddo/running-do...
Docker CLI + Docker Desktop GUI: https://www.docker.com/products/docke...
OrbStack (Docker Desktop GUI Alternative) Native Apple Silicon (Free for personal use): https://orbstack.dev/
Colima (Open Source Docker alternative for cmd line usage): https://github.com/abiosoft/colima
Timestamp:
00:00 Start
00:14 If it throws tinygrad library is too old with uv venv
00:38 Copy the python commands from shell script
00:47 Use uv run to execute commands
00:55 Allow/Enable TinyGPU.app Extension
01:19 TinyGPU Driver Extension is Installed Successfully
01:24 Download CUDA 12.8 Docker Image
01:32 Install Docker Desktop or OrbStack and Running in Background
01:39 Be Patient with Docker Build & Container Setup
01:56 Process is Completed (2mins to Setup)
01:59 Add to PATH
02:05 ENV Parse Issue
02:15 Run the local LLM CLI
02:42 Downloads Recommended GGUF File(s)
02:48 Finally Worked!
02:52 Spoiler: TinyGPU Doesn't Work Outside of Tinygrad Library
03:07 Outro
03:16 End (Not Sponsored by Steve Austin 3:16)