Cheapest Local AI Server?
I bought this motherboard because it was only $150, and it turned into a home lab for Proxmox, GPU passthrough, and local AI experiments. I learned about hybrid CPU and GPU inference with large local models (GLM 4.7, Minimax 2.5 and Qwen 3.5) on a pretty modest setup.
Chapters (timestamps):
00:00 Intro
00:16 The $150 motherboard
01:21 Hybrid LLM inference GPU + CPU
01:42 Supermicro Motherboard unboxing
02:09 Xeon CPU Assembly
02:30 DDR4 ECC RAM Installation
02:54 Silverstone case
03:13 Ik_llama.cpp
03:23 Test setup and model selection
04:31 Benchmark Summary
04:35 Benchmark Bot
05:02 Balance GPU + CPU
05:35 Chosen Model Quants
06:01 ik_llama.cpp common flags
06:15 Model specific flags
06:25 GLM 4.7
06:52 Minimax 2.5
07:18 Qwen 3.5
07:37 GPU Utilization
07:56 CPU and RAM bottleneck
08:34 Active Parameters GLM vs. QWEN
09:04 Recap and what I learned
09:45 Outro and what’s next
Related Videos:
Build Your Own AI Server -
• Build Your Own AI server
Ways to support:
Subscribe -- https://www.youtube.com/c/JunYamog?su...
Affiliate Link Disclaimer:
These are not sponsored, I bought everything myself.
The links below are affiliate links. They won’t cost you anything extra, but if you use them, I may earn a small commission that helps support the channel and future builds.
Some parts that I used for the build with affiliate links:
CPU Tray - https://s.click.aliexpress.com/e/_c3D...
CPU Bracket - https://s.click.aliexpress.com/e/_c4D...
CPU Cooler/Heatsink - https://www.aliexpress.com/item/10050...
Xeon 6154 Gold - https://www.aliexpress.com/item/10050...
Supermicro X11DPi-NT - https://s.click.aliexpress.com/e/_c3h...
Open Chassis Case - https://s.click.aliexpress.com/e/_c3b...
T30 Torx - https://s.click.aliexpress.com/e/_c4o...
Videos Mentioned:
@Level1Techs - AI and You Against the Machine: Guide so you can own Big AI and Run Local
• AI and You Against the Machine: Guide so y...
@juliaturc1 - How LLMs survive in low precision | Quantization Fundamentals
• How LLMs survive in low precision | Quanti...
References:
Ik_llama.cpp - https://github.com/ikawrakow/ik_llama...
Level1Techs forum ik_llama.cpp thread -
https://forum.level1techs.com/t/full-...