Part 1: AI Budget Build Guide - Local AI Hardware Project ~£1k
In Part 1 of the Local AI Budget Build Project, we build a complete AI workstation without spending an arm and a leg.
In this video I will explain the goals of the project, and walk you through the entire process of build the platform, and getting set up with Linux Mint.
In this video you'll learn:
• The full bill of materials and component choices
• How to build the system step-by-step
• Budget, mid-range, and high-end GPU options
• Common mistakes to avoid during assembly
• How this hardware is designed to work with PatterOS (coming soon)
Link to project repo:
https://github.com/Daniel-Parke/PatterOS
BOM & Build Guide:
https://github.com/Daniel-Parke/Patte...
****************************************
The long term goals of this project are as follows:
• How cheap can we make the final hardware build, while still maintaining sufficient quality and token generation speeds?
• How much can we reduce the required power with GPU tuning whilst maintaining sufficient performance?
• Can we profile the power/energy consumption of local AI hardware relative to model complexity and inference intensity?
• What is the smallest model that can be run to generate reliable and consistent high quality outputs?
• Can we design an OS that is an extension of Linux Mint, that will simplify the installation and set up process for a range of rig configurations (Multi GPU, mixed manufacturers, etc.).
• When everything is said and done, is it worth the bother for users compared to cloud based offerings?
The hypothesis: 1 high quality token/s is infinitely more valuable than 0. Everything after that is just impressive gains with diminishing returns.
The budget build project aims to see how far we can take this concept with local hardware, whilst maintaining an acceptable level of output quality
*****
GUIDES:
To follow this guide and the entire project you will need to install an Ubuntu based Linux distro.
For this project I select Linux Mint - Cinnamon edition:
Linux Mint ISO: https://www.linuxmint.com/edition.php...
To flash a USB drive with the Linux iso you will need to use either Rufus, or Balena Etcher. I prefer Rufus as it works more reliably with old hardware, but Balena etcher will work fine here also.
Download ISO flasher:
Rufus: https://rufus.ie/en/
https://etcher.balena.io/
Flashing the USB drive with Linux is very straightforward, please following the below guides for specific instructions if this is your first time installing Linux:
https://linuxmint-installation-guide....
*****
BOM & Build Guide:
https://github.com/Daniel-Parke/Patte...
AI Rig Parts List:
• Machinist X99 PR9-H mini-ATX motherboard (£47.02)
• Xeon E5-2650 v4 CPU (Bundled with CPU)
• TR-AXP120 CPU Cooler (£22.99)
• SK Hynix 64GB DDR4 RAM (£112.46)
• Micron 512GB NVMe SSD (£36.65)
• RTX 2070 Super / RX 7900 XTX / RTX 3090 Ti (£140 | £665 | £843)
• Corsair RM850e 850W Gold Power Supply (£86.99)
• Open Frame Test Bench (£26.99)
• Artic MX-7 Thermal Paste (£7.39)
• CR2302 CMOS Battery (£3.33)
Total Platform Build Cost for each GPU:
NVIDIA RTX 3090 Ti - £1,187
AMD RX 7900 XTX - £1,009
NVIDIA RTX 2070 Super - £484
****************************************
00:00 Introduction
01:09 Build Components
05:53 Motherboard
07:34 RAM
10:02 NVMe
11:46 CPU Cooler Mount
13:42 Thermal Paste
14:49 CPU Cooler
17:29 Planning
18:06 Power Cables
20:04 GPU
22:00 Removing GPU
22:55 GPU Power
24:22 Final Check + Boot
25:09 Trouble in Paradise
26:00 Admission of our Guilt
26:42 Great Success
27:07 First POST
27:43 Linux Mint Install
33:04 First Boot
35:37 Project Goals