Private Offline AI Chat In Linux In Under 2 Minutes - Open WebUI (Requires Docker)
UPDATE: Here is a shell script that will check if Docker is installed, and will it install it if not. Then it will install the NVIDIA Toolkit followed by Open WebUI:
https://pastebin.com/aAgUHy4v
BASIC INSTRUCTIONS:
Step 1 (Install Docker): https://docs.docker.com/engine/instal...
Step 2 (Install NVIDIA for Docker): https://docs.nvidia.com/datacenter/cl...
Step 3 (Install Open WebUI): https://github.com/open-webui/open-webui
Step 4 (Install Ollama LLMs): https://ollama.com/library
COMMANDS:
curl -fsSL https://nvidia.github.io/libnvidia-co... | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
&& curl -s -L https://nvidia.github.io/libnvidia-co... | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
sudo apt install nvidia-docker2
sudo systemctl restart docker
sudo docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama