Raspberry Pi 5 RAM 8GB ollama benchmark for running local LLMs
Raspberry Pi 5 RAM 8GB ollama benchmark for running local LLMs
Ollama needs to be installed first.
https://ollama.ai
ollama-benchmark needs to be downloaded
https://github.com/aidatatools/ollama...
🔗 Author links: https://linktr.ee/jasonchuangai
⏱️ Timestamp
0:00 Introduction
0:45 Installation of ollama-benchmark
3:05 Check models
5:20 Benchmark models explaination
5:55 Run benchmark
8:59 Throughput rate of mistral:7b model (1.43 tokens/s)
9:12 Throughput rate of llama2:7b model (0.93 tokens/s)
9:22 Throughput rate of llava:7b model (0.336 tokens/s)
9:34 Thank you