Local AI Homelab
A personal systems environment for running, monitoring and comparing local AI models across several devices.
Objective
My aim was not simply to run a few local language models. I wanted to manage devices with different operating systems from one place, compare GPU workloads, keep remote access secure and apply changes in a way that could be reversed when necessary.
System structure
The laboratory consists of a MacBook for administration and development, a Raspberry Pi 400 for central control tasks, two desktop computers and a Linux laptop. The devices communicate over a private Tailscale network; general services are not exposed directly to the public internet.
Custom commands bring together device status, GPU utilisation, Docker services, network connections and power management. Before applying changes, I use a preview, explicit confirmation and a defined way to reverse the change when necessary.
Model evaluation
I do not compare local models by speed alone. The evaluation also considers code generation, structured output, source-based responses, language and reasoning tasks, response stability, latency and resource use.
Evaluations are carried out as structured model comparisons. Task definitions, result files and model versions are documented.
My contribution
I designed the network structure, device roles, command structure, monitoring system and model-evaluation approach. I operate the environment day to day, managing hardware, Linux, containers, networking and AI components as one system.
Outcome
The laboratory can be managed remotely without exposing general services to the public internet. Two desktop computers serve as GPU compute nodes. Monitoring, temperature control, power limits and tools for running and evaluating models are part of the active system.
Technologies
Linux · macOS · Docker · NVIDIA Container Toolkit · Ollama · Open WebUI · Tailscale · Python · Bash · systemd · Beszel · Uptime Kuma
This project is not publicly accessible. In an appropriate professional discussion, I can present an anonymised network diagram, monitoring views and the model-evaluation approach. Real IP addresses, device and user names, email addresses and network identifiers are concealed in published screenshots.