Current Lab
A September 2026 snapshot: what I run, how it is wired, and what it does.
The short version
Five x86 hosts, an eleven-node ARM device cluster I call the Fellowship, and three GPUs doing AI work in parallel. Home AI, the orchestrator I designed and run, schedules everything, routes each request to the right model, and keeps the lights on.
Alongside the AI stack: a self-hosted media library for friends, Home Assistant with real-time energy monitoring from three UPS units, and a nightly stack-health report to Discord. Everything is version-controlled in local git mirrors first, and github.com/drhawktopus gets the polished pieces once they are ready to be seen.
About a year of applied-AI work on this setup, with the intensive engineering arc in the most recent several months. It is a lab, not an employer's production, and it runs real workloads: training runs, captioning sweeps, adversarial self-review, distributed evaluation. The broader substrate goes back further: system administration, automation, networking, Docker, Home Assistant. The AI half is newer and moves fast.
Compute
Ryzen 7 5800X · Radeon RX 7900 XTX 24 GB · 32 GB DDR4, all-SSD. This is where the heavy work lives: the 32B QLoRA fine-tuning, Flux rendering, and heavyweight reasoning through the 27B tier on wake-on-demand, on a card whose driver-level stall I root-caused. Fast working storage for training runs.
RTX 3070 8 GB, clock-locked at 1500 MHz because its factory boost profile is unstable under sustained compute. Quick-chat host: the 8-9B model for routine questions and tool orchestration, plus the captioning tier. Models unload when idle.
RTX 3070 Ti 8 GB, also clock-locked at 1500 MHz for the same reason. Runs Plex directly and hosts the Hyper-V layer that carries the media-stack VM and the Home Assistant VM as separate guests. Rebuilt from scratch in summer 2025 after I pulled it back from an off-site location.
Small x86 host. Earmarked as a general-compute node; not yet tasked.
x86 desktop running Tolkien, the container that coordinates the Fellowship. It is not limited to that role and takes other work when tasked.
Eleven ARM devices with seventy-six CPU cores between them: seventy-two arm64, plus four armv7 from a legacy device named Sméagol that I brought back to life with a custom armv7 build of llama.cpp. Distributed CPU workloads: OCR, captioning, evaluation sweeps.
The hardware rationale: why three GPUs instead of one big card
Budget and thermodynamics. One 7900 XTX does the heavy lifting but costs more than two 3070s combined. Splitting work across three cards buys parallelism: captioning runs on a secondary GPU while training occupies Mothership, and the third card covers failover. Every GPU host runs the same order-taking runner, so each host is a factory in the lab's multi-factory model: orders serialize per card and parallelize across hosts. A single scheduler places orders, and each factory reports card state, current workload, and health-daemon verdicts.
Both 3070-class cards needed a clock lock. Their factory boost profiles are unstable under sustained compute; I root-caused it, locked both at 1500 MHz, and they pass hour-long soaks cleanly. That trades some throughput for uptime.
Storage
Forty-six terabytes spinning for the main media library and data. All-SSD on Mothership for fast working storage during training and rendering. A twelve-terabyte drive docked at my desk holds second copies of whatever I decide deserves one, plus overflow from Mothership's SSDs, and it is on hand for anything that needs bulk space.
The AI stack
A three-GPU local LLM cluster with tiered routing across a real model roster:
- 8-9B general tier: quick chat, tool orchestration, routine questions. Runs mostly on BigHonker's 3070 (Qwen 3.5 9B). Failover is Mothership with a small-model swap on the 24 GB card; Softserve's 3070 Ti is the absolute last resort because it is also the Plex host. Vision is handled by a separate 8B VLM (Qwen 3 VL 8B). A 14B lands on Mothership when a mid-tier task needs more room.
- 27B heavyweight tier: Mothership's 7900 XTX, wake-on-demand. Qwen 3.6 27B for think-hard work and adversarial review; MedGemma 27B for the nursing-persona critic.
- Deep-think tier: Qwen 3.6 35B-A3B (a mixture-of-experts) and Qwen 3 Coder 30B ladder up for overnight planner and plan-critic passes on the same 24 GB card, with a model swap between phases.
- Embeddings: Nomic-embed-text under RAG grounding.
- External Claude API only on my explicit opt-in. Not the default path.
The orchestrator, Home AI, is a FastAPI service with per-user grounding through RAG over my own maintained knowledge library. Citations are enforced through tool calling. Scheduling across the fleet is deterministic, and a forward-auth gate sits at the edge.
Distributed CPU workloads are placed onto the Fellowship by a capability-aware scheduler. The cluster handles embarrassingly parallel tasks: captioning sweeps, OCR passes, evaluation matrices.
How routing actually works
BigHonker comes first for routine chat and tool orchestration. Mothership is the failover: I wake it for a 14B mid-tier task if BigHonker is busy, or for the 27B heavyweight tier on think-hard work. Softserve is the absolute last resort, because it is the Plex host and stays unburdened unless nothing else is available. The external API is used only when I explicitly ask. Same pattern as the smart-home voice layer: try local before going out.
RAG grounding pulls from the curated knowledge library, and the citation gate means an ungrounded claim is dropped, not invented.
Services and infrastructure
- Self-hosted media library: Plex serves the content. Friends have per-user OAuth-gated access with capability tiers, and a self-service request-and-fulfillment workflow: ask for a title, the system routes it, and you get a notification when it is ready.
- Cloud front door: a DigitalOcean droplet running Caddy, with a Tailscale overlay back to the house, reverse-proxies the public subdomains. Zero residential inbound ports are open.
- Home Assistant: real-time energy telemetry from three UPS units. The smart-home voice layer runs through Home AI.
- Observability: a nightly stack-health Discord report, and an aggregated dashboard on a homepage tile.
- Version-controlled everything: Docker Compose, the Caddyfile, service configs, the dashboard itself.
Recent and current work
- Adversarial self-engineering (Project 12): concluded August 31, 2026. Home AI now writes its own code changes, critiques them internally, disputes its own rewrites, and gates everything through a multi-suite test wall with human approval. The orchestrator suite runs 2,424 tests (two skipped) at the deploy gate and keeps growing; the foundry and the Fellowship head carry their own suites; every deploy has to pass its lane. Phases 0 through 5 run live. Phase 6, the bounded overnight-autonomy loop, is built and validated, and it arms only on a human's say.
- Native Intelligence, the umbrella program behind the AI work on this site: the lab run as one intelligence. Local-first compute placed across the ladder from the ARM cluster to the 24 GB card, self-engineering under human gates, evaluation and review machinery for the models and for the agents, and a training arm that turns the lab's operational history into better local collaborators. Everything but the training arm is live; that arm has not run yet.
- Fellowship distributed compute: the ensemble experiment, nearly through its 1,172 arms, with the appraisal waiting on my time.
- This portfolio site.
The self-engineering pipeline in brief
Project 12 proved an end-to-end loop: Home AI plans a change to its own code, edits the file, runs unit tests and a compile check, creates a checkpoint, deploys through a canary that runs the full orchestrator test gate, waits for my approval, promotes to live, then verifies health endpoints after cutover. The canary either reaches the live stack or rolls back; it is never left half-deployed. I approve each promotion. It is slow by design.
Contact
nickcrowley97@gmail.com, or github.com/drhawktopus for the code.