Home AI: the orchestrator that runs my lab
A local AI orchestrator in front of the GPU fleet and the ARM cluster: named components, capability-matched scheduling, per-user grounding, and a short list of safety invariants that every autonomous behavior has to satisfy.
Home AI (HAI) is the FastAPI orchestrator that runs my lab. It is the one door into all of the compute: the x86 GPU factories and the ARM cluster I call the Fellowship. Chat, grounded retrieval, tool calls with citations, and the longer-running work all go through it. It runs in my home lab, and as of September 2026 it serves me and a small circle of friends; the only way in from outside is the front door described on the friend onboarding page. It is a working system that I keep changing in the open, on the lab it runs. Its code stays private, because it carries my hosts, my users, and my configuration; the public pieces of the lab are the training recipe and the upstream driver report.
The architecture separates concerns by named component: Kirk (the orchestrator), Spock (the scheduler), Tolkien (the head of the ARM cluster), a coding runner sandbox, and a hostops broker. Every autonomous behavior binds to an explicit human-approval gate, and violating any standing invariant counts as a critical defect even when every test passes. The AI and compute map draws the whole arrangement on one page, and the lab overview map places it among the hosts.
Named components
The orchestrator itself. A FastAPI application that owns chat, grounding, the tool loop, user identity pinning, and the top of the routing decision. This is what a user talks to in the chat interface.
The scheduler. It decomposes an order into steps and places each step on the factory whose declared capabilities match what the step requires.
The head node of the Fellowship, the ARM cluster. Kirk and Spock see the cluster as one placement target; Tolkien alone distributes work inside it.
A sidecar container that owns the coding workspaces and the durable task, project, decision, and checkpoint stores. It is isolated from the live orchestrator, so generated code cannot reach the running system without an explicit human deploy.
A privileged external container that runs host-level operations and the self-deploy state machine. It is reachable only from inside the orchestrator over an authenticated header, and it is the surface that actually lands deploys on live hosts. The host-administration tools the AI uses on registered hosts, the ops agent, are an extension of this broker.
The naming is structural, and it names software, not hardware: the hosts keep their own names. Star Trek names go to the components that run the GPU fleet (Kirk the orchestrator, Spock the scheduler). Lord of the Rings names go to the ARM cluster and its head (Tolkien over the Fellowship). Different substrate, different naming family, readable from the name alone.
Multi-factory scheduling
Each factory declares its own capabilities to the scheduler through a push endpoint, so the factory is the authority on what it can do right now, not a static table. A step that needs a particular model's weights, the ARM pool, or the coding sandbox goes to whichever factory declared that capability. A step no factory can satisfy is held, never misrouted. Placement is capability-first but not capability-only: deferrable work also passes an energy-price gate, so non-admin deep-think runs are held when electricity is above a ceiling and heavy jobs target the cheapest hours, detailed on the energy page.
The same order-taker contract runs on every factory: accept an order from a queue directory, claim the lane, publish card state and a heartbeat, resume idempotently, honor pause and priority, retire the order by every exit path, and count failures as failures instead of writing them up as success. The Fellowship implements the same contract, which makes it a factory by construction and not a special case.
Default routing: quick chat on the 8-9B tier, heavy think-hard work on the 27B tier, and Softserve, the media host, reserved for last resort so it stays unburdened. The current lab, Fellowship, and QLoRA pages cover the fleet topology and the training runs.
Grounding and the tool loop
HAI grounds every answer against a curated library of about 2,600 chunks, and the library grows with the work. Grounding is per user: different users are authorized to different corpus namespaces, and each user sees their own conversation history and notes as retrievable material.
The tool loop enforces citations. When a claim depends on a source, HAI has to produce that source through a tool call, not paraphrase it from memory. Retrieval is per chunk, with the first heading of each source document prepended to every chunk, which turns document titles into an editorial lever over what the model reaches for.
Consent-scoped tool surfaces
Some tools touch a subject's data, the render tool that uses trained adapters among them. Those pass through a per-subject consent check on every request, and the check fails closed. The gate, its scope model, and why it exists are on the QLoRA page.
The control surface
The chat interface presents five controls. Three of them set request parameters; the other two are presentation and a launcher. The point is that the user chooses the cost of an answer: a quick question stays on the small tier, and hard work asks for the big card on purpose.
| Persona | Which hat HAI wears: general, nursing, or law. Nursing exposes a sub-mode selector and pins the model tier. |
| Model | Auto (the smart cascade), the quick 8-9B tier, or the 27B tier forced. Forcing the 27B is a per-user permission. |
| Effort | Normal, hard (the think-hard reasoning path), or absurd (a detached maximum-budget run for hard work that can wait hours). Hard and absurd both run on the 27B heavyweight tier and both answer a user's request. The self-engineering loop is a different lane, on the deep-think models listed on the current lab page. |
| View | Presentation only. It controls whether tool trails and reasoning journals open or stay collapsed, and it touches no request parameter. |
| Plan | The launcher for the planner, a separate flow for structured multi-step work. It sets no request parameter of its own. |
The query contract stayed byte-identical across the interface redesigns: the first three controls set request parameters that already existed, and the other two add nothing to the request. The panel is a rendering, not a new interface.
Safety invariants
HAI ships with an explicit invariant file that the self-engineering loop reads on every review pass. The loop and the human deploy gates enforce them. They are a discipline, not a proof of safety.
- Identity pinning. Every endpoint that acts on per-user data pins non-admin users to their authenticated identity header. Self-asserted fields are trusted only on the LAN.
- Human-only approval. Nothing HAI writes may create, approve, or modify backlog work items. The approved backlog is a read-only mount, and overnight proposals go to a separate file that never runs without a human.
- Sandbox-only overnight. Overnight coding happens in sandboxed workspaces with the deploy and live-apply tools mechanically stripped. Nothing reaches the live system without a human deploy through the hostops broker.
- External APIs are opt-in. No code path may call an external commercial LLM API without per-use human approval.
- Existing tests stand. Tests cannot be weakened, renamed, or deleted unless the work item names them for edit.
- Secrets stay out. Never read, log, or commit environment files, tokens, or webhook URLs.
- Fail toward safety. On ambiguity, a guard or gate failure resolves to "flag a human," never to silent success.
- Remedies need consent. Stopping, requeuing, or changing factory work requires an approved remedy proposal. The human's yes is per proposal and re-verified at execution time.
- The librarian reflex. Real work is not finished until it is written down. Journal appends and handoff notes are the only library write paths: no sensitive material, no speculation stated as fact.
Deployment lifecycle
The self-deploy state machine runs in the hostops broker, outside the orchestrator being replaced. It verifies against a whole-tree mirror hash, builds and brings up a canary copy on a separate port while the live orchestrator stays untouched, runs the pinned regression suite (2,424 tests, two skipped, and growing), then waits for my approval before promoting to live. Auto-rollback fires on a canary or live verification failure. Deploys are slow by design, and every promotion needs explicit consent. The whole loop is drawn on the self-engineering map.
Observability
A health endpoint reports service status, including the current self-engineering phase, which decides what kinds of overnight work are eligible. The homepage dashboard renders a tile per factory and per running order. Each factory publishes card-state heartbeats with structured fields (state, detail, ETA, step, total steps, seconds per step), so a tile can show "step X of Y" without parsing free text. Cross-factory alerts are host-prefixed so two boxes never get confused for each other.
Engineering detail: rolling-TTL claims, the explicit-null trap, and idle-factory publishing
Rolling-TTL claim. A factory's declared state ages out if the factory dies. Capability declarations are pushed, not written to a file, because a file cannot free its own card when the process holding it dies and a rolling-TTL claim can. When the push stops, the lane goes free on its own.
Explicit-null trap. A GPU-side telemetry daemon once had every push rejected for eleven hours while a hand-rolled probe from the same box returned success. The daemon shipped an explicit null in a field with a non-null default; the probe omitted the key. Every push-endpoint field has a concrete default, omitting a field uses it, and sending an explicit null fails validation. Proving my own probe works proves nothing about the real payload.
Idle factories must still publish. Capabilities used to ride only on during-run pushes, so a factory idle since before the feature existed reported zero capabilities, the scheduler saw nothing, and no work could be placed there: unroutable exactly when it was available. The fix was explicit idle publishing with an empty run name.