Homelab / AI

Wiki Chat

RAG over the wiki and service handbooks — plus read-only tools for RAM, CPU, disk, and uptime when those values are never in the wiki.

Wiki Chat answers questions about the homelab documentation — and keeps two sources clearly apart. Handbook questions come from the indexed wiki and service handbooks, with numbered citations. Questions about the current machine state (RAM, CPU, disk, uptime) go to read-only tools, because those values never live in the pages.

Boundaries: Open WebUI is free-form local chat. The Ops AI showcases are the multi-pass pipeline for patch notes and alert analysis. Wiki Chat stays handbook RAG plus targeted metrics — no blind ops.

Purpose

Two answer paths, kept distinct:

  • Handbook — answer only from the loaded markdown excerpts; citations as [1], [2]. If the context has no answer, the model says exactly «Das steht nicht im Wiki.» Follow-ups use the last three turns; the question is first condensed into a standalone search query.
  • Live state — the language model calls tools (get_host_metrics, list_known_hosts). Numbers come only from the tools; the footer names source and time.

Corpus

The index mirrors the same content as the wiki portal: markdown from the portal repo plus service handbooks from the lab repos (sparse checkout, aligned with the sync catalog). Periodic refresh and incremental re-index: only changed sections are re-embedded. Moved sections keep embeddings via stable content-hash IDs.

Preparation

  1. Frontmatter → metadata (title, service, repo, URL)
  2. Split on markdown headings #, ##, ###
  3. Sub-chunks of 1000 characters with 15 % overlap
  4. Breadcrumb (Title > h1 > h2 > h3) prepended for embedding context

Configuration qwen3-embedding-8b-instr-c1000-k8-hyb:

Building block Choice
Embedding local Qwen3 Embedding 8B — instruction prefix on the query only, documents plain
Store Chroma (persistent index)
Lexical in-process BM25
Fusion Reciprocal Rank Fusion (k = 60)
Top-K 8 chunks

Hybrid search raises gold MRR from 0.740 to 0.808 (376 questions); on numeric questions 0.827 vs 0.678. Ports, unit names, and version strings land better than dense retrieval alone.

Wiki answer

Follow-up condensation with Qwen3.5 9B, then hybrid search, then streaming with Qwen3.6 27B (num_ctx 16384, temperature ~0.1). Optional LLM reranking is off in production.

Measured on 361 wiki questions: accuracy about 4.47 / 5; all 45 negative questions correctly refused. Spec cues («laut Wiki», «needs», «recommended») stay on the handbook path — even when RAM or CPU appear in the sentence.

Tools for live state

Peer to the RAG path. Via LangChain bind_tools the model gets exactly two functions:

  • get_host_metrics(host, metric) — metrics ram, disk, cpu, uptime
  • list_known_hosts — known host names from the wiki service table

At most three tool rounds. The prompt requires: only numbers from the tools; otherwise say honestly that no value is available. If the model makes no call or cannot use tools, a deterministic formatted answer from the same readers follows — invented numbers in neither path.

Access is read-only: fixed Prometheus instant queries, else the Proxmox API. No SSH, no free-form PromQL from the question. Host names come from the wiki inventory, not from the model. The footer names the source (Prometheus or Proxmox API) and the time.

Routing: clear «now / free / currently» goes straight to the tools, skipping wiki search. Weaker live questions only when the best vector hit is below threshold or the wiki says «Das steht nicht im Wiki.»

flowchart TD
  frage[Question] --> verdichtung[Condense]
  verdichtung --> route{Live state?}
  route -->|now or free| tools[Tools]
  route -->|handbook| suche[Hybrid search]
  suche --> chroma[Chroma]
  suche --> bm25[BM25]
  chroma --> fusion[RRF]
  bm25 --> fusion
  fusion --> wiki[Qwen3 with wiki sources]
  wiki -->|not in wiki| tools
  tools --> metrics["get_host_metrics: RAM CPU disk uptime"]
  metrics --> prom[Prometheus]
  metrics --> pve[Proxmox API]
  prom --> liveAntwort[Answer with source and time]
  pve --> liveAntwort

Operations and evaluation

One container (Python 3.12, Gradio), one replica with a persistent index, Recreate because of a writable volume. Image build on git tag into the private registry, rollout via GitOps. Models run on a dedicated Ollama machine, not inside the chat pod.

The Streamlit evaluation UI and the 2D/3D chunk map stay workstation lab tooling: 376 questions (15 live, 272 with a gold page). Live-check verifies host, metric, number with unit, and named source — 13 of 15 on the 26 Oct run.

More context: Ops AI showcases, Observability, Services.