Your AI assistant doesn’t hand you the best thinking available. It defaults to the most-published.
The mechanism is a corollary to Larman’s Laws of Organizational Behavior. Original thinkers write slowly and rarely. Displaced managers become coaches, coaches become consultants, consultants become prolific content producers, and that derivative layer outwrites the primary sources by something like an order of magnitude. Models train on the written record, so they inherit the ratio. Ask a general-purpose model how to run an incident review or structure a hard decision, and the default answer is the consultant-frequency consensus, not the sources you’d stake a real decision on. Web search doesn’t fix it by itself. It samples the same distribution: more language, not more truth.
I’ve open-sourced grounded-forge, my working answer to that problem. It’s also a hands-on example of the principle I build everything around: put the LLM where it earns its place, and stay deterministic where consistency, auditability, and cost win.
What it does
You pick the sources you trust. An LLM reads each one in full, not chunked, under a structured nine-pass protocol that produces verbatim-cited references and pre-projects each source onto every task your assistant serves: decision-making, stakeholder engagement, after-action review. A source-only audit checks every claim against the text before anything ships.
At runtime the model doesn’t re-project raw chunks onto your task. It picks a pre-projected, pre-audited artefact and answers from that bounded, cited material. That significantly reduces the room to hallucinate, and the source-only audit that gates every claim before it ships is, in practice, quite good at catching what would otherwise slip through. Not zero, but a different order of risk from a model reshaping fragments under time pressure. You pay for the synthesis once, under audit, and every query after that is selection rather than re-derivation.
That’s the whole bet. The expensive, error-prone work is reading the source and projecting it onto your task. Standard retrieval does that again on every query, from raw chunks, with the model reshaping fragments under time pressure. grounded-forge moves it to ingestion time, does it once under discipline, and reads the result at runtime.
Where it wins, and where it doesn’t
The evaluation results are in the repo, including where the architecture loses.
On canonical public material with clean filenames, a naive corpus read wins and the matrix is pure overhead. In the evaluation, tidy author-and-topic filenames gave the naive read enough routing signal to win outright. The likely reason is that the name keys a strong training prior, so the model routes in one read and spends the rest on content, while the projection machinery re-derives what the prior already holds. I ship that as a bounded claim, not a blanket one.
The matrix earns its keep where your organisation’s real knowledge lives: engagement documents, incident histories, your own frameworks, the material no training run has seen and no model can route without help. There the training prior has nothing to invoke, and the curation does the work. A second result pointed the same way. The lens layer, which reweights a distillation through a role or a methodology stance, beat its lensless variant by 0.6 rubric points on the same non-public prompt, in the direction the lens was designed to move. Directional, on a small sample, and reported as such.
One honest caveat on the headline number. The 99.4%-clean audit figure measures the internal consistency of the source-only protocol, the same model family auditing its own output, not independent verification. It’s a discipline check, not a truth oracle. It makes the assistant’s fidelity to the chosen source inspectable; it doesn’t establish that the source is right about the world. That judgement stays yours.
For builders
The architecture is a reference × task matrix. One axis is your sources. The other is the tasks your assistant actually does. Each cell is a distillation: one source projected onto one task, carrying verbatim quotes and evidence markers in-band, so the load-bearing claims arrive already cited. Retrieval becomes a lookup into that grid rather than a reshape of chunks.
The demo corpus ships 28 sources across five task axes, 119 distillations in all. It isn’t 28 × 5: some source-task pairs are deliberately skipped where the projection would add nothing the matrix doesn’t already carry elsewhere. Those 119 cells compile into five distributable assistants from one build command. Two of them, decision and stakeholder, are built from the identical source library and the identical runtime indexes; they differ only in their task distillation directory and the CLAUDE.md that briefs the assistant for that task. diff -rq between the two apps returns exactly those three lines. That’s the matrix made concrete: one corpus, many task readings, each shipped as its own assistant.
Fork it, strip the demo content in one command, and ingest your own corpus. The reference tier is the audit-of-record and stays at corpus level; the compiled app ships only the distillations and the routing indexes it needs. It runs in Claude Code today. On licensing, the substrate (build system, scripts, skills) is MIT, the long-form prose I wrote is CC BY 4.0, and each source-derived reference inherits its own source’s licence, stamped in the file’s frontmatter. The corpus is plain markdown throughout, so an operator who walks away takes their files and goes. Nothing in the design locks the corpus to me or to any one vendor.
Ingestion runs with a 9-pass protocol: here’s the tl;dr:
The organisational work
Many AI programmes start at the coder-tooling layer and stop there. Roll out a chatbot, wire up autocomplete, call it done. The harder and more valuable work sits above that layer. It’s deciding what your organisation treats as authoritative, and building the machinery that holds the line when a model would otherwise reach for the consultant-frequency mean.
Pick your sources. Project them onto your work. Make the assistant read through your windows, not the training distribution’s.
If you want to build on it, the repo is here: https://github.com/chrisgagne/grounded-forge.
Leave A Comment