Every organization running a legacy system eventually hits the same wall: the code works, but nobody can say with confidence what it does anymore. The people who wrote it are gone, the specs are stale or contradictory, and the “documentation” is scattered across dozens of partial write-ups that don’t agree with each other. Ask an LLM to just summarize it all and you get a confident-sounding document that quietly drops the parts it couldn’t reconcile, or worse, blends two contradictory sources into a single plausible-but-wrong sentence.

That’s not a hypothetical for me — I built this against a real legacy billing system: 56 batch components plus 85 inbound services, decades of PL/SQL and Java, with existing functional docs that were “decent but leave a lot to the implementation.” The generic version of the problem is one every enterprise with a legacy estate recognizes: audits, onboarding, M&A due diligence, and compliance reviews all need functional documentation that’s actually faithful to the source, not just fluent.

The tool is a pipeline (`python -m pmdocs <stage>`) that goes from raw source to a queryable knowledge base in eight resumable stages: stage0 builds deterministic “grounding” catalogs (every PL/SQL callable, every opcode used, every table) so later steps aren’t hallucinating structure; stage1 extracts a technical read of each component straight from the actual code; generate turns that into a business-altitude functional doc; judge runs an adversarial groundedness check (GO / REVIEW / REJECT) on every doc before it’s trusted; translate produces the Italian delivery copy; ingest merges each doc into a shared wiki of functional-area pages; and query answers a question by reading the wiki and producing a cited synthesis. Every stage records what it finished, so you can stop and resume instead of re-running from zero — on this project that meant processing 56 components, then later adding 85 more services and 96 more components without starting over.

The part that actually makes this trustworthy rather than just fluent: every sentence in the wiki carries a Provenance table row saying which source it came from, and instead of silently picking a side when two sources disagree, the wiki writes `[CONFLITTO: … fonti: A vs B]` and `[SME-REQUIRED: …]` markers into the page and routes them to a human. On the real run this caught something a plain summarizer would have missed or smoothed over — two components disagreeing on a tax-rate value, one of them silently producing empty tax subtotals in production. That’s a real defect surfaced by the documentation process itself, not a documentation nicety.

There’s also a full-screen terminal app (`python -m pmdocs triage`, built in Textual) for the humans who have to close those gaps. It has two tabs: Triage, which lists every open conflict and open question across the wiki (1,268 of them on the full run), lets you jump straight to the marker line in an inline editor, and records a structured, audited decision (`d` to decide, `Ctrl+S` to save, with every resolution logged to an append-only audit trail); and Query, where you type a question in Italian, watch a live-progress overlay while the agent reads the wiki, and get back a cited answer you can render straight to HTML in your browser.

The hardest problem turned out not to be generating the documentation — it was making it *updatable* without becoming stale. Merging sources into a coherent page is lossy: once you’ve blended three sources into one clean sentence, you can no longer tell by reading it which source said what. I wrote out the mental model as “the wiki is cooked food, the sources are raw ingredients” — and the fix is that provenance is the thing that buys back updateability, because it lets you grep for exactly which pages a changed source touched instead of re-cooking the whole wiki. The trap underneath that: conflict markers and cross-links are derived from *multiple* sources, so a naive provenance grep alone misses them — you have to explicitly chase that “blast radius” or you leave stale disagreement flags sitting on pages that no longer disagree.

The second lesson was about what actually makes an LLM-as-judge trustworthy for this kind of work. It’s tempting to bolt on “have Claude score the output” and call it a quality gate, but self-preference bias (a model over-rating its own output), verbosity bias, and a bias toward fluency over faithfulness mean an ungrounded judge just measures whether the prose reads well — the opposite of what a functional doc needs. The two transformations here (code → functional description, and many documents → one coherent synthesis) needed two separately-anchored judges, and cheap deterministic checks (does every claim carry a source? is every source represented?) caught more real errors than the LLM judge did.

The obvious next step is the “update” operation on top of the wiki — I designed and partially built the add/revise/retract semantics (a worklist builder that finds a changed source’s footprint via provenance plus its blast radius) so a source revision becomes a bounded, surgical re-ingest instead of a full rebuild. That distinction — “update when you can point at the damage, rebuild when the damage is everywhere” — is a general operating principle for any org running a synthesized knowledge base that has to survive contact with a changing reality.

At team or organization scale, this pattern generalizes past one legacy system: any group maintaining institutional knowledge from heterogeneous, sometimes-contradictory sources — compliance documentation, integration inventories, onboarding material for an acquired codebase — needs the same three things: grounded generation instead of unconstrained summarization, an explicit mechanism for surfacing disagreement rather than silently resolving it, and a human-in-the-loop triage flow so SMEs spend their limited time only on the parts the system is actually unsure about, not re-reviewing everything. The SME-trigger rate — how often the pipeline needs a human — is itself a useful KPI: as prompts and grounding improve, it should fall, and that number is a more credible signal of system quality than any single “looks good” review.