Keeping a Knowledge Base Current Without a Full-Time Editor

A retrieval assistant is only as good as the documents behind it. We build systems that answer from a company's own material rather than general model knowledge, and the question we get most often isn't about accuracy or latency. It's about who keeps the source documents honest once the project team moves on. Most organisations can't justify a full-time knowledge editor, and the ones that try usually find out the role turns into a backlog nobody clears. Good news: maintenance can be spread around, scheduled, and mostly driven by the assistant's own usage data.

Why Stale Answers Are Worse Than No Answers

An assistant inherits the freshness of whatever it indexes. Nothing in the model corrects a procedure that went out of date two quarters ago. It will summarise the obsolete version fluently, with complete composure. And that's the expensive part. Confident wrong answers cost more than silence, because staff act on them without a second check, and the mistake shows up later in a customer conversation or an invoice.

The failure mode is invisible. Retrieval still returns a well-formed passage, ranking still works, no error appears in any log. Attaching the source document to every answer changes that: the reader sees which file was used, notices it's the old onboarding checklist, and reports it. Attribution turns a silent failure into a reportable one.

Where Documentation Actually Rots

Rot is uneven. Pricing pages, onboarding steps, security policies and API references drift fast, while glossaries, company history and regulatory background sit unchanged for years. Put all of it on one review calendar and you burn effort on the stable material while still missing the fast-moving parts.

Two patterns cause most of the damage:

• Duplicate procedures in different folders. The same instructions exist twice, age at different speeds, and retrieval will happily pick whichever copy scores better - often the older one.
• Superseded documents that were never removed. Nobody deletes. They rename to "v2" and leave both in the index.

So the practical first step is an inventory organised by change frequency rather than by department, with a review interval assigned to each set.

Let the Assistant Tell You What Is Broken

Query logs are a free editorial backlog. What people ask tells you exactly what the knowledge base fails to cover, and collecting it costs nothing. Four signals I watch:

1. Questions where retrieval returned nothing relevant.
2. Answers whose cited source is old relative to its set's review interval.
3. Repeated rephrasings of one question, which usually mean the first answer did not land.
4. Topics with no matching document at all.

Contradictions between two retrieved documents deserve special attention. When the assistant surfaces both, one of them should almost certainly have been retired.

Tip: review the top unanswered queries weekly and fix the three most frequent gaps. That beats attempting a full documentation audit, which teams start and rarely finish.

Ownership Instead of an Editorial Team

Every document set gets a named owner who already works with that material daily. Support owns troubleshooting guides, finance owns billing, engineering owns the API reference. Nobody is being asked to learn new content, only to confirm what they already know.

Owners review on a schedule tied to how fast their set drifts, not on a uniform quarterly calendar imposed across the company. Reviews stay short and binary: still correct, needs an edit, or retire it. That third option carries more weight than people expect - pulling a document out of the index is the fastest correctness fix available, and it requires no writing at all.

Metadata and Pipeline Work That Removes Manual Effort

Some of the burden disappears into the plumbing. Store effective date, owner and next review date as metadata, so freshness becomes something you filter and rank on instead of guessing at. Re-index on change instead of rebuilding nightly, and an edit made in the morning is answerable before lunch. Where you can, sync from the system of record - the wiki, the ticketing system, the repository - rather than uploading copies that fork the moment they land.

Tip: mark superseded documents as archived rather than deleting them. Retrieval skips archived material while the audit trail survives intact.

Deployment location changes the plumbing but not the discipline. Cloud or on-premise, the ingestion job still has to know when a document changed.

A Maintenance Routine That Fits in an Hour a Week

The whole practice compresses into one weekly session:

• Read the week's low-confidence and no-result queries, then group them into themes.
• Check the sources cited most often and confirm the top few are still accurate.
• Route each gap to its document owner as a small edit, never as a documentation project.
• Retire anything that contradicted a newer document during the week.
• Log what changed, so the next reviewer reads the history instead of rediscovering it.

Frequently Asked Questions

Can the assistant simply ignore old documents automatically?

Partly. Freshness can be weighted in ranking, and stale sets can be excluded with date filters. What the system can't work out is whether a two-year-old security policy is obsolete or still binding. Age and validity are different properties. That judgement stays with the owner.

How much documentation do we need before this is worth deploying?

Coverage of the questions people actually ask matters far more than volume. A small, current set that handles the top recurring questions beats a large archive nobody maintains.

Does running on our own infrastructure make maintenance harder?

The editorial work is identical. The difference is operational: self-hosted deployments put the ingestion schedule, storage and access control under your own change process rather than a vendor's.

What This Buys You

Maintenance becomes a routine with a small, predictable cost instead of a periodic rescue project. Source attribution turns every user into a proofreader without asking anyone to proofread. Distributed ownership plus usage signals replaces the full-time editor role most teams can't justify hiring, and the knowledge base stays useful because it gets corrected exactly where it's wrong rather than rewritten wholesale.