How Often to Reindex, and What Breaks When You Forget
An index is a snapshot. It captures your documents at the moment they were embedded, and nothing after that. The assistant answers from that snapshot, never from the live source, so every reply carries the accuracy of whenever the last sync ran. So reindexing frequency isn't really a technical setting. It's a promise about how old an answer may be, and I like to write that promise down before anyone touches a schedule. Once it's a contract, you can negotiate it with the business instead of leaving it as a hidden engineering default.
What Actually Breaks When the Index Falls Behind
Staleness is quiet. That's the problem. A superseded procedure gets quoted confidently, because the retrieved chunk still exists and still looks authoritative, complete with a citation. Pricing, policy and version numbers hurt most. People act on those immediately.
- Stale answer, valid citation: the source link works, the content moved on.
- Ghost documents: deleting a file changes nothing until the index learns about it.
- Broken source links: renamed or moved files produce citations pointing nowhere, which destroys trust faster than a wrong answer does.
- Duplicated chunks left behind by a partial run.
- Silent coverage gaps where new documents were never ingested at all.
Let the Document, Not the Calendar, Set the Cadence
We group sources into tiers by two things: how fast their content moves, and how expensive a wrong answer would be. Release notes, price lists and incident procedures need near-continuous sync. A handbook rewritten once a year? Not even close.
- Event-driven for systems emitting change webhooks
- Nightly for content management systems and shared drives
- Weekly for stable handbooks
- On-demand for archives
Tiering keeps compute proportional to risk. Tip: record the cadence in the source registration itself, so nobody has to remember which corpus runs on which schedule.
Incremental Sync Versus Full Rebuild
Incremental sync compares a change signal and touches only what moved. That's what routine schedules should use. Content hashing beats modification timestamps here, and it isn't close - timestamps flip when nothing meaningful changed, which inflates cost and churns embeddings for free. And deletions need explicit handling. A pass that looks only for new and modified files will never remove anything. Ever.
Full rebuilds belong to structural changes: a new embedding model, different chunking, a revised metadata schema, or a corpus you no longer trust. Never mix embedding models inside one index, because vectors from different models simply are not comparable. Upgrading the model means rebuilding, not patching.
Rebuilding Without Taking the Assistant Down
Build into a new index, verify it, then switch the pointer. A failed embedding run then never leaves users querying a half-populated corpus. Keep the previous index alive until the replacement has served real traffic - rollback is the only cheap fix for a bad chunking decision, and you will want it eventually. Cloud deployments make parallel indexes trivial. On customer-owned infrastructure it's a different conversation: disk and GPU budget for two copies has to be planned ahead. Embedding throughput and rate limits govern how long a rebuild takes, so measure it once against the real corpus and treat that number as a constraint, not an estimate.
Tip: run every rebuild against a fixed set of known questions, comparing answers and cited sources before and after. Chunking regressions surface there rather than in front of users.
Instrumentation That Tells You the Index Is Stale
Log how old each cited chunk was when it was served, then alert on the tail, not the average. Averages lie about this. Compare source document counts against indexed counts regularly too, because a widening gap means ingestion is failing quietly. Record every sync run with per-source outcomes - otherwise a connector that's been erroring for weeks stays invisible. And a spike in no-context answers usually means new terminology arrived before its documents did.
- Last successful sync per source
- Documents added and removed
- Failed items, with reasons
- Oldest chunk served in a real answer
Where Ownership Belongs
Reindexing fails as a purely engineering concern. The people who know a document is obsolete are the ones who wrote it. Give content owners a way to trigger a resync for their own sources instead of filing a ticket and waiting. Deletion policy needs a named owner too - someone has to decide whether archived material stays retrievable or vanishes from answers. In regulated environments, being able to state exactly which document version produced an answer matters as much as the answer itself.
FAQ
Does reindexing everything nightly solve the problem?
It hides it. Full nightly rebuilds burn embedding budget, still leave up to a day of staleness on urgent content, and mask which sources are genuinely failing to sync.
How do we know a document was removed from the source system?
Only by comparing the full source listing against the index. New-and-modified checks cannot detect absence, so plan a periodic reconciliation pass.
Do we need to reindex after changing the prompt or swapping the answering model?
No. Prompts and answering models sit above retrieval. Reindexing is required only when the embedding model, chunking or metadata schema changes.
What We Would Decide First
Start from the freshness promise per source tier, then pick mechanics that meet it. Don't choose an interval and hope. Use content hashes, handle deletions explicitly, keep one embedding model per index, stay able to roll back. And measure staleness where it hurts - the age of chunks actually served - not where it's convenient to measure. Forgetting doesn't give you a slow assistant. It gives you a confident one, citing something that stopped being true.
