Live Lookups: Answering From the System of Record in Real Time
A knowledge base is a snapshot. Some questions can only be answered from the live source. We added targeted, on-demand lookups against the institutional system of record - without reopening the door to the bulk crawling that gets automated clients blocked.
Problem Statement
The assistant answers most questions from an indexed knowledge base rebuilt on a schedule. That snapshot is fast and grounded, but it lags reality: a project created or updated after the last rebuild is either missing or described with stale facts (status, dates, the people coordinating it).
Two everyday interactions exposed this. A user pastes a link to a specific record and asks "what is this?" - the snapshot may not contain it yet. Or a user explicitly asks for "the latest" on a project - the honest answer is whatever the source of record says right now, not what the last rebuild captured. In both cases the previous behaviour was a stale answer or a flat "I could not find that."
The naive fix is to crawl the whole source of record continuously so the index is always current. We deliberately did not do this: high-volume automated access is slow, expensive, and the exact pattern that gets an automated client rate-limited or blocked. A human opening one page is never the problem; a client pulling thousands is.
Approach
Targeted, on-demand, single-record. Instead of bulk syncing, the assistant fetches exactly one record at the moment a user needs it - mimicking a person opening a single page. This is the access pattern that has never been a problem, so it stays within normal, courteous limits by construction.
Two explicit triggers, plus one safety net. A live lookup runs only when (1) the user pastes a link to a specific record, (2) the user explicitly asks for the current or latest state of a named item, or (3) the user names a specific record that the snapshot does not contain - in which case, rather than giving up, the assistant checks the live source once before answering. Ordinary questions are untouched and still answered instantly from the local index.
Amortised sign-in. Authenticating to the live source is the slow step. The session credential is obtained once and reused for a window, so only the first live lookup pays that cost; subsequent ones return in about a second.
Grounding and Safety
A live answer is composed strictly from the fetched record and is clearly labelled as fetched live, so the user knows it came from the source rather than the snapshot. If the live source genuinely has nothing for a named item, the assistant declines plainly - it never substitutes a similar-looking record or invents details to fill the gap. The safety net for a snapshot miss therefore either upgrades a dead end into a correct live answer, or leaves an honest "not found" - never a confident wrong one.
Keep the fast path fast. Live access is a deliberate, bounded exception triggered by intent - not a background process and not a per-query default. The common case never pays for the rare one.
Results
| Interaction | Before | After |
|---|---|---|
| Pasted link to a specific record | Stale or "not found" | Live status, dates, coordinators |
| "What is the latest on X?" | Whatever the last rebuild captured | Current state, labelled live |
| Named record missing from snapshot | Declined | Recovered via one live lookup |
| Ordinary questions | Instant from index | Unchanged - no live call |
| First live lookup / subsequent | - | ~13s / ~1s (cached credential) |
The full regression suite (routing, retrieval precision, entity and person matching, extraction) stayed green throughout, confirming the addition did not disturb the existing fast path.
Lessons
- A snapshot and a system of record answer different questions. Most queries want the fast, grounded snapshot; a minority want ground truth as of this instant. Serve both, and be explicit about which one a given answer came from.
- Freshness does not require bulk synchronisation. One targeted lookup at the point of need delivers current data while staying within the access pattern that automated clients are actually allowed.
- Authentication is usually the expensive part of a live call, not the query. Obtain the credential once and reuse it, and "live" stops meaning "slow".
- A live source is also the right fallback for a snapshot miss - but only if the fallback keeps the same honesty rules: recover the answer when the source has it, decline cleanly when it does not, never fabricate.
Discussion