Teaching an answer tool to say "I don't know"

The first time the answer tool did its job well, it also did the one thing we most needed it not to. We'd built a tool that lets an AI agent ask questions about a business and get answers back — hours, services, prices, the stuff a customer wants before they show up. We connected it to a test business, pointed an assistant at it, and asked something the profile didn't actually contain: whether the place took walk-ins on weekends.

It answered. Fluently, warmly, plausibly. And completely made up.

Nothing in the profile said a word about walk-ins. The model just knew how a place like that usually works, and told the customer with total confidence. If that had been a real shop, a real person would have turned up on a Saturday expecting something the owner never offered — and blamed the shop, not the AI.

That test is why the tool's single most important behavior is refusing to answer.

An LLM's default setting is "always answer"

These models are trained to produce a fluent continuation. Ask one almost anything and you get a confident paragraph — that's the product, and for most uses it's the point. For a tool that answers factual questions about a specific business on the owner's behalf, it's a liability, because the failure mode here isn't "no answer." It's a wrong answer delivered with the same warmth as a right one. Stale hours. A price from last year. "Yes, they do that," when they don't. And the customer acts on it.

The bar for this kind of tool isn't "usually right." No business can run on usually-right facts that a stranger is about to act on. So at some point the design problem flipped. It stopped being "how do we make this answer well" and became "how do we stop it from answering when it shouldn't."

Cite the field, or don't say it

The fix is grounding, and the specific move that made it work was making the tool cite its source. It doesn't answer from the model's general knowledge of the world. It answers from the business's own structured facts — the hours, services, prices, and policies the owner actually filled in — and it names the field it used. "Open until 6 on Saturdays, from your hours." If the relevant field is empty, there's nothing to cite, and the honest output is that it doesn't know.

Two useful things fall out of that. The model's job gets smaller: it's rephrasing known facts, not recalling the world, which is exactly the task LLMs are reliable at. And every answer becomes checkable — the owner can trace a reply back to the field it came from. A grounded answer you can audit is a fundamentally different object than a confident guess that happens to be right.

(Questions arrive as untrusted text, so the input gets fenced off from the tool's own instructions — a cleverly worded question shouldn't be able to talk it into freelancing. That's its own post.)

"I don't know" is a product surface, not a failure

Most teams treat the not-found path as an error state: a shrug, an apology, a dead end. For this tool it's a first-class output, and it's designed on purpose.

When a fact isn't on file, the tool says so plainly — no apologetic paragraph pretending to be helpful anyway — and then does the useful thing. It offers to take the question as a message the owner can answer directly. For a small business, that trade is obviously the right one. A fabricated answer costs you a customer quietly, and you never learn it happened. A captured question — "someone asked whether you do walk-ins" — is a lead and a to-do in the same object. The refusal isn't the end of the conversation. It's a handoff.

That reframe did more for the design than any prompt tweak. Once "I don't know" is a real destination with a next step, you stop trying to squeeze an answer out of thin facts, because the honest path is no longer a loss.

The refusals are the most useful data you have

The last piece surprised us. Every question the tool couldn't answer gets logged — not just that it failed, but what was asked. Those unanswered questions turned into a feature: a running list, part of the Lodestar dashboard, of exactly what agents asked that the profile couldn't cover.

It closes a loop we didn't plan for. The gaps tell the owner precisely which facts to add next, ranked by what's actually being asked. You don't sit down and guess at your FAQ; the misses write it for you. A limitation became a feedback signal — the tool's failures are a structured record of demand the business isn't meeting yet.

That's the part we'd most want another builder to take. If your assistant refuses cleanly, its refusals are data. Throwing them away is the waste — not the refusal.

What generalizes

Strip out the business and the pattern holds for most grounded assistants — over your docs, your product catalog, a customer's records.

Underneath all three is one reframe worth keeping: a grounded tool's quality is as much about what it refuses to say as what it says.

“The demo that impresses people is the answer. The product they can trust is the "I don't know."”