Prompts teach patterns, not facts

Teach the model how to read signals. Let changing facts arrive when the task runs.

Section
Fa (法)
Written
Length
1,159 words
Hexagram
夬
111110
A checksum, not a fortune
Source
Markdown
为学日益,为道日损。Dao De Jing, chapter 48. Learning adds a little each day; the Dao takes a little away.

I want prompts to teach a way of finding out. Facts that can change should arrive when the task runs. Put them in permanent instructions and you have created another copy of the world to maintain.

I kept making this move in my workflow engine: take a fact out of the prompt and give it a runtime source. Which tools exist? What language is the user using? Is a service connected for this request? Each has an owner better placed to answer than a paragraph written earlier.

“Available” is doing too much work

Suppose a prompt says a tool is available. It might mean either that the product supports it or that this particular request can use it successfully. Those are different claims. The first can remain true after the second stops being true. The model still receives an unqualified assertion.

A tool list creates a similar problem. If the prompt contains the catalog, adding a capability requires a prompt edit. The backend may already know about the tool, but the model has to wait for another description to catch up. That is a maintenance dependency we introduced ourselves.

Language needs a more careful distinction. Asking the model to use the user’s language is a behavioral instruction. Declaring which language this user is using is a statement about the current request. I would keep the instruction and supply the current information separately. Hard-coding the languages I happen to have considered can restrict the cases I have not considered.

My test is simple: would this sentence stop being true if the user, connection, or tool catalog changed? If so, I want to know who updates it. Better wording cannot compensate for putting that responsibility in the wrong place.

Give the fact an owner

The sources are function results, request context, and catalog schemas. These are different ways to get information to the model when it needs to act.

A function result is what a program returns after doing something. To judge whether a connection works or an operation succeeded, read the result from the program responsible for checking or executing it. A tool’s existence should not count as evidence that its latest call worked.

Request context is information accompanying this particular task. It can carry the user’s language and the available connections. The same prompt should be able to make different decisions from different context without acquiring a separate set of permanent answers for each user.

A catalog schema describes tools through structured fields: what they are called, what they do, and what inputs they accept. The model learns to read that structure. New capabilities can then enter through the catalog instead of becoming another paragraph of product documentation in the prompt.

None of this makes a signal automatically true. A check can fail. Information can be stale. I want missing information to remain visibly missing, rather than have the model fill the gap with an old assertion from its instructions. Knowing where an answer came from gives us a chance to judge whether it supports the next action.

What the prompt still has to teach

Removing facts does not remove the need for instructions. The model still needs to understand which information applies to this task, how to interpret a tool’s input requirements, and what to do when a result is absent or reports failure.

That is what I mean by a pattern. If the catalog specifies inputs, prepare those inputs. If the result says an operation failed, do not report completion. The relationship belongs in the instructions. The current field names belong with the tool that defines them.

In Forge, my workflow project, the design document puts constraints into schemas and validators. Schemas define the shape of data; validators check whether it meets the contract. The prompt stays thin, with examples. A requirement that a structural check can enforce should not depend entirely on the model remembering a request.

There are limits to what that design establishes. Forge’s philosophy document also lists retry behavior and approval capabilities that still need work. Teaching the model to recognize failure does not establish that recovery has been implemented. The signal, the decision, and the available action each need an account of their own.

My local prompt guidelines suggest a practical editing test: remove a rule and see whether an actual evaluation fails. An evaluation here means checking task outputs against requirements. Text does not earn a permanent place merely because someone once added it.

Examples need room to generalize

Few-shot prompting means giving the model a small set of examples to learn from. When correcting prompts built around individual cases, I have emphasized that examples must generalize. They should not turn into questions constructed around answers we already know.

I want an example to show a relationship. The connection state changes, so the decision changes. The catalog changes, so the model looks for a suitable tool again. A fixed tool paired with a fixed answer may teach an association while leaving the useful reasoning unstated.

Adding every bad output and its preferred answer can turn the prompt into an answer book. I would rather change the tool name, language, or input and ask whether the guidance still works. Success on the original question is not enough to establish that the model learned the intended pattern.

An example also cannot certify the current task. It may demonstrate how to interpret a successful result. It cannot establish that this operation succeeded. The example supplies a method; the current result supplies the evidence.

The work moves somewhere

There is a cost to this approach. Someone has to maintain the catalog, expose connection state, and define what happens when information cannot be obtained. We trade repeated instruction edits for engineering that supplies usable state. That work does not disappear because the prompt became shorter.

Fetching information can also add delay and consume context. Checking everything before every small decision would be excessive. I would retrieve what the task needs and treat incomplete information as an expected condition. I cannot prescribe a useful refresh interval for every task.

Some tasks arrive with complete inputs that will not change during execution. Put those inputs in the request. Adding an external lookup to obey a slogan serves no purpose. My concern is with assertions that decay inside long-lived instructions; facts absolutely belong in the model’s context when it needs them.

I also distinguish fixed constraints from open judgment. A required data format belongs in a structural check. A judgment that benefits from examples and reasons still needs those examples and reasons. I do not want either category forced into the other’s machinery.

I cannot yet claim that the latency and maintenance tradeoff pays off for every task. But I can inspect a prompt sentence by sentence: does this teach the model how to read a signal, or does it preserve an answer that may already have changed? The latter sends me back to the system responsible for knowing.