Asia/Shanghai
BlogMay 27, 2026

Designing AI Workflows with Context

How to decide what an AI system should know, remember, retrieve, and ignore
Li Zimu
Designing AI Workflows with Context
AI features often look like prompt design from the outside. In practice, the more important work is context design. Context is the material the model uses to reason. It can include the user's request, previous actions, retrieved documents, tool results, system rules, product state, and temporary task memory. Adding more context is not always better. The useful question is: what does the model need to know to make the next decision correctly? Durable intent is what the user is trying to accomplish. Task state is what has happened so far. Those two should not be treated as the same thing. For example, "prepare a deployment checklist" is durable intent. "Nginx config already passed validation" is task state. The system should preserve both, but use them differently. Durable intent helps the system stay aligned. Task state helps it avoid repeating work or skipping a required step. Retrieval should be specific. Dumping a large document set into context can make the model less reliable because the important signal is surrounded by noise. A useful retrieval layer usually considers:
  • What question is being answered?
  • Which source is authoritative?
  • How recent does the source need to be?
  • Is the content safe to expose to this task?
  • Does the retrieved material need citations or traceability?
Retrieval is not just search. It is part of the product's trust model. When a model calls a tool, the result becomes new context. That context should be compact, structured, and easy to inspect. If a tool returns a huge unstructured blob, the next model step has to infer too much. If the tool returns clear fields, statuses, and errors, the workflow becomes easier to control. Tool design is context design with an API boundary. Some information should be left out:
  • Secrets and credentials.
  • Irrelevant historical messages.
  • Internal implementation details the model does not need.
  • User data outside the current permission scope.
  • Low-confidence retrieved content.
Ignoring information is not a weakness. It is how the system keeps the model focused and the product boundary clear. Users may never see the context window, but they feel its quality. They feel it when the system remembers the right thing, forgets the irrelevant thing, cites the right source, chooses the right tool, and recovers from a failed step. That is why I treat context design as part of product engineering, not just prompt writing.
Share this post: