Build the capability.
Define the boundary.
These are the layers we design around, not a catalog of a hosted product. Every capability needs a test, a permission boundary and a way to stop.
Orchestration
Make the execution path inspectable before making it longer.
What can be built
Represent a task as steps with typed inputs, explicit state transitions, a step budget and a stopping rule. Use a fixed workflow when the sequence is already known; reserve model choice for genuinely variable decisions.
Where it breaks
A fluent plan is not evidence that a dependency exists. Open-ended loops can repeat failed actions or drift from the task. Without a budget, a failure state and a safe recovery path, do not grant autonomous execution.
CHECK → Inject a failed step; confirm that the loop stops or takes the declared recovery path.
Tools & retrieval
Treat every tool call as a boundary crossing.
What can be built
Validate a proposed call against a schema, authorize it outside the model, and normalize its response. Retrieve only from permitted sources and attach document identifiers, versions and evidence spans to grounded answers.
Where it breaks
Valid syntax does not establish permission or truth. Search can return stale or irrelevant evidence. A timed-out write may already have committed: retrying it blindly can duplicate a side effect.
CHECK → Test malformed arguments, missing evidence, access denial and ambiguous write outcomes.
Memory & state
A context window is not a source of record.
What can be built
Separate transient conversation, durable task state and retrieved evidence. Record provenance and freshness. Store checkpoints after confirmed transitions so an interrupted task can resume from known state.
Where it breaks
Summaries lose detail; stale memory can overwrite a newer fact. Parallel runs can conflict without version checks. Never promote a model inference into authoritative state without validation.
CHECK → Resume from a checkpoint with stale evidence; require revalidation before a consequential action.
Human collaboration
Ask at the point where the answer changes the action.
What can be built
Pause for missing scope, conflicting evidence or consequential actions. Show the proposed operation, affected resource and expected side effect. Bind approval to the exact action and current state, with an expiry rule.
Where it breaks
A vague “continue?” is not informed approval. Human review can be overloaded, and an approval can become stale. If the action changes after review, the earlier decision does not authorize the new action.
CHECK → Change an approved action's arguments; confirm that the old approval can no longer be used.
Sometimes the right agent is no agent. If a deterministic workflow covers the task and its exceptions, use it. Model-driven decisions add uncertainty and another thing to evaluate.