There is a recurring pattern in how teams build AI agent systems. They choose an orchestration framework -- LangGraph, CrewAI, Autogen, or a custom graph implementation -- and then ask it to do two jobs at once: coordinate what the agent does (sequencing, routing, tool calls, state transitions) and govern what the agent is allowed to do (policy enforcement, authorization, audit, rollback). The framework does the first job well. It was not designed for the second.
The result is a system where orchestration and governance are fused into a single layer. Routing logic contains authorization checks. Graph edges encode business rules. Conditional nodes implement policy. The system works -- until someone asks a question the framework cannot answer: "Which policy was active when this agent denied the refund?" or "Can we roll back the authorization change without redeploying the agent?" or "Who approved the rule that allows agents to escalate directly to billing?"
These are governance questions. Orchestration frameworks do not answer governance questions, for the same reason that a process scheduler does not answer questions about access control. The concerns are architecturally distinct. This article draws the precise line between them.
What an Orchestration Layer Does
An orchestration layer coordinates the execution of an agent's workflow. It manages the sequence of operations, routes control flow based on intermediate results, handles tool invocations, maintains state across steps, and recovers from transient failures. It answers the question: "given the current state, what should happen next?"
LangGraph, currently the most widely adopted agent orchestration framework, implements this as a directed graph. Nodes represent computation steps (LLM calls, tool invocations, data transformations). Edges represent transitions between steps. Conditional edges route control flow based on the output of a node. State is maintained across the graph traversal as a typed state object that nodes can read from and write to.
This is a powerful abstraction for coordination. It handles the complexity of multi-step agent workflows: branching on model output, parallelizing tool calls, retrying on failure, checkpointing for resumability. Temporal's durable execution model provides a complementary perspective: orchestration is about making workflow execution reliable in the face of failures, timeouts, and distributed system complexity. It is infrastructure for coordination.
What orchestration is not -- and what no orchestration framework claims to be -- is a governance system. The orchestration layer knows how to route. It does not know whether the route it is taking is authorized. It knows how to call a tool. It does not know whether the agent should be permitted to call that tool in this context. It knows how to sequence steps. It does not know whether the business rule embedded in step three is the current version or whether it was superseded last Tuesday.
What a Decision Plane Does
A decision plane is the architectural layer where all consequential logic that could affect a governed outcome is evaluated, enforced, and recorded. It answers a fundamentally different question from the orchestration layer: not "what happens next?" but "is what is about to happen permitted, and under what terms?"
The decision plane has four responsibilities that the orchestration layer does not and should not carry:
1. Policy Evaluation
The decision plane evaluates policies -- named, versioned rules that specify the conditions under which an action is permitted, denied, or constrained. When an agent is about to issue a refund, the decision plane evaluates the refund policy against the specific facts of the request (account tenure, refund history, amount, reason code) and returns a determination: allow, deny, or allow with constraints. The orchestration layer does not evaluate policies; it receives the decision plane's determination and acts on it.
2. Authority Management
The decision plane manages who is authorized to do what -- including which agents, which tools, and which human operators have authority over which actions. Authority is not routing. An orchestration layer can route a request to the refund tool, but the question of whether this agent, in this context, with this customer's account, is authorized to execute a refund is an authority question that belongs in the decision plane.
3. Audit Record Generation
Every evaluation through the decision plane generates an immutable decision trace: a record of what was evaluated, which policy version was applied, what facts were assessed, and what determination was produced. This audit record is generated at the point of policy evaluation, not at the point of action execution. The distinction matters because the orchestration layer may take actions that the decision plane did not evaluate -- those ungoverned actions are invisible to the audit record.
4. Independent Rollback
Policies in the decision plane are versioned independently of the orchestration code. A policy change can be rolled back without redeploying the agent, without modifying the graph, and without touching the orchestration layer. This independence is a design requirement, not a convenience. When a policy change produces unexpected behavior in production, the rollback path must be faster than an engineering deployment cycle. If rolling back a policy requires a code deploy, the policy is not in the decision plane -- it is in the application.
The Conflation Problem: What Happens When You Merge Them
When orchestration and governance are merged into a single layer, four specific failure modes emerge. Each one is a direct consequence of asking the orchestration framework to carry governance responsibilities it was not designed for.
Failure Mode 1: Business Rules Encoded as Graph Edges
A LangGraph conditional edge that checks if account.tenure > 90 and refund.amount < 500: route_to_auto_refund is a business rule masquerading as a routing decision. It works -- the edge routes correctly. But the rule is now embedded in the graph definition. It cannot be tested independently of the graph. It cannot be versioned independently of the application code. It has no owner (who owns a conditional edge?). It generates no audit record of its evaluation. And when the refund policy changes from $500 to $750, someone must find this edge in the graph code, change it, and redeploy the agent.
This is the most common conflation pattern, and The Gradient's analysis identifies it as the root cause of governance failures in agent systems: teams embed policy in orchestration because it is the fastest path to working behavior, and then discover that the policy is invisible, untestable, and ungovernable.
Failure Mode 2: Authorization in Application Code
When authorization checks live in the orchestration layer, they take the form of if/else statements scattered across the codebase. An agent's tool-calling node checks if agent.role == "billing_admin": allow_refund(). This check works, but it is not a governance control -- it is application logic that happens to implement authorization. It cannot be audited centrally. It cannot be queried ("which actions is the billing_admin role authorized to perform?"). It cannot be changed without a code deploy. And when someone adds a new tool node without the authorization check, the omission is invisible -- there is no enforcement point that catches the gap.
Failure Mode 3: No Decision-Level Audit Trail
Orchestration frameworks produce execution logs: which nodes fired, in what order, with what outputs. These are observability records, not governance records. They tell you what the agent did. They do not tell you what the agent was permitted to do, which policy governed the permission, and whether the action was within the terms of that permission. When a regulator or auditor asks "show me the policy that governed this agent's decision," an execution log is not an answer.
a16z's analysis of the emerging agent infrastructure stack identifies the governance layer as architecturally distinct from the orchestration and memory layers precisely because it must produce a different kind of record. Execution logs and decision traces serve different consumers, answer different questions, and require different data.
Failure Mode 4: Coupled Rollback
When a business rule embedded in the orchestration layer produces unexpected behavior, rolling back the rule means rolling back the orchestration code. This is a deployment operation, not a policy operation. It affects everything in the deployment, not just the rule that changed. It requires engineering involvement. It takes minutes to hours instead of seconds. And in the meantime, every agent instance is operating under the incorrect policy.
A properly separated decision plane allows rule rollback in seconds, independent of the orchestration layer, without affecting any other aspect of the agent's behavior. The rollback is a policy state change, not an application deployment.
The Architectural Boundary in Practice
The clean boundary between orchestration and governance can be described by a single principle: the orchestration layer asks the decision plane for permission before taking any consequential action. The orchestration layer handles everything about how to execute; the decision plane handles everything about whether to execute.
In practice, this means that every consequential action in the agent's workflow -- issuing a refund, sending a communication, modifying an account, escalating to a human, invoking a tool with side effects -- passes through an enforcement point. The enforcement point is the boundary between the two layers. It is the place where the orchestration layer pauses, presents the facts of the situation to the decision plane, receives a determination, and proceeds (or halts) accordingly.
| Concern | Orchestration Layer | Decision Plane |
|---|---|---|
| Core question | "What happens next in the workflow?" | "Is this action permitted under current policy?" |
| Typical implementation | LangGraph, CrewAI, Temporal, custom DAG | Policy engine, rules repository, decision platform |
| State managed | Workflow state, tool outputs, conversation context | Policy versions, authority grants, evaluation history |
| Change frequency | Engineering sprint cadence (weeks) | Business cadence (days, sometimes hours) |
| Change mechanism | Code deploy | Policy promotion (draft, shadow, canary, active) |
| Rollback speed | Minutes to hours (redeploy) | Seconds (version revert) |
| Audit output | Execution trace (what happened) | Decision trace (what was permitted and why) |
| Testability | Integration tests requiring full agent run | Unit tests against policy rules with typed inputs |
| Ownership | Engineering team | Business owner per policy domain |
This table is not describing two competing systems. It is describing two complementary layers that serve different purposes, change at different cadences, and are owned by different stakeholders. They must both exist. Merging them produces a system that is coordinated but ungoverned.
A Concrete Example: The Refund Agent
Consider an AI agent that handles customer refund requests. The orchestration workflow is straightforward: receive the request, retrieve account data, assess the request, determine the action, execute the action, notify the customer.
In a conflated architecture, the "assess the request" node contains the refund policy as conditional logic: check tenure, check amount, check refund history, determine eligibility. The logic is in the graph. It works until the refund policy changes, at which point someone edits the node, redeploys the agent, and hopes the change is correct.
In a separated architecture, the orchestration graph handles the workflow sequence. When the graph reaches the "determine the action" step, the orchestration layer calls the decision plane with the relevant facts: account tenure (typed integer), refund amount (typed currency), prior refund count in the last 12 months (typed integer), account standing (typed enum). The decision plane evaluates the current active version of the refund policy against these facts and returns a determination: allow, deny, or allow with constraint (e.g., "allow as credit, not cash refund"). The orchestration layer acts on the determination.
The difference is not in the outcome -- both architectures can produce the same refund decision. The difference is in what happens after:
- When the refund policy changes, the separated architecture updates the policy in the decision plane without touching the orchestration code.
- When an auditor asks what policy governed a specific refund, the separated architecture produces the exact policy version and the facts it evaluated.
- When the new policy produces unexpected behavior, the separated architecture rolls back the policy in seconds without redeploying the agent.
- When the compliance team wants to review all active refund policies, the separated architecture provides a queryable policy repository -- not a codebase search.
Why Orchestration Frameworks Cannot Add Governance Retroactively
A natural response to this argument is: "why not just add governance features to the orchestration framework?" Some teams attempt this by building policy-evaluation nodes within their LangGraph graphs, or by adding authorization middleware to their orchestration pipeline. This approach produces something that looks like governance but lacks its essential properties.
The problem is architectural, not feature-level. Governance requires that policies be managed as first-class entities: named, versioned, independently testable, independently deployable, and independently auditable. An orchestration framework manages workflows as first-class entities. Adding policy evaluation as a node in a workflow does not make the policy a first-class entity -- it makes it a subordinate component of the workflow.
The practical test is simple: can you change a policy without changing the workflow? Can you test a policy without running the workflow? Can you audit a policy without reading the workflow code? Can you roll back a policy without redeploying the workflow? If any answer is no, the policy is not in a decision plane -- it is in the orchestration layer wearing a governance costume.
Temporal's durable execution model illustrates this principle from the orchestration side. Temporal deliberately separates workflow definitions (the sequence of activities) from the activities themselves (the actual business logic). The workflow orchestrates; the activities execute. This separation is what makes Temporal workflows durable and replayable. The same principle applies to governance: the orchestration layer orchestrates; the decision plane governs. The separation is what makes governance auditable and independently manageable.
Integration Patterns: How the Two Layers Communicate
The integration between orchestration and decision plane follows one of three patterns, depending on the latency requirements and the complexity of the policy evaluation.
Pattern 1: Synchronous Evaluation (Most Common)
The orchestration layer calls the decision plane synchronously before executing a consequential action. The call includes the typed facts relevant to the policy, and the response includes the determination and the trace identifier. The orchestration layer blocks until the determination is received. This is the simplest pattern and is appropriate when the decision plane evaluation adds less than 50ms of latency -- which it should for any well-implemented policy engine evaluating structured rules.
Pattern 2: Pre-Evaluation at Workflow Start
For workflows where the set of possible actions is known at the start, the orchestration layer can request a batch evaluation from the decision plane before the workflow begins: "for this account, in this context, which of these actions are permitted?" The decision plane returns a set of determinations, and the orchestration layer uses them throughout the workflow without additional calls. This pattern reduces latency at the cost of freshness -- if a policy changes mid-workflow, the cached determinations may be stale.
Pattern 3: Asynchronous Evaluation with Enforcement Queue
For high-throughput systems where synchronous evaluation would create an unacceptable bottleneck, the orchestration layer can submit evaluation requests to a queue and proceed with a default determination (typically "deny" for safety-critical actions, "allow" for low-risk actions). The decision plane evaluates asynchronously and, if the determination differs from the default, triggers a correction. This pattern is appropriate only for actions that are reversible; for irreversible actions, synchronous evaluation is required.
The Organizational Dimension
The architectural separation has an organizational corollary. When orchestration and governance are fused in a single layer, the engineering team owns everything: the workflow logic, the business rules, the authorization checks, and the audit records. Every policy change requires an engineer. Every rule update requires a deploy. Every audit question requires someone who can read the code.
When they are separated, the ownership model changes. The engineering team owns the orchestration layer -- the workflow definitions, the tool integrations, the state management. Business stakeholders own the policies in the decision plane -- the refund rules, the authorization grants, the escalation criteria. Each policy has a named owner who can read, test, and propose changes to their policies without touching application code.
This is not a theoretical benefit. It is the practical consequence of making policies first-class entities. When the VP of Customer Success needs to change the escalation threshold for enterprise accounts, the change happens in the decision plane, goes through draft-shadow-canary-active promotion, and is live in production without an engineering sprint. When the compliance officer needs to review all active policies that govern financial decisions, the decision plane provides a queryable inventory -- not a codebase grep.
The goal is not to remove engineering from governance. It is to make governance a first-class organizational concern that does not bottleneck on engineering deployment cycles. The architecture enables the organizational model; the organizational model justifies the architecture.
Building the Boundary
If your agent system currently has business rules embedded in orchestration code, the path to separation is incremental. You do not need to rebuild the agent. You need to identify the policy decisions embedded in the graph and extract them.
Start with the three most consequential decisions your agent makes -- the actions where a wrong outcome would cause the most damage. For each one, identify the facts the decision evaluates, the conditions it checks, and the outcome it produces. Write those as explicit, named rules in a format that can be versioned independently of the orchestration code. Build the enforcement point: the place in the orchestration graph where the workflow pauses and calls the decision plane. Route the three decisions through the enforcement point. Log every evaluation.
You now have a decision plane -- small, focused, and governing the most consequential actions. From here, expand. Extract more rules. Add more enforcement points. Build the safe rollout pipeline. Assign policy owners. The boundary between orchestration and governance becomes clearer with each extraction, and the system becomes more governable with each decision that moves from application code to the decision plane.
Your orchestration framework is good at orchestration. Let it do that job. Governance is a different job, with different requirements, different stakeholders, and different operational properties. It deserves its own layer.