When an AI system denies a loan application, cancels a subscription, flags an expense report, or routes a customer complaint to a collections queue, someone will eventually ask: "Why did it do that?" In a regulated environment, "eventually" means "during the next audit." In a high-stakes operational environment, it means "within the hour of the incident."
Teams without decision traces cannot answer this question reliably. They can pull request logs, dig through model outputs, trace back through orchestration code, and assemble a plausible account — but they cannot produce an authoritative, immutable record that shows exactly what the system knew, which rules it applied, and what outcome those rules produced. That gap is the gap between an AI system that is observable and an AI system that is defensible.
Decision traces close that gap. This article explains what they are, what makes them defensible, how to implement them, and how they satisfy the specific technical requirements of three compliance frameworks: the EU AI Act's Article 13, SEC Rule 17a-4, and SOC 2 Type II. It also covers the common implementation mistakes that produce traces that look complete but fail under scrutiny.
What a Decision Trace Is — and What It Is Not
A decision trace is the append-only, immutable record generated when a decision system evaluates inputs against rules and commits to an outcome. It is not a general-purpose log. It is a specific audit artifact with specific required contents.
The distinction matters because teams routinely confuse decision traces with three other log types that are necessary but insufficient:
Observability Logs
Observability logs — request/response pairs, LLM input/output records, token counts, latency metrics — capture what the model did. They answer "what did the model produce?" They do not answer "what rule-governed decision did the system make based on that output?" A model that generates the text "deny this application" has produced an output. The decision to actually deny the application happens at the decision layer, not at the model inference layer. Only a decision trace captures that evaluation.
Event Logs
Event logs capture system state transitions: "subscription status changed from active to cancelled at 14:32:07." They answer "what happened?" but not "why did the system decide to make this happen?" Without the rules evaluated and the inputs assessed, an event log entry is an outcome without a rationale.
Application Logs
Application logs capture infrastructure events: errors, warnings, service calls, performance metrics. They are essential for debugging and monitoring, but they are not structured around the decision-making semantics that compliance frameworks require.
ISACA's 2025 analysis of agentic AI auditing found that the most common failure in AI audit preparation is conflating these log types — organizations believe they have adequate audit evidence because they have comprehensive observability, when in fact they have no decision-layer records at all. The two complement each other; neither substitutes for the other.
The Three Defensibility Properties
A decision trace is legally and operationally defensible when it satisfies three properties. These are not implementation details — they are design requirements that must be established in the architecture before a single trace is written.
1. Immutability
An immutable trace cannot be altered after creation. This sounds obvious, but many logging implementations allow records to be updated, overwritten, or soft-deleted. A trace that can be modified is not evidence — it is a mutable record whose validity can be challenged. Immutability is typically implemented through append-only storage (write-once data stores, cryptographic hash chaining, or write-protected audit tables) combined with access controls that prevent any application code from modifying existing records.
2. Completeness
A complete trace set has a record for every committed decision — there are no "silent" decisions that affect system outcomes without generating a trace. Completeness is harder to achieve than it sounds because consequential decisions in AI systems often happen in multiple places: in orchestration code, in model function calls, in rule evaluations, and in post-processing steps. A trace architecture that only captures some of these decision points is structurally incomplete.
IBM's guidance on trustworthy AI agents for compliance emphasizes that completeness requires explicit architectural design — all decision-making logic must route through a single, instrumented decision layer. Systems where logic is distributed across prompts, code, and API calls cannot achieve completeness without significant refactoring.
3. Replay Fidelity
Replay fidelity means: given the same inputs recorded in the trace and the same rule versions recorded in the trace, re-evaluating the decision system produces the same outcome. This property requires that rules be versioned (so the version evaluated at decision time can be retrieved) and that inputs be captured completely (so the evaluation can be reconstructed).
Replay fidelity is the property that answers "could you show me exactly how that decision was made?" in a regulatory inquiry. Without it, you can show that a decision happened; you cannot show that it was deterministically produced by specific, reviewable logic.
What a Decision Trace Captures: The Minimal Schema
A decision trace needs to capture enough information to satisfy all three defensibility properties. The following schema represents the minimal viable record — each field has a specific purpose and cannot be omitted without degrading one of the three properties.
| Field | Type | Purpose | Defensibility Property |
|---|---|---|---|
trace_id | UUID | Unique identifier for this evaluation event | Completeness (enables gap detection) |
timestamp | ISO 8601 with timezone | When the evaluation occurred | All three (anchors the record in time) |
agent_identity | string | Which agent or system triggered the evaluation | Completeness (establishes who acted) |
input_atoms | array of {name, type, value} | The typed facts evaluated at decision time | Replay fidelity (reconstruct the evaluation) |
rules_evaluated | array of {rule_name, rule_version, outcome} | Which rules fired, at which version, with which result | Replay fidelity (version-pinned rule reference) |
decision_outcome | enum (system-defined) | The committed decision produced by rule evaluation | Completeness (the authoritative result) |
action_taken | string (nullable) | The downstream action executed based on the decision | Completeness (links decision to execution) |
schema_version | semver string | Version of the trace schema used | Immutability (supports forward-compatible parsing) |
This schema is intentionally language-agnostic and implementation-agnostic. It can be stored in a relational database, an append-only log system, a write-once object store, or a dedicated audit database. The storage choice matters for immutability guarantees; the schema matters for defensibility.
What Not to Include in a Decision Trace
Decision traces should not include raw LLM outputs, conversation history, or model reasoning chains. Those belong in observability stores with different retention, access, and governance policies. Decision traces should not include PII beyond what is strictly necessary to identify the subject of the decision — typically a hashed or tokenized identifier, not raw personal data.
Dynatrace's guidance on AI audit trail architecture recommends treating decision traces and observability logs as separate data products with separate governance policies — different access controls, different retention schedules, and different encryption requirements. Conflating them increases compliance risk in both directions: too much PII in audit records and too little decision-layer detail in observability records.
Three Compliance Scenarios
Decision traces are not primarily a technical artifact — they are a compliance artifact with specific uses in specific regulatory frameworks. Three scenarios illustrate how the schema above maps to concrete compliance requirements.
Scenario 1: EU AI Act Article 13 — Transparency for High-Risk Systems
Article 13 of the EU AI Act requires that high-risk AI systems enable human oversight by providing "appropriate information" about how automated decisions were made. In practical terms, this means being able to produce, for any specific decision affecting a natural person, a record of the inputs used and the logic applied.
A complete decision trace satisfies this requirement directly: input_atoms documents what the system knew; rules_evaluated (with version references) documents the logic applied; decision_outcome documents the result. The combination allows a compliance officer to reconstruct the decision path without relying on model explanations, which are inherently non-deterministic. See The EU AI Act's Article 13 Problem for a full breakdown of these transparency obligations.
Scenario 2: SEC Rule 17a-4 — Financial Record-Keeping
SEC Rule 17a-4 requires broker-dealers to retain certain business records in a non-rewritable, non-erasable format for defined retention periods (typically three to six years depending on record type). For AI-assisted financial decisions — trade recommendations, suitability assessments, risk flag determinations — this requirement extends to the decision records that support those actions.
The immutability requirement in a decision trace architecture maps directly to the Rule 17a-4 "non-rewritable, non-erasable" standard. An append-only trace store with cryptographic integrity verification satisfies this technical requirement. The timestamp, agent_identity, and rules_evaluated fields satisfy the "who made the decision and how" documentation requirements that regulators expect during examinations.
Scenario 3: SOC 2 Type II — Evidence of Operating Effectiveness
SOC 2 Type II audits require evidence that stated controls operated effectively over the audit period — typically 6 to 12 months. For AI systems, this means demonstrating that authorization controls, access policies, and decision rules were actually applied to every relevant decision, not just that they were defined.
Decision traces provide this evidence at scale. For any sampled decision in the audit period, the trace record shows which rules were evaluated, that those rules match the controls documented in the SOC 2 description, and that the decision outcome followed from rule evaluation rather than from an undefined process. Galileo's research on AI agent compliance notes that organizations with structured decision traces reduce SOC 2 evidence collection time by a significant margin compared to those relying on general observability logs.
Implementation Considerations
Async Logging for Performance
Synchronous trace writes add latency to every decision. For most systems, the right architecture is to write decision traces asynchronously: the decision evaluation completes and returns its outcome synchronously, while the trace record is queued and written asynchronously with a guaranteed-delivery mechanism. The key constraint is that the trace must be committed before the action is considered "done" — you cannot accept that a decision happened without a corresponding trace existing.
PII Handling
Decision traces will often contain personal data as part of the input ATOMs — account identifiers, transaction amounts, geographic attributes. Treat this data according to your data residency and privacy requirements: use tokenized identifiers where possible, apply field-level encryption for sensitive values, and define retention periods that match the regulatory context (EU AI Act data minimization principles, CCPA retention limits).
Retention Policies
Retention requirements vary by jurisdiction and use case: SEC Rule 17a-4 requires up to six years for certain records; EU AI Act documentation requirements extend to ten years for certain high-risk system records; SOC 2 audit periods typically require 12 months of evidence. Define retention tiers by decision category, and implement automated enforcement — do not rely on manual deletion.
Gaps and Silent Decisions
The hardest implementation problem is ensuring completeness — that there are no decisions being made outside the traced decision path. Implement a completeness check: compare the count of committed actions (subscription cancellations, loan denials, approval grants) against the count of trace records for the same decision type in the same time window. Persistent discrepancies indicate a decision path that bypasses the trace infrastructure. These gaps are not a logging problem — they are an architectural problem that requires routing the bypassing logic through the decision layer.
Decision Traces in Production: The Memrail Pattern
Teams building decision trace infrastructure from scratch face significant implementation work: designing the schema, building append-only storage, instrumenting the decision layer, implementing completeness monitoring, and maintaining the system over time as the rule set evolves.
Memrail's Decision Traces implement this pattern as a managed platform capability. Every evaluation through the Memrail Decision Plane automatically generates a trace record that satisfies all three defensibility properties — immutability, completeness, and replay fidelity — without requiring teams to design or maintain the underlying infrastructure. Trace records include full ATOM capture, version-pinned rule references, and outcome documentation, with built-in retention controls and access governance.
The practical advantage is consistency: traces are generated uniformly for every decision, not selectively where engineers remembered to add logging. The architecture guarantees completeness by design — if a decision is not going through the Decision Plane, it is not going through Memrail, which is itself an architectural signal.
The Practical Test: Can You Answer Five Questions?
The test of an adequate decision trace implementation is whether you can answer five questions about any specific committed decision in your system, for any decision in the past 90 days, within two minutes:
- What were the exact input values (ATOMs) the system evaluated when making this decision?
- Which rules (by name and version) were evaluated, and what did each one return?
- Which agent or system identity triggered the evaluation?
- What was the committed outcome, and what downstream action was taken?
- If you re-ran the evaluation with the same inputs and the same rule versions, would you get the same outcome?
If you can answer all five questions with documented evidence — not inference, not reconstruction, but records — your decision trace implementation is defensible. If any question requires investigation, guesswork, or "I think what happened was," you have an architectural gap, not a logging gap.
For the governance layer that makes these decisions auditable from the start, see The Agent Governance Stack: Four Layers Every Enterprise Needs Before Going to Production. For the vocabulary that grounds decision traces in a coherent system, see ATOMs, EMUs, and the Decision Plane: A Vocabulary for AI Decision Infrastructure.