There is a question that ends most AI governance demos. Not the hard philosophical one about consciousness or alignment. The brutally practical one: "Take any specific decision your system made eighteen months ago and reproduce it — same verdict, same reasoning, same governance record — offline, without calling your API." Most teams cannot. Not because they lack monitoring. Not because they do not care about compliance. But because the thing that was deployed was never designed to be reproduced. It was designed to be fast. Reproduction is an architectural property, not a logging afterthought.
This article is for the internal audit and model risk audience that is now being handed AI systems to validate and is realizing that "we have audit logs" is not the same answer as "the decision is reproducible." It explains precisely what replay means, why probabilistic systems structurally cannot provide it, what engineering properties make it possible, and how auditors use it once it exists. The worked example is EVE CoreGuard's Verify portal — a live demonstration that offline replay is not theoretical.
What "replay" means precisely
Same inputs plus same policy version equals same verdict, regenerable on demand
Replay is not a synonym for "we kept a log." A log is a record of what the system reported about itself. Replay is something more demanding: given the same inputs that governed a specific decision and the same versioned policy that was active at the time, the evaluation engine produces an identical verdict — and this process can be executed entirely offline, without calling back to any vendor system, months or years after the fact.
The definition has three load-bearing words. Same verdict — not approximately the same, not in the same direction, but bit-for-bit identical (ALLOW, BLOCK, or MODIFY). Versioned policy — the exact policy pack that evaluated the decision, pinned and immutable, not whatever the current policy happens to be today. Offline — the verifier has no dependency on a live service call. If the vendor's API is unavailable, the replay still works. If the vendor goes out of business, the replay still works.
Replay = (stored inputs digest + pinned policy version) → (re-executed deterministic evaluation) → (identical signed verdict). The signed certificate is not a summary of the decision. It is the cryptographic artifact from which the verdict can be independently derived and confirmed.
Not a log of what happened — a re-executable record
The distinction between a log and a re-executable record is the one most teams miss, because the two look identical from the outside. Both are stored. Both have timestamps. Both contain a verdict field. The difference is what you can do with them.
A log is self-attested. It says "the system reported that the verdict was BLOCK." There is no independent way to confirm that. The record could have been written incorrectly. It could have been edited. It could reflect a different decision than the one that was actually used. An auditor who wants to validate it is, in practice, trusting the organization's own data pipeline — not an ideal position for an independent review.
A re-executable record is something different. The signed certificate carries enough information — the inputs digest, the policy identity and version, the verdict, the timestamp, the cryptographic signature, and the key identifier — that any party holding the public key can independently run the verification and confirm that nothing has changed since the moment of the original decision. The vendor is not in the loop. The organization's own infrastructure is not in the loop. The math is the verifier.
Why probabilistic systems cannot replay
Temperature, sampling, model version, and hidden state
When the enforcement verdict — the ALLOW/BLOCK/MODIFY — flows through a language model, a classifier with a temperature setting, or any system that introduces stochastic variation, replay becomes impossible in a precise sense. The problem is not that the system is bad at its job. The problem is structural: the same input, fed to the same model on two different runs, will produce outputs that differ. Temperature introduces randomness into sampling. Model versions change silently under the hood. Context windows carry hidden state. The output is a function of inputs plus a source of randomness that is not recorded and cannot be recovered.
This matters for audit in a specific way. An auditor validating a control by re-performance needs to run the same process and get the same answer. If re-performance sometimes returns BLOCK and sometimes returns ALLOW for identical inputs, then what is being tested? The answer is: nothing, with any reliability. You are measuring the distribution of a stochastic process, not confirming that a specific control was applied.
| Property | Deterministic enforcement (EVE CoreGuard) | Probabilistic / model-based evaluation |
|---|---|---|
| Same verdict on identical input | Always — 100% match | Not guaranteed — score varies by run |
| Offline re-performance possible | Yes — verifier requires no vendor call | No — model must be live and version-pinned |
| Verdict attributable to a specific rule | Yes — rule ID and policy version in certificate | Partial — score, not rule attribution |
| Tamper evidence on the record | Ed25519 signature (prod) / HMAC-SHA256 | Application log — self-attested, mutable |
| Audit finding if verdict varies | Structurally impossible — deterministic | Likely — variance is the control's normal behavior |
| Policy version pinned to each record | Yes — immutable, per-decision reference | Rarely — model version may not be captured |
The table above is also the conversation auditors are beginning to have with AI teams. When a model validation group asks "can you reproduce the exact verdict for case ID 4471823," they are not asking whether you can regenerate a similar output. They are asking whether the control is repeatable in the precise sense that the word "control" implies.
SR 11-7, the Federal Reserve's supervisory guidance on model risk management, places documentation and reproducibility at the center of what it means to operate a model responsibly. When a model is used in a consequential decision — lending, fraud, servicing — examiners expect that the decision can be explained, attributed, and re-performed. A verdict that varies by run is, in SR 11-7 language, an indicator of a control that is not well-understood and not well-managed.
The four ingredients of a replayable decision
Deterministic evaluation — no stochastic step in the verdict path
The first ingredient is the one that makes everything else possible. If the evaluation engine introduces any probabilistic step between inputs and verdict — a language model, a scoring threshold that varies by temperature, a classifier whose outputs drift — replay is foreclosed before it begins. Deterministic evaluation means fixed rule logic: the same inputs, the same policy state, the same result, every time. Not approximately. Exactly.
EVE CoreGuard's Layer A enforcement verdict is zero-LLM. There is no language model in the path between the proposed action and the ALLOW/BLOCK/MODIFY verdict. The verdict is a function of the inputs and the rule set — nothing else. The same inputs produce the same verdict on the first evaluation and on the ten-thousandth. This is what makes it a control rather than a heuristic.
Pinned, versioned policy — the exact ruleset, immutably referenced
Deterministic evaluation against a shifting ruleset is not replayable. If the policy that governs a decision today is different from the policy that governed the same decision eight months ago, you cannot reproduce the original verdict by running the current evaluation engine against the current policy. Replay requires that the policy is versioned and that each decision record contains an immutable reference to the exact policy version that evaluated it.
This sounds obvious, but most enforcement implementations skip it. The evaluation logic is in application code, which changes continuously. The "policy" is often a configuration file with no version history tied to the individual decision record. When an auditor asks which policy governed a decision from the third quarter of last year, the honest answer is: we cannot tell. The policy has been updated sixteen times since then and we do not know which version was live on that specific date.
Policy versioning, as a first-class architectural property, means that each signed decision certificate carries a policy ID and a policy version hash. The policy pack at that version is archived and can be recovered. The auditor can confirm, without ambiguity, that the policy described in the record was the policy applied. This is what runtime attestation provides.
Signed input and verdict capture — tamper-evident, not self-attested
The third ingredient is the one that separates evidence from reporting. Signing the decision certificate — the inputs digest, the policy reference, the verdict, the timestamp — with a private key means that anyone holding the corresponding public key can confirm two things: that the certificate has not been altered since it was issued, and that it was issued by a system holding the private key at the time it claims. Neither of those properties is available from an application log.
EU AI Act Article 12 requires automatic logging of AI system events over the system lifecycle for high-risk systems. Signing those records is what elevates them from logging to evidence. A signed record is tamper-evident by construction. An unsigned record is tamper-evident only to the extent that you trust the organization that wrote it — a weaker foundation for a third-party audit.
Offline verifier — no vendor call required
The fourth ingredient is often the one enterprise security and procurement teams ask about first, and rightly so. If verification of a decision record requires calling back to the vendor's API, then several things become true that an auditor will not like: the audit depends on the vendor being available; the audit depends on the vendor not having changed the verification logic; and the organization cannot perform an independent check without the vendor's participation. That is not independence. That is reliance.
An offline verifier is a standalone tool — or in its simplest form, a script — that accepts a stored certificate and a public key and returns a pass/fail result. It has no network dependency. The EVE CoreGuard Verify portal demonstrates this: paste in a decision certificate, and the portal confirms or refutes its authenticity without calling any EVE AI Core backend system. The cryptographic verification happens client-side, against the published public key. Any auditor with the certificate and the public key can do the same thing independently, in any environment, at any time.
What auditors do with replay
Once replay exists as an engineering property, auditors have three concrete uses for it. These are not theoretical — they map directly to the activities that internal audit, model validation, and external examiners conduct during a review of an AI-governed process.
Replay in EVE CoreGuard — the certificate anatomy
A signed decision certificate from EVE CoreGuard is not a proprietary format. Every field serves a specific evidentiary function. Understanding what each field proves is useful for audit teams that are designing their own testing programs.
| Certificate field | What it contains | What it proves to an auditor |
|---|---|---|
| inputs_digest | SHA-256 hash of the canonicalized input payload | That the inputs recorded are the exact inputs that were evaluated — any alteration invalidates the certificate |
| policy_id | Unique identifier of the regulatory policy pack applied | Which ruleset governed this decision (e.g., lending_v1, banking_aml_v1) |
| policy_version | Semantic version + content hash of the policy pack | The exact, immutable version of the rules — allows the policy to be recovered and re-run |
| verdict | ALLOW, BLOCK, or MODIFY | The binding disposition returned by the deterministic gate |
| violated_rules | List of rule IDs that triggered the verdict | Rule-level attribution — which specific rules caused a BLOCK or MODIFY, not a score |
| timestamp | ISO 8601 UTC timestamp of evaluation | When the governance control was applied — independently verifiable against external time sources |
| signature | Ed25519 signature (production) or HMAC-SHA256 over the canonical certificate payload | Tamper evidence — any alteration to any other field invalidates this signature |
| key_id | Identifier of the signing key used | Which public key to use for verification — supports key rotation without ambiguity |
The Verify portal at /verify accepts a certificate in this format and performs the cryptographic check client-side. The verifier confirms that the signature over the canonical payload is valid given the published public key, and that the inputs digest matches the inputs provided. The verdict in the certificate is then trusted as authentic — not because EVE AI Core says so, but because the math confirms it. That is the structural difference between a vendor attestation and an independently verifiable record.
For teams comparing governance approaches across vendors, the EVE CoreGuard vs observability platforms comparison addresses how signed replay certificates fit into a broader monitoring stack. The short answer is that replay and observability are complementary: observability tracks trends across the stream of decisions; replay validates the integrity of individual decisions within that stream. See also the EVE Proof evidence model for the full technical specification of how certificates are constructed and chained.
Designing for replay from day one
Replay is an architectural decision, not a feature you can add after the fact. Organizations that discover the requirement during an exam rather than during system design face a much harder path. The engineering commitments to make at the outset are straightforward in principle, but easy to defer:
- Capture inputs at the governance layer, not downstream. The inputs that the governance engine evaluated must be captured at the moment of evaluation, not reconstructed later from application state. Reconstruction is not replay. If the inputs stored in your record are a reconstruction, an auditor cannot confirm they match what was actually evaluated.
- Pin and archive policy versions. Every policy version that was ever active in production must be archived in a form that can be recovered. Continuous-delivery pipelines that overwrite policies in place destroy the ability to re-perform evaluations from historical periods. Version identifiers must be immutable and tied to a content hash, not to a deployment timestamp.
- Sign at the evaluation moment, not at log ingestion. The signature must be produced by the evaluation engine at the time of the decision — not by a log aggregator downstream. If signing happens downstream, the chain of custody between the evaluation and the signature is broken, and the tamper evidence is weakened.
- Publish public keys and manage key rotation transparently. Auditors need the public key to verify certificates. Key rotation must be documented, and old public keys must remain available for the retention period of the decisions they signed. A key that has been rotated without a published rotation record makes certificates from the prior key unverifiable.
- Define your retention period by the regulatory context. EU AI Act Article 12 requires logging over the system lifecycle. SR 11-7 model validation cycles create their own retention expectations. Certificates must be retained for at least as long as the decisions they govern may be examined — which, in lending and financial services contexts, is often measured in years, not months.
The cost of designing for replay from day one is low — it is primarily a requirement for deterministic evaluation, policy versioning, and signing at the evaluation moment. The cost of retrofitting replay after deployment is high, and in some cases, the historical record is simply lost. The decisions made before the system was designed for replay cannot be made replayable retroactively.
Where replay fits in the broader governance stack
Replay does not replace the other layers of an AI governance program. It completes the evidence layer — the one that connects the written policy in your GRC system to the individual signed decisions that an examiner can test. The enforcement plane is where replay originates; the trust center and procurement artifacts describe how the enforcement layer is operated; and observability tools watch the aggregate stream of decisions over time.
What replay adds to that stack is the ability to close the loop on any individual decision at any time. An examiner can pull a certificate, verify it offline, and confirm that the policy governance claim in your GRC system was actually executed — not just documented. That is the evidentiary value that turns a governance program from a collection of documentation into a defensible control environment.
For teams building out this capability, the full platform comparison maps where replay and signed evidence sit relative to other categories of AI governance tooling. The cryptographic proofs article provides a deeper primer on the signing mechanisms. And the banking infrastructure article addresses how replay maps to specific SR 11-7 and ECOA/Reg B examination expectations.