Agency

To explain why a system behaved a certain way, we must know who or what made a decision, and under what authority. That means tracking agency as a first-class concept, not just a string like "principal_user_id": 42, but an accountable actor whose identity, intent, and context can be traced and validated.

To evaluate this, we define five levels of agency integrity:

• A0: Unattributed: Actions have no associated actor or are logged with anonymous strings
• A1: Logged: Actions include basic identifiers that are recorded but not validated or enforced
• A2: Bound: Actions are bound to authenticated actors, using cryptographic identities
• A3: Authorized: The system enforces who is allowed to do what, based on runtime context
• A4: Verifiable: Agency is explicitly modeled, validated, and testable

This slide appeared in a 1979 IBM internal presentation and captures a truth that’s only grown more urgent: a computer can never be held accountable. But what happens when intelligent systems are making real decisions that affect real users? Imagine a dataflow pipeline that ingests a customer’s entire financial history, feeds it through a predictive model informed by machine learning, and denies them credit. Who is accountable for that decision?

The answer goes far deeper than authentication or authorization. It’s not enough to know who triggered the pipeline. We need to know the provenance of the model that made the prediction. What data was it trained on? When? By whom? Under what assumptions? Historically, delegating to machines meant decision trees in code plus auth/authz. Now it means code, training data, trained models, and inference chains, each with its own lineage. Agency in this context isn’t just about identity. It’s about the full causal history of a decision, human and machine alike.

A0: Unattributed

Actions occur without any associated actor, and there’s no causal history to interrogate.

You can’t tell:

• Who or what triggered this
• Whether it was human, code, or model
• What version of the model made the decision
• What data trained it
• What inference path was taken
• Whether it should have been allowed

Think back to the credit denial example. At A0, the decision happened, the customer was denied, and the trail is cold. You can’t even begin to answer “who is accountable?” because there’s nothing to trace.

Legacy systems and modern ML pipelines alike can be found at A0: models deployed without versioning, training data never tracked, hyperparameters never checked into source control, inference decisions that are ephemeral. Vibe coding accelerates the problem, but it exists wherever intelligent systems make decisions without preserving their lineage.

A1: Logged

Events record metadata about who or what acted, but none of it is verified. You have identifiers, but you can’t trust them.

A principal_actor_id might have been passed in a payload or spoofed in an HTTP header. A model_id: "pricing-v2.3" might be stale, wrong, or just a string someone passed in. A training_run_id might not match what was actually used. The metadata exists, but it’s not authoritative.

At this level, logging is a side channel, not the system’s book of record. It’s best-effort: maybe the developer remembered to log a line, maybe they didn’t. Whatever they did log can be unpacked manually by an operator, or perhaps collected by something more intelligent like a log scraper. Operators and support engineers can:

• Manually correlate identifiers across log entries
• Build basic reports from whatever was captured
• Grep their way toward an answer when something goes wrong

But you cannot:

• Guarantee the identity is correct
• Verify that the logged model is what actually ran
• Enforce authorization
• Trace back to training data or hyperparameters with confidence

At A1, you see glimpses of maturity. Hyperparameters might be checked into source control. Model versions might be recorded somewhere. Training runs might be documented. But it tends to be scattered and inconsistent, especially in large systems. Imagine a large bank: various teams are logging, but the quality varies wildly across departments. Some teams are rigorous. Others log almost nothing. There’s no uniformity, and no guarantee that what you need will be there when you need it.

A2: Bound

Actions are tied to authenticated actors, and “actor” now includes humans, services, and models. Identities are validated using:

• OIDC tokens and verified human sessions
• mTLS certificates and SPIFFE/SPIRE workload identity
• Signed model artifacts and model registries with verified checksums
• Cryptographic binding of model versions to inference events

At A1, model_id: "pricing-v2.3" was just a string. At A2, the model artifact is hashed, signed, and the inference event proves which model actually ran.

You can:

• Bind events to cryptographic identity for humans, services, and models
• Reject actions from unauthenticated sources or unsigned models
• Prove who or what initiated a state change

Consider when decisions get audited: a regulator asks why a customer was denied credit, legal discovery demands all decisions affecting a specific account, a compliance review requires evidence of which models ran in production last quarter. At A2, identity is cryptographically verified. You can produce a defensible chain showing exactly which model artifact, signed by which party, made which decision at which time.

This unlocks real traceability. The causal chain still has gaps: you can prove which model ran, but not necessarily what data trained it, what hyperparameters shaped it, or whether the training pipeline itself was trustworthy. Authorization is also still missing.

A3: Authorized

At A2, you can prove who or what acted. At A3, you enforce what they’re allowed to do.

Authorization applies to humans, services, and models. For humans and services, you might use RBAC, ABAC, or policy engines like OPA or AWS Cedar.

For models, authorization means controlling what decisions a model is permitted to make:

• Model X is approved for credit decisions under $10K, but requires human approval above that threshold
• Model Y can operate autonomously during business hours, but flags decisions for review outside that window
• Model Z version 2.3 is authorized for production; version 2.4 is staging-only
• Certain model classes are permitted for low-risk decisions but prohibited from high-stakes ones

At this level, unauthorized actions are blocked by design. A signed model that passes A2 verification can still be rejected if it’s not authorized for the action it’s attempting.

Policies themselves become auditable artifacts. When authorization logic is externalized into version-controlled policy bundles, every policy change has a commit history, every evaluation can be replayed, and the rules governing decisions are as traceable as the decisions themselves.

A4: Verifiable

At A3, you enforce what actors are allowed to do. At A4, the entire agency chain becomes verifiable and testable.

This is where the causal loop finally closes. At A2, you could prove which model ran. At A3, you could enforce what it was allowed to do. At A4, you can trace the full chain: identity → policy → model artifact → training run → training data → hyperparameters. Each link is cryptographically or contextually provable.

You can:

• Verify the complete lineage of any decision, from action back to training data
• Assert agency in tests and simulate entire chains of authority
• Run hypothetical scenarios to validate what would have happened under different policies or models
• Produce evidence that satisfies auditors, regulators, and legal discovery

Remember the IBM slide: “A computer can never be held accountable.” At A4, you’ve built the infrastructure to hold the humans and processes behind the computer accountable. When a model denies a customer credit, you can trace that decision to: who approved the model for production, who trained it, what data it was trained on, what hyperparameters shaped its behaviour, and what policy authorized it to act.

Example: A CreditDenied event can be traced to model risk-scorer-v2.1, signed by the ML platform team, trained on dataset credit-history-2024-q3 with documented hyperparameters, operating under policy credit-decisions-v4 that permits automated denials only for scores below a threshold. The human who approved that model for production is on record. The training pipeline is auditable. Agency, when preserved and enforced to this level, is the accountability layer of explainable systems.