Exploring the Agentic Future

The Voices

CONCEPTUALIZATION: You see it beginning. Hallucinating, demented agents with your company’s prod keys, and nobody knows how they work. They’ll gain access to every bit and byte. They’ll hallucinate, corrupt, leak. It’s all changing so fast. We can make things with them. Glorious things. Beautiful, profitable things. But oh… the risk.

ANCIENT REPTILIAN BRAIN: Everything is a risk, silly fish! You forget so quickly! Just a sad little ancient fish with proto-hands, crawling from pond to pond trying not to be gobbled up. Now you use those little fishy hands to press buttons on magic boxes and call it software engineering! My, how you’ve grown!

VOLITION: STOP IT. You’re a professional. You know this will end badly, and the author knows it too. This blog post, this series, whatever it is, it’s speaking to you. It’s trying to warn you. You clicked for a reason. Maybe because you want to embrace progress, but not have the musket explode in your hands.

CONCEPTUALIZATION: The machines are making decisions now. In finance, commerce, war, and science, little black boxes whisper judgements about our collective future. We used to know how systems behaved. Now we don’t.

DRAMA: Never mind the details, we must vibe code. We will talk to the LLM like we are speaking in tongues, like we are interfacing with the ouija board again. Conjure the spirits, it is the only way we will feel something again. I think I… feel something… but I don’t know if it’s good.

HALF LIGHT: SHUT DOWN YOUR EMOTIONS! ONLY BURNDOWN CHARTS CAN EXIST IN THE REALM OF SCRUMMASTERS! To hesitate is to die. The sprint ends on Friday, and your dignity ends with it. State machine diagrams are far too complex, an utter betrayal of the technical spike. ONLY BOX DRAWINGS MUST EXIST, WITH ARROWS THAT POINT IN ONE DIRECTION. NEVER BACK. NEVER SIDEWAYS. ONLY FORWARD.

You reach for structure and they slap your hand away. “Move faster!” they scream, as the floor collapses beneath them.

CONCEPTUALIZATION: Even the cautious ones. The planners. Even they gave in… stopped thinking, stopped designing. Their systems are just as opaque now, wrapped in strange abstractions that cannot be explained in any natural language.

Two failures, running in parallel. A generation that borrowed engineering titles without the due diligence, oversight, or accountability that real engineering demands, and the incentive structures that rewarded it. Now the systems are becoming agentic.

The Real Crisis

Developers were never just typists, at least not in the way the industry talks about them. Their primary value was as the event horizon. A good developer always knew where the line was between a safe change and a dangerous one. The best developers placed themselves in front of that line, absorbing risk so nobody on their team crossed it, often at great personal cost. They were the speed limit. Remove that speed limit and ask yourself what happens next.

Consider any publicly traded company under pressure to deliver. If you handed their leadership a magic box that auto-generated code, would they run it at 5% capacity, still double their current output, and guarantee safety? Or would they crank it past 100%, accept the risk, and deliver a better quarterly number? The worst would push it to 130%, knowing it could cost safety, lives, or even the survival of the company, as long as they personally benefited before they exited. We already know the answer.

That’s the real crisis. Not that AI writes bad code. The real crisis is that the same leadership and incentive structures that produced two decades of slop now have a tool that removes the last speed limit between bad decisions and production. Be careful what you wish for. We may be entering the monkey’s paw era of software development.

We were willing, in a millisecond, to declare “writing code” obsolete. Yet we’ve spent almost no time redesigning decision-making, approval, and accountability. If companies are having those conversations, they certainly aren’t as public as the prognostications about the end of developers. That’s the sad current state of the industry, and something we’re completely rejecting here and now.

Decisions Without Humans

Consider the table saw. An incredible tool that shaped modern woodworking, but also one of the most dangerous machines in any workshop. Catastrophic injuries were accepted as an occupational hazard for decades. Then SawStop came along: the blade carries a small electrical signal, and when skin contact changes the conductivity, a spring-loaded aluminum brake fires into the blade and stops it within five milliseconds. The blade retracts below the table and your mistake leaves a nick instead of an amputation. Innovation doesn’t have to mean accepting catastrophic risk. It means accepting calculated risk, and the calculation must acknowledge all options and be honest about the pros and cons.

Humans have always been innovative and always taken risks, but we’ve also always tried to hedge them. In software, we did this first with waterfall and formal design methodologies. During the ZIRP era, we hedged with layers of automation: CI/CD, monitoring, alerting. But there was always a human backstop. The expert programmer who could read the code was the operator who answered the page. Humans in the loop have always acted like that potentially lifesaving electrical signal that fires the brakes.

Today, decisions are increasingly made autonomously, often justified after the fact, and sometimes without any human involved at all. We can no longer rely on a button click to signal intent, or a dashboard to explain behaviour after the fact. We need systems that can encode, trace, and retain the flow of decision-making end-to-end. We need systems that are self-explaining.

Everything as a System

Everything is a system. A team is a system that builds systems, and those systems automate things. If you take systems thinking seriously, every decision point is a process that can be mapped. How was a decision made? That’s a process. How was a deployment approved? That’s a process too.

Now apply that logic to a catastrophic production incident. We start with the symptom: a customer complained about data loss. Now we trace the chain all the way back. Not just to a server or a piece of code, but through the entire organization. Teams. Decisions. Events.

• How was the project funded? What gate approved it?
• How was the delivery scheduled? What process set the timeline?
• How was the code reviewed? What gate signed off?
• How was it tested? What was covered, and what wasn’t?
• How was deployment approved? What process released it?

That entire chain is essentially an event journal. A set of facts. Just because many of those facts aren’t part of our customer-facing production systems doesn’t mean they’re not events that could be studied.

From “ten years ago we raised money” to “five minutes ago we took production down”, that entire chain is a system. We could map every step in the process, from as-defined to as-executed, and circle the ones that broke down.

Imagine a scenario where every single step to release a brand new, highly hyped feature succeeded except one: automated testing during the merge to main timed out, so the project lead made a judgment call to merge and push without waiting for tests to complete. That decision to skip automated testing wound up missing a bug introduced during a rebase.

Now follow the timeline from that point forward.

This isn’t a real incident. But it could be, easily. The one that happens everywhere, every week, at companies of every size. The details are interchangeable.

In our hypothetical incident, the bug wasn’t caught by tests. Logging saw something, but not enough to trigger an alert. Monitoring saw a latency spike, but it was below threshold. The operator glanced at the dashboard and moved on. The first real signal came from customers.

This is the gap. Look at the yellow and red rows. Now look at the “Evidence?” column. Some steps have logs. Some have metrics. Some have nothing but a Slack thread or someone’s memory. The steps closest to the customer, the ones that actually caught the problem, were the furthest from the code.

This is systems thinking. This is why we need to stop living within the lie that developers are fungible resources, and instead ask ourselves how we’re going to use these incredible new tools to build better, more robust systems, not how to use them to game headcount math.

What’s Next

We’ve traced the path that led us here: from the early days of deliberate software design, through the chaos of ZIRP-era delivery culture, to the economic and architectural headwinds now shaping our future. The tools we use, the systems we inherit, and the tradeoffs we normalise are all downstream of the incentives that shaped them. And those incentives have changed.

What hasn’t changed is the value of technical tradecraft. A return to it might be the only thing that prevents us from crossing the event horizon, the point beyond which our systems become so opaque, so fast, and so autonomous that no amount of after-the-fact forensics can explain what went wrong or why.

If our systems are going to operate reliably under real-world pressure, they need to be structured to preserve meaning. That means encoding not just what happened, but why, who decided, and under what assumptions. That’s what Observable Reflective Architecture (ORA) provides: an approach to building systems that are explainable by design.