From control to confidence

There's a conversation I keep hearing from developers right now about AI-generated code, and I think most of it comes down to one question. Are you more or less confident in the code now that AI wrote most of it?

I understand the concern. I do. AI gets things wrong. Sometimes wildly so. It can misunderstand the assignment, introduce bugs, and open up security holes.

I'm not arguing those risks aren't real. My argument is that we have always been deluding ourselves.

We had this idea that because a human being wrote the code, or because we could theoretically read every line, we somehow had control of the chaos. We didn't. We never did. Nobody understands every square inch of a modern software stack. You install a framework and you inherit all of its flaws, issues and defects. You bring in dependencies you didn't write, and those dependencies bring in more dependencies you didn't write. You call APIs you don't control. You deploy onto infrastructure where, at some point, you are absolutely trusting somebody else's judgment.

That uncertainty is real, but it is definitely not new. AI just makes it much harder for us to pretend otherwise.

Control vs. confidence

I think we're asking the wrong question when we say, "How do we keep a human in the loop so we can maintain control?" I don't think control is the thing we should be optimizing for. I think we should be optimizing for confidence. Those are different things.

Instead of pouring all our energy into making sure a human can inspect every line of generated code, we should be investing in testing harnesses, QA systems, security tooling, observability, deployment infrastructure, and everything else that tells us whether the output is actually good. At some point, "I read every line of code" is not a particularly scalable quality strategy. And honestly, it probably never was despite repeated assertions to the contrary.

The interesting thing is that we're starting to see tooling emerge around exactly this problem.

Netlify recently released AXIS, an open-source scoring framework for what they call Agent Experience. The easiest way to describe it is Lighthouse for AI agents. You give it a scenario and a rubric, point an agent at a service, and it watches what happens. Did the agent accomplish the goal? Did it use the service correctly? Did it recover when something went wrong? Then it gives you a score you can actually track over time.

What I find especially interesting is what Netlify is doing with it internally. They're moving toward running those scenarios as part of their development process, so if the experience gets worse, the build fails. That's a fundamentally different approach to confidence. You don't sit there and watch the agent make every decision. You define what good looks like, you build the harness, you run the test, and if quality drops, the system will tell you.

That idea extends well beyond Agent Experience, too. LangSmith lets teams evaluate entire agent workflows: the final result, the tool calls along the way, the trajectory the agent followed, and whether the overall behavior met the criteria you defined. You can run those evaluations against known datasets before you ship, score real production interactions afterward, and wire the results into CI so a regression fails the build. Braintrust is coming at it from the observability side. Trace the model calls. Trace the tool invocations. Score what happened. Then take a bad production run and turn it into a regression test so you don't make the same mistake twice.

That, to me, is the direction this has to go. Did the system behave the way we expected? Did it accomplish the thing? Did it break any of the rules? Did performance get better or worse? Did we introduce a security problem?

Those are much more useful questions. And if we can answer them reliably, I don't know why my confidence needs to depend on whether I personally watched the code being written or whether I wrote it myself.

Congrats, you are now an engineering manager

This also changes what the job is.

Our developers and engineers are increasingly not developers and engineers in the way we've traditionally thought about them. They're now engineering managers–everybody just got a promotion! Not necessarily in title, and not necessarily with a team of people reporting to them. But functionally? Absolutely.

They're defining work, setting expectations, and creating constraints. They're delegating and reviewing outcomes. They're figuring out when to intervene and when to let the system keep going. They're managing agents. And I think that requires us to think bigger and broader than we have in the past.

I've learned this lesson before

There's another reason this feels familiar. I've already had to learn this lesson once.

A huge part of my evolution as a leader has been learning how to let go of control–something which does not come naturally for me. 

For a long time, I believed being responsible for the work meant having my hands in the work. I needed to understand what was happening. I needed to be close enough to intervene. I needed to know the decisions being made were the decisions I would have made.

Pretty quickly though, I learned that that does not scale.

What changed for me was building an extremely good team. Finding people I believed in who also believed in me. Giving them autonomy, but also spending a lot of time establishing what good decision-making actually looks like, so they understood the principles underneath the work instead of just following instructions for it.

And then the difficult part: trusting them. Every day. Letting somebody make a decision differently than I would have. Letting them own things. Letting them occasionally get something wrong. Being available when they need me without inserting myself every time they don't.

That has always been hard for me. I'm better at it than I used to be, though. When I look around at the team we've built, I realize I actually figured some of this out. Not perfectly. Not even close. But in ways I see a lot of people in leadership positions continue to struggle with.

The lesson wasn't that I needed to stop caring about the outcome. It was almost the opposite. I had to care enough about the outcome to stop making myself the bottleneck. I had to replace control with something better: good people, clear expectations, shared principles, autonomy, feedback, trust, and some systems that tell me when something is going off the rails. Which sounds an awful lot like what we're talking about with agents…

The answer isn't to stand over their shoulder and watch every move. It's to build the right environment around them. Define what good looks like. Give them enough context to make sound decisions. Create the checks that catch problems before they become disasters. Then let them cook.

That doesn't mean people and AI agents are interchangeable. They very obviously aren't. But the management problem feels weirdly familiar. At some point, leadership requires accepting that you can't personally control every decision and still build something bigger than yourself. Maybe engineering is arriving at the same point.

But why?

I keep seeing people treating the constraints of their current system as impediments to progress. I’m hearing a lot of this kind of stuff:

• AI can't do this because our process requires that.

• AI can't do that because our architecture works this way.

• AI can't be trusted because we've always had a developer manually do this.

Okay. But why?

Some constraints are real. Of course they are. But some of them only exist because human beings used to have to do the work by hand. One of the biggest opportunities here is to stop being scared of technology and stop operating exclusively within your constraints. Rather, use technology to work around your constraints. Or, even better: find a completely different paradigm that releases you from the constraint altogether.

That's much more interesting to me than figuring out how to use an agent as a slightly faster developer. I don't want the agent to be a slightly faster version of me. I want to know what becomes possible when I stop assuming the way I've always done the job is the way the job has to be done.

The part that's personal

There's probably something personal underneath all of this for me.

When I became a developer, part of what appealed to me was control. I started as a designer and I loved the craft–but design exists within the eye of the beholder. It’s inherently subjective. You can make something beautiful and somebody else can hate it. Development felt more concrete to me; the function either works or it doesn't. The test either passes or it doesn't. There's something safe in that objective space that appeals to my insecurities.

But the hardest parts of my career have increasingly been the moments when I've had to recognize I don't actually control as much as I thought I did. Maybe that's the shift happening now, at a much larger scale and people are kind of losing their minds over it. Maybe the job isn't to control the chaos? Maybe the job is to build systems that can operate confidently inside of it.

That doesn't mean blindly trusting AI or shipping whatever the machine spits out and hoping for the best. It means changing where we put our effort. Less energy spent preserving the illusion that a human can maintain complete control over the system. More energy spent building systems that can tell us, with a high degree of confidence, whether the thing actually does what we want it to. 

If developers really are becoming engineering managers for teams of agents, that's the skill we need to get good at anyway. Building an environment where good work is more likely to happen, bad work gets caught more quickly, and the system keeps moving without us gripping the steering wheel every second.

I think that's the part that still makes people uncomfortable. And it probably should. But the sooner we come to terms with how little control we actually have, the better IMO.