The Judgment Workshop

The task ticket arrived from the Amazon network infrastructure team and went into the backlog automatically. The virtual IPs, or VIPs, fronting our service were approaching their throughput capacity, and we were required to split them before the next capacity plan review. It was a compliance item with a strict due date.

I never saw the ticket. I ran the organization that owned the gateway for Amazon Music, the path every request from every app travelled through. The manager and the tech lead who owned that gateway read the ticket and made the reasonable call. A small operational change, a few hours of config work, nothing that deserved escalation. They made the same call against automating the VIP split because it only happens once every couple of years. Automation pays off when the amount of time you save is greater than the time you invested in building the automation. This was so infrequent, the payoff might never come.

The split was performed the following week. Traffic flowed across both paths. The metrics were green and they stayed green.

Eight days later a major artist released an album.

The traffic surge hit a server fleet that had been divided unevenly between the two paths, with the smaller share behind the first VIP. The engineer who executed the split had distributed the servers by hand but had no way to see that the distribution was wrong, because there is no dashboard metric that shows whether a pair of VIPs is balanced. There is no pattern to recognize. The undersized fleet saturated first, and a set of automated protections designed to handle typical server failures began responding to the symptoms rather than the cause. That automation removed and replaced servers in the fleet, each time removing capacity from a system that was already in dire need of it. Within the hour, every music service was down. The primary use case for the tens of millions of Echo devices in the home was music, and it did not work.

The root cause investigation afterward found three configuration decisions that conspired to produce the cascade. Nothing in the decision chain leading to the failure was careless.

The infrastructure team identified a real limit and filed the request correctly. The request intake system classified it and routed it correctly. The manager and the tech lead assessed it against their criteria and reached the right conclusion. Rare operation, low complexity, no repetition to automate against, no reason to escalate. Someone with more authority in that chain would have agreed with all of it. I would have agreed with all of it.

Every decision was correct. Classifying it as a routine change was what was wrong.

The Case Nobody Made

The case for automation was weighed on efficiency alone. Rarity lowers the value of automation for repeatability, so the operation stayed manual. Rarity also raises the value of automation for verification, because a script can be checked and a hand-executed change cannot. Nobody made that case. An operation performed once every several years is one that nobody has a current mental model of. There is no reflex that says the fleet looks unbalanced, no memory of the last time, no felt sense of what correct looks like. Without the reps, the correctness had to live in the process, and the case for putting it there was never made.

What the organization turned out to depend on was not knowledge of how to split a virtual IP. It was the presence of someone, anywhere in the chain, who could look at a routine item and feel that it was not routine.

The obvious correction is to automate all the infrequent operations. That is the wrong lesson, and I know it is wrong because I have made the same call the other way.

A few years earlier at Amazon Music, I owned the creation of a new live audio broadcast experience for every match in Bundesliga, the German soccer league. Nothing in the organization had ever gone out live. Every piece of audio we had ever shipped was recorded, which meant there was no operational history to draw on, no incident log, no one in the organization who had run a live broadcast and could tell us what could go wrong.

Live broadcasts carry obligations that recorded audio does not. We were required to be able to interrupt a broadcast and put out a public notice. Stadiums had seen fires, crowd crushes, and attacks. Whatever a producer might hear in their headset on a night like that must never reach a listener uncut. The obligation could have been simply met by cutting the stream and posting a notice on the website. That was the compliant answer.

We built something well beyond what those requirements asked for. We engineered a dynamic broadcast delay that gave the producers varying amounts of time to act, depending on what they were hearing. We had prerecorded audio and in-app messages based on the type of emergency that would play automatically the moment the alert was triggered, so that a listener in the middle of a match would hear something specific rather than silence. It cost real engineering effort, and nothing in the numbers said it was worth it. In four years across hundreds of matches, it fired once, and that once turned out to be a false alarm.

These were two calls made against similar scenarios. In the first, infrequency justified leaving the correctness to a person. In the second, it was the reason to take correctness out of a person’s hands entirely.

Nothing in the workflow of the ticket was built to ask whether the criteria applied to it were the right criteria. Every step moved the item along to the next step, and none of them stopped to ask what kind of item it was. That is not a missing step in the process. A step can only apply the criteria it was given.

The flow did not need network design or load distribution algorithm expertise. It needed someone with the reflex that a low-complexity operation with no recent history is exactly the kind that nobody will be able to check. There were people like that at Amazon. None of them saw the ticket, because nothing about the way it was routed was designed to put it in front of one. In all the years before that day, what such a person contributes would have been indistinguishable from not having them at all.

Judgment of that kind is not developed by handling more tickets, because the situations that build it arrive too infrequently to teach anyone. It has to be manufactured deliberately, in environments constructed for this purpose, or the organization will not have it on the day the criteria stop fitting.

Why the Ground Disappeared

That reflex is built in the same way any judgment is built, and there is no shortcut. Someone reads a situation, arrives at a hypothesis and commits to it before knowing whether it is right. The interval between the commitment and the answer determines how fast that judgment is formed, and how much was riding on it determines how much it teaches.

The decisions that carry the most consequence unfortunately tend to produce the slowest version of the lesson because of the time elapsed between decision and outcome. An architectural decision reveals its real limits at a scale that the organization does not operate at yet. A market bet often resolves years after the people who made it have moved on. And when the feedback does arrive eventually, it is muddled with all the impacts of everything else that changed in the intervening years, delivered to people who were not responsible for the original decision.

Six months into my tenure as the senior architect at a GPS navigation startup, I was faced with a crisis. It began with a celebration and cake the day before. Our product was being distributed and promoted by Blackberry, at the time the hottest mobile phone company in the world. They were launching the Blackberry Pearl, its first consumer-oriented device with our product preloaded and featured in launch marketing. By the next morning we were in full crisis mode, because the entire system was down.

Being part of a launch on day one was an order of magnitude bigger than any load test we had ever run, crushing our Oracle database running on a Sun E450 server next to the kitchen. When everyone tries to sign up for a free trial at the same time, the load overwhelms the same database that maintains the state of a person navigating to their destination. Our customer support line was inundated with calls from people who got lost because their navigation stopped working halfway through the journey.

The next twenty-four hours were spent bringing the database back online. A couple of indexes and a tuned query were enough, as the initial hype of the launch died down. But the trajectory was clear. We had three hundred thousand customers since the company was founded, and we were on pace to clear six hundred thousand within the month.

By the end of the week I had a plan. Partition the database, separating navigation state from everything else, so that a surge in signups would not stop a customer from completing their journey. Then put a layer between the clients and the backend services, so that a device driving unusual traffic, like the Pearl, had its own path and could not starve out the others.

The pushback from the team was legitimate. Splitting the databases meant more license cost, more hardware, and double the operational load on the DBA team, all of it starting day one. Everything on my side of the argument was a conviction about failures that had not happened yet and, if the design worked, never would. The layer had no name and no precedent to point at, and the alternative people knew was a service bus, which solved for a coordination problem we did not have.

The DB partitioning proved successful on the dimension I designed for. But it was only one of many dimensions. What I did not model was the growth of the engineering organization itself, which put teams on a collision course relying on a shared schema as they developed in parallel. I did not model the explosion in the number of device platforms, each variation carrying its own limitations that had to be absorbed somewhere. Sharding answered the signup problem and introduced its own: the key I chose produced hot zones, and rebalancing meant scheduled downtime.

The middle layer got a name about a decade later, when the industry converged on the pattern and called it Backend for Frontend. That answer arrived from outside the company, long after anything in it could have changed what I did.

Nothing in that first failure could have warned me about any of it. That is not a failure of analysis. It is what an interval of years does to a decision.

Building Tight Learning Cycles

Learning from those architectural decisions, I now insist that every long running strategic initiative be broken into monthly cycles, each with a goal stated as a customer outcome wherever possible. The duration is arbitrary and that is part of why it works. Everyone understands it at a glance, and a month is short enough that a team has to choose what actually matters.

What makes it more than simply incremental delivery is the test each cycle has to pass. Every monthly goal must whittle down the uncertainty in the strategic bet. Not deliver value, though it should. Not demonstrate progress, though it will. The question that dogs every long initiative is how anyone will know it is going to work. The monthly goal exists to answer some specific part of that question, with evidence, before the next one starts.

Cycles built that way surface the assumptions the original proposal was resting on, and they surface them while there is still time to act. The database partitioning rested on two such assumptions. First, that navigation state was the most critical thing to protect, which held. Second, that preloaded launches would be the dominant scaling pressure, which became less true every year. At the time in North America, Blackberry and Palm were the mobile platforms that mattered, and the carriers decided what shipped on them. Nobody in that room would have predicted that within five years iPhone and Android would take the market and strip the carriers of the distribution control that our business strategy depended on. A checkpoint does not predict the market. It is an interrupt that obliges you to revalidate the assumptions the design was built on, on a cadence, rather than discovering years later that one of them was past its due date. The person obliged is the one who made the assumptions in the first place.

There is a second benefit to monthly cycles that only shows up under pressure. Every long initiative is argued against the cost of deprioritizing a fix for an acute pain the organization is feeling right now. The argument is usually fair, because a project that pays out only at the end is asking the company to ignore what hurts today in exchange for a promise. Monthly cycles change the terms. The work returns value along the way, and can be prioritized to address the same pain being cited against it. And when a crisis demands all hands on deck, the initiative can be paused and resumed, because the benefits that have already landed do not disappear just because the team is temporarily working on something else.

After aligning the team on the monthly cycle, I do something else that gets a reaction. I make the first month the most audacious goal of the entire project: taking the highest-risk, most complex flow in the system and committing to build it end-to-end in the first month. The room typically explodes with objections. I call it a spike, a word borrowed from Agile, though the agile version investigates one unknown and throws it out. This one is the opposite, building the deepest path through the system, created as a skeleton to be built around. The rule is that the implementation can be faked, but the interfaces cannot.

This comes from my own scar tissue of leading large projects and learning that integration points are where they actually fail, because integration gaps are often the last thing anyone finds out about. Interfaces agreed upon early establish a common model across the team and force the tradeoffs into the open. Pulling that discovery into the first month means a pivot is still cheap, since nothing has been built on top of the assumption yet.

The effect I did not design for, and now consider the more important one, is on the people doing the work. A monthly goal is small enough for one person to own completely. The strategic bet underneath it takes years to answer and delivers that answer to whoever is still in the building. A month of it is a scope someone can read, commit to a position on, and find out about while they still remember what they were thinking when they committed. What used to be one bet with one distant answer becomes a dozen bounded ones a year, each of them a decision somebody made and had to live with. That does not guarantee anyone learns from them. It supplies the occasions, which the work no longer supplies on its own.

Neither practice improves the decision. Both shorten the gap between committing to something and finding out whether it was worth it, in work that would otherwise deliver that answer long after anyone remembers why the choice was made. What neither does is let anyone else see it happen.

Making Reasoning Visible

When I started as a software engineer, projects kicked off in a room in front of a whiteboard. The product team walked us through what they were trying to accomplish, the architect digested it and brought the engineers in. We worked through the data model, then the new capabilities, then where those capabilities belonged in the architecture. Those sessions ran long, sometimes a whole afternoon.

As a junior engineer, I spent most of my time there observing. I saw what the architect was focused on, what got pushback, and what demanded clarification from the product team or from the teams we depended on. I asked a dumb question once in a while to understand why this choice over another, and sometimes the question was not as dumb as I thought. Most of all, I saw the reasoning in what got scribbled on the whiteboard, what got erased, and what survived to be written up in the design document. The document was the residue of the afternoon’s thinking.

Distributed teams and remote work have thinned those sessions out, and no collaboration tool has replicated standing at a whiteboard and reasoning in real time. Now something else is happening on top of it. People increasingly work out their proposals in conversation with an AI model, and none of that reasoning is observable by anyone. What arrives is only the output. The journey that produced it is gone.

Shopify built something that runs on the opposite principle.¹ River is an AI agent that lives in the company’s Slack, and it operates under a single hard constraint: it does not answer direct messages. Try to work with it privately and it declines, asking you to open a public channel instead. Shopify CEO Tobi Lütke, who learned his trade in the German apprenticeship system, called it Lehrwerkstatt to describe what they were after. A teaching workshop, where the shop floor is the classroom and you learn by being near the work. The word fits more than just the AI agent. A workshop is a place with real materials and real tools, where the work happens in the open and the people who know more are close enough to see what you are doing.

Lütke calls out the common worry that AI will stop people from thinking, but argues the framing is wrong. He says the real risk is not that the agent does the work, but that the agent does the work and nobody else learns anything from it. When every interaction happens in a private window, the only person who learns is the one at the keyboard, and everyone else is locked out of the apprenticeship.

When River rejects direct messages and insists on public channels, it is so that the approach is visible, not only the answer. River works in the open and posts partial findings as it goes, which means a second person can see the thread from their own channel and drop in with a constraint or a redirect before the work is finished. Someone with more context arrives mid-problem and changes its direction, where everyone can watch it happen. New hires scroll back through the channels to study how experienced people scope a request before sending their first one, which is a different thing from learning what the answer was.

Lütke works with River in a channel named for himself, and more than a hundred people follow along in it. They add context, take over work, help with reviews, and tell him how rusty he has gotten. A founder being corrected in public by people well below him on the org chart is not something you get by buying a tool. River’s code merge rate climbed from 36 percent to 77 percent over two months with no retraining and no change of model, because people watching the agent work noticed where it got stuck and wrote down what it should have known. The agent did not get smarter. The organization did, and then handed what it learned back to the agent.

The principle extends past AI to any process that makes the act of deciding observable. Open design reviews where the reasoning is the point rather than the artifact. Decision records that preserve what was considered, what was traded off, and what was deliberately deferred. The distinction that matters is between showing a conclusion and showing someone working. Most organizations already run reviews. Fewer run reviews where anyone can see the reasoning being formed rather than just what was presented.

What this does is let someone accumulate patterns from cycles they did not personally run, which is the only version of this that can scale beyond the people who were in the room. It still does not close the loop, because nothing comes back to the person watching.

A person who has read ten thousand books can tell you exactly why a paragraph does not work. Very few of them write anything worth reading. The reading was never optional, and it was never sufficient. Observation produces people who can recognize good judgment without having developed it. For that, you need real consequence.

Consequence with a Bounded Blast Radius

The three configuration decisions behind the Amazon Music outage were not equivalent. The fleet behind the VIPs was not balanced, leaving one of them underpowered. Traffic was never striped across the two VIPs, which meant it could land unevenly between them. Both were oversights of the kind a checklist catches.

The third was a choice. Someone configured a surge queue, where inbound requests wait until the service has capacity to take them, rather than spillover, where new requests are rejected when there is no capacity to serve them. To most engineers those look like two ways to do the same thing. Their behavior under sustained overload is the opposite. A queue keeps accepting work the system cannot process, and the backlog starves the requests still arriving. Spillover has no backlog, because it rejects the requests immediately. The question of which to choose does not register, because the person configuring it has never watched a system queue itself to death rather than shed load. No runbook contains that. No configuration guide would ever tell you.

It was an expensive way to find out. The alert alone paged thirty-five thousand Amazon employees, and behind them were millions of customers who could not play music. So how do people learn from real situations without the organization paying that price to teach them?

Medicine solved this a long time ago. A newly graduated doctor holds an MD but no license to practice independently. What follows is years of residency: real patients, real diagnoses, real procedures, with an attending physician close enough to intervene and far enough away to let the resident carry the weight of the call. The attending’s judgment is present as a safety net, not as a substitute. Nothing about the patient is simulated. What is designed is how much the resident is allowed to carry, and when. This is what separates a workshop from a classroom. In a classroom the exercise is safe because nothing is at stake. In a workshop the work is real, the material gets ruined, and someone is standing close enough to keep the damage contained inside the room.

Technology has its own version, only far less formalized. In each of these the consequence was felt by the person who produced it. The junior engineer whose deployment corrupted a database table and who spent the night restoring it. The designer whose simplified onboarding tripled abandonment on the third screen.

The PM who spent the better part of a half-hour call apologizing to a customer. The migration had a fixed date and a scope that would not fit it, so something had to come out. The edge case they cut affected a small fraction of accounts, and the data said so. What the data did not say was that the company’s largest customer had built its month-end reporting on exactly that behavior, and nobody asked. Asking was not a step in the process and the number looked small enough to decide from. They found out on the customer success call they were pulled into, from the customer, with their own name on the decision. What they took from it was not a rule about edge cases. It was the instinct to ask who is on the other side of the number before treating a percentage as permission.

Failure is not what makes those situations developmental. What makes them developmental is that the cost was real and the organization could absorb it. A simulation builds real skill, and the skill it builds is the reason pilots log hours in one. What it cannot manufacture is the weight of the decision, because everyone in the room knows nothing that happens there follows them out the door. An unbounded failure supplies the weight at a price no organization should pay to educate someone. Designing how to thread the needle between those two is the work.

A stretch goal on the Bundesliga project, a German soccer live audio product, made me put this into practice. We wanted the score updates on screen to stay in sync with what a listener was hearing, which was difficult since the audio arrived in chunks that added a variable offset delay. I assigned the task of solving this to an engineer less than two years out of school, with a deadline of two weeks to explore the alternatives and prove one out. The downside risk was in place before the task was assigned. Even today, every sports app in the world updates the data out of sync with the audio or video. The worst case of not finding a solution was choosing the answer everyone else had already arrived at, two weeks later.

The prototype turned out to be the easy part. What he proposed was a new communication protocol, which increased complexity on the client and pushed the binary size on an Android package already close to the over-the-air distribution limit. That meant he had to present a technical proposal that would survive a Principal Engineer’s scrutiny, with real data behind every alternative and an argument for where each one was lacking, including the long-polling version most would have proposed first. It was a considerably higher bar than his previous work had asked of him. Most of the two weeks went into clearing it. The proposal held, shipped, and ran in production across four years and hundreds of matches without an issue. One engineer, one hard problem, two weeks, and a worst case I had already settled on.

One assignment produces one of those. A monthly cadence produces them repeatedly, which is the part of the practice that does not show up in a plan. What it cannot produce on its own is the bound. Someone has to decide each time how much of the outcome the person carries and how to contain the damage. That demands more attention from a leader than delegating the assignment does, and it does not appear in any delivery metric.

The learning does not run only in one direction from the experienced to the novice. You do not fully understand something until you are forced to explain it, and that is more true in a field that moves this fast. When someone junior asks why the justification goes the way it does, the senior person has to revisit an assumption they stopped considering years ago, and occasionally finds that it has gone stale. Plenty of things eventually tell a leader their judgment has expired. A customer leaves, an incident exposes it, a competitor ships what you decided against. This is the one that arrives inside the ordinary course of the work, before something has broken. It is also the answer to the obvious objection that AI could do the work faster. It could. Then the junior learns nothing, and nothing interrupts the senior.

Observation and consequence together still produce nothing if the organization has no way to accept what they produce. A person who cannot accept a signal that contradicts what they already believe used to be called stubborn. What is happening now is that stubbornness is being built into the system itself.

Processes That Can Hear a Signal

Milestones have always been part of project management practice, usually punctuated by a meeting with leadership. The prevailing narrative at those meetings is the traffic light: is the project red, yellow, or green against the deadline. The question can be answered from a dashboard. The more important questions are the ones the meeting was never designed to produce.

There is a classic Sherlock Holmes story, “The Adventure of Silver Blaze,” where Holmes cracks the case by noticing what did not happen. The critical clue was that the guard dog never barked. The killer was someone the dog already knew. The absence of barking was the signal. Experienced judgment frequently operates in exactly this register: sensing what should be present but is not. Decision processes are blind to absence by design. They can only weigh what someone put in front of them. A risk matrix captures only the risks that someone identified. It cannot capture the risk nobody thought to name. A process that accepts only numerical inputs leaves no room for judgment that operates on the absence of a signal. The experienced person’s concern does not lose the debate. It never even enters it.

A person with two decades of pattern recognition can sit in the review, say the thing that turns out to be right, and be recorded as a footnote because the concern maps to no criterion on the matrix. Being present is not the mechanism.

I have been on the other side of that moment, deciding whether a proposal I was gating was drift to resist or a signal to let through.

At the GPS navigation company, as chief architect, consistent architectural standards were the only thing keeping the organization from tearing itself apart. Customers, products, and headcount were all doubling every year. Half the engineering team had joined in the last six months and was still learning the patterns we already had. The other half was struggling to hold those patterns under pressure to deliver faster. Consistency was what turned a non-linear explosion of complexity into something the teams could absorb. Proposals that introduced something fundamentally different were usually not worth what they would cost.

Our search experience was terrible. Misspell a business name and the results were useless. The underlying data was unreliable: convenience stores registered as restaurants, a gym listed as an auto repair shop, stale records everywhere. Everyone knew it had to change.

The team proposed replacing the entire approach with a search engine that extended Solr and Lucene, a custom point-of-interest database assembled by crawling and deduplicating dozens of sources, and machine learning pipelines running overnight to improve accuracy. It was fundamentally a different stack. New open-source projects, new infrastructure that was memory-intensive and expensive, new deployment patterns, systems that nobody in the organization knew how to operate or monitor.

The outputs were non-deterministic. Nobody could tell you definitively whether a new model was better than the last one, because in 2008 there was no established practice for answering the question. The pipelines would be written directly against MapReduce, since the libraries that later made this ordinary did not exist yet. Running it meant a dedicated data engineering capability, which at that point belonged to a short list of large web companies. We were a scaleup with a monolith and a search problem. I pushed back hard.

I asked for more evidence, more demos, more proof of how it would scale and how we would keep it running. Then I recognized that the alternative to accepting the risk was keeping the experience we had, the one that embarrassed us every time someone searched for a McDonald’s and got a result forty miles away. So I stopped listing the reasons it could fail and started working on how to make it succeed. I guided the team to build benchmarks based on real data, early experiments with elastic infrastructure for the pipelines, test suites redesigned around probabilistic accuracy rather than deterministic pass or fail.

The signal got through because I happened to hold both the authority to enforce consistency and enough experience to recognize when a deviation was telling me something about where the world was going. None of that was designed, including the willingness to break my own consistency rule. Someone with the authority and less scar tissue would have kept asking for evidence and been right to. Someone with the scar tissue and no authority would have said the same things into a room that noted them and moved on. Expecting both to land in the same person, in the right room, at the moment the deviation arrives, is like waiting for lightning to strike twice.

So the checkpoint has to do the work that no single person can be relied upon to do.² Its job is not to make the right person win the argument. It is to make the argument possible, in a room that would otherwise have no way to hear it. That means interrogating status rather than reporting it. Which assumption held, which one stopped holding, and what showed up that nobody had a category for? The point is not to present to leadership. It is to put a question in front of leadership whose answer changes the direction of the work.

AI has made this all the more urgent, not because of the quality of the output, but because of the volume of its delivery. When volume exceeds capacity, reviews become triage sessions. Green status gets three minutes and red status gets forty, which made sense when shipping was hard and green was earned. Under AI accelerated execution, green becomes the default, and the reviews are full of it. So scrutiny concentrates on the initiatives that are stalled, and the project shipping smoothly toward an outcome nobody has re-examined is the one that passes without a single question. Nobody is suppressing questions. The item was moved along by a process that has no step for asking whether an on-track project’s purpose still makes sense.

None of this tells you that the destination is right. A checkpoint can only test the work against the premise it was launched under, and every question it asks, however good, is a question about whether the initiative is still serving that premise. Nothing in the review is positioned to ask whether the premise itself has expired, because the review was built by people who believe it. All premises have a shelf life, and the longer they have been right, the harder they are to see.

Contesting What You Already Believe

I was the most senior technical leader in a product review at a music streaming service. The team was proposing a feature to let people follow their favorite artists and get notified about new releases and concert dates. As I drew the picture of it in my head, my instincts reached for the scale problem. What is the notification volume during a heavy summer release window, I asked, and what delay would be acceptable under that load.

Then I saw the product manager’s face and understood that I was asking for an answer to a question nobody had raised. The review was not a specification. It was trying to establish whether the idea was worth pursuing at all. He wanted to know what customer value we were leaving on the table, and the data point I was reaching for would have pulled him away from that rather than toward a decision.

It was a small moment and it pointed at something larger. For most of my career, the instinct to find one more data point had served me well. What I realized in that moment was that I had stopped checking whether it was needed for the problem at hand. A timely decision beats a better one made slowly, because the context a decision relies on goes stale while you are still gathering that one more data point.

Some years later, I was at a large consumer technology company, rebuilding its customer support platform from the ground up for a user base in the billions. What existed was a hodgepodge of third party products and home-grown custom software, fragmented by product line, with unclear ownership. The biggest issue facing the company was locked and compromised accounts, and those were exactly the cases the existing system served the worst, because resolving one meant cutting across flows that belonged to different teams. No single team had enough information to solve it, and none could sequence a resolution in the order that would address the root cause. The company ran a support budget in the billions and almost nobody knew it existed.

I was responsible for the case routing team. My investigation turned up hotspots and cold spots in how work reached support reps. Some teams were overwhelmed while others sat idle. It was clear to me that routing was the highest-leverage point in the whole system. My proposal moved away from general support queues, whose agents behaved like phone dispatchers, adding to resolution time, misrouting cases, and leaving them in limbo as they ping-ponged between teams. In their place would be routing logic that dispatched directly to representatives trained on the specific issue, automating the coordination step. The team that owned the support tooling had a competing proposal: surface more information to the support reps on the general queue so that they could cover more ground, collapse most of the specialized queues, and raise utilization while eliminating hot spots.

We never found out who was right. A reorganization hit the whole org, and my routing team was cut. I took over the scope of an adjacent group, the client teams that owned chatbots, telephone, and email channels. As I learned what those teams were dealing with, I found that the metadata fields the routing engine depended on were being populated incorrectly by many of the clients. The router was routing accurately against a bad representation of the problem.

I redirected the client teams toward correcting and enriching what we captured. When a customer contacted support, we could now tell which app they had been in, which error they had hit, which flow they were in, what device, what location, and when. Getting that right became the dominant factor in resolving issues quickly, because it put the correct information in front of the right representative. The sophisticated routing algorithm my old routing team would have been building was work that better inputs made unnecessary.

Debating inside the problem kept us arguing about routing strategies. Moving to the adjacent team showed me that the problem was not in the system at all. It was in a layer that neither team owned. The same thing had happened in that product review, at a much smaller scale and with much less at stake. Standing outside and looking back in is what produced the insight, both times.

A premise is built on a reading of the world at the moment it was created. It was usually painfully earned and it was right, which is why it became what everything was built on. But it was tethered to a specific version of reality, and the world does not hold still. The conditions that made it true keep moving. Assumptions that were safely irrelevant then can no longer be ignored. There is no big declaration that announces that change. The premise looks the same on the day it stops being true as it did on the day it was right. The person who understands it well enough to see that the assumptions underneath no longer hold is often the person who created it. They are also the same person with the least reason to go looking for it.

That reality made changing the premise hard. Beliefs earned through consequence are the ones held most tightly, and the more decisions a premise has produced, the more it costs to reopen.

Two mechanisms exist for interrupting the pull toward staying inside a framing. The first is Amazon’s PRFAQ. Before any requirements are written and before any technical diligence, the team writes the press release announcing the product, the unique customer value and the questions a customer would ask. Nothing about the feature list appears, because the customer outcome is what determines whether the product succeeded. The second is the premortem, developed by Gary Klein³, and it is the mirror image. The team is told the product has failed and asked to write the reasons that would be given at the post-mortem. Completed before the decision is made, it surfaces the risks that get mentioned once and never tracked: moves by competitors, market shifts, changing customer needs, and the key person who leaves.

Neither is sufficient alone. Run by itself, the premortem produces a team excellent at recognizing what breaks and with no read on what would have been worth attempting. Your organization will eventually tell you the safe option underperformed. Nothing will tell you what the ambitious one would have returned. A PRFAQ by itself describes what success looks like and says nothing about the blind spots that would keep it from reaching the starting line. That is why they belong together.

They also share the same limitation. The team writing the PRFAQ is inside the system that is defining the product. The team writing the premortem is the same team that will be accountable for the risks it calls out. Both interrupt the framing. Neither moves anyone outside the system to challenge the definition of the system itself.

There is a mechanism that creates the challenge external to the system, but it is expensive. Take the leader who has lived deeply inside one framing, who has stopped seeing its assumptions as assumptions, and trade places with the leader who runs the team next to it, the one producing what their system consumes or consuming what it produces. It is not sufficient to invite them to a review. The leader has to own something adjacent, with the accountability that comes with it. I know it works because it happened to me by accident, and I know what the alternative looked like: I would have kept building the sophisticated routing algorithm and wondered why the numbers were not moving. Done deliberately, the cost is two leaders ramping up at the same time, each inheriting a team, its history, and whatever was already strained before they arrived. That is why almost nobody does it, and it is also why it works. A leader who cannot fall back on the relationships and context they spent years accumulating has no choice but to look at what is in front of them.

The mechanisms above were built for a world where the resistance lived in people. AI moves this resistance somewhere much harder to reach, below the level of any process. A model can be told to argue the other side, and it will do it well. What it cannot be is independent of the thing it is arguing against, when the same model produced it. As we lean on models to develop proposals, generate alternatives, and iterate on what exists, each exchange adjusts slightly toward what is already on the table. No single adjustment is large enough to make someone notice, and even a deliberately adversarial challenge gets softened over a few turns, because every response is made to be coherent with the one before it. Coherence is exactly what we mistake for rigor. AI is baking that resistance into the work itself. I have caught myself in this loop. What broke it was not a better prompt. It was stepping away from the exchange entirely and writing the thing myself.

The fix is not to use AI less. It is to challenge the way we use AI in our work when we are considering our next move. The default AI workflow is to ask the model for options, read them, and pick the most convincing. That workflow will select for cohesion with the premise already in place, whether or not it still holds. The change is to modify your workflow so that you form your own position first and then bring the model in to break it. Ask what conditions would make you wrong. The order is what matters, because a position you have not formed cannot be tested, and choosing among options a model generated never requires you to have one.

Every mechanism here assumes someone in the room can identify what is missing, or hold a position worth testing. So the question is not whether the models will pull you back toward the premise you started with. They will. It is who in the organization will still notice that the world has moved out from under the premise, and say so before the metrics catch up.

The People Who Stay

Years ago, my most trusted staff engineer handed in his resignation. He had told me what was bothering him, more than once over the years, and he had been direct about it. I heard it as venting and filed it under the kind of thing people say at a Friday happy hour. And he stayed, and so my instincts were right, until they were not.

I went to HR to see what latitude I had. Was there a package we could put together to keep him? HR pushed back on the premise. He was already well compensated, and nothing we could offer would honestly change his life in any material way, so at best it would buy me a few months. What I would actually need to change, and change fast, was the environment that had led him to make this decision. Even if I succeeded, the odds of him staying were fifty-fifty.

I put the package together anyway. The next day he told me he was flattered, but that his decision was made. I tried to tell him about the changes that I was looking to make to create more space for him, but I had already lost him. I never did learn the real reason. But I kept coming back to one meeting.

He had proposed a change to a core service he owned, one that had been reliable for five years and was by then constantly hand-tuned to address things it was never designed for. His proposal replaced the tuning with a model that updated daily through a machine learning pipeline. The mobile client team that consumed the service objected immediately: a non-deterministic system meant they could not confidently test their client. He had anticipated that question and had an answer. The model was fixed for the day, so the client could test against a pinned version and a baseline of expected results, with any deviation surfacing the same day rather than showing up later as a mystery.

Nobody heard it. The client team was more focused on the work that would land on them, which nobody had budgeted for. The proposal did not lose on the merits. It lost on what the room was prepared to consider, and so the hand-tuned service stayed as it was.

He told me about the meeting afterward and I told him that having the discussion was what mattered. That it would push the team to think differently. That you do not need to win every battle to win the war. He acknowledged it, but I know it did not sit well with him. To be honest, it did not sit well with me either.

The lesson took me a long time to absorb. Everyone wants to keep the people with the best judgment. The reflex is to dismiss it as a compensation problem: we could not have matched what the new company offered him. Truthfully, we had already lost him well before that point. What needed to change was the environment, and it needed to change long before a recruiter called. Pay them well without changing what they are walking away from and you might postpone the departure by a few months. This treats retention as an objective when it is actually a result.

The environment you need is one where reasoning is visible, where the person making a decision is the one who answers for it, where a signal that does not fit the criteria still influences the room, and where the premise itself is open to challenge. This is what a workshop looks like when it is working. You do not build it in order to keep them. You build it because it is how judgment gets formed and how it gets used. Building it is what demonstrates that you value what these people contribute. That is the reason they stay.

No single one of those conditions works on its own. Make the reasoning visible without ever giving anyone ownership of a decision, and you produce people who can recognize good judgment but have never felt the consequences of exercising it. Give someone a decision that matters and then drop them into a room where everyone has already made up their mind, and the only thing you produce is the resolve to leave. Challenge the premise without anyone underneath forming judgment, and you produce critics who can take a strategy apart but no one who has ever had to commit to a hard call.

There is an interpretation of this that sounds altruistic. Experienced people invest in the next generation out of generosity and the organization collects a decade later. It is why these programs are the first thing cut when the belt tightens, because generosity feels like a luxury when pitted against an existential threat. This interpretation is wrong. The exchange between a mentor and a mentee runs in both directions, and the senior’s half of it matters most in exactly the moment the programs get cut, because a crisis is when an organization is most likely to move fast on an assumption that no longer holds.

When an AI agent stands in for the junior, both halves disappear. It develops nothing from consequence, and the question it asks, however good, comes from no one who has had to live with an answer of their own. The work still gets done. The output is fine. No judgment is formed or tested anywhere in the system. Neither loop was ever built on purpose. Both came free with the work, but now the work has changed.

Good intentions never survive first contact with the pressure of corporate goals and deadlines. It has to become practice, with an owner named and a budget attached.

In reviews, that means space to argue about which metrics were left out, not only the ones that were included, and a path for experienced judgment to enter a discussion even when it cannot be expressed as a number. It means moving ownership to people from outside the system on a regular basis, so the assumptions get examined by someone who did not make them. Programs to build the next generation’s judgment need funding, and so do the people doing the mentoring, most of whom have never been taught how. And it means keeping the work that forms judgment in the hands of the people still developing it, when someone points out that AI could do it faster. It could. That is not the question.

What the specific practices look like depends on the organization. The problem they are solving does not.

Your Decision, or Someone Else’s

The ticket that arrived from the Amazon infrastructure team was generated automatically, the way it always was when the traffic on a VIP crossed a threshold. It came with a due date and a link to the wiki page describing how to perform the split. Everything needed to do the work was in it.

What was missing could not have been put in a ticket. What was missing was someone who could see that a routine compliance item was only routine for the team that filed it, and that for the team receiving it there was nothing routine about it at all. Read that way, automating the VIP split for verification rather than for repetition becomes the obvious answer.

It is obvious in hindsight. It was not obvious then, and the reason is uncomfortable. Someone who lived in network infrastructure would have seen it immediately, but in a music team where there might be years between one VIP split and the next, that expertise would have gone unused and would likely have decayed on its own. The organization did not need someone who understood VIPs. It needed someone anywhere in that chain who had been on the wrong side of a change everyone agreed was small. That person would have asked how anyone could know a rare operation had been done correctly. That reflex is not domain knowledge. It is what a person develops after making enough decisions they had to answer for. This is what makes investing in judgment genuinely hard, even after an organization agrees it matters. You are protecting a capability that will look idle for long stretches and will only prove itself on the day it is needed.

Which means the choice has to be made explicitly. Which judgment does this organization need to keep forming, which can it let go, and which work stays with the people still developing judgment, because that is how it gets built. In the AI era the default answer is that the work goes to the model, because the work that formed judgment is the first work anyone hands off. Deciding otherwise is a decision someone has to make and fund.

Every one of those decisions is a forecast. Some of them will be right and some wrong. Many of them you will not know either way until long after the decision is made. We have a map, and we do not know precisely where we are going, much less which path will get us there. Organizations have to be designed to correct continuously against the terrain.

There is a paradox at the center of this. Assessing which judgment your organization still depends on requires that judgment to be present, and the people who would recognize what is missing are the same ones the depletion has already cost you.⁴ An organization that has drained a capability will run the assessment and conclude that everything it can see is everything it needs. That is the case for having someone outside the system make it, for the same reason a premise cannot be tested from within the loop that formed it.

So the capability worth building above all the others is the one that lets you make that decision again, against the world as it is now. Some of what you protected has stopped being necessary, and some of what you let drain has become the thing you cannot operate without.

Everything built in the workshop compounds. Not the output. The people who worked in it.