Acceleration Without Understanding

The objective is good judgment, not human judgment

A useful starting position: a great deal of human judgment in operating systems should be replaced, and much of it already has been.

Nobody estimates optical loss by looking at a fiber. The instrument is better, and the argument for the instrument was settled long ago. The same holds across a long list of decisions that were once matters of experienced opinion and are now matters of measurement — protection switching, thermal management, load balancing, alarm correlation. In each case a machine performs the judgment more consistently, more quickly, and at a scale no person could match, and the resulting system is better for it.

Capable AI systems extend that list considerably, and there is no principled reason to stop extending it. The objective in operations was never to preserve human judgment as such. It was to produce good judgment, and where a machine demonstrably produces better judgment on a given decision, delegating that decision is an improvement rather than a loss.

This matters as a starting point because the more common framing — that automation threatens something valuable in human expertise — leads to defending the wrong things. Plenty of operational discretion is worth removing. What deserves protection is not human involvement but decision quality, and decision quality depends on conditions that have nothing to do with who or what is doing the deciding.

The question worth asking is therefore narrower and more practical. Before an organization accelerates a process with a machine, what does it need to understand about that process, and what happens when it turns out not to understand it?

The invoice that nobody could specify

The clearest way in is through work that appears completely routine.

An organization decides to automate invoice review. The case is strong. The volume is high, the work is repetitive, the staff performing it are expensive and bored, and the process is documented. The documented process has four steps: the invoice arrives, it is reviewed for discrepancies, it is routed for approval according to value and category, and it is paid.

Implementation begins with what should be a formality — specifying what the agent is supposed to do.

What counts as a discrepancy?

The answer comes back as a description of a person rather than a rule. The analyst who has done this longest knows which ones to stop. Pressed further, it emerges that they have a working sense of which vendors submit reliably, which categories run over for legitimate reasons, and what an invoice from a particular class of work should look like at a particular time of year. None of this is written anywhere, and the analyst has never been asked to articulate it because the organization has never needed it in articulable form.

How does the analyst know which ones to stop?

Partly the system flags them. The flag is a colour in a screen, and it has been trusted for years because it is broadly right.

What makes the system flag them?

Nobody in the room knows. The rule was configured at some point by someone in the finance systems group, possibly against a tolerance that made sense when it was set, and the person who set it has moved on. The rule is doing real work in the process and exists as an unexamined artifact.

At this point the automation project has stopped being an automation project. It has become a discovery exercise, and what it is discovering is that the organization did not have a process. It had a working arrangement: a documented workflow, plus a set of software behaviors nobody had recently examined, plus undocumented decision rules living in one or two people, plus historical exceptions preserved as habit, plus a layer of individual judgment that was genuinely load-bearing and genuinely invisible.

None of this was concealed. It accumulated the way all working arrangements accumulate — by successive small adaptations that worked, none of which was large enough to require documentation. The organization was operating it successfully every day and could not describe it.

Automation as an audit

The reason this pattern matters is that it is almost universal and rarely anticipated, and it puts the difficulty in the wrong place on the project plan.

An attempt to automate a process reliably exposes several things at once. It surfaces decision logic that was never documented, because the machine cannot infer what the analyst infers. It identifies where institutional knowledge is concentrated in individuals, which is usually a shorter list than anyone expects and constitutes a standing operational risk regardless of whether automation proceeds. It reveals dependencies on system behaviors nobody currently understands. It exposes objectives that were never specified precisely enough to be optimized — whether the goal is to pay faster, to pay more accurately, to preserve working capital, or to reduce headcount, which are four different objectives producing four different agents. It shows where the evidence the decision requires is thin or absent. And it makes visible how exceptions have actually been handled, which is generally by a person who knows what to do rather than by any defined path.

Every one of these is a finding about the organization rather than about the technology. They would all be true without AI, and in most cases they were true and unexamined for years.

This is not an argument against automating the process. It is an observation about sequence. The work of establishing what the decision rules are, where the evidence comes from, what the objective actually is, and how exceptions should be resolved is prerequisite engineering. It has to be done by somebody. Automation does not remove it; automation is what makes skipping it visible, and usually at the point where a budget has already been committed and a date has already been promised.

An organization that treats that discovery as a project failure has misread it. The information was always missing. The project is the first thing that required it to exist.

Where the physical world sets the boundary

Everything above concerns processes that live inside systems, where the evidence the machine reasons over is at least present somewhere. Field operations add a harder constraint.

There is a great deal that capable systems can do well in this environment, and the potential is substantial. They can interpret test results against expected values, correlate alarms with physical topology, narrow a fault to a probable domain, retrieve the maintenance and construction history of a specific asset, recommend which test to run next, check a proposed action against permit and access constraints, and sequence a workflow so the right resource arrives with the right equipment. Each of those compresses time in a process where time is expensive, and several are things people do badly because they require recalling more history than a person holds.

The boundary is not in the reasoning. It is in the evidence.

A machine cannot reason about a physical condition the organization never observed. If optical loss on a span matters to a decision, somebody has to have measured it, with a calibrated instrument, and the result has to have been recorded against the right asset. If burial depth matters, the evidence of depth has to originate at the moment of construction, because it cannot be recovered afterwards without excavation. If the condition of a cabinet matters, someone or something has to have looked at it. No amount of model capability substitutes for an observation that was never made, and no amount of correlation across adjacent data recovers a measurement that does not exist.

This makes the operating chain concrete. Physical reality produces evidence, but only if something captures it. Evidence acquires meaning through context — what this asset is, where it sits, what has been done to it. Context is bounded by constraints, which are the permits, access windows, safety rules and commitments that determine what is actually permissible. Machine reasoning operates across those three and produces a determination or a recommendation. That drives a physical action, and the physical action generates new evidence, which either feeds back into the record or is lost.

Every link in that chain is a capability the organization either has or does not have, and the quality of the machine reasoning is capped by the weakest one. This has a direct implication for sequencing: the value an organization can extract from AI in field operations depends heavily on the evidence environment, and an organization with poor records and thin instrumentation will get less from a better model than an organization with good records will get from a worse one.

It also sets a design requirement that is easy to state and frequently ignored. A system operating in this environment should be able to recognize when the available evidence does not support a determination, and say so — requesting an observation rather than producing a confident answer from insufficient input. An agent that returns the evidence does not establish this; measure the loss at the midpoint is more valuable in operations than one that always returns an answer, and considerably harder to build, because the training and evaluation pressure runs the other way.

Six failures that look identical

When an automated system produces a wrong operational answer, the finding is usually recorded as a model problem. That diagnosis is right some of the time and expensive when it is not, because the responses diverge sharply.

The evidence may have been wrong — a test recorded against the wrong asset, an as-built that does not match the ground, a reading from an instrument out of calibration. No change to the model fixes this, and a better model may make it worse by reasoning more confidently from bad input.

The context may have been missing. The evidence was correct and the system did not know something it needed, such as the fact that this route was rebuilt two years ago or that this site is fed from a different path than the records suggest.

The objective may have been underspecified. The system optimized what it was asked to optimize, and the organization wanted something it had not stated. This one is particularly common and particularly hard to see, because the output is not wrong against the specification.

The process logic may never have been documented. The system implemented the written process, and the written process was not what the organization does.

The decision criteria may have been inadequate. The thresholds were arbitrary, inherited, or set for conditions that no longer apply — the red flag in the invoice system, configured against a tolerance nobody has revisited.

Or the reasoning may genuinely have been poor, which does happen and is the only one of the six that a better model addresses.

Treating all six as the same failure produces one of two organizational responses, both wrong. The system is abandoned as unreliable, and the underlying deficiencies stay in place for the humans to keep absorbing. Or the model is replaced, at cost, and the same failure recurs because the cause was never in the model.

The argument is about sequence

Nothing here counsels against automating operations, and an operator who waits for a complete evidence environment before beginning will wait indefinitely while competitors compress cycle times.

The caution is narrower. Automation applies force to a process. Where the process is well understood, the evidence is sound and the objective is clear, that force produces speed and consistency, and the gains are real. Where those conditions are absent, the same force produces faster, more consistent, and more confidently presented versions of decisions the organization was already making badly, at a scale that makes them harder to notice and harder to unwind.

What separates those two outcomes is not the capability of the system. It is whether the organization understood, before it accelerated, what its own process actually was, where its evidence came from, and what it was trying to achieve.

Most of the work of acquiring that understanding is worth doing whether or not anything is ever automated. An organization that has documented its real decision rules, located its concentrated knowledge, specified its objectives and assessed its evidence is better run for having done so — and is, at that point, ready to accelerate. That is the order these things belong in, and the opposite of the order in which they are usually attempted.