Where human judgement belongs in an AI workflow

The temptation to put a person at the end

When organisations discuss the risks of using AI, somebody will often suggest keeping a human in the loop. It sounds reassuring, and sometimes it is an appropriate safeguard. But I am not convinced that placing a person at the end of an automated sequence tells us very much about how responsibility will work in practice. If the person is expected to approve a recommendation without seeing the evidence, understanding the alternatives or having time to question the result, their involvement may be little more than a formality.

This matters particularly in manufacturing, where everyday workflows contain decisions of very different kinds. Recording the arrival of a document is not the same as deciding whether a product meets a customer’s specification. Finding previous examples of a defect is not the same as authorising a concession. A system can assist with all of these activities, but the kind of assistance and the controls required should depend on the nature of the decision. Before designing the workflow, I am looking for where judgement is actually being exercised and why.

Judgement is distributed through the work

It is easy to imagine a process as a sequence of steps, with routine administration at the beginning and an important decision at the end. Actual work is rarely so tidy. An employee may decide that a customer enquiry is sufficiently clear to proceed, recognise that a drawing refers to an obsolete revision, or notice that a requested delivery date is inconsistent with production commitments. These decisions may be made quickly, almost without comment, because the person has developed experience of the work. They are nevertheless consequential. They determine which information is accepted, which cases are escalated and which risks become visible to somebody else.

When I investigate a workflow, I therefore try to find the moments at which someone could reasonably have done something different. What information did they have? What were they uncertain about? Which rules applied, and where did they need to interpret the situation? Who would have been affected if they had chosen incorrectly? This often reveals that judgement is present much earlier than the formal approval stage. It may also reveal that the organisation relies heavily on a few experienced individuals whose reasoning has never been made explicit.

There is a practical difference between applying a stable rule and interpreting an uncertain situation. If a record is incomplete, a system can check whether required fields are present. If an order exceeds a defined credit limit, it can route the case for approval. But deciding whether an unusual quality deviation is acceptable may depend on engineering evidence, the intended use of the product, customer requirements and the consequences of failure. The decision may involve rules, but it is not necessarily reducible to a single rule. Treating these activities as equivalent simply because they appear in the same workflow creates unnecessary risk.

What AI can usefully contribute

AI can be particularly helpful in preparing the ground for judgement. It may retrieve previous incidents, compare documents, identify missing information or assemble a summary of evidence that would otherwise take an employee considerable time to collect. In a supplier quality investigation, for example, the system might identify earlier complaints involving the same component, extract the recorded causes and show whether corrective actions were documented. An engineer could then examine the relevant evidence rather than spending much of the investigation locating it.

The usefulness of this assistance depends on how the result is presented. A confident paragraph that conceals gaps in the records is much less helpful than a clear account of what was found, where it came from and what remains uncertain. A system that acknowledges it cannot establish whether a corrective action was completed is more useful than one that quietly assumes completion because a report contains a proposed action. The first response preserves an important question for the engineer. The second risks turning an absence of evidence into an apparent fact.

There is also a difference between recommending an action and carrying it out. A system might suggest that a quality incident resembles a previous case, or that a particular production order deserves attention. Whether it should be permitted to change a record, release stock or communicate a commitment to a customer is a separate question. The consequences of those actions determine the controls required. In some cases, a reversible administrative update may be acceptable within well-defined limits. In others, the organisation should require explicit human authorisation supported by evidence that the person can inspect.

The problem with nominal approval

Human review is often discussed as though it were a single, reliable control. In practice, the quality of that review depends on the conditions under which it happens. A production manager receiving dozens of AI-generated recommendations may find it difficult to examine each one carefully, particularly if the system has been correct most of the time. The more fluent and authoritative the recommendations appear, the easier it may become to accept them without investigating the assumptions beneath them.

This is why the review needs to be designed around the decision, rather than reduced to an approval button. The reviewer needs to know what the system is proposing, what evidence supports the proposal, what information is missing and what would happen if the recommendation were wrong. They also need sufficient authority and time to reject it. If an employee is measured only on how quickly they clear a queue, we should not be surprised if the review gradually becomes procedural rather than substantive.

Some decisions may benefit from more structured review. An engineer considering a proposed disposition for non-conforming material might need to see the specification, measured deviation, intended application and relevant customer requirements. The system could assemble this information and highlight gaps, while the engineer remains responsible for the disposition. The review then has a clear purpose: exercising technical judgement on a defined body of evidence. That is a stronger arrangement than asking somebody to confirm that an AI-generated answer looks reasonable.

Designing for uncertainty and escalation

One of the most important questions is what happens when the system cannot reach a dependable conclusion. Organisations sometimes regard these cases as failures because they interrupt automation. I think they can be a sign of a well-designed boundary. A system that recognises missing information, conflicting records or an unfamiliar situation can direct the case to someone with the knowledge and authority to resolve it. The workflow should make this possible without forcing the employee to reconstruct what the system has already examined.

The escalation route needs to be practical. If a question is referred to an engineer, what will they receive? Can they inspect the original documents? Will they know which checks were performed and which assumptions were made? Is there a clear record of the final decision? These details matter because judgement is not simply an individual mental activity. It is part of an organisational process in which decisions must be communicated, implemented and sometimes explained later.

It is also important to distinguish uncertainty that better information can resolve from uncertainty inherent in the decision itself. A missing inspection result may be resolved by obtaining the measurement. A judgement about the acceptable balance between delivery pressure and quality risk may require consultation and formal authority. AI can assist in both situations, but it should not be expected to eliminate the second merely by processing more documents.

Making responsibility visible

A useful AI workflow should make it easier to establish who decided what, on what basis and with what authority. That does not require an elaborate governance framework for every small business. It does require clarity about the decisions that matter. For a limited pilot, I am looking to establish the proposed system outputs, the actions they may influence, the circumstances requiring review and the person or role responsible for the consequential decision. The next test is whether those arrangements still hold when information is incomplete or a case is unusual.

There is a wider benefit to doing this work. Many organisations already have unclear decision boundaries, even before AI is introduced. People rely on experienced colleagues, informal approvals and assumptions about who is entitled to make a particular call. Examining where judgement belongs in an AI workflow can reveal those arrangements and provide an opportunity to improve them. The value may come as much from clarifying the work as from the technology that eventually supports it.

I am interested in AI systems that strengthen professional judgement by making relevant evidence easier to obtain and uncertainty harder to overlook. That is a more demanding design objective than simply keeping a person in the loop, but it is also more useful. The question is not whether a human appears somewhere in the process. It is whether the organisation has preserved meaningful responsibility at the points where the consequences of a decision require it.