Human in the Loop: how to transform a job with AI

We usually recognise a job by its name or by the outcome we expect from it. Yet to understand how that work is done, we need to distinguish the actions that make it up. What does the person receive? What information do they seek? What leads them to choose one path and discard another?

Some actions transform information; others produce something in the physical world. Decisions connect what we know with what should happen next. Breaking down that set of activities helps us recognise where each capability contributes and what we need to preserve when reorganising the work.

Comparing two quantities may require no more than a conventional application. Recognising a pattern may justify machine learning; interpreting a request or preparing an explanation may benefit from generative AI. The choice starts with understanding what transformation is needed and what information is available.

The ability to carry out a task also depends on conditions. A tool may work well with complete data and need help when a reference is missing. Alongside allocating tasks, we therefore need to recognise when each part can proceed and when someone else must intervene.

We call this human involvement within an automated process Human in the Loop. The expression becomes meaningful when we can identify where a person is needed, what they receive and what they must return for the work to continue.

The agent uses a model and tools to look up information or carry out actions. The person contributes the capability that this part of the process requires. For the exchange to work, both must meet around a comprehensible task: what remains unresolved, what makes it possible to resolve it and what changes with the response.

Consider a reply that requires expert knowledge. The agent can find information, connect it and prepare a draft. The person’s work changes: they receive a proposal accompanied by the material needed to understand it and judge whether it addresses the issue.

For that review to have value, it must be able to change what happens next. If an action requires approval, it waits for the decision. A correction returns to the process; uncertainty may call for another enquiry. Keeping a record of those interventions allows us to review how the system works, although the model does not automatically learn merely from receiving them.

Here we encounter a human capability different from expert judgement: acting on material objects and situations. In a warehouse, for example, the agent can indicate a location and the worker can move the item, record the result or report what prevented them from doing so.

We then need to understand what the system can verify. A scanned code confirms that a label was read; on its own, it does not prove where the box ended up. A photograph may provide another signal. We design the check around what we need to observe and turn to a person when the evidence is insufficient.

If most cases can proceed within the expected conditions, human involvement concentrates on matters that need additional judgement. Rather than resolving every case from beginning to end, the person can attend to situations arriving from several executions.

That possibility depends on a concrete relationship: how much attention is requested and how much the reviewer can provide. It also matters whether enquiries can wait and whether they arrive simultaneously. A visible queue, with owners and priorities, helps organise that workload and recognise when more help is needed.

The agent can gather information, prepare cases and maintain their state while different people contribute what is needed: a review, a missing piece of information or an action. This combines the system’s speed of preparation with the team’s capabilities.

Preparing a task and being able to perform it are different capabilities, however. If the system generates more work than people can handle, the backlog grows. Organising assignments, allowing obstacles to be reported and limiting how much work is started are all part of the same design.

A person can request an analysis, an agent can consult another tool, and an expert can resolve a question before the process continues. These exchanges connect because the result of one task provides what the next one needs.

The network remains meaningful when we can reconstruct those relationships. Each case needs to retain its identity, its state and responsibility for the next step. We can start with a small exchange and connect others once we understand what they contribute and how to check their results.

When someone says “it depends” or “you can tell straight away,” there is something worth understanding. What does it depend on? What detail are they recognising? We can reconstruct a case, preserve the explanation and use AI to distinguish tasks, data, criteria and exceptions.

The interpretation we develop goes back to the expert: we need to check whether it expresses their judgement or whether we have connected something incorrectly. These iterations help us reach sufficient understanding for our purpose. What someone usually does still needs to be discussed before it becomes a rule in the system.

How to capture expert knowledge with audio and AI

A proof of concept turns our interpretation of the work into something observable. We choose a limited scope and agree on what would need to happen for us to consider it successful. We then try cases comparable to those in the current process.

We include a routine situation, missing information, an uncertain case and an intervention that does not arrive in time. We measure quality, total time, review workload and cost. This lets us distinguish which parts of the design work, what we misunderstood and what needs adjusting before we expand its scope.

A faster step can pass the difficulty on to the next one. We therefore need to measure the journey: from the start of a case until we obtain a result of the required quality. This helps us recognise whether we have freed capacity or merely moved the backlog elsewhere.

We then need to ask what we can do with that capacity. Handling cases that currently wait, improving a reply or doing more work are different possibilities. Financial savings depend on how the change can be put to use; they cannot be inferred solely from the time an AI takes.

The job changes as we redistribute its tasks and decisions. Some interventions are no longer needed; others require more judgement, knowledge or ability to act. The people who do the work need to participate in that design and be able to learn to use what is introduced.

The first test leaves knowledge, criteria and tools that we can reuse. On that basis, we decide the next step: adjust what still does not work, apply what we learned to another task or connect a new exchange. Each advance expands our possibilities from something we have already understood and been able to check.