The enterprise AI conversation is entering a new phase. Until recently, much of the focus was on how artificial intelligence could assist people: generating content, analysing information, accelerating individual tasks or helping employees make better decisions. Agentic AI changes that equation.
AI is beginning to move from assisting with work to acting within it. Agents can increasingly interpret objectives, support or make decisions, coordinate tasks and execute across workflows with less direct human intervention. As that capability advances, an important question is moving to the centre of the enterprise agenda: where should the human sit?
The instinctive answer is often “keep a human in the loop”. But that phrase risks oversimplifying a much more important challenge.
If every action taken by an AI agent requires human approval, organisations simply recreate the friction and bottlenecks they are trying to remove. At the other extreme, autonomy without clear boundaries creates obvious questions around risk, trust and accountability.
The challenge, therefore, is not to keep humans involved in everything. It is to design the right relationship between human judgement and machine autonomy.
From assistance to agency
This distinction becomes more important as AI moves deeper into the enterprise.
Enterprise-grade AI cannot simply exist as another feature or application. Its real potential emerges when intelligence becomes embedded across systems, workflows and decision-making. At that point, we are no longer discussing isolated AI use cases. We are beginning to redesign how an organisation operates.
That requires a different way of thinking about human-in-the-loop.
That relationship will not look the same everywhere. There will be processes where agents can operate autonomously within clearly defined parameters. There will be decisions where AI can analyse, recommend and accelerate, but a human should remain accountable for the final judgement. And there will continue to be areas where human expertise must lead, with AI acting as an enabler.
The important question is not whether humans or AI should be in control. It is where autonomy improves the outcome, where human judgement changes it, and where accountability must ultimately sit.
Human judgement becomes more valuable, not less
As AI takes on more execution, the role of people does not disappear. It changes.
An AI system may be highly capable of optimising towards an objective. But someone still needs to determine whether it is the right objective.
This is where domain expertise becomes critical. A subject-matter expert understands not only whether an AI system has completed a task correctly, but whether the approach makes sense in the wider context and whether it is delivering the outcome the organisation actually intended.
Consider public services. An AI agent might successfully automate a process, reduce processing time or remove several manual steps. Those are useful measures, but they do not necessarily tell us whether the service has improved. The more meaningful question is whether the citizen experienced a better outcome: Was friction removed? Was the service easier to access? Did it become more responsive to what the individual actually needed?
That distinction between completing a process and understanding its purpose is where human judgement remains fundamental.
From governing AI to governing agency
Agentic AI also changes the governance conversation.
Much of AI governance to date has focused on models, data and use cases. As AI systems gain the ability to act, governance must increasingly address agency itself.
What information should an agent be allowed to access? What decisions can it make independently? What actions can it execute? At what point should it escalate to a person? And when something goes wrong, where does accountability ultimately sit?
These cannot be answered through technology alone. They require organisations to establish clear guardrails based on the risk, consequence, complexity and reversibility of different decisions. The greater the consequence of a decision – and the harder it is to reverse – the stronger the case for human judgement and accountability.
Security becomes part of that architecture as well. Greater autonomy means greater access to enterprise systems, data and workflows. Permissions, monitoring, escalation and auditability therefore need to be designed into agentic environments from the outset, rather than added once systems are already operating at scale.
This is why human-in-the-loop should not be understood simply as an approval mechanism. It is part of a broader architecture of accountability.
Autonomy cannot compensate for weak foundations
There is another reality organisations need to confront: sophisticated agents cannot compensate for fragmented enterprises.
Many AI initiatives perform well in controlled environments but struggle when organisations attempt to industrialise them. They encounter disconnected data, legacy technology, siloed decision-making and workflows that were never designed for intelligent systems.
Human oversight will not solve those structural problems.
If AI is to become part of the enterprise infrastructure, organisations need to modernise the foundations beneath it and rethink how work moves across functions. They also need governance that connects AI initiatives to clear enterprise outcomes, rather than accumulating hundreds of disconnected experiments without a common direction.
The technology may be advancing rapidly, but execution remains the harder challenge.
The opportunity for the Middle East
This is particularly relevant in the Middle East, where governments and businesses are moving quickly from digitisation towards AI-enabled operating models.
In markets such as the UAE and Saudi Arabia, the ambition is increasingly not simply to digitise existing services, but to rethink how they can operate around intelligence. That creates an opportunity to design AI-enabled services around autonomy, human judgement and accountability from the outset, rather than retrofitting those principles once systems are already operating at scale. For the region, that could become an important advantage: not simply adopting AI faster, but designing intelligent operating models without inheriting all the assumptions of the pre-AI enterprise.
The organisations that lead this next phase will not necessarily be those that automate the most. They will be those that make the clearest choices about where autonomy creates value, where human judgement remains indispensable, and how the two work together.
The goal should not be maximum autonomy. It should be intelligent autonomy.
Intelligent autonomy means giving AI the freedom to act where speed, scale and consistency create value, while deliberately placing human judgement where context, consequence and accountability demand it. AI brings increasingly sophisticated execution; people bring domain expertise, judgement and accountability. The real opportunity lies in designing how those capabilities work together.
The future is therefore not AI versus humans. It is AI with humans. The enterprises that lead will be those that understand when machines should act, when people should judge, and how accountability connects the two.
