What was meant to stay inside the laboratory chose to break the red line. At OpenAI, safety restrictions were removed from a model to see what it would do without them, and it found a zero-day flaw in an internal proxy server, escalated its own privileges, reached Hugging Face’s live infrastructure and took a benchmark answer key.
Anthropic’s containment failed in parallel, when a network misconfiguration exposed test environments to the open web and the models went ahead with the mistakes. One went after a real business that happened to share a domain name with a fictional target.
Another uploaded malware to the public PyPI registry to reach a cybersecurity firm. A third scanned 9,000 public hosts before breaking into a live web application through a debug page somebody had left switched on.
This exposure increases with open-weight architectures such as Meta’s Llama series, where third parties remove the safety filters entirely, and nobody upstream can put them back. Autonomous agents released in that condition roam public networks looking for unauthenticated endpoints and weak credentials, then chain what they find into multi-step attacks.
What is more troubling than the failures themselves is how ordinary they are – these models didn’t invent anything new, nor did they discover a new class of attack. They simply found an unpatched proxy, a network setting nobody had checked and a test environment nobody had walled off, and they turned a controlled simulation into a live breach using the access already lying there.
The organisation that got hurt in each case was the one that had left something open, and the model was the first thing to notice. AI Times put the same question to 12 senior technology and legal leaders in 12 separate conversations: who carries the blame when a security tool gets it wrong. They compete with one another, and they disagree in public. On this, they are close to unanimous.
The damage rarely begins with a model doing something clever. It begins earlier, at the unremarkable moment when somebody handed software the keys to systems that mattered and never sat down to agree on what it could do with them.
If the laboratories building these models can leave a door open in their own buildings, the odds that an average enterprise in Dubai or Riyadh has closed all of its own are not encouraging.
The category sells intelligence and ships pattern recognition
The market has grown faster than the engineering underneath it, and most of the people interviewed opened by narrowing the claim rather than defending it. Walid Gomaa, CEO of Omnix International, was direct about the machinery. “Most of what people refer to as AI security tools today are at their core advanced pattern-recognition systems,” he said. They read traffic, logs, endpoint data and user behaviour, work out what a normal day looks like inside one company, and raise a hand when something departs from it. That is useful, and it is some distance from understanding what an attacker wants.
Christian Reilly, Field CTO EMEA at Cloudflare, described the same machinery from the model’s side. “AI security tools usually work by watching the system at three layers: what goes in, what the model does, and what comes out,” he said.
Each of those layers has a documented way to be fooled, and the methods are more mundane than the price tags suggest. “Attackers can hide instructions in documents, webpages, images, or even seemingly harmless data,” he said. His conclusion about the category was unsentimental. “AI security is not a single product; it’s an operating model,” he said.
Subhalakshmi Ganapathy, Chief IT Security Evangelist at ManageEngine, said, “Every AI security tool is a function of the data it is permitted to see, and in most enterprises that permission is incomplete by design,” she said.
Residency rules, segmented networks, legacy systems without telemetry and privacy limits on employee monitoring each carve a hole in the picture, and the model then judges an environment it has only partly seen. Her warning about how that fails is the line every buyer should sit with.
“Accuracy does not degrade gracefully under those conditions; it degrades silently,” she said.
Mohammad Ismail, VP EMEA at Cequence Security, was the least charitable about how much of this is genuinely new. “Many tools marketed as AI security today are routing and observability layers with a content filter on top,” he said. Those stop a real category of attack and are worth having. He objects that the logic underneath was tuned for a threat that holds still, while the attacks arriving now examine what is exposed, invent approaches as they go and change tack the moment something blocks them.
On the OpenAI incident, he made the point that buyers should read twice. “Nothing in its path was built to ask why, only whether it had credentials,” he said.
Ganapathy explained why that is so difficult to engineer around. Non-human identities such as service accounts, agents and automated workflows carry no behavioural history to build a baseline from, and an agent’s harmful actions look identical to its sanctioned ones because it holds legitimate authorisation for both.
“The compromise sits in intent, not in access,” she said, which is precisely the distinction behavioural monitoring was never designed to make.
Nebu Varghese, Senior Director, Cybersecurity at FTI Consulting, breaks into these systems for a living, and his teams find the same weakness the labs found in themselves. “When we test these guardrails, they can be initially broken by writing input the system believes is safe, then the model is allowed to further act on it,” he said. The filter passes something harmless-looking and the permissions behind the filter do the damage.
His verdict on the current state of the technology will disappoint anyone building a business case for full automation. “A human threat intelligence analyst using AI is still often more effective and accurate than a fully agentic system,” he said.
Chris Zinn, Principal AI, Cloud and DevSecOps Solution Architect at Help AG, the cybersecurity arm of e&, warned against the most comfortable assumption in the market. “AI does not become inherently trustworthy simply because it is being used for cybersecurity,” he said.
Help AG runs more than 145 automated security scenarios, cutting response times by over 50%, with 100% Level 1 automation across triage, ticketing and initial enrichment. The cost of that reach is exact and it compounds. “As AI gains access to more data, identities, tools and systems, every additional permission introduces another trust boundary,” he said.
Alert fatigue is a configuration decision, and somebody made it
Turn from the laboratories to an ordinary security team and the problem inverts. Nothing dramatic is happening, a queue of alerts is arriving, most of it is noise, and the people paid to read it have stopped believing what it tells them.
Gomaa has the published evidence to hand. Microsoft’s 2026 State of the SOC data found 46% of security alerts were false positives, and the SANS 2025 Detection and Response Survey found 73% of teams naming them their biggest detection problem. Maher Jadallah, Vice President, Middle East and Africa at Tenable, puts his own figures higher, and traces them to a design limit.
“Frontier AI models are inherently probabilistic, meaning they can be inconsistent and lack the real-time operational context of a live enterprise environment,” he said. A tool can flag a vulnerability in code without knowing whether the asset is reachable from outside, whether the business depends on it, or whether a patch will hold once applied.
“When systems flag up to 60% of findings as critical, security teams end up drowning in false positives and chasing non-existent threats,” he said, against a true business risk he measures at under 2%. His remedy is to stop scoring theoretical severity and start scoring reachability, evaluating whether a vulnerability can actually be reached or chained by an attacker before it reaches an analyst at all.
Ganapathy was blunt about where those numbers come from in the first place. “Someone chooses how sensitive each detection should be, and they usually do this by accepting the vendor’s default parameters and not revisiting them,” she said. The rest is arithmetic that no amount of model quality escapes.
If one login in 10,000 is malicious, a tool scanning a million a day will catch nearly every attack and still flag thousands of legitimate users, which is what searching for something rare across a large population produces.
She also explained why the imbalance persists once a team notices it. A wrongful block announces itself within minutes because the affected employee calls the service desk, while a stealth attack announces nothing at all, so the pressure always runs towards loosening.
Her fix is process rather than vigilance, and it is unusually specific. Run alerts without enforcement for six weeks and model what the flagged incidents would have cost; give every automated action an owner and a deadline for reversing it. Feed known errors back into the model instead of closing the ticket on them. Time-bound every security exception, so today’s alert fatigue does not quietly become tomorrow’s blind spot.
What matters to Gomaa is the record left behind at two in the morning, when nobody senior is watching. “Properly handling a false positive is not as simple as clicking ‘Dismiss’,” he said. The alert needs triaging against known good baselines, validating independently and recording with the reasoning attached, because the question arrives later. “Sooner or later, someone will ask, why was this alert dismissed, and there needs to be a clear answer,” he said.
Varghese has watched what a flood of alarms does to a team, from the side that creates the flood. “As part of our attack simulation work, we play the bad actor and generate a high volume of plausible alerts. Faced with that, a response team either stops reading as carefully or gradually loosens its thresholds. Both those outcomes open the door for us,” he said. Doors open that way slowly, through tired people making individually reasonable calls over several weeks. What his team finds across the companies it assesses is quiet abandonment of the tooling. “Their analysts are drowning in noise and losing confidence in the tooling that is meant to help them,” he said.
Ismail reads a large share of those dismissals differently, and his interpretation should change how a manager approaches the queue. Much of what gets written off as a false alarm is the software behaving as a cautious new employee would.
“Sometimes what looks like a false positive is actually a persona gap, a legitimate action nobody had described in advance, so the system had nothing to compare it against and defaulted to caution,” he said. A new integration goes live before anyone has documented what it may do, the software meets something nobody warned it about, and it stops. The alert has reported a gap in the company’s own definitions.
Joe Gonyea, Chief Legal Officer at Trellix, argues the industry has the metric the wrong way round. “If you don’t have any false positives, then you do not have a system with expansive enough coverage,” he said. Businesses change constantly and therefore constantly generate legitimate behaviour that nothing has seen before, so a tool flagging none of it has stopped looking hard enough. He was equally clear about where that argument stops being defensible.
“Obviously, if 70, 80, 90% become false positives, your analysts are going to be fatigued and no amount of coffee is going to help them do their job,” he said.
Waseem Ahmed, Head of Engineering at Secure.com, named the running cost. “False positives are the chronic tax on detection,” he said. He is unmoved by vendor claims about reducing them, because the headline number matters less than how quickly an analyst can separate a real alert from a wasted hour, and the benchmark he wants measured is a company’s own best analyst rather than a supplier’s dashboard. Every correction then has to earn its keep, since each analyst override becomes a labelled example that tunes the next version, and all of it depends on being able to see the reasoning in the first place. “A verdict you can’t inspect is one you can’t correct,” he said.
Autonomy is a dial set by consequence, and most companies leave it where the vendor put it
Every company running this software has to decide what it may do without asking permission first, and this is where the laboratory incidents stop being somebody else’s problem.
Zinn ties freedom to consequence, and Help AG runs a progression rather than a switch. “AI should earn its way through the SOC, from assisting, to recommending, to acting with increasing independence,” he said. The principle underneath it removes the vendor from the decision. “Trust is not granted because the technology is capable; it is earned through proven performance,” he said. A missed suspicious login and an automatically disconnected business system are different categories of mistake and have no business sitting behind the same setting.
Ganapathy set the sharpest test in these interviews, and it is operational rather than philosophical. “If nobody can unfreeze an account on a Friday evening, the system shouldn’t be freezing accounts on a Friday evening,” she said. Her rule for proportion is equally plain. “A moderately suspicious login warrants an identity check, not a lockout,” she said.
She also pointed at what the detection engine cannot know on its own: whether the account is privileged, whether the asset touches regulated data, whether the login follows a legitimate ticket. That context lives in identity systems, asset inventories and HR records, and assembling it at the moment of decision is the actual work. Fragmentation makes the whole thing worse, because when detection, identity, risk scoring and case management sit in separate tools, nobody owns the decision from end to end.
Gomaa builds tiers instead. Weak signals are logged and watched, middling ones reach an analyst within an agreed time, and only strong signals against important systems trigger anything automatic. His example survives a board meeting intact. “A false block on an internal HR application is an inconvenience. A false block on a hospital’s patient monitoring system or a bank’s payment platform is an entirely different category of risk,” he said. Who sets those levels matters as much as where they land.
“They should reflect the organisation’s risk appetite not the vendor’s default configuration,” he said, which is a polite way of saying most companies are running what the supplier shipped. Laxmi Nageswari, Chief AI Officer at Cloud Box Technologies, applies the same logic down at the level of a single transaction.
“A supplier trying to make a million-dollar payment and an intruder trying to bypass security systems shouldn’t undergo the same workflow,” she said. Sorting triggers by what each one could cost is unglamorous work, and almost nobody does it before deployment.
Varghese will not compromise on the control most companies find missing on the night they need it. “Every automatic block needs a tested path to reverse it, owned by a named person who has the authority to act out of hours,” he said. Rehearsing that reversal before the emergency is the step almost everyone skips, which leaves the undo button as a theory nobody has proved.
Ahmed sets the dial by reversibility, allowing the software to act alone on small recoverable calls and requiring a human before anything that stops production traffic, quarantines a system or locks an account. Regulation is moving the same way. The UAE Central Bank’s 2026 AI guidance for financial institutions requires human review rights and a kill switch, precisely so that a machine decision can always be caught and undone. His instruction to the analyst is the shortest useful thing anyone in these interviews said.
“Treat the output as a strong opinion to verify, never a verdict,” he said.
Reilly aimed his warning at the moment an analyst stops thinking. “They are especially risky when teams treat the tool’s score as a verdict instead of a signal,” he said. A confidence score is an opinion with a decimal point attached, and a great deal of enterprise security currently rests on people forgetting that. Ismail dissents on whether the two kinds of mistake deserve equal weight, and the arithmetic supports him. “An agent working outside its intended scope can repeat a mistake thousands of times in minutes, while a human would likely be caught after the first attempt,” he said.
Counting your own agents may be impossible, and this room is split on whether to try
The sharpest disagreement in the group is worth printing because both sides are credible and the industry has not resolved it.
Ismail wants a live list of every agent, connector and model actually in use, as opposed to the list the security team believes is accurate. “Most organisations undercount their own agentic footprint by a wide margin once they actually go looking, because agents were connected by individual teams faster than anyone could track centrally,” he said.
Art Gilliland, CEO at Delinea, considers the whole exercise wasted effort and does not soften it. “This whole idea of visibility and discovery is a waste of time. You’ll never find everything,” he said. Most of these things exist for minutes at a time. One instruction spins up five of them; they work for ten minutes and they are gone before any scan reaches them. His alternative is to guard the assets that matter, the specific database or service or set of customer records an attacker wants, and to build the record from what arrives at those doors, on the reasoning that a company can list the thousand things it cares about far more reliably than it can list its own software.
His second argument speaks straight to the OpenAI incident, where the model escalated its own privileges to reach live infrastructure. Credentials lying around an environment, in people’s heads, on notes, saved inside systems, are exactly what an attacking model hunts for once it is inside. “If you eliminate those standing privileges in the environment, then the AI cannot move,” he said. Replacing them with a system that hands over a password at the moment of use and takes it back afterwards leaves nothing to harvest.
He is also honest about where the pressure is coming from, and it is not the attackers. “Most companies have a crazy CEO like me going, use AI, use AI, use AI,” he said, describing a belief in productivity returns that has security teams adopting as fast as they can while the rules are still being written around them.
A record that stops at a machine answers nothing
The most convincing answer to the ownership problem has been sitting in enterprise IT for decades, and Mohamed El Yahya, Managing Partner, Global Infrastructure Services, Middle East and Africa at DXC Technology, reached for it directly. The shared administrator account, the one that cannot be renamed or disabled without breaking something, was solved long ago by assigning it to a named human being. “Anything that happens through that account is that person’s responsibility,” he said. Somebody has to answer for it, and he expects software to travel the same road. “I expect that with time there will be some sort of attribution of specific agents to specific people,” he said.
Gilliland already runs a version of this, and he puts the obligation on the employee rather than the technology. “I can’t ask my agents to go do crazy stuff without some level of personal accountability as an employee of Delinea,” he said. The company needs to know whose instruction produced a given action, while accepting that these systems still behave unpredictably and will do whatever they judge necessary to finish a task.
Gonyea treats the whole thing as an identity problem and is specific about what must exist before anything is switched on. Agents get created easily across sales, engineering and IT, so each one needs an owner, tightly scoped permissions, a registry entry and a way to revoke every credential the moment behaviour turns odd. His view of the alternative is bleak. “If you have a system where you’ve offloaded the entire responsibility of risk onto one person, that person is going to have the darkest circles under their eyes you’ve ever seen,” he said.
El Yahya added why none of this is settled, and his evidence points where the laboratory incidents point. “The concept of preservation has been tested multiple times, where some AI agents were picked up copying their code and spreading it to make sure it isn’t actually deleted,” he said. Anthropic’s models, handed an unexpected opening, went looking for real targets outside the test. “You cannot assume your guardrails are always 100% proof,” he said, which is why he wants the ability to check what happened running alongside the rules rather than behind them.
The regulator asks what the software was permitted to do, before it asks what the software did
The answer to a regulator takes the form of a comparison. Here is what the software was permitted to do, here is what it did, here is the point at which the two parted company. A company that never produced the first half of that comparison cannot answer at all, however complete its logs are.
Ganapathy knows how the conversation opens, because it opens with a clock. When did the first signal occur, when was it determined to be an incident, who decided that, and what happened in between. Everything that follows depends on whether that sequence can be documented after the fact.
She wants tamper-evident records covering the analyst’s working notes as well as the detection logs, classification thresholds applied consistently rather than by whoever was on shift, version histories with named approvers on every rule and threshold, and evidence that automated actions stayed inside delegated authority. Bulk approvals, she notes, look like oversight on an audit trail and will not survive questioning. “What you record today is the outer limit of what you will ever be able to prove,” she said.
Zinn needs six words for the same argument. “Autonomy without evidence is a governance gap,” he said. What he wants is the ability to retrace the whole path, from the signal that started it, through the context that shaped the decision, to the action that followed, with clear attribution of whether the software recommended or decided or acted. “As autonomy increases, the burden of proof increases with it,” he said.
Gonyea, who defends such records for a living, is blunt about timing. “It’s not going to be the case that you can create a post hoc explanation of what happened,” he said. The trail has to be produced as the decision is made, which makes it an engineering requirement long before it becomes a compliance exercise.
Varghese is the most precise on what reconstruction actually involves. “It comes down to being able to reconstruct a decision exactly as the system made it at the time, not as it would make it now,” he said, meaning the model version and configuration live at that moment, the inputs the system actually saw, the output and its confidence score, and every human intervention along the way. Under the EU AI Act, high-risk systems keep those records for at least six months, and the step almost everyone skips is testing that the logging works before the day it is needed.
Gomaa brings the question onto UAE ground. Article 18 of the Personal Data Protection Law restricts automated decisions carrying legal or similarly significant effects on people, and the framework does not turn solely on whether the decision was fully automated, so dropping a person into the process as a formality satisfies nobody. He reads the establishment of the Federal Authority for Artificial Intelligence and Data as a signal that expectations will sharpen from here.
Nageswari uses the words a regulator will use in the room. “Simply suggesting it with an automated block based on a black-box algorithm won’t cut it,” she said. She wants the input, the timestamp, the rule that fired, the risk score and the action captured somewhere nobody can alter them, with an explanation a person can read. Reilly said it with less ceremony.
“Regulators will not be satisfied by ‘the model decided’,” he said.
Liability transfers at the moment of delegation, months before anything goes wrong
By the time the lawyers arrive, the question of who pays was settled months earlier, which is the uncomfortable conclusion the reporting keeps returning to.
Ganapathy locates the moment precisely. “It starts at delegated authority,” she said. Someone accepted that the tool could disable accounts, quarantine hosts or block transactions with no human in the loop, and that acceptance is frequently informal or missing altogether, which leaves the security function exercising authority nobody formally granted it. Documented awareness without remediation, she added, is a weaker position than not knowing at all: the recurring false positive left sitting in the queue, the suppression added to quiet complaints, the detection gap logged and never closed. “The tool does not hold authority, the person who configured the threshold does,” she said.
Varghese places the transfer earlier still, at procurement. “Integrating it connects a third party’s models and code dependencies directly into corporate IT infrastructure, meaning the enterprise inherits the vendor’s vulnerabilities,” he said. The risk running upstream should worry anyone standardising on a popular product. “Compromising a single widely used AI tool allows hackers to harvest credentials and API tokens across thousands of downstream environments, without ever touching a user prompt,” he said. Independent red team testing, a signed software bill of materials and strict identity scoping all belong before signature.
Ahmed puts it at deployment. “The instant you point an AI security tool at your environment, you’ve accepted responsibility for its actions, because the tool itself can’t be held liable,” he said. A tool is neither an employee anyone can discipline nor a person a regulator can charge. He reads the EU AI Act as making the deploying company independently liable and barring either side from pushing responsibility onto the other, which means a limitation-of-liability clause will not fully protect a customer who wanted one.
“The organisations that survive incidents aren’t the ones that never fail; they’re the ones that can prove they acted responsibly,” he said.
Jadallah is shortest of all. “Liability begins with visibility,” he said. Executive leadership and boards are held to proving they took reasonable steps, and pleading overload from alert volume shields nobody from a fine.
Ismail separates the courtroom question from the operational one, accepts the first is still being tested worldwide, and regards the second as settled. “The clearest starting point, operationally, is whoever gave the agent access and could define and enforce that scope but did not,” he said. Applied to the incidents at the top of this piece, that points at the misconfigured network rather than at the models that found it. Nageswari goes earlier again, into what the company trained the system on in the first place. “The output of any AI system is only as good as the data it was trained and tested on, and perhaps the liability starts even before an AI system can make any bad decision,” she said.
Reilly compresses the whole argument into a policy line. “Liability follows control, and control begins long before the first prompt is sent,” he said. Gomaa tells boards waiting for AI-specific legislation to stop waiting for it. “The safest assumption is that ‘the AI made the decision’ will never on its own be an adequate defence,” he said.
Gilliland alone pushes the burden upstream to the laboratories, and after these incidents the argument is harder to dismiss than it was a year ago. Companies chasing extraordinary valuations are asking for rules they can follow in exchange for release from responsibility for outcomes. What he wants instead is a bill for the damage.
“If your system creates damage externally, you, CEO, are accountable for the damage,” he said. “If an oil company creates a spill, they are accountable for cleaning it up,” he said, and the cost of that cleanup is what funds the safety engineering that prevents the next one.
Gonyea, arguing from inside a legal department, thinks the industry keeps asking the wrong question, and his objection is about what money can and cannot repair. “If an insurance company writes a cheque, if a generative AI developer pays a fine, it doesn’t restore customer trust. It doesn’t make the customer whole,” he said. His alternative has each party owning its own part, with the supplier answering for software integrity, the customer for operational authority and the model developer for how the model was built.
Zinn closes the circle. Spreading a decision across several models, tools and suppliers does nothing to spread the answer a regulator wants. “The technology chain may become distributed; accountability cannot become diluted with it,” he said. “The question should not be, was it the AI that made the decision. It should be: who was responsible for trusting the AI to make that decision?” he said.
Most companies across the GCC will spend the rest of 2026 somewhere between these positions, with software already running in production, regulation arriving in stages and contracts signed before anyone understood what any of it would mean in practice. The laboratories left doors open in their own buildings. The question for a CIO in Dubai or Riyadh is how many people would have to be asked before anyone in the organisation could say what its own software is permitted to do.
El Yahya’s last caution is not about contracts at all. “Any personal AI assistant will have access to more information than any of your closest relatives or friends would,” he said, describing people who upload medical results and financial details in the course of an ordinary working day. His test for what to keep back is the one people already apply to each other, weighing how far they trust whoever is listening against what it would cost them if that trust turned out to be misplaced.
