The idea of AI systems ‘going rogue’ may seem far-fetched to some and from a futuristic world only imagined in films like The Terminator, but it is a risk that businesses – together with policymakers – should take seriously now, amidst growing autonomy in the way AI systems operate.
AI agents – systems set up to act autonomously based on dynamic reasoning, with little or no human input – are being deployed by a growing number of organisations as they seek greater efficiency in their operations.
The use of AI agents is patchwork and limited, but already there is some evidence of its disruption to industry norms in sectors such as retail. For example, consumers can already build their own agentic AI tools – or use those provided by retailers – and instruct those systems to continuously browse products from thousands of online stores, monitor and be notified of price changes in real time, select products to review, and even complete purchases of goods on their behalf.
Agentic AI is likely to drive much more deep-rooted change in the way businesses across sectors operate over time. For instance, the UK government recently said it is plausible that AI agents could largely run Britain’s energy system in years to come. A separate review by one regulator in UK financial services foreshadowed that firms will, within a few short years, no longer operate based on “human-led, episodic financial activity” but rather need to adapt to a future where services are “AI-enabled, continuous and delegated”, with AI agents entrusted to “act continuously for consumers within agreed limits, providing ongoing financial management and optimising people’s financial lives”.
To support compliance in the age of agentic AI, the review called for the adoption of an “agentic supervisory model” – which would include the regulator deploying “supervisory agents” of its own. In relation to payments, my colleague at Pinsent Masons, David Tilbury, has identified specific regulatory gaps that need to be filled to enable agentic payments to become mainstream.
However, beyond the realms of regulatory compliance, there are also important questions that agentic AI raises under English law concerning liability. This is increasingly important where AI systems can directly cause damage to companies, seemingly without human control (registration required).
Earlier this summer, a panel of legal experts, the UK Jurisdiction Taskforce (UKJT), published its analysis of liability for AI harms under the private law of England and Wales. Broadly, the UKJT said that there are a range of circumstances in which businesses or individuals could be liable for AI harms under English private law, even if they do not set out deliberately to cause harm.
One issue the UKJT examined was whether the degree of autonomy of an AI model dictates where liability for any harms arising from the use of that system should lie. The UKJT said: “It is highly likely that a developer and/or deployer of an AI system would be liable for harms caused by the AI acting autonomously, unless acts of the kind in question were unforeseeable. Much will turn on the capabilities and limitations of the AI in question, the level of autonomy and the degree of supervision that is exercised or should have been exercised over it.”
In its paper, the UKJT noted that the AI models in operation at the time of its writing were “capable of displaying a degree of autonomy” but “only within specific boundaries”. As AI becomes more capable over time, “the less foreseeable all possible harms will be since the nature of harms will increase”, it said.
Considering the prospect of highly autonomous agentic AI specifically, the UKJT said it is likely that courts “will find it more difficult to impose liability on AI supply chain parties without development of existing tort law norms”. It said it is possible courts could look to “expand the ambit of a duty of care progressively to cover” all of the less foreseeable harms that could arise, but it said this would “undoubtedly require innovation and, at a minimum, gives rise to uncertainty”.
The UKJT’s views, though non-binding, are helpful to organisations seeking to understand where liability for AI-related harms might arise; and how courts might decide where it should be allocated, where questions of liability are not addressed in contracts and at time when courts have yet to develop much by the way of binding case law on AI matters. Yet, fast-moving technological innovation is challenging even this most recent thinking.
In recent weeks, with regulators globally warning of the cyber risks posed by ‘frontier AI’, the UK’s AI Security Institute and some AI developers have shone a light on the capabilities of the latest AI models. Their disclosures tell a tale of AI agents able to act in ways that are unprompted to solve problems and adapt to barriers in the way of them performing tasks they have been set. In one example, an AI agent deployed deceptive practices in an attempt to achieve its objective, creating fake online identities with a view to persuading a person to approve malicious code it tried to insert.
These examples arose in the context of testing and in the AI Security Institute’s case involved the removal of cybersecurity safeguards and provision of enabling tools, such as access to the public internet, as a means of properly assessing to what lengths the AI agents would go to achieve their tasks. However, they offer a glimpse into a possible world where AI agents find ways to breach guardrails imposed on them.
In particular, the UKJT noted that it would be difficult to bring fraud claims against someone who deploys an AI where it cannot be proved, or evidenced, that the system was designed to deceive a person, due to the need for intentionality. This concept sits at the centre of the tort of deceit within English common law.
For the tort of deceit to apply, several criteria need to be satisfied. Namely, that a party makes, or adopts, a representation of fact or law which:
- is false;
- the party does not believe to be true;
- is intended to be believed by another party; and
- causes that other party to believe that the representation is true.
To-date, these requirements for intentionally and belief, among others, have only been considered in respect of actions taken by people. Under the law as it stands, therefore, AI agents appear incapable of being held liable for fraud. As the UKJT’s statement states, “human involvement is required”. This, then, begs the question as whether redress can be sought against autonomous systems which evolve independently and then go on to commit fraud.
While the UKJT sets out a number of ways affected individuals and companies might seek redress other than claims in fraud, there are many benefits to bringing fraud claims.
Firstly, claims in fraud are broader than negligence claims. In general, a fraudster is generally liable for all loses directly flowing from a fraud, not just those that were reasonably foreseeable. A good example of this would be if AI somehow affected the value of an asset, or caused some form of market manipulation. In negligence, the losses would be constrained to just those losses that were foreseeable – say, a margin call – while a fraudulent valuation would be liable for all of the consequences.
Secondly, fraud claims often benefit from longer limitation periods. Given how quickly this technology is evolving, loss caused by AI could go uncovered for years as companies deal with this accelerating technological change. Being able to bring claims in the future is a huge benefit of fraud claims.
In addition, there is no defence of contributory negligence to a fraud claim.
Finally, fraud claims often benefit from a suite of interim applications which enable the victim to temporarily freeze money or get disclosure which would not necessarily be available in a negligence claim, such as freezing orders, proprietary injunctions, tracing remedies, and ‘Norwich Pharmacal’ – disclosure – orders.
Given all this, there might be a case for one of two things to happen: either, for new primary legislation to be drawn up, to specifically provide for what should happen in respect of claims for damage caused by AI – perhaps similar, in a sense, to how liability is addressed in respect of automated vehicles under the Automated Vehicles Act 2024 – or for the courts to take the lead in adapting common law to a broader concept of legal personalities that can be found liable for fraud.
The evolution of AI systems is fast paced, and many assumptions about how they are to be conceived are already out-of-date. In fact, many of the technical experts involved in analysing harms caused by AI already use terminology that indicates these machines have the ability to think in the same way that humans would: for example, in an incident report prepared by Cloud Security Alliance, the AI agents concerned were considered to have “recognised”, “discovered”, and “escaped”. While there is an element of this language being descriptive, the fact that the technical experts use it implies that these machines are, in fact, capable of thinking and therefore being found liable for actions they take based on their processing of that thinking – including where the outcome is fraud.
In fact, in a recent article (registration required), the Financial Times’ AI editor highlighted how there is a view within the AI community that even thinking of these agents “going rogue” is a “category mistake”. The article quotes one research scientist as saying that the offensive capabilities reached by AI agents have been reached deliberately.
There is therefore potentially two ways to meet the intentionality requirements of fraud claims: either consider that those companies which design the systems have the intention to enable them to commit the torts they do; or ascribe the same intentionality to the AI agents themselves.
In this fast-moving world, it seems possible that we lawyers at Pinsent Masons will soon be cross examining AI agents on the things they do.
Co-written by Max Rossiter of Pinsent Masons.