Agentic AI: the basics

The predictions of growth, made by analysts MarketsandMarkets, reflect the fast-changing nature of AI technologies, while a study by Deloitte (46-page / 6.9MB PDF) highlights the eagerness of businesses to explore the potential of AI systems that can act in an adaptive way, without ongoing human instruction, to perform operational functions and make decisions.

In this guide, we look at how early adopters of agentic AI tools can manage the risks associated with the technology, explore where legal responsibilities lie for the way those tools operate, and examine how agentic AI is raising questions relating to legal personality, regulation, and intellectual property (IP) rights.

What is agentic AI?

The concept of AI agents – robots or systems that are programmed to perform tasks on an autonomous basis by processing data and interacting with their environment – has been around for some time.

Deep Blue, the chess-playing computer system that famously defeated chess champion Gary Kasparov in a series of matches in 1997, is an example.

Deep Blue’s victory was a triumph for expert programming and supercomputing. It was programmed with the benefit of chess grandmaster input and it had access to 200 million data points to help it decide what game strategy to pursue – with state-of-the-art processing power for its age enabling it to process the data and take decisions quickly.

Commercial examples of AI agents include customer service chatbots, which can evaluate a problem and serve customers with information within pre-defined parameters, as well as tools on asset manager platforms that help to guide retail investors on selecting financial products to invest in based on how the investor responds through a decision tree-style process.

The difference between AI agents and agentic AI, however, is that agentic AI systems are set up to act autonomously on the basis of dynamic reasoning, with little or no human input. This works through a process known as ‘chaining’, where agentic AI systems start with a specific prompt and a set goal and deliver their end result by performing a series of sequential tasks, with the process being informed by the outputs – and the systems’ interpretation of those – at each stage. Agentic AI systems determine how to act by perceiving information they receive and adapting to their environment without needing precise human instruction.

In operating, agentic AI systems leverage ‘static’ AI agents and other tools and resources, to build additional capabilities, like reasoning, planning and self-evaluation.

For organisations, agentic AI provides a raft of opportunities. It can help them achieve back-office efficiencies, undertake complex analysis, and support with financial risk management, fraud detection, and with supply chain and logistics management, for example.

Risk and governance

The sophistication with which organisations have responded to the development of AI tools has matured as the technology – and the risks and opportunities it brings – has evolved.

Agentic AI not only enhances existing risks but introduces new risk areas due to its autonomous nature. Managing these risks without constraining the system to effectively remove its utility is the challenge, particularly with the technology at a nascent stage.

Responsible businesses seeking to explore the potential of agentic AI should educate themselves on the risks and opportunities and identify low risk use cases to pilot and work up from, particularly given some concerns that there is a rush to deployment without proper governance and research.

The risks

Specific challenges exist around misaligned actions where the agentic AI system takes actions that are not aligned with an organisation’s policies and values.

Bias in the way agentic AI operates can also arise, because of which tools the system selects to achieve its overall goals or if it elects to modify a dataset.

There are increased cybersecurity risks – not just to the system itself but to its external touchpoints, which might be targeted by cyber criminals and then used to manipulate the agentic AI through malware, to pursue unintended goals or compromise commercially sensitive or personal data.

As with other forms of generative AI, there is also the risk of agentic AI systems generating false, but plausible, information – and using it in a way that could cause harm. For example, these ‘hallucinations’ could lead to mistakes if the agentic AI system uses the information to ask other tools to complete actions.

There are also governance risks that businesses deploying agentic AI must consider. For example, users could place an over-reliance on the capabilities of agentic AI. The chaining effect can mean unexpected outcomes arise and transparency and explanation issues are compounded with agentic AI systems. More broadly, employee resentment or disengagement could make it harder to maintain cohesion and effective governance, since greater effort will be required to manage this across the whole system’s lifecycle – from procurement and development, to deployment, management, evaluation and assurance.

Actions businesses can take

Organisations should build on existing legal and ethics AI policies, education and lifecycle governance and assurance frameworks against the backdrop of existing regulations and developing best practice on policy, principle and guidance.

Examples of useful frameworks to refer to include the EU AI Act and its codes and standards to come, the international standard ISO 42001, the US National Institute of Standards and Technology (NIST) AI risk frameworks, cybersecurity standards and legal requirements globally, and data protection laws and relevant guidance, such as the European Data Protection Board’s report on privacy risks of LLMs.

Specific projects will need to be assessed and managed accordingly. At the outset, information needs to be sought on system uses, developer information, system components, functionality and the tools used, the right to access the full model specification, and on whether there is a requirement that the chain of thought must be human intelligible.

There needs to be an initial risk assessment and ongoing risk evaluation, as well as clear assignment of responsibilities to appropriate team members. Data flows should be mapped and appropriate guardrails put in place – including around effective data management, testing, accessibility of components, permitted levels of agency and appropriate restrictions, data quality, transparency, fairness, explainability, evaluation and accountability.

Agents’ access to external tools should be strictly controlled, and agents should have the lowest necessary privileges to do the job. Strict memory controls and isolation of memory between agents and sessions should be implemented, along with memory audits and integrity checking. Logging and security monitoring are also important measures. Considering how best to keep a human in the loop as the technology matures will be another important area to address.

Businesses should also adopt robust cybersecurity risk management measures and processes, to determine if and where something has gone wrong and how to fix it in a dynamic environment. Probing for vulnerabilities through ‘red teaming’ is an option that businesses could consider. This would involve ethical hackers stress testing systems and enables businesses to better understand how their incident response plans will hold up during an incident.

Businesses will also want to monitor for the development of new standards and benchmarks as these can provide practical guidance to developers and deployers, though they will need to keep things under review to ensure they remain relevant and useful as the technology continues to advance.

Allocating responsibilities

The nature of agentic AI increases complexity in terms of the ability to evaluate the system, find and address errors and issues, and audit behaviours and outputs, as well as in relation to the overall assessment of compliance and assigning of responsibility for actions.

It is overly simplistic to assert that developers or providers should be responsible for the development stage and deployers for deployment given the multiplicity of tools and resources agentic AI will make use of. Deployer integration, fine tuning and modification activities may mean they may become a provider for regulatory purposes – such as under the EU AI Act – and this will affect the appropriate allocation of contractual risk and responsibility.

Given the complexity of systems and the increased difficulty of transparency and explainability, establishing where and why something goes wrong, when unexpected outcomes occur and raise issues around blame and causality, a rigorous, well-informed approach to procurement is needed. This should entail suitable warranties and ongoing commitments to transparency, explainability, monitoring, audit and accessible assistance. Suitable KPIs and detailed governance arrangement should also be put in place.

Businesses with strong bargaining positions could consider imposing presumptions of responsibility in some circumstances. Current performance metrics should be considered in the light of changing technology and the specific context. They could, for example, take account of the system’s goal completion rate, contextual understanding, user satisfaction, and continuous improvement.

Whatever the contractual terms, businesses deploying agentic AI should ensure employees are closely involved to ensure safe, responsible operations and outcomes. Responsibilities should be allocated to ensure ongoing close monitoring.

The question of legal personality

There has been discussion ongoing for years as to whether, and if so in which circumstances, AI systems, chatbots or robots have or should have legal personality, and associated legal rights and duties, in their own right.

The idea that robots should be considered in law as legal persons was floated by MEPs in the context of proposals they asked the European Commission to draw up in relation to new civil law rules on robotics in 2017. However, hundreds of experts from across the worlds of science, technology, law and ethics opposed the idea and resulted in the MEPs abandoning the idea.

In 2020, in a precursor to what resulted in the development of the EU AI Act and reforms to EU product liability rules, the European Parliament said that “any required changes in the existing legal framework should start with the clarification that AI-systems have neither legal personality nor human conscience”. It added that “all physical or virtual activities, devices or processes that are driven by AI-systems may technically be the direct or indirect cause of harm or damage, yet are nearly always the result of someone building, deploying or interfering with the systems” and concluded, as a result, that it was “not necessary to give legal personality to AI-systems”.

The Canadian case of Moffatt v Air Canada reaffirmed the principle that, generally, organisations are responsible for the acts or omissions of the computer systems they use and for misrepresentations they make to the public, irrespective of whether it comes from a human representative or an automated chatbot.

Similarly, in the US case of State Farm Mutual Auto Insurance Co. v Bochorsf, the court had no trouble in holding that the insurance company was bound by a contract renewing a policy formed by its computer.

Further developments in IP law further support the established legal position. The UK government is currently considering removing scope for computer-generated works to benefit from copyright protection, citing a lack of evidence of the use of the provisions currently in effect. In addition, courts around the world have dismissed the notion that AI systems can own or be assigned patent rights under current legislation.

The basic premise at the heart of these various developments is that AI systems are only a tool and are the result of someone building, deploying or adjusting them. However, the increasingly autonomous nature of an AI system’s operations does start to stretch the assertion that it only acts in accordance with the information and direction supplied by its human programmers.

A series of articles and academic papers have explored the issue and drawn analogy with commercial agents, while the concept of ‘digital co-workers’ is also now being widely discussed instead of mere use of AI ‘tools’.  However, the legal position in relation to legal personality in many countries is clear and to change this would require a wholesale rethink of the law – something that will not happen quickly.

For businesses, having appropriate policies, governance and safeguards in place is the best mechanism to defend against potential claims of negligence. These measures, as well as suitable contractual terms on data protection, cybersecurity and IP, among other things, will help contractual parties address risk and responsibility issues.

Regulation

Arguably, the most significant piece of AI-related legislation that has been drawn up to regulate use of the technology and those that provide or deploy it is the EU AI Act. The broad territorial reach of the legislation means how to comply with the AI Act is something businesses all around the world need to consider in the content of developing or using agentic AI.

The AI Act has introduced a tiered, risk-based system of regulation, under which some AI systems are completely prohibited and others – ‘high risk’ AI systems – subject to the strictest requirements. Further rules apply to so-called ‘general purpose AI’ (GPAI) models.

As well as impacting on EU-based businesses that provide, deploy, distribute or import AI systems or GPAI models, the AI Act applies to organisations outside the EU where they place those systems or models on the EU market or putting them into service on that market or where their outputs are used in the EU.

Given the components of agentic AI, the rules on GPAI models are particularly relevant to providers. Among other things, they require transparency over training data and the operation of an EU copyright law-compliant copyright policy. In addition, the providers need to draw up technical documentation, including training and testing process and evaluation results, and make that information available to regulators, as well as to businesses downstream from them in the supply chain that that intend to integrate the GPAI model into their own AI system, to inform them on capabilities and limitations. Additional requirements apply to the providers of GPAI models ‘with systemic risk’.

It is possible that certain agentic AI could be classed as both a GPAI model and ‘high-risk’ AI system under the AI Act, which would entail providers having to meet two sets of compliance obligations. For high-risk AI, requirements apply around risk management, technical design and documentation, record keeping, data governance, human oversight, downstream information provision, and conformity declarations, among other things.

Standards are being developed to provide more detail on these requirements and contract terms will need to dovetail with these. The standards could act as a guide for deployers of agentic AI seeking to procure systems from providers, to help information negotiation over contract terms, even if the requirements around ‘high risk’ AI do not apply in their context.

IP considerations

For businesses deploying agentic AI, there are IP risks to consider – including around what data is stored and shared by agentic AI systems, their users, or the third-party tools they interact with.

Businesses should seek to undertake their own testing of agentic AI during procurement to better understand how the system works and the data-related risks, to decide what company data they are comfortable exposing to the system. Their attitude towards risk should be underpinned by carefully negotiating data sharing terms with the agentic AI provider.

As well as considering the potential for inadvertently enabling third party access to commercially sensitive or confidential information, the risks of sharing personal data – of staff or customers – should also be considered. The EDPB’s report on AI privacy risks and mitigations in relation to LLMs is an extremely useful resource in this regard.