AI survey places emphasis on supplier oversight

Ian Stott and Luke Scanlon of Pinsent Masons were commenting after the Bank of England and Financial Conduct Authority survey found that third-party AI systems are becoming both more prevalent and more material across the sector.

According to the survey, 86% of UK financial services firms now use AI in some way, including within their own operations and IT functions, but also to support risk and compliance, sales and marketing, and human resources and legal teams. Most AI tools now used in the sector are generative-AI systems.

The use of AI agents, tools that operate with a degree of autonomy with little human input, is also becoming widespread, with 80% of firms already using AI agents and a further 14% planning to deploy them over the next year. However, the AI agents in use typically have limited autonomy and are mostly being deployed to support decision-making. Less than 10% of survey respondents said they expect their AI agents to “approve or decline customer outcomes; execute trades or place orders; initiate, execute or block payments; or change limits, thresholds and controls” within the next 12 months.

Of the AI systems in use in UK financial services, 39% are provided by third parties, up from 17% in 2022, while third-party systems account for more than half of “high-materiality AI systems” in operation. The survey identified that many third-party AI systems relied on by financial services firms are hard to substitute, but also highlighted firms’ concerns around information security, operational resilience and transparency, as well as the challenges they see in negotiating contractual protections with third-party providers.

Stott, an expert in financial services regulation, said: “As AI becomes more embedded, more agentic and more dependent on third parties, traditional governance frameworks are going to be tested. It is no longer enough to have an AI policy, an inventory and some governance committees.”

“Boards and senior management increasingly need to be able to answer some core questions, in real time: Where is AI operating? What is it influencing? What data and third parties does it depend on? Who is accountable? And, critically, how do we know it is producing the outcomes we expect? That is about regulatory observability: moving from having governance to being able to see, understand and evidence control in practice. The survey findings suggest that challenge is becoming very real, very quickly,” he said.

Scanlon, who specialises in technology contracting in financial services, said: “As regulated businesses increasingly rely on third-party services enabled by AI, it is clear that the regulators expect firms to invest more in oversight. The FCA’s recent publication on good and poor practices confirms that firms need to consider a range of risks presented by third-party AI use, including resilience, overreliance on AI vendors, and their ability to plan effectively for future change.”

“The FCA has highlighted the need for contractual requirements to obtain notice and further information about significant model changes. This means that it is not enough to simply understand how the model works today; firms must also have a commitment from third-party providers to be notified before significant changes could introduce new or different risks,” he added.

Earlier this year, Pinsent Masons highlighted that the use of AI by suppliers in service delivery can alter the risk profile and, in some cases, would justify financial services firms seeking increased protections in their contracts. Pinsent Masons also said financial services firms should change the way they conduct supplier due diligence to account for the use of agentic AI tools.

Access the Pinsent Masons report on AI-enabled service delivery