Australian firms face increasing pressure to monitor AI performance drift over time

One area of focus is ensuring that AI systems continue to perform accurately and reliably throughout their lifecycle, and remain within the risk tolerances organisations are prepared to accept for their intended use cases.

Much of the regulatory and public discussion around AI risk has centred on the problem of AI hallucinations. However, regulators are also paying increasing attention to a less widely discussed risk: model drift.

In its April 2026 industry letter, the Australian Prudential Regulation Authority (APRA) highlighted concerns about regulated organisations' "reliance on point in time and sample-based assurance models, despite these methods being ill suited to probabilistic models that learn, adapt and degrade over time". The comments underscore a growing regulatory expectation that organisations continuously monitor the performance of AI systems rather than relying solely on pre-deployment testing.

Broadly speaking, drift refers to the deterioration of an AI system's performance over time due to changes in data or changes in the relationships between input and output variables. Often referred to as model decay, drift can reduce the reliability of predictions, increase error rates and contribute to poor decision-making.

Drift can arise in several ways, including:

• data drift, where the characteristics or distribution of input data change over time;
• concept drift, where the relationship between inputs and outputs changes; and
• label drift, where the frequency or meaning of outcome labels changes over time.

Regulators are increasingly emphasising that the deployment of AI systems is not a "set and forget" exercise. Ongoing monitoring, validation and testing are needed to ensure systems continue to operate as intended and produce reliable outcomes.

One challenge is that performance degradation may not be immediately apparent. Unlike a conventional technology failure, drift can occur gradually and without obvious warning signs. If left undetected, it can lead to flawed decisions, biases, increased costs, customer harm, reputational damage and, in some cases, regulatory consequences.

For organisations developing AI systems or models in-house, regulators increasingly expect appropriate governance arrangements, monitoring practices and review processes that can identify drift and maintain performance within acceptable parameters. This includes defining performance thresholds, implementing ongoing testing and establishing processes for remediation where performance deteriorates.

The issue is equally relevant where AI systems or models are supplied by third parties. Organisations remain accountable for managing risks arising from outsourced technologies and should ensure they have sufficient transparency and oversight of supplier practices. Contractual arrangements may need to provide appropriate audit, assurance or reporting mechanisms to enable effective oversight.

In practice, this may involve understanding how suppliers monitor for drift, the nature and currency of training data, the frequency of model retraining, and the controls used to maintain system performance over time.

The risk of drift also highlights the growing importance of developing internal AI governance capabilities. While many organisations have moved quickly to adopt AI tools, governance frameworks and internal expertise have not always developed at the same pace. As AI becomes more deeply embedded in business operations, organisations are increasingly needing to invest in expertise spanning data science, risk management, compliance and technology governance.

AI adoption is accelerating as organisations seek to realise the technology's potential benefits. As APRA's comments suggest, however, assurance practices are not always keeping pace with the increasing use of AI systems. In many cases, this reflects a broader challenge: organisations are still developing their understanding of how AI systems function, how risks emerge and how those risks should be managed throughout the technology lifecycle.

To maximise the benefits of AI while minimising risk exposure, organisations need a fundamental understanding of how AI systems operate and how their performance can change over time. Drift is only one of several AI-related risks, but it is an important reminder that effective AI governance requires ongoing oversight long after implementation.