Model Drift
What model drift is
An AI model is trained on data from a particular period and domain. When it is deployed in production, it encounters real-world data that may differ from its training data in ways that gradually accumulate. The language used in a professional domain evolves. Regulatory frameworks change. Client behaviour shifts. Market conditions move. The model, trained on data that reflected a previous reality, begins to produce outputs that are less accurate, less relevant or less consistent with current expectations.
This is model drift. It is not a failure or a bug. It is an inherent characteristic of AI systems deployed in dynamic environments. A model that was highly accurate at deployment will, without active monitoring and maintenance, become less accurate over time as the gap between its training data and the current environment widens.
Drift can be gradual and difficult to detect without systematic monitoring. Individual outputs may remain plausible while the overall accuracy of the system decreases. In a professional services context, where the outputs of AI systems influence decisions with real consequences for clients, undetected drift represents a significant governance risk.
Types of drift
Data drift occurs when the statistical properties of the inputs the model receives in production change relative to the inputs it was trained on. A contract review model trained predominantly on English law contracts will experience data drift if it begins receiving increasing volumes of contracts governed by other legal systems, even if those contracts look superficially similar.
Concept drift occurs when the relationship between inputs and the correct outputs changes over time, even if the inputs themselves remain similar. A risk assessment model trained before a significant regulatory change will experience concept drift as the criteria for what constitutes risk shift, even if the documents it is reviewing remain formally similar to those it was trained on.
Managing drift
Managing model drift requires active monitoring of system performance in production, comparison of current outputs against a ground truth, and a defined process for retraining or fine-tuning the model when performance degrades below an acceptable threshold. This is the operational infrastructure that MLOps provides. Without it, drift goes undetected until it manifests as a visible failure, by which point it may have influenced a significant volume of decisions.
Insight
For law firms and financial services businesses, model drift has direct governance implications. A system whose accuracy is decreasing undetected is a system whose outputs are being relied upon to a degree that is no longer warranted.
In a regulated environment, the ability to demonstrate that AI systems are monitored for performance degradation and that there is a defined process for addressing it when it occurs is increasingly expected as part of a credible AI governance framework.
Designing the monitoring and remediation process for drift is a requirement of the Orchestration stage, not an operational detail to be addressed after deployment.
Misconceptions
Model drift is not caused by the model changing. The model itself is fixed after training. Drift occurs because the environment the model operates in changes, creating a gap between the model's learned understanding and current reality.
Drift does not happen suddenly. It accumulates gradually, which makes it easy to miss without systematic monitoring. Waiting for visible output failures before investigating drift means accepting a period of degraded performance that could have been caught earlier.
More capable foundation models do not eliminate drift. A more capable model may be more robust to certain kinds of drift, but all models deployed in dynamic environments are subject to it. Monitoring remains necessary regardless of the model used.