On How AI will (not) transform risk management

The modern history of financial risk management is, in one sense, a story about not having enough people.

In the late 1950s, risk in banking still lived in paper files and human memory. Credit officers in New York or London judged borrowers by ledgers, visits, and the occasional whispered reference. When Fair, Isaac and Company began to promote statistically derived credit scores, they appealed not to romance but to arithmetic and scale: a score could be applied consistently, to hundreds of thousands of borrowers, by clerks who did not know them and would never meet them. It was, among other things, a solution to a staffing problem.

Over the following decades, the same pattern repeated. Each new apparatus of risk control emerged not only from intellectual conviction but from a quiet recognition that there were simply too many decisions, too many transactions, too many counterparties for human scrutiny alone.

From prudence to parameters

By the late 1980s, this logic had hardened into policy. The first Basel Accord, agreed in 1988, did more than set minimum capital ratios. It translated the old, somewhat mystical language of “prudence” into explicit risk weights and percentages. Bankers and supervisors could now point to a table rather than a hunch.

As markets deepened and balance sheets grew, the profession sought tools that would keep pace. Value-at-risk, arriving in the 1990s, promised to compress an entire trading book into a single number: how much might be lost, with a given probability, over a given horizon. The attraction was not merely mathematical elegance. A VaR report allowed a handful of risk officers to oversee the positions of hundreds of traders; it multiplied their reach.

Basel II, published in 2004, extended the programme. Large banks were invited to build internal models for credit and operational risk, estimating probabilities of default and loss-given-default using their own data. It was, in part, a concession to reality. There were not enough supervisors, in Basel or anywhere else, to review every loan and every control. There could, however, be standards for models, tests for their performance, and capital charges derived from their outputs.

The myth crystallising in this period was simple: if only the parameters were correct, the risks would be under control. The crisis of 2007–2009 revealed how fragile that assumption could be, but the basic predicament remained. There were still more credit files, more exotic structures, more streams of market data than any number of analysts could reasonably absorb.

Machine learning and the overworked risk officer

The first wave of machine learning in finance should be understood against that backdrop. Banks did not adopt decision trees, gradient boosting, and neural networks solely because they were incrementally more accurate. They adopted them because the workload had become unmanageable.

In credit, machine learning models could sift through thousands of variables—cash-flow patterns, repayment histories, sector indicators, macroeconomic signals—and surface a concise risk assessment. What had once required a team of analysts working for weeks could be rendered as a live, updating score, accompanied by a ranked list of factors most strongly associated with default.

In fraud detection, the resource imbalance was even starker. A large retail bank might process millions of transactions a day. Manual review teams could plausibly examine only a minute fraction of suspicious cases; rules-based filters, though helpful, generated oceans of false positives. Machine learning systems were introduced as triage nurses: prioritising alerts, downgrading obviously benign cases, and escalating those with unusual or fast-evolving patterns. Human investigators still made the final call, but the machines decided who reached their desks.

Here the reality quietly diverged from the popular image. Within the folklore of Silicon Valley, AI arrived as an almost mystical intelligence. Inside financial institutions, it was first and foremost a labour-saving device.

The risks no one has time to see

Nowhere is this clearer than in the domain of financial crime and know-your-customer (KYC) controls.

For two decades, legislators and standard-setters have required banks to identify their customers, verify beneficial ownership, understand the nature of business relationships, and monitor accounts for suspicious activity. The letter of these requirements is demanding; the volume of actual work they generate is enormous. Each new client file must be checked, each change in ownership recorded, each alert investigated. The result in many institutions has been a permanent backlog.

The most acute risks often lurk in these backlogs. A missing passport copy, a beneficial owner not fully verified, a high-risk jurisdiction not correctly flagged—these are not glamorous failures of high finance. They are small absences, scattered across thousands of records, that collectively create opportunities for money laundering, sanctions evasion, and reputational disaster. Particularly the issue of beneficial ownership of legal entities is a problem I've observed repeatedly at banks and other financial intermediaries in Switzerland; heck, until 2016, beneficial ownership was not formally established for operating companies! I estimate about 5% of legal entities are properly documented and verified when it comes to ownership. But the age of the centralized share register will come, even if slowly.

AI is particularly well suited to this neglected terrain.

Machine learning systems can review KYC records at scale, scanning for inconsistencies, missing documents, or patterns that historically correlate with later problems. A client whose stated activity does not match transactional behaviour, a chain of ownership with unusual depth, a cluster of customers sharing contact details or introducers: all can be surfaced automatically. The system does not replace the compliance officer; it arranges her in-tray, placing the most worrying files on top.

Similarly, where controls remain manual—four-eyes checks on payments, approvals for limit increases, ad hoc spreadsheet reconciliations—AI can act as an additional, automated control. It can cross-check that approvals align with policy, highlight cases where manual reviewers consistently override certain rules, or flag outliers in behavioural patterns that human supervisors might overlook simply because they have too many screens open and too little time.

In this guise, AI is best understood as a force multiplier for scarce human attention. It excells not only at detecting novel forms of risk, but at illuminating familiar ones that organisations know about and simply cannot reach.

Algorithms under stress

The same period that saw AI embedded in credit and compliance also produced some instructive shocks.

The “Flash Crash” of May 2010, in which major U.S. equity indices plunged and recovered within minutes, was often described as a seizure of the market by its own algorithms. In popular retellings, the machines momentarily escaped human control. The more sober reconstruction painted a different picture: a human decision—an unusually large sell order—interacting with a microstructure already saturated with automated strategies. The event did not reveal a malevolent artificial intelligence; it revealed a finely tuned, tightly coupled system whose internal feedback loops could quickly become unstable.

For risk managers, the lesson was unnerving. Complex automated agents could, in aggregate, create dynamics that no single risk model had anticipated. Capacity constraints reappeared at a higher level. No organisation, however well staffed, could trace every interaction among trading algorithms in real time.

To hallucinate or not hallucinate

The modern term “hallucination” suggests something dramatic: models spinning elaborate fictions out of thin air; in fairness, the early GPT era did in fact suggest a significant underlying issue in how LLMs reason. In practice, serious institutions go to considerable lengths to ensure that their AI systems behave more like meticulous clerks than overconfident fabulists. The most reliable configurations of large language models in risk and compliance are not left to improvise, but are tightly moored to precise context, curated data, and explicit rules about what they may and may not say.

The first discipline is grounding. When an AI assistant is asked to summarise a credit file, review KYC documentation, or explain a policy, its remit can be confined to the relevant documents and records. Retrieval systems supply the model with specific contracts, transaction histories, and regulatory excerpts; prompts instruct it to answer only from those sources and to acknowledge gaps rather than speculate. In this mode, the model acts less as a storyteller and more as a controlled interface over a well-defined corpus. The narrower and clearer the context, the less room there is for invention. Large context performance has also significantly improved among all LLMs (although people are right to question e.g. how accurate a 1 million token window can really be at the moment; Google has also abandoned the 2 million context window for now).

The second discipline is model choice and architecture. Institutions that take risk seriously do not deploy experimental systems on critical workflows; they use the best-validated models they can obtain, often with fine-tuning on domain material and extensive pre-production testing. Around the model sits a lattice of guardrails: output filters to block prohibited content, schema constraints that force answers into predictable formats, confidence thresholds that trigger escalation, and tooling that checks factual claims against reference data where possible. The result is not infallibility, but a marked reduction in the free-form, unanchored responses that earned “hallucination” its ominous reputation.

A third line of defence is workflow design. In high-stakes settings, AI rarely has the last word. Drafts of KYC reviews can be generated by a model, but sign-off remains with a human analyst. Anomalous transaction clusters may be identified algorithmically, yet they still pass through experienced investigators who understand local business practices and historical cases. Properly arranged, these workflows exploit the machine’s appetite for detail and the human’s capacity for context and doubt. They also create opportunities to detect and correct model errors before they crystallise into operational failures.

This does not mean that concerns about hallucination are misplaced; they are, however, easily misdirected. Public demonstrations of language models fabricating citations or inventing biographies are alarming, but they rarely resemble the constrained, audited configurations used in regulated financial environments. Moreover, the comparison that matters is not between an idealised, error-free human and a flawed machine, but between fallible humans working unaided and fallible humans equipped with tools that make certain mistakes less likely and more visible.

Historically, human error has been a dominant source of operational losses in finance: misread limits, miskeyed trades, overlooked red flags in customer files, fatigue-induced approvals granted without full review. AI will introduce its own failure modes, and it would be complacent to assume that these will always be smaller or more manageable. Yet it is equally misleading to treat hallucination as an exotic new peril that dwarfs all others. In many risk and compliance functions, unassisted human oversight already tolerates a level of inconsistency and omission that a well-governed AI system, operating under constraint, can help to reduce rather than exacerbate.

The prudential stance, then, is neither alarm nor enchantment. It is to treat hallucination not as a monstrous anomaly but as one more category of model risk to be identified, bounded, and monitored—alongside bias, drift, and misuse. With clear context, strong models, and robust guardrails, the spectre of hallucination becomes less a reason to shun AI in risk management and more a reminder that judgment, accountability, and design still sit firmly with human institutions, however sophisticated their machines may become.

Generative AI and the two-sided sword

The arrival of large language models and generative systems in the early 2020s added a new layer to the story.

On the defensive side, banks began to deploy these tools as universal interpreters. They summarised policy documents and regulatory circulars, assisted staff in drafting reports, and digested the sprawling output of traditional risk systems into more coherent narratives. In compliance, language models reviewed transaction notes, emails, and chat logs to help identify potential misconduct or gaps in existing controls. In KYC, they assisted with extracting and reconciling information from corporate registries, shareholder lists, and identification documents that once demanded painstaking manual entry.

On the offensive side, criminals seized the same capabilities. Deepfaked voices and images undermined biometric authentication. Fraudulent investment platforms appeared with convincingly generated content. Phishing messages became less stilted, more tailored, and more numerous. In several widely reported cases, synthetic video and audio were used to impersonate corporate officers in order to authorise transfers that would once have been stopped by simple incongruity.

Again, the imbalance of resources was central. Fraudsters no longer needed a team of skilled forgers; a modest collection of tools sufficed. Banks responded by stepping up their own AI defences, but the contest now resembled an arms race, fought at machine speed.

How AI is changing the craft

Set against this history, several features of AI’s transformation of risk management stand out.

First, it expands coverage. Traditional controls are necessarily selective: sample-based audits, manual file reviews, periodic KYC refreshes. AI systems can, at least in principle, examine entire populations—every transaction, every customer file, every payment instruction—searching for anomalies. This does not guarantee perfection, but it materially alters the ratio between risks known and risks simply unseen.

Second, it improves triage. In domains overwhelmed by alerts—fraud monitoring, sanctions screening, suspicious activity reports—the key constraint is not the existence of controls but the capacity to act on them. AI models can rank and filter these alerts with a nuance unavailable to simpler rules, lifting precision and freeing scarce investigators to concentrate on the most consequential cases.

Third, it enables layered control. Rather than replacing existing manual or rules-based processes, AI can sit alongside them, providing an independent view. A payment may pass standard rule checks yet still be flagged by an anomaly detector; a KYC file may satisfy the checklist yet appear, in a network analysis, uncomfortably close to known high-risk entities. Disagreements between these layers are often the most informative points of inquiry.

Fourth, it sharpens forward-looking analysis. By tracing complex dependencies, AI-assisted models contribute to more detailed scenario analysis and stress testing. They can, at least under stable conditions, illuminate how particular shocks might propagate through portfolios, funding models, and counterparties, helping institutions identify vulnerabilities that traditional ratio-based metrics would miss.

None of this dissolves the need for human judgment. It's also important to remember that humans provide an assurance layer, or legal accountability. That need won't go away. I've sometimes cynically called law firms assurance providers, rather than legal consultants. It does, however, rearrange where that judgment is applied—from line-by-line inspection of records that machines can parse, to the design, oversight, and challenge of the systems that now perform that inspection at scale.

The boundaries of the map

The limitations of AI fall into equally concrete categories.

Opacity remains a central concern. Many of the most effective models are not meaningfully interpretable to those who must sign off on their use. “Explainability” tools can help, but they often transform one kind of abstraction into another rather than restoring the intuitive transparency of a traditional scorecard. This matters acutely in areas such as credit decisions and KYC, where institutions must justify their actions to customers and regulators alike.

Bias is another structural issue. When the past is unjust, models trained on historical data will reproduce that injustice unless actively constrained. In credit and customer due diligence, where correlations with geography, occupation, or social networks are strong, the line between legitimate risk differentiation and unfair discrimination can be thin.

There is also the problem of procyclicality and herding. As more institutions adopt similar models and datasets, their actions risk becoming synchronised. If many banks use comparable AI tools to rate borrowers, detect fraud, or assess country risk, their responses to model signals may converge—tightening credit to the same sectors at the same time, or exiting particular markets en masse. The collective result can be to amplify rather than dampen stress.

Data fragility imposes further constraints. AI’s strength lies in patterns; crises are, almost by definition, breaks in pattern. Models trained on long periods of relative calm may perform poorly when confronted with abrupt regime changes—pandemics, geopolitical shocks, or novel monetary arrangements. In such moments, the value of old-fashioned scenario imagination, and of seasoned scepticism toward tidy outputs, becomes evident.

Finally, there is the adversarial dimension. Unlike earthquakes or storms, human opponents adapt. Once fraudsters understand which behaviours are being flagged, they adjust their tactics. Once they learn how a KYC system scores risk, they structure ownership and cash flows to skate just within acceptable boundaries. AI helps defenders keep up, but it also supplies attackers with new ways to probe and evade.

Entangled futures

The arc from early credit scoring to today’s generative systems suggests a pattern. Each wave of innovation in risk management has arisen when the volume and complexity of financial activity exceeded the capacity of human scrutiny. Each wave has promised not only better insight but relief from chronic resource constraints. Each has, in turn, introduced new forms of dependence and new varieties of risk.

Artificial intelligence continues that trajectory. Its most enduring contribution may not be glamorous breakthroughs in prediction, but the more prosaic ability to illuminate the sprawling hinterland of “known unknowns”: the KYC files no one has had time to revisit, the manual controls that operate faithfully but invisibly, the alerts that accumulate faster than they can be read. In those territories, its capacity to extend human attention is genuinely transformative.

Yet the fundamental structure of the craft remains. Risk management in finance has never been a project of elimination; it has always been a project of judgment under constraint. AI alters the constraints. It redraws the map. But it does not abolish the need to decide—consciously, and with eyes open—where to focus, what to believe, and who should bear the consequences when even the most sophisticated systems fail to see what lies just beyond their carefully curated data.