English edition: AI Act and model risk — who is accountable for fraud systems?

Welcome to the Nordic Fraud Briefing. This is our English edition. In this special episode, we examine the AI Act, model risk, and the wider compliance framework around the systems banks use to fight fraud. Our audience is fraud and anti-money-laundering specialists, and our perspective is Swedish and Nordic banking.

Picture a perfectly legitimate payment being blocked. A risk model detects something unusual. An AI assistant gives the investigator a convincing explanation. Yet the customer still cannot get a clear answer. Later, the bank discovers that its supplier recently changed the model, and an important piece of input data was missing. Who should have caught that? And who has the authority to let the system keep running?

We will follow a completely fictional bank and a customer called Ellen. The distinction we need to keep in view is between the system's legal classification, the quality of the model, and the bank's responsibility for the complete decision process. Those questions are connected. They do not always produce the same answer.

Our reference date is the ninth of November, twenty twenty-six. The AI Act has been amended during the year. We use that updated framework, and distinguish rules already in force from future high-risk requirements and our own practical recommendations. Every person, system, and event in the bank example is invented. The voice you are hearing is synthetic, generated with Boson AI's Higgs Audio and the English reference voice Belinda. Sources and the transcript are on the episode page.

Part one: four regulatory perspectives on the same payment.

Our fictional bank uses machine learning to rank payments by fraud risk. A set of rules turns those scores into actions. Some payments proceed. Others trigger an extra check. A smaller number are stopped for investigation. A language model then summarises the case for the investigator. Already, we have several systems and several different uses of artificial intelligence.

The AI Act asks what kind of system this is, what it is intended to do, and what role the bank plays. The General Data Protection Regulation asks why personal data is being processed, whose rights are affected, and whether a decision is solely automated. DORA concerns digital operational resilience, including technology dependencies and third-party risk. The bank's wider governance must address model error, customer harm, and accountability across the process.

Take resilience as an example. A service can be available, recover quickly, and still produce unreliable fraud assessments. Equally, a lawful purpose for processing personal data does not establish that a model is accurate enough for a particular decision. Bring the controls together, but retain the distinction between the questions they answer.

A scoring model can work as designed and still be used for the wrong purpose. It can be classified correctly under the AI Act and still be poorly validated. A contract can contain all the right words while the bank has no workable fallback. That is why the same real-world process needs several assessments.

Start by mapping the chain from data to customer impact. What information comes in? Which model produces which signal? Which rule causes a payment to stop? What can the investigator see? What does the customer hear? Where can a person change the outcome? The inventory should follow the use of the system, rather than stop at the product name in the purchasing catalogue.

There is also a boundary between AI and other forms of automation. A simple instruction that always blocks payments above a fixed amount is not automatically an AI system. A machine-learning model may fall within the definition. Assess how the solution actually works. Marketing language about intelligence does not determine its legal classification, and a conventional rule can still have serious consequences for a customer.

In our example, Ellen cannot complete an important payment. The bank needs to explain both the model's contribution and the action that followed. If each team owns only its own technical component, responsibility for the whole process can disappear between the boxes. Our practical recommendation is to appoint a clear owner for the use case, with the authority to bring together payments, fraud, legal, privacy, and technology specialists.

That does not mean removing the independence of control functions. The business owns its risk. Risk management, compliance, and the data protection officer have distinct responsibilities. Internal audit must be able to assess the arrangements independently. The allocation of responsibility should remain understandable even when one supplier provides several parts of the chain.

Part two: the fraud exemption and the boundary of high risk.

Annex Three of the AI Act lists several uses that may be classified as high risk. One category covers systems intended to evaluate the creditworthiness of natural persons or establish their credit scores. Within that particular category, there is an explicit exception for systems used to detect financial fraud. That wording is highly relevant for fraud teams.

The exception belongs to that category. It does not say that everything a bank calls fraud falls outside the entire Act. The purpose, the functions, and any other potentially relevant categories still need to be assessed. Nor does an exception under the AI Act remove obligations under privacy law or the bank's wider governance framework.

Suppose the bank initially uses a model to detect account takeover. Later, the credit department wants to use the same score as part of an assessment of a customer's ability to repay. The use has changed. The bank cannot simply retain its old classification by pointing to the model's original name.

Profiling creates another common misunderstanding. Article Six contains a rule under which certain systems within Annex Three cannot benefit from an exception to high-risk classification when they profile natural persons. That does not mean every fraud model that profiles people automatically becomes high risk. First establish whether the system falls within the relevant Annex Three category.

Biometrics also needs careful distinctions. Verification whose sole purpose is to confirm a claimed identity has an exception within the category concerning remote biometric identification. Comparing a face with the identity a person claims is different from searching for that person across many identities. But the classification exception does not create a general permission to process biometric data under the privacy rules.

The payment model, an identity check, and a customer-facing assistant may therefore need separate analyses. A single procurement contract does not turn them into one legal use case. Prohibited AI practices also require a separate check. An anti-fraud label cannot make a practice lawful if it is otherwise prohibited.

Document the reasoning: intended purpose, affected people, decisions supported, and operational limits. Record what would trigger reassessment. Adding credit assessment, changing the biometric function, or allowing an assistant to take decisions could all be relevant changes. Classification should be a reasoned position that follows the business as it develops.

Part three: the updated timetable and the bank's roles.

Older presentation slides need checking here. The amending regulation entered into force on the twenty-seventh of July, twenty twenty-six. The high-risk provisions for the uses in Annex Three have been moved to the second of December, twenty twenty-seven. For certain high-risk systems forming part of regulated products, the date is the second of August, twenty twenty-eight.

This is not a general pause on all AI rules. Parts of the framework already apply. Article Four, on AI literacy, has been amended but remains in place. Providers and organisations deploying systems must take measures supporting the development of their staff's AI literacy, taking account of knowledge, experience, education, and the context of use. They do not have to guarantee a specified individual level of competence.

For a fraud analyst, a useful training question is how to distinguish a risk indicator from evidence. For a model owner, it also means understanding limitations, changes, and the circumstances in which operation should stop. A generic course certificate does not, by itself, demonstrate that these questions are addressed. Our recommendation is to connect training to actual responsibilities, local examples, and the decisions people are expected to make.

Article Fifty contains particular transparency obligations. They concern situations including direct interaction with AI, machine-readable marking, and certain synthetic content. A customer chatbot and an internal risk score are different situations. A visible statement that something uses AI does not automatically satisfy a separate obligation concerning machine-readable marking.

The special deadline of the second of December, twenty twenty-six concerns a limited marking requirement for providers of certain systems already on the market before the second of August, twenty twenty-six. It should not be described as postponing every transparency obligation. Older high-risk systems also have specific transitional provisions. Existing systems are not generally exempt for all time merely because they were introduced before a particular date.

The bank's role is a separate question from the timetable. It may develop a system and also deploy it. Under certain conditions, it may take on provider responsibilities by putting its own name on a high-risk system, making a substantial modification, or changing the intended purpose so that a previously different system becomes high risk. Ordinary configuration does not automatically create that change of role.

Swedish banks need to follow European rules together with Swedish supervisory arrangements. For operations in Norway and Iceland, incorporation into the European Economic Area framework and national steps also need checking. A Nordic group should not put one date against every legal entity without examining those differences.

Part four: model risk begins with what the model is allowed to do.

Return to Ellen's blocked payment. The risk score is high, but what does that number actually mean? Is it a calibrated probability of fraud, or simply a ranking? What time period does it refer to? Has the model been validated for this payment type? A score displayed to two decimal places can look more precise than its evidence supports.

Model risk arises when errors in a model, its assumptions, or its use can cause harm. Even a statistically effective model can be used outside its intended scope. A system trained on card fraud has not automatically been tested for socially engineered account transfers, or for decisions to end a customer relationship.

Our practical governance approach begins with a model inventory. Record the owner, purpose, data, version, users, influence on decisions, and known limitations. Link the model to the rules and workflows that consume its output. An inventory containing only a name and a purchase date offers little help during an incident.

Assess model risk in terms of consequences and dependence. A model that helps organise internal drafts may warrant different controls from one that blocks access to a customer's money. The same model can carry different risks in two workflows. This internal assessment of model risk does not have to match the AI Act's legal high-risk classification.

The way a model is acquired does not remove that need. An internally developed model, a ready-made supplier service, and a solution built using openly available model weights can all create customer risk. Open availability does not guarantee reliable training data, sufficient documentation, or support during an incident. Assess the solution actually being used, including its dependencies and the people responsible for operating it.

Before production, establish a clear basis for approval. What must the model demonstrate? Which weaknesses, if any, are accepted temporarily? Who may accept them, and when must that decision be revisited? Distinguish developer testing from an assessment by someone sufficiently independent to challenge suitability and limitations.

International model-risk frameworks can provide useful comparisons, but they do not automatically become Swedish law. The British prudential supervisor, for example, sets out principles covering inventories, governance, development, independent validation, and risk mitigation within its own scope. A Nordic bank needs to interpret such material in its own legal and operational setting.

For our bank, the important outcome is that someone can say: this model is approved for this purpose, with these limits and these safeguards. A supplier's claim of high accuracy is not a local acceptance of risk. The bank must examine what the result means for its own customers, data, and decisions.

Part five: data, ground truth, and testing that resembles real life.

Fraud models face a difficult ground-truth problem. Not every fraud is discovered, and information may arrive long after the payment. Meanwhile, the bank is more likely to investigate activity its existing system already flagged. The observed outcomes are therefore selective. A model may learn the bank's previous investigation choices as much as it learns criminal behaviour.

Research on credit-card fraud has long described class imbalance, changing patterns, and delayed feedback. These remain relevant methodological problems even when today's products are more advanced. Older studies do not establish how well a particular supplier's current system performs in a Nordic bank. They help identify the questions to ask.

A simple example shows why overall accuracy can mislead. If only a small proportion of payments are fraudulent, a system that approves everything can classify a very high percentage correctly. Yet it detects no fraud at all. The bank needs to measure both the fraud it finds and the legitimate customers it wrongly affects.

Useful measures include the proportion of alerts that represent actual fraud, the proportion of fraud detected, and the amounts protected. But connect them to investigation capacity, waiting times, customer friction, and remaining harm. More correct alerts can still produce a worse operational result if the queue becomes so long that action arrives too late.

Validate on periods that come after the training period. Check that future information has not slipped into historical tests. Something learned only after a customer complaint must not be treated as though the bank knew it when the payment began. Otherwise the test measures a capability that does not exist in production.

Examine performance across channels and relevant customer groups as well. A difference needs investigation and explanation. That does not give the bank unrestricted permission to collect sensitive data for every conceivable fairness test. Lawful grounds, special-category restrictions, and data minimisation must also be assessed for validation data.

At our fictional bank, an input sometimes goes missing after the supplier's update. Testing must therefore cover data quality and failure modes, rather than the model file alone. Try missing values, changed formats, and delayed feeds. Document whether the system then allows payments, blocks everything, or uses a fallback check. These are practical safety choices with direct consequences for customers.

Part six: human control, explanations, and model changes.

Ellen's case reaches an investigator. The screen displays a red risk indicator and a ready-made explanation. If the investigator lacks time, supporting information, and authority to change the decision, the presence of a person offers limited protection. An approval button does not, by itself, create meaningful review.

Specific human-oversight requirements exist for high-risk systems when the relevant provisions apply. For other fraud systems, similar controls may be justified by customer risk, bank governance, or data-protection requirements. Be clear about the actual basis for a requirement.

Investigators need to know what the signal means and what it does not demonstrate. Which information can be checked? What alternative explanations exist? When should they contact the customer or another specialist? Authority to depart from the recommendation must be combined with documentation and feedback. That allows the bank to correct individual errors while also identifying recurring weaknesses in the model or the wider process.

A model explanation is not automatically a causal explanation. A feature contributing strongly to a risk score does not prove fraud. A generative assistant producing a polished justification may invent a connection that the model never used. Customer explanations must therefore be grounded in the real case and the actual decision chain.

Model changes require the same practical approach. A new version, training dataset, threshold, or data source can affect results. Not every change is equally significant, and not every change needs the same checks. The bank needs a risk-based change process with clear criteria for renewed validation and approval.

Monitor data and outcomes in production. A change in the distribution of inputs can be a warning, but does not alone prove deteriorating performance. Equally, stable inputs can conceal changes in criminal behaviour. When outcomes arrive late, use supplementary indicators and be explicit about what is still unknown.

For our bank, we recommend a reconstruction exercise. Reproduce a particular payment using the model version, configuration, and information actually available at the time. If that cannot be done, both incident investigation and customer review become harder. Logging must remain proportionate and protected. Keeping everything forever is not a general answer to the evidence problem.

Part seven: privacy and the customer's right to challenge a decision.

Data-protection rules apply even where the fraud model is not high risk under the AI Act. The bank needs a clear purpose and a lawful ground for processing personal data. The importance of fighting fraud does not make every source necessary or every new use compatible with the original purpose.

Article Twenty-Two of the General Data Protection Regulation concerns decisions based solely on automated processing that produce legal effects or similarly significant effects for a person. Every internal risk score is not automatically covered. The bank needs to examine how the score affects the real decision and how significant its consequences are.

If the provision applies, citing legitimate interests under Article Six is not enough. The exceptions for these automated decisions require separate assessment. They include contractual necessity, specific legal authorisation with safeguards, or explicit consent. The applicable protections and rights must then be assessed in context.

The bank cannot avoid that analysis by adding a person who routinely accepts the model's result. Review must be real, supported by relevant information, competence, and an ability to influence the outcome. If Ellen's account is effectively restricted by the model while the investigator merely confirms the screen, the bank needs to understand what that process actually means.

A European Court judgment on automated credit assessment from February twenty twenty-five illustrates the importance of understandable explanations. It concerned a particular credit case, but provides guidance on meaningful information about the logic of an automated decision. A mathematical formula alone, or an impenetrable technical description, is not enough to allow a person to understand and challenge the outcome.

This does not mean that a bank generally has to publish source code or every fraud threshold. Confidentiality and protected information require a proper legal assessment. But trade secrets do not create a blanket right to dismiss information requests. Distinguish a useful explanation for a customer from instructions that would help someone evade the controls.

Also assess whether a data-protection impact assessment is required before processing likely to create high risk begins. Any fundamental-rights impact assessment under the AI Act is a different exercise. One form does not automatically replace the other. Involve the data protection officer early, and document how necessity, minimisation, correction, retention, and the customer's ability to seek review work in practice.

Part eight: DORA and the supplier that cannot take over the bank's responsibility.

DORA has applied since the seventeenth of January, twenty twenty-five. It is an existing framework for digital operational resilience in the financial sector. For our bank, dependence on a model platform, data services, and cloud infrastructure belongs within its overall technology-risk management. This is not a requirement to prepare for at some distant AI deadline.

The bank retains responsibility when it uses external technology services. A supplier's certification or supervisory status does not replace the bank's own risk assessment. DORA also extends beyond traditional outsourcing. Relevant contractual arrangements for technology services belong in the register of information, distinguishing services supporting critical or important functions from other services.

Do not confuse two different concepts. The bank's assessment that a function is critical or important is not the same as a supplier being designated critical at European level. Nor does using a model for fraud automatically make every individual component critical. Analyse the business dependency and the consequences of failure.

Contracts have baseline requirements, including the service description, responsibilities, locations, data protection, access to data, service levels, and incident assistance. Services supporting a critical or important function attract additional requirements, including monitoring, audit, contingency arrangements, and transition on exit. Do not describe every enhanced clause as an identical obligation for every small technology contract.

For a model supplier, clarify which changes could affect the bank's use. Who can switch the model? What happens if a data source disappears? What documentation and test environment are available? These are matters for a proportionate contract and a functioning change process. DORA does not mean every technical update always requires the bank's explicit prior approval.

An exit plan must be usable. Can the bank keep protecting payments if the service fails or the contract ends? Can data be retrieved in a usable format? Does the alternative depend on the same subcontractor? For critical or important functions, documented and sufficiently tested plans are needed, rather than a termination clause alone.

We recommend rehearsing a specific disruption: fraud scores become unavailable during a busy payment period. Which payments continue, which are reviewed, and how do customers get help? Also assess whether the event triggers reporting obligations. A technology outage, a personal-data breach, and a model problem require different assessments that may need to run in parallel.

Part nine: generative AI and agents in fraud operations.

Language models can help summarise cases, search regulations, and prepare drafts. A well-written answer is not proof that its contents are correct. In our fictional case, the assistant claims Ellen's payment was stopped because of a particular recipient region. That information does not appear in the actual decision evidence.

This creates a distinct model risk. An explanation can sound credible while misrepresenting both the model and the case. The assistant should not fill gaps with assumptions. Test whether statements can be linked to verifiable sources, and whether investigators can distinguish evidence, interpretation, and draft wording.

A shared test set can include routine cases, borderline cases, and cases with incomplete information. Test whether the assistant can avoid giving a definite answer when evidence is missing. Check whether it separates the customer's allegation from information the bank has verified. An unsupported statement written confidently can do more harm than an uncertainty that is clearly acknowledged.

If the assistant reads customer messages, documents, or web content, the bank must also guard against embedded instructions intended to redirect the system. Treat that content as information, rather than authorised commands. Test that the assistant neither discloses another customer's information nor takes actions the sender is not entitled to request.

There is a substantial difference between a drafting aid and an agent that acts. Once a system can restrict an account, change a case, or send a customer communication, clear permissions and limits are needed. Our recommendation is to begin with narrow tasks and assess every expansion of authority. Pay particular attention to actions affecting customers' money and rights.

For personal data, understand what the supplier receives, how long it retains it, and whether it uses it for other purposes. Processor agreements, subprocessors, and any international transfers need to be assessed against the real service. Availability in Europe does not, by itself, answer where or how the data is processed.

During the summer of twenty twenty-six, the European supervisory authorities emphasised governance and the management of cyber risks associated with advanced AI models. This was not a declaration that every new model is prohibited. It reinforces the need to assess security, dependencies, and resilience alongside potential benefits.

Measure more than drafting speed. Track factual errors, unsupported claims, missed information, information leakage, and the frequency of human corrections. Examine whether users become excessively dependent on suggestions. A shorter handling time is a limited gain if customer decisions become less accurate or harder to explain.

Part ten: a shared set of controls that people can actually use.

How does our bank put this together without creating five parallel folders? What follows is our practical proposal. Start with a shared description of the use case: purpose, responsible business area, system roles, data flows, customer impact, and clear boundaries on what the system may do.

Connect the legal assessments to that same description. Document AI classification, the basis for personal-data processing, any assessment of automated decisions, and DORA dependencies. Mark what applies now and what prepares the bank for future high-risk requirements. A change in use should reopen the relevant assessments.

Add the model's test evidence and approval. What demonstrates that it works in the intended environment? Which uncertainties remain? Who challenged the conclusions? Who accepted the residual risk? Make the distinction visible between developer results, independent review, and the business decision to use the model.

Information sharing between banks also needs a separate legal assessment. Permission to use data for internal fraud analysis does not automatically permit sharing it with a common model or another bank. Check purpose, confidentiality, recipients, and conditions for each flow. A technical ability to analyse data jointly does not create a legal entitlement to share it.

Then test the whole decision chain using Ellen's fictional payment. Can the bank reconstruct the inputs and version? Can the investigator understand and change the outcome? Does the customer receive an accurate explanation? Can incorrect information be corrected wherever it has spread? Can the business continue if the supplier disappears?

Management reporting should cover both effectiveness and harm. Track fraud detected and amounts protected alongside mistaken blocks, long waits, overturned decisions, and incidents. Report uncertainty in the outcome data as well. A green dashboard should not depend on treating everything the bank has not yet discovered as problem-free.

We leave our fictional bank with a clearer chain of responsibility. It has assessed the AI Act's fraud exception without treating it as a general release from accountability. The model's limitations are understood. Customer rights and supplier commitments are connected to real workflows. That is the organisational capability that makes the rules useful in practice, including when a model or an input fails in an unexpected way.

Take three questions into your next meeting. Have we classified the actual use? Have we tested the model in its real environment? And is responsibility clear all the way to the customer decision? This has been the Nordic Fraud Briefing's English special on the AI Act, model risk, and compliance in fraud systems, with a reference date of the ninth of November, twenty twenty-six. Thank you for listening.