AI in Germany

Having reviewed France, we now move onto Germany.

The subject of artificial intelligence (AI) or machine learning is a huge topic in Germany. Both the German Central Bank (Bundesbank) and the German Federal Financial Supervisory Authority (BaFin) have published several statements on these topics, some of which we touch on below. German banks are increasingly using machine learning for their processes in order to achieve gains in quality and efficiency, but with these efficiencies come numerous challenges including, to name but a few, the often-cited black box dilemma and appropriate data quality.

As regards the regulator's approach to machine learning, the Bundesbank has already issued a couple of informative policy documents including a Policy Discussion Paper in 2020 on the use of AI and machine learning in the financial sector and a joint consultation with BaFin on the concrete use of machine learning in risk models of Pillars I and II of the regulatory frameworks for banks and insurers. BaFin also published in 2021 a paper setting out general principles for the use of algorithms, with a focus on algorithms in the area of Big Data and AI and machine learning.

Importantly, BaFin's supervisory focus is not on the algorithm itself but rather how the supervised institution actually incorporates the algorithm into the decision-making processes. This is on the basis that an algorithm that is suitable for one particular context may yield unusable results in another situation or contravene the conditions with which an approval has been granted. As German supervisors, BaFin is doing everything it can to uphold the principle that humans must bear responsibility, even when innovative technologies are used in complex, integrated systems. There is also no legal basis in Germany for the general approval of algorithms or algorithm-based decision-making processes. However, there are certain special cases regulated by law in which the scope of an algorithm is defined and for which there are at least general provisions and minimum requirements in place for the procedures used. Even in such cases, though, BaFin does not grant its general approval but examines whether a procedure is suitable for its purpose.

Last year, BaFin issued an article regarding a study on the risks of using AI, machine learning, and algorithmic decision-making by banks for granting loans. "The automated decision-making techniques used in the lending process are currently typically based on proven machine learning techniques such as logistic regression." Whilst noting the advantages algorithm-based decision-making systems offer, the article also drew attention to the social risks "where algorithms [may] seize on, reinforce and even widen existing forms of discrimination." The German banking industry has been working to eliminate such discrimination, for example, some "explicitly no longer use critical characteristics such as residence and origin as parameters for risk classification procedures ... However, it is not possible to entirely eliminate correlations between financial parameters such as net income or assets and personal characteristics such as gender, residence or origin. With regard to such correlations, however, it needs to be remembered that considering the economic circumstances of borrowers is required by German supervisory law."

As regards the EU AI Act, Germany will approve this pioneering piece of legislation following the compromise that was reached at the beginning of the year.