High risk AI myth-busting

The European Commission's new draft guidelines make high-risk classification easier - and dispel some common myths.

The more I hold trainings for AI developers, the more myths I hear about risk classification. A lot of developers are eager to avoid their products being "high-risk", which is understandable.

But the European Commission is working to clarify, in plain language, how exactly an AI system gets classes as high-risk. Their new draft guidelines are long - 154 pages - but really helpful, and they clear up some commons myths among providers:

Myth 1: “My product requires human-in-the-loop, so it’s not high-risk”

I’m pretty sure this myth comes from using ChatGPT, Copilot, and Claude; you’ll inevitably get some sentence about how HITL is a part of responsible AI and risk reduction. 

First, "HITL" is not a term ever used in the Act.

Second, lacking human-in-the-loop is not what makes an AI system high-risk - so adding human-in-the-loop doesn’t make an AI system not-high-risk. High-risk is determined by whether a system poses a high risk to fundamental rights. And if your system is high-risk, you must have specific huaman oversight meaures.

The draft guidelines say:

“The provider cannot exempt and categorise an AI system as ‘low risk’ simply by adding to it a requirement for human involvement”

(item 71, Annex III guidelines)

Myth 2: “Our product is really flexible - if users decide to use it in a high-risk manner, that’s on them.” 

I get this logic, but the logic of the Act hinges on how the provider defines intended use, and on reasonably foreseeable misuse. Responsibility can’t be handed over to the user, if the technical capability exists. 

The opposite is almost true: the more flexible an AI system is marketed as or described, the more likely it could be to be high-risk.

The draft guidelines say:

“If the instructions for use… present the AI system as broadly applicable across a generality of contexts and functions, and do not consistently limit its application or exclude high-risk uses, the system’s  intended purpose will be deemed to also encompass high-risk use cases and therefore qualify as high-risk… Merely asserting (for example in the terms of service) that high-risk uses are excluded is insufficient to avoid the system from being considered high-risk.”

(Item 12, general principles guidelines) 

We work with great providers of complex, high-risk AI systems - also explicitly addressed in the draft guidelines. If you want to second opinion on your risk class, get in touch!