The political deal clinched by European Union lawmakers late Friday over what the bloc is billing as world’s first comprehensive law for regulating artificial intelligence includes powers for the Commission to adapt the pan-EU AI rulebook to keep pace with developments in the cutting edge field, it has confirmed.
Lawmakers’ choice of term for regulating the most powerful models behind the current boom in generative AI tools — which the EU Act refers to as “general purpose” AI models and systems, rather than using industry terms of choice, like “foundational” or “frontier” models — was also selected with an eye on futureproofing the incoming law, per the Commission, with co-legislators favoring a generic term to avoid a classification that could be chained to use of a specific technology (i.e. transformer based machine learning).
“In the future, we may have different technical approaches. And so we were looking for a more generic term,” a Commission official suggested today. “Foundation models, of course, are part of the general purpose AI models. These are models that can be used for a very large variety of tasks, they can also be integrated in systems. To give you a concrete example, the general purpose AI model would be GPT-4 and the general purpose AI system would be ChatGPT — where GPT-4 is integrated in ChatGPT.”
As we reported earlier, the deal agreed by the bloc’s co-legislators includes a low risk tier and a high risk tier for regulating so-called general purpose AIs (GPAIs) — such as models behind the viral boom in generative AI tools like OpenAI’s ChatGPT. The trigger for high risk rules to apply on generative AI technologies is determined by an initial threshold set out in the law.
Also as we reported Thursday, the agreed draft of the EU AI Act references the amount of compute used to train the models, aka floating point operations (or FLOPs) — setting the bar for a GPAI to be considered to have “high impact capabilities” at 10^25 FLOPs.
But during a technical briefing with journalists today to review the political deal the Commission confirmed this is just an “initial threshold”, affirming it will have powers to update the threshold over time via implementing/delegating acts (i.e. secondary legislation). It also said the idea is for the FLOPs threshold to be combined, over time, with “other benchmarks” that will be developed by a new expert oversight body to be set up within the Commission, called the AI Office.
Why was 25 FLOPs selected as the high risk threshold for GPAIs? The Commission suggests the figure was picked with the intention of capturing current gen frontier models. However it claimed lawmakers did not discuss nor even considered whether it would apply to any models currently in play, such as OpenAI’s GPT-4 or Google’s Gemini, during the marathon trilogues to agree the final shape of the rulebook.
A Commission official added that it will, in any case, be up to makers of GPAIs to self assess whether their models meet the FLOPs threshold and, therefore, whether they fall under the rules for GPAIs “with systemic risk” or not.
“There are no official sources that will say ChatGPT or Gemini or Chinese models are at this level of FLOPs,” the official said during the press briefing. “On the basis of the information we have and with this 10^25 that we have chosen we have chosen a number that could really capture, a little bit, the frontier models that we have. Whether this is capturing GPT-4 or Gemini or others we are not here now to assert — because also, in our framework, it is the companies that would have to come and self assess what the amount of FLOPs or the computing capacity they have used. But, of course, if you read the scientific literature, many will point to these numbers as being very much the most advanced models at the moment. We will see what the companies will assess because they’re the best placed to make this assessment.”
“The rules have not been written keeping in mind certain companies,” they added. “They’ve really been written with the idea of defining the threshold — which, by the way, may change because we have the possibility to be empowered to change this threshold on the basis of technological evolution. It could go up, it could go down and we could also develop other benchmarks that in the future will be the more appropriate to benchmark the different moments.”
GPAIs that fall in the AI Act’s high risk tier will face ex ante-style regulatory requirements to assess and mitigate systemic risks — meaning they must proactively test model outputs to shrink risks of actual (or “reasonably foreseeable”) negative effects on public health, safety, public security, fundamental rights, or for society as a whole.
While “low tier” GPAIs will only face lighter transparency requirements, including obligations to apply watermarking to generative AI outputs.
The watermarking requirement for GPAIs falls in an article that was in the original Commission version of the risk-based framework, presented all the way back in April 2021, which focused on transparency requirements for technologies such as AI chatbots and deepfakes — but which will now also apply generally to general purpose AI systems.
“There is an obligation to try to watermark [generative AI-produced] text on the basis of the latest state of the art technology that is available,” the Commission official said, fleshing out details of the agreed watermarking obligations. “At the moment, technologies are much better at watermarking videos and audio than watermarking text. But what we ask is the fact that this watermarking takes place on the basis of state of the art technology — and then we expect, of course, that over time the technology will mature and will be as [good] as possible.”
GPAI model makers must also commit to respecting EU copyright rules, including complying with an existing machine readable opt-out from text and data mining contained in the EU Copyright Directive — and a carve-out of the Act’s transparency requirements for open source GPAIs does not extend to cutting them loose from the copyright obligations, with the Commission confirming the Copyright Directive will still apply on open source GPAIs.
As regards the AI Office, which will play a key role in setting risk classification thresholds for GPAIs, the Commission confirmed there’s no budget nor headcount defined for the expert body as yet. (Although, in the small hours of Saturday morning the bloc’s internal market commissioner, Thierry Breton, suggested the EU is set to welcome “a lot” of new colleagues as it tools up this general purpose AI oversight body.)
Asked about resourcing for the AI Office, a Commission official said it will be decided in the future by the EU’s executive taking “an appropriate and official decision”. “The idea is that we can create a dedicated budget line for the Office and that we will be able also to recruit the national experts from Member States if we wish to on top of contractual agents and on top of permanent staff. And some of these staff will also be deployed within the European Commission,” they added.
The AI Office will work in conjunction with a new scientific advisory panel the law will also establish to aid the body to better understand the capabilities of advanced AI models for the purpose of regulating systemic risk. “We have identified an important role for a scientific panel to be set up where the scientific panel can effectively help the Artificial Intelligence Office in understanding whether there are new risks that have not been yet identified,” the official noted. “And, for example, also flag some alerts about the models that are not captured by the FLOP threshold that for certain reasons could actually give rise to important risks that governments should should look at.”
While the EU’s executive seems keen to ensure key details of the incoming law are put out there in spite of there being no final text yet — because work to consolidate what was agreed by co-legislators during the marathon 38 hour talks that ended on Friday night is the next task facing the bloc over the coming weeks — there could still be some devils lurking in that detail. So it will be worth scrutinizing the text that emerges, likely in January or February.
Additionally, while the full regulation won’t be up and running for a few years the EU will be pushing for GPAIs to abide by codes of practice in the meanwhile — so AI giants will be under pressure to stick as close to the hard regulations coming down the pipe as possible, via the bloc’s AI Pact.
The EU AI Act itself likely won’t be in full force until some time in 2026 — given the final text must, once compiled (and translated into Member States’ languages), be affirmed by final votes in the parliament and Council, after which there’s a short period before the text of the law is published in the EU’s Official Journal and another before it comes into force.
EU lawmakers have also agreed a phased approach to the Act’s compliance demands, with 24 months allowed before the high risk rules will apply for GPAIs.
The list of strictly prohibited use-cases of AI will apply sooner, just six months after the law enters into force — which could, potentially, mean bans on certain “unacceptable risk” uses of AI, such as social scoring or Clearview AI-style selfie scraping for facial recognition databases, will get up and running in the second half of 2024, assuming no last minute opposition to the regulation springs up within the Council or Parliament. (For the full list of banned AI uses, read our earlier post.)