ZEW Proposal for Evaluating Risky Generative AI // AI Act: ZEW Calls for External Safety Evaluations through Red Teaming

AI Act: ZEW Calls for External Safety Evaluations through Red Teaming

Risky Generative AI has to be evaluated.

The EU's recently adopted AI Act stipulates that general-purpose AI (GPAI) models with systemic risk will need to undergo particularly rigorous testing. This includes popular generative AI models such as OpenAI's GPT4. Researchers at ZEW Mannheim are now proposing guidelines for the systematic evaluation of such models. The proposal stems from a research project funded by the Baden-Württemberg Stiftung.

"The evaluation of GPAI with systemic risks requires well-defined goals, clear roles, as well as proper incentive and coordination schemes for all parties involved. Only then can we expect reliable evaluation results - and these should be reported in a standardised manner. To avoid conflicts of interest, the evaluation should be conducted by independent third parties. This could lead to the emergence of a specialised market for independent AI adversarial testing," summarises Dr. Dominik Rehse, co-author of the proposal and head of the ZEW Research Group "Digital Market Design".