Recital 12

(12) It is necessary to define clearly the scope of the prohibition, including in particular the extent of providers’ and deployers’ obligations. This prohibition should not prevent providers from developing the technical capabilities of AI systems to generate or manipulate images, videos, audio or similar material. As concerns providers, the prohibition should be limited to the placing on the market or the putting into service of AI systems that generate or manipulate non-consensual intimate material or child sexual abuse material in two cases. First, it should cover systems intended to generate or manipulate such material. Second, it should cover systems where such generation or manipulation is a reasonably foreseeable and reproducible outcome and there are no reasonable and adequate technical safety measures and other safeguards in place, taking into account reasonably foreseeable misuse, to reliably prevent, and where necessary correct, that outcome and correct observed or reported misuse, including circumvention of such measures. Technical measures and other safeguards to prevent the generation of such material could include data cleaning, refusal training, safe prompt design and output controls, runtime prompt guardrails, content classification and filtering mechanisms, usage restrictions, abuse detection mechanisms, and notice and action mechanisms. Such preventive measures should be reasonable for the specific AI system, and are considered adequate if they align with the state-of-the-art measures and demonstrably prevent or sufficiently reduce in each specific case the likelihood of generating or manipulating such material, taking into account known and reasonably foreseeable misuse, including reasonably foreseeable circumvention of the preventive measures without significant technical modification. For providers retaining effective control over AI systems, for instance through a platform or a web interface, that could include following and reporting methods for misuse cases in full compliance with Union privacy and data protection law. In cases of observed or reported circumvention of the preventive measures or other safeguards, adequate corrective measures should be taken provided that such measures are reasonable, taking into account the specific AI system, including its release and distribution strategy (such as open-source releases). As concerns deployers, the use of an AI system should be prohibited only where the deployer uses an AI system for the purpose of generating or manipulating non-consensual intimate material or child sexual abuse material. This includes cases where a deployer uses or misuses AI systems placed on the market or put into service that lack reasonable and adequate preventive measures or where a deployer circumvents the preventive measures or uses for such purposes lawful AI systems not intended to generate or manipulate such material. The prohibition on use therefore does not cover the use of an AI system for lawful purposes, such as the generation or manipulation of material other than non-consensual intimate material or child sexual abuse material, even in cases where the AI system lacks reasonable and adequate safeguards that should have been put in place by the provider, nor does it cover accidental generation or manipulation of such content. Concerning the prohibition regarding non-consensual intimate material, where an AI system is intended for generation or manipulation of material falling under this prohibition, measures and other safeguards should include appropriate means for the distribution of the AI system aimed at enabling the reliable collection and demonstration of consent of the depicted person to such generation or manipulation, in compliance with Regulation (EU) 2016/679. The prohibition regarding non-consensual intimate material should be limited to realistic depictions of intimate parts, notably the genitals, pubic area, anus, exposed buttocks or exposed female breasts, nipples or areolae, or of sexually explicit activity. This ‘realism’ refers to the depiction of the person’s face, voice or their body in a credible real-life manner, regardless of the realism of the context of that depiction or whether it fully corresponds to the actual voice or appearance of the depicted person. Conversely, it excludes cartoonish or physically impossible depictions of a person’s body. The prohibition of non-consensual intimate material does not affect the generation or manipulation of other forms of nude material, such as material that does not depict identifiable natural persons, realistic partially nude depictions where intimate parts are not revealed and sexually explicit activities are not depicted, or non-realistic artistic nude works that do not realistically depict identifiable natural persons engaged in sexually explicit activity or depict their intimate parts. It also does not cover generative AI applications where intimate parts are not exposed or, if exposed, this is subject to the freely given, specific, informed, unambiguous and explicit consent of the depicted person (for example, try-on applications and medical applications, such as medical anatomical simulations and mammograms); this prohibition does not preclude the exceptional use of AI systems generating or manipulating nude depictions of the intimate parts of an identifiable person, in accordance with fundamental rights law, including data protection law, and applicable medical law, for the purpose of medical diagnosis and treatment by medical professionals where the person concerned is incapable of consent (for instance in an emergency situation). Finally, the prohibition on ‘manipulating’ non-consensual intimate material excludes cases where pre-existing intimate material is manipulated in a way that does not increase the exposure of any depicted intimate parts or alter the nature of any depicted sexually explicit activities, for instance the mere enhancement of an existing image depicting intimate parts or video depicting sexually explicit activities, such as changing the background, adding a text heading or enhancing the contrast or the brightness. Conversely, any manipulation of material, including material that already depicts intimate parts or sexually explicit activity, that increases the level of exposure of any depicted intimate parts or alters the nature of any depicted sexually explicit activities, falls under the scope of this prohibition.

← Recital 11 · All articles · Recital 13 →

Source: EUR-Lex (Cellar) · retrieved 2026-10-14 · Text as adopted (Official Journal); later amendments are not incorporated in this text.