OUT-LAW ANALYSIS 7 min. read

KIDS Act: safety-by-design considerations

A young Asian girl holding a locked smartphone with parental control notification on the screen

Online platforms could be required to build child safety into their services by default. fadfebrian/iStock


Providers of a range of digital services and AI systems will have to rethink the way their platforms are designed under new EU-wide child online safety legislation that has been proposed.

General ‘safety-by-design’ obligations are envisaged under Article 8 of the proposed new EU KIDS Act and are supplemented by a range of more prescriptive proposed rules that would influence the design of popular online and digital services, from social networks and video-sharing platforms, to app stores, online games, AI companions and chatbots.


Read more on the EU KIDS Act proposals


The general obligations would require providers of those services and systems to design their platforms so that they “ensure a high level of privacy, safety and security” of children under 18 years of age. Although the proposed regulation would not directly regulate content, the underlying assumption is that, by embedding safety by default, providers not only mitigate the risks associated specifically with design elements but also reduce the likelihood of minors being exposed to content, interactions or functionalities that may be harmful to them. The proposal would not affect existing rules which govern the availability of specific categories of content, such as the EU Audiovisual Media Services Directive and content rules under national law.

This duty would apply regardless of whether the in-scope services or systems are accessible with an account. In practice, this means that providers would generally have to assume that each of their unregistered or signed-out users are under-18 and design the service for those users accordingly. This is unless the provider can confirm the user is an adult via age assurance measures.

A whole chapter of the EU KIDS Act is devoted to ‘age assurance’, a term that broadly encompasses both age estimation as well as age verification but which expressly excludes “self-declaration by recipients of the service”. The chapter sets standards that age assurance solutions providers could deploy would need to meet, including around accuracy and privacy.

As such, the requirement to ensure users only access services or systems that are designed for children applies until the point that a provider can satisfy itself that a user is an adult, in line with the prescribed standards around age assurance.

For providers, this might mean running two or more different versions of the services or systems they operate, underpinned by a mechanism for ensuring non-child-friendly content, features and functionality is only accessible to signed-in users they can be confident are over the age of 18. This will inevitably reshape the product experience, particularly given that many online media experiences currently take place in a signed-out state. Requiring those experiences to be designed by default in a way that ensures privacy, safety and security suitable for children could therefore have significant implications for how all users discover, navigate and consume content practically.

These considerations are complicated by the fact that more age-specific access and account restrictions, envisaged under the EU KIDS Act proposal, would also need to be factored into design considerations. Those rules, if implemented, would govern how children under the age of 15 access digital services, and what they see and can do when access is enabled, as well as control when and how under-13s access video-sharing platforms “specifically designed” for users in that age group.

Chapter 3 sets out further design requirements for each category of in-scope service. The EU KIDS Act proposal differentiates between services based on their functionalities and restricts particular design features accordingly.

Social networking services and video-sharing platforms

Articles 9 to 13 apply to online social networking services and video-sharing platform services and include:

Addictive design (Article 9)

Providers must not design their services in a way that is intended, or could reasonably be foreseen, to encourage compulsive or excessive use by children. What constitutes 'compulsive’ or ‘excessive’ use is not defined, which presents challenges for providers. However, prohibited features include autoplay without effective interruption, attention-regaining notifications, incentivising content sharing or live transmission, and engagement streak mechanics. Time-limited access measures must also be deployed to protect school time and core sleep hours. The list leaves several practical questions unresolved, including where the line is drawn between an “attention regaining” notification and a routine notification. The recitals indicate that artificial timing of notifications and a change in platform behaviour when a user loses interest will be relevant.

Recommender systems (Article 10)

Providers that use recommender systems must design the information suggested and the optimisation of those systems to ensure a high level of privacy, safety and security of minors, including by not exploiting a minor’s vulnerability or attention. In particular, recommender systems must prioritise explicit user preferences (e.g. deliberate selection of topics of interest and quality-based signals) over implicit engagement signals such as browsing behaviour and time spent viewing content, which should be disabled by default. The line can be blurred in some cases, for example, where a user proactively engages with content about a particular subject.

Recommender systems must also not collect personal data of minors outside the service and must not expose minors to information that poses a risk to their privacy, safety or security, including if such risk occurs through repeated exposure.

Safe settings (Article 11, 14(1)b) and 15(1)(b))

Providers must ensure that certain settings are switched off by default, including geolocation and other tracking features, contact synchronisation and recommendations of contacts, and camera and microphone access. These settings must remain defaulted off until a minor turns 15. If the minor changes the settings before then, the settings should revert to off at the end of the session. Features and settings which increase social comparison or misrepresenting minor’s image (e.g. beauty filters) must not be available to minors.

Contact and interaction safeguards (Article 12)

Providers must put in guardrails around interactions between children and other users, including initiation of contact with the minor, contact recommendations, addition of minors to groups, and allowing minors to anonymously block other users. Other safeguards include preventing other users from enticing or manipulating minors into approving direct contact with them, and limiting their visibility of the child’s information.

Among other things, providers must prevent children from livestreaming and must ensure that other recipients of the service are unable to download or take screenshots of children's contact, location or account information, or any content they upload or share.

Safety and security of economic transactions (Article 13)

Providers must ensure that any economic transaction on their service is presented clearly and transparently. Where virtual currencies, such as community tokens or in-game currencies, can be used to make a purchase, the corresponding monetary value of the child’s local currency must also be displayed. Acknowledging that minors are still developing their commercial literacy, the Act requires that providers must not design their services in a way that encourages excessive, impulsive or unwanted spending by minors. Again, what qualifies as “excessive”, “impulsive” and especially “unwanted”, will be challenging to determine in practice.

AI companions and general conversational chatbots

Article 14 sets out specific design requirements applicable to AI companions and general conversational chatbots. These include:

Addictive design

Providers must protect minors from exposure to addictive design features that simulate interpersonal relationships which may create emotional dependencies with AI companions or general conversational chatbots by ensuring the design does not encourage compulsive or excessive use, setting time limits, and interrupting service use. The term 'emotional dependencies' is not defined in the proposal, which may give rise to ambiguity and difficulties in applying this in practice. Potentially any form of reassurance or affirmation given by an AI could give rise to some element of emotional dependency – and, arguably, an AI companion that doesn’t relate to the user is not really a companion at all.

Safe settings

The same safety settings set out above as applicable to social networking services and video-sharing platforms must be applied, including, importantly, a requirement that minors' previous interactions are not used to inform subsequent interactions unless doing so is necessary to protect the minor’s safety. At present, it is not clear what is meant by a minor's safety in this context, or what information may be collected for this purpose.

Economic transactions

The same requirements governing the safety and security of economic transactions set out above as applicable to social networking services and video-sharing platforms must be applied.

Testing and evaluation

Providers must perform state-of-the-art testing and evaluation of risks to the health, safety and fundamental rights of minors, including their physical, mental and emotional wellbeing.

Online games

Article 15 sets out specific design requirements applicable to online games. These include:

Addictive design

Providers must ensure that compulsive or excessive use of the game by minors is not encouraged, by applying the requirements of Article 9(1), (2)(b) and (d).

Safe settings

Providers must ensure that safe settings, as set out above, are applied by default.

Contact and interaction safeguards

Providers must ensure that contacts between minors and other players of the game are subject to a high level of privacy, security and safety, by reference to Article 12 (1) and (3).

Further, providers of online games must put in place safeguards to prevent the game from being used to entice minors to initiate contact on other services.

Software application stores

Finally, the proposed KIDS Act imposes specific requirements on software application stores to implement an age-rating system to establish the “age-appropriateness” of software disseminated through their service. This age-rating must take into account the evolving capacities of a child, and the methodology must be made publicly available in clear and accessible terms. Accordingly, providers of software application stores must not allow minors to access or purchase software that is inappropriate for their age. Any age assurance measures used in this context must be compliant with the age assurance requirements of the proposed Act.

Co-written by Michelle Seel of Pinsent Masons.

We are processing your request. \n Thank you for your patience. An error occurred. This could be due to inactivity on the page - please try again.