Ai’s Advances May Improve The World, But Without The Right Protections, The Tech Could Threaten Children Everywhere, Experts Warn

• Speaking at the Fortune Brainstorm AI conference Tuesday, experts warned that without awareness and the right protections, AI could be dangerous to children. The increased use of the technology to create AI-generated sexual abuse material, is especially worrying. Yet the EU has been at the forefront of tackling this threat through multiple laws.

AI is already in the process of improving lives, but it is also making the lives of children harder, and protections are needed, said experts at the Fortune Brainstorm AI conference Tuesday. 

Technology has for years affected young people, with some studies claiming social media, in particular, may be to blame for mental health problems and increased isolation among youth. Yet AI poses a unique and singular threat to children, according to experts.

Baroness Joanna Shields, OBE, is trying to help protect children against sexual abuse facilitated by AI. Shields, founder of the WeProtect Global Alliance, has helped bring together more than 100 governments to share best practices for combatting online harms and crimes through her organization.

Shields executive produced a film, Protect Us, which strikes right at the heart of the dangers of AI when it comes to young people. The film, a portion of which was shown during the session, highlights the use of “nudify” apps to create sexually explicit material featuring minors, with three of every 10 kids either having experienced this or knowing someone who has experienced it, Shields said.

“The impact is devastating, and then they're using these nudify apps in 'sextortion' cases, where they're actually enticing young people to upload an image of themselves, a sexually explicit image, thinking they're talking to somebody that likes them and that they've developed a relationship with, and then they use that for extortion,” Shields said during the session.

Shields confirmed the Protect Us film will be included in the curricula of schools in different countries to bring awareness to the issue.

As the risks to children increase with AI, Emilio Puccio, the Secretary General of the European Parliament Intergroup on Children’s Rights, said the EU government is stepping up to protect them.

The EU is approaching the issue from criminal standpoint, with new legislation criminalizing AI-generated sexual abuse material. At the same time, other legislation is trying to look at the problem through a lens of prevention, including through the trailblazing AI Act passed in May 2024, but also through the Digital Services Act and the Digital Markets Act, Puccio said.

“We have understood that criminalizing only happens after the harm is already done. So we need to actually tackle the problem upstream and tackle the infrastructure that's made this possible,” Puccio said.

Other highlights:

• Baroness Joanna Shields, OBE, Founder, WeProtect Global Alliance: “We can't productize humanity. We can't productize a person and create something that is a substitute for a real human.”
• Emilio Puccio, Secretary General of the European Parliament Intergroup on Children’s Rights: “Somehow we're meant to believe that there's a sort of a difference between the online world and in the real world. But in fact, there is none.”

This story was originally featured on Fortune.com