Latest

Roblox details how it reports harmful activity to law enforcement

The company says its safety systems, age checks and monitoring tools drive higher reporting volumes and a more proactive approach.

Roblox details how it reports harmful activity to law enforcement
Image: Roblox

Roblox said on Thursday that it has refined how it reports suspected child sexual exploitation and other serious harms to law enforcement, describing a system built around aggressive detection, age checks and real-time text monitoring. In a post by Chief Safety Officer Matt Kaufman, the company said its reporting volume has risen as its safety systems have become more sophisticated, but that the increase does not necessarily mean more abuse is taking place.

Roblox said it works with law enforcement agencies around the world to address harms on the platform. A central part of that effort is the National Center for Missing & Exploited Children, where Roblox reports suspected instances of child sexual exploitation, as required of online service providers under U.S. federal law. The company said NCMEC acts as a global clearinghouse for those reports and routes them to domestic and international law enforcement agencies. Roblox said it recently visited NCMEC for a collaborative conversation about improving the quality and actionability of its reports.

The company also said it partners with the International Centre for Missing & Exploited Children, which provides resources, tools and training for educators and law enforcement globally. In a statement included in the post, Pilar Ramírez, vice president of national capacity building at ICMEC, said: “ICMEC values its partnership with Roblox.” She said the collaboration has supported capacity-building initiatives with law enforcement and other stakeholders, including specialized training, resources and tools to strengthen knowledge of online child safety issues.

How Roblox says its reports reach law enforcement

Roblox said it does not share the details of each report publicly, but includes aggregate numbers in its transparency reports and, for child sexual exploitation, in NCMEC’s annual report. The company also warned that direct comparisons between platforms can be misleading because reporting reflects platform design, reporting methodology and scale, including daily active users.

One example Roblox gave was the reports it sends to NCMEC about incidents that fall into what it called an “informational” category. The company said these are usually cases between minors, such as young players sharing or repeating explicit language, rather than interactions involving predatory adults. Roblox said about 40% of its reports to NCMEC fall into that category. It said those cases are handled with warnings, heightened monitoring, account restrictions and, where warranted, account closures.

Roblox said its approach is intentionally proactive because of its younger audience. The company said it looks aggressively for bad behavior and reports it through the proper channels, and that some of the guardrails it has built are uncommon for gaming platforms and can be unpopular. Kaufman wrote that the company wants to err on the side of caution.

Chat limits, age checks and non-encrypted monitoring

Roblox said players cannot share photos or videos in chat, which prevents chat from being used to share child sexual abuse material. It also said the ability to see, share or display external hyperlinks is restricted to specific pages and older age groups, a policy designed to keep young users from being directed off Roblox into less monitored environments where they could be asked for personal information or asked to share inappropriate content.

The company said it does not encrypt chat. That choice, Roblox said, allows its safety systems and specialists to monitor text interactions, spot grooming behavior earlier and intervene in real time. Roblox tied that decision directly to its reporting numbers, saying its ability to monitor all communication in real time leads to comparatively high law enforcement reporting and reflects its commitment to safety.

Age verification is another part of that structure. Roblox said it was the first major gaming platform to require age checks to access chat features, limiting chat to similar age bands and giving parents control over who their child can chat with. The company said it automatically assigns players to account types that align access, content, communication settings and parental controls with their age.

Machine learning, moderator training and cross-platform signals

Roblox said its machine learning models are designed to identify off-platform pivot attempts and predatory patterns earlier. Because communication on Roblox is text- or voice-based, the company said it has built systems to detect very early conversational signs of grooming, enticement and other subtle ways bad actors try to push personal boundaries. Those systems are trained to flag conversations for closer review and monitoring.

Once a conversation crosses into one of the areas Roblox is required to report to NCMEC, the company said it can act quickly. It said that approach has raised its proactive detection rate, and that the platform often catches bad behavior before users report it.

Roblox also said moderators receive specialized training led by an instructional designer and an expert with experience in the signs of child exploitation. Through simulated scenarios, moderators learn to recognize subtle, context-dependent coercion techniques. After training, they review user-submitted tickets with support until they are ready to work independently.

For violations it defines as “critical harms,” meaning those that threaten a user’s safety, Roblox said it investigates what happened and how it can be prevented in the future. It also said it trains systems and reviewers on emerging threat models, including misuse of generative AI, using material from Thorn, NCMEC and law enforcement. The company said language and slang change over time, and people who intend harm look for new ways to evade filters, so it continually retrains models on emerging language patterns, workarounds and real-world adversarial behavior.

Roblox said it is also an active contributor to the Tech Coalition’s Lantern project, which allows tech companies to securely share signals to help detect bad actors operating across different platforms. The company said the most serious online risks, including child endangerment, are not confined to one service, and that sharing information early and often can help law enforcement connect the dots across platforms and identify people who intend to cause harm.

Roblox links higher reporting to stricter safety design

Roblox said rising reporting volume does not necessarily mean there is more problematic behavior on the platform. The company said its detection systems have become more sophisticated and robust, and that they are built on the assumption that someone will try to bypass protections. It said everything it builds is meant to catch those attempts when they happen.

Kaufman said Roblox will continue to work around the clock to improve early detection and prevention. The company said it will keep refining safety systems, enhancing parental tools and working with parents and caregivers, while staying close to child safety experts and law enforcement so the platform remains a positive environment for its community.

Roblox closed the post by pointing readers to its safety information page for details on its architecture, policies and parental controls.

See all
See all

1 comment

  1. gamer_jr

    the no-encrypted chat bit stands out more than the report count to me. if they can actually spot grooming in real time, that’s one of the few times a privacy tradeoff makes sense on roblox

Add a comment