Media & Entertainment

Roblox responds to the hack that allowed a child’s avatar to be raped in its game

Comment

There’s a special place in Hell for people who think it’s funny to rape a 7-year-old girl’s avatar in an online virtual world designed for children. Yes, that happened. Roblox, a hugely popular online game for kids, was hacked by an individual who subverted the game’s protection systems in order to have customized animations appear. This allowed two male avatars to gang rape a young girl’s avatar on a playground in one of the Roblox games.

The company has now issued an apology to the victim and its community, and says it has determined how the hacker was able to infiltrate its system so it can prevent future incidents.

The mother of the child, whose avatar was the victim of the in-game sexual assault, was nearby when the incident took place. She says her child showed her what was happening on the screen and she took the device away, fortunately shielding her daughter from seeing most of the activity. The mother then captured screenshots of the event in order to warn others.

She described the incident in a public Facebook post that read, in part:

At first, I couldn’t believe what I was seeing. My sweet and innocent daughter’s avatar was being VIOLENTLY GANG-RAPED ON A PLAYGROUND by two males. A female observer approached them and proceeded to jump on her body at the end of the act. Then the 3 characters ran away, leaving my daughter’s avatar laying on her face in the middle of the playground.

Words cannot describe the shock, disgust, and guilt that I am feeling right now, but I’m trying to put those feelings aside so I can get this warning out to others as soon as possible. Thankfully, I was able to take screenshots of what I was witnessing so people will realize just how horrific this experience was. *screenshots in comments for those who can stomach it* Although I was immediately able to shield my daughter from seeing the entire interaction, I am shuddering to think of what kind of damage this image could have on her psyche, as well as any other child that could potentially be exposed to this.

Roblox has since issued a statement about the attack:

Roblox’s mission is to inspire imagination and it is our responsibility to provide a safe and civil platform for play. As safety is our top priority — we have robust systems in place to protect our platform and users. This includes automated technology to track and monitor all communication between our players as well as a large team of moderators who work around the clock to review all the content uploaded into a game and investigate any inappropriate activity. We provide parental controls to empower parents to create the most appropriate experience for their child, and we provide individual users with protective tools, such as the ability to block another player.

The incident involved one bad actor that was able to subvert our protective systems and exploit one instance of a game running on a single server. We have zero tolerance for this behavior and we took immediate action to identify how this individual created the offending action and put safeguards in place to prevent it from happening again. In addition, the offender was identified and permanently banned from the platform. Our work on safety is never-ending and we are committed to ensuring that one individual does not get in the way of the millions of children who come to Roblox to play, create, and imagine.

The timing of the incident is particularly notable for the kids’ gaming platform, which has more than 60 million monthly active users and is now raising up to $150 million to grow its business. The company has been flying under the radar for years, while quietly amassing a large audience of both players and developers who build its virtual worlds. Roblox recently stated that it expects to pay out its content creators $70 million in 2018, which is double that of last year. 

Roblox has a number of built-in controls to guard against bad behavior, including a content filter and a system that has moderators reviewing images, video and audio files before they’re uploaded to Roblox’s site. It also offers parental controls that let parents decide who can chat with their kids, or the ability to turn chat off. And parents can restrict kids under 13 from accessing anything but a curated list of age-appropriate games.

However, Roblox was also in the process of moving some of its older user-generated games to a newer system that’s more secure. The hacked game was one of several that could have been exploited in a similar way.

Since the incident, Roblox had its developers remove all the other potentially vulnerable games and ask their creators to move them over to the newer, more fortified system. Most have done so, and those who have not will not see their games allowed back online until that occurs. The games that are online now are not vulnerable to the exploit the hacker used.

The company responded quickly to take action, in terms of taking the game offline, banning the player and reaching out the mother — who has since agreed to help Roblox get the word out to others about the safeguards parents can use to protect kids in Roblox further.

But the incident raises questions as to whether kids should be playing these sorts of massive multiplayer games at such a young age at all.

Roblox, sadly, is not surprised that someone was interested in a hack like this.

YouTube is filled with videos of Roblox rape hacks and exploits, in fact. The company submits takedown requests to YouTube when videos like this are posted, but YouTube only takes action on a fraction of the requests. (YouTube has its own issues around content moderation.)

It’s long past time for there to be real-world ramifications for in-game assaults that can have lasting psychological consequences on victims, when those victims are children.

Roblox, for its part, is heavily involved in discussions about what can be done, but the issue is complex. COPPA laws prevent Roblox from collecting data on its users, including their personal information, because the law is meant to protect kids’ privacy. But the flip side of this is that Roblox has no way of tracking down hackers like this.

“I think that we’re not the only one pondering the challenges of this. I think every platform company out there is struggling with the same thing,” says Tami Bhaumik, head of marketing and community safety at Roblox.

“We’re members of the Family Online Safety Institute, which is over 30 companies who share best practices around digital citizenship and child safety and all of that,” she continues. “And this is a constant topic of conversation that we all have – in terms of how do we use technology, how do we use A.I. and machine learning? Do we work with the credit card companies to try to verify [users]? How do we get around not violating COPPA regulations?,” says Bhaumik.

“The problem is super complex, and I don’t think anyone involved has solved that yet,” she adds.

One solution could be forcing parents to sign up their kids and add a credit card, which would remain uncharged unless kids broke the rules.

That could dampen user growth to some extent — locking out the under-banked, those hesitant to use their credit cards online and those just generally distrustful of gaming companies and unwanted charges. It would mean kids couldn’t just download the app and play.

But Roblox has the momentum and scale now to lock things down. There’s enough demand for the game that it could create more of a barrier to entry if it chose to, in an effort to better protect users. After all, if players knew they’d be fined (or their parents would be), it would be less attractive to break the rules.

More TechCrunch

Featured Article

I’m rooting for Melinda French Gates to fix tech’s broken ‘brilliant jerk’ culture

Women in tech still face a shocking level of mistreatment at work. Melinda French Gates is one of the few working to change that.

2 hours ago
I’m rooting for Melinda French Gates to fix tech’s  broken ‘brilliant jerk’ culture

Blue Origin has successfully completed its NS-25 mission, resuming crewed flights for the first time in nearly two years. The mission brought six tourist crew members to the edge of…

Blue Origin successfully launches its first crewed mission since 2022

Creative Artists Agency (CAA), one of the top entertainment and sports talent agencies, is hoping to be at the forefront of AI protection services for celebrities in Hollywood. With many…

Hollywood agency CAA aims to help stars manage their own AI likenesses

Expedia says Rathi Murthy and Sreenivas Rachamadugu, respectively its CTO and senior vice president of core services product & engineering, are no longer employed at the travel booking company. In…

Expedia says two execs dismissed after ‘violation of company policy’

Welcome back to TechCrunch’s Week in Review. This week had two major events from OpenAI and Google. OpenAI’s spring update event saw the reveal of its new model, GPT-4o, which…

OpenAI and Google lay out their competing AI visions

When Jeffrey Wang posted to X asking if anyone wanted to go in on an order of fancy-but-affordable office nap pods, he didn’t expect the post to go viral.

With AI startups booming, nap pods and Silicon Valley hustle culture are back

OpenAI’s Superalignment team, responsible for developing ways to govern and steer “superintelligent” AI systems, was promised 20% of the company’s compute resources, according to a person from that team. But…

OpenAI created a team to control ‘superintelligent’ AI — then let it wither, source says

A new crop of early-stage startups — along with some recent VC investments — illustrates a niche emerging in the autonomous vehicle technology sector. Unlike the companies bringing robotaxis to…

VCs and the military are fueling self-driving startups that don’t need roads

When the founders of Sagetap, Sahil Khanna and Kevin Hughes, started working at early-stage enterprise software startups, they were surprised to find that the companies they worked at were trying…

Deal Dive: Sagetap looks to bring enterprise software sales into the 21st century

Keeping up with an industry as fast-moving as AI is a tall order. So until an AI can do it for you, here’s a handy roundup of recent stories in the world…

This Week in AI: OpenAI moves away from safety

After Apple loosened its App Store guidelines to permit game emulators, the retro game emulator Delta — an app 10 years in the making — hit the top of the…

Adobe comes after indie game emulator Delta for copying its logo

Meta is once again taking on its competitors by developing a feature that borrows concepts from others — in this case, BeReal and Snapchat. The company is developing a feature…

Meta’s latest experiment borrows from BeReal’s and Snapchat’s core ideas

Welcome to Startups Weekly! We’ve been drowning in AI news this week, with Google’s I/O setting the pace. And Elon Musk rages against the machine.

Startups Weekly: It’s the dawning of the age of AI — plus,  Musk is raging against the machine

IndieBio’s Bay Area incubator is about to debut its 15th cohort of biotech startups. We took special note of a few, which were making some major, bordering on ludicrous, claims…

IndieBio’s SF incubator lineup is making some wild biotech promises

YouTube TV has announced that its multiview feature for watching four streams at once is now available on Android phones and tablets. The Android launch comes two months after YouTube…

YouTube TV’s ‘multiview’ feature is now available on Android phones and tablets

Featured Article

Two Santa Cruz students uncover security bug that could let millions do their laundry for free

CSC ServiceWorks provides laundry machines to thousands of residential homes and universities, but the company ignored requests to fix a security bug.

2 days ago
Two Santa Cruz students uncover security bug that could let millions do their laundry for free

TechCrunch Disrupt 2024 is just around the corner, and the buzz is palpable. But what if we told you there’s a chance for you to not just attend, but also…

Harness the TechCrunch Effect: Host a Side Event at Disrupt 2024

Decks are all about telling a compelling story and Goodcarbon does a good job on that front. But there’s important information missing too.

Pitch Deck Teardown: Goodcarbon’s $5.5M seed deck

Slack is making it difficult for its customers if they want the company to stop using its data for model training.

Slack under attack over sneaky AI training policy

A Texas-based company that provides health insurance and benefit plans disclosed a data breach affecting almost 2.5 million people, some of whom had their Social Security number stolen. WebTPA said…

Healthcare company WebTPA discloses breach affecting 2.5 million people

Featured Article

Microsoft dodges UK antitrust scrutiny over its Mistral AI stake

Microsoft won’t be facing antitrust scrutiny in the U.K. over its recent investment into French AI startup Mistral AI.

2 days ago
Microsoft dodges UK antitrust scrutiny over its Mistral AI stake

Ember has partnered with HSBC in the U.K. so that the bank’s business customers can access Ember’s services from their online accounts.

Embedded finance is still trendy as accounting automation startup Ember partners with HSBC UK

Kudos uses AI to figure out consumer spending habits so it can then provide more personalized financial advice, like maximizing rewards and utilizing credit effectively.

Kudos lands $10M for an AI smart wallet that picks the best credit card for purchases

The EU’s warning comes after Microsoft failed to respond to a legally binding request for information that focused on its generative AI tools.

EU warns Microsoft it could be fined billions over missing GenAI risk info

The prospects for troubled banking-as-a-service startup Synapse have gone from bad to worse this week after a United States Trustee filed an emergency motion on Wednesday.  The trustee is asking…

A US Trustee wants troubled fintech Synapse to be liquidated via Chapter 7 bankruptcy, cites ‘gross mismanagement’

U.K.-based Seraphim Space is spinning up its 13th accelerator program, with nine participating companies working on a range of tech from propulsion to in-space manufacturing and space situational awareness. The…

Seraphim’s latest space accelerator welcomes nine companies

OpenAI has reached a deal with Reddit to use the social news site’s data for training AI models. In a blog post on OpenAI’s press relations site, the company said…

OpenAI inks deal to train AI on Reddit data

X users will now be able to discover posts from new Communities that are trending directly from an Explore tab within the section.

X pushes more users to Communities

For Mark Zuckerberg’s 40th birthday, his wife got him a photoshoot. Zuckerberg gives the camera a sly smile as he sits amid a carefully crafted re-creation of his childhood bedroom.…

Mark Zuckerberg’s makeover: Midlife crisis or carefully crafted rebrand?

Strava announced a slew of features, including AI to weed out leaderboard cheats, a new ‘family’ subscription plan, dark mode and more.

Strava taps AI to weed out leaderboard cheats, unveils ‘family’ plan, dark mode and more