AS SHOOTERS TURN TOWARD NEW SOCIAL NETWORKS, OLD SAFEGUARDS FAIL
Before two 18-year-old men allegedly killed 31 people in separate shootings over the past two weeks, they turned to a variety of social media apps to share troubling private messages. Both men - one killed by police in Uvalde, Texas, and the other charged in the Buffalo shooting - used a combination of disappearing-video app Snapchat, Instagram direct messages, chat app Discord and social app Yubo to meet people and share their violent plans with acquaintances. In Buffalo, the suspect also used the video streaming platform Twitch to publicize his deadly attack.
These apps - many of which have been adopted by Gen Z as teens and young people seek out more-private corners of the Internet - are ill-equipped to police such content. They are fundamentally designed to keep communications private, presenting different challenges than Facebook, YouTube and Twitter, where violent screeds and videos have been algorithmically amplified to millions of viewers.
The way that generation uses social media more generally could render years of work to spot and identify public signs of upcoming violence obsolete, social media experts warn.
“There is this shift toward more-private spaces, more-ephemeral content,” said Evelyn Douek, a senior research fellow at the Knight First Amendment
Institute at Columbia University. “The content moderation tools that platforms have been building and that we’ve been arguing about are kind of dated or talking about the last war.”
Texas Gov. Greg Abbott (R) said
Wednesday that the Texas gunman, whom authorities have identified as
Salvador Rolando Ramos, wrote on social media that “I’m going to shoot my grandmother” and “I’m going to shoot an elementary school” shortly before the attack. Facebook confirmed that the messages were sent privately but declined to say which of its social networks were used.
Stephen Garcia, who considered himself Ramos’s best friend in eighth grade, previously told The Washington
Post that Ramos used the Yubo app, a platform where users can swipe on each other’s profile, Tinder-style, or hang out in live-streaming rooms and virtually
“meet” other users by playing games and chatting.
Yubo spokeswoman Amy Williams said in an email that the company is not able to release information outside of direct requests from law enforcement, but that the company is investigating an account that has been banned from its platform.
“We are deeply saddened by this unspeakable loss and are fully cooperating with law enforcement on their investigation,” she said.
In the case of Buffalo shooting, the alleged gunman,
Payton Gendron, sent an invitation to an online chatroom on the instant messaging platform Discord that was accepted by 15 users, which allowed them to scroll back through months of Gendron’s voluminous writings and racist screeds, The Washington Post has reported. Users who clicked through to the room also could view an online video stream, where footage of the Buffalo attack was broadcast. That attack was also broadcast on Twitch, a live-streaming service popular among video game users.
Twitch was able to remove the stream within two minutes after the gunman began shooting, Angela
Hession, the company’s head of trust and safety, said previously. The site has an all-hours escalation system in place to address urgent reports, such as livestreamed violence.
Discord has since said the messages were visible only to the suspect until he shared them with others the day of the attack.
In the wake of high-profile mass shootings in recent years, communities, school districts and tech companies made major investments in safety systems aimed to root out violent screeds in the hopes of preventing an attack before it happens. The Uvalde Consolidated
Independent School District has previously used an artificial intelligence-backed program to scan social media posts for potential threats years before the attack, although it’s unclear whether it was in use at the time of the shooting.
But these tools are ill-equipped to address the surging popularity of live video streaming and private or disappearing messaging, which are increasingly used by young adults and teens. Those messages are then closed off to outsiders, who might be able to spot the warning signs that a troubled individual might be about to inflict harm on themselves and others.
These newer social networks also have far less history dealing with violent content, and they’re less likely to have policies and personnel in place to respond to the incitement of violence on their services, experts said.
“For smaller sites or newer sites, they’re having the moments that bigger services like Facebook and YouTube were having in 2015 and 2016,” said Emma Llansó, the director of the free expression project at the Center for Democracy and Technology, a nonprofit backed by major tech companies.
The shooters’ adoption of these upstart apps reflects a larger generational shift among social media use. Gen Z, teens and young adults born after 1996, have been flocking to apps that emphasize private messaging, live-streaming or allow their users to post content that disappears from public profiles after a certain amount of time.
They have largely shunned legacy social media apps such as Facebook, Twitter and YouTube, that rose to popularity by providing public and open spaces to communicate with the world.
The new apps’ role in the shootings have caught the attention of the New York and New Jersey state attorneys general, who in the wake of the Buffalo shooting launched probes into Discord and Twitch.
“Time and time again, we have seen the real-world devastation that is borne of these dangerous and hateful platforms,” New York attorney general Letitia James (D) said in a statement announcing the probe after the Buffalo shooting. “We are doing everything in our power to shine a spotlight on this alarming behavior and take action to ensure it never happens again.”
Social media has played a prominent role in many mass shootings, and there have been high-profile instances where gunmen have posted about their plans online in plain sight and have not been caught.
Republican lawmakers, who have long resisted measures to expand background checks or limit access to guns, aimed to put a spotlight on the role of social media in the Texas shooting on Thursday. “The common theme of almost all of these mass shootings is the social alienation of sick young men, often fueled by social media,” tweeted Sen. Dan Sullivan, R-Alaska. He did not mention gun access in the post.
Tech industry officials pushed back, warning that such tweets could distract from broader policy questions about gun control.
“Some people will try to make it about Facebook so that it’s not about guns,” tweeted Brian Fishman, former director of counterterrorism, dangerous organizations and content policy at Facebook. “Don’t let them.”
Tech giants have also been caught up in a years-long power struggle as they seek to balance privacy with policing content on their sites and demands from law enforcement agencies.
Facebook and other companies have moved toward strong encryption, technology that scrambles the contents of a message so that only the sender and receiver can see it. WhatsApp and Apple iMessage use it, as well as messaging apps like Signal. And Facebook has said it wants to introduce encrypted messaging as a default setting to Instagram and Facebook Messenger, prompting backlash from politicians and officials in law enforcement who have warned that the broad adoption of this technology can leave them in the dark and made it more difficult for them to investigate violence.
Some major tech companies do scan messages for harmful content, such as child sexual abuse or spam. But experts warn that monitoring more private communication spaces is a delicate balance.
“It’s always going to be a cat and mouse game,” Douek said. “These are just sort of intractable problems. But that doesn’t mean that we can’t improve or we should let platforms off the hook.”
“There is this shift toward more-private spaces, more-ephemeral
content.” —Evelyn Douek