Almost simultaneously, Ireland’s Data Protection Commission launched an investigation into Grok, X’s generative AI chatbot, for producing sexualized deepfakes which might have included personal data of Europeans, including children. Even British prime minister Keir Starmer, who has signed a sweeping “Technology Prosperity Deal” with the U.S. has, spoken about the need to “protect children’s wellbeing” from Grok. And earlier this month, French police searched the Paris offices of X as part of a process that X described as a “politicized criminal investigation.” 

With Australia having set a precedent for “age-gating” the Internet through legislation, France’s under-15 ban is now due to come into force in September. The UK already requires age verification for certain content via the Online Safety Act, and Spain, Slovenia, Denmark, Germany, and Greece are among those considering similar measures. ​​The social and political consensus is striking. A 30-country Ipsos survey found strong majorities in every country supporting bans for under-14s.

Elon Musk responded to the Spanish prime minister’s comments about social media being essentially a failed state, rife with criminality and a disregard for law, by calling him “a tyrant and traitor to the people of Spain.” The U.S. government has only been marginally more restrained. The House Judiciary Committee’s report accused Europe of mounting a decade-long campaign to “censor the global internet.” 

The involvement of the U.S. government, and its consistent defence of U.S. tech companies, means the battle is increasingly less about European regulators and Silicon Valley and more about what appears to be a profound ideological mismatch. “Though often framed as combating so-called ‘hate speech’ or ‘disinformation’,” said the Republican legislators’ report, the EU was working to “censor true information and political speech about some of the most important policy debates in recent history — including the Covid-19 pandemic, mass migration, and transgender issues.” Meanwhile, a recently published report in Europe shows how Silicon Valley companies spent 151 million euros lobbying far right European parliamentarians in 2025 to water down regulations. 

Big Tech forging links to the European far right dovetails with a Trump administration in which senior figures, including Donald Trump himself, endorse certain candidates in elections and routinely repeat far right talking points as part of an “unapologetic defense of Western civilization.” And now the U.S. State Department has openly touted the building of a “freedom.gov” portal that enables people to access restricted content, even if it contravenes local laws in sovereign countries.

But European regulations are not the only challenge to the impunity with which social media platforms seem to be able to act. As momentum builds to hold social media platforms to account in Europe, in the U.S. Meta owner Mark Zuckerberg has been defending Instagram in a Los Angeles courtroom. He was testifying in a lawsuit, one of several hundred filed in U.S. civil courts, alleging that social media platforms are addictive, harm the mental health of children and that platforms are aware of these effects but do little to safeguard teenage users from harm. 

The lawsuits have the effect of making the Australian, European, and perhaps global attempt to ban teens from setting up social media accounts appear necessary. But Paige Collings, a digital policy expert at the Electronic Frontier Foundation and board member at European Digital Rights, said that bans are politically attractive precisely because they are simple. “Complex problems require complex solutions,” she said. “It’s more expensive. It’s longer-term. It can’t just be implemented overnight. But blocking under-16s from social media — that is something you can implement overnight.” 

Collings is among a growing chorus of experts that are cautious about embracing bans as a comprehensive solution. For instance, she explains, to ban children, platforms first need to know who is a child. This relies on national digital ID systems, facial recognition, and third-party age verification. In all scenarios, Collings said, “we are trusting that these services and platforms are not storing this information, not selling the information,” often without meaningful guardrails to ensure that is the case. 

When the UK introduced age restrictions last summer, searches for VPNs surged as users of all ages tried to avoid giving away personal information. Now the government has floated expanding restrictions to VPN usage to plug enforcement gaps. The purpose of a VPN itself is to preserve the privacy of its user, however imperfectly. VPNs are essential tools for businesses to secure communications, for journalists to protect sources, and for citizens in restrictive environments to access independent information. Forcing identification to use them fundamentally undermines their purpose. And when the argument for banning them is framed around the protection of children, it reinstates the urgency of an entire infrastructure required to keep children off the internet and risks normalizing identity checks as conditions for access to online spaces. In a digital economy where personal data is highly valuable, such measures raise the question of who ultimately benefits.Beyond privacy concerns, Collings points out that age-gating can become “a fantastic tool for censorship with no accountability or remedy.” It does, in fact, do in part what the U.S. government and Silicon Valley companies say it does, which is restrict speech. At an AI summit in Delhi, French president Emmanuel Macron dismissed Silicon Valley’s invocation of censorship as a defense against European regulation. “Free speech,” he said, “is pure bullshit if nobody knows how you are guided through this… having no clue about how the algorithm is made, how it is tested and where it will guide you — the democratic biases of this could be huge.” But, forcing accountability and improving safety would perhaps be better than a blanket ban where the cutoff is 14 or 16, leaving everyone else to take cover as best they can in a “digital Wild West,” to borrow the Spanish prime minister’s phrase.

Related Articles

Turn off, tune out: Australia takes its kids off social media

Isobel Cockerell

The global battle to age-gate the internet

Lydia Morrish

Lying about your age? This AI will see right through it

Chris Stokel-Walker

The post Europe vs Big Tech: A battle for democracy? appeared first on Coda Story.

It came from Christopher Wylie. He’s a data scientist and the whistleblower who cracked open the Cambridge Analytica scandal in 2018. I had just started working with him on a new investigation into AI. Chris was supposed to be meeting me, but he had found himself trapped in Dubai in a party full of Silicon Valley venture capitalists.

“I don’t know if you can hear me — I’m in the toilet at this event, and people here are talking about longevity, how to live forever, but also prepping for when people revolt and when society gets completely undermined,” he had whispered into his phone. “You have in another part of the world, a bunch of journalists talking about how to save democracy. And here, you've got a bunch of tech guys thinking about how to live past democracy and survive.”

A massive storm and a once-in-a-generation flood had paralyzed Dubai when Chris was on a layover on his way to Perugia. He couldn’t leave. And neither could the hundreds of tech guys who were there for a crypto summit. The freakish weather hadn’t stopped them partying, Chris told me over a frantic Zoom call. 

“You're wading through knee-deep water, people are screaming everywhere, and then…  What do all these bros do? They organize a party. It's like the world is collapsing outside and yet you go inside and it's billionaires and centimillionaires having a party,” he said. “Dubai right now is a microcosm of the world. The world is collapsing outside and the people are partying.”

Chris and I eventually managed to meet up. And for over a year we worked together on a podcast that asks what is really going on inside the tech world.  We looked at how the rest of us —  journalists, artists, nurses, businesses, even governments — are being captured by big tech’s ambitions for the future and how we can fight back. 

Mercy was a content moderator for Meta. She was paid around a dollar an hour for work that left her so traumatized that she couldn't sleep. And when she tried to unionize, she was laid off.

Our reporting took us around the world from the lofty hills of Twin Peaks in San Francisco to meet the people building AI models, to the informal settlements of Kenya to meet the workers training those models.

One of these people was Mercy Chimwani, who we visited in her makeshift house with no roof on the outskirts of Nairobi. There was mud beneath our feet, and above you could see the rainclouds through a gaping hole where the unfinished stairs met the sky. When it rained, Mercy told us, water ran right through the house. It’s hard to believe, but she worked for Meta. 

Mercy was a content moderator, hired by the middlemen Meta used to source employees. Her job was to watch the internet’s most horrific images and video –  training the company’s system so it can automatically filter out such content before the rest of us are exposed to it. 

She was paid around a dollar an hour for work that left her so traumatized that she couldn’t sleep. And when she and her colleagues tried to unionize, she was laid off. Mercy was part of the invisible, ignored workforce in the Global South that enables our frictionless life online for little reward. 

Of course, we went to the big houses too — where the other type of tech worker lives. The huge palaces made of glass and steel in San Francisco, where the inhabitants believe the AI they are building will one day help them live forever, and discover everything there is to know about the universe. 

In Twin Peaks, we spoke to Jeremy Nixon, the creator of AGI House San Francisco (AGI for Artificial General Intelligence). Nixon described an apparently utopian future, a place where we never have to work, where AI does everything for us, and where we can install the sum of human knowledge into our brains. “The intention is to allow every human to know everything that’s known,” he told me. 

Later that day, we went to a barbecue in Cupertino and got talking to Alan Boehme, once a chief technology officer for some of the biggest companies in the world, and now an investor in AI startups. Boehme told us how important it was, from his point of view, that tech wasn’t stymied by government regulation. “We have to be worried that people are going to over-regulate it. Europe is the worst, to be honest with you,” he said. “Let's look at how we can benefit society and how this can help lead the world as opposed to trying to hold it back.”

I asked him if regulation wasn’t part of the reason we have democratically elected governments, to ensure that all people are kept safe, that some people aren’t left behind by the pace of change? Shouldn’t the governments we elect be the ones deciding whether we regulate AI and not the people at this Cupertino barbecue?

“You sound like you're from Sweden,” Boehme responded. “I'm sorry, that's social democracy. That is not what we are here in the U. S. This country is based on a Constitution. We're not based on everybody being equal and holding people back. No, we're not in Sweden.” 

As we reported for the podcast, we came to a gradual realization – what’s being built in Silicon Valley isn’t just artificial intelligence, it’s a way of life — even a religion. And it’s a religion we might not have any choice but to join. 

In January, the Vatican released a statement in which it argued that we’re in danger of worshiping AI as God. It's an idea we'd discussed with Judy Estrin, who worked on building some of the earliest iterations of the internet. As a young researcher at Stanford in the 1970s, Estrin was building some of the very first networked connections. She is no technophobe, fearful of the future, but she is worried about the zealotry she says is taking over Silicon Valley.

What if they truly believe humans are replaceable, that traditional concepts of humanity are outdated, that a technological "god" should supersede us? These aren't just ideological positions – they're the foundations for the world being built around us.

“If you worship innovation, if you worship anything, you can't take a step back and think about guardrails,” she said about the unquestioning embrace of AI. “So we, from a leadership perspective, are very vulnerable to techno populists who come out and assert that this is the only way to make something happen.” 

The first step toward reclaiming our lost agency, as AI aims to capture every facet of our world, is simply to pay attention. I've been struck by how rarely we actually listen to what tech leaders are explicitly saying about their vision of the future. 

There's a tendency to dismiss their most extreme statements as hyperbole or marketing, but what if they're being honest? What if they truly believe humans, or at least most humans, are replaceable, that traditional concepts of humanity are outdated, that a technological "god" should supersede us? These aren't just ideological positions – they're the foundations for the world being built around us right now. 

In our series, we explore artificial intelligence as something that affects our culture, our jobs, our media and our politics. But we should also ask what tech founders and engineers are really building with AI, or what they think they’re building. Because if their vision of society does not have a place for us in it, we should be ready to reclaim our destiny – before our collective future is captured.

Our audio documentary series, CAPTURED: The Secret Behind Silicon Valley’s AI Takeover is available now on Audible. Do please tune in, and you can dig deeper into our stories and the people we met during the reporting below.

Dig deeper into our CAPTURED series

Who owns the rights to your brain?

Isobel Cockerell

In Kenya’s slums, they’re doing our digital dirty work

Isobel Cockerell

Stop Drinking from the Toilet!

Judy Estrin

The post Captured: how Silicon Valley is building a future we never chose appeared first on Coda Story.

Over the last few weeks the US has been thrown into a tailspin over the impending divest-or-ban law that might possibly block the youth of America from accessing their favorite short-form video app. But if you've only been following the Supreme Court's hearing on TikTok you may have totally missed an entirely separate Supreme Court hearing on whether or not southern American states like Texas are constitutionally allowed to block porn sites like Pornhub. As of this month, 17 US states have blocked Pornhub for refusing to adhere to "age-verification laws" that would force Pornhub to collect users' IDs before browsing the site, thus making sensitive, personal information vulnerable to security breaches. 

But it's not just US lawmakers that are questioning what's allowed on their corner of the web. 

Following a recent announcement that Meta would be relaxing their fact checking standards Brazilian regulators demanded a thorough explanation of how this would impact the country's 100 million users. Currently the Brazilian government is "seriously concerned" about these changes. Which itself is almost a verbatim repeat of how Brazilian lawmakers dealt with X last year, when they banned the platform for almost two months over how the platform handled misinformation about the country's 2023 attempted coup.

Speaking of X, the European Union seems to have finally had enough of Elon Musk's digital megaphone. They've been investigating the platform since 2023 and have given Musk a February deadline to explain exactly how the platform's algorithm works. To say nothing of the French and German regulators grappling with how to deal with Musk's interference in their national politics.

And though the aforementioned Chinese Great Firewall has always blocked the rest of the world from the country's internet users, last week there was a breach that Chinese regulators are desperately trying to patch. Americans migrated to a competing app called RedNote, which has now caught the attention of both lawmakers in China, who are likely to wall off American users from interacting with Chinese users, and lawmakers in the US, who now want to ban it once they finally deal with TikTok.

All of this has brought us to a stark new reality, where we can no longer assume that the internet is a shared global experience, at least when it comes to the web's most visible and mainstream apps. New digital borders are being drawn and they will eventually impact your favorite app. Whether you're an activist, a journalist, or even just a normal person hoping to waste some time on their phone (and maybe make a little money), the spaces you currently call home online are not permanent. 

Time to learn how a VPN works. At least until the authorities restrict and regulate access to VPNs too, as they already do in countries such as China, Iran, Russia and India. 

A version of this story was published in this week’s Coda Currents newsletter. Sign up here.

The post Blocking Pornhub and the death of the World Wide Web appeared first on Coda Story.

For years, Silicon Valley's tech moguls have systematically engineered a world where truth is optional, equity and justice are expendable, and facts are toxic waste. By dismantling both fact-checking operations and DEI programs, Meta stands to save millions - with DEI programs already facing cost-cutting measures in 2023, the move shows that the only responsibility Zuckerberg appears to take seriously is the bottom line. The surprising part isn't that Meta has stopped pretending to care about anything but their power and profit - it's that we were ever naive enough to believe they did.

The consequences of this decision will play out globally, and few understand those consequences better than Maria Ressa. The founder of Rappler, was among the first to document how social media platforms enabled the rise of authoritarianism in her native Philippines, where Facebook became so dominant that it "rewired our people's brains." Ressa, a 2021 Nobel Peace Prize laureate. says "propaganda is like cocaine - you take it once or twice, you're okay. But if you take it all the time, you become an addict. And we are all addicts today." When she spoke these words at Coda's Zeg Festival last June, they felt like a warning. Now they read like a prophecy fulfilled.

Her warning wasn't just about addiction to propaganda – it was about the deliberate architecture of our digital world. "These tech companies are engineering a world without facts," she says, "and that's a world that's right for a dictator."

The Engineering of Chaos

Tech pioneer Judy Estrin frames the problem in stark infrastructural terms: "Digital platforms mix 'digital water' and 'sewage' in the same pipes, polluting our information systems and undermining the foundations of our culture, our public health, our economy and our democracy."

This pollution isn't accidental - it's a feature, not a bug. Meta's announcement, coinciding with Elon Musk's open championing of far-right movements in Europe, reveals a profound transformation in Silicon Valley. Tech moguls who once felt pressured to champion openness and truth are now racing to shed any pretense of responsibility.

It isn't just about catering to Donald Trump and the sentiments of his followers. The shift is about how tech companies view their stakeholders. Where platforms once felt compelled to respond to pressure from employees, users, and advertisers concerned about digital pollution, they've now consolidated power solely around profits. The workforce  that once served as a guardrail for online behavior has been neutralized - a trend Elon Musk pioneered when he bought Twitter. And Meta's move to end its 'Diverse Slate Approach' to hiring and representation goals, while adding Trump allies like the Ultimate Fighting Championship supremo Dana White to its board, shows exactly where power now lies.

The Infrastructure of Authoritarianism

For years, we've analyzed electoral manipulation, documented democratic backsliding, and tracked the rise of strongmen while treating platforms as mere conduits rather than active architects of our political reality. The entire debate around content moderation appears in retrospect to have been a carefully crafted distraction – a game of Whack-a-Mole that kept us focused on individual pieces of content rather than the systemic nature of the problem. As one former Meta employee said, "It's like putting a beach shack in the way of a massive tsunami and expecting it to be a barrier."

"Facebook's 'fact checking' initiative was at heart always a PR exercise," argues Emily Bell, whose research at the Tow Center at Columbia University focuses on the intersection of platforms, media and information integrity. "Nothing has changed about the platform's mission: to make money from the exploitation of IP and data created for free."

By abandoning civic responsibility, while disingenuously claiming to be acting in the interests of free speech, Zuckerberg and Musk aren't so much transforming their platforms as finally being honest about what these platforms have always been: engines of engagement designed to maximize profit and power, regardless of societal cost. The real shift isn't in their behavior – it's in our belated recognition that no meaningful conversation about democracy can exclude the role of the broligarchy in shaping our information ecosystem.

The Future of Truth

Tech platforms have wielded the First Amendment much like the gun lobby has wielded the Second: turning constitutional protections into a weapon against regulation and accountability. Just as gun manufacturers claim they bear no responsibility for how their products are used, platform owners insist they're merely providing neutral spaces for free expression – all while their algorithms amplify lies and fuel society's most self-destructive impulses.

And we are all complicit. The endless scrolling of TikTok and Instagram, the ease of WhatsApp communications, the ability to instantly connect with friends and family across the globe – these aren't just corporate products, they're now fundamental to our daily lives. But in our embrace of this convenience, we've sleepwalked into a future where the rejection of facts isn't just the domain of authoritarian governments in Moscow or Beijing, but of giant tech companies in Silicon Valley.

Many respected journalists and human rights defenders lent their credibility to Meta's Oversight Board – a body that could review a handful of content decisions but had little effect on the platform's fundamental design or business model. "The Oversight Board is absolutely the wrong problem [to address]," Ressa says. "They tried to call it the Supreme Court for content. Content is not the problem. The distribution and the rate of distribution is the problem. The design of the platform, none of which they have any power over. But yet they were able to get very credible people."

The Path Forward

The solutions we've pursued – from fact-checking initiatives to content moderation boards – have been mere band-aids applied on a deep systemic wound. As platforms poured millions into lobbying and institutional capture (Meta spent $7.6 million on lobbying the U.S. government in just the first quarter of 2024), we settled for superficial fixes that left their core business model untouched. As long as news organizations treat platforms as mere distribution channels rather than existential threats to information integrity, we will remain trapped in a cycle of ineffective half-measures. 

Before journalists point fingers solely at tech platforms, we should also look in the mirror - especially those of us who've made careers out of dealing in facts and telling truths. Journalists, researchers, scientists, educators - we're all part of this story. While tech platforms may at some point be regulated (though good luck with that during a Trump administration), we need to get real about our own role in this mess.

While we must figure out how to work toward systemic change, there is still power in how we choose to engage with these platforms. Every scroll, every share, every moment of attention we give is a choice. By being more conscious about where we get our information, how we verify it, and most importantly, how we pass it on, we can start reclaiming some control over our information environment. Small individual actions - from supporting independent journalism to thinking twice before spreading unverified content - add up to collective resistance against a system designed to exploit our worst impulses.

Those of us in journalism and media must also ask ourselves: Have we been complicit in providing cover for systems we knew were fundamentally broken? Have we prioritized our convenience and digital reach over the integrity of information? Most importantly, are we ready to acknowledge that our industry's survival, and arguably that of democracy overall, depends on confronting these platforms' role in undermining the very foundations of factual discourse?

The answers to these questions will determine whether we can rebuild an information ecosystem that serves society rather than corrodes it. But the very first step towards a society in which facts matter and truth has value, is admitting that the destruction of truth wasn't an accident - it was by design.

*Disclosure: Maria Ressa serves on Coda's Board of Directors.

Related Articles

Stop Drinking from the Toilet!

Judy Estrin

Silicon Savanna: The workers taking on Africa’s digital sweatshops

Erica Hellerstein

How Silicon Valley is helping Putin and other tyrants win the information war

Natalia Antelava

The post The death of truth was by design appeared first on Coda Story.

These currents connect the crises happening in different parts of the world into a global web of intricately connected problems. It may not be obvious, but Silicon Valley is right at the very heart of this web. Home to some of the richest and most powerful men on earth, Silicon Valley is the birthplace of the technology that has given us so much convenience and also taken so much away from us. 

The world may be on fire, but things are going well for Silicon Valley’s most powerful men. Sam Altman, the CEO of OpenAI, which is now officially worth $157 billion and Mark Zuckerberg, whose $72 billion dollar wealth surge this year could now make him the richest person on earth. 

Both are in a position to address some of the world’s greatest problems, yet both choose to avoid any responsibility, and instead choose to obscure and deflect. 

Take AI-powered disinformation in this election for example. It’s rampant, scary and consequential for American democracy. Sam Altman’s response? He wants us to be patient. In his recent letter worthy of a techno-optimism medal Altman argues that it would be a “mistake to get distracted by any particular challenge. Deep learning works and we will solve the remaining problems”.  

Zuckerberg says he wants Meta to be remembered for “building big,” not safe. Meta no longer even engages in a whack-a-mole game of fact checking and content moderation. Along with Google, Amazon and X, Meta has essentially dismantled its Trust and Safety team that at least tried to mitigate the real life damage caused by the algorithmic promotion of hateful content. Mark Zuckerberg, who wore an “Aut Zuck Aut Nihil.” “Either Zuck or Nothing” shirt as he presented his latest meta verse at the company’s annual developer conference. As for life in this world, he is apparently done with politics. 

It takes a very special kind of privilege to ask for patience in the face of a major, life threatening, world changing crises. The attitude is familiar to anyone who has seen authoritarianism up close: the goal of an authoritarian is to secure a monopoly on money and power. Maintaining a monopoly of the narrative is the way of achieving that. Human suffering may not be the objective, but if that’s what it takes to achieve the desired outcome, then it’s just collateral damage. 

I spent a lot of my week speaking to people who could be considered “collateral damage”: people in Beirut, where unprecedented escalation of violence between Israel and Iran is wreaking havoc on millions of lives. Friends in Ukraine, where Russia is making territorial gains while continuing to bomb, kill and maim civilians.  

As well as my own family in Georgia, where the Kremlin is making political gains: the Russian state propaganda machine is now openly backing an autocratic, populist government that is about to use a democratic tool–elections–to pull the country deeper into its autocratic orbit. The government’s campaign strategy resembles blackmail. “If you don’t vote for us, Russia will do to you what it did to Ukraine,” is literally the message of the election billboards the Georgian government put up this week.  

The roots of each of these crises are buried deep in the history of individual places, but so much of the journalism we do at Coda brings us back to Silicon Valley. 

The valley is the modern day equivalent of the heart of the Roman empire; a place of extreme abundance, fantastic innovation and terrifying detachment from the rest of the world.  
For this reason, it has never been more important to connect the dots between the patterns that weave into the web of our modern life.

The post Global crises, local consequences: how Silicon Valley shapes our world appeared first on Coda Story.

Anthropologist Dragana Kaurin told me this week about a strange encounter she had with it some years back.

“I opened Facebook and I saw a face and a name I recognized. It was my first grade teacher,” she told me. Kaurin is Bosnian and fled Sarajevo as a child, at the start of the war and genocide that took hundreds of thousands of lives between 1992 and 1995. One of Kaurin’s last memories of school life in Sarajevo was of that very same teacher separating children in the classroom on the basis of their ethnicity, as if to foreshadow the ethnic cleansing campaign that soon followed.

“It was widely rumored that our teacher took up arms and shot at civilians, and secondly, that she had died during the war,” she said. “So it was like seeing a ghost.” Now at retirement age, the teacher’s profile showed her membership in a number of ethno-nationalist groups on Facebook. 

Kaurin spent the rest of that day feeling stunned, motionless. “I couldn’t function,” she said.

The people who designed the feature probably didn’t anticipate that it would have such effects. But even after more than a decade of journalists like The New York Times’ Kashmir Hill showing various harms it could inflict — Facebook has suggested that women “friend” their stalkers, sex workers “friend” their clients, and patients of psychiatrists “friend” one another — the “People You May Know” feature is still there today.

From her desk in lower Manhattan, Kaurin now runs Localization Lab, a nonprofit organization that works with underrepresented communities to make technology accessible through collaborative design and translation. She sees the “People You May Know” story as an archetypical example of a technology that was designed without much input from beyond the gleaming Silicon Valley offices in which it was conceived.

“Design is always political,” Kaurin told me. “It enacts underlying policies, biases and exclusion. Who gets to make decisions? How are decisions made? Is there space for iterations?” And then, of course, there’s the money. When a feature helps drive growth on a social media platform, it usually sticks around.

This isn’t a new story. But it is top of mind for me these days because of the emerging consensus that many of the same design mistakes that social media companies have made over the past 15 years will be repeated in the so-called “AI revolution.” And with its opaque nature, its ability to manufacture a false sense of social trust and its ubiquity, artificial intelligence may have the potential to bring about far worse harms than what we’ve seen from social media over the past decade. Should we worry?

“Absolutely,” said Kaurin. And it’s happening on a far bigger, far faster scale, she pointed out.

Cybersecurity guru Bruce Schneier and other prominent thinkers have argued that governments should institute “public AI” models that could function as a counterweight to corporate, profit-driven AI. Some states are already trying this, including China, the U.K. and Singapore. I asked Kaurin and her colleague Chido Musodza if they thought state-run AI models might be better equipped to represent the interests of more diverse sets of users than what’s built in Silicon Valley.

Both researchers wondered who would actually be building the technology and who would use it. “What is the state’s agenda?” Kaurin asked. “How does that state treat minority communities? How do users feel about the state?”

Musodza, who joined our conversation from Harare, Zimbabwe, considered the idea in the southern African context: “When you look at how some national broadcasters have an editorial policy with a political slant aligned towards the government of the day, it’s likely that AI will be aligned towards the same political slant as well,” she said.

She’s got a point. Researchers testing Singapore’s model found that when asked questions about history and politics, the AI tended to offer answers that cast the state in a favorable light.

“I think it would be naive for us to say that even though it’s public AI that it will be built without bias,” said Musodza. “It’s always going to have the bias of whoever designs it.”

Musodza said that for her, the question is: “Which of the evils are we going to pick, if we’re going to use the AI?” That led us to consider that a third way might be possible, depending on a person’s circumstances: to simply leave AI alone.

This piece was originally published as the most recent edition of the weekly Authoritarian Tech newsletter.

Related Articles

Advertising erectile dysfunction pills? No problem. Breast health? Try again

Erica Hellerstein

When Meta suspends influential political accounts, who loses?

Fiona Kelliher

Lithuania goes after bots following spikes in pro-Russian propaganda

Amanda Coakley

and

Ellery Roberts Biddle

The post How tech design is always political appeared first on Coda Story.

This story was updated at 6:30 ET on October 16, 2023

Wabe didn’t expect to see his friends’ faces in the shadows. But it happened after just a few weeks on the job.

He had recently signed on with Sama, a San Francisco-based tech company with a major hub in Kenya’s capital. The middle-man company was providing the bulk of Facebook’s content moderation services for Africa. Wabe, whose name we’ve changed to protect his safety, had previously taught science courses to university students in his native Ethiopia.

Now, the 27-year-old was reviewing hundreds of Facebook photos and videos each day to decide if they violated the company’s rules on issues ranging from hate speech to child exploitation. He would get between 60 and 70 seconds to make a determination, sifting through hundreds of pieces of content over an eight-hour shift.

One day in January 2022, the system flagged a video for him to review. He opened up a Facebook livestream of a macabre scene from the civil war in his home country. What he saw next was dozens of Ethiopians being “slaughtered like sheep,” he said. 

Then Wabe took a closer look at their faces and gasped. “They were people I grew up with,” he said quietly. People he knew from home. “My friends.”

Wabe leapt from his chair and stared at the screen in disbelief. He felt the room close in around him. Panic rising, he asked his supervisor for a five-minute break. “You don’t get five minutes,” she snapped. He turned off his computer, walked off the floor, and beelined to a quiet area outside of the building, where he spent 20 minutes crying by himself.

Wabe had been building a life for himself in Kenya while back home, a civil war was raging, claiming the lives of an estimated 600,000 people from 2020 to 2022. Now he was seeing it play out live on the screen before him.

That video was only the beginning. Over the next year, the job brought him into contact with videos he still can’t shake: recordings of people being beheaded, burned alive, eaten.

“The word evil is not equal to what we saw,” he said. 

Yet he had to stay in the job. Pay was low — less than two dollars an hour, Wabe told me — but going back to Ethiopia, where he had been tortured and imprisoned, was out of the question. Wabe worked with dozens of other migrants and refugees from other parts of Africa who faced similar circumstances. Money was too tight — and life too uncertain — to speak out or turn down the work. So he and his colleagues kept their heads down and steeled themselves each day for the deluge of terrifying images.

Over time, Wabe began to see moderators as “soldiers in disguise” — a low-paid workforce toiling in the shadows to make Facebook usable for billions of people around the world. But he also noted a grim irony in the role he and his colleagues played for the platform’s users: “Everybody is safe because of us,” he said. “But we are not.”  

Wabe said dozens of his former colleagues in Sama’s Nairobi offices now suffer from post-traumatic stress disorder. Wabe has also struggled with thoughts of suicide. “Every time I go somewhere high, I think: What would happen if I jump?” he wondered aloud. “We have been ruined. We were the ones protecting the whole continent of Africa. That’s why we were treated like slaves.”

To most people using the internet — most of the world — this kind of work is literally invisible. Yet it is a foundational component of the Big Tech business model. If social media sites were flooded with videos of murder and sexual assault, most people would steer clear of them — and so would the advertisers that bring the companies billions in revenue.

Around the world, an estimated 100,000 people work for companies like Sama, third-party contractors that supply content moderation services for the likes of Facebook’s parent company Meta, Google and TikTok. But while it happens at a desk, mostly on a screen, the demands and conditions of this work are brutal. Current and former moderators I met in Nairobi in July told me this work has left them with post-traumatic stress disorder, depression, insomnia and thoughts of suicide.

These “soldiers in disguise” are reaching a breaking point. Because of people like Wabe, Kenya has become ground zero in a battle over the future of content moderation in Africa and beyond. On one side are some of the most powerful and profitable tech companies on earth. On the other are young African content moderators who are stepping out from behind their screens and demanding that Big Tech companies reckon with the human toll of their enterprise.

In May, more than 150 moderators in Kenya, who keep the worst of the worst off of platforms like Facebook, TikTok and ChatGPT, announced their drive to create a trade union for content moderators across Africa. The union would be the first of its kind on the continent and potentially in the world.

There are also major pending lawsuits before Kenya’s courts targeting Meta and Sama. More than 180 content moderators — including Wabe — are suing Meta for $1.6 billion over poor working conditions, low pay and what they allege was unfair dismissal after Sama ended its content moderation agreement with Meta and Majorel picked up the contract instead. The plaintiffs say they were blacklisted from reapplying for their jobs after Majorel stepped in. In August, a judge ordered both parties to settle the case out of court, but the mediation broke down on October 16 after the plaintiffs' attorneys accused Meta of scuttling the negotiations and ignoring moderators' requests for mental health services and compensation. The lawsuit will now proceed to Kenya's employment and labor relations court, with an upcoming hearing scheduled for October 31.

The cases against Meta are unprecedented. According to Amnesty International, it is the “first time that Meta Platforms Inc will be significantly subjected to a court of law in the global south.” Forthcoming court rulings could jeopardize Meta’s status in Kenya and the content moderation outsourcing model upon which it has built its global empire. 

Meta did not respond to requests for comment about moderators’ working conditions and pay in Kenya. In an emailed statement, a spokesperson for Sama said the company cannot comment on ongoing litigation but is “pleased to be in mediation” and believes “it is in the best interest of all parties to come to an amicable resolution.”

Odanga Madung, a Kenya-based journalist and a fellow at the Mozilla Foundation, believes the flurry of litigation and organizing marks a turning point in the country’s tech labor trajectory. 

“This is the tech industry’s sweatshop moment,” Madung said. “Every big corporate industry here — oil and gas, the fashion industry, the cosmetics industry — have at one point come under very sharp scrutiny for the reputation of extractive, very colonial type practices.”

Nairobi may soon witness a major shift in the labor economics of content moderation. But it also offers a case study of this industry’s powerful rise. The vast capital city — sometimes called “Silicon Savanna” — has become a hub for outsourced content moderation jobs, drawing workers from across the continent to review material in their native languages. An educated, predominantly English-speaking workforce makes it easy for employers from overseas to set up satellite offices in Kenya. And the country’s troubled economy has left workers desperate for jobs, even when wages are low.

Sameer Business Park, a massive office compound in Nairobi’s industrial zone, is home to Nissan, the Bank of Africa, and Sama’s local headquarters. But just a few miles away lies one of Nairobi’s largest informal settlements, a sprawl of homes made out of scraps of wood and corrugated tin. The slum’s origins date back to the colonial era, when the land it sits on was a farm owned by white settlers. In the 1960s, after independence, the surrounding area became an industrial district, attracting migrants and factory workers who set up makeshift housing on the area adjacent to Sameer Business Park.

For companies like Sama, the conditions here were ripe for investment by 2015, when the firm established a business presence in Nairobi. Headquartered in San Francisco, the self-described “ethical AI” company aims to “provide individuals from marginalized communities with training and connections to dignified digital work.” In Nairobi, it has drawn its labor from residents of the city’s informal settlements, including 500 workers from Kibera, one of the largest slums in Africa. In an email, a Sama spokesperson confirmed moderators in Kenya made between $1.46 and $3.74 per hour after taxes.

Grace Mutung’u, a Nairobi-based digital rights researcher at Open Society Foundations, put this into local context for me. On the surface, working for a place like Sama seemed like a huge step up for young people from the slums, many of whom had family roots in factory work. It was less physically demanding and more lucrative. Compared to manual labor, content moderation “looked very dignified,” Mutung’u said. She recalled speaking with newly hired moderators at an informal settlement near the company’s headquarters. Unlike their parents, many of them were high school graduates, thanks to a government initiative in the mid-2000s to get more kids in school.

“These kids were just telling me how being hired by Sama was the dream come true,” Mutung’u told me. “We are getting proper jobs, our education matters.” These younger workers, Mutung’u continued, “thought: ‘We made it in life.’” They thought they had left behind the poverty and grinding jobs that wore down their parents’ bodies. Until, she added, “the mental health issues started eating them up.” 

Today, 97% of Sama’s workforce is based in Africa, according to a company spokesperson. And despite its stated commitment to providing “dignified” jobs, it has caught criticism for keeping wages low. In 2018, the company’s late founder argued against raising wages for impoverished workers from the slum, reasoning that it would “distort local labor markets” and have “a potentially negative impact on the cost of housing, the cost of food in the communities in which our workers thrive.”

Content moderation did not become an industry unto itself by accident. In the early days of social media, when “don’t be evil” was still Google’s main guiding principle and Facebook was still cheekily aspiring to connect the world, this work was performed by employees in-house for the Big Tech platforms. But as companies aspired to grander scales, seeking users in hundreds of markets across the globe, it became clear that their internal systems couldn’t stem the tide of violent, hateful and pornographic content flooding people’s newsfeeds. So they took a page from multinational corporations’ globalization playbook: They decided to outsource the labor.

More than a decade on, content moderation is now an industry that is projected to reach $40 billion by 2032. Sarah T. Roberts, a professor of information studies at the University of California at Los Angeles, wrote the definitive study on the moderation industry in her 2019 book “Behind the Screen.” Roberts estimates that hundreds of companies are farming out these services worldwide, employing upwards of 100,000 moderators. In its own transparency documents, Meta says that more than 15,000 people moderate its content in more than 20 sites around the world. Some (it doesn’t say how many) are full-time employees of the social media giant, while others (it doesn’t say how many) work for the company’s contracting partners.

Kauna Malgwi was once a moderator with Sama in Nairobi. She was tasked with reviewing content on Facebook in her native language, Hausa. She recalled watching coworkers scream, faint and develop panic attacks on the office floor as images flashed across their screens. Originally from Nigeria, Malgwi took a job with Sama in 2019, after coming to Nairobi to study psychology. She told me she also signed a nondisclosure agreement instructing her that she would face legal consequences if she told anyone she was reviewing content on Facebook. Malgwi was confused by the agreement, but moved forward anyway. She was in graduate school and needed the money.

A 28-year-old moderator named Johanna described a similar decline in her mental health after watching TikTok videos of rape, child sexual abuse, and even a woman ending her life in front of her own children. Johanna currently works with the outsourcing firm Majorel, reviewing content on TikTok, and asked that we identify her using a pseudonym, for fear of retaliation by her employer. She told me she’s extroverted by nature, but after a few months at Majorel, she became withdrawn and stopped hanging out with her friends. Now, she dissociates to get through the day at work. “You become a different person,” she told me. “I’m numb.”

This is not the experience that the Luxembourg-based multinational — which employs more than 22,000 people across the African continent — touts in its recruitment materials. On a page about its content moderation services, Majorel’s website features a photo of a woman donning a pair of headphones and laughing. It highlights the company’s “Feel Good” program, which focuses on “team member wellbeing and resiliency support.”

According to the company, these resources include 24/7 psychological support for employees “together with a comprehensive suite of health and well-being initiatives that receive high praise from our people," Karsten König, an executive vice president at Majorel, said in an emailed statement. "We know that providing a safe and supportive working environment for our content moderators is the key to delivering excellent services for our clients and their customers. And that’s what we strive to do every day.”

But Majorel’s mental health resources haven’t helped ease Johanna’s depression and anxiety. She says the company offers moderators in her Nairobi office with on-site therapists who see employees in individual and group “wellness” sessions. But Johanna told me she stopped attending the individual sessions after her manager approached her about a topic she shared in confidentiality with her therapist. “They told me it was a safe space,” Johanna explained, “but I feel that they breached that part of the confidentiality so I do not do individual therapy.” TikTok did not respond to a request for comment by publication.

Instead, she looked for other ways to make herself feel better. Nature has been especially healing. Whenever she can, Johanna takes herself to Karura Forest, a lush oasis in the heart of Nairobi. One afternoon, she brought me to one of her favorite spots there, a crashing waterfall beneath a canopy of trees. This is where she tries to forget about the images that keep her up at night. 

Johanna remains haunted by a video she reviewed out of Tanzania, where she saw a lesbian couple attacked by a mob, stripped naked and beaten. She thought of them again and again for months. “I wondered: ‘How are they? Are they dead right now?’” At night, she would lie awake in her bed, replaying the scene in her mind.

“I couldn’t sleep, thinking about those women.”

Johanna’s experience lays bare another stark reality of this work. She was powerless to help victims. Yes, she could remove the video in question, but she couldn’t do anything to bring the women who were brutalized to safety. This is a common scenario for content moderators like Johanna, who are not only seeing these horrors in real-time, but are asked to simply remove them from the internet and, by extension, perhaps, from public record. Did the victims get help? Were the perpetrators brought to justice? With the endless flood of videos and images waiting for review, questions like these almost always go unanswered.

The situation that Johanna encountered highlights what David Kaye, a professor of law at the University of California at Irvine and the former United Nations special rapporteur on freedom of expression, believes is one of the platforms’ major blindspots: “They enter into spaces and countries where they have very little connection to the culture, the context and the policing,” without considering the myriad ways their products could be used to hurt people. When platforms introduce new features like livestreaming or new tools to amplify content, Kaye continued, “are they thinking through how to do that in a way that doesn’t cause harm?”

The question is a good one. For years, Meta CEO Mark Zuckerberg famously urged his employees to “move fast and break things,” an approach that doesn’t leave much room for the kind of contextual nuance that Kaye advocates. And history has shown the real-world consequences of social media companies’ failures to think through how their platforms might be used to foment violence in countries in conflict.

The most searing example came from Myanmar in 2017, when Meta famously looked the other way as military leaders used Facebook to incite hatred and violence against Rohingya Muslims as they ran “clearance operations” that left an estimated 24,000 Rohingya people dead and caused more than a million to flee the country. A U.N. fact-finding mission later wrote that Facebook had a “determining role” in the genocide. After commissioning an independent assessment of Facebook’s impact in Myanmar, Meta itself acknowledged that the company didn’t do “enough to help prevent our platform from being used to foment division and incite offline violence. We agree that we can and should do more.”

Yet five years later, another case now before Kenya’s high court deals with the same issue on a different continent. Last year, Meta was sued by a group of petitioners including the family of Meareg Amare Abrha, an Ethiopian chemistry professor who was assassinated in 2021 after people used Facebook to orchestrate his killing. Amare’s son tried desperately to get the company to take down the posts calling for his father’s head, to no avail. He is now part of the suit that accuses Meta of amplifying hateful and malicious content during the conflict in Tigray, including the posts that called for Amare’s killing.

The case underlines the strange distance between Big Tech behemoths and the content moderation industry that they’ve created offshore, where the stakes of moderation decisions can be life or death. Paul Barrett, the deputy director of the Center for Business and Human Rights at New York University's Stern School of Business who authored a seminal 2020 report on the issue, believes this distance helped corporate leadership preserve their image of a shiny, frictionless world of tech. Social media was meant to be about abundant free speech, connecting with friends and posting pictures from happy hour — not street riots or civil war or child abuse.

“This is a very nitty gritty thing, sifting through content and making decisions,” Barrett told me. “They don't really want to touch it or be in proximity to it. So holding this whole thing at arm’s length as a psychological or corporate culture matter is also part of this picture.”

Sarah T. Roberts likened content moderation to “a dirty little secret. It’s been something that people in positions of power within the companies wish could just go away,” Roberts said. This reluctance to deal with the messy realities of human behavior online is evident today, even in statements from leading figures in the industry. For example, with the July launch of Threads, Meta’s new Twitter-like social platform, in July, Instagram head Adam Mosseri expressed a desire to keep “politics and hard news” off the platform.

The decision to outsource content moderation meant that this part of what happened on social media platforms would “be treated at arm’s length and without that type of oversight and scrutiny that it needs,” Barrett said. But the decision had collateral damage. In pursuit of mass scale, Meta and its counterparts created a system that produces an impossible amount of material to oversee. By some estimates, three million items of content are reported on Facebook alone on a daily basis. And despite what some of Silicon Valley’s other biggest names tell us, artificial intelligence systems are insufficient moderators. So it falls on real people to do the work.

One morning in late July, James Oyange, a former tech worker, took me on a driving tour of Nairobi’s content moderation hubs. Oyange, who goes by Mojez, is lanky and gregarious, quick to offer a high five and a custom-made quip. We pulled up outside a high-rise building in Westlands, a bustling central neighborhood near Nairobi’s business district. Mojez pointed up to the sixth floor: Majorel’s local office, where he worked for nine months, until he was let go.

He spent much of his year in this building. Pay was bad and hours were long, and it wasn’t the customer service job he’d expected when he first signed on — this is something he brought up with managers early on. But the 26-year-old grew to feel a sense of duty about the work. He saw the job as the online version of a first responder — an essential worker in the social media era, cleaning up hazardous waste on the internet. But being the first to the scene of the digital wreckage changed Mojez, too — the way he looks, the way he sleeps, and even his life’s direction.

That morning, as we sipped coffee in a trendy, high-ceilinged cafe in Westlands, I asked how he’s holding it together. “Compared to some of the other moderators I talked to, you seem like you’re doing okay,” I remarked. “Are you?”

His days often started bleary-eyed. When insomnia got the best of him, he would force himself to go running under the pitch-black sky, circling his neighborhood for 30 minutes and then stretching in his room as the darkness lifted. At dawn, he would ride the bus to work, snaking through Nairobi’s famously congested roads until he arrived at Majorel’s offices. A food market down the street offered some moments of relief from the daily grind. Mojez would steal away there for a snack or lunch. His vendor of choice doled out tortillas stuffed with sausage. He was often so exhausted by the end of the day that he nodded off on the bus ride home.

And then, in April 2023, Majorel told him that his contract wouldn’t be renewed.

It was a blow. Mojez walked into the meeting fantasizing about a promotion. He left without a job. He believes he was blacklisted by company management for speaking up about moderators’ low pay and working conditions.

A few weeks later, an old colleague put him in touch with Foxglove, a U.K.-based legal nonprofit supporting the lawsuit currently in mediation against Meta. The organization also helped organize the May meeting in which more than 150 African content moderators across platforms voted to unionize.

At the event, Mojez was stunned by the universality of the challenges facing moderators working elsewhere. He realized: “This is not a Mojez issue. These are 150 people across all social media companies. This is a major issue that is affecting a lot of people.” After that, despite being unemployed, he was all in on the union drive. Mojez, who studied international relations in college, hopes to do policy work on tech and data protection someday. But right now his goal is to see the effort through, all the way to the union’s registry with Kenya’s labor department.

Mojez’s friend in the Big Tech fight, Wabe, also went to the May meeting. Over lunch one afternoon in Nairobi in July, he described what it was like to open up about his experiences  publicly for the first time. “I was happy,” he told me. “I realized I was not alone.” This awareness has made him more confident about fighting “to make sure that the content moderators in Africa are treated like humans, not trash,” he explained. He then pulled up a pant leg and pointed to a mark on his calf, a scar from when he was imprisoned and tortured in Ethiopia. The companies, he said, “think that you are weak. They don’t know who you are, what you went through.”

Looking at Kenya’s economic woes, you can see why these jobs were so alluring. My visit to Nairobi coincided with a string of July protests that paralyzed the city. The day I flew in, it was unclear if I would be able to make it from the airport to my hotel — roads, businesses and public transit were threatening to shut down in anticipation of the unrest. The demonstrations, which have been bubbling up every so often since last March, came in response to steep new tax hikes, but they were also about the broader state of Kenya’s faltering economy — soaring food and gas prices and a youth unemployment crisis, some of the same forces that drive throngs of young workers to work for outsourcing companies and keep them there.

Leah Kimathi, a co-founder of the Kenyan nonprofit Council for Responsible Social Media, believes Meta’s legal defense in the labor case brought by the moderators betrays Big Tech’s neo-colonial approach to business in Kenya. When the petitioners first filed suit, Meta tried to absolve itself by claiming that it could not be brought to trial in Kenya, since it has no physical offices there and did not directly employ the moderators, who were instead working for Sama, not Meta. But a Kenyan labor court saw it differently, ruling in June that Meta — not Sama — was the moderators’ primary employer and the case against the company could move forward.

“So you can come here, roll out your product in a very exploitative way, disregarding our laws, and we cannot hold you accountable,” Kimathi said of legal Meta’s argument. “Because guess what? I am above your laws. That was the exact colonial logic.”

Kimathi continued: “For us, sitting in the Global South, but also in Africa, we’re looking at this from a historical perspective. Energetic young Africans are being targeted for content moderation and they come out of it maimed for life. This is reminiscent of slavery. It’s just now we’ve moved from the farms to offices.”

As Kimathi sees it, the multinational tech firms and their outsourcing partners made one big, potentially fatal miscalculation when they set up shop in Kenya: They didn’t anticipate a workers’ revolt. If they had considered the country’s history, perhaps they would have seen the writing of the African Content Moderator’s Union on the wall.

Kenya has a rich history of worker organizing in resistance to the colonial state. The labor movement was “a critical pillar of the anti-colonial struggle,” Kimathi explained to me. She and other critics of Big Tech’s operations in Kenya see a line that leads from colonial-era labor exploitation and worker organizing to the present day. A workers’ backlash was a critical part of that resistance — and one the Big Tech platforms and their outsourcers may have overlooked when they decided to do business in the country.

“They thought that they would come in and establish this very exploitative industry and Kenyans wouldn’t push back,” she said. Instead, they sued.

What happens if the workers actually win?

Foxglove, the nonprofit supporting the moderators’ legal challenge against Meta, writes that the outcome of the case could disrupt the global content moderation outsourcing model. If the court finds that Meta is the “‘true employer’ of their content moderators in the eyes of the law,” Foxglove argues, “then they cannot hide behind middlemen like Sama or Majorel. It will be their responsibility, at last, to value and protect the workers who protect social media — and who have made tech executives their billions.”

But there is still a long road ahead, for the moderators themselves and for the kinds of changes to the global moderation industry that they are hoping to achieve.

In Kenya, the workers involved in the lawsuit and union face practical challenges. Some, like Mojez, are unemployed and running out of money. Others are migrant workers from elsewhere on the continent who may not be able to stay in Kenya for the duration of the lawsuit or union fight.

The Moderator’s Union is not yet registered with Kenya’s labor office, but if it becomes official, its members intend to push for better conditions for moderators working across platforms in Kenya, including higher salaries and more psychological support for the trauma endured on the job. And their ambitions extend far beyond Kenya. The network hopes to inspire similar actions in other countries’ content moderation hubs. According to Martha Dark, Foxglove’s co-founder and director, the industry’s working conditions have spawned a cross-border, cross-company organizing effort, drawing employees from Africa, Europe and the U.S.

“There are content moderators that are coming together from Poland, America, Kenya, and Germany talking about what the challenges are that they experience when trying to organize in the context of working for Big Tech companies like Facebook and TikTok,” she explained.

Still, there are big questions that might hinge on the litigation’s ability to transform the moderation industry. “It would be good if outsourced content reviewers earned better pay and were better treated,” NYU’s Paul Barrett told me. “But that doesn't get at the issue that the mother companies here, whether it’s Meta or anybody else, is not hiring these people, is not directly training these people and is not directly supervising these people.” Even if the Kenyan workers are victorious in their lawsuit against Meta, and the company is stung in court, “litigation is still litigation,” Barrett explained. “It’s not the restructuring of an industry.”

So what would truly reform the moderation industry’s core problem? For Barrett, the industry will only see meaningful change if companies can bring “more, if not all of this function in-house.”

But Sarah T. Roberts, who interviewed workers from Silicon Valley to the Philippines for her book on the global moderation industry, believes collective bargaining is the only pathway forward for changing the conditions of the work. She dedicated the end of her book to the promise of organized labor.

“The only hope is for workers to push back,” she told me. “At some point, people get pushed too far. And the ownership class always underestimates it. Why does Big Tech want everything to be computational in content moderation? Because AI tools don’t go on strike. They don't talk to reporters.”

Artificial intelligence is part of the content moderation industry, but it will probably never be capable of replacing human moderators altogether. What we do know is that AI models will continue to rely on human beings to train and oversee their data sets — a reality Sama’s CEO recently acknowledged. For now and the foreseeable future, there will still be people behind the screen, fueling the engines of the world’s biggest tech platforms. But because of people like Wabe and Mojez and Kauna, their work is becoming more visible to the rest of us.

While writing this piece, I kept returning to one scene from my trip to Nairobi that powerfully drove home the raw humanity at the base of this entire industry, powering the whole system, as much as the tech scions might like to pretend otherwise. I was in the food court of a mall, sitting with Malgwi and Wabe. They were both dressed sharply, like they were on break from the office: Malgwi in a trim pink dress and a blazer, Wabe in leather boots and a peacoat. But instead, they were just talking about how work ruined them.

At one point in the conversation, Wabe told me he was willing to show me a few examples of violent videos he snuck out while working for Sama and later shared with his attorney. If I wanted to understand “exactly what we see and moderate on the platform,” Wabe explained, the opportunity was right in front of me. All I had to do was say yes.

I hesitated. I was genuinely curious. A part of me wanted to know, wanted to see first-hand what he had to deal with for more than a year. But I’m sensitive, maybe a little breakable. A lifelong insomniac. Could I handle seeing this stuff? Would I ever sleep again?

It was a decision I didn’t have to make. Malgwi intervened. “Don’t send it to her,” she told Wabe. “It will traumatize her.”

So much of this story, I realized, came down to this minute-long exchange. I didn’t want to see the videos because I was afraid of how they might affect me. Malgwi made sure I didn’t have to. She already knew what was on the other side of the screen.

Related stories

Meta cozies up to Vietnam, censorship demands and all

Dien Nguyen An Luong

The Albanian town that TikTok emptied

Isobel Cockerell

How TikTok influencers exploit ethnic divisions in Ethiopia

Endalkachew Chala

The post Silicon Savanna: The workers taking on Africa’s digital sweatshops appeared first on Coda Story.

Lundy thought the ad could educate social media users about the risk factors for breast cancer, but it never saw the light of day. Instead, Instagram rejected it for violating its policies on nudity and sexual activity.

For more than a year, Lundy’s company, Complex Creatures, has struggled to find a home for its content on Instagram. The platform has rejected scores of the company’s advertisements and posts since its account went live in June 2022. Lundy co-founded Complex Creatures with her sister, a breast cancer survivor, to raise awareness about the disease and provide health and wellness products for women undergoing breast cancer treatment. But the content rejections came rolling in as soon as she started posting. It didn’t take long for Lundy to realize that Meta, owner of Instagram, was nixing her content because of its subject matter: the breast. 

“How do you desexualize the breast?” she asked. “It’s so much of what we’re trying to do.” But platforms like Instagram, Lundy said, “don’t want to let us.” In a call over Zoom, she shared some screenshots of her company’s censored content. One was a post about how massages can improve breast health, featuring a photo of a woman’s hands fully covering her breasts. “But they’re allowed to do this,” she sighed, pulling up an advertisement from a men’s health brand for an erectile dysfunction treatment containing an image of a hand clutching an eggplant with the caption: “Get hard.” The censorship, she added, “is an ongoing challenge. We’re talking about breast cancer and breast health.” Access to the right information about the disease and its risk factors, she explained, can be a matter of “life and death.”

The censorship that Lundy routinely confronts on Instagram is part of a deeper history at Meta, which has long faced criticism for censoring material about breasts on Facebook. But it’s not just breast-focused content that’s not getting through. Lundy belongs to a community of nonprofits and startups focused on women’s health that face routine — and often bewildering — censorship across Facebook and Instagram. 

I spoke with representatives from six organizations focused on women’s health care globally, and they told me that while Meta regularly approves advertisements for material that promotes men's sexuality and sexual pleasure, it regularly blocks them from publishing advertisements and posts about a wide range of health and reproductive services aimed at women, including reproductive health, fertility treatments and breast care. Often, these posts are rejected on the grounds that they violate the company's advertising policies on promoting sexual pleasure and adult content.

This kind of censorship comes at an existential moment for the U.S.-based reproductive rights community after the Supreme Court’s overturning of Roe v. Wade — the nearly 50-year-old ruling that legalized abortion across the U.S. — in 2022. As I reported in March 2023, abortion opponents have sought to clamp down on abortion speech online in the post-Roe era, introducing policies in Texas, Iowa, and South Carolina that would prohibit websites from publishing information about abortion. That’s on top of censorship that reproductive rights groups already face when they try to post content about accessing abortion care on platforms like Instagram and Facebook — even in countries where the procedure is legal. 

According to Emma Clark Gratton, a communications officer for the Australia chapter of the international reproductive health nonprofit MSI Reproductive Choices, the organization is routinely blocked from running ads about abortion services on Facebook, often for violating the company’s advertising policy on social issues, elections, and politics. Abortion is “totally legal” in Australia, Clark Gratton explained, but on Meta’s platforms, it is “still very restricted in terms of what we can post.” The organization’s clinical team in Australia, she added, can advertise for vasectomy services on Facebook, “but they definitely couldn’t do an ad promoting abortion services, which is literally what they do. They’re an abortion provider.”

Women First Digital, a group that provides information resources about abortion globally, has dealt extensively with restrictions on social media networks. Michell Mor, a digital strategy manager with the organization, put it to me this way: “Because big tech is from the United States, everything that happens there is replicated around the world.”

The impact of these restrictions reaches well beyond social media, says Carol Wersbe, chief of staff for the Center for Intimacy Justice, a nonprofit that has been tracking Meta’s rejections of health-related ads. 

“Advertising represents so much more than just a company getting an ad on Facebook,” Wersbe told me. “It's visibility, access to information. If we can't advertise for things like pelvic pain and endometriosis, how do we ever reduce the stigma from those topics?” 

In January 2022, the Center for Intimacy Justice published a survey of 60 women’s healthcare startups about their experiences with censorship on Facebook and Instagram. The participating companies offer products and services for a range of women’s healthcare needs, from fertility and pregnancy support to postpartum recovery, menstrual health, and menopause relief. All of the companies surveyed reported having their ads rejected by Instagram and Facebook, and half said their accounts were suspended after Meta removed their ads. According to the report, ads were frequently taken down after they were flagged for promoting “adult products and services,” which are not permitted under the company’s advertising policies.  

Some ads that didn’t make the cut featured products to relieve side effects of menopause; another included background about consent in school sexual education courses. During the same time period, the report points out, Meta approved ads for men’s sexual health products, including treatments for premature ejaculation, erectile dysfunction pills promising to help consumers “get hard or your money back” and men’s lubricants to “level up your solo time.” The platform allowed these ads despite its own rules prohibiting ads from promoting products and services that “focus on sexual pleasure.”

Meta quietly updated its advertising guidelines after the report came out, stating that ads for family planning, contraception, menopause relief, and reproductive health care are allowed. Though the social media giant expanded the scope of permissible advertisements on paper, Wersbe says the status quo remains unchanged. “Across the board, we're still seeing our partners experiencing rejections,” she explained. The censorship that she and others in the field are observing cuts across languages, markets, and continents. “Facebook’s ads policy is a global policy, so when it changes something it affects their whole user base,” explained Wersbe. “We’ve seen rejections in Arabic, Spanish, French, Swedish, Swahili. It’s really pervasive.”

In March 2023, the organization filed a complaint with the U.S. Federal Trade Commission, urging the agency to investigate whether Meta is engaging in deceptive trade practices by rejecting ads from women’s health organizations that comply with its stated advertising policies, while allowing similar advertisements promoting men’s sexual health. The complaint alleges that the social media giant is unevenly applying its ads rules based on the gender of the target audience. These removals, it argues, constitute discriminatory censorship and perpetuate “inequality of access to health information and services for women and people of underrepresented genders.” 

In reporting this story, I contacted Meta with questions about the Center for Intimacy Justice’s report, the Federal Trade Commission complaint, and the rejection of Lundy’s advertisements. A spokesperson responded and shared the company’s published Community Standards, but declined to comment on the record.

Alexandra Lundqvist told me that alongside the outreach challenges that these issues create, ad rejections also make it harder for women-led health companies to get a leg up among investors. Lundqvist is a communications lead with The Case for Her, an investment firm that funds women’s sexual health organizations worldwide, including the Center for Intimacy Justice. “The general Silicon Valley big tech investor is not going to go to a women’s health company, especially when they can’t really advertise their work because they get blocked all the time. When these companies can’t advertise their work, they can’t scale, they can’t get funding,” Lundqvist explained. That exacerbates inequities that women and nonbinary entrepreneurs already face in securing investments from the male-dominated venture capital industry, creating a negative feedback loop for companies marketing products by and for women. “There is a big systems impact,” she added.

Lundy, who says her breast health company continues to experience widespread rejections despite Meta’s policy update, believes the censorship has a corrosive effect on consumers and creators alike. The content takedowns make it harder for entrepreneurs like herself to reach customers, make money, and attract investors. But they also prevent people from learning potentially life-saving information about breast cancer.

“There’s not a lot of information out there about breast health,” she said, describing her own lack of awareness about the disease prior to her sister’s diagnosis at age 37. “We had no family history,” she told me. “Her gynecologist missed it and she had never had a mammogram.” The experience, she continued, “really illuminated how much we didn’t know about our breasts.”

Lundy and her sister founded the company in part to address the information vacuum that left them both in the dark — to reach people before diagnosis and support those with the disease through treatment. But Meta makes that mission harder. “We want to normalize the breast,” she said, “but it’s almost like the algorithm and the people making the algorithms can’t think about a breast or a woman’s body in any way other than sexuality or arousal.” The censorship that Complex Creature routinely faces for posting material on Instagram about breast health, Lundy told me, “feels like the patriarchal system at work.”

The morning after our call, Lundy emailed me an update: a photo of two squashes meant to resemble breasts hanging side by side — the visual for an Instagram ad about her company’s summer sale. The post, she wrote, “was rejected last night. They’re gourds.”

Related Articles

When Meta suspends influential political accounts, who loses?

Fiona Kelliher

Texas lawmakers want to erase abortion from the internet

Erica Hellerstein

What a law designed to protect the internet has to do with abortion

Erica Hellerstein

The post Advertising erectile dysfunction pills? No problem. Breast health? Try again appeared first on Coda Story.

The speech came back to haunt him on June 29, when Meta’s Oversight Board recommended that the company suspend the prime minister for six months for breaking the platform’s rules against threatening or inciting violence.

Later that day, Hun Sen beat the company to the punch and deleted his own page. It was a stunning move in Cambodia, where the prime minister has used the platform to trumpet his policy positions and lash out at his opponents to the nearly 14 million followers he has amassed since joining Facebook in 2015.

Some of his posts have had immediate real-world consequences. In February 2023, the forced closure of one of Cambodia’s last independent news outlets, Voice of Democracy, played out entirely on Hun Sen’s Facebook over two days. Angered by an article he claimed was erroneous, Hun Sen threatened in a post to revoke VOD’s license if the outlet didn’t apologize promptly.

After VOD expressed “regret” for any confusion the story caused, Hun Sen responded via Facebook that the statement was insufficient and said that the Cambodian Ministry of Information would revoke the outlet’s license.

“Is it acceptable to use words of ‘regret’ and ‘forgiveness’ instead of the word ‘apologize?’ For me, I cannot accept it,” Hun Sen wrote in the post. “Look for jobs elsewhere,” he added. Police and ministry officials arrived at VOD’s office the next morning with an order to cease publishing.

But now the future of Hun Sen’s page is uncertain. A few weeks after he deleted his account, his assistant reinstated it, ahead of the national elections. And Meta, which owns Facebook, has yet to officially decide whether to follow the recommendation of its Oversight Board and proceed with the six-month suspension. This means that the account could go offline again — and take with it a digital archive attesting to the more recent chapters of Hun Sen’s 38-year regime.

“Facebook was the key, important way for him to communicate his political messages to his audience and fans,” said Sokphea Young, a Cambodian research fellow at University College London who has studied the visual messaging of Hun Sen’s Facebook page. “However many people don’t like Prime Minister Hun Sen, the account is very important for the collective memory of Cambodian people and Cambodian history.”

And Cambodia is hardly alone in this. Around the world, speech coming from government officials has increasingly spilled over onto social media platforms. But companies like Meta and Twitter can decide to remove posts or entire accounts at any moment, regardless of how this might affect public access to information about state actors and institutions. Neither company has a policy on archiving state accounts, and, with a few exceptions, states don’t require companies to do this either.

In the mid-1990s, libraries, universities and governments around the world became concerned about losing electronic records to the fast-evolving digital sphere. But archiving from social media platforms has remained an “unloved” area of public policy, even as more and more government data has landed there, said William Kilbride, the executive director of the U.K.-based Digital Preservation Coalition, an advocacy group that works with public and private institutions around the world on archiving. 

Some governments with robust archiving capabilities deal with social media platforms on an individual basis to maintain records. The U.S. National Archives and Records Administration, for instance, has worked with Twitter to “freeze” previous versions of accounts linked to the presidency on the original platform. The U.K.’s National Archives maintains a social media database with Twitter and YouTube archives.

But major platforms have not created broader global policies around such programs, and they aren’t always transparent about how long they internally retain deleted or suspended accounts or those of deceased people. There are also technical challenges: Meta actively works to prevent scraping, a technique that archivists use to gather and then preserve such data. Finding automated ways to capture pages’ full context — such as comments on posts — is also “really difficult,” Kilbride said. 

Even within existing archival relationships, platforms still have the upper hand. After the January 6, 2021 riots in Washington, D.C., Twitter announced it would not allow a federally preserved version of former U.S. President Donald Trump’s @realDonaldTrump tweets — which the National Archives had been working to preserve — to appear on the site after it “permanently banned” him, as would be typical with other accounts linked to the presidency.

But after Elon Musk bought the company, the account was reinstated on Twitter. In an email exchange, the National Archives would not confirm whether its prior efforts to preserve @realDonaldTrump are ongoing but said to “continue to check back for addition[al] content as it is added in the future.” The handle does not appear alongside other accounts the agency has made separately available on its website.

“The public record has been privatized and now sits on these platforms,” Kilbride said. “Suddenly, it’s the National Archives’ or whoever’s job to try to figure out what on earth to do.”

“They have no duty of transparency,” Kilbride added of the platforms. “There’s no accountability.”

Although most governments have national archiving laws, many lack the resources to enforce them on social media or store mass amounts of data on independent servers, putting them at a further disadvantage in preserving material when accounts — or an entire platform — suddenly go dark.

In June 2021, former Nigerian President Muhammadu Buhari received a 12-hour suspension on Twitter over a tweet targeting Igbo people, one of the biggest ethnic groups in the country, writing he would “treat them in the language they understand.” Two days later, he blocked access to Twitter countrywide, making his own and other government-related accounts inaccessible within Nigeria.

‘Gbenga Sesan, the executive director of the digital rights group Paradigm Initiative Nigeria, said that in his circles, “nobody cared” at the time about preserving Buhari’s account locally: He was more focused on the thousands of requests for help accessing virtual private networks (VPNs) pouring in from across Nigeria. Plus, digital experts knew that a local block on Twitter didn’t mean the accounts had been lost, he said. 

In fact, as Nigerians continued to access Twitter with VPNs, Buhari’s account — mainly a place to share propaganda and party information — was little-missed for the roughly six months that Twitter was officially blocked. “I don't ever remember going there to check what was said. A few times, I tweeted that silence was a better option for Buhari, because every time he speaks, the country gets angry,” Sesan said.

Still, Sesan wants to see social media platforms create archiving partnerships with governments on a global scale. But the Buhari episode also showed the need for a more expansive view of preservation: On its own, Buhari’s account would provide a slim portrait of Nigeria’s online history at the time. And what’s more, governmental partnerships would only work if both sides had mutual good will to preserve materials.

“You’ll find the digital aides sharing more historical facts than the president himself,” Sesan said.  “That, I think, is the major context when it comes to presidential archives and information: That kind of information also matters.”

Challenges with archiving also arise when it comes to posts that shine light on human rights abuses, war crimes and other atrocities that demand documentation in the service of future legal investigations and historical inquiry, particularly on Meta and YouTube. Last month, Meta’s Oversight Board called on the company to publicly address archiving practices in a decision about a video of Armenian prisoners of war. The video showed the faces of injured and deceased soldiers, raising questions about revealing the identities and locations of prisoners in conflict zones. Although the board agreed Meta was correct to leave up the content with a warning screen, it recommended the company commit to preserving evidence of atrocities, develop public protocols for preservation and explain how long it internally retains data and considers preservation requests. Meta has not yet responded publicly and did not respond to a request for comment.

In recent years, social media platforms have faced scrutiny for helping to spread hate speech and disinformation in places such as Myanmar, Kenya and India, making the platforms eager to appear quick to remove content or accounts spewing violent rhetoric. 

While deplatforming violent actors can be crucial to limiting offline violence, digital historians and researchers say it also causes public records to disappear from the internet before they have the chance to collect them.

The nonprofit Mnemonic grew out of the civil war in Syria and maintains four archives documenting evidence of potential human rights violations in Syria, Sudan, Yemen and Ukraine. As social media platforms have increased their use of automated tools that try to remove harmful content, the group has seen human rights-related material taken down more frequently, according to Maria Mingo, a policy and advocacy manager at Mnemonic. 

The organization stores its archives on independent servers. About one-quarter of the two million YouTube videos it has archived from Syria since 2014 have disappeared from YouTube itself. About one-tenth of the 2,000 Twitter accounts from the same archive have been removed during the same period.

In May 2023, Musk announced that Twitter would begin removing and archiving “inactive” accounts. Mingo said that rule could present a “huge problem” for jailed activists whose accounts, and the information they collected at great personal risk, suddenly disappeared.

“If the content is taken down so, so quickly — unless platforms preserve and are able to engage with relevant stakeholders about the existence of the content — we won’t be able to do anything with it, we won’t be able to request it, we won’t be able to in any way try to use it,” Mingo said. 

“We can’t preserve something that no longer exists, or that we don’t know exists,” she added.

In Cambodia, questions around preservation and collective memory have persisted for decades. During the 1970s, the Khmer Rouge genocide wiped out nearly one-quarter of the population. Images of torture, starvation and detention have become irreplaceable to “memorialize how things went wrong in that period,” said researcher Young.

Right now, Hun Sen’s Facebook page provides an unmatched record of the current regime, replete with personal exchanges between Hun Sen and his followers. But it also has a more reflective style. He has long favored posting black-and-white or old photographs of himself or family members, dating back to shortly after the Khmer Rouge era when he came to power. In a recent post after the account was reinstated, he shared an undated photograph of himself as a young man walking in a green, placid background, along with a message about the upcoming one-sided national election, which his party won in a landslide.

“Today is the last day of the party campaign, and also the day of great expectations for the Cambodian People’s Party in the upcoming election. I wish you all, the family of the ‘angel party,’ success countrywide,” the caption read, referring to the ruling party’s logo.

Such photographs have been central to the page for years, according to Young, who, over four years, has tracked the prime minister’s habit of contrasting black-and-white and color photos as a visual representation of his mythical political journey.

The idea is to show himself bringing Cambodia out of the darkness and into the light, a human representation of peace protecting the country from plunging back into civil war.

“As a copy of the history of Cambodia, maybe [his account] should be in a museum somewhere, in the next 40, 50 years, so the new generation can see,” Young said. “This is what Facebook was like, and this is what the prime minister was like, during the new era of digitalization.”

Related Articles

The occupational hazards of cleaning the internet

Erica Hellerstein

In the Khmer Rouge’s last stronghold, myths from the Cambodian genocide still reign

Fiona Kelliher

Cambodian prime minister unleashes social media crackdown to seal grip on country

Chaewon Chung

The post When Meta suspends influential political accounts, who loses? appeared first on Coda Story.

In early February, three archbishops of the Orthodox Tewahedo Church — one of the oldest churches in Africa that dates back to the 4th century — accused fellow church leaders of discriminating against the Oromo people, who constitute the largest ethnic group in Ethiopia’s population of 120 million. While church members come from a diverse array of ethnic backgrounds, worship services are predominantly conducted in the liturgical language of Ge’ez and in Amharic, which is a language primarily spoken by the Amhara people. Amharic is the dominant language of Addis Ababa, Ethiopia’s capital, and the working language of the federal government. This linguistic predilection underlines the cultural clout of Amharic. 

After the three archbishops — all of Oromo lineage — made their allegations of discrimination public, they were excommunicated by church authorities. They then declared their plans to form a breakaway synod, triggering an instant public outcry. The cleavages underlying Ethiopia’s civil conflict bubbled to the surface and devolved into violent skirmishes, resulting in a combined total of 30 fatalities in the southern Ethiopian town of Shashemene and in Addis Ababa.

But what was a serious political crisis for the church and for the country amounted to a prime opportunity for TikTok influencers seeking to spread their messages and turn a profit along the way.

A quick scroll through live sessions on TikTok reveals heated political discussions in Amharic, Oromo and Tigrinya, in which participants exchange barbs and strategize on how to confront their adversaries. Zemedkun Bekele is prominent among them. A self-proclaimed defender of the Orthodox Tewahedo Church, he is known for his forceful, admonitory videos that are often over an hour long. Bekele began broadcasting threats against the breakaway synod, claiming to have video evidence that its leaders had engaged in homosexual activity and threatening to release the tape to the public. Accusations like this resonate deeply in a nation steeped in conservatism, where homosexuality is viewed with considerable disdain.

A known social media influencer who had already been banned from both Facebook and YouTube in 2020 for violating their policies on hate speech and the promotion of violence, Bekele re-established himself on TikTok in February 2023, just in time to jump into the fray. Since then, he has amassed a dedicated audience of more than 203,000 TikTok followers, most of whom appear to be members of the Amhara ethnic group and followers of the Orthodox Tewahedo Church. 

In the midst of the crisis, Bekele also launched attacks against a senior church teacher, Daniel Kibret, who has become a staunch ally of Prime Minister Abiy Ahmed. Drawing on the fact that the prime minister comes from a mixed religious background (his father is Muslim and his mother is Christian), Bekele made unfounded claims that Kibret had secretly converted to Islam.

In a video with more than 19,000 views, Bekele maintained that he would not relent. “We will not back down without making sure the Ethiopian Orthodox Church is as big as the country itself. We will not back down without toppling Abiy Ahmed,” he said. “We will not back down without hanging Daniel Kibret upside down.” The video was posted on February 4, the same day that the three bishops declared their intentions to secede from the Church.

Another account called TegOromo also saw swift growth surrounding the church controversy. TegOromo has a passionate following and is on the opposite side of the conflict from Bekele. The person who runs the account has expressed support for the Oromo religious leaders who sought to establish the independent synod. The account's moniker fuses the first three letters of “Tegaru” with “Oromo” — a calculated move to represent harmony between the Tigrayan and Oromo ethnic groups.

With more than 60,000 followers, TegOromo’s account is marked by overt threats, inflammatory language and aggressive rhetoric. One TikTok video urged supporters to “chop the Amharas like an onion.” This video was later removed from the platform, but copies of it remain accessible. In a live session, a TegOromo follower called on Oromo people to “kill all Amharas” and even specified that children should not be spared. TegOromo cheered him on, urging other followers to answer the call and take up arms.  

Despite the controversial nature of TegOromo’s content, the influencer's popularity suggests a burgeoning trend. Republishing his material or circulating incisive and satirical clips featuring TegOromo has become a reliable strategy for Ethiopian content creators seeking higher engagement.

In another instance, the spotlight turned toward two emerging TikTok influencers, Dalacha and Betayoo, who garnered attention for their adept use of vitriol. In one video, Dalacha, who identifies as Oromo, launched a barrage of insults and sexual slurs at his rival TikToker, Maareg, who identifies as Amhara. The episode exemplified the depths to which Dalacha was willing to stoop in order to denigrate Maereg. Dalacha used language that reduced the Amhara community to mere cattle. It was intended only to amplify the prevailing animosity between the two ethnic groups.

In another video, Betayoo, who consistently identifies as Amhara, used similarly troubling language, employing both sexual and ethnic slurs. She directed her insults toward a rival TikToker who identifies as Tigrayan and who has publicly expressed disdain for the Amhara community. Betayoo's actions escalated beyond targeting an individual. She proceeded to insult the entire Tigrayan community, expressing a desire for their eradication. 

The videos I reference above also all contain clear violations of TikTok’s terms of service, yet they remain on the platform. TikTok's Community Guidelines strictly prohibit hate speech or hateful behavior and promise to remove such material from their platform. Accounts and/or users that engage in severe or multiple violations of hate speech policies are promptly banned from the platform. Despite these guidelines, plenty of Ethiopians who have exhibited hateful behavior remain active on the platform and continue to produce content for significant numbers of followers.

When I approached TikTok staff members to alert them about the videos and ask them to comment for this piece, they did not respond. It is difficult to definitively prove that this kind of discourse directly contributes to violence on the ground. But it is clear that discussions of political violence and religious conflicts on TikTok often result in the spread of misinformation and amplify interethnic hatred. Clips containing these influencers’ offensive remarks have also seeped onto other platforms, such as YouTube and Facebook, where reposting or critiquing such content has become a low-effort method for content creators to gain engagement.

Given the sheer volume of such live streams, TikTok's moderation team may be overwhelmed, struggling to monitor these discussions and remove inappropriate content. It is also worth noting that all of these accounts are run primarily in Amharic, Oromo or Tigrinya, languages that are spoken by millions of Ethiopians in and outside of the country but that have historically been underrepresented on major social media platforms. TikTok does not publicly disclose how many staff members or content moderators it employs for reviewing content in these languages.

All this engagement is not driven purely by political vitriol — it is also a pursuit of profit. The TikTok LIVE feature has seen a swift uptick in popularity among Ethiopian users, catalyzing an emergence of politically-minded influencers who reap economic rewards through virtual gifts. These gifts can be converted into TikTok “diamonds,” which are in turn redeemable for actual cash.

Crafting politically-charged clickbait, designed to fan the flames of ethnic and religious discord, is emerging as a common tactic for financial gain. It has had especially strong uptake among individuals in the Ethiopian diaspora. Many of the most impactful Ethiopian TikTok figures are actually located in Western nations. Zemedkun Bekele, for instance, lives in Germany.

Amid the ongoing crisis, Bekele proudly claimed to have received one of the most sought-after TikTok LIVE gifts — the lion, which translates to a little over $400 in real-world currency. He has prominently featured a video on his profile displaying a virtual lion roaring at the screen, serving as both a symbol of his influence and a testament to the economic gains that one can reap through this kind of engagement on TikTok.

In a 2021 essay, former New York Times media critic Ben Smith showed how TikTok's algorithmic recommendation framework has helped to intensify cultural, linguistic and ideological divides among its global user base. The unfolding situation in Ethiopia could serve as a case study for Smith’s argument. With the videos I described — in addition to hundreds of others — the platform's content dissemination strategy appears to inadvertently encourage distinct factions to isolate themselves and push each other to commit hate speech and even physical violence.

The rise of these online strongholds poses significant challenges to promoting inclusive, cross-cultural understanding that TikTok claims to want to foster. Users now risk becoming trapped in ideological echo chambers, detached from diverse perspectives and viewpoints, and increasingly vulnerable to politically-motivated disinformation.

At the core of the issue lies the question of accountability. What obligation does TikTok, and by extension other social media platforms, have to curtail the spread of divisive content, particularly when it is financially incentivized? And moreover, could the pursuit of profit from politically-charged content inadvertently pave the way for more extreme or hazardous content, potentially triggering threats of violence in real life?

In the end, for onlookers familiar with Ethiopian culture and politics, it is clear that the platforms that invite us to share our lives online are failing to mediate the complexities of the world they seek to engage with.

Related Articles

How Somali workers in the US are fighting Amazon’s surveillance machine

Erica Hellerstein

In India, a trans woman stands up to the ‘YouTube Baba’

Alishan Jafri

and

Vipul Kumar

Uganda is targeting reproductive rights alongside its ‘anti-gay’ bill

Prudence Nyamishana

The post How TikTok influencers exploit ethnic divisions in Ethiopia appeared first on Coda Story.

On April 6, Trixie found herself standing on a stage, before the shaven-headed, heavyset Baba himself. With cameras rolling, two men held her in place while the Baba, draped in long white robes, accused her of being infected with the “disease of queerness.” By posting videos like these on social media, the Baba has made a fortune in just three years. He claims to have a “godly” cure for terminal illnesses and a variety of other personal and psychological complaints. 

He also specializes in conversion therapy — in which he claims to “pray the gay away” — and offers a special prayer package to “reconvert” transgender people and align their gender identity to the sex they were assigned at birth. Trixie’s family paid about $1,830 for her “treatment,” a sizable sum in a country in which the average monthly wage is below $500.

The Baba’s promises to banish homosexuality and to “cure” transgender people appeal to longstanding popular prejudices in Uttar Pradesh and other parts of India. Even the federal government is currently arguing in the Indian Supreme Court that gay marriage is an “urban elitist concept.” For most of the country, the government insists as it attempts to put the brakes on the Supreme Court hearings that would determine whether India should legalize same-sex marriage, the “notion of marriage itself necessarily and inevitably presupposes a union between two persons of the opposite sex.” And this notion is “deeply embedded in religious and societal norms.”

From his estate, the Baba regularly livestreams his “healing” sessions to tens of thousands of viewers. His most popular videos on YouTube, where he has a verified channel, have surpassed one million views. He also commands an impressive following on Facebook, where he maintains multiple pages. His social media pages all link to the ashram’s website, which boasts testimonials from his patients, instructions for devotees and a market for the Baba’s health products.

The Baba — aka Santosh Singh Bhadauria — is what is known in India as a “godman,” a self-styled guru who has managed to persuade people that he possesses spiritual powers. Godmen are similar to televangelists, and their followers might once have been called holy rollers. As with televangelists, godmen are frequently found to be conmen, criminals and sex offenders. Bhadauria has been in trouble with the law for decades, accused of various crimes though yet to be convicted of any.

Last month, a doctor who challenged Bhadauria by calling out his theatrics as cheap quackery was allegedly assaulted at Bhadauria’s behest. Among the types of cures Bhadauria enacts in public spectacles on his estate, attended by thousands of followers, is the ritual “murder” of “Muslim ghosts” that he claims have possessed the bodies of Hindus. The Muslim ghosts are exorcised with a toy gun.

Trixie knew little about Bhadauria’s methods before agreeing to visit his ashram. She was just trying to keep her parents happy. On reaching the estate, she found that a recording of his exhortations was being broadcast to hundreds of devotees. People were screaming and crying as if they'd been possessed by a spirit, she said.

Uncomfortable with the atmosphere, Trixie tried to walk away but says she was physically restrained by the Bhadauria's security. The next morning, Bhadauria showed up in person. He addressed Trixie’s family directly. Homosexuality, he said, was a disease, and Trixie, as someone infected by it, was “filled to the brim with filth.”

Her mother stood beside her, silently watching as Bhadauria continued to rant. That was when Trixie realized, she told us, that she had lost her mother to the propaganda, a far more cruel betrayal than Bhadauria’s crude abuse. 

“Parents can be wrong sometimes too,” she told her mother in front of Bhadauria and the audience. They were the only words she would utter during the “therapy” session. Had she tried to argue her case, she told us, she would have felt “like a dog barking without reason.” 

Bhadauria’s conversion therapy is emblematic of the transphobia and homophobia of Indian society. This prejudice is seeing a resurgence as the Supreme Court prepares to rule on the question of same-sex marriage. Despite a long history of gender fluidity in Indian theology, mythology and culture, the Hindu nationalist government led by Prime Minister Narendra Modi argues that the push for LGBTQ rights is a result of increasing Western influence and the decline of so-called Indian values. 

The Indian judiciary, though, has taken a more liberal line. Consensual gay sex was decriminalized in India in 2018, when the Supreme Court overturned a colonial law. “History owes an apology to members of the community,” said one of the judges, “for the delay in ensuring their rights.” And last year, a state high court ordered the prohibition of conversion therapy, ruling that it constituted misconduct when performed by medical professionals. 

If the Supreme Court does legalize same-sex marriage, it would be yet another ruling that defies the values and beliefs of some of India’s most powerful political actors. While no judgment has yet been made, trolls have targeted the court on social media and its chief justice was attacked as “woke” even before he was appointed to the bench. Government officials and mainstream media personalities have also piled on, insisting that the judges would be undermining tradition and imposing their own values on the country. Religious representatives from all of India's major religions, in a rare show of togetherness, have teamed up to oppose the marriage equality petition before the Supreme Court.

Social activist Indrajeet, the founder of “Yes, We Exist,” a digital LGBTQ+ awareness initiative, told us in an interview that although the Indian right wing says same-sex marriage is a Western imposition, Indian conservatives are also taking their cues from the West. In the West, particularly in the United States, transphobia has become an endemic political hot-button topic and is similarly framed as an issue beloved by liberal elites rather than one of existing civil rights being unequally applied. 

The Indian government says it is for elected legislators to decide on the fate of same-sex marriage rather than for unelected judges. But the Indian constitution — as a government website helpfully points out — gives all Indians the right to equal treatment before the law.

It is to the courts that a transgender woman like Trixie has to turn to get redress for the ordeal she was forced to endure at Bhadauria’s ashram. “If you do not self-determine and do prayers to be a boy,” Bhadauria said to Trixie, “you'll become a girl and will get beaten by boys. Even if you marry a boy, he will beat you too.”

Turning to Trixie’s mother, Bhadauria said, “The only way left to cure him is by doing prayers. If he doesn't do it by himself, you should do this for him.” Bhadauria also insinuated that Trixie’s transition was sexual in nature, a perversion rather than a deeply felt identity. This vein of transphobia has been contested at length by scholarship on and by queer people. 

Pointing to the scholarship, though, is not always a helpful strategy when confronted by hate speech on social media and the socially permitted behavior of quacks like Bhadauria. Indrajeet, the founder of the LGBTQ+ awareness initiative, explained that social media sites have become key platforms for the likes of Bhadauria. Their brand of hate is easily spread on these platforms and enables them to attract new followers and expand their reach. It also allows them to monetize their polarizing content. 

Although many of these social media pages and channels are riddled with hate speech and discriminatory messages, platforms routinely fail to take action against violations of their own rules of conduct.

For instance, though Meta does not offer clear guidance about organic content promoting conversion therapy on Facebook, the company expressly prohibits advertisements offering such services. Google (YouTube’s parent company) prohibits the promotion of conversion therapy in its publisher policies. Both companies have a broad ban on the kinds of hate speech and discriminatory language that characterize Bhadauria’s content.

Bhadauria’s video of his encounter with Trixie was highly visible both on Facebook and YouTube for two weeks after their “therapy session.” It has since been removed from Facebook, but the video is still up on YouTube, where Bhadauria has 439,000 followers. Indrajeet and other activists we spoke with expressed concern that these videos, spread by spiritual leaders with significant social influence, could be used to justify physical attacks on queer people in the public eye.

Zainab Patel, a trans woman, activist and one of the petitioners in the pending marriage equality Supreme Court case, told us that Bhaduria’s attempt to “treat” Trixie is against Indian law. All forms of conversion therapy against queer people were banned in 2021 by the National Medical Commission of India which described such therapy as “professional misconduct,” following an order from the Madras high court which has jurisdiction in the southern state of Tamil Nadu. 

These therapies can take the form of pseudo-religious rituals but can also involve measures as extreme as “corrective rapes.” Independent research has proven that conversion therapy practices increase the risk of self-harm among queer people. It is why, Zainab says, it is essential that legal action is taken against self-appointed holy men like Bhadauria.

“After watching Trixie's video,” Zainab told us, “we can say that she has been subjected to humiliation, stigmatization and discrimination.” Trixie’s parents, Zainab added, “along with the spiritual person to whom she was taken, can both be punished under the Transgender Protection Act.” This also means that both Facebook and YouTube could be compelled, by a court order, to remove the material. But so far, most of the footage remains online, garnering thousands of views and untold advertising revenue.

Akkai Padmashali, a transgender rights activist, pointed out that while in other democracies the numbers of openly homosexual and bisexual legislators are growing, India’s LGBTQ community has no representation in parliament to stand up for their concerns. Instead, the court case on marriage equality has become an opportunity for politicians to grandstand on matters of religious tradition. "I believe that I am bound to follow constitutional morality,” Akkai told us, “and not any social construct, cultural or religious morality.” But, in India, that is an increasingly rare position.

For Trixie, her ordeal does have a silver lining — she has found her voice and an inner strength. She counts it as a small victory that the video of her conversion therapy is no longer on Facebook. Now, she says, she is ready to take on more transphobic propaganda on her Instagram, where she has found many new supporters and followers.

Related Articles

India reopens its Khalistan wounds

Alishan Jafri

The demolition of dissent in India

Tusha Mittal

Why India’s defamation laws are hurting its democracy

Ashish Khetan

The post In India, a trans woman stands up to the ‘YouTube Baba’ appeared first on Coda Story.

Maya Amerson is one of these people. Amerson began working as a content moderator for Reddit in 2018, reviewing violent and disturbing content and identifying posts that violated the company’s terms of use. After three years on the job, she began suffering from panic attacks and symptoms of post-traumatic stress disorder. She repeatedly sought support from her employer but didn’t get it. Now, she is taking Reddit to court.

Currently pending before the San Francisco Superior Court, Amerson’s lawsuit alleges that the Reddit management ignored her requests to move to a different position with less exposure to this kind of material, even after she returned from a 10-week medical leave following her PTSD diagnosis. The lawsuit also claims that Amerson’s supervisors belittled her after she came back to the office. This led to her resignation in September 2022. She filed the suit three months later. When we asked Reddit for its side of the story, we were told that we could attribute the following to a company spokesperson: “At Reddit, our employees' well-being is a top priority and we offer a robust set of resources for content moderators.”

The allegations in Amerson’s lawsuit are specific to Reddit, a social media giant valued at $6.6 billion that has garnered praise for its unique and decentralized approach to content moderation. But they tell a story that goes well beyond Reddit. A growing number of legal actions are taking aim at tech platforms over their treatment of content moderators and the psychological hazards of the work. These cases highlight the tension inherent in a job that cannot yet be automated and is routinely exported to low-wage contractors overseas.

They also raise an essential and unresolved set of questions: Is vicarious trauma inevitable in the work of content moderation? Or can the tech giants who hire scores of moderators around the world do more to meet the psychological needs of the workforce? If so, are any of the major platforms doing it right?

A brief history of the “front page of the internet”

Founded in 2005 as the aspirational “front page of the internet,” Reddit allows people to create discussion groups, known as subreddits, on topics that interest them. Each of the over 100,000 subreddits across the platform has its own ethos and culture. And each subreddit has its own set of rules that are enforced in addition to the company’s own content policy.

What makes all this work? At the heart of Reddit lies an army of thousands of volunteers who carry out the grueling work of content moderation. Known on the platform as mods, these volunteers oversee each subreddit, removing content and banning accounts that violate their established community rules and norms. Some of the platform’s largest subreddits, like r/politics, have more than 1,000 volunteer mods. The company also employs a paid staff of moderators, like Amerson, to ensure that the content posted on the site does not run afoul of the law or of the company’s own content policy.

Across Reddit, it is these volunteer mods who conduct most of the content moderation on the site. According to Reddit’s most recent transparency report, volunteer moderators were behind 59% of all content removals in 2021, while paid content moderators handled most other removals. 

Reddit characterizes its moderation approach as “akin to a democracy, wherein everyone has the ability to vote and self-organize, follow a set of common rules, and ultimately share some responsibility for how the platform works” and maintains that its bottom-up system “continues to be the most scalable solution we’ve seen to the challenges of moderating content online.” 

In doing this work, however, volunteer mods are providing essential quality control services for Reddit without getting paid a dime. A 2022 study from Northwestern University attempted to quantify the labor value of Reddit’s estimated 21,500 volunteer moderators and concluded that it is worth at least $3.4 million to the company annually. 

While the volunteer moderators do their work for free, many are unaware of the monetary value of the labor they perform for the company. “We wanted to introduce some transparency into this transaction to show that volunteer moderators are doing a very valuable and complex job for Reddit,” said Hanlin Li, the co-author of the report.

Volunteer moderators often describe their work as a “labor of love,” Li explained, but they also have been forthcoming about the need for more institutional support to combat hate speech and harassment. “Over the years we’ve seen a lot of protests by moderators against Reddit, essentially because they thought that they were not supported adequately to do this volunteer job,” she explained. 

This has been a longstanding tension at Reddit, which developed a reputation early on for taking a laissez-faire approach to content moderation and hosting subreddits rife with bigotry and hate speech. This came to a head with a misogynistic incel community that formed on the platform in 2013 and became the ground zero for the 2014 Gamergate online harassment campaign that targeted women in the video game industry. The following year, when Reddit’s former CEO Ellen Pao announced that the platform would ban five subreddits for harassment, she was inundated with online abuse. Shortly after, she resigned. The incel subreddit‌ wasn’t banned by the website until 2017.

Despite its unique reliance on a volunteer-driven content moderation model, the site has flown under the radar of researchers and reporters, generating less attention than other social networks. But the lawsuit could change this.

Lawsuits target tech giants over moderator trauma, mistreatment

Maya Amerson’s case against Reddit is a reminder that the company, despite its unique approach to moderation, is not immune to the same kinds of allegations of worker harm that have formed the basis of lawsuits against the world’s largest tech platforms. 

That includes TikTok, Facebook and YouTube. TikTok was sued by two former content moderators last spring: They alleged that the company failed to create a safe and supportive working environment as they reviewed disturbing material including videos of bestiality, necrophilia and violence against children. Facebook agreed to a $52-million settlement in 2020 to compensate former content moderators who claimed their work caused psychological harm. And YouTube recently settled a $4.3-million lawsuit filed by former moderators who said they developed mental health issues while on the job. 

These suits all focus on a relatively new labor force and speak to an industry-wide concern about the toll of moderators’ work. “Generally speaking, content moderation is kind of the new frontier,” Nicholas De Blouw, one of Amerson’s attorneys, told me. “In 1992, there wasn’t such a thing as content moderation. I think that’s the interesting aspect of these cases, is seeing how the law protects people that are in roles that never really existed.”

Many social media platforms have shifted away from employing moderators in-house and now outsource their jobs to third-party contractors. Vendors like Accenture, CPL and Majorel employ people all over the world, from India to Latvia to the Philippines, typically expecting fast turnaround times on content review but offering relatively low hourly wages in return. In lawsuits, employees allege they’ve been exploited, made to work in substandard conditions and denied adequate mental health support for the psychological effects of their work. 

After TIME magazine showed how moderators in Kenya — working for Sama, a subcontractor of Meta — were paid as little as $2.20 per hour to review violent material while operating in a “workplace culture characterized by trauma,” one worker took both Sama and Meta to court. Sama has since canceled its remaining contracts with Meta. A separate investigation by the Bureau of Investigative Journalism highlighted the plight of a “legion of traumatized” and underpaid Colombian moderators working under grueling conditions for a TikTok contractor. “You have to just work like a computer,” a moderator said of the job in Colombia. “Don’t say anything, don’t go to bed, don’t go to the restroom, don’t make a coffee, nothing.” 

Some social media researchers have suggested that eliminating the outsourced labor model could be a step toward improving conditions for workers. New York University professor Paul M. Barrett, who authored a 2020 report on outsourced content moderation, has called on companies to stop farming out the work and instead employ moderators in-house, where they would theoretically have more access to mental health support, supervision and proper training. 

But moving the job in-house alone won’t resolve all the alleged issues with this line of work, as Amerson’s lawsuit against Reddit makes clear. After all, she was not an outsourced moderator — she was a direct employee of the company. As Barrett explained, directly employing moderators does not guarantee that they will receive everything that employees in these roles need.

“There’s an irreducible aspect to this work that makes it hazardous,” he said. “If you’re doing this work, there’s a danger you’re going to run into very difficult, offensive, and unsettling content. You can’t really completely avoid that. The question is: How well-trained are you, how well-supervised are you, and are there options for you to get the kind of mental and emotional support that would seem to be common sense in connection with this kind of work?” 

Are any companies doing this well? Barrett said he was “unaware of a major platform doing content moderation well in terms of directly employing moderators who receive proper training, supervision and mental health support.”

What would incentivize platforms to implement these changes? While they could reform themselves, put an end to outsourced moderation and provide in-house employees with better benefits, compensation, oversight and training, there’s no reason to expect this to happen. Under Elon Musk’s leadership, Twitter has moved in the opposite direction, gutting content moderation teams. Layoffs at YouTube have left the company with just one employee overseeing global misinformation policy.

In an area that shows no appetite for self-regulation, actual oversight may be the only solution. Barrett has proposed enhancing the consumer protection authority of the Federal Trade Commission to regulate the social media industry. Such an expansion could give the FTC the authority to ask companies outright about their content moderation practices and to verify whether they live up to their commitments to consumers outlined in their terms of service and community standards guidelines, Barrett explained.

“They make promises just like other companies,” he said. “My argument is that the FTC could ask: So how many moderators do you employ? Do you have enough people to get this done? How do they interact with the automated system? And is all of that adequate, really, to fulfill the promises you make?”

A policy shift like this also would require Congressional action, which is hard to imagine in our gridlocked era of governance. But without major changes to the industry, it’s also hard to imagine lawsuits like Amerson’s — and the concerns that undergird them — going away anytime soon.

Related Articles

What a law designed to protect the internet has to do with abortion

Erica Hellerstein

Online harassment is on the rise — and Elon Musk’s Twitter takeover isn’t helping

Mariam Kiparoidze

Platforms like Twitch have tried to combat harassment against women. It’s not working

Katia Patin

The post The occupational hazards of cleaning the internet appeared first on Coda Story.

After the decision, red states moved quickly to ban or severely limit access to the procedure. This made the virtual sphere uniquely important for people seeking information about abortion, especially those living in states that have outlawed the procedure with little or no exceptions. 

Google searches for abortion medications increased by 70% the month following the court ruling. People flocked to social media platforms and websites with resources about where and how to end a pregnancy, pay for an abortion or seek help to obtain an abortion out of state. 

Despite state laws criminalizing abortion, these digital spaces are legally protected from liability for hosting this kind of content. That’s thanks to the landmark Section 230 of the 1996 Communications Decency Act, the 26 words that are often credited with creating today’s internet as we know it. Thanks to Section 230, websites of all kinds are protected from lawsuits over material that users might post on their platforms. This legal shield allows sites to host speech about all kinds of things that might be illegal — abortion included — without worrying about being sued.

But the future of 230 is on shaky ground. Next month, the U.S. Supreme Court will hear oral arguments on a case that challenges the scope of the landmark internet law. The Court’s decision could have sweeping consequences for digital speech about abortion and reproductive health in a post-Roe America. 

THE BACKGROUND

When armed ISIS assailants staged a series of attacks in central Paris in November 2015, an American college student named Nohemi Gonzalez was among the 130 people who lost their lives. Her family has since taken Google (the owner of YouTube) to court. Their lawyers argue that the tech giant aided and abetted terrorism by promoting YouTube videos featuring ISIS fighters and other material that could radicalize viewers and make them want to carry out attacks like the one that killed Nohemi. Central to the case is YouTube’s recommendation algorithm, which feeds users a never-ending stream of videos in an effort to keep them hooked. Independent research has shown that the algorithm tends to promote videos that are more “extreme” or shocking than what a person might have searched to begin with. Why? Because this kind of material is more likely to capture and sustain users’ attention.

Section 230 protects Google from legal liability for the videos it hosts on YouTube. But does it protect Google from legal liability for recommending videos that could inspire a person to join a terrorist group and commit murder? That is the central question of Gonzalez v. Google. If the Supreme Court decides that the legal shield of Section 230 does not apply to the recommendation engine, the outcome could affect all kinds of videos on the platform. Any video that could be illegal under state laws — like abortion-related content in the post-Roe era — could put the company at risk of legal liability and would probably cause Google to more proactively censor videos that might fall afoul of the law. This could end up making abortion and reproductive health-related information much harder to access online.

If this all sounds wonky and technical, that’s because it is. But the Court’s decision has the potential to “dramatically reshape the internet,” according to Eric Goldman, a professor at California’s Santa Clara University School of Law specializing in internet law. 

Algorithmic systems are deeply embedded in the architecture of online services. Among other things, websites and social media platforms use algorithms to recommend material to users in response to their online activity. These algorithmic recommendations are behind the personalized ads we see online, recommended videos and accounts to follow on social media sites and what pops up when we look at search engines. They create a user’s newsfeed on social media platforms like Facebook and Twitter. They have become a core feature of how the internet functions.

WHAT ARE THE STAKES IN A POST-ROE AMERICA?

If the Supreme Court rules in the plaintiffs’ favor, it could open up a vast world of possible  litigation, as websites and platforms move assertively to take down content that could put them at legal risk, including speech about abortion care and reproductive health. Platforms then would face the threat of litigation for recommending content that stands in violation of state laws  — including, in thirteen cases, laws against abortion. 

“That's going to dramatically affect [the] availability of abortion-related material because, at that point, anything that a service does that promotes or raises the profile of abortion-related material over other kinds of content would no longer be protected by Section 230, would be open for all these state criminal laws, and services simply can't tolerate that risk,” Goldman explained. 

In this scenario, technology companies could not only be exposed to lawsuits but could even find themselves at risk of criminal charges for algorithmically recommending content that runs afoul of state abortion bans. One example is Texas’ anti-abortion “bounty” law, SB 8, which deputizes private citizens to sue anyone who “aids or abets” another person seeking an abortion. If the Court decides to remove Section 230’s shield for algorithmic amplification, websites and platforms could be sued for recommending content that helps a Texas resident to obtain an abortion in violation of SB 8. Most sites would likely choose to play it safe and simply remove any abortion-related speech that could expose them to criminal or legal risks.

The abortion information space is just one realm where this could play out if the Court decides that Section 230’s protections do not apply to algorithmic promotion of content. Anupam Chander, a law professor at Georgetown University who focuses on international tech regulation, explained: “Making companies liable for algorithmically promoting speech when they haven't themselves developed it will lead to the speech that is most controversial being removed from these online services.”

Goldman had similar concerns. “We’ve never had this discussion about what kind of crazy things could a state legislature do if they wanted to hold services liable for third-party content. And that's because Section 230 basically takes that power away from state legislatures,” he said. “But the Supreme Court could open that up as a new ground for the legislatures to plow. And they're going to plant some really crazy stuff in that newly fertile ground that we've never seen before.”

Consider the #MeToo movement. Section 230 protects platforms against defamation lawsuits for hosting content alleging sexual harassment, abuse or misconduct. Without the law’s shield, the movement could have had a different trajectory. Platforms may have taken down content that could have exposed them to lawsuits from some of the powerful people who were subjects of allegations.

“That kind of speech, which we have seen the internet empower over the last decade in ways that have literally reshaped society, would lead to the kind of liability concerns that would mean that it would be suppressed in the future,” Chander added. “So, when someone claims that Harvey Weinstein assaulted them, companies are in a difficult position having to assess whether or not they can leave that up when Harvey Weinstein's lawyers might be sending cease and desist and saying, ‘we're going to sue you for it for defamation.’” 

Proponents of Section 230, who have long argued that changing or eliminating the law would end up disproportionately censoring the speech of marginalized groups, are hoping to avoid this scenario. But it’s hard to predict how the Supreme Court justices will rule in this case. Section 230 is one of the rare issues in contemporary American politics that doesn’t map neatly onto partisan or ideological lines. As I reported for Coda in 2021, conservative and liberal politicians alike have taken issue with Section 230 in recent years, introducing dozens of bills seeking to change or eliminate it. Both U.S. President Joe Biden and former president Donald Trump have called for the law to be repealed. 

“This is not just a left-right issue,” Chander explained. “It has this kind of strange bedfellows character. So I think there's a real possibility here of an odd coalition both from the left and the right to essentially rewrite Section 230 and remove much of its protections.”

If the Supreme Court decides that platforms are on the hook legally for recommendation algorithms, it may be harder for people seeking abortions to come across the information they need, say, in a Google search or on a social media platform like Instagram, as those companies will probably take down (or geoblock) any content that could put them at legal risk. It feels almost impossible to imagine this scenario in the U.S., where we expect to find the world at our fingertips every time we look at our phones. But that reality has been constructed, in large part, on the shoulders of Section 230. Without it, the free flow of information we have come to expect in the digital era may become a relic of the past — when abortion was a constitutional right and information about it was accessible online. The Supreme Court’s decision on this tech policy case could, once again, turn back the clock on abortion rights.

Related Articles

Women develop a code to discuss abortion on TikTok

Rebekah Robinson

Roe reversal puts spotlight on Meta’s abortion content moderation policies

Erica Hellerstein

How changing a 26-word US internet law could impact online expression everywhere

Erica Hellerstein

The post What a law designed to protect the internet has to do with abortion appeared first on Coda Story.

On April 27, Musk took a cheap shot at Gadde, mocking past decisions made by her team. Gadde, who is female and hails from India, soon faced a barrage of harassing messages, including racist and sexist insults. Musk was all but proving that point that many had made surrounding his bid to buy the company: under his ownership, Twitter may become a much more hostile place, especially for women and LGBTQ people.

Whether it’s a top executive at Twitter, a BBC reporter in Iran, or a Muslim feminist activist in India, or a journalist at The Washington Post in the U.S., women and sexual minorities — especially those with a public profile — face various kinds of online abuse, from violent threats and hate speech on social media to doxxing attacks that expose their personal information. Since the start of the pandemic, online abuse and harassment has become more severe for women, especially women of color, and LGBTQ people.

Many advocates say tech companies should do more to tackle online abuse, but with people like Musk in charge, it’s hard to see this happening any time soon.

For the foreseeable future, it will be up to those who are targeted to take individual actions to protect themselves. Last week, I spoke with a few key voices dealing with online harassment and gathered their insights.

Speak out, but don’t feed the trolls

Zeba Warsi, a journalist from India, has extensive experience with online abuse. 

“In the beginning I used to reply to every single troll who would lash out with abusive language or be disrespectful. I used to respond to them and then I started realizing when I would respond to people — especially to accounts which were just like bots, with close to no followers, who were thriving on hate — I was in fact giving them more traction by engaging with them. So then I eventually stopped responding.”

Though it doesn’t mean she’s not speaking out. “Address the issue on your own platform, post about it, tweet it, but don’t engage with them. Don't give them the platform, they don't deserve it,” Warsi told me.

The latest major attack she endured was in January, when she appeared in a fake auction app in India, called Bulli Bai, that included profiles and photographs of more than 100 Muslim women, offering them up “for sale.” Like Warsi, most of the women featured in the app are Muslim and do public-facing work — among its targets were politicians, other journalists, and the Pakistani activist and Nobel laureate Malala Yousafzai.

She said disengaging with the trolls has helped her mental wellbeing. Warsi is now pursuing a journalism degree at Columbia University.

“I decided to take a break from my career in India and get this distance from all that hate. And despite all that distance, it still haunts me. You wake up to it, on the 1st of January, the beginning of new year and you get so many messages from your friends and colleagues that your picture is being plastered on this disgusting, degrading, humiliating, fake auction.”

“It gets to you,” she said. I had a very distressing phone call with my father back home in India. He was really upset and he was really scared for my safety, even though I'm not physically in India, because that's the kind of fear that it creates.”

Separate the personal from the professional

Warsi also recommends keeping personal and professional lives separated online as much as possible. 

“I have this sort of division between what is public for me and what is private. I think it helps me to have more control over my social media. So my Twitter is public where I put out my work and my Instagram is private, which is only for my friends and family. So I keep my account private so random Twitter or Instagram trolls are not going to find me there,” Warsi said. 

Keep a record

Gwen Taylor, a program manager at Glitch, a U.K.-based nonprofit working to eliminate online abuse, suggests documenting the abuse.

“Documenting online abuse is a really important step, not only to empower yourself to understand the patterns that are happening, but also to empower you to report it if you decide to do so, and it’s validating, in a way, that it is difficult and it is traumatic,” they said.

Be an ‘active bystander’

Warsi and Taylor both said when it comes to online abuse, having support and community is essential. But being a part of that supporting network also requires knowledge of how to do it.

Taylor says reaching out to people is extremely useful, as online abuse can be so isolating, but it requires thoughtfulness: “Whether you know them or not, just don't take action without their consent. Online abuse can feel very disempowering. You might think you're doing the right thing by reporting it, or by replying to it, but actually that might not be what the person wants. So reaching out, checking in with the person. Maybe giving them options of like, 'I was thinking of doing this. Is that okay? Is there something else you want?'”

Having a network that can report abuse on your behalf can be very powerful and much more effective than doing it alone, says Taylor.

They pointed to a group of women parliamentarians across countries in Africa who had formed a WhatsApp group to organize this kind of support. “Whenever one of them receives online abuse they go into chat and they're like, 'Can you help?' It's really powerful, having this network that empowered people that can go in and be like 'Yes, I'm going to come and report that for you and help,'” Taylor told me.

Amplify their voice, not their victimhood

The other advice Taylor gave on how to empower people being harassed and abused online is to amplify the person, not the abuse.

“Not amplifying the post that's getting the abuse, but in general, just amplifying their messages, their work. If it's an artist, retweeting. If it's somebody writing, saying that you like their writing or encouraging other people to read it,” they told me. “Because often what happens in incidents of online abuse is that the conversation ends up getting totally focused around the abuse. And actually, we want to take away from that. We don't want to give the person being abusive that power.”

Related Articles

Platforms like Twitch have tried to combat harassment against women. It’s not working

Katia Patin

Egypt’s TikTok crackdown targets young female influencers

Mat Nashed

How disinformation became a new threat to women

Nina Jankowicz

The post Online harassment is on the rise — and Elon Musk’s Twitter takeover isn’t helping appeared first on Coda Story.

This was 2014, during the first Russian invasion of Ukraine. After the Ukrainian army pushed Russian-backed forces out of the city, Garmash joined a group of volunteers who quickly got to work, scrubbing and fixing their hometown. But one thing they couldn’t fix was the fallen television tower that had once overlooked the city. Russian-backed militants used it to beam the Kremlin’s message at residents of Slovyansk during the three month long occupation and destroyed it before they left. 

“How will we get the local news?” Garmash remembers asking a local journalist as they cleaned up a street in Slovyansk that July. “There will be no local news,” she replied. 

She was wrong. Within weeks of that conversation, Garmash launched a new media site and named it 6262.com.ua, a reference to Slovyansk’s city code. 

“People really needed local news. And all I needed to provide it was the internet and social media,” Garmash tells me.

By the time Russia invaded again, in February of 2022, Garmash was running the city’s most popular, most trusted local news site. But as Russian tanks rolled towards Kyiv and Western sanctions kicked in, local journalism in Slovyansk was silenced once again. This time, it seems that Meta, the parent company of Facebook and Instagram — not Russia — was to blame.  

“People we serve no longer get our news in their Facebook and Instagram feeds. In that sense what is happening with Facebook is not all that different from what happened with the TV tower back in 2014,” Garmash tells me.

‘WE CAN’T GET THEIR VOICES OUT’

Meta has mobilized resources in response to the war in Ukraine, and the company says it is taking the issue of disinformation around the war seriously. Staffers sent us this statement two weeks ago:

"We're taking significant steps to fight the spread of misinformation on our services related to the war in Ukraine. We've expanded our third-party fact-checking capacity in Russian and Ukrainian, to debunk more false claims. When they rate something as false, we move this content lower in Feed so fewer people see it, and attach warning labels. We also have teams working around the clock to remove content that violates our policies. This includes native Ukrainian speakers to help us review potentially violating Ukrainian content. In the EU, we've restricted access to RT and Sputnik. Globally we are showing content from all Russian state-controlled media lower in Feed and adding labels from any post on Facebook that contains links to their websites, so people know before they click or share them.” 

I read it to Andrey Boborykin, who manages some of the biggest Facebook publishers in Ukraine, in addition to serving as the executive director of Ukrayinska Pravda, one of Ukraine’s largest dailies. He laughs.

The organic reach of Russian propaganda voices in the West has indeed been curbed — Facebook is blocking pages for RT and Sputnik in the EU, as noted above. But for Ukrainian publishers, none of this makes much difference. 

Ukrainian newsrooms are being flooded by graphic images from the frontlines of the war. It’s newsworthy, at times vital content that is in public interest but it is impossible for editors to know what they are allowed to publish on Facebook and Instagram because Meta, Boborykin says, “never made attempts to identify key controversial topics and provide additional guidance to publishers on how to treat these topics on their platform.” 

And even where there are rules, they are confusing and inconsistent. Here is just one example: it is impossible to cover the war in Ukraine without mentioning Azov Battalion, a key group fighting Russians in eastern Ukraine. But a mere mention of Azov Battalion can be considered a violation of community standards. The punishment for such a violation is a “strike” and several strikes could result in their accounts being blocked or suspended. 

Recently, Meta made a temporary change to its hate speech policy, allowing calls for violence against Russian soldiers in the context of the Ukraine invasion. But when Ukrayinska Pravda posted stories about Azov Battalion cheering after hitting the enemy targets in Mariupol their pages got “strikes.” 

Things are especially dire for publishers on the frontlines: dozens of small, independent newsrooms in eastern Ukraine, who have recently lost their ability to promote their posts to their communities. 

“We woke up one day to the news of invasion, and the next day to the news of all of our Facebook and Google ad accounts being blocked. We contacted both. Google fixed the issue within twelve days. We are still waiting for Facebook,” Garmash told me. 

Boborykin says restricting advertisement is normally used by Meta to curb what the company calls “coordinated inauthentic behavior” of state-backed accounts who use ads to promote propaganda, hatred and fake news. Blocking ad capabilities is part of Meta’s effort to combat disinformation on its platform. But what is happening in eastern Ukraine illustrates something else. It is an example, Boborykin says, of platforms applying a “one-fits-all” policy without any attempt to understand the local context. 

“If you are a small publisher from eastern Ukraine, there is a high chance that right now you don’t have any advertising capabilities and you have your pages blocked,” says Boborykin. 

As a result, Boborykin says 31 newsrooms, including 6262.com.ua, are experiencing a massive drop in Facebook revenue and audience. In addition to his day job, Boborykin works with the Media Development Foundation and is currently running emergency fundraisers for local Ukrainian newsrooms. Limits that Facebook has imposed on them, he says, are affecting wartime fundraising too. 

“We can’t promote their pages, we can’t get their voices out,” Boborykin says. “It’s crazy because it means that [local publishers] are cut off from their communities. And many of them are already cut off physically, because they’ve had to flee. If they don’t flee they work under shelling. It’s crazy that they have to be dealing with technical constraints imposed by Facebook on top of it all,” he says. 

In early March, in order to continue operations, Valerii Garmash moved most of his 14-strong team away from the frontline in eastern Ukraine, keeping only a few journalists in Slovyansk. “In extreme times like this, people need information more than ever. But they are no longer seeing us on their feeds,” he says.

On Facebook, in what appears to be the result of new company policies on Ukraine, 6262.com.ua has seen an 80% drop in audience since the war began. The numbers are similar on Instagram. Financing independent journalism is never easy, but Garmash has taken a unique approach, providing spin-off services like video production and social media consulting to local businesses.

Garmash says his team runs 25 business pages on Facebook alone. The local vet clinic, the city’s pharmacy and a clothing shop are among 6262.com.ua’s clients. But now they can’t get on the feeds of their community members either.

I asked Meta staffers I am in touch with whether they are aware of the huge losses that their company’s policy brought to small and struggling independent publishers in Ukraine. Their reply reads that “Meta remains committed to building systems that promote and protect news content on our platforms, in order to help news publishers, large and small, better make money and serve their local communities.”

“We can't respond to the specific claims reported on by Coda Story as these details were not shared with us prior to publication,” the statement goes on “but we do partner with international institutions such as Reuters and ICFJ as well as regional and local organizations — including in Ukraine — to train journalists and newsroom professionals and get a better understanding of the challenges they face.”

In the last two months, staffers at 6262.com.ua have contacted Meta at least 40 times. They have yet to receive a response. 

NO ANSWER: A GLOBAL PROBLEM

The experience of the team of 6262.com.ua is playing out for independent media across eastern Ukraine, and even beyond its borders. 

We recently profiled two independent newsrooms in Georgia, a country also partly occupied by Russia, that saw their audiences decline by as much as 90%t after Facebook blocked some of their posts about the war in Ukraine. The reasons why the posts were blocked are unclear, but both newsrooms suspect that they were reported by Russian trolls. 

After the piece was published, a Facebook representative asked me to pass on his personal details to the journalists we profiled and promised to review their cases. I did and journalists followed up with Facebook directly. Two weeks have gone by, and neither television station has gotten a clear answer from Facebook. 

An estimated 26 million people in Ukraine use Facebook every month. “These platforms are crucial for us,” Boborykin says. Having worked across the African continent and closely watched Facebook’s controversies in places like Myanmar, Boborykin says he has no illusions about Meta’s business model, or any issues with it, for that matter. The problem, he says, is the way that Meta deals with people and organizations they like to call partners. 

“What they have done in the case of Ukraine is 1% of what they could have done,” Boborykin says. “Have better news partnerships, reach out to local publishers, make lists of people and media organizations that you trust. Reply to their messages.” 

What would you say to Mark Zuckerberg if you met him? I ask Valerii Garmash, the founder of 6262.ua before we hang up. 

“I’d tell him that in Ukraine he is violating his own mission,” Garmash says. “He set up Facebook to give people power to build communities. He is destroying ours.”

This story originally ran in our weekly Disinfo Matters newsletter that looks beyond fake news to examine how manipulation of narratives, rewriting of history and altering our memories is reshaping our world. We are currently tracking the war in Ukraine. Sign up here.

Related Articles

Outside the US, Elon Musk’s vision of a rules-free Twitter is expected to unlock violence and civil strife

Ellery Roberts Biddle

How Silicon Valley is helping Putin and other tyrants win the information war

Natalia Antelava

How changing a 26-word US internet law could impact online expression everywhere

Erica Hellerstein

The post Fleeing Russian bombs while battling Facebook. A Meta problem Ukrainian journalists did not need appeared first on Coda Story.

What would happen in countries vulnerable to social unrest and communal violence if the company threw its content rules out the window and embraced an absolute commitment to free speech?

Elon Musk wants to find out. In his recent bid to buy the company, the Tesla and SpaceX CEO wrote of his belief in Twitter’s “potential to be the platform for free speech around the globe.” Musk pledged to “unlock” that potential, implying that he would ditch the company’s content rules and simply let the tweets flow.

In case you missed it: Musk bought 9% of shares in the company in mid-March, a figure that only became public last week, prompting Twitter’s leadership to offer him a seat on its board of directors. Musk entertained the offer, but then had a second thought: Why not just buy the whole company? The board opted to deploy a so-called “poison pill” strategy, effectively preventing a Musk takeover.

But there was still time to wonder what might happen if the world’s wealthiest person got his way. Academic experts cautioned that an absolute free speech policy would turn the platform into a cesspool of hate speech, spam, and porn. Veteran tech critics pointed out that Musk’s ideas about content moderation were popular in the earlier days of the internet, and that time has proven that they really don’t work at scale. Across the political spectrum in the U.S., pundits speculated on whether this would pave the way for Donald Trump to return to the platform.

What would it mean for the majority of Twitter users, who live outside the U.S.?

“That just doesn’t work in a country like India,” said Nikhil Pahwa, a tech expert and founder of Medianama, an India-focused tech policy publication based in New Delhi. India is Twitter’s third-largest market after the U.S. and Japan.

“We have real world consequences from the kind of speech that Twitter enables. Our political parties are really, really adept at understanding how the algorithms work, how to create trends, how to make something shareable,” Pahwa said. “What they excel at is essentially fueling hate.”

In recent years, India’s ruling Bharatiya Janata Party and other hardline Hindu nationalist groups have made Twitter, alongside Facebook and WhatsApp, an essential platform for promoting their agendas, sometimes inciting violence against religious minorities, Muslims in particular.

“I think we’re in a situation where we need more moderation of hateful content and not less. I don’t think Musk understands or cares for whether people are getting polarized or killed in India,” Pahwa told me.

While more than 20 million Indians use Twitter on a regular basis, others have left or avoided the platform for exactly these reasons. A female researcher I spoke with, who studies gender-based harassment online, declined to be quoted for this story, citing concern that she would be attacked as a result.

Twitter’s policies prohibit hate speech, harassment, and incitement to violence, but it has a poor track record of enforcing these rules, especially for posts that are not in English.

“Everyone thinks they know how to do content moderation until it becomes their job,” said Mishi Choudhary, founder of the Software Freedom Law Center, a tech policy group in New Delhi.

“I am not sure how [Musk] plans to address censorship by proxy that countries like India demand,” she wrote in a message.

The Modi government is known for pressuring the company to remove certain posts and reinstate others. In 2021, officials updated India’s IT Rules and began requiring large foreign tech platforms to create locally-staffed grievance programs for content removal and related disputes. It took several months, and a police visit to Twitter’s local offices, before the company complied.

Twitter has faced similar kinds of pressure in sub-Saharan Africa, where it plays a significant role in national politics in the region’s largest markets, Nigeria and Ethiopia.

In Nigeria, Twitter in 2020 became the digital ground zero for #EndSARS, a social movement protesting police brutality that played out both online and in cities across the country.

“Twitter created a special emoji for the EndSARS protest, and also verified some major handles that promoted the protests. [Former CEO Jack Dorsey] himself raised some money for them via Bitcoin,” said Nwachukwu Egbunike, a media and communications scholar at Pan-Atlantic University in Lagos. 

“The feeling around government quarters is that Twitter really sided with protesters,” he said.

Less than a year later, the government banned Twitter altogether, after moderators took down a tweet posted by President Muhammadu Buhari that contained a veiled threat against Igbos, one of the country’s largest ethnic groups.

This went on for seven months. When they lifted the ban, officials announced that they had reached an agreement with Twitter, under which the company would “act with a respectful acknowledgement of Nigerian laws and the national culture and history,” and alluded to a code of conduct meant to govern the relationship. This document has not been made public.

“One has the impression that Twitter gave in or compromised Nigerian digital rights in order to get unbanned,” said Egbunike. “If this agreement is true, and the Nigerian government has the power to pull down tweets, where does that leave Nigerians?” he asked.

Egbunike’s question would be a good one for Elon Musk. The governments of both Nigeria and India have demonstrated that if companies like Twitter want to stay accessible in their countries, they need to be prepared to comply with censorship demands and the whims of whichever party is in power. 

In theory, regular people can still say whatever they like online, but between rules like these and political parties’ online influence operations and troll armies, the costs of doing so can be pretty high. 

Victims of violence stoked on the platforms pay the highest price of all. Endalk Chala, a communications professor at Hamline University and former blogger, described the role Twitter has come to play in Ethiopia’s ongoing civil conflict. Twitter has made some efforts to curb problematic speech coming from pro-government voices, Chala explained, but different ethnic groups continue to promote violence and hate on the platform.

“On Twitter, if a person from one ethnolinguistic group makes fun of a person from another, and that speech is available for people who feel attacked and derided, [members of the target group] will be harmed,” said Chala. “People are dying every day now for things like this,” he said.

“There is really bad content, in English, Amharic and in other Ethiopian languages. The content moderation on Twitter doesn’t work really well,” he said.

What if, as Musk advocates, the company simply stopped trying to moderate speech in Ethiopia?

“I am all for free speech,” Chala said. “But if it’s this messy now, you can’t imagine what would happen without it.”

Related Articles

How Silicon Valley is helping Putin and other tyrants win the information war

Natalia Antelava

Platforms like Twitch have tried to combat harassment against women. It’s not working

Katia Patin

Warnings to Journalists Blur Twitter’s Transparency in Pakistan

Umer Ali

The post Outside the US, Elon Musk’s vision of a rules-free Twitter is expected to unlock violence and civil strife appeared first on Coda Story.

“You cannot post or comment for 3 days”

“You can’t go live for 63 days”

For Afghan journalist Shafi Karimi, the list of restrictions that Facebook has imposed on him goes on and on. “I am blocked and I am losing an audience, and people are losing vital information,” says Karimi, who is covering Afghanistan from exile in France.

He is not the only one. From Afghanistan to Ukraine, and much of the rest of the non-English speaking world, journalists are losing their voice. Not only because of the increasingly oppressive governments that target them, but also because policies created in Silicon Valley are helping oppressors of free speech peddle disinformation. 

Over the past month, Karimi has sent numerous appeals to Facebook, but has gotten no reply. And so last week, Karimi pushed his way through a champagne-sipping crowd of journalists and media representatives at a reception that Meta, Facebook’s parent company, threw at the International Journalism Festival in Perugia, Italy. 

The festival is one of the industry’s key annual events and a rare opportunity for journalists like Karimi to speak to big tech company representatives directly. 

Karimi found a Meta staff member and, shouting over the crowd, tried to explain to him how all independent voices on Afghanistan are being affected by Facebook’s poorly thought-out policy that seems to indiscriminately label all mentions of the Taliban as hate speech and then summarily remove them. He explained that Facebook is an essential platform for people stuck in the Taliban-imposed information vacuum and that blocking those voices benefits first and foremost the Taliban itself. The Meta representative listened and asked Karimi to follow up. Karimi did – twice – but never heard back. 

As Karimi was pleading with one Meta employee, I cornered another one across the crowded reception hall, to deliver a similar message from a different part of the world. A friend working for an independent television station in Georgia had asked me to pass on that her newsroom had lost a staggering 90% of its Facebook audience since they began covering the war in Ukraine.  The station, called Formula TV, made countless attempts to contact Meta but received no response. 

Was there anything these Meta staffers could do to help?

‘Facebook pages of real, independent journalism are dying’

“We went from reaching 2 million people to reaching 200,000,” Salome Ugulava, Formula’s chief digital editor told me. The drop followed a warning they received from Facebook after its algorithm flagged a quote from Ukrainian President Volodymyr Zelensky as hate speech. The post, which she showed me, was merely a translation into Georgian of a post by Zelensky himself. 

This seemingly technical error caused the station to lose 90% of its audience, but also a chunk of its revenue. “Monetization has been suspended. It is a harsh punishment,” Ugulava said. 

The Tbilisi-based opposition TV station Mtavari saw similar declines when it ran a story about Azov Battalion, a Ukrainian volunteer paramilitary group that has been defending Mariupol, where some of the worst atrocities of the war have occurred. Facebook removed the story on grounds that it was “a call to terrorism.” The Azov Battalion is controversial because of the far-right and even neo-Nazi leanings of many of its members, but only Russia categorizes it as a terrorist organization. 

“We were already under constant attacks from Georgian government troll farms, but since the invasion Russian-backed organizations began reporting us too. Azov incident was one of many,” Nika Gvaramia, the channel’s Director General told me. 

Gvaramia says many of the Ukraine-related stories that Mtavari posted in early March were taken down by Facebook in early April, weeks after they were first published and seen by millions of people. The Mtavari team has good reason to believe they are being reported by Kremlin-backed accounts. 

“The worst part is that there is no warning mechanism, there is no obvious criteria for these takedowns and appeals to Facebook take weeks … Facebook pages of real, independent journalism are dying,” Gvaramia said. His channel’s engagement, he said, dropped from 22 million in early March to 6 million in early April.  

For a place as small as Georgia (population 3.5 million) the consequences of these takedowns are huge. Like in many countries outside the West, Facebook has become the nation’s virtual public square – the place where people gather to discuss and debate their future. This discussion is existential in Georgia, because the future is so precarious: twenty percent of the country is already occupied by Russia, and many fear that Ukraine’s invasion will push Georgia further into Moscow’s embrace. Kremlin-funded disinformation campaigns have put independent media covering the situation under massive pressure. When media outlets like Formula TV and Mtavari disappear from people’s Facebook feeds, the very ideas of liberal democracy disappear from the public debate. 

“The power that Facebook has is scary. The way it is using it is even scarier,” a Russian journalist, who did not want to be named due to security concerns, told me. Her account was suspended after she was reported to Facebook by numerous accounts accusing her of violating community standards. She suspects the accounts that reported her were working on behalf of the Russian government. Like Karimi, who says Facebook is helping the Taliban, she says Facebook’s policies are aiding Vladimir Putin’s agenda. 

“Silicon Valley is helping Putin to win the information war. It is insane and it has to stop,” she said “But we don’t know how to tell them this, because it is impossible to speak to them directly.”

None of this is new

Meta has been accused of promoting hatred and disinformation around the world before, from Nigeria to Palestine to Myanmar where the company was famously accused of fueling the genocide of Rohingya Muslims. And experts in regions like the Middle East and Africa have noted that even though tech platforms are failing to adequately address the content crisis around Ukraine, they have brought a faster and more robust public response in this case than in places like Syria or Ethiopia. 

With each new crisis, Meta has made new promises to better account for all the cultural and linguistic nuances of posts around the world. The company even put out an earnest-sounding human rights policy last year that focused on these issues. But there’s little evidence that its practices are actually changing. Facebook does not disclose how many moderators it employs, but estimates suggest around 15,000 people are charged with vetting content generated by Facebook’s nearly 3 billion users globally. 

“It is like putting a beach shack in the way of a massive tsunami and expecting it to be a barrier,” one moderator told me. I cannot name him, or say where he is, because Facebook subcontractors operate under strict non-disclosure agreements. But he and other people with access to moderators told me that the Ukraine war is the latest proof that Facebook’s content moderation model does more harm than good. 

Facebook moderators have 90 seconds to decide whether a post is allowed to go up or not. From Myanmar to Ukraine and beyond, they are dealing with incredibly graphic images of violence or highly contextual speech that typically doesn’t line up with Facebook’s byzantine rules on what is and is not allowed. The system, in which posts live or die depending on a quick decision of an overworked, underpaid and often traumatized human, takes a toll on the mental health of the moderators. But it is also damaging the health of the information ecosystem in which we live. 

“The weight of this war is falling on outsourced moderators, who have repeatedly sounded the alarm,” says Martha Dark, director of Foxglove Legal, a UK-based tech justice non-profit group that is working on issues of mistreatment of Facebook content moderators around the world.   

“Despite its size and its colossal profits, Ukraine has shown that Facebook's systems are totally unequipped to deal with all-out information war,” says Dark. “No one is saying moderating a war zone is an easy task. But it's hard to shake the sense that Facebook isn't making a serious effort at scaling up and fixing content moderation – because to do so would eat into its profit margins. That's just not good enough,” Dark said.

Facebook has pledged to reply to my questions about the cases of Shafi Karimi from Afghanistan and FormulaTV in Georgia in the coming days – I’ll report on it as soon as I hear back. In the meantime, my contacts there offered this response, which is attributed to a “Meta spokesperson”: 

"We're taking significant steps to fight the spread of misinformation on our services related to the war in Ukraine. We've expanded our third-party fact-checking capacity in Russian and Ukrainian, to debunk more false claims. When they rate something as false, we move this content lower in Feed so fewer people see it, and attach warning labels. We also have teams working around the clock to remove content that violates our policies. This includes native Ukrainian speakers to help us review potentially violating Ukrainian content. In the EU, we've restricted access to RT and Sputnik. Globally we are showing content from all Russian state-controlled media lower in Feed and adding labels from any post on Facebook that contains links to their websites, so people know before they click or share them.”

It’s true that the company has deplatformed some of the most prominent sources of Russian disinformation, such as Russian state broadcasters. And we can’t know what impact some of these other measures are having, due to the company’s lack of transparency about its actual day-to-day content moderation decisions. But the real power of Facebook, which is arguably the most potent communication tool in the world at the moment, lies in organic, peer-to-peer shares and that’s where so much disinformation flourishes. 

“We can no longer cover the war,” says the Georgian journalist Salome Ugulava. “Our followers are not seeing us on their feeds.” 

It’s not just Facebook: Twitter is facing similar accusations of doing a terrible job policing its platform when it comes to Ukraine. “You are failing,” tweeted journalist Simon Ostrovsky who is covering Ukraine for PBS Newshour. “Hundreds of sock puppet accounts attack every tweet that counters the Kremlin narrative, meanwhile you fall for coordinated campaigns to suspend genuine accounts.”

This story originally ran in our our weekly Disinfo Matters newsletter that looks beyond fake news to examine how manipulation of narratives, rewriting of history and altering our memories is reshaping our world. We are currently tracking the war in Ukraine. Sign up here.

The post How Silicon Valley is helping Putin and other tyrants win the information war appeared first on Coda Story.

It was 2016, and lawmakers in Pakistan had passed a sweeping and controversial internet law granting the government the power to censor online content, including criminalizing hate speech and defamation. The bill, part of a national plan to combat terrorism, came in the wake of a 2014 terrorist attack on a school in Peshawar that killed more than 130 children. 

Authorities said its implementation would help defend the country against threats to national security and cybercrime; critics argued that its broadly defined provisions could be used to muzzle online expression and content critical of the government. One prominent Pakistani legal expert, who has helped a number of other countries draft internet laws, said it was “the worst piece of cybercrime legislation in the world.”

Aziz, co-founder of the Pakistani digital rights group Bolo Bhi, campaigned against the law as it was being debated, speaking out against its potential impact on freedom of speech. Five years after the passing of the Prevention of Electronic Crimes Act (PECA), Aziz said it has been used against journalists and dissidents, as well as women who have come forward with allegations of sexual harassment online, but then been charged with defamation.

The outcome, Aziz said, “just confirms and validates the apprehensions we raised when the law was being introduced.” The U.S. State Department echoed critics’ concerns, concluding in a 2021 human rights report on Pakistan that it “gives the government sweeping powers to censor content on the internet, which authorities used as a tool for the continued clampdown on civil society.” Between 2016 and 2020, Pakistan’s ranking in the global internet freedom index fell from 31st to 26th, according to the U.S.-based nonprofit Freedom House. 

Pakistan’s assault on digital freedom is once again on Aziz’s mind, as U.S. politicians threaten to change a landmark internet law that could affect the bounds of online expression well outside of their country’s borders. The 1996 law, Section 230, shields websites from legal liability for the content users post online. The 26-word mandate, signed by President Bill Clinton and widely referred to as the “digital Magna Carta,” has allowed tech giants like Facebook, YouTube, and Twitter, to flourish without worrying about legal liability. 

Yet Section 230 has come under fire from American lawmakers on both sides of the aisle as a bipartisan appetite to regulate large technology companies has grown, especially following the January 6 insurrection at the U.S. Capitol and the deplatforming of former President Donald Trump from Facebook and Twitter. Both events have ratcheted up calls on the right and the left to hold social media platforms accountable for their content moderation decisions, with Republicans principally concerned about biases against conservative voices on social media, and Democrats focused on the role of platforms in spreading disinformation and extremism. 

Section 230 has become a flashpoint in American politics and the subject of numerous attempted legislative reforms in the past two years. Since 2020, roughly two-dozen bills have been introduced, by Congressional Democrats and Republicans alike, to slash or chip away at it. Some would repeal the law altogether; others would strip away platforms’ liability shield for posted content involving civil rights, harassment, child sexual abuse and more. 

While the proposals vary and reflect a broad range of concerns, from censorship to cyberstalking, it’s unclear which, if any, will pass. But what is obvious is that there is significant political will to eliminate or reconfigure Section 230. In a January 2020 interview, Joe Biden — now President of the United States — stated that it should be “immediately” revoked, while former president Donald Trump repeatedly called to repeal the law during his time in office.

Yet, while much of the debate in the U.S. has focused on the domestic impact of a post-Section 230 internet, lawyers, human rights campaigners, and digital rights activists around the world predict that eliminating the law could have global consequences. Among their fears are that scrapping or significantly amending the law could set a precedent for countries from Asia to Latin America to introduce new policies that place severe restrictions on digital speech or make platforms liable for the material posted on their sites, causing them to aggressively remove content that could expose them to lawsuits. 

Eric Goldman is a professor focusing on internet law at California’s Santa Clara University School of Law and has written extensively about Section 230. As he put it, “To the extent that the U.S. stops trying to fight for free speech online, every other country in the world is going to fall even further on the censorship scale.”

Farieha Aziz expects a U.S. decision on Section 230 to influence the conversation on tech regulation in Pakistan, where the government has repeatedly tightened control of the internet in recent years. In addition to the cybercrime law, authorities have banned dating applications over concerns about “immoral” content and unveiled regulations imposing penalties, including hefty fines and potential bans, on social media platforms that violate government requests to take down content. 

“The first thing that happens over here is the government tends to borrow from the narrative of, ‘Oh, but this happens elsewhere as well, this happens in democracies,” she said. If the U.S. changes Section 230, Aziz predicts that Pakistani authorities are “immediately going to draw from it and use this as a justification that, “Look, even in the U.S. they believe the companies need to be held liable.’”

Internet reform and global consequences

So, how could scrapping a U.S. internet law affect how people around the world engage with digital platforms? 

Section 230 protects websites and platforms like Facebook, Amazon, Twitter, and YouTube from lawsuits over the content users create, like comments on online news stories, restaurant reviews on Yelp, videos on YouTube, social media posts and more. 

One of Section 230’s foremost experts, Jeff Kosseff, an assistant professor of cybersecurity law in the United States Naval Academy's Cyber Science Department, describes it as the “26 words that created the internet.” The Electronic Frontier Foundation, a U.S.-based nonprofit that advocates for digital privacy and free speech, refers to it as “the most important law protecting internet speech.” 

For critics, however — including U.S. lawmakers — Section 230 has given social media giants too much legal protection and shielded them from accountability for their approach to content moderation. Democrats who have spoken out against it say that the legal shield removes the incentive for platforms to moderate harmful content, including disinformation, hate speech, conspiracy theories, and extremism. Republicans opponents, meanwhile, argue that it provides cover for social media companies to censor voices on the right. 

Though the two sides have both embraced reform, they have different objectives, underscoring how legislation scaling back platform immunity can be used to advance vastly different political agendas. Democrats want companies to more aggressively moderate harmful content, such as hate speech and misinformation, while Republicans want to limit the abilities of the same social media companies to moderate postings on their platforms.

While most countries have yet to pass local liability legislation, Section 230 has acted as the “law across the world,” according to Anupam Chander, a law professor at Georgetown University who specializes in international tech regulation. The European Union has taken a similar, but less sweeping approach, passing legislation in 2000 shielding platforms from liability if they are unaware of illegal content or quickly comply with requests to take it down.

Chander says a change to Section 230 will have “ripple effects across the world.” It could license other countries to introduce policies making platforms liable for content hosted on their sites, forcing them to more aggressively moderate and remove material that could expose them to litigation or violate local laws.

“There's a lot of speech that might lead to liability across the world. And I think that is a risk of 230 changes,” Chander said. “The precedent it will set for the most free-speech-promoting democracy in the world to hold platforms responsible for the millions of speech messages that they offer each day is really quite stunning.”

Latin America and deplatforming

In Latin America, the U.S debate on Section 230 is already influencing conversations about platform liability. A recently proposed draft bill in Mexico would give the country’s telecommunications regulator (IFT) the power to overrule social media networks in decisions about content moderation. It would also require platforms with more than one million users to get authorization from the IFT to operate in the country and impose fines of up to $4.4 million on digital platforms for non-compliance. Human Rights Watch said the legislation would “place the harshest restrictions on free speech that Mexico has seen in decades.”

Mexico’s draft bill came a month after Donald Trump’s suspension from Facebook and Twitter, a decision that infuriated President Andrés Manuel López Obrador, who had a warm relationship with the former U.S. president, and vowed to lead an international campaign against social media censorship. 

Javier Pallero, an Argentina-based policy director for the digital rights group Access Now, draws a line connecting Mexico’s proposal, which was drafted by a member of Lopez Obrador’s party, Trump’s social media suspension and bipartisan support for Section 230 reform. 

“If the country that used to champion this limitation of liability is now questioning it, then that generates the space for other countries and lawmakers to spark proposals of reform,” he said.

In Brazil, too, it appears that U.S. wrangling over Section 230 has fast-tracked efforts at internet reform. At least four bills related to content moderation and platform liability have been proposed since the U.S. Capitol riot, including two that explicitly reference Trump’s deplatforming from social media outlets. 

Like Republicans in the U.S., conservative politicians in Brazil have set their sights on what they see as excessive censorship of right-leaning voices. One proposal would ban platforms like Facebook and Twitter from moderating content unless they received a court order; another would make platforms liable for financial damages when they censor or ban content that expresses a user’s opinion.

The bills — all introduced by Congressional members of the right-wing Partido Social Liberal, which was President Jair Bolsonaro's party before he split from it in 2019 — mirror Republican lawmakers’ concerns in the U.S. about suppression of right-wing voices on social networks. 

Artur Pericles, head of research at InternetLab, a technology-focused research center in the capital, Sao Paulo, told me that he had noted similarities between the Brazilian platform liability bill for opinion-related content moderation and legislation introduced by U.S. Senator Josh Hawley, a Republican from Missouri, in June 2020. Hawley’s bill — the Ending Support for Internet Censorship Act — would only extend Section 230 protections to companies that are able to prove political neutrality by getting an immunity certification from the Federal Trade Commission. 

“Both the left and right agree that platforms have too much power,” Pericles explained. “But then the left thinks that platforms should be more accountable to combat misinformation and hate speech. The right thinks that the platforms should have less power so they are not able to censor people on the right.”

Bolsonaro has also thrown his weight behind the push to regulate content moderation. In mid-May, the government published a draft executive order that would bar platforms from taking down content unless they have a court order, an option that could bypass Congressional efforts, and become law with the swipe of Bolsonaro’s pen. 

Though Pericles suggested the proposals would be unlikely to survive legal challenges, he said the executive order, in particular, has already had an impact on public discourse, highlighting platform liability and drawing attention to politicians’ efforts to limit content moderation. “Even if it doesn't make it into law, it definitely has changed the conversation and has given it a new sense of urgency,” he said. “People on the right who support the president have been drawing on the conversation in the United States for a while now.”

Illustration by Teona Tsintsadze

The post How changing a 26-word US internet law could impact online expression everywhere appeared first on Coda Story.

Accusations of misogynistic comments, threats of rape, revenge porn and doxxing spread to other social media platforms, not just the ones centered on gaming. From YouTube to Twitter and Reddit, big-name content creators — almost exclusively male — were getting away with creating online worlds that amplified the harassment women have fought for decades to eradicate from their workplace.

That online platforms function as workplaces for content creators and that harassment and abuse threaten their livelihoods is rarely addressed in discussions on platform regulation and the industry’s self-policing policies. Many of the users involved are financially dependent on ad revenue from their online streams or posts, making these platforms essentially their place of business.

“In a normal workplace if your employees are getting harassed this way, you would do something,” said Karen Skardzius, who researches Twitch streamers and platform regulation at York University in Toronto. “If this was a store and you had someone come in and scream all these expletives and hateful language at one of your employees, you would chuck them out of the store. But instead, here, that’s engagement with the platform.”

Over several days in June 2020, there were dozens, and later hundreds of posts ranging from accusations of inappropriate comments during gaming streams and rape at industry conferences. One New York-based streamer, Jessica Richey, started collecting the posts in a Google spreadsheet. By July 3, it listed over 400 accusations of harassment, manipulation and sexual assault.

Platforms and gaming executives responded with promises to rid their spaces of bad behavior. “I’ve heard your voices,” tweeted Twitch CEO Emmett Shear on June 23. He promised to extinguish “systemic sexism, racism and abuse” in the gaming industry. Nearly six months later, Twitch, which is owned by Amazon, announced a new hateful conduct and harassment policy, which went into effect on January 22. 

The updated policy eschewed any major upheaval on the company’s gaming network. It now promises a “much lower tolerance for objectifying or harassing behavior” and a ban on sending unsolicited nude images and videos, an issue many users flagged in June 2020. Users can now report other gamers for making repeated comments on perceived attractiveness.

“Our recent policy updates take a clearer and more consistent stance against hate and harassment, and enable us to enforce our guidelines more consistently,” said a Twitch spokesperson in an email. They added that the policy was just one of “a number of projects underway to address hate and harassment.”

The upgraded policy puts Twitch a step ahead of other platforms, such as YouTube, Reddit, and Twitter, where female and non-binary content creators say harassment is also pervasive.

For YouTuber Pieke Roelofs, the threats haven’t stopped coming. After accusing professional YouTuber Alexander McKechnie of raping her in 2016, Roelofs has faced a stream of death threats and hateful messages from his fans, who number in the millions. McKechnie’s videos about science and the future have been collectively viewed over 145 million times on his channel. Roelofs says this dedicated fan base has been waging a relentless campaign against her. 

YouTube, Twitter, Reddit — name the platform and Roelofs has been harassed, doxed and threatened by other users. After a court case in the Netherlands, where Roelofs lives, opened a criminal investigation in 2018, the abuse only intensified. Today she is left fending off online hate from McKechnie’s fans with the limited toolkit provided by the platforms where she makes her living.

Roelofs’ income took a big hit. Searching her name online pulls up pages of tweets, subreddits and comment threads calling her a liar and worse. With limited options for reporting and removing the content, she says she’s left with a choice: putting up with the stream of online hate or leaving her professional field entirely. Since the abuse began she’s turned off comments on her videos, resulting in YouTube’s algorithm downgrading her content.

“People are taken hostage by these companies and their rules, people feel they can’t leave,” Roelofs said. “I want them to start taking responsibility for these huge worlds they have created online.”

McKechnie, known by his username Exurb1a, has not responded to multiple requests for comment.

Google-owned YouTube has faced several high-profile calls for greater accountability for harboring harassment and hate speech since Roelofs first reported McKechnie’s channel in 2017.

In June 2019, YouTube came under scrutiny for allowing Steven Crowder, a right-wing video blogger, who at the time had 3.8 million subscribers, to stay on the platform. Crowder had been sending homophobic and racist messages to journalist Carlos Maza. A public outcry followed, with YouTube responding with updates to its anti-harassment policy. The platform also demonetized Crowder’s account, temporarily stripping him of the ability to generate ad revenue on his videos, but still keeping him on YouTube. The actions made Crowder a hero on the right and he gained 1.4 million subscribers since his temporary ban.

Accusations that Twitter, Facebook, YouTube and other social media giants over-regulate content to discriminate against conservative voices are in stark contrast to the voices who say platforms are toothless when it comes to content moderation.

Moments of reckoning come and go, say many women and non-binary content creators, marked by initial optimism, strongly worded statements from industry CEOs and an ensuing lack of meaningful change. 

In some cases, as with Crowder, a platform’s actions — such as demonetizing an account or a temporary ban from the website — backfire, ultimately fueling increased popularity for the person targeted by a platform for punitive measures. 

New moderation policies unveiled by platforms with much fanfare are often framed as “updates” or “clarifications” to pre-existing language.

Skardzius says she often sees women resorting to posting their own detailed harassment policies on their channels, outlining what will get another user banned or reported. As for male streamers, “their profiles don’t have that,” she said. “When they do have rules it’s something like ‘no politics and no religion.’ They’re just not targeted by this kind of stuff.”

Other streamers rely on customized bots to filter out hateful content on their accounts. Natalie Casanova — known as ZombiUnicorn on Twitch, where they were named streamer of the year in 2020 — protects their profile with bots that screen out messages from other users for words like “whore” and “rape.”

Casanova has especially relied on these filters since last June, when they accused British YouTuber Tom Cassell, known online as Syndicate, of rape. Cassel still streams on Twitch to three million followers nearly every day. He has almost 10 million subscribers on YouTube.

Cassell did not respond to a request to comment for this piece.

Hate-filled messages continue to find Casanova, despite the bots and the complaint they have submitted to YouTube and Twitch. YouTube never responded to Casanova’s allegations against Cassell. Twitch sent Casanova a confirmation email on July 8, 2020 saying it had received the message and asking for more information. Casanova attached the police report they filed concerning the alleged rape, contact information for the police detective examining the case and dozens of witness statements. Casanova has not heard from the platform since.

“Nothing has literally ever happened,” Casanova said. “I think at least getting an email from this alleged investigation team saying, ‘Hey, sorry it's been like eight months, but we came to a decision and we couldn't do anything’ … that would at least be a first step.”

The women and non-binary gamers and streamers interviewed for this piece did offer their own ideas for reporting tools they wish were on the negotiating table. Such as bans based on IP addresses, rather than usernames, which could slow down how quickly harassers are able to re-join platforms after being banned, said Casanova.

Skardzius says many of the women she spoke to for her dissertation demanded more reliable channels for contacting actual people, not anonymous chat bots, working at these companies about complaints.

In the meantime, the abuse across the platforms carries on for creators like Roelofs and Casanova. Roelofs' most recent death threat came via a direct message on Twitter on January 31, accompanied by photographs of dead bodies. Casanova is messaged by the same troll every day from an anonymous account on Twitter. Every time they block the account, the user messages again from a new one.

The post Platforms like Twitch have tried to combat harassment against women. It’s not working appeared first on Coda Story.

That all began to change in late August, with a video posted from the city of Chita in Siberia. Soundtracked by Billie Eilish’s hit “Bad Guy,” it showed a teenage boy inside a gilded Orthodox church. Looking over his shoulders and crossing himself, he leaned over and lit a cigarette on a church candle. “This will be the crime of the year,” said an off-camera voice. 

The video was widely shared and a criminal case opened, accusing the boy of “offending the feelings of believers” — a charge carrying a sentence of up to three years in prison. After he published three online apologies, the local branch of the church appealed to authorities to drop the case.

Later that month came a second court action, in which a TikTok user was fined $250 for encouraging her followers to attend a protest in the city of Yekaterinburg. 

“People being imprisoned for their speech online is hardly news for us. It happens a lot on other platforms,” said Sarkis Darbinyan, founder of the Moscow-based Digital Rights Centre. “In this sense, there is no hope for TikTok. Most likely, it will start to comply and censor everything that Russian-owned platforms already do.”

While TikTok’s free-speech honeymoon may now be over, thousands of videos are still being uploaded and viewed by young Russians every day. As such, it opens a rare window to what this demographic is thinking and talking about.

Much like Twitter or YouTube, TikTok, which has 20 million users in Russia, publishes its trending topics. Lately, protests in Belarus and the poisoning of Russia’s main opposition figure Alexey Navalny have dominated the conversation. TikTok also counts the number of views for trending topics: constitutional reform in Russia has 107 million, while a hashtag in support of a recently incarcerated governor in eastern Russia has a staggering 274 million.

This is especially remarkable considering that TikTok has an aggressive content moderation team, who in the first half of this year removed more than 104 million videos from the platform globally. Last month, reports surfaced of how TikTok specifically censors hashtags related to LGBTQ issues in countries including Jordan, Turkey and Russia. 

The results of the company’s content moderation strategy can be seen in its second-ever transparency report, published on September 22. Between January 1 and June 30, moderators removed more than 700 pieces of content, in response to requests filed by national governments. Russia took the top spot, responsible for nearly 300 requests.

In recent months, TikTok, which is owned by the Beijing-based company ByteDance, has come in for considerable regulatory scrutiny in the EU and U.S. over its collection of user data. Before a September court decision temporarily blocked the move, President Donald Trump also attempted to ban U.S. online stores from hosting the application.

But that hasn’t stopped young Russians posting.

Kiril Fedorov, a 27-year-old LGBTQ activist from Saint Petersburg, believes the chances of facing prosecution under Russia’s “gay propaganda” law are slimmer on TikTok than any other platform. 

“Right now TikTok is saved by its reputation as something apolitical and unserious,” he said. “But there’s really a lot of political content there, a lot about human rights, feminism, LGBT and so on.”

Federov, who has 90,000 likes on his account, and other Russians get around the platform’s censorship tools by writing posts and hashtags using Cyrillic instead of the European alphabet, or a mix of both. 

By Russian standards, Federov’s videos are extreme, including explicit jokes about gay sex and drag impersonations of prominent state officials. In one of his most popular clips, he plays out a scenario in which he runs into an ex-boyfriend in a Gulag camp after being jailed for LGBT activism.

Mainstream opposition figures have also jumped on the TikTok bandwagon, albeit begrudgingly. Alexey Navalny created an account this summer. Counterintuitively, he documented the process on YouTube, posting a video in which he said, “I’m not even sure why I’m doing this.” 

Denis Kaigorodov, a 20-year-old TikToker from Tambov, central Russia, is part of an account named @fear_patriots, along with a group of local amateur comedians. 

His top-performing post so far was made in June. In it, he called for voters to come out against a controversial amendment to Russia’s constitution that would allow President Vladimir Putin to remain in office for life. Although thousands of young Russians flocked to the platform ahead of the polls, the amendment passed with 78% of the vote, prompting widespread allegations of electoral fraud.

Since then, Kaigorodov, who has 240,000 likes on his account, has made a series of comic videos mocking the result. In them, he accuses officials of stuffing ballots and states that members of the electorate cast repeated fraudulent votes.

“Political jokes do really well with audiences,” he explained. “People start to think you possess some kind of special courage if you’re making a joke like that in Russia.”

This reporting was supported by the Russian Language News Exchange.

The post Is TikTok’s free-speech honeymoon over in Russia? appeared first on Coda Story.

Published by Australia’s Strategic Policy Institute, the report describes how China’s official line on the humanitarian crisis has been promoted to TikTok users around the world. Of the top 20 videos on TikTok’s Xinjiang hashtag, only one is critical of the Chinese Communist Party.

The report, which also looked at how the video-sharing platform’s LGBTQ hashtags in the Middle East and Russia are being shadow banned, studied how the app treated the program of oppression, surveillance and control currently underway in China’s northwest region.  It described TikTok as “a powerful political actor with a global reach,” with the ability to covertly control flows of information on its platform. 

While the subject of Xinjiang and China’s oppression of Muslim minorities is widely discussed across other social media platforms such as Facebook and Twitter, the Xinjiang hashtag on TikTok, owned by tech giant Bytedance, is noticeably free of criticism. A scroll through TikTok’s #Xinjiang videos shows users glossy propaganda videos and happy vlogs made by state media-linked accounts, Chinese influencers and production companies, while content that’s critical of the regime is relegated to the bottom of the feed. 

When researchers at the Australian institute analyzed all 444 videos on the Xinjiang hashtag, they found that only 5.6% of videos were critical of the crackdown on the Uyghurs. Almost half of the top 100 videos were either propaganda, presenting Xinjiang in a highly idealized light, or outwardly pro-CCP. 

“Do not rely on the view of the world that TikTok provides you, because it is very distorted,” said Fergus Ryan, an analyst at ASPI and the report’s lead researcher. “The CCP has enormous leverage over this company, and their temptation to then use that leverage to subtly tweak the algorithms is going to be irresistible.”

Bytedance is subject to China’s security, intelligence, counter-espionage and cyber security laws. “Bytedance itself works hand in hand with public security bureaus in China to produce and disseminate propaganda,” said Ryan, describing how ASPI’s research showed TikTok’s feeds being flooded with content from its Bytedance-owned, firewalled-off Chinese counterpart, Douyin. TikTok denied cross-posting content from Douyin, but said that its users may be doing so.

Last year, TikTok was criticized for censoring discussions of Uyghur oppression after the company banned an Afghan-American political activist who posted a video discussing Xinjiang. 

“TikTok remains committed to creating a fun, authentic, and safe place for our users,” a TikTok spokesperson said in an emailed statement. TikTok did not answer questions about being used as a platform for Chinese state propaganda. 

As reporting from Coda Story pointed out in August, the Xinjiang hashtag feed is flooded with videos, hailing Xinjiang as a tourist destination, and featuring happy Uyghurs living an idyllic rural lifestyle. 

Following up on Coda’s coverage, the ASPI report identified several characters within the Xinjiang hashtag. One particularly prominent user, Jessica Zang, has more than 16,000 TikTok fans and is an employee of state owned China Global Television Network and a member of China’s ruling Community Party (CCP). “Have no idea that my video were on top under #Xinjiang… that’s new to me,” said Zang in a Facebook message, adding that she wanted to use her English skills to post on the international version of the app. “I just wanted to share the beauty and fun things in China.” Her videos proliferate on the #Xinjiang hashtag, showing snippets from a recent trip to the region during the pandemic. 

Another account on the hashtag is @guanvideo, a Chinese production company that also produces videos for Beijing-owned Global Times.

The Xinjiang hashtag also features videos from Uyghurs within China, such as a young woman called @aygul_uyghur, whose account describes her as “just a simple girl from Xinjiang.” Her most popular video has racked up more than 100,000 views. In the past, Uyghurs living in exile have pointed to social media accounts like these as clearly propaganda-oriented, for the simple fact that they are allowed to post to western audiences. 

For many Uyghur citizens in Xinjiang, using a virtual private network in Xinjiang (a necessity to post on TikTok) is a shortcut to arrest and detainment, though there appear to be exceptions for users who use the video-sharing platform to promote the Chinese party line. 

“The result, even for TikTok users perusing the topic, is a depiction of Xinjiang that glosses over the human rights tragedy unfolding there and instead provides a more politically convenient version for the CCP,” the report said.

President Trump has given Bytedance a deadline of September 20 to sell its U.S. TikTok business and destroy all its copies of U.S. user data, or be banned in the U.S. In India, the app – which dominated the social media space in the country – was banned in June as part of an ongoing standoff between Delhi and Beijing, losing Bytedance 120 million monthly users.

An analysis by ASPI of Bytedance’s career page suggests that the company is continuing to hire workers in China to monitor international TikTok content. 

“Even if hypothetically they were able to completely sever the content moderation from China, that doesn't get to the core problem: What is happening in the algorithm,” said Ryan. “The algorithm is a total black box.”

The post Welcome to TikTok’s sanitized version of Xinjiang appeared first on Coda Story.

Organized by Austria’s Erste Foundation, which encourages citizens to become active members of society, the conference — “How the EU can take the lead in governing technology globally” — addressed the growing need for the regulation of large technology companies, such as Facebook or Google. 

“Trust is lost. Stakes are very high. We need to pull this all into public accountability,” said the keynote speaker, Marietje Schaake, former member of the European Parliament from the Netherlands. Schaake was speaking to the media at the sidelines of the conference. 

Silicon Valley’s giants stand accused of a plethora of far-reaching systemic flaws ranging from discriminatory race and gender practices in algorithmic decision making to enabling the use of micro-targeting to spread political propaganda. In the EU, the big tech platforms have come under increasing scrutiny with the bloc’s ground-breaking data protection regulation (GDPR) coming into effect last year. 

Europe is already home to the world’s strictest data privacy laws and Schaake is one of the most prominent voices arguing the EU can do much more. Having served as a Member of the European Parliament for a decade, the Dutch politician was behind some of Europe’s key digital initiatives, such as the Digital Agenda for Europe, which was originally conceived in 2010 to speed up the roll-out of high-speed internet in Europe. In 2017, she was singled out by Politico as one of the 28 most influential Europeans. After she resigned at the end of her parliamentary term this year, she was appointed the international policy director of Cyber Policy Center at Stanford University. 

Schaake’s argument contends that since companies like Facebook, Twitter or Google have effectively become the guardians of how information flows, people around the world have lost trust in information sources of any kind. She blames the advertising model whereby “paying instead of convincing” determines one’s position in the marketplace of ideas.  

“Governing for profit optimization allows blatant conspiracy to rise above news stories in search results,” Schaake said in Vienna. “Companies like Facebook allow their clients to advertise against categories such as ‘Jew Hater’, or ‘Hitler did nothing wrong’.” 

In an effort to counter micro-targeted disinformation campaigns, the European Commission proposed a new Digital Services Act earlier this year. These are mostly connected to Russian online influence operations in Europe. For example, the UK-based impact agency 89up reported that Russia spent up to $5.18 million on attempts to influence British voters to vote Leave in 2016’s UK’s EU membership referendum. 

The act is due to be presented at the end of 2020 and is widely expected to contain updated transparency rules on political advertising which would require big tech platforms to subject their algorithms to more stringent forms of regulation. 

More recently, the European Commission’s incoming president Ursula Von der Leyen pledged to create “a Europe fit for the digital age” by enhancing the Digital Services Act with proposals including ethical regulations for AI and new taxes for tech companies.

Algorithms and accountability 

Many experts in Europe today agree on two areas of regulation that could help alleviate the problem of manipulation on social media and micro-targeting for political goals: Content moderation and holding social media algorithms accountable.

The first would make social media companies liable for the content of their platforms. Algorithmic accountability would then force big tech companies to reveal the formulas they use to collect data and match them with advertisers. 

“The debate about intermediary liability is really at the heart of this question. Should companies like YouTube or Twitter be liable for what happens on their platforms?” Schaake asked journalists at the conference. 

Katarzyna Szymielewicz from the Polish NGO Fundacja Panoptykon, which has been engaged in legal battles pushing for the implementation of the right to explanation of automated decisions in Poland, sees a clear link between liability and algorithmic accountability.

“Liability doesn’t concern only content. It’s more about the way tech platforms moderate the discussion. The users might create the content, but the companies create the algorithm. They should have certain transparency obligations,” she said. 

A host of advocacy groups in Europe, but also increasingly more in the United States, who call for algorithmic accountability of big tech companies demand the introduction of legislation that would make the inner workings of the algorithms used by companies like Facebook or Google transparent. 

“It’s not like we want them to reveal all their codes as part of the solution,” said Szymielewicz. “They use hundreds of these algorithms on a daily basis; that would be impossible… We just want them to share with the public the decisions behind building these algorithms, how do they optimize them.” 

Schaake says that she has observed a change in the way companies like Facebook engage with the legislators. 

She says that just three years ago, Facebook was still in denial that the information flows on the platform could have any impact on countries’ electoral processes, let alone assuming any responsibility for the content shared by its users. 

“I remember sitting on a panel on intermediary liability back in 2016. I asked Facebook representatives if they thought their platform could somehow influence the outcome of the upcoming American elections. They told me: ‘don’t watch too many movies’”. 

Since then, she says, especially after the 2018 Cambridge Analytica scandal, Facebook is more aware about how the platform can manipulate users.  

“They [Facebook] now say they have a lot to learn. They say these things are very complicated, but that they are working on it,” Schaake added. “I think that they are really stressed out about all the pressure they are currently under, so they want to create an impression that they want to cooperate on this, but that’s very far from reality.” 

As the global debate about how digital technologies pose serious challenges to the integrity of democratic institutions gathers momentum, new hurdles have emerged. Facebook’s recent announcement that the company will not fact check political ads has led to calls for the voluntary suspension of online political advertising before next month’s UK election. The problems with digital campaigning has also cast a shadow on the 2020 U.S. presidential election.

The issue was given more prominence last week when the actor and comedian Sacha Baron Cohen gave a speech attacking Facebook and other social media platforms for enabling the spread of hate speech and misinformation. Speaking about Facebook’s decision to not fact check political ads, Baron Cohen said that under its own rules, the company would have allowed Adolf Hitler to run propaganda on its platform.

Facebook’s Asia-Pacific Policy Communications Director Clare Wareing said that the company has been working with experts and legislators across Europe on four areas where the company thinks regulation could help. 

“We are committed to working with experts and policymakers to shape the rules of the internet and have outlined four areas where we think regulation could help — harmful content, election integrity, privacy and data portability,” she said in an email. 

Since Facebook, like other big tech companies, considers its algorithms a trade secret, algorithmic accountability is not among the four areas cited. 

However, there have been initiatives on both sides of the Atlantic to release the codes from the so called “black boxes”. Earlier this year, a Democratic member of the U.S. House of Representatives Yvette Clarke introduced the Algorithmic Accountability Act, which would require large tech companies to audit their algorithms for potential abuses of human rights. 

Some version of the right to explanation is already enshrined in the EU’s GDPR, which mandates that users are able to ask for the data behind the algorithmic decisions made for them, including advertising programs, and social networks. 

Thomas Lohninger, the director of Vienna-based digital rights NGO Epicenter.works and a former policy advisor for European Digital Rights, says that the EU should be much more confident in going beyond the existing regulation.  

“Sadly, our politicians often feel like they are powerless when it comes to Google or Facebook,” he said. “But we have seen that the past two legislative terms in the EU have woken up to these issues.” 

After the wave of misinformation campaigns during the last EU elections, there is an appetite in Brussels to go beyond the existing regulation. The European Commission is currently carrying out an in-depth analysis into algorithmic transparency. The Commission will engage with experts, academics, industry, civil society and policy-makers throughout the project.  

“Two studies are currently under evaluation, focusing in particular on governance consideration, oversight and algorithmic auditing technologies,” the EU’s Digital Single Market spokesperson Inga Höglund said in an email. The results of the study are expected to be used in policy-making decisions about algorithms

“Preliminary work on the project has already informed several policy steps taken by the Commission, such as regulation on ‘platform-to-business’ trading practices, which establishes an obligation on online platforms and search engines to be transparent about the general parameters which determine online ranking, she added. 

Implementation requires architecture

Joanna Goodey who heads the Freedom and Justice department at the EU’s Agency for Fundamental Rights says that, whenever the forthcoming Digital Services Act is eventually enacted, the key thing will be its implementation. 

“You can have the best law drafted in the world, but the reality is the implementation,” she said during a panel discussion at the conference. “Do we have an architecture strong enough in the EU? One can argue that the data protection authorities have been grossly under-resources to deal with the GDPR.” 

Schaake says that so far it has been the tech companies who have shown more initiative in setting rules and standards.

“As soon as anti-trust investigations were announced in Europe and the U.S., we’ve seen tech companies hiring more lobbyists for their campaigns,” she said. “They are also funding a lot of civil society organizations, as well as many academic programs, that are working on tech policy.” 

She continued: “The question is then not whether we will see rules and regulation, but who is in charge of it and based on which principles and desired outcome.”

The post At the forefront of Europe’s battle for tech transparency appeared first on Coda Story.