How Amazon’s algorithms push people towards extremist content
A recently released report highlights how Amazon’s book recommendation algorithms can lead people to literature about extremism, white nationalism, and conspiracy theories, including QAnon and Covid-19 disinformation.
The study, published in April by the Institute for Strategic Dialogue, a UK-based think tank researching extremism, analyzed Amazon’s book recommendation algorithms for literature about white nationalist and far-right content, coronavirus denialism, and conspiracies about QAnon and fraud in the 2020 U.S. presidential election.
The analysis found that the platform’s recommendation algorithms, which suggest titles and authors to people browsing through books on the platform — while relatively harmless for most users — can create a disturbing pipeline into extremist literature, directing customers toward conspiracy theories and white nationalist literature.
The study found Amazon’s algorithmic recommendations “could serve as a gateway into a broader universe of conspiracy theories and misinformation, or to increasingly radical far-right and white nationalist content.”
Amazon’s recommendation algorithm suggests books to users by telling them what other customers who clicked on the book bought and viewed. This algorithm, according to the ISD, creates a feedback loop. People who click on a book about a conspiracy are fed recommendations for the same and other conspiracies, including books about QAnon, debunked claims about vaccines and fraud in the U.S. 2020 presidential election.
A descent into this algorithmic dystopia could take you to the book page for The Hammer is the Key to the Coup: How Obama, Brennan, Clapper, and the CIA spied on President Trump, General Flynn … and everyone else, written by the proponents of a debunked election fraud conspiracy about the 2020 U.S. presidential election. The page suggests links to books about the Illuminati, the Rothschilds, the “Scamdemic,” Pizzagate, and aliens.
“I was shocked by the cornucopia of weirdness in the recommendations,” Elise Thomas, the author of the report and an ISD analyst, told me. She was especially alarmed to see books on the platform by the founder of the Order of Nine Angles (O9A), a UK-based Nazi satanic group that has been tied to a number of terror offences, prompting calls to outlaw it as a terrorist organization.
“These beliefs are very extreme, and I was genuinely quite taken aback to see them just sitting on Amazon,” she said. “As a result of doing this research, I learned about other 09A texts I didn’t previously know about because of Amazon’s recommendations. I think that’s an example of the potential harm here.”
Other examples directing readers to extremist literature abound. Click on the book Whiteness: The Original Sin, and recommendations include titles about European ethnostates, “white identity politics,” and New World Order conspiracies.
Amazon recently removed the 1978 white supremacist novel The Turner Diaries, but a search for the book will now direct people to The Anarchist’s Cookbook, a bomb-making guide published in 1971 by William Powell that has been linked to acts of violence including the Columbine shooting and the Oklahoma City bombing.
An Amazon spokesperson said the company takes “concerns from the Institute for Strategic Dialogue seriously and are committed to providing a positive experience for our customers. Similar to other stores that sell books, we provide our customers with access to a variety of viewpoints and our shopping and discovery tools are not designed to generate results oriented to a specific point of view.”
While any conversation about banning books is complex, Thomas says an overhaul of Amazon’s algorithm for extremist books could offer one solution. This would involve turning off the automatic recommendation of books promoting conspiracy theories, disinformation, or extremist views.
“The books are still on the platform, you can find them if you search for them, but if you search for one and go to that page it’s not going to recommend you 20 others,” she said.
The ISD report marks the most recent investigation into Amazon’s search algorithms. Previous analyses have also examined the role played by the digital giant’s search algorithms in leading customers into conspiracies and misinformation. A January 2021 study by researchers at the University of Washington, for example, revealed that nearly 10.5% of searches involving the term “vaccine” promote books containing anti-vaccine conspiracies and health misinformation.
The story you just read is a small piece of a complex and an ever-changing storyline that Coda covers relentlessly and with singular focus. But we can’t do it without your help. Show your support for journalism that stays on the story by becoming a member today. Coda Story is a 501(c)3 U.S. non-profit. Your contribution to Coda Story is tax deductible.
The Big Idea
China's repression of journalists: no more borders, no more constraints
Why targeting ethnic minority journalists is central to China’s crackdown on the press
China’s crackdown on Uyghurs reaches the Arctic
China ordered a Uyghur journalist extradited to Xinjiang. His wife has taken to the Istanbul streets to stop it
China’s Global Dragnet
Immersive simulation attempts to pierce apathy over the Uyghur genocide
Threatened, harassed, punished: The Uyghur translators defying China to tell Xinjiang’s story