It starts with a simple search term in the Department of Justice’s Epstein Library. “Blue eyes.” Hundreds of results. Jeffrey Epstein’s international trafficking agents send him pictures and descriptions of blue-eyed young girls: potential victims to be dispatched to his various homes. “I spotted two skinny blond blue eyes 21 years old ladies in Monaco last weekend and asked them for CVs,” one agent, whose name has been redacted, wrote. “Trying her best to move from her small town to Moscow; English isn't great. Could be fun for Paris, blue eyes,” wrote another. “Can't understand if her breast is real. Otherwise very pretty and sweet…Very blue eyes as we like.” 

One of Epstein’s victims wrote of being chosen for her eye color in a journal entry later shared with federal prosecutors. "Superior gene pool?!? Why me?" she wrote, describing Epstein's worldview as "Nazi like." "It makes no sense. Why my hair color and eye color?" 

Epstein — himself blue-eyed — seemed to prefer both his victims, and the people he bankrolled, to have blue eyes. “All of my fundees have blue eyes,” he boasted in one email. In the entryway of his Manhattan townhouse, he displayed dozens of prosthetic eyeballs in a frame. Epstein made notes and sent article links to his contacts asking if having blue eyes meant you were more intelligent or a “genius”. He even had a list of scientists and tech leaders with blue eyes — including Elon Musk, Peter Thiel, and Google’s Ray Kurzweil. “Total — 70 people Blue eyes — 41 Unclear (might be blue, but not 100% sure)” the list says. Appearing in the files — whether on this list or elsewhere in Epstein's records — does not connote legal wrongdoing.

Going deeper into the files, Epstein and his network of contacts discussed beliefs about how physical characteristics and race might denote intelligence. They exchanged emails about population control. They spoke of engineering women’s sex drives, building designer babies, and living in a world full of superintelligent humans that could merge with robots. They spoke of getting rid of the elderly, the infirm, and the poor.

The files offer a glimpse into a world where ideas about eugenics and race science have never gone away. On the contrary, they run through our elite universities, through the most powerful companies in Silicon Valley, and through the tech industry itself. Epstein’s was an exclusive club that counted among its members people who harbor dreams of re-engineering human minds and bodies, seizing control of our collective future, and building technology that, they hope, will one day merge with — or even replace — all of us.

In 2002, two decades before the launch of ChatGPT, Epstein hosted an Artificial Intelligence summit on his Caribbean island. In the years that followed, he cultivated close, regular contact with a network of  (predominantly male) scientists, researchers, academics and tech leaders working at the vanguard of AI, biotech, genetics and cognitive science, meeting them at universities like Harvard and at his various homes. 

In August 2018, a year before Epstein was found dead in his jail cell, he was in email correspondence with software consultant and bitcoin investor Bryan Bishop about funding a project to create “designer babies” — children with genes cherrypicked for their looks, health, strength, immune systems, sleep needs and even, in Bishop’s imaginings, abilities to live on a different planet. 

  “Attached is the doc you requested, it's the "use of funds" spreadsheet for the designer baby and human cloning company,” Bishop wrote to Epstein. “This gets us out of our self-funded ‘garage biology’ phase to the first live birth of a human designer baby, and possibly a human clone, within 5 years. Once we reach the first birth, everything changes and the world will never be the same again.”

Bishop went on to discuss how his ultimate ambition was to make “practically unlimited modifications to the cells before generating an embryo.”

In response to a request for comment, Bishop sent Coda a publicly available set of answers to frequently asked questions about designer babies.

“The reason people have an aversion to eugenics, and rightfully so, is because countries used genocide and sterilization to prevent reproduction by populations that they didn’t like. We have no intention of doing anything of the sort,” Bishop writes in the public FAQ. “‘Designer baby’ simply describes a child whose genome has been intentionally altered or chosen by their parents, rather than left entirely to the genetic lottery of natural conception.”

“It’s such a great subject,” Epstein responded after he read Bishop’s proposal. “We need to get a read on legal. Can’t do anything where US rules apply to US citizens regardless of where [they are].” 

Building a super-race of humans, and parachuting humanity into a different evolutionary era — or even obsoleting the human race as we know it — is a running theme in the Epstein files, and an increasingly prominent ambition for tech evangelists today.

“It’s eugenics all the way down,” said Jacob Metcalf, a founding partner at Ethical Resolve, a consulting firm working with tech companies to develop their ethics protocols. A common fantasy in tech circles, he said, is “to essentially control human destiny. And a lot of the times that human destiny is for humans to be replaced. That's the really bleak thing here. What could be more eugenic than getting rid of humans.”

In 2008, Epstein began conversations with the computer scientist Ben Goertzel. Over the years, Epstein would send Goertzel more than $360,000 to fund the researcher’s plans to build towards Artificial General Intelligence (AGI), a term Goertzel himself popularized.

“I remain eager to move forward on working together to accelerate progress toward a human-obsoleting thinking machine,” Goertzel wrote to Epstein in May 2008. Eighteen years on, and the idea of obsoleting humans with artificial intelligence is widely discussed in the tech world.

When asked to comment on his exchange with Epstein, Goertzel told Coda: “I do think we will create forms of transhuman intelligence going beyond the scope of humanity as we know it, but I also very much hope and envision a strong role for humans even after this happens.”

Goertzel went on to describe a future where the world reaches the “Singularity” — a Silicon Valley buzzword signifying a tipping point where AI surpasses human intelligence. “I do think AI will eventually gain its own superhuman autonomy, but I think this can happen in a way that respects and nourishes human life rather than being harmful to it,” he said. “Epstein and I discussed this face to face a few times and indeed I was a bigger fan of the human species than he was, and more optimistic about its flourishing post-Singularity.” 

In an email to Epstein, Goertzel laid out a scenario where AI systems would start running their own economic activity. He envisioned this Artificial Intelligence economy acting as a “parasite to overcome the regular human economy” that would eventually “gain its own superhuman autonomy.” The ideas Epstein and Goertzel exchanged mirror a broader conversation unfolding in the tech world that imagines a future where ultimately, human labour could be rendered superfluous, and ultimately be replaced by artificial intelligence and robots.

Together, Goertzel and Epstein also discussed modifying human brains — a concept popular in Silicon Valley today, where numerous brain-computer projects are researching ways to cognitively enhance the human brain, and alter human personality, memory, and mental capabilities.

In 2008, when Epstein told Goertzel he was “off to jail” for a year, after he was convicted of soliciting a minor for prostitution, Goertzel suggested the solution to his problems might one day be solved if human brains could be re-programmed.

“According to my understanding, the girls you were involved with were old enough to know what they were doing, so society really has no ‘moral right’ to lock you up,” Goertzel wrote to Epstein. “This is a fucked-up society we live in. But past ones have really been no better -- the fault is really w/ the human brain architecture, which is precisely what I'm aiming to supercede in my AGI work.”

When asked to comment on these remarks — and in particular the implication that Epstein’s problems might be solved if his accusers' brains were one day re-engineered — Goertzel told Coda: “This was a general observation that the messed-up nature of our society generally is rooted in the way our brains have evolved... and that advanced tech will let us modify our brains to make ourselves and thus our society better.  There was no implication intended (nor stated) that women’s brains are any more or less messed up or in need of improvement than men’s.”

Goertzel reflected that his comments on Epstein’s victims being “old enough” were “regrettable and unfortunate in hindsight,” adding that his impression was that Epstein had been involved with adult women, not “disgustingly curating high school students for sexual purposes. I should have paid more attention.” 

In 2013, three and a half years after Epstein was released from jail, Goertzel approached Epstein for funding to build a “toddler robot”. Given Epstein’s criminal history of abusing minors, this has inevitably attracted attention online. “When we were discussing measuring the IQ of robot toddlers, the topic was never sexualized in any way,” Goertzel told Coda when asked about the project. “While I had nothing to do with Epstein's perverse sexual tastes or abuse of women, what I have read about his awful doings in the newspapers relates to his interest in teenage girls not toddlers.”

Epstein was particularly interested in funding projects that built — like Goertzel’s –- on transhumanist theories. Transhumanism is a worldview that captivates many of the most prominent tech leaders in Silicon Valley today. It believes in a future when the human body can be endlessly altered, genetically engineered, and ultimately fused with artificial intelligence. 

“Transhumanism is a much more radical concept than eugenics,” explained Timnit Gebru, a computer scientist and researcher who has written extensively about eugenicist ideas within artificial intelligence. “In eugenics, you're trying to create a more superior human by breeding humans through generations. In transhumanism, you're trying to get rid of humans altogether.”

For transhumanists, she added, “their idea is to get rid of any undesirable properties they see with humans."

Perhaps the most well-known proponent of transhumanism in the Epstein files is Peter Thiel.

“I think you would prefer the human race to endure, right?” New York Times journalist Ross Douthat asked Thiel last year. “Uh—,” Thiel said. “This is a long hesitation!” Douthat said. “Should the human race survive?” “Yes, but I would like us to radically solve these problems,” Thiel said. “We want you to be able to change your heart and change your mind and change your whole body.”

Thiel’s name appears in the files more than 2000 times, and Epstein reportedly invested some $40 million into Valar Ventures, a firm co-founded by Thiel. The two spoke of building secret societies and shared an interest in transhumanism and cryogenics — Epstein wanted to freeze his brain and penis when he died, so that one day he could be revived, while Thiel has also stated his body will be frozen after his death. 

They also appeared to share an interest in bringing an end to the democratic systems of today, imagining a different system altogether. Epstein, for his part, spent his life puppeteering the most powerful people in the world and undermining democratic systems. Thiel, meanwhile, first expressed his own anti-democratic views in 2009 when he wrote: “I no longer believe that freedom and democracy are compatible,” adding that since women were allowed to vote, the notion of a capitalist democracy became impossible. When the Brexit vote came through, Epstein wrote to Thiel: “Brexit, just the beginning.” Thiel asked — “of what”; Epstein said – “Return to tribalism, counter to globalization, amazing new alliances.” 

Globalization — and the idea of internationally powerful governing bodies — is a worldview that both Epstein and Thiel seemed to distrust. In March, in a palazzo in Rome, a stone’s throw from the Vatican, Thiel gave one of his infamous lectures in which he espoused his views about an “antichrist” that gets in the way of technological progress. This antichrist, he suggested, could be an internationally powerful body; the product of globalization. I stood outside the palace as attendees — priests, students, researchers — mutely hurried out, refusing to speak to the cluster of reporters waiting for Thiel’s black Mercedes. 

“He has a totally irrational side, which lives on fear, of what danger might happen,” one audience member told me of Thiel on condition of anonymity, recalling how, up close, Thiel looked haunted and ill. “His head is full of future scenarios, which is what’s killing him. I think he’s scared.”

Thiel did not respond to multiple requests for comment. 

Epstein didn’t confine himself to lofty conversations about a future collapse of the global order or re-engineering humanity. He also had ambitions for his own personal eugenics project. In 2019, it emerged that he wanted to seed the world with his DNA — and reportedly have 20 women impregnated at a time at Zorro ranch, his New Mexico property.

Epstein tried to recruit Virginia Giuffre for this very project. He “fantasized about improving the human race by fathering children who carried his superior genes,” she recounted in her memoir, published posthumously late last year.  “He’d talk about using his Zorro ranch as a literal breeding ground to propagate babies.” When Giuffre was 18 years old, she recalled, Epstein asked if she would carry his child and hand over all legal rights to it – “like a modern-day handmaid.”

In a haunting diary entry from another Epstein victim, written between the ages of 16 and 17 and shared with federal prosecutors, a girl describes being told she will be sent to Zorro ranch — possibly to participate in the very same project. “Go to New Mexico? What in the hell? This makes no sense. What about school?” she writes, describing how Epstein chose her for her hair color and eye color, and tried to convince her she would create “perfect offspring.”

The teenager chronicles her pregnancy, pasting a sonogram into the scrapbook, before giving a traumatic account of giving birth with Ghislaine Maxwell beside her. “Ghislaine said to push all the pain away. I don't understand. Blood and water all over the bed.” As the baby was born, she writes, Maxwell covered her eyes. “I saw between her fingers this tiny head and body in the doctors hands.”

The girl describes hearing the baby’s “tiny cries” before “they took her.”

“I’m nothing but your property and incubator,” the teenager writes of Epstein. The diary is a terrifying piece of evidence that appears to link to Epstein’s longstanding fixation with creating genetically bespoke humans. The diary author’s lawyers, Wigdor LLP, declined to comment.   

Epstein’s fever-dreams of creating an army of children carrying specific genes reflect a broader trend of “pronatalism” — a movement historically tied to eugenics — that’s thriving in Silicon Valley.

 Millions of dollars of funding are currently being poured into projects creating “superbabies,” while billionaire tech oligarchs including Elon Musk — whose name appears more than 1000 times in the files — reportedly want to use surrogates “to reach legion-level before the apocalypse.” Musk did not respond to requests for comment.

In the files, women appear either as victims, as objects, or as vessels for genetic engineering experiments. They are an inconvenient reality, people to be controlled and re-booted. Epstein wrote a 2013 email implying that women “are like shrimp. You throw away the head and keep the body.” 

“The obsession with "artificial" life appears tied to a masculine desire to try control the production of life – ultimately ridding themselves of their dependency on women," said Gabriella Razzano, Co-Founder of OpenUp, a social impact tech lab based in Cape Town, who is also a senior advisor at the African AI Observatory. “I think there is important work to be done on tying the narratives that are very revealing in the Epstein files to understand how, and why, technology is being developed as it is.” 

The trading of ideas about intelligence — both artificial and human — takes a particularly sinister turn in a 2016 exchange between Epstein and the cognitive scientist and AI researcher Joscha Bach, whose research Epstein funded to the tune of $300,000.

Bach writes to Epstein about a study claiming that “black children outperform white children in motor development, even in very poor and socially disadvantaged households, but they lag behind (and never catch up) in cognitive development even after controlling for family income.”

Epstein responds with racist ideas about his notion of how to “make blacks smarter”, adding — “maybe climate change is a good way of dealing with overpopulation. The Earth’s forest fire. Potentially a good thing for the species,” before contemplating a world with “too many people,” where “many mass executions of the elderly and infirm make sense.” 

Epstein then imagines creating a future “Übermensch” — a superior human with cherry-picked attributes. “What I like is the idea that ubermensch could be the melding of humans, put together in one brain,” Epstein writes. This bespoke human, he suggests, would include traits from marginalized groups, who he appears to believe have a stronger awareness of how to navigate power structures because of their historical exclusion. “An increased motor system, an increased awareness, an increased status calculator (Blacks, jews, women). Ubermensch could be the combination of the best of humans, not the best of a specific race or gender. Fun idea.” 

Bach told Coda in a statement: “I was summarizing a scientific study in a private email. Studies like this get often abused in ideological discourse to justify discrimination, which I strongly oppose and condemn.”

“I am firmly opposed to any form of racial discrimination, and I reject the use of group-level statistical claims to make judgments about individuals or to justify unequal treatment.” 

He continued: “It goes without saying that if global warming were to lead to a reduction in the human population, it would be accompanied by immeasurable suffering. Our civilization would break down, leading to a return to dark ages, in which the elderly and infirm were often killed, because people could not support them, and often did not care about supporting them. Every reasonable person understands that this is horrible and not desirable in any way.”

Epstein “was often callous about human suffering in a way that I found disturbing but worth understanding, as a window into the perspectives of the rich and powerful,” Bach added. 

Alongside Epstein’s conversations about mass executions for the old and and the sick, he was also interested in Silicon Valley’s dream concept of living forever — he had numerous email conversations with the longevity guru Peter Attia about prolonging his own lifespan, and funded a Harvard project geared towards “the end of aging.” In an email to Attia, Epstein mused: “I’m not sure why women live past reproductive age at all.” Attia, who published a statement about his relationship to Epstein, did not respond to requests for comment. 

This interest in “longevity” — living for as long as possible, even living forever, is popular among the elite precisely because they find themselves in an elite class, says David Robert Grimes, a scientist and disinformation expert who has written about longevity and race science in Silicon Valley. “They're both sides of the same coin — the Silicon Valley eugenics, and also the longevity stuff. They promote an idea that ‘we are exclusive and we are special',” he said. "It helps them to justify deep social inequality."

The tech elite did not inherit this ideology by accident. Stanford University, the intellectual heart of Silicon Valley, was once a major hub for the American eugenics movement, which later helped to inspire Nazi race laws. Stanford’s founding president, David Starr Jordan, was a prominent eugenicist, campaigning for forced sterilization of people with undesirable genetic traits. The university removed his name from its buildings in 2020 — but in Palo Alto, his beliefs did not disappear with the nameplate.

"Instead of eugenics we just call it longevity or biohacking," Christopher Wylie, the Cambridge Analytica whistleblower who has spent years investigating Silicon Valley's belief systems, said on a panel with me at a journalism conference last year. "It's the same."

The ideology Epstein bankrolled in private is being built in public. It’s a vision of the future in which a select few get to upgrade and extend their lives, while tightening their grip on the systems that determine which humans are worth investing in — and which are not.

It sounds like a dark sci-fi fantasy, except, as the files show, that fantasy is being funded and pushed into reality. Most of us will never be in the rooms where these ideas are discussed. All of us will live with the results.

Related Articles

The future according to Silicon Valley’s prophets

Isobel Cockerell

Peter Thiel is building a parallel justice system — Powered by AI

Nic Dawes

When I’m 125?

J. Paul Neeley

The post “All my fundees have blue eyes.” Epstein and the tech world’s dark ideology appeared first on Coda Story.

Sections:

We think of Silicon Valley as a nexus of tech moguls, innovators, power brokers and venture capitalists. But something bigger and more ideological is unfolding in the Valley — the building of an entire religion. Tech evangelists talk about Artificial Intelligence as if they’re building a higher power. Elon Musk believes AI will help us find a “digital God;” while biohacker and tech entrepreneur Bryan Johnson is adamant: “I think the irony is that we told stories of God creating us,” he said in an interview earlier this year. “And I think the reality is that we are creating God. We are creating God in the form of superintelligence.”

According to the tech prophets, the future is something the rest of us don’t have any control over — in part, they say, because we don’t understand the tech enough to have the power or the authority to regulate it, and in part because the prophets themselves don’t want to bear any responsibility for the products they create. So how should we think about Silicon Valley’s version of the future, what promises are they really making, and how can we regain control over the story of the future? 

This time two years ago, I was staying at an eco-retreat deep in the rainforest in Costa Rica. It was supposed to be a break from work — a time to unplug, recharge, sleep in a bamboo “pod” to the soundtrack of howler monkeys and toucans, that sort of thing. Instead, as often happens when I’m trying not to think too hard, I came across an interesting story. It began when I noticed my fellow retreaters all came from California. They were unplugging too: and arguably, they needed it more than me, because they all worked in tech. What I had thought was a rustic Costa Rican-owned eco-lodge was actually a favorite techbro getaway, founded by burnt-out former tech innovators, who had invested their money into helping their other burnt-out friends recover from burnout. 

Over my days in that steamy jungle, I learned that the place I was staying in often ran psychedelic retreats for venture capitalists, engineers, tech workers, and crypto-bros, and that the entire valley surrounding us was gradually being taken over by similar retreats. Parcels of land were being sold off to Californian buyers, with indigenous people pushed out before being invited back into “the space” to guide psychedelic rituals and help the tech bros unlock their “creative flow” and dream up their latest innovations.

Right now, Silicon Valley’s elite are obsessed with accelerating towards a future where the human race is re-engineered and the world’s resources are in the hands of a very few. After I got back from my trip, I couldn’t stop thinking about how psychedelics are being used to help some of the world’s most powerful tech evangelists build a vision of expanded human consciousness and fuel their ambition to build hyper-intelligent AI models, pushing them to accelerate towards evolutionary transformation, with all the problems and delusions that entails — and what that means for the rest of us. 

“Come watch me trip balls,” Bryan Johnson, the longevity entrepreneur (whose catchphrase is “don’t die”), proclaimed recently, before livestreaming himself taking a ‘heroic dose’ of magic mushrooms. Johnson, who believes the tech world is “building God with superintelligence” is determined to live until he can eventually merge with a machine and live forever. In recent years, he’s been trying myriad interventions to biohack his body — everything from injecting himself with his son’s blood plasma to taking over 100 supplements a day — in an attempt to live longer. Experimenting with psychedelics is his latest venture, but he’s far from alone in the tech world. OpenAI’s Sam Altman has publicly said a psychedelic retreat was “life-changing;” while Elon Musk says he has used ketamine for depression, and Google’s Sergey Brin has invested millions into a psychedelic research project.

Upon my return from Costa Rica, I spoke to Johns Hopkins psychedelic humanities lecturer Neşe Devenot, who described how, spurred on by psychedelics, the tech elite are building a conviction that they are “the chosen steward of technology to help transmute the current phase of humanity and consciousness into a new form.”

The thing is, while psychedelic brews like ayahuasca have been used in shamanic practices within indigenous groups for centuries, the practice has been hijacked by the tech world — not to forge a closer connection with nature, or to confront their own existence, but to imagine a future where we transcend nature, transcend death, and terra-form the planet with datacenters to power ever-expanding artificial intelligence systems.

“A tech bro on acid is still a tech bro — they just become a psychedelically amplified tech bro,” is how writer and media theorist Douglas Rushkoff put it to me last year. “These guys have a hallucinatory confidence over their plans. And they’re developing tech that is as potentially disruptive to civilization as nuclear weapons.” Here are some of the most psychedelically inflected visions for the future that the tech bros are building for us and, soberly, let’s also look at what the costs of those visions are.

We’ll live in Utopia* 

Believers: Jeff Bezos, Ray Kurzweil, Elon Musk

Tech leaders like Jeff Bezos and Ray Kurzweil promise us a solved world. They say that with the help of AI, we can hack our way back into paradise. Some talk about it as “the Singularity” — a world where AI is billions of times more intelligent than humans — and say we just won’t be able to predict or even conceive of what the future will look like once we build artificial intelligence that powerful. But the most optimistic tech evangelists believe it will be a kind of heaven.

“It is a renaissance; it is a golden age. We are now solving problems with machine learning and artificial intelligence that were in the realm of science fiction for the last several decades,” says Amazon CEO Jeff Bezos. “By the time we get to the 2040s, we’ll be able to multiply human intelligence a billionfold. That will be a profound change that’s singular in nature,” adds computer scientist Ray Kurzweil, who has written extensively on the Singularity.

In our podcast Captured, tech workers described what their utopia might look like from their San Francisco condos: “I see a city filled with gardens, filled with communities, a place where people can raise their kids together, a place where people can find a place to belong. And maybe there's sci-fi elements to that,” engineering physicist Andrew Cote told us, staring out over the horizon.

The catch: But once everything is solved, what will we do with our time? Philosopher Nick Bostrom asks us to imagine what Utopia would actually look like — and whether it’s something we actually want: “Imagine we have all this technological abundance, and we’ve somehow managed not to use it to oppress one another or wage war, but have some reasonably good arrangement. What would human lives be like?” Well, for one thing… 

We’ll live forever*

Believers: Bryan Johnson, Peter Thiel 

Talk to anyone in Silicon Valley right now and they’ll wax lyrical about ways to live forever. At present, they accept it’s medically impossible — but they believe the day is coming when technology will let us transcend our bodies.

“I’m basically a brain with limbs… the rest is kind of undifferentiated,” said AI builder Kyle Morris when speaking to us for Captured, showing us the vast range of supplements he took to live long enough to see a technological shift where we’ll be able to merge with machines and continue to consciously live beyond the limits of our bodies. Bryan Johnson, tech CEO and leader of the “don’t die” movement, has experimented with injecting his son’s blood plasma into his veins in a bid to live longer — though he says it didn’t really work.

The catch: *Not everyone will live forever. Only those who can afford it. “I suspect we're going to see a class divide between people who can live hundreds of years and people who live less than 50. That’s going to be a civil war of some sort, I would anticipate,” Kyle Morris told us.

We’re all going to die*

Believers: Elon Musk, Daniel Kokotajlo, Effective Altruists

This might seem contradictory, but in San Francisco it makes sense: there are two camps — those who believe AI will allow us to live forever, and those who believe it will kill us all. There’s also people who believe both outcomes are a possibility. Elon Musk, for example, says there’s “only a 20% chance of annihilation” by super-powerful artificial intelligence programs.

While reporting for Captured, we spoke to Effective Altruists protesting outside Meta: “Pause AI because we don’t want to die!” they chanted. Earlier this year, a group of AI researchers released AI2027, a piece of science fiction charting the rise of runaway artificial intelligence, ending in a brutal showdown where every human is killed by an AI-activated biological weapon, and the Earth is terraformed by datacenters, laboratories, and particle colliders.

*Except the tech-bro survivalists. Tech enthusiasts — with money — believe their inventions could trigger a catastrophic event on Earth: a global pandemic, climate breakdown, nuclear war, or AI apocalypse. They’re quietly prepping. Some are building bunkers in Montana. Others see New Zealand as the ideal bolthole. Peter Thiel has constructed a fortified estate there, designed as a survival outpost.

We’ll never have to work again*

Believers: Sam Altman, Mark Zuckerberg, Alex Blania

Tech leaders building artificial intelligence talk openly about how they’re transforming the entire economy. They tell us that the world of work, as we know it, may not exist for much longer. “Entire classes of jobs will go away and not come back,” is how OpenAI CEO Sam Altman puts it. Jobs as we know it will change forever. For Captured, we spoke to nurses who are already seeing chunks of their jobs taken over by artificial intelligence, and even a comedian who worries a day will come when AI starts writing her peers’ jokes. Already, entire industries are feeling the effects of AI takeover. But if we don’t have to work, how will we get paid? Silicon Valley has an answer for that too: Universal Basic Income, an old idea retrofitted for the AI age. The idea with UBI, is that we'll all get an allowance, a regular payment, no strings attached. That payment will replace income that would previously have come from a job. We traveled to Kenya to look at the prototype for one of these systems in action: a concept called World, that gives you a monthly allowance of around $50. In return, you must submit your iris biometrics to World’s database via a camera device called the Orb. When the Orb arrived in Kenya, there were enormous, chaotic queues at shopping malls, packed with people vying to submit their iris data and get onto World’s system and get hold of the handouts. 

The catch: Universal Basic Income sounds great in principle, but if you think deeper, it will completely change what it means to be human. If we don’t work, don’t pay taxes, then we as humans will no longer contribute to society and the economy. We’ll then become completely reliant on — and powerless against — the whims and wishes of those in power, with no way to protest, or strike, if they’re unhappy with how things are going. If we accept Silicon Valley’s vision of the future where we depend on handouts from our tech overlords, we’d concede our freedom, independence and autonomy to a new set of masters. Beyond that, it’s difficult to imagine what we would do all day — as a species — if we didn’t have to work. “If there's nothing we need to do–if we could just press a button and have everything done, like, then what do we do all day long? What gives meaning to our lives?” philosopher Nick Bostrom mused while speaking to us for Captured.

Nation states will not exist*

Believers: Balaji Srinivasan, Peter Thiel, Marc Andreessen

“Very few institutions that predated the internet will survive the internet,” Balaji Srinivasan, the former CTO of Coinbase, said in a lecture recently–and by that, he means governments, and countries themselves. After all, governments come with a whole host of irritating traits that tech leaders loathe–they regulate companies, make them pay taxes, tell them what they can and can’t do. Why not secede, then, from those countries entirely, and build your own? Srinivasan is one of the leading thinkers behind the idea of a “networked state” — a successor to the nation state, built and enabled by tech. 

Proponents of the networked state dream of having digital statehoods; “startup nations” where they’ll be free of taxes and regulations, free of the bureaucracy of living in, well, a traditional country. They’re already doing it: pushing to draft legislation to create “freedom cities” in the U.S. — something Trump’s 2024 campaign proposed, enclaves unshackled by federal law where tech engineers can try out startups and clinical trials free from regulation or approval from federal agencies. Meanwhile on an island off the coast of Honduras is Prospera, a semi-autonomous “private city” backed by Sam Altman, Marc Andreessen and Peter Thiel, that’s marketed as a libertarian fantasy utopia. 

The catch: The idea of getting rid of stifling government bureaucracy and living in a world without borders is an idealistic dream held by many people, not just tech leaders. But, as the Silicon Valley elite envisions it, we would replace sovereign nations with a collection of private, giant gated communities that would hoard resources, money, and power, while locking everyone else out. A world where democracies no longer exist and elected leaders are replaced by digital moguls would be a world that serves clients, not citizens, and cares only for profit and innovation, a world where international human rights laws are thrown out. 

We’ll spread out into the stars*

Believers: Elon Musk, Jeff Bezos, Richard Branson

But what if we could take this idea of building crypto-states further — and leave Earth entirely to build Silicon Valley outposts on Mars, or on the moons of Jupiter? Not only transcend our bodies, but transcend the Earth itself — after all, if we can’t fix the planet, we can just leave it. Jeff Bezos talks about moving “all polluting industry into space” and leaving Earth as a nature reserve — one of the tech industry’s many technofixes for climate change. And all of Elon Musk’s ventures, from Tesla to X, are designed to support his ultimate mission: making the human species “multiplanetary.”

“They want to ensure the light of consciousness persists by reducing the probability of human extinction,” says Émile P. Torres, a philosopher who used to be part of what they call the emergent “cult” of Silicon Valley. Torres told us about the tech bros’ vision of a utopian future where humans conquer the universe and plunder the cosmos. It sounds like something out of science fiction — and indeed it is: when we visited AI frat houses during our reporting for Captured we found bookshelves stuffed with science fiction about space and colonizing the universe. 

Harvard historian Jill Lepore has a different way of seeing it — she calls it “extra-terrestrial capitalism,” mimicking a colonialist vision of expanding indefinitely, taking our extractivist mindset into the stars. 

The catch: Not everyone will be able to travel into space — or perhaps, not everyone will be able to stay on Earth. If you read enough sci-fi, and listen to enough conversations in Silicon Valley, you can envision all sorts of different outcomes: Mars becoming a penal colony filled with slave workers extracting resources; Mars becoming independent from Earth; only the super-rich and elite able to leave Earth as the planet burns. In Musk and other tech-bro survivalist visions of the future, they imagine a global pandemic, climate meltdown or nuclear war extinction event — perhaps thanks to the runaway Artificial Intelligence they themselves built — and see space as the ultimate off-ramp for a chosen few. 

“It’s important to get a self‐sustaining base on Mars… because it’s far enough away from Earth that it’s more likely to survive than a moon base,” Musk told the audience at South By Southwest in 2018. “In the hopefully unlikely event that something terrible happens to Earth, there’s a continuance of consciousness on Mars. One of the benefits of Mars is life insurance for life collectively,” he said this year. 

We’ll have all human knowledge in our brains*

Evangelists: Elon Musk, Bryan Johnson

Why bother with school when you could install a chip in your brain? Right now, tech leaders are working on building chips — like Musk’s venture, Neuralink — that we can insert in our brains, so that one day, we can merge with machines. When we met engineers in San Francisco, they told us about their ultimate ambition: to put all human knowledge inside human brains, from birth. 
“That’s the purpose of the education system, right?” said Jeremy Nixon, the founder of AGI House, which brings together AI workers into a houseshare in San Francisco.

But why not skip over all that and simply install a chip into our brains, so that even from birth we can know everything, all at once. Imagine, we’ll be able to speak every language on Earth, we’ll know all of human history, all of science. Ok, we might not be able to discover anything new — but our future will be boundless. “You hold your phone and it’s like a better prefrontal cortex. It tells you how to get places, tells you how to plan. It gives you answers. It gives you a better memory. I see in the next 50 years, that's going to enter us, that's going to become part of us,” Kyle Morris, another member of the AGI House, told us. 

The catch: Not everyone will necessarily be able to get this supersonic brain — and those enhancements will only come to those who pay. So, as tech leaders see it, could there one day be an underclass of people who can’t afford — or don’t want to install — these brain enhancements? And will those with enhanced brains then oppress those without them? Just as the world is becoming harder and harder to navigate now without a smartphone, perhaps in the future it will become harder to navigate without a chip in your brain — will you be able to travel, move freely, do simple errands? Last week, Mark Zuckerberg said that people without smart glasses like Meta’s model, that give them instant and constant access to an AI assistant, will be at a cognitive disadvantage. 

Climate change will be fixed by tech*

Evangelists: Larry Page, Elon Musk, Bill Gates


There’s an idea we came across while reporting in Silicon Valley that climate change, while problematic, is nothing much to worry about, because one day soon it too, like everything else, will be fixed by some technological intervention. Perhaps we’ll geoengineer the skies to create “sunscreen for the Earth” (as one pair of tech evangelists-turned-guerilla geoengineers dubbed it); perhaps we’ll finally figure out nuclear fusion (that’s a favourite prediction in Silicon Valley circles), or we’ll figure out how to get our oceans to sequester carbon. In November, Elon Musk proposed that “A large solar-powered AI satellite constellation would be able to prevent global warming by making tiny adjustments in how much solar energy reached Earth.” Though artificial intelligence datacenters suck up vast quantities of water and spew carbon into the atmosphere (Google’s newest datacentre in the UK will emit 570,000 tonnes of CO2 a year, according to planning documents), the tech leaders tell us: we’ll figure out the answers sooner or later; or AI will do it for us. 

The catch: Geoengineering, while a favorite pipedream of tech enthusiasts, could have unpredictable, and Earth-shattering consequences. Climate experts say processes like these could throw Earth into deeper chaos by cooling the world unevenly and wreaking havoc on our climate systems. And once we start the process of solar geoengineering, we won’t be able to stop — we’ll have to keep spewing chemicals into the atmosphere to cool down the sun, or face a rapid and catastrophic heating event. Who would even be in charge of geoengineering the planet; and who would decide if it was safe enough?

Related Articles

Silicon Valley’s sci-fi dreams of colonizing Mars

Isobel Cockerell

Who decides our tomorrow? Challenging Silicon Valley’s power

Natalia Antelava

When I’m 125?

J. Paul Neeley

The post The future according to Silicon Valley’s prophets appeared first on Coda Story.

There was no specific tipping point that made Logan Lane get rid of her smartphone. One day, the thought just arrived. “I was like, I just can’t fucking do this anymore.” And she put away the device that had dominated her life since she was 11. “I spent about five of my developmental years just tied to my smartphone,” she says. Logan, 20, bought a basic flip phone, and re-learned to navigate the world, without social media, GPS, and without the constant, nagging cry for attention from her smartphone that had punctuated her days.

She grieves the early adolescence she lost to her phone. “In the years when you’re supposed to be reading and playing, we were on our phones and computers,” she says. “We had those years of play stolen from us.” 

Lane is the founder of the Luddite Club, a solidarity network of “former screenagers” growing a movement across America. Together, they’re pledging to give up their devices, choosing instead a life of voluntary exile from the digital world. 

To speak to Lane, I placed an international call to her flip-phone — an act that already felt anachronistic. The line crackled as we talked and her train rattled through New York City. For a moment, the world felt analog again. 

Lane is part of the first generation with no memory of life before smartphones — a generation that became addicted to their phones before anyone truly understood the cost. “There’s no one person to blame,” she said. “Even though I was only 11 or 12 years old when I got a phone, I was responsible for facilitating this addiction in my life. But at the same time, I was a child.” 

All around Lane on the subway, all along the train — and along every train in New York City; every train in every major city in the world — people stared into their smart devices. The smartphone penetration rate for the world is about 60%; in the U.S. it’s at 91%. Just a decade ago, global penetration was 10%, but now many of us can’t leave a room, let alone the house, without our phones.

Rising in response is a resilient counterculture; a growing group of people who have had enough. People who long for a simpler, more three-dimensional life in which they have control over their digital existence, and their thoughts and data are not harvested, nudged, monitored. So they check out. Power off their smartphone; lock it in a drawer; give it away; throw it in the trash. Hope they’ll never have to use one again. The Luddite Club now has local chapters all over the U.S., and young people are flocking to the myriad offline events where they talk about reclaiming their lives from tech capture. 

“I’m excited to read on the train in peace, to not look at social media, post or check up on exes, looking for validation or a small dopamine hit. I’ll get dopamine the right way,” a young woman recently wrote on Reddit. “It will be difficult at first,” someone responded, “but it will become more freeing after you break your chains.” Another young man wrote that he had “just wasted ten years of my life living in an alternate reality.” Having made the switch, he called on others to “come back to the real world and enjoy the struggles and solutions of analog life.”

These conversations unfold in a radical corner of the internet where thousands of people a day come to discuss getting rid of their smartphone. The “dumbphones” subreddit has the intimacy of an addiction support group. The page is full of pictures of what people call their “everyday carry” gear, the tech they bring with them on a typical day. For people of a certain age, the pictures are transfixing, nostalgic: Motorola Razr flip phones, old Nokias, candy-colored iPod minis, notebooks, A to Z maps, point-and-shoot cameras, MP3 players. The photos hark back to a moment in time before everything — as the Luddites see it — started to go wrong.

It was, if you want to put a specific year to it, 2006. Facebook had just opened up its usership beyond students, and tens of thousands of users were signing up every day. Back then, Facebook reunited long lost schoolfriends, lovers, even relatives. Independent musicians blew up overnight on Myspace. Social media felt like something that would make people more open and connected. The first iPhone was still a year away. We still knew how to navigate our world without Google maps. We still read books on commutes, took pictures on cameras and uploaded them in their joyful hundreds to Facebook for fun. The 2008 crash hadn’t happened. Attention algorithms didn’t yet exist. The tech companies still felt like harbingers of a better, more connected future. 

Daisy Krigbaum, a dumbphone advocate who now runs a business around it, calls that era “the sweet spot.” It was a time, she says, “when online social platforms were there to facilitate in-person correspondence. They just filled the gap between when you could see somebody in person. You could talk to your friend while they’re abroad. You could talk to a family member who's bedridden. But then it evolved into a monster.”

The “sweet spot” is something Judy Estrin remembers well. One of the internet’s early architects, Estrin is a Silicon Valley veteran who helped build the foundations of the web in the 1970s. When I spoke to her at a sunny cafe in Palo Alto last year, she described the last days before technology stopped being built to cater to our needs. “It was human-centred,” she said of the internet back then. “It wasn’t until we got into the Cloud, mobile, social, that the dynamic shifted and it became more about humans adapting to the technology.” 

One thing that had kept tech companies in check, Estrin explained, was limitations on computing power. “There were constraints on the technology. We kept moving up against processing, bandwidth, storage.” But once computing power got cheaper, those constraints disappeared. “The culture changed,” she said. Instead of designing carefully, companies could just keep adding features. “The design aesthetic was these continuous scrolling feeds. The design of mobile became more and more massively online.”

She remembered how computer scientists started designing for mobile first. “We stopped having to think in terms of constraints. We just started brute-forcing everything.” And then tech began not to respond to our lives but to shape them. “It was in 2010, 2011, 2012,” Estrin said, “you could see the incentives of the system and the ad-driven markets just completely starting to shift things.” She said she felt guilty for not noticing this switch sooner — and for playing some role in the world Silicon Valley created; the world we all live in today. “ I did and do feel increasingly disappointed. Just disappointed with the technologies that we created,” she said. “ I think that I was so heads down and focused for so many years, between building companies and raising my son. And I think that I, then at some point, picked up my head. And it's like, well, why wasn't I paying attention to this stuff? What was I doing?”

For dumbphone business-owner Daisy Krigbaum and her partner Will Stults, the wake-up moment came on a transformative night in 2022. One night, after hours of scrolling beside each other on the couch, they finally looked up.

After basking in blue light and “looking at mindless stuff” for “an unreal amount of time,” Krigbaum said, they turned to each other and admitted they had a problem. 

They decided to forgo the tech that had been dominating their existence. First, like Lane, they had to come to terms with the time they had lost, and why. “ I think we both feel really grateful to have been born kind of on the cusp of the post-information age where we still had some foundational social skills,” said Krigbaum, who is 28. “I already feel impoverished by how much of my adolescence took place online.”

“Society's relationship with tech has at least migrated to the point where we're willing to admit that most or all of us have some sort of problem,” added Stults. “None of us have a completely healthy relationship with technology.” They started to look at flip phones and old-style cellphones to switch over to but found the experience of detangling their life from smartphones filled with knotty inconveniences, workarounds and sacrifices. 

Contemporary life is full of small dependencies that keep people tethered to their phones — apps for work, school portals, two-factor authentication, maps, music, messaging. One tiny function you rely on can hold you hostage to the whole device. “It’s such a confusing world to get off a smartphone,” Stults said. So he and Krigbaum founded an online store called dumbwireless selling dumbphones, and running a hotline to help people through the process. “We thought if we could streamline it a little bit, then people might be more inclined to follow their better instinct in those moments when they are like, ‘I can't do this anymore,’” said Krigbaum.

Krigbaum herself uses a Lightphone: it’s a new type of device built for digital exiles. Alongside another phone called the Kompakt, these phones are intentionally boring. The screens are e-ink. They have maps, messages, a calculator, an alarm clock, and of course a telephone. The Kompakt can “sideload” any other apps you need, like Slack, Spotify and WhatsApp. But they don’t pull and nag at your attention. 

At upwards of $200, these hybrid, dull phones are the ultimate connoisseur's choice for someone who wants to live in the modern world without being dependent on an attention-demanding device. But the true radicals go further, returning to the flip phones and Nokias of the “sweet spot” era, saying the joy of going back to the dumbphone is reclaiming parts of your brain — like your sense of direction — that have atrophied from smartphone use.

On Reddit’s dumbphones forum, people talk about the bigger aim of doing without the conveniences their phones provide and regaining control over their thought patterns. “Everything is a fucking struggle without a smartphone. The whole world is set up around them,” one Redditor wrote last month. “But I am focussed, I feel capable, I am so much more compassionate and understanding of others. I have more patience. I am less angry and more in control of my emotions. My anxiety is practically gone.”

Every so often, the author Zadie Smith — perhaps the world’s most famous flip-phone user — is reminded of the horrors of analog life, she told Ezra Klein on his podcast. “ Disaster. We're at a party at three in the morning, there's no way to get home, forget about it, walk five miles, disaster. Once a year. And every time it happened, I would think that was bad, but is it as bad as having my very consciousness colonized every moment of the day? And I'd be like, no. Definitely no competition.” It’s a trade-off dumb phone users are happy to make – lose their phone but regain their consciousness.

The other thing dumb phone users cherish is the solitude they get back. True solitude – where there’s no constant companion in your pocket that can listen to the sound of your voice, feel the pads of your fingertips, track your expressions, and follow you through your home city.

Someone who knows the importance of such solitude is Issa Amro, a Palestinian activist living in Hebron, on the West Bank. Hebron is one of the most intensely surveilled places on Earth, where the Israeli military uses facial recognition programs called Blue Wolf, Red Wolf, and White Wolf to track Palestinians. “I feel that I live in a lab and I’m a simulation object,” Amro told me, describing how the systems rely heavily on smartphones for data collection and enforcement. 

In 2022, Amro filmed an Israeli soldier beating an Israeli-Jewish activist. The video was much-shared in Israel, and Amro knew it would only be a matter of time before he was arrested. So he gave his smartphone to a friend who drove a taxi around the city. When the police came for him, they were intent on getting hold of it.  “The Israeli police were crazy to get my phone. And I refused to give it to them,” he remembers. Meanwhile, the phone’s location was moving all over Hebron, hidden in the taxicab. “My friends moved it from one car to another, trying to hide it. The phone was going all around the city until I was released,” he said with a laugh.

After that, he traveled to Jordan and bought an analog phone with buttons — the first he’d owned in years. “Buy it from a random place, when you travel somewhere, go and buy one,” he advised. He swaps out his smartphone for the analog phone “to feel better,” when he wants to have a moment of respite from the suffocating surveillance of life in the occupied West Bank. 

“It’s a very bad feeling to know all your life is being watched. Not just your political activities, but your [personal] life too. If you want to have a date, or something for yourself, the occupation will use it against you.” Once Amro started using the analog phone, the Israeli forces took notice. They didn’t like it. Just last month, as he was crossing the border from Jordan into Palestine, the customs officer rifled through his bag, looking for his smartphone. When the officer found the small analog phone, he took a picture of it and sent it to his superiors. 

“I was waiting, waiting, waiting,” Amro said. “Then the interrogators came.”

They grilled him about the phone. “ What’s wrong with you?” they said. “Why do you carry an analog phone? What do you do with it, who did you contact, and where is your smartphone?”

“I told them, ‘I’m not doing anything illegal. I live in Hebron. My house has one camera in the front and one in the back. Whenever I get in or get out, you know about it. Wherever I go, you know. My life has no privacy. Why do you care if I have an analog phone or a smartphone?’”

The border police questioned him solidly for two hours about the phone. “Everything is built on surveillance now and digitalization,” Amro said. “So if you go analog, you really make it hard for them. In the past, intelligence systems depended on analog tactics — on people. Now they depend on machines.”

They wanted him to have a smartphone because, as Amro put it, “The phone documents everything.”

He feels solidarity with other analog phone users around the world. “Whenever I see someone else with one, I feel — Here’s a friend. We are the same family.”

Sometimes, with his analog phone, Amro does nothing more than go to the forest for a moment of peace. “We’re skimming nature from our life, and it’s really important to understand the threat of digitalization. Going back to nature is really important.” 

It’s a sentiment New York-based writer August Lamm, who has made a zine about dumbphones, shares. She can palpably feel the outside world re-entering her life since she got rid of her phone. “I feel more present and attuned to my surroundings, and I can feel my life changing,” she said. “My days feel long and rich and open, and I can trust my thoughts more because I don't feel they've been fed to me.”

She talked about how the physical realm opened up to her when she got rid of her smartphone, with its Instagram account and its tens of thousands of followers. She regained a sense of her surroundings. “If you live for fifty years and you’re aware every day of what’s going on around you, and you’re listening to people, and you’re present, that is more valuable than living into your nineties and when you flash back through your life it’s just screens.”

Lamm wants to push to maintain a critical minority of society who isn’t captured by smartphone use, who don’t own them and will never own them. “I would love to live in a world where people say, ‘Wait, do you have a smartphone?’ as a matter of courtesy.” 

It feels like an impossible dream, as big tech companies move to capture even more areas of our lives. From the moment the scans of our unborn bodies are uploaded by our parents to Instagram, to our school days dominated by Google classroom, to our first phone, to every thought we commit to a search term or AI model, to every beat of our heart recorded by our smart watch, to the steady decline of our health, to our hospital appointments booked on our phones, to the day we die and condolences are posted on our page, the phone is ever-present. “ Our brains are captured. The industry is captured. Our politics is captured. We're captured in so many different ways,” Judy Estrin said. “Our leadership is captured. In every industry, we’re captured by this mentality and worship of growth and innovation.” 

And if tech leaders have their way, there’ll be a time when the smartphone is no longer an external device — but part of our bodies. Kyle Morris, a young AI builder I met in San Francisco last year, called the smartphone a “better prefrontal cortex. It tells you how to get places, tells you how to plan. It gives you answers. It gives you a better memory. I see in the next 50 years, that it's going to enter us. That it's going to become part of us.” He held up his phone in front of me: “It's weird that we have these like external things that we're using. People are going to start retrofitting themselves with improved memory, improved vision.” 

As companies like Neuralink push towards merging technology with the body, and AI seeps into every corner of our world, Lamm says she still has days where she feels powerless and alone. 

When Google rolled out AI search, with no way to turn it off, she broke down. “I  googled how to undo an AI overview, and there wasn't a way to do it. And I had a total meltdown… I was like, this is evil. Like, I can't even do a Google search without being confronted by AI.”

I ask Lamm and Lane what they’ll do in the future in the face of this capture. With each passing year, it gets more difficult to live without a smartphone. The pandemic — which saw countries around the world rolling out QR code greenpasses — cemented this, as restaurants spurned paper menus, airlines stopped issuing paper tickets, health services made it so hospital appointments could only be booked on apps. Recently, Lamm couldn’t apply for a UK Visa because she needed a smartphone to do it. She can’t get an electric car because you can only pay for electricity with a QR code. So what then — when the drawbridge finally rises and modern life necessitates a smartphone?

Lamm has thought about this, and once she gets to her conclusion, it becomes as sci-fi as the imaginings of the tech workers who want to put chips in our heads. “There needs to be another option,” she reflected. “In the worst case scenario, people just defect from society and say, ‘Ok, there’s at least a few thousand of us that want to just live a normal life and we’ll go off and continue living a normal life somewhere else,’” she said. She quoted from Dave Eggers’s cult novel “The Every,” where a small tribe of tech-skeptics calling themselves the “Trogs” try to live outside a world where surveillance capitalism and tech have become all-encompassing. 

“It wouldn’t be a commune situation because ideally through this activism, it would kind of be more of a split in society, rather than founding a new society,” Lamm said. “It would be like enough people that it would feel like normal life, and you just wouldn't interact with the tech.”

As my line with Logan Lane, the Luddite Club founder, crackled again, I asked her the same question — what will she do when life becomes impossible without a smartphone, when tech capture becomes complete? “I’m just like, fuck it. I’ll get to it when I get to it. But I am not OK with it. I am going to do everything I can before then to try to prevent that.” She paused. “I'm not so worried about what people in Silicon Valley think people want.” As her train went into a tunnel, the line went dead, and she continued with her journey — in exile from the digital world; fully present in the physical world.

Drop in image 1: Teona Tsintsadze. Motion by Anna Jibladze . Drop in image 3: Teona Tsintsadze/Creative Commons (CC BY 4.0) Reinhold Möller, Motion by Anna Jibladze. Drop in image4 : Teona Tsintsadze/ Creative Commons (CC BY 4.0)Ermell/Reinhold Möller.

PART OF THE BIG IDEAS

---

Captured

This Big Idea explores how this new technology is not just intended to redefine the way we work, but what it means to be human.

Explore Captured

---

The Age of Exile

This Big Idea explores how displacement has evolved from historical punishment into a defining condition of our time.

Explore The Age of Exile

Related Articles

Welcome to the age of exile

Natalia Antelava

The price of exile: a Syrian photographer trapped by the laws that saved her

Sara Kontar

I risked prison to keep the Uyghur culture alive

Abduweli Ayup

The post The digital exiles: Why people are abandoning their smartphones appeared first on Coda Story.

This is just the beginning. “Innovation related to artificial intelligence could displace 6-7% of the US workforce if AI is widely adopted,” says a recent Goldman Sachs report.

In the last week, over 53,000 people signed a statement calling for “a prohibition on the development of superintelligence.” A wide coalition of notable figures, from Nobel-winning scientists to senior politicians, writers, British royals, and radio shockjocks agreed that AI companies are racing to build superintelligence with little regard for concerns that include “human economic obsolescence and disempowerment.”

The petition against superintelligence development could be the beginning of organized political resistance to AI's unchecked advance. The signatories span continents and ideologies, suggesting a rare consensus emerging around the need for democratic oversight of AI development. The question is: can it organize quickly enough to influence policy before the key decisions are made in Silicon Valley boardrooms and government backrooms?

But it’s not just jobs we could lose. The petition talks about the “losses of freedom, civil liberties, dignity… and even potential human extinction.” It reflects a deeper unease about the quasi-religious zeal of AI evangelists who view superintelligence not as a choice to be democratically decided, but as an inevitable evolution the tech bros alone can shepherd.

Coda explored this messianic ideology at length in "Captured," a six-part investigative series available as a podcast on Audible and as a series of articles on our website, in which we dove deep into the future envisioned by the tech elite for the rest of us.

During our reporting, data scientist Christopher Wylie, best known as the Cambridge Analytica whistleblower, and I spoke to the Swedish philosopher Nick Bostrom, whose 2014 book foresaw the possibility that our world might be taken over by an uncontrollable artificial superintelligence.

Do we believe in Bostrom’s version of the future where society plunges into dystopia or utopia? Or is there a middle way? Judge for yourself whether his warnings still sound theoretical.

This conversation has been edited and condensed for clarity.

Christopher Wylie: To start, could you define what you mean by superintelligence and how it differs from the AI we see today?

Nick Bostrom: Superintelligence is a form of cognitive processing system that not just matches but exceeds human cognitive abilities. If we're talking about general superintelligence, it would exceed our cognitive capacities in all fields — scientific creativity, common sense, general wisdom.

Isobel Cockerell: What kind of future are we looking at — especially if we manage to develop superintelligence?

Bostrom: So I think many people have the view that the most likely scenario is that things more or less continue as they have — maybe a little war here, a cool new gadget there, but basically the human condition continues indefinitely. 

But I think that looks pretty implausible. It’s more likely that it will radically change. Either for the much better or for the much worse.

The longer the timeframe we consider — and these days I don’t think in terms of that many years — we are kind of approaching this critical juncture in human affairs, where we will either go extinct or suffer some comparably bad fate, or else be catapulted into some form of utopian condition.

You could think of the human condition as a ball rolling along a thin beam — and it will probably fall off that beam. But it’s hard to predict in which direction.

Wylie: When you think about these two almost opposite outcomes — one where humanity is subsumed by superintelligence, and the other where technology liberates us into a utopia — do humans ultimately become redundant in either case?

Bostrom: In the sense of practical utility, yes — I think we will reach, or at least approximate, a state where human labor is not needed for anything. There’s no practical objective that couldn’t be better achieved by machines, by AIs and robots.

But you have to ask what it’s all for. Possibly we have a role as consumers of all this abundance. It’s like having a big Disneyland — maybe in the future you could automate the whole park so no human employees are needed. But even then, you still need the children to enjoy it.

If we really take seriously this notion that we could develop AI that can do everything we can do, and do it much better, we will then face quite profound questions about the purpose of human life. If there’s nothing we need to do — if we could just press a button and have everything done — what do we do all day long? What gives meaning to our lives?

And so ultimately, I think we need to envisage a future that accommodates humans, animals, and AIs of various different shapes and levels — all living happy lives in harmony.

Cockerell: How far do you trust the people in Silicon Valley to guide us toward a better future?

Bostrom: I mean, there’s a sense in which I don’t really trust anybody. I think we humans are not fully competent here — but we still have to do it as best we can.

If you were a divine creature looking down, it might seem like a comedy: these ape-like humans running around building super-powerful machines they barely understand, occasionally fighting with rocks and stones, then going back to building again. That must be what the human condition looks like from the point of view of some billion-year-old alien civilization.

So that’s kind of where we are.

Ultimately, it’ll be a much bigger conversation about how this technology should be used. If we develop superintelligence, all humans will be exposed to its risks — even if you have nothing to do with AI, even if you’re a farmer somewhere you’ve never heard of, you’ll still be affected. So it seems fair that if things go well, everyone should also share some of the upside.

You don’t want to pre-commit to doing all of this open-source. For example, Meta is pursuing open-source AI — so far, that’s good. But at some point, these models will become capable of lending highly useful assistance in developing weapons of mass destruction.

Now, before releasing their model, they fine-tune it to refuse those requests. But once they open-source it, everyone has access to the model weights. It’s easy to remove that fine-tuning and unlock these latent capabilities.

This works great for normal software and relatively modest AI, but there might be a level where it just democratizes mass destruction.

Wylie : But on the flip side — if you concentrate that power in the hands of a few people authorized to build and use the most powerful AIs, isn’t there also a high risk of abuse? Governments or corporations misusing it against people or other groups?

Bostrom: When we figure out how to make powerful superintelligence, if development is completely open — with many entities, companies, and groups all competing to get there first — then if it turns out it’s actually hard to align them, where you might need a year or two to train, make sure it’s safe, test and double-test before really ramping things up, that just might not be possible in an open competitive scenario.

You might be responsible — one of the lead developers who chooses to do it carefully — but that just means you forfeit the lead to whoever is willing to take more risks. If there are 10 or 20 groups racing in different countries and companies, there will always be someone willing to cut more corners.

Wylie: More broadly, do you have conversations with people in Silicon Valley — Sam Altman, Elon Musk, the leaders of major tech companies — about your concerns, and their role in shaping or preventing some of the long-term risks of AI?

Bostrom: Yeah. I’ve had quite a few conversations. What’s striking, when thinking specifically about AI, is that many of the early people in the frontier labs have, for years, been seriously engaged with questions about what happens when AI succeeds — superintelligence, alignment, and so on.

That’s quite different from the typical tech founder focused on capturing markets and launching products. For historical reasons, many early AI researchers have been thinking ahead about these deeper issues for a long time, even if they reach different conclusions about what to do.

And it’s always possible to imagine a more ideal world, but relatively speaking, I think we’ve been quite lucky so far. The impact of current AI technologies has been mostly positive — search engines, spam filters, and now these large language models that are genuinely useful for answering questions and helping with coding.

I would imagine that the benefits will continue to far outweigh the downsides — at least until the final stage, where it becomes more of an open question whether we end up with a kind of utopia or an existential catastrophe.

A version of this story was published in this week’s Coda Currents newsletter. Sign up here.

Related Articles

Who owns the rights to your brain?

Isobel Cockerell

Captured: how Silicon Valley is building a future we never chose

Isobel Cockerell

When I’m 125?

J. Paul Neeley

The post Finding meaning in human lives appeared first on Coda Story.

As sunlight filtered through glass onto worn stone walls, Cambridge Analytica whistleblower Christopher Wylie recounted a dinner with a Silicon Valley mogul who believes drinking his son's blood will help him live forever.

"We've got it wrong," Bryan Johnson told Chris. "God didn't create us. We're going to create God and then we're going to merge with him."

This wasn't hyperbole. It's the worldview taking root among tech elites who have the power, wealth, and unbounded ambition to shape our collective future.

Working on “Captured: The Secret Behind Silicon Valley's AI Takeover” podcast, which we presented in that church in Perugia, we realized we weren't just investigating technology – we were documenting a fundamentalist movement with all the trappings of prophecy, salvation, and eternal life. And yet, talking about it from the stage to my colleagues in Perugia, I felt, for a second at least, like a conspiracy theorist. Discussing blood-drinking tech moguls and godlike ambitions in a journalism conference felt jarring, even inappropriate. I felt, instinctively, that not everyone was willing to hear what our reporting had uncovered. The truth is, these ideas aren’t fringe at all – they are the root of the new power structures shaping our reality.

“Stop being so polite,” Chris Wylie urged the audience, challenging journalists to confront the cultish drive for transcendence, the quasi-religious fervor animating tech’s most powerful figures. 

We've ignored this story, in part at least, because the journalism industry had chosen to be “friends” with Big Tech, accepting platform funding, entering into “partnerships,” and treating tech companies as potential saviors instead of recognizing the fundamental incompatibility between their business models and the requirements of a healthy information ecosystem, which is as essential to journalism as air is to humanity.

In effect, journalism has been complicit in its own capture. That complicity has blunted our ability to fulfil journalism's most basic societal function: holding power to account.

As tech billionaires have emerged as some of the most powerful actors on the global stage, our industry—so eager to believe in their promises—has struggled to confront them with the same rigor and independence we once reserved for governments, oligarchs, or other corporate powers.

This tension surfaced most clearly during a panel at the festival when I challenged Alan Rusbridger, former editor-in-chief of “The Guardian” and current Meta Oversight Board member, about resigning in light of Meta's abandonment of fact-checking. His response echoed our previous exchanges: board membership, he maintains, allows him to influence individual cases despite the troubling broader direction.

This defense exposes the fundamental trap of institutional capture. Meta has systematically recruited respected journalists, human rights defenders, and academics to well-paid positions on its Oversight Board, lending it a veneer of credibility. When board members like Rusbridger justify their participation through "minor victories," they ignore how their presence legitimizes a business model fundamentally incompatible with the public interest.

What once felt like slow erosion now feels like a landslide, accelerated by broligarchs who claim to champion free speech while their algorithms amplify authoritarians.

Imagine a climate activist serving on an Exxon-established climate change oversight board, tasked with reviewing a handful of complaints while Exxon continues to pour billions into fossil fuel expansion and climate denial. 

Meta's oversight board provides cover for a platform whose design and priorities fundamentally undermine our shared reality. The "public square" - a space for listening and conversation that the internet once promised to nurture but is now helping to destroy - isn't merely a metaphor, it's the essential infrastructure of justice and open society.

Trump's renewed attacks on the press, the abrupt withdrawal of U.S. funding for independent media around the world, platform complicity in spreading disinformation, and the normalization of hostility toward journalists have stripped away any illusions about where we stand. What once felt like slow erosion now feels like a landslide, accelerated by broligarchs who claim to champion free speech while their algorithms amplify authoritarians.

The Luxury of Neutrality

If there is one upside to the dire state of the world, it’s that the fog has lifted. In Perugia, the new sense of clarity was palpable. Unlike last year, when so many drifted into resignation, the mood this time was one of resolve. The stakes were higher, the threats more visible, and everywhere I looked, people were not just lamenting what had been lost – they were plotting and preparing to defend what matters most.

One unintended casualty of this new clarity is the old concept of journalistic objectivity. For decades, objectivity was held up as the gold standard of our profession – a shield against accusations of bias. But as attacks on the media intensify and the very act of journalism becomes increasingly criminalized and demonized around the world, it’s clear that objectivity was always a luxury, available only to a privileged few. For many who have long worked under threat – neutrality was never an option. Now, as the ground shifts beneath all of us, their experience and strategies for survival have become essential lessons for the entire field.

That was the spirit animating our “Am I Black Enough?” panel in Perugia, which brought together three extraordinary Black American media leaders, with me as moderator.

“I come out of the Black media tradition whose origins were in activism,” said Sara Lomax, co-founder of URL Media and head of WURD, Philadelphia’s oldest Black talk radio station. She reminded us that the first Black newspaper in America was founded in 1827 - decades before emancipation - to advocate for the humanity of people who were still legally considered property.

Karen McMullen, festival director of Urbanworld, spoke to the exhaustion and perseverance that define the Black American experience: “We would like to think that we could rest on the successes that our parents and ancestors have made towards equality, but we can’t. So we’re exhausted but we will prevail.”

And as veteran journalist and head of the Maynard Institute Martin Reynolds put it, “Black struggle is a struggle to help all. What’s good for us tends to be good for all. We want fair housing, we want education, we want to be treated with respect.”

Near the end of our session, an audience member challenged my role as a white moderator on a panel about Black experiences. This moment crystallized how the boundaries we draw around our identities can both protect and divide us. It also highlighted exactly why we had organized the panel in the first place: to remind us that the tools of survival and resistance forged by those long excluded from "objectivity" are now essential for everyone facing the erosion of old certainties.

The Power of Protected Spaces

If there’s one lesson from those who have always lived on the frontlines and who never had the luxury of neutrality – it’s that survival depends on carving out spaces where your story, your truth, and your community can endure, even when the world outside is hostile.

That idea crystallized for me one night in Perugia, when during a dinner with colleagues battered by layoffs, lawsuits, and threats far graver than those I face, someone suggested we play a game: “What gives you hope?” When it was my turn, I found myself talking about finding hope in spaces where freedom lives on. Spaces that can always be found, no matter how dire the circumstances. 

I mentioned my parents, dissidents in the Soviet Union, for whom the kitchen was a sanctuary for forbidden conversations. And Georgia, my homeland – a place that has preserved its identity through centuries of invasion because its people fought, time and again, for the right to write their own story. Even now, as protesters fill the streets to defend the same values my parents once whispered about in the kitchen, their resilience is a reminder that survival depends on protecting the spaces where you can say who you are.

But there’s a catch: to protect the spaces where you can say who you are, you first have to know what you stand for – and who stands with you. Is it the tech bros who dream of living forever, conquering Mars, and who rush to turn their backs on diversity and equity at the first opportunity? Or is it those who have stood by the values of human dignity and justice, who have fought for the right to be heard and to belong, even when the world tried to silence them? 

As we went around the table, each of us sharing what gave us hope, one of our dinner companions, a Turkish lawyer, offered a metaphor in response to my point about the need to protect spaces. “In climate science,” she said, “they talk about protected areas – patches of land set aside so that life can survive when the ecosystem around it collapses. They don’t stop the storms, but they give something vital a chance to endure, adapt, and, when the time is right, regenerate.”

That's what we need now: protected areas for uncomfortable truths and complexity. Not just newsrooms, but dinner tables, group chats, classrooms, gatherings that foster unlikely alliances - anywhere we can still speak honestly, listen deeply, and dare to imagine.

More storms will come. More authoritarians will rise. Populist strongmen and broligarchs will keep fragmenting our shared reality.

But if history has taught us anything – from Soviet kitchens to Black newspapers founded in the shadow of slavery - it’s that carefully guarded spaces where stories and collective memory are kept alive have always been the seedbeds of change.

When we nurture these sanctuaries of complex truth against all odds, we aren't just surviving. We're quietly cultivating the future we wish to see.

And in times like these, that's not just hope - it's a blueprint for renewal.

The post The capture of journalism and the illusion of objectivity appeared first on Coda Story.

It came from Christopher Wylie. He’s a data scientist and the whistleblower who cracked open the Cambridge Analytica scandal in 2018. I had just started working with him on a new investigation into AI. Chris was supposed to be meeting me, but he had found himself trapped in Dubai in a party full of Silicon Valley venture capitalists.

“I don’t know if you can hear me — I’m in the toilet at this event, and people here are talking about longevity, how to live forever, but also prepping for when people revolt and when society gets completely undermined,” he had whispered into his phone. “You have in another part of the world, a bunch of journalists talking about how to save democracy. And here, you've got a bunch of tech guys thinking about how to live past democracy and survive.”

A massive storm and a once-in-a-generation flood had paralyzed Dubai when Chris was on a layover on his way to Perugia. He couldn’t leave. And neither could the hundreds of tech guys who were there for a crypto summit. The freakish weather hadn’t stopped them partying, Chris told me over a frantic Zoom call. 

“You're wading through knee-deep water, people are screaming everywhere, and then…  What do all these bros do? They organize a party. It's like the world is collapsing outside and yet you go inside and it's billionaires and centimillionaires having a party,” he said. “Dubai right now is a microcosm of the world. The world is collapsing outside and the people are partying.”

Chris and I eventually managed to meet up. And for over a year we worked together on a podcast that asks what is really going on inside the tech world.  We looked at how the rest of us —  journalists, artists, nurses, businesses, even governments — are being captured by big tech’s ambitions for the future and how we can fight back. 

Mercy was a content moderator for Meta. She was paid around a dollar an hour for work that left her so traumatized that she couldn't sleep. And when she tried to unionize, she was laid off.

Our reporting took us around the world from the lofty hills of Twin Peaks in San Francisco to meet the people building AI models, to the informal settlements of Kenya to meet the workers training those models.

One of these people was Mercy Chimwani, who we visited in her makeshift house with no roof on the outskirts of Nairobi. There was mud beneath our feet, and above you could see the rainclouds through a gaping hole where the unfinished stairs met the sky. When it rained, Mercy told us, water ran right through the house. It’s hard to believe, but she worked for Meta. 

Mercy was a content moderator, hired by the middlemen Meta used to source employees. Her job was to watch the internet’s most horrific images and video –  training the company’s system so it can automatically filter out such content before the rest of us are exposed to it. 

She was paid around a dollar an hour for work that left her so traumatized that she couldn’t sleep. And when she and her colleagues tried to unionize, she was laid off. Mercy was part of the invisible, ignored workforce in the Global South that enables our frictionless life online for little reward. 

Of course, we went to the big houses too — where the other type of tech worker lives. The huge palaces made of glass and steel in San Francisco, where the inhabitants believe the AI they are building will one day help them live forever, and discover everything there is to know about the universe. 

In Twin Peaks, we spoke to Jeremy Nixon, the creator of AGI House San Francisco (AGI for Artificial General Intelligence). Nixon described an apparently utopian future, a place where we never have to work, where AI does everything for us, and where we can install the sum of human knowledge into our brains. “The intention is to allow every human to know everything that’s known,” he told me. 

Later that day, we went to a barbecue in Cupertino and got talking to Alan Boehme, once a chief technology officer for some of the biggest companies in the world, and now an investor in AI startups. Boehme told us how important it was, from his point of view, that tech wasn’t stymied by government regulation. “We have to be worried that people are going to over-regulate it. Europe is the worst, to be honest with you,” he said. “Let's look at how we can benefit society and how this can help lead the world as opposed to trying to hold it back.”

I asked him if regulation wasn’t part of the reason we have democratically elected governments, to ensure that all people are kept safe, that some people aren’t left behind by the pace of change? Shouldn’t the governments we elect be the ones deciding whether we regulate AI and not the people at this Cupertino barbecue?

“You sound like you're from Sweden,” Boehme responded. “I'm sorry, that's social democracy. That is not what we are here in the U. S. This country is based on a Constitution. We're not based on everybody being equal and holding people back. No, we're not in Sweden.” 

As we reported for the podcast, we came to a gradual realization – what’s being built in Silicon Valley isn’t just artificial intelligence, it’s a way of life — even a religion. And it’s a religion we might not have any choice but to join. 

In January, the Vatican released a statement in which it argued that we’re in danger of worshiping AI as God. It's an idea we'd discussed with Judy Estrin, who worked on building some of the earliest iterations of the internet. As a young researcher at Stanford in the 1970s, Estrin was building some of the very first networked connections. She is no technophobe, fearful of the future, but she is worried about the zealotry she says is taking over Silicon Valley.

What if they truly believe humans are replaceable, that traditional concepts of humanity are outdated, that a technological "god" should supersede us? These aren't just ideological positions – they're the foundations for the world being built around us.

“If you worship innovation, if you worship anything, you can't take a step back and think about guardrails,” she said about the unquestioning embrace of AI. “So we, from a leadership perspective, are very vulnerable to techno populists who come out and assert that this is the only way to make something happen.” 

The first step toward reclaiming our lost agency, as AI aims to capture every facet of our world, is simply to pay attention. I've been struck by how rarely we actually listen to what tech leaders are explicitly saying about their vision of the future. 

There's a tendency to dismiss their most extreme statements as hyperbole or marketing, but what if they're being honest? What if they truly believe humans, or at least most humans, are replaceable, that traditional concepts of humanity are outdated, that a technological "god" should supersede us? These aren't just ideological positions – they're the foundations for the world being built around us right now. 

In our series, we explore artificial intelligence as something that affects our culture, our jobs, our media and our politics. But we should also ask what tech founders and engineers are really building with AI, or what they think they’re building. Because if their vision of society does not have a place for us in it, we should be ready to reclaim our destiny – before our collective future is captured.

Our audio documentary series, CAPTURED: The Secret Behind Silicon Valley’s AI Takeover is available now on Audible. Do please tune in, and you can dig deeper into our stories and the people we met during the reporting below.

Dig deeper into our CAPTURED series

Who owns the rights to your brain?

Isobel Cockerell

In Kenya’s slums, they’re doing our digital dirty work

Isobel Cockerell

Stop Drinking from the Toilet!

Judy Estrin

The post Captured: how Silicon Valley is building a future we never chose appeared first on Coda Story.

Judy Estrin has been thinking about digital connectivity since the early days of Silicon Valley. As a junior researcher at Stanford in the 1970s she worked on what became the Internet. She built tech companies, became Cisco’s Chief Technology Officer, and served on the board of Disney and FedEx. Now, she’s working to build our understanding of the digital systems that run our lives.

We can’t live without air. We can’t live without water. And now we can’t live without our phones. Yet our digital information systems are failing us. Promises of unlimited connectivity and access have led to a fractionalization of reality and levels of noise that undermine our social cohesion. Without a common understanding and language about what we are facing, we put at risk our democratic elections, the resolution of conflicts, our health and the health of the planet. In order to move beyond just reacting to the next catastrophe, we can learn something from water. We turn on the tap to drink or wash, rarely considering where the water comes from–until a crisis of scarcity or quality alerts us to a breakdown. As AI further infiltrates our digital world, a crisis in our digital information systems necessitates paying more attention to its flow.

Water is life sustaining, yet too much water, or impure water, makes us sick, destroys our environment, or even kills us. A bit of water pollution may not be harmful but we know that if the toxins exceed a certain level the water is no longer potable. We have learned that water systems need to protect quality at the source, that lead from pipes leach into the water, and that separation is critical–we don’t use the same pipes for sourcing drinking water and drainage of waste and sewage.

Today, digital services have become the information pipes of our lives. Many of us do not understand or care how they work. Like water, digital information can have varying levels of drinkability and toxicity–yet we don’t know what we are drinking. Current system designs are corroded by the transactional business models of companies that neither have our best interests in mind, nor the tools that can adequately detect impurities and sound the alarm. Digital platforms, such as Instagram, TikTok, or YouTube, don’t differentiate between types of content coming into their systems and they lack the equivalent of effective water filters, purification systems, or valves to stop pollution and flooding. We are both the consumers and the sources of this ‘digital water’ flowing through and shaping our minds and lives. Whether we want to learn, laugh, share, or zone-out, we open our phones and drink from that well. The data we generate fuels increasingly dangerous ad targeting and surveillance of our online movements. Reality, entertainment, satire, facts, opinion, and misinformation all blend together in our feeds. 

“Technology is neither good nor bad; nor is it neutral."

Digital platforms mix “digital water” and “sewage” in the same pipes, polluting our information systems and undermining the foundations of our culture, our public health, our economy, and our democracy. We see the news avoidance, extremism, loss of civility, reactionary politics, and conflicts. Less visible are other toxins, including the erosion of trust, critical thinking, and creativity. Those propagating the problems deny responsibility and ignore the punch line of Kranzberg’s first law which states, “technology is neither good nor bad; nor is it neutral." We need fundamental changes to the design of our information distribution systems so that they can benefit society and not just increase profit to a few at our expense.

To start, let us acknowledge the monetary incentives behind the tech industry’s course of action that dragged the public down as they made their fortunes. The foundational Internet infrastructure, developed in the 1970s and 80s, combined public and private players, and different levels of service and sources. Individual data bits traveled in packets down a shared distributed network designed to avoid single points of failure. Necessary separation and differentiation was enforced by the information service applications layered on top of the network. Users proactively navigated the web by following links to new sites and information, choosing for themselves where they sourced their content, be it their favorite newspaper or individual blogs. Content providers relied heavily on links from other sites creating interdependence that incentivized more respectful norms and behaviors, even when there was an abundance of disagreements and rants.

Then the 2000s brought unbridled consolidation as the companies that now make up BigTech focused on maximizing growth through ad-driven marketplaces. As with some privatized water systems, commercial incentives were prioritized above wellness. This was only amplified in the product design around the small screen of mobile phones, social discovery of content, and cloud computing. Today, we drink from a firehose of endless scrolling that has eroded our capacity for any differentiation or discernment. Toxicity is amplified and nuance eliminated by algorithms that curate our timelines based on an obscure blend of likes, shares, and behavioral data. As we access information through a single feed, different sources and types of content–individuals, bots, hyperbolic news headlines, professional journalism, fantasy shows, and human or AI generated–all begin to feel the same.

Toxicity is amplified and nuance eliminated by algorithms that curate our timelines based on an obscure blend of likes, shares, and behavioral data

Social media fractured the very idea of truth by taking control of the distribution of information. Now. Generative AI has upended the production of content through an opaque mixing of vast sources of public and private, licensed, and pirated data. Once again, an incentive for profit and power is driving product choices towards centralized, resource intensive Large Language Models (LLMs). The LLMs are trained to recognize, interpret, and generate language in obscure ways and then spit out, often awe inspiring, text, images, and videos on demand. The artificial sweetener of artificial intelligence entices us to drink, even as we know that something may be wrong. The social media waters are already muddied by algorithms and agents, as we are now seeing “enshittification” (an aptly coined term by Cory Doctorow) of platforms as well as the overall internet, with increasing amounts of AI generated excrement in our feeds and searches.

We require both behavioral change and a new more distributed digital information system–one that combines public and private resources to ensure that neither our basic ‘tap’ water or our fancy bottled water will poison our children. This will require overcoming two incredibly strong sets of incentives. The first is a business culture that demands dominance through maximizing growth by way of speed and scale. Second is our prioritization of convenience with a boundless desire for a frictionless world. The fact that this is truly a “wicked problem” does not relieve us of the responsibility to take steps to improve our condition. We don’t need to let go entirely of either growth or convenience. We do need to recommit to a more balanced set of values.

As with other areas of public safety, mitigating today’s harms requires broad and deep education programs to spur individual and collective responsibility. We have thrown out the societal norms that guide us to not spit in the proverbial drink of the other, or piss in the proverbial pool. Instead of continuing to adapt to the lowest common decency, we need digital hygiene to establish collective norms for kids and adults. Digital literacy must encourage critical thinking and navigation of our digital environments with discernment; in other words, with a blend of trust and mistrust. In the analog world, our senses of smell and taste warn us when something is off. We need to establish the ability to detect rotten content and sources–from sophisticated phishing to deep fakes. Already awash in conspiracy theories and propaganda, conversational AI applications bring new avenues for manipulation as well as a novel set of emotional and ethical challenges. As we have learned from food labeling or terms of service, transparency only works when backed by the education to decipher the facts.

The artificial sweetener of artificial intelligence entices us to drink, even as we know that something may be wrong.

Mitigation is not sufficient. We need entrepreneurs, innovators, and funders who are willing to rethink systems and interface design assumptions and build products that are more proactive, distributed, and reinforcing of human agency. Proactive design must incorporate safety valves or upfront filters. Distributed design approaches can use less data and special purpose models, and the interconnection of diverse systems can provide more resilience than consolidated homogeneous ones. We need not accept the inevitability of general purpose brute force data beasts. Human agency designs would break with current design norms.  The default to everything looking the same leads to homogeneity and flattening. Our cars would be safer if they didn’t distract us like smart phones on wheels. The awe of discovery is healthier than the numbing of infinite scrolls. Questioning design and business model assumptions require us to break out of our current culture of innovation which is too focused on short term transactions and rapid scaling. The changes in innovation culture have influenced other industries and institutions, including journalism that is too often hijacked by today's commercial incentives. We cannot give up on a common understanding and knowledge, or on the importance of trust and common truths.   

We need policy changes to balance private and public sector participation. Many of the proposals on the table today lock in the worst of the problems, with legislation that reinforces inherently bad designs, removes liability, and/or targets specific implementations (redirecting us to equally toxic alternatives). Independent funding for education, innovation, and research is required to break the narrative and value capture of the BigTech ecosystem. We throw around words like safe, reliable, or responsible without a common understanding of what it means to really be safe. How can we ensure our water is safe to drink? Regulation is best targeted at areas where leakage leads to the most immediate harm–like algorithmic amplification, and lack of transparency and accountability. Consolidation into single points of power inevitably leads to broad based failure. A small number of corporations have assumed the authority of massive utilities that act as both public squares and information highways–without any of the responsibility.

Isolation and polarization have evolved from a quest for a frictionless society with extraordinary systems handcrafted to exploit our attention. It is imperative that we create separation, valves, and safeguards in the distribution and access of digital information. I am calling not for a return to incumbent gatekeepers, but instead for the creation of new distribution, curation, and facilitation mechanisms that can be scaled for the diversity of human need. There is no single answer, but the first step is to truly acknowledge the scope and scale of the problem. The level of toxicity in our ‘digital waters’ is now too high to address reactively by trying to fix things after the fact, or lashing out in the wrong way. We must question our assumptions and embrace fundamental changes in both our technology and culture in order to bring toxicity levels back to a level that does not continue to undermine our society.

The post Stop drinking from the toilet! appeared first on Coda Story.

This is bad for people, but it could also sound wonky, banal or even “small bore” to tech experts. Yet it is precisely the kind of existential threat that we should worry about when we look at the consequences of bringing artificial intelligence into our lives.

You might remember this spring, when the biggest and wealthiest names in the tech world gave us some pretty grave warnings about the future of AI. After a flurry of opinion pieces and full-length speeches, they found a way to boil it all down to a simple “should” statement: 

“Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”

This sentence and its most prominent signatories (Sam Altman, Bill Gates and Geoffrey Hinton among them) swiftly captured the headlines and our social media feeds. But have no fear, the statement’s authors said. We will work with governments to ensure that AI regulations can prevent all this from happening. We will protect you from the worst possible consequences of the technology that we are building and profiting from. Oh really?

OpenAI CEO Sam Altman then jetted off on a global charm tour, on which he seems to have won the trust of heads of state and regulators from Japan to the UAE to Europe. A week after he visited the EU, the highly anticipated AI Act had been watered down to suit his company’s best interests. Mission accomplished.

Before the tech bros began this particular round of spreading doom and gloom about blockbuster-worthy, humanity-destroying AI, journalists at ProPublica had published an investigation into a much more clear and present threat: Cigna’s PXDX algorithm, the very subject of the aforementioned lawsuit. 

In its official response to ProPublica’s findings, Cigna had noted that the algorithm’s reviews of patients’ claims “occur after the service has been provided to the patient and do not result in any denials of care.” 

But hang on a second. This is the U.S., where medical bills can bankrupt people and leave them terrified of seeking out care, even when they desperately need it. I hear about this all the time from my husband, who is a physician and routinely treats incredibly sick patients whose conditions have gone untreated for years, even decades, often due to their being uninsured or underinsured. 

This is not the robot apocalypse or nuclear annihilation that the Big Tech bros are pontificating about. This is a slow-moving-but-very-real public health disaster that algorithms are already inflicting on humanity. 

Flashy tools like ChatGPT and LensaAI may get the lion’s share of headlines, but there is big money to be made from much less interesting stuff that serves the banal needs of companies of all kinds. If you read about what tech investors are focused on right now, you will quickly discover that the use of AI in areas like customer service is expected to become a huge moneymaker in the years to come. Again, forget the forecasted human extinction by robots that take over the world. Tech tools that help “streamline” processes for big companies and state agencies are the banal sort of evil that we’re actually up against.

Part of the illusion that seems to drive statements that prophesy human extinction is that technology will start acting alone. But right now, and for the foreseeable future, technology is the result of a multitude of choices made by real people. Right now, tech does not act alone.

I don’t know where we’d be without this kind of journalism or the AI researchers who have been studying these issues for years now. I’ve plugged them before, and now I’ll do it again — if you’re looking for experts on this stuff, start with this list.

And now I’ll plug a new story of ours. Today, we’re publishing a deep dive that shows how a technical tool, even when it’s built by people with really good intentions, can contribute to bad outcomes. Caitlin Thompson has spent months getting to know current and former staff at New Mexico’s child welfare agency and speaking with them about a tool that the agency has been using since 2020. The tool’s intention? To help caseworkers streamline decisions about whether a child should be removed from their home, in cases where allegations of abuse or neglect have arisen. This is a far cry from the ProPublica story, in which Cigna seems to have quite deliberately chosen to deny people’s claims in order to cut costs. This is a story about a state agency trying to improve outcomes for kids while grappling with chronic staffing shortages, and it shows how the adoption of one tool — well-intentioned though it was — has tipped the scales in some cases, with grave effects for the kids involved. Give it a read and let us know what you think.

GLOBAL NEWS

Google and Meta are facing new legal challenges over violent speech on their platforms. The families of nine Black people who were killed in a supermarket in Buffalo, New York in 2022 have filed suit against the two companies, arguing that their technologies helped shape the ideas and actions of Payton Gendron, the self-described white supremacist who murdered their loved ones. The U.S. Supreme Court has already heard and decided to punt on two cases with very similar characteristics, reasoning that the companies are shielded from liability for speech posted by their users under Section 230 of the Communications Decency Act. So the new filings may not have legs. But they do reflect an increasingly widespread feeling that these platforms are changing the way people think and act and that, sometimes, this can be deadly.

The Saudi regime is using Snapchat to promote its political agenda — and to intimidate its critics. This should come as no surprise: An estimated 90% of Saudis in their teens and 20s use the app, so it has become a central platform for Saudi Crown Prince Mohammed “MBS” bin Salman to burnish his image and talk up his economic initiatives. But people who have criticized the regime on Snapchat are paying a high price. Earlier this month, the Guardian reported allegations that the influencer Mansour al-Raqiba was sentenced to 27 years in prison after he criticized MBS’ “Vision 2030” economic plan. Snapchat didn’t offer much in the way of a response, but Gulf-based media have reported​ on the company’s “special collaboration” with the Saudi culture ministry. It’s also worth noting that Saudi Prince Al Waleed bin Talal — who is Twitter, er, X’s, biggest shareholder after Elon Musk — is a major investor in the company.

WHAT WE’RE READING

• Writing for WIRED, Hossein Derakshan, the blogger who was famously imprisoned in Iran from 2009 until 2015, reflects on his time in solitary confinement and what it taught him about the effects of technology on humanity.

• Justin Hendrix of Tech Policy Press has written a new essay on the “cage match” between Elon Musk and Mark Zuckerberg, the “age of Silicon Valley bullshit” and the overall grim future of Big Tech in the U.S. Read both pieces, and then take a walk outside.

The post The AI apocalypse might begin with a cost-cutting healthcare algorithm appeared first on Coda Story.

It served her well when she worked in child protective services in Hobbs, a small oil town in southeastern New Mexico.

She looks at it this way: “If you give in to emotion, the job’s not going to get done. You don’t process emotion. You walk in on a scene, and the first thing you see isn’t a tragedy. The first thing you see is a checklist of things you need to do to resolve the issue.”

But when Woodward looks back on all the horrible things that she witnessed as a caseworker, the weight of the decisions she had to make is almost too much to bear.

“Each decision that you make changes your life. Every single, solitary decision that I made, I still carry it,” Woodward said when we met in the spring.

Woodward used to work for the state of New Mexico’s Children, Youth, and Families Department, supporting children who had been the victims of abuse or neglect. She was part of the CYFD staff teams that deliberated on whether to take kids away from their parents and put them into foster care.

Woodward is still haunted by the memories of one little girl in particular. Woodward had reason to believe that the girl, who was living in foster care with her grandfather, was being abused.

But something stood in the way: It was a safety assessment tool that the state requires caseworkers like Woodward to use. Formally known by its somewhat clunky brand name — “Structured Decision Making” — the tool is meant to help determine whether a child is in great enough danger to be removed from their home. 

Her concern was based on more than just a hunch. The girl’s mother had told Woodward that the grandfather was an abuser – he had raped her when she was young. Woodward took this information to her team and called for another office to send a caseworker to investigate. But that caseworker’s report, based on the tool, indicated that there was no reason for concern about the girl’s safety. Despite Woodward’s pleas, CYFD staff decided to keep the girl with her grandfather.

It became clear months later that Woodward was right — the little girl’s grandfather had been sexually abusing her all along. The girl was eventually taken away from her grandfather and placed in a different foster home.

The agency is adamant that the tool isn’t meant to supersede a caseworker’s judgment. “It's not about giving the job of a caseworker to an electronic tool,” said Sarah Meadows, the head of the agency’s research, assessment and data bureau. “That’s 100% not what it’s intended to do.”

But in cases like this one, it felt to Woodward as if the tool had won out.

“You can no longer go on all of your training, all of your experience in the field. It’s a moot point because the tool said so,” Woodward told me.

“You're going off of a scoring system now. And if the family doesn’t meet the score, you have to turn around and walk out.”

Across the U.S., child welfare agencies are looking to algorithms and risk assessment tools to help support the arduous labor of caseworkers in child protective services agencies. The hope is that these tools will help caseworkers make better and more equitable decisions that will ultimately improve outcomes for vulnerable children. But these agencies’ problems run deep. Oftentimes, there is no single tool or policy solution that can fix them.

Facing high rates of child abuse and neglect, the New Mexico Children, Youth, and Families Department rolled out the Structured Decision Making safety scoring tool in 2020. The goal was to help the agency decide whether or not children are safe living with their parents and to assess the risk of future abuse if a child remains in their home. But in the face of severe staffing shortages and a push to remove kids from their families in only the most extreme cases, former CYFD staff and children’s attorneys in New Mexico say that the safety scoring tool has been replacing caseworker judgment and leaving some kids in harm’s way.

New Mexico had the 15th highest rate of child abuse or neglect in the 2021 fiscal year, a drop from the 8th highest in 2020. About a third of children who died from abuse, neglect or homicide between 2015 and 2021 had prior involvement with child welfare, according to the New Mexico Department of Health.

One of them was named James Dunklee Cruz. There were countless warning signs that the little boy was at risk of harm. When he was just a few months old, caseworkers found ample evidence of neglect: The home where he lived with his mother was roach-infested and strewn with trash and dog feces. 

In October 2019, when Dunklee Cruz was four, he was brought to the hospital with multiple injuries, including a black eye, a bruised penis and an injured shoulder. He told a CYFD investigator that he had been abused by three people in his life. But somehow those allegations were declared unsubstantiated. The Strategic Decision Making tool classified Dunklee Cruz as “safe with a plan,” allowing him to stay with his mother.

Two months later, James Dunklee Cruz died as a result of blunt force trauma to his head and torso at the hands of Zerrick Marquez, one of the men he had named as his abuser two months before. Marquez pleaded guilty to killing Dunklee Cruz and was sentenced to 30 years in prison.

CYFD conducted nine investigations into allegations of abuse and neglect during the boy’s short life. Caseworkers put what they call “safety plans” in place for Dunklee Cruz, but this wasn’t enough to keep him safe. These details appear in a publicly posted child fatality review summary report. The section of the document drawing on the child’s autopsy also describes a litany of injuries, including “healing jaw fractures and healing subdural hemorrhage indicating significant blunt head trauma that occurred earlier than the acute injuries” — in other words, injuries that didn’t kill him but proved that Dunklee Cruz was at risk of serious harm before his final days.

A wrongful death lawsuit is also working its way through the court. A complaint filed in December 2022 in a federal district court in New Mexico accuses CYFD of failing in their duty to protect the boy and states that Dunklee Cruz’s mother repeatedly violated the safety plans CYFD put in place. The complaint also specifically points to the Structured Decision Making tool. 

It reads: “Over the span of his four years of life, CYFD investigators repeatedly failed to rely on accurate and well-documented facts when it utilized the agency’s Safety Risk Assessment Tool, causing its repeated contacts with James to result in flawed and underestimated risk assessment and flawed decision-making resulting in James’ death.”

When I asked about the boy’s case, CYFD offered this response: “The death of James Dunklee Cruz is tragic. The loss of any child is felt deeply and grieved by our caseworkers and staff. Regarding the function of the tool in this case, he was identified as safe with a plan. This means that the safety assessment tool identified at least one danger indicator and that in order for the child to stay in the custody of the parent, a plan was required. Our caseworkers worked with James’ mother to find a safe place for her to live and alternative childcare for James to mitigate against the threats that were identified by the caseworker.”

What happened to James Dunklee Cruz reflects the most significant problem that former CYFD workers raised when they talked to me about the Structured Decision Making safety tool: It doesn’t always convey how much danger kids are truly facing.

The tool’s launch coincided with a change in the agency’s approach to decision-making about when to remove a child from their family’s home. This was a part of a nationwide shift with the passage of the federal Family First Preservation Services Act, a policy that was designed to keep children “safely with their families to avoid the trauma that results when children are placed in out-of-home care.”

The safety tool doesn’t tell caseworkers what to do. It is meant to facilitate a conversation between the worker and their supervisor about whether to declare the child “safe,” “safe with plan” or “unsafe.” The tool sets the tone for what, ideally, should be an extensive, in-depth dialogue between people from across the agency. But due to staffing shortages, it doesn’t always play out this way. The tool “doesn’t take into account that there's not enough workers, there's not enough supervisors,” said Matt Esquibel, a regional manager at CYFD.

Some former caseworkers have told me that, in this context, the assessment takes on an outsized role in determining a child’s fate.

“It's not meant to streamline or fast-track decisions, but it helps focus the conversations, which is helpful to supervisors and to workers,” said Meadows, the head of the agency’s research, assessment and data bureau.

Former CYFD workers told me that the risk and safety assessments did not always match what they observed about the level of danger a child was facing, particularly when it came to substance abuse, domestic violence or repeated involvement with child protective services.

“We saw issues with the safety tool immediately,” said one former CYFD worker who had knowledge of the tool and reviewed investigations in which it was used. She requested anonymity out of fear of retaliation. 

The former CYFD worker said she would see cases in which she thought a child should have been removed from the home but the safety tool didn’t reflect that.

“I’m reading a report that comes and I’m reading their notes that they’ve entered. And then I’m looking at their safety assessment, [and it] does not match what I’m reading,” she said.

Workers are only allowed to check off a danger on the safety tool if they can observe or otherwise prove it. But investigators don’t always have time to do multiple home visits or to gather more information, said Esquibel. They may not be able to gather all the details right away, and children may not initially disclose abuse. There is an “override” for the risk assessment that requires supervisor approval. If the worker thinks that the risk score is too low, they can bump the score up one level. CYFD’s Meadows emphasized that workers should use their judgment and critical thinking, work with supervisors and override the tool if necessary.

“I think the workers and supervisors do the best that they can when they’re out there,” Esquibel said. “But your assessment is only as good as what information you're gathering or who’s available at the time.” 

Ultimately, the former CYFD staff member who requested anonymity thinks the assessments are not capturing the seriousness of some cases and that the consequences for kids are real.

“I think it’s leading to dangerous situations for children,” she said. “I think the agency is leaving the children in situations based on that tool when they should be removing them.”

Meadows said that shouldn’t happen. “If a worker feels strongly that a child is unsafe and they don’t want to walk away from that child in that home, they shouldn't. Safety tool be damned,” she said.

Even though she’s moved on from CYFD, this all still weighs heavily on Ivy Woodward, who has worked with children for most of her career. Before working in child welfare, Woodward, who is Native American and Hispanic, taught elementary school on the Apache reservation in San Carlos, Arizona and in southwestern New Mexico. In 2017, CYFD brought her on as a permanency planning supervisor. This meant she worked on cases where the agency had credible evidence that a child was being abused or neglected at home. The work spoke to core elements of Woodward’s personality. 

“I’m a protector,” Woodward told me. “You can do a lot to me and get away with it. But if I see somebody doing something to someone else, that triggers my inner anger.” 

She calls it like she sees it and pushes back when she disagrees. “I don’t know what is broken in my head, but I question everything,” Woodward said.

When Woodward left CYFD in the summer of 2020, she and a colleague filed a lawsuit alleging that they faced retaliation after raising concerns about a case in which a child was severely injured after she and her siblings were returned to their parents. The agency settled the lawsuit for more than $300,000 without acknowledging liability.

Woodward has a fast, forceful way of speaking, a reflection of her often overly-caffeinated state. But when she talks about the kids she worked with at CYFD and the horrible things she heard or saw on the job, her voice gets a little higher. Her emotions begin to flow.

“You do have to be able to turn off the emotions and make those cold, hard decisions when the time comes to make them,” she said. “But until that time comes, you have to see people, not casework.”

Woodward now lives across the border in a tiny county in West Texas with her husband and two daughters. She works as the chief of juvenile probation and coordinates emergency management for the county.

In some ways, Woodward was an outlier among other CYFD staff. Many start working with the agency soon after college and have little or no experience in child welfare. The agency struggles to stay fully staffed — this spring, nearly a quarter of positions were unfilled. A July 2022 review by an outside consultancy found that CYFD employees felt overwhelmed by the work they were being asked to do. Staff said they would rush from one emergency to the next and had little ability to make progress on other cases.

This is not unique to New Mexico. A caseworker I spoke with in Indiana described feeling like he was stretched so thin that he would race from one emergency to the next without ever having time to put out the fires. He felt like he was just identifying a fire and then moving on to the next one.

The issues of understaffing and high turnover rates were top of mind for many CYFD workers. High turnover isn’t just bad for morale. It directly affects the ability of those who remain on staff to do their work. When one person leaves, those who stay have to absorb their caseload. It is daunting. The review described a “culture of fear” in which staff were afraid that if something bad were to happen with a case, they would be punished or “scapegoated.”

And the forced intimacy of the work can be grating and even traumatic. Caseworkers must regularly intervene in painful moments of struggle and conflict within families, and they are sometimes met with resistance. As agents of the state, they are caught between a bureaucracy that requires them to treat each situation as consistently and objectively as possible and real life-and-death conflicts in which people’s actions are largely driven by emotion.

For strained child welfare agencies, algorithms and risk assessment tools are an attractive solution to the vexing challenge of maintaining consistent decision-making practices.

Some states have experimented with predictive analytics, with limited success. Illinois used an algorithm to estimate the likelihood that a child would die or be seriously injured as a result of abuse or neglect. Social workers were flooded with cases erroneously determined to be urgent, while children that the algorithm deemed low-risk were dying. The state soon stopped using the tool after the Illinois Department of Children and Family Services declared it ineffective. 

A child welfare algorithm in Allegheny County, Pennsylvania is currently facing scrutiny from the U.S. Department of Justice. Using arrest records, Medicaid data and documented struggles with substance abuse, the algorithm generates a score from 1 to 20 that determines whether to open a neglect investigation. Reporting by the AP found that the algorithm disproportionately flagged Black children for neglect investigations. There was also evidence that the algorithm did the same for parents with disabilities.

Torn between pushback against opaque algorithms and the desire to use technology to streamline decision-making, some states are turning to scoring tools that are less opaque and less automated. New Mexico’s Structured Decision Making tool, created by the nonprofit Evident Change, is one of them. Oregon, New Hampshire and California also use assessment tools built by Evident Change.

Structured Decision Making offers a checklist that is meant to help the investigator understand “the risk of imminent and serious harm,” according to a CYFD progress and impact report. Children are ranked safe, safe with plan, which involves in-home services, or unsafe, which is grounds for removal. There’s also an actuarial risk scoring tool, which is meant to assess “the likelihood of any future maltreatment” and additional CYFD involvement in the next 18 to 24 months, if the child remains with the family.

The safety scoring tool asks about abuse or neglect, including physical or sexual abuse, failure to meet the child’s basic needs, unsafe living conditions, emotional harm or unexplained injuries. Both assessments are intended to guide caseworkers to think about risk factors, vulnerabilities and the impact on the child.

“What Structured Decision Making tries to do is to help workers and supervisors make accurate, consistent and equitable decisions at these high-stakes moments,” said Phil Decter, the director of child welfare at Evident Change.

Structured Decision Making is also “intended to reduce bias, whether that's related to race, ethnicity, socioeconomic status, making sure that we're not conflating poverty with neglect,” said CYFD’s Meadows.

But in New Mexico, as the Dunklee Cruz case and insights from caseworkers make clear, the tool does not always work as intended. And the tool can't solve some of CYFD's biggest problems. The agency doesn’t have the workers to meet the needs of the population. Emblematic of a national trend, CYFD is chronically understaffed. Workers juggle heavy caseloads and often have precious little time to dedicate to each child’s case.

The safety tool isn’t meant to fix that. CYFD says hiring is a priority. “Structured Decision Making is not intended to replace human beings in terms of lightening their caseload,” Meadows said. “The role of it is to create consistency, making sure that we're looking at every angle of the case, every potential impact to a child.”

But for caseworkers racing from one emergency to the next, the tool begins to play a different role. It sometimes becomes a shortcut, they told me — a stand-in for real human decision-making, in a system already weighed down by the rigid requirements of the state.

Reed Ridens remembers everything about the day the state took him away from his father almost seven years ago. It was a typical January afternoon at school. About an hour before classes ended, Ridens, who was 15 at the time, was pulled out of orchestra practice and brought to a conference room. Waiting for him were two of his teachers, the school social worker, representatives from CYFD, a police officer and his dad. 

“I’m just looking around like, what is going on?” Ridens recalled. 

For nearly an hour, the adults in the room went back and forth about whether Ridens’ dad could take care of him. There were concerns, they said, about neglect and his father’s alcoholism. 

“The entire time, I was just sitting there, crying, like, ‘Hey, please don’t take me out of my home,’” Ridens said.  

His protests were futile. Ridens stayed in the foster care system until he was 18, moving between 15 different placements. It left him with a deep-seated trauma, compounded by his father’s death four years ago.

“I felt like the state was taking me out of my household and then not doing any better for me than my father did. And in fact, actually putting me in worse-off situations,” he said. 

“I don’t really feel like they saw me as a person,” he told me when we met in Albuquerque.

“I feel like they didn’t see me as more than a list of checkmarks. I feel like they didn’t see my dad as anything more than a monster.”

Today, kids in a position like Ridens’ are not only dealing with adults trying to decide what’s best for them. Their fate is also shaped by tools like Structured Decision Making.

How did New Mexico get here? In part, the objective was to prevent the wrong kids from entering foster care, said Beth Gillia, the former deputy secretary of CYFD. 

“Foster care really should be the absolute last resort in extreme circumstances where needs cannot be met in the home and where a child cannot be safe at home,” she told me.

The state paid the nonprofit organization Evident Change $1.3 million to develop a risk and safety assessment tool, according to a state legislative finance committee report. The nonprofit creates similar tools for criminal justice, education and adult protective services. 

After a pilot in some counties in 2019, including in the country where James Dunklee Cruz lived, Structured Decision Making was rolled out statewide in January 2020.

The tool works best in situations where there is plenty of time and staff capacity to dedicate to this kind of deliberation. But CYFD’s investigations unit was short almost 25% of its workforce as of May 2023, according to the state’s public statistics dashboard, and maintains a steep turnover rate.

“If a child welfare organization is not being resourced well, if it’s understaffed or if caseloads are high, it’s going to be hard for optimal work to happen in any situation,” said Decter, who previously worked in child welfare in Massachusetts. “Good decision-making takes time.”

A report presented to a CYFD steering committee found that, according to focus groups made of CYFD workers, Structured Decision Making is “not being used as it was designed to be utilized. They go out and do their investigation and then come back in and click whatever needs to be clicked to show it has been done.” 

A former investigator in Hobbs told me that the Structured Decision Making tool just added more work to her plate. 

“It didn’t take a whole lot of time, but it was just another tedious step that you’re going through when you’ve already made up your mind,” she said.

As a result, she said, some people rushed through checking boxes on the safety tool. 

“I watched people go click, click, click, click, click, and just move on,” she said. It wasn’t the deciding factor. But she did feel like it could be “manipulated” to justify a certain decision.

CYFD says this isn’t how it’s supposed to be used. “Safety assessment is not a quick activity,” said Meadows. “Workers should take their time with it, really do their best to engage the family to get as much information as possible so that the safety assessment is accurate.”

Ivy Woodward, the former supervisor in Hobbs, had concerns about the safety scoring tool from the very beginning. In particular, she worried about how it dealt with a caregiver’s substance use, which is not listed as one of the danger indicators that must be checked in order for the agency to remove a child. In a sharp pivot from New Mexico’s previous assessment, substance use is treated as a “complicating factor” rather than a deciding one.

The risk tool adds points if the parent struggles with substance abuse. However, the tool doesn’t weigh substances differently. Meth gets the same number of points as marijuana, for example. 

In the Structured Decision Making training, Woodward and some of the other experienced caseworkers challenged this, fearing that it would put children at risk. The discussion got so heated that the head of the agency came to intervene. Woodward said she was effectively shut down. It was clear that the agency would be using the tool, whether she liked it or not.

Other CYFD workers and child welfare attorneys also raised concerns about how the safety and risk assessments handle drug abuse, a factor affecting almost one-third of children who were victims of maltreatment in 2020, according to statistics from the U.S. Department of Health and Human Services. 

While investigators are supposed to consider substance use in their decision about removing a child, it’s not supposed to be the sole reason for removal. This is part of a recent change in the agency’s approach to substance use. Caseworkers are now told to focus not solely on substance use, but rather on the impact substance use has on the caregiver’s ability to care for their children, said Gillia, the former deputy secretary of CYFD. 

“It’s only if the substance use interferes with parenting that it becomes abuse or neglect,” Gillia said. “So I think what the tool is trying to do is force a look at what parenting behavior is impacted by the substance use.” 

Phil Decter at Evident Change says the safety tool also helps when it comes to an inexperienced workforce. It has detailed instructions that help workers decide whether to check ‘yes’ or ‘no’ for a danger indicator. It points staff without a background in child welfare in the direction of things to look for, he said. 

But out in the field, Woodward sees problems with this. The decisions are so monumental — literally life or death. For Woodward, the tool is not a substitute for a seasoned supervisor guiding less experienced staff through decisions.

“It becomes a crutch for a lack of confidence,” said Woodward. “I don't think that being armed with a piece of paper and a laptop is an adequate replacement for someone who's been in the trenches for 20 years and can tell them this is what you do.”

And the tool doesn’t capture the unspoken cues that an investigator may notice, like a child who can’t make eye contact with a family member or won’t answer open-ended questions, Woodward said. 

The safety tool has an “other” option where investigators can write in safety concerns not addressed by the nine danger indicators. But that should be used “rarely and infrequently,” said Decter. “That’s by design. The other danger indicators should be sufficient.”

The success of the tool depends on how it’s used, and this is where Woodward hit roadblocks in Hobbs. She said her supervisors would tell her she was paying attention to things that the safety tool said weren’t an issue, rather than focusing on what she was called upon to investigate. Woodward felt like she was being instructed to ignore history, context and other dangers that she knew were significant from past experience. 

Information about those more subtle cues may be presented to a judge if CYFD files a petition to remove a child. But if the tool indicates that the child doesn’t need to be removed, the case likely won’t reach that stage.

Former CYFD staff like Gillia emphasized that the agency wants to keep kids living with their families unless they are clearly at risk of imminent and grave harm. The agency settled a lawsuit in 2020 that accused the state of failing to take adequate care of foster children in CYFD custody.

But former caseworkers I spoke with worried that the tool was being used as a way to all but ensure that kids would remain in the home, even in cases where it might leave them at risk. The worry, for people like Ivy Woodward, was that the tool was being used to justify decisions that had already been made.

Evident Change emphasizes that “tools don’t make decisions, people make decisions.” But former CYFD workers told me they worried that this particular tool has an outsized impact on the agency’s final decision.

CYFD commissioned a report from an outside group, Collaborative Safety, to look at what went wrong in five specific cases from 2021 in which children died. In the report, released in July 2022, staff involved in those cases said that sometimes the Structured Decision Making tool would say the child is “safe,” even if the worker felt there were “significant concerns with the family.”

“This places staff in the position where they perceive they cannot act on those concerns as it would go against what the tool’s output is,” wrote the report’s authors.

“Investigators were just using the tool as the end-all-be-all to a decision and an assessment. That’s not correct. We don’t want it to substitute their good judgment,” former CYFD Secretary Barbara Vigil told members of the New Mexico House Appropriations and Finance Committee in February 2023. In response to the Collaborative Safety report, CYFD announced they would overhaul their training protocols and pledged to “make sure that every member of staff uniformly knows how to use the tool, including through enhanced training to investigators and supervisors statewide.”

The former CYFD worker I spoke to who requested anonymity saw this reflected in the investigations she reviewed. “I don’t even know how many cases I reviewed where it’s like, you should have removed that kid immediately. And they didn’t because of the safety tool,” she said. “We would always say, use your common sense. This is a guide.” But some workers and managers still put too much emphasis on the tool.

Esquibel said the tool played a major role in facilitating decision-making. “The weight is 100% on your safety assessment because that's really the snapshot of what happened the day that that worker was there,” he said.

CYFD’s Meadows put it differently: “It's not just a snapshot in time,” she said. “Safety assessments are not a one-off, one and done thing. Safety is assessed on an ongoing basis when we have an open case because sometimes it does take effort and time to learn more about a family or child situation.”

Woodward doesn’t think the tool should carry so much weight. Instead, it should be “something in your toolbox that you utilize to help you through the process,” she said. “I don't think they should be used as the ultimate decision maker.”

When Vanna was first removed from her parents at age five, the adults in her life told her that her parents were going on vacation. 

She remembers a woman pulling up to their house and talking to her parents. Her mother was crying, her father was trying to calm her down. The strange woman went up to her younger brother, who was four at the time, and said, “How would you like to go somewhere else?”

“I looked at her, I said, ‘You’re not taking my brother,” said Vanna. 

Vanna, who is now 21 and using a nickname to preserve her privacy, has been fiercely protective of her little brother since they were small. 

As the woman stood talking to her parents, Vanna tried to get him out of his car seat. “And I tried to run with him, and she started running after us. And she said, ‘I’m not trying to take your brother. I’m trying to take you both. You’re going to this lady. Your parents are going on vacation.” 

She didn’t realize until later that she wasn’t returning home. Vanna spent 13 years in foster care until she aged out at 18. She estimates she lived in more than 50 placements. 

In foster care, Vanna felt like she was treated like a case number. Someone else made decisions about every aspect of her life. Someone else had power over her.

“I got numb. I became this robot. You want me to be a puppet, guys? I’ll be a puppet. Pull my strings and do whatever you want because that’s how you treat me,” she said. 

Vanna would tell the adults around her what she wanted, but she didn’t feel like they listened.

“They would always say, ‘Honey, we wouldn't make any decision if it wasn't going to be safe for you or if we weren't keeping your best interests in mind,’” she said. “How do you know what my best interest is?”

The safety assessment that’s currently in place rolled out statewide the year Vanna aged out of the system. But when she looks back at her own experience, systems like this still worry her. She thinks the assessments used to make decisions need to be more personalized, otherwise they do more harm than good. 

“How do you put everyone in the same box, the same population? You put them under the same microscope, but they’re not the same. They’ve had individual situations,” she said.

If the assessments are too generalized, kids won’t end up getting the help they need, Vanna said. Just as the assessments used to evaluate their needs are flattened and standardized, the care kids get is too.

For people like Vanna, many aspects of the child welfare system were dehumanizing. Ernie Holland, who worked at CYFD for 25 years, thinks that by relying on assessment tools like Strategic Decision Making, the agency could make these effects even worse. When he left, he ran the Guidance Center, a nonprofit that offers mental health and other community-based services in Hobbs. 

Even as a young child protective services investigator, the weight of the decisions he was making never escaped him. He shares Ivy Woodward’s belief that “each decision you make changes your life.”

“Unless you’ve gone around the block three or four times to screw up your courage to knock on somebody’s door and ask them why they sodomized their infant, you don’t know what it’s like,” he said. “I’ve been there, done that, and I know what it's like. And I know you’re risking some of yourself doing that work.”

That pressure never goes away. Holland still remembers a family whose case he managed nearly 50 years ago. He’s still not sure he made the right decision. 

As the agency relies more on standardized assessments, he worries humanity gets removed from the equation. 

For Holland, there’s a big difference between being able to say, “I made the decision based on this tool” and “I made the decision.”

“If you can hide behind an assessment tool,” Holland told me, “it’s not personal anymore. If you get it to where it’s not a personal decision, the kid loses. If you’re making life and death decisions, you damn well better own ‘em.”

This project was supported by the Global Reporting Centre and The Citizens through the Tiny Foundation Fellowships for Investigative Journalism.

Related Articles

How Somali workers in the US are fighting Amazon’s surveillance machine

Erica Hellerstein

Fingerprinting employees could cost Illinois businesses billions

Caitlin Thompson

What a law designed to protect the internet has to do with abortion

Tamara Evdokimova

The post Inside New Mexico’s struggle to protect kids from abuse appeared first on Coda Story.

In early February, three archbishops of the Orthodox Tewahedo Church — one of the oldest churches in Africa that dates back to the 4th century — accused fellow church leaders of discriminating against the Oromo people, who constitute the largest ethnic group in Ethiopia’s population of 120 million. While church members come from a diverse array of ethnic backgrounds, worship services are predominantly conducted in the liturgical language of Ge’ez and in Amharic, which is a language primarily spoken by the Amhara people. Amharic is the dominant language of Addis Ababa, Ethiopia’s capital, and the working language of the federal government. This linguistic predilection underlines the cultural clout of Amharic. 

After the three archbishops — all of Oromo lineage — made their allegations of discrimination public, they were excommunicated by church authorities. They then declared their plans to form a breakaway synod, triggering an instant public outcry. The cleavages underlying Ethiopia’s civil conflict bubbled to the surface and devolved into violent skirmishes, resulting in a combined total of 30 fatalities in the southern Ethiopian town of Shashemene and in Addis Ababa.

But what was a serious political crisis for the church and for the country amounted to a prime opportunity for TikTok influencers seeking to spread their messages and turn a profit along the way.

A quick scroll through live sessions on TikTok reveals heated political discussions in Amharic, Oromo and Tigrinya, in which participants exchange barbs and strategize on how to confront their adversaries. Zemedkun Bekele is prominent among them. A self-proclaimed defender of the Orthodox Tewahedo Church, he is known for his forceful, admonitory videos that are often over an hour long. Bekele began broadcasting threats against the breakaway synod, claiming to have video evidence that its leaders had engaged in homosexual activity and threatening to release the tape to the public. Accusations like this resonate deeply in a nation steeped in conservatism, where homosexuality is viewed with considerable disdain.

A known social media influencer who had already been banned from both Facebook and YouTube in 2020 for violating their policies on hate speech and the promotion of violence, Bekele re-established himself on TikTok in February 2023, just in time to jump into the fray. Since then, he has amassed a dedicated audience of more than 203,000 TikTok followers, most of whom appear to be members of the Amhara ethnic group and followers of the Orthodox Tewahedo Church. 

In the midst of the crisis, Bekele also launched attacks against a senior church teacher, Daniel Kibret, who has become a staunch ally of Prime Minister Abiy Ahmed. Drawing on the fact that the prime minister comes from a mixed religious background (his father is Muslim and his mother is Christian), Bekele made unfounded claims that Kibret had secretly converted to Islam.

In a video with more than 19,000 views, Bekele maintained that he would not relent. “We will not back down without making sure the Ethiopian Orthodox Church is as big as the country itself. We will not back down without toppling Abiy Ahmed,” he said. “We will not back down without hanging Daniel Kibret upside down.” The video was posted on February 4, the same day that the three bishops declared their intentions to secede from the Church.

Another account called TegOromo also saw swift growth surrounding the church controversy. TegOromo has a passionate following and is on the opposite side of the conflict from Bekele. The person who runs the account has expressed support for the Oromo religious leaders who sought to establish the independent synod. The account's moniker fuses the first three letters of “Tegaru” with “Oromo” — a calculated move to represent harmony between the Tigrayan and Oromo ethnic groups.

With more than 60,000 followers, TegOromo’s account is marked by overt threats, inflammatory language and aggressive rhetoric. One TikTok video urged supporters to “chop the Amharas like an onion.” This video was later removed from the platform, but copies of it remain accessible. In a live session, a TegOromo follower called on Oromo people to “kill all Amharas” and even specified that children should not be spared. TegOromo cheered him on, urging other followers to answer the call and take up arms.  

Despite the controversial nature of TegOromo’s content, the influencer's popularity suggests a burgeoning trend. Republishing his material or circulating incisive and satirical clips featuring TegOromo has become a reliable strategy for Ethiopian content creators seeking higher engagement.

In another instance, the spotlight turned toward two emerging TikTok influencers, Dalacha and Betayoo, who garnered attention for their adept use of vitriol. In one video, Dalacha, who identifies as Oromo, launched a barrage of insults and sexual slurs at his rival TikToker, Maareg, who identifies as Amhara. The episode exemplified the depths to which Dalacha was willing to stoop in order to denigrate Maereg. Dalacha used language that reduced the Amhara community to mere cattle. It was intended only to amplify the prevailing animosity between the two ethnic groups.

In another video, Betayoo, who consistently identifies as Amhara, used similarly troubling language, employing both sexual and ethnic slurs. She directed her insults toward a rival TikToker who identifies as Tigrayan and who has publicly expressed disdain for the Amhara community. Betayoo's actions escalated beyond targeting an individual. She proceeded to insult the entire Tigrayan community, expressing a desire for their eradication. 

The videos I reference above also all contain clear violations of TikTok’s terms of service, yet they remain on the platform. TikTok's Community Guidelines strictly prohibit hate speech or hateful behavior and promise to remove such material from their platform. Accounts and/or users that engage in severe or multiple violations of hate speech policies are promptly banned from the platform. Despite these guidelines, plenty of Ethiopians who have exhibited hateful behavior remain active on the platform and continue to produce content for significant numbers of followers.

When I approached TikTok staff members to alert them about the videos and ask them to comment for this piece, they did not respond. It is difficult to definitively prove that this kind of discourse directly contributes to violence on the ground. But it is clear that discussions of political violence and religious conflicts on TikTok often result in the spread of misinformation and amplify interethnic hatred. Clips containing these influencers’ offensive remarks have also seeped onto other platforms, such as YouTube and Facebook, where reposting or critiquing such content has become a low-effort method for content creators to gain engagement.

Given the sheer volume of such live streams, TikTok's moderation team may be overwhelmed, struggling to monitor these discussions and remove inappropriate content. It is also worth noting that all of these accounts are run primarily in Amharic, Oromo or Tigrinya, languages that are spoken by millions of Ethiopians in and outside of the country but that have historically been underrepresented on major social media platforms. TikTok does not publicly disclose how many staff members or content moderators it employs for reviewing content in these languages.

All this engagement is not driven purely by political vitriol — it is also a pursuit of profit. The TikTok LIVE feature has seen a swift uptick in popularity among Ethiopian users, catalyzing an emergence of politically-minded influencers who reap economic rewards through virtual gifts. These gifts can be converted into TikTok “diamonds,” which are in turn redeemable for actual cash.

Crafting politically-charged clickbait, designed to fan the flames of ethnic and religious discord, is emerging as a common tactic for financial gain. It has had especially strong uptake among individuals in the Ethiopian diaspora. Many of the most impactful Ethiopian TikTok figures are actually located in Western nations. Zemedkun Bekele, for instance, lives in Germany.

Amid the ongoing crisis, Bekele proudly claimed to have received one of the most sought-after TikTok LIVE gifts — the lion, which translates to a little over $400 in real-world currency. He has prominently featured a video on his profile displaying a virtual lion roaring at the screen, serving as both a symbol of his influence and a testament to the economic gains that one can reap through this kind of engagement on TikTok.

In a 2021 essay, former New York Times media critic Ben Smith showed how TikTok's algorithmic recommendation framework has helped to intensify cultural, linguistic and ideological divides among its global user base. The unfolding situation in Ethiopia could serve as a case study for Smith’s argument. With the videos I described — in addition to hundreds of others — the platform's content dissemination strategy appears to inadvertently encourage distinct factions to isolate themselves and push each other to commit hate speech and even physical violence.

The rise of these online strongholds poses significant challenges to promoting inclusive, cross-cultural understanding that TikTok claims to want to foster. Users now risk becoming trapped in ideological echo chambers, detached from diverse perspectives and viewpoints, and increasingly vulnerable to politically-motivated disinformation.

At the core of the issue lies the question of accountability. What obligation does TikTok, and by extension other social media platforms, have to curtail the spread of divisive content, particularly when it is financially incentivized? And moreover, could the pursuit of profit from politically-charged content inadvertently pave the way for more extreme or hazardous content, potentially triggering threats of violence in real life?

In the end, for onlookers familiar with Ethiopian culture and politics, it is clear that the platforms that invite us to share our lives online are failing to mediate the complexities of the world they seek to engage with.

Related Articles

How Somali workers in the US are fighting Amazon’s surveillance machine

Erica Hellerstein

In India, a trans woman stands up to the ‘YouTube Baba’

Alishan Jafri

Uganda is targeting reproductive rights alongside its ‘anti-gay’ bill

Prudence Nyamishana

The post How TikTok influencers exploit ethnic divisions in Ethiopia appeared first on Coda Story.

But the success of the e-commerce giant’s rapid-fire delivery model depends on what happens inside Amazon’s “fulfillment centers” — sprawling warehouses where workers sort and pack orders for shipment, all under the watchful eye of technical systems that track their every move. For years, workers have said that the company’s algorithmically-driven approach pushes them to the brink, treating them “like robots” in the service of meeting unattainable productivity quotes and driving up injuries in the process.

On May 16, lawmakers in Minnesota passed a pioneering workplace safety bill that could improve labor conditions for Amazon employees subjected to the company’s worker tracking system. Organizers behind the legislation say it will provide the strongest labor protections in the nation for people working in warehouses like Amazon’s.

Work in Amazon warehouses is overseen entirely by technology: Algorithms track workers’ speed and productivity, measure the so-called “time off task” that employees spend logged out of their workstations and alert managers when workers don’t meet their productivity quotas. Mohamed Farah, a 50-year-old Amazon employee who came to the U.S. in the mid-1990s as a refugee from Somalia, works a 10-hour shift packaging items for shipment at a Minnesota warehouse. He said the company’s grueling pace of work and “time off task” rules have worn on workers’ bodies and minds, including his own. “They say you have to pack a minimum of 80 boxes per hour, but you cannot do it,” he told me. “If you try to pack 80 per hour, you cannot go to the bathroom. If you go to the bathroom, the rate is down.” 

Amazon’s “time off task” measurement is a constant source of worry for many workers. The company tracks the time employees are gone from their workstations. If you spend too much time away from your station, you get in trouble. Internal company documents obtained by VICE revealed that Amazon can fire employees who accumulate 30 minutes of “time off task” on three separate days over the course of a year. The documents also showed that the company tracks the amount of time employees spend in the bathroom. Some employees have described needing to urinate in bottles while working to avoid penalties for using the bathroom.

Farah, who has worked for Amazon for seven years, said that workers get hurt trying to keep pace with packaging quotas. He has come home with three injuries over the last few years. “You go home feeling very tired. Headache, muscle aches, leg aches,” he told me. “They want us to work like robots.”

Farah’s experience is common across the company. A recent survey of more than 2,000 Amazon workers across eight countries found that the company’s performance monitoring and tracking system has taken a physical and emotional toll on employees’ well-being: 57% of respondents said their mental health suffered due to the company’s productivity monitoring, and 51% claimed it harmed their physical health. Amazon has twice the injury rate of comparable warehouses in the U.S., according to a recent analysis of injury data submitted to federal safety regulators.

“You do a lot of bending and back-and-forth walking for hours. You get thirsty, and you go to the bathroom, and it’s on a different floor,” explained Qali Jama, a 39-year-old Amazon warehouse worker who also hails from Somalia. “And then you go to the bathroom, which only has two toilets. If those toilets are occupied, you need to wait to go to the bathroom. The whole time you’re gone your time accumulates, it adds up. And next thing you know, the manager goes up to you and tells you a couple of days later, ‘You have time off task.’”

It was these conditions that fueled a raft of organizing efforts in Amazon facilities across the U.S., including the nation’s first successful union drive at a company warehouse in New York last year. Workers from East Africa, like Qali, were among the first Amazon employees in the nation to confront the company over its labor practices and have been at the helm of organizing efforts in Minnesota. 

The Minnesota bill, which state lawmakers passed on May 16, will not just apply to Amazon — though lawmakers who supported the legislation said it was spurred by reports of injuries at Amazon and a lack of transparency around the company’s productivity quotas. The policy will require any warehouse with more than 250 employees to provide employees with the quotas and work speed metrics used to evaluate workers’ performance. The law also requires for this information to be communicated in employees’ preferred language. Organizers say this will force Amazon to be transparent with its employees about the company’s often opaque workplace productivity metrics — a system they claim increases injuries among workers. The legislation also prohibits employers from imposing quotas on workers that prevent them from taking bathroom, food, rest and prayer breaks.

The law is the product of years of organizing by East African migrant workers, many of whom came to Minnesota as refugees escaping Somalia’s civil war in the 1990s and formed what became the largest Somali diaspora community in the United States. Somali workers now make up large numbers of Amazon’s labor force at warehouses. They were the first in the country to take on Amazon’s labor practices in 2018, when a cohort of workers in the Minneapolis area staged a walkout over working conditions at local warehouses, forcing the company to the negotiating table. 

“Before anyone in the labor movement, we took on Amazon,” said Abdirahman Muse, the executive director of the Awood Center, a Minnesota-based nonprofit that advocates for East African workers’ rights. “And everybody thought we were crazy. But we were not.” Muse compared the Minnesota Somali workers’ trajectory to “the story of David and Goliath” — a small group of refugees and immigrants “facing one of the biggest companies in the world.”

It was the realities of working inside Amazon’s warehouses that prompted Qali to begin organizing for better labor conditions last year. “When I started working there, I said, ‘This is not right what they are doing,’” she recalled. “I always felt like we were slaves there. I always fought against them, I knew my rights. I felt that there were people who have only been in the United States for 30 days. They need money. They come from hunger. They will take anything. And I think that’s what Amazon depends on.” 

“I want people when they come to America to know that they are still human,” she added. “This country does stand for what they believe, but you have to speak — and act.”

Related Articles

Amid chaos, Pakistan shut down the internet to little effect

Javeria Khalid

Immigrating to the US? ICE wants your biometrics

Erica Hellerstein

Texas lawmakers want to erase abortion from the internet

Erica Hellerstein

The post How Somali workers in the US are fighting Amazon’s surveillance machine appeared first on Coda Story.

But not for our colleagues from greater China, perhaps the world’s first mover on all things authoritarian tech. Years later, I worked on a story about how shutdowns were routine in China’s ethnic minority regions and could last for months at a time. Political riots in Lhasa triggered a shutdown that lasted from March until December of 2008. In 2009, protests in Xinjiang, home to a majority of China’s Uyghur Muslim population, led to a yearlong blackout. Chinese authorities were ahead of the curve on surveillance tech, too. In 2017, Xinjiang residents were made to install Jingwang, a surveillance software system as pervasive as products like Pegasus, but with added features like a remote control option that would allow the operator to manipulate the user’s phone.

Xinjiang and the systematic oppression of Uyghurs was the focus of one of the two major U.S. congressional hearings on China last week. But only one made major media headlines — and it wasn’t the hearing on what U.S. officials now refer to as the Uyghur genocide.

Instead, the congressional grilling of TikTok CEO Shou Zi Chew grabbed most of the attention. Members of Congress from both parties put on a spectacle of Cold War-style grandstanding, complete with the Sinophobic pigeonholing of Chew. Representatives repeatedly suggested that Chew was an associate of the Chinese Communist Party. And when Texas Republican Dan Crenshaw insisted that Chew himself was Chinese, the executive had to state for the record that he is from, and lives in, Singapore, an entirely separate country.

The Chinese government’s abysmal human rights record and mass incarceration of Uyghur Muslims did make it into the hearing, as representatives heard again how in 2019, TikTok was removing content concerning human rights in western China. But the TikTok hearing wasn’t really about human rights, in China, or in the U.S. 

It also didn’t seem designed to get new information from the CEO — on a few occasions, representatives literally cut him off before he could answer their questions. Instead it was, as Republican Committee Chair Cathy Rodgers put it, about “American values” and a desire to show anti-China toughness towards the leader of TikTok, the parent company of which indeed is Chinese. 

Right now, TikTok is subject to more regulatory scrutiny and requirements than any other major social media platform in the U.S., essentially because of its connection to China. All the while, the most clear and present threats posed by China keep failing to capture enough public attention to drive meaningful change in how the U.S. responds to the challenge of this enormous world power.

U.S. government agencies are now banned from using commercial spyware, thanks to an executive order from President Biden. The order is limited to spyware that poses security risks or “significant risks of improper use by a foreign government or foreign person.” Although this doesn’t quite constitute a wholesale ban on spyware, it is an important step, especially following last year’s revelations that the FBI had considered using NSO Group’s Pegasus — software that is perhaps best known for being abused by governments — as an investigative tool. The order explicitly bans spyware that enables the collection of “information on activists, academics, journalists, dissidents, political figures, or members of non-governmental organizations or marginalized communities in order to intimidate such persons” — key details reflecting the reality of how spyware has been used to undermine democratic institutions around the world. 

Mexican president Andres Manuel López Obrador admitted that his government spied on the human rights defender Raymundo Ramos. The announcement followed the release of internal documents showing how the Mexican military used NSO Group’s Pegasus spyware to surveil Ramos, who had been helping families facing threats from drug traffickers. The president’s office also took the opportunity to cast doubt on the veracity of the full set of documents, which became public due to a major hacking operation but were then reviewed and verified by technical and legal experts. In any case, the admission is a big deal for Mexico, where the executive branch has a long history of denying or simply turning a blind eye to the evidence of state abuses of due process and human rights.

Russian authorities are using facial recognition to stop protesters before they even hit the streets. Authorities are using surveillance systems in the Moscow metro to identify likely protesters and stop them before they even appear at a protest, often using social media posts as grounds for arrest. Although the facial recognition system has certainly attracted more attention since Russia began its war in Ukraine, it is nothing new — Moscow first deployed the technology in 2017. But a review of 2,000 pending criminal cases by Reuters, with the aid of the Russian human rights group OVD-Info, shows that the system has been used to prosecute hundreds of protesters since the start of the war.

WHAT WE’RE READING

This week, lots of people read a New York Times op-ed by Tristan Harris and Aza Raskin, of the Center for Humane Technology, and historian Yuval Harari, about the apparent impending doom of AI. The piece drove a lot of experts nuts. I don’t want to spill more ink on the subject, but the issues at hand are really important. So I’m recommending two Twitter threads by esteemed scholars who know this stuff much better than most, have nothing to gain from speaking about tech in outlandish terms and are great at separating hype from reality.

• Bentley University math professor Noah Giansiracusa dressed down the op-ed almost line by line. In my favorite tweet, he points out that the authors describe chatbots as “humanity's most consequential technology” and asks, “Are you seriously putting chatbots above antibiotics, pasteurization, the internet, cell phones, smart phones, cars, planes, electricity, the light bulb..?”

• “Design Justice” author and former MIT professor Sasha Costanza-Chock launched a fascinating, forward-looking thread with this provocation: “Generative AI systems are trained upon vast datasets of centuries of human creative and intellectual work. They should thus belong to the commons, to all humanity, rather than to a handful of powerful for-profit corporations.” 

The post U.S. lawmakers grill TikTok, but it’s all bark and no bite appeared first on Coda Story.

After the decision, red states moved quickly to ban or severely limit access to the procedure. This made the virtual sphere uniquely important for people seeking information about abortion, especially those living in states that have outlawed the procedure with little or no exceptions. 

Google searches for abortion medications increased by 70% the month following the court ruling. People flocked to social media platforms and websites with resources about where and how to end a pregnancy, pay for an abortion or seek help to obtain an abortion out of state. 

Despite state laws criminalizing abortion, these digital spaces are legally protected from liability for hosting this kind of content. That’s thanks to the landmark Section 230 of the 1996 Communications Decency Act, the 26 words that are often credited with creating today’s internet as we know it. Thanks to Section 230, websites of all kinds are protected from lawsuits over material that users might post on their platforms. This legal shield allows sites to host speech about all kinds of things that might be illegal — abortion included — without worrying about being sued.

But the future of 230 is on shaky ground. Next month, the U.S. Supreme Court will hear oral arguments on a case that challenges the scope of the landmark internet law. The Court’s decision could have sweeping consequences for digital speech about abortion and reproductive health in a post-Roe America. 

THE BACKGROUND

When armed ISIS assailants staged a series of attacks in central Paris in November 2015, an American college student named Nohemi Gonzalez was among the 130 people who lost their lives. Her family has since taken Google (the owner of YouTube) to court. Their lawyers argue that the tech giant aided and abetted terrorism by promoting YouTube videos featuring ISIS fighters and other material that could radicalize viewers and make them want to carry out attacks like the one that killed Nohemi. Central to the case is YouTube’s recommendation algorithm, which feeds users a never-ending stream of videos in an effort to keep them hooked. Independent research has shown that the algorithm tends to promote videos that are more “extreme” or shocking than what a person might have searched to begin with. Why? Because this kind of material is more likely to capture and sustain users’ attention.

Section 230 protects Google from legal liability for the videos it hosts on YouTube. But does it protect Google from legal liability for recommending videos that could inspire a person to join a terrorist group and commit murder? That is the central question of Gonzalez v. Google. If the Supreme Court decides that the legal shield of Section 230 does not apply to the recommendation engine, the outcome could affect all kinds of videos on the platform. Any video that could be illegal under state laws — like abortion-related content in the post-Roe era — could put the company at risk of legal liability and would probably cause Google to more proactively censor videos that might fall afoul of the law. This could end up making abortion and reproductive health-related information much harder to access online.

If this all sounds wonky and technical, that’s because it is. But the Court’s decision has the potential to “dramatically reshape the internet,” according to Eric Goldman, a professor at California’s Santa Clara University School of Law specializing in internet law. 

Algorithmic systems are deeply embedded in the architecture of online services. Among other things, websites and social media platforms use algorithms to recommend material to users in response to their online activity. These algorithmic recommendations are behind the personalized ads we see online, recommended videos and accounts to follow on social media sites and what pops up when we look at search engines. They create a user’s newsfeed on social media platforms like Facebook and Twitter. They have become a core feature of how the internet functions.

WHAT ARE THE STAKES IN A POST-ROE AMERICA?

If the Supreme Court rules in the plaintiffs’ favor, it could open up a vast world of possible  litigation, as websites and platforms move assertively to take down content that could put them at legal risk, including speech about abortion care and reproductive health. Platforms then would face the threat of litigation for recommending content that stands in violation of state laws  — including, in thirteen cases, laws against abortion. 

“That's going to dramatically affect [the] availability of abortion-related material because, at that point, anything that a service does that promotes or raises the profile of abortion-related material over other kinds of content would no longer be protected by Section 230, would be open for all these state criminal laws, and services simply can't tolerate that risk,” Goldman explained. 

In this scenario, technology companies could not only be exposed to lawsuits but could even find themselves at risk of criminal charges for algorithmically recommending content that runs afoul of state abortion bans. One example is Texas’ anti-abortion “bounty” law, SB 8, which deputizes private citizens to sue anyone who “aids or abets” another person seeking an abortion. If the Court decides to remove Section 230’s shield for algorithmic amplification, websites and platforms could be sued for recommending content that helps a Texas resident to obtain an abortion in violation of SB 8. Most sites would likely choose to play it safe and simply remove any abortion-related speech that could expose them to criminal or legal risks.

The abortion information space is just one realm where this could play out if the Court decides that Section 230’s protections do not apply to algorithmic promotion of content. Anupam Chander, a law professor at Georgetown University who focuses on international tech regulation, explained: “Making companies liable for algorithmically promoting speech when they haven't themselves developed it will lead to the speech that is most controversial being removed from these online services.”

Goldman had similar concerns. “We’ve never had this discussion about what kind of crazy things could a state legislature do if they wanted to hold services liable for third-party content. And that's because Section 230 basically takes that power away from state legislatures,” he said. “But the Supreme Court could open that up as a new ground for the legislatures to plow. And they're going to plant some really crazy stuff in that newly fertile ground that we've never seen before.”

Consider the #MeToo movement. Section 230 protects platforms against defamation lawsuits for hosting content alleging sexual harassment, abuse or misconduct. Without the law’s shield, the movement could have had a different trajectory. Platforms may have taken down content that could have exposed them to lawsuits from some of the powerful people who were subjects of allegations.

“That kind of speech, which we have seen the internet empower over the last decade in ways that have literally reshaped society, would lead to the kind of liability concerns that would mean that it would be suppressed in the future,” Chander added. “So, when someone claims that Harvey Weinstein assaulted them, companies are in a difficult position having to assess whether or not they can leave that up when Harvey Weinstein's lawyers might be sending cease and desist and saying, ‘we're going to sue you for it for defamation.’” 

Proponents of Section 230, who have long argued that changing or eliminating the law would end up disproportionately censoring the speech of marginalized groups, are hoping to avoid this scenario. But it’s hard to predict how the Supreme Court justices will rule in this case. Section 230 is one of the rare issues in contemporary American politics that doesn’t map neatly onto partisan or ideological lines. As I reported for Coda in 2021, conservative and liberal politicians alike have taken issue with Section 230 in recent years, introducing dozens of bills seeking to change or eliminate it. Both U.S. President Joe Biden and former president Donald Trump have called for the law to be repealed. 

“This is not just a left-right issue,” Chander explained. “It has this kind of strange bedfellows character. So I think there's a real possibility here of an odd coalition both from the left and the right to essentially rewrite Section 230 and remove much of its protections.”

If the Supreme Court decides that platforms are on the hook legally for recommendation algorithms, it may be harder for people seeking abortions to come across the information they need, say, in a Google search or on a social media platform like Instagram, as those companies will probably take down (or geoblock) any content that could put them at legal risk. It feels almost impossible to imagine this scenario in the U.S., where we expect to find the world at our fingertips every time we look at our phones. But that reality has been constructed, in large part, on the shoulders of Section 230. Without it, the free flow of information we have come to expect in the digital era may become a relic of the past — when abortion was a constitutional right and information about it was accessible online. The Supreme Court’s decision on this tech policy case could, once again, turn back the clock on abortion rights.

Related Articles

Women develop a code to discuss abortion on TikTok

Rebekah Robinson

Roe reversal puts spotlight on Meta’s abortion content moderation policies

Erica Hellerstein

How changing a 26-word US internet law could impact online expression everywhere

Erica Hellerstein

The post What a law designed to protect the internet has to do with abortion appeared first on Coda Story.

WeChat is an omnipresent Chinese messaging app, and Kokas, a media studies professor at the University of Virginia, needed it to talk to Chinese sources for her research. But, as Kokas told me, it soon became “a very meta experience.” To have WeChat on her personal phone meant that “you were subjecting yourself to precisely the type of surveillance that you were writing about.”

In the book, Kokas analyzes how Chinese firms and the Chinese government gather data on U.S. citizens for political and commercial gain, putting U.S. national security at risk. China is able to do this, Kokas points out, in part because the U.S. government does not have substantial regulations in place to protect users and their data.

“By tracing how China and the US have shaped the global movement of data, I hope this book empowers citizens around the world to navigate the complex terrain created by Silicon Valley, Washington, and Beijing,” she writes.

I recently spoke with Kokas on the phone. Our conversation has been edited for length and clarity. 

What do “digital sovereignty” and “data trafficking” mean in layman's terms? 

Digital sovereignty is the idea of control over a country’s digital resources. Digital sovereignty is something that we see in countries that are trying to protect their digital domain from oversight from other countries. The Chinese government has a more expansive vision called cyber sovereignty, which is that any digital space that a country touches should be part of their digital domain.

Data trafficking is the movement of data from one country to another without the consent of users and without their understanding of the implications of their data being moved between national data regimes. For example, if I sign up for TikTok here in the U.S. and I find out that my data has been accessed in another country, that would be data trafficking. 

My favorite line in the book is when you write, “Most people are simply not exciting intelligence targets.” So what are the implications of data trafficking for most Americans in their daily lives? 

People are afraid that they are individually going to be targeted, and there are some scary stories, but ultimately the more interesting data for the Chinese government and for Chinese firms is actually at scale. So while you might not personally be interesting, you plus all of your neighbors, or you plus all of the people in your state, yield really rich insights that can enable the tracking and mapping of a whole society.

And while most people aren’t that interesting, there are specific subgroups that face intensive targeting, like Hong Kong democracy activists, as well as Uyghur and Tibetan activists. 

I also think there are other layers that are significant. One is economic risk. U.S. companies can’t gather data in China the same way that Chinese companies can in the United States, and that creates a fundamental asymmetry in the development of the digital economy in ways that will have long-standing implications for the development of products. At a certain point, it’s not necessarily just about spying or surveillance. It’s about what types of products you can build.

The third issue is national security. These platforms are becoming essential in daily life and the functioning of society. For example, TikTok now functions as a form of critical communications infrastructure. Chinese firms have also become involved in gathering and using health data and agricultural data from the United States. If that breaks down or if the Chinese government decides to pull participation from these firms, which they can do, it leads to a fundamental destabilization of key areas in the U.S. and global economy — areas like communication, health, food production. 

That’s not a risk that I think most people want to take.

Do you think the United States is at fault for not better protecting user data? Or is China more at fault for taking advantage of those weaknesses? 

A lot of China’s ability to go into other countries and propose tech platforms that rapidly gather data builds on the fact that U.S.-based companies have already been there. A great example of this is TikTok being officially based in the Cayman Islands. This is a classic move by U.S. firms to escape U.S. government scrutiny. And TikTok adopted this, so while their headquarters are officially in Beijing, they’re domiciled in the Cayman Islands. The other thing that U.S. firms pioneered was a lack of algorithmic transparency. And that’s at the foundation of a lot of these business models from which many Chinese entrepreneurs learn to grow their businesses.

The first and most important thing the U.S. government should do is pass national data regulations that have actual enforcement requirements in place. But there are significant differences within the U.S. government about what is and is not acceptable in terms of government oversight over corporations, as well as oversight over data. And even if laws are passed, enforcement is still really challenging. 

You present these issues as being contested, but it seems that the U.S. isn’t putting up much of a fight. 

The title should be something like, “China is taking over the digital world, and the U.S. kind of agreed to it.” But people I interviewed in the U.S. government and tech corporations would argue that by not heavily regulating the U.S. digital landscape, U.S. platforms are able to grow and compete with China that way. The other aspect is this resistance to changing U.S. data governance policies because that would be “letting China win” by adopting too many aspects of the Chinese model. I don’t fully agree with that framework. 

You wrote that you felt a sense of urgency while working on the book. Why did you feel that way? 

A lot of people outside China haven’t experienced China’s digital control directly, so they don’t understand the seriousness of what it means for that model to be exported and how difficult it is to put the genie back in the bottle once it’s out.

Related Articles

Repressive regimes around the world are nationalizing the internet and isolating people

Liam Scott

How TikTok opened a window into China’s police state

Isobel Cockerell

China’s Digital Wall Around Tibet

Nithin Coca

The post China is gaining control of the world’s data as the US stands by appeared first on Coda Story.

Even in Democratic-majority cities, public anxiety about crime seems to be peaking. Determined to assuage people’s concerns (and keep their votes), major cities including San Francisco, Chicago, and New Orleans are turning to technical surveillance as a solution.

This marks a big shift, especially for a city like San Francisco, which in 2019 became the first U.S. city to ban the use of facial recognition technology by local public agencies, including the police. Boston, Portland, Oakland, and Jackson, Mississippi, have since followed San Francisco’s lead, passing similar restrictions of their own that prevent public agencies from using privately-developed technologies to identify individuals in the course of criminal investigations or other procedures. 

Spearheaded by privacy advocates and buoyed by mass protests against police abuse after the killing of George Floyd, these policies were intended to keep cities from treading into the legal and ethical gray area where facial recognition technology currently sits. 

But now the tide seems to be turning. A recent poll found some 65% of voters in San Francisco report feeling less safe today than they did in 2019.

“We went from a long-term view to an extremely short-term view,” explained Tracy Rosenberg, the advocacy director for Oakland Privacy, a group that advocates for surveillance oversight in the Bay Area. 

“The narrative that was dominant in 2019 was the long-term implications of the ubiquitous use of facial recognition, which is basically the end of public anonymity. And I think that narrative has largely been replaced by a narrative that [says]: ‘Who cares about the future when right now your car is getting stolen or your store is being looted?’ And that basically the short-term implications on your life right now are more important than any sort of future surveillance state that might develop.”

Public concern about crime has clearly gone up, but national crime data reveals a complex picture. The murder rate spiked in 2021, reaching its highest point in nearly 25 years, but now appears to be decreasing, with homicides in major cities down nearly 5% in 2022. All other kinds of violent crime have held steady or dropped since 2019, according to Pew. And cities’ experiences with violent crime are not uniform. As of November 2022, murders have increased by nearly 30% in New Orleans and Charlotte compared to the same time period in 2021, and decreased in others, including San Francisco and Oakland. 

Despite San Francisco’s pioneering ban on the use of facial recognition technology, in September 2022 the city’s Board of Supervisors passed a policy that will allow law enforcement to access the video footage of private security cameras in real time. During a 15-month pilot phase, San Francisco police will be able to view up to 24 hours of live video footage from private surveillance cameras during criminal investigations and large public events. 

In a letter to city officials, a coalition opposing the ordinance, including the American Civil Liberties Union (ACLU) of Northern California and the San Francisco Public Defender's Office, argued the proposal “massively expands police surveillance” and could give officers the ability to “surveil any large gathering of people in San Francisco, including the crowds that gather for the Pride Parade, street markets, and other political and civic events.”

The Electronic Frontier Foundation’s Matthew Guariglia described the Board’s decision as an attempt to “[put] voters at ease that something, anything is being done about crime.”

These San Francisco legislators are not alone. Their decision reflects a broader trend playing out in left-leaning cities nationwide. Cities are expanding the use of surveillance technology to reduce crime, or at least assuage some citizens’ concerns about crime, sometimes without clear evidence that these tools are effective as such. These cities also risk entrenching a permanent surveillance infrastructure that may be difficult to dismantle down the road. “The history of surveillance suggests that it's not easy to put the genie back in the bottle,” argues Rosenberg. 

One of  the most high-profile examples of this dynamic comes out of New Orleans, where lawmakers are poised to expand police surveillance less than two years after passing a sweeping facial recognition ban. In July, the New Orleans City Council voted to allow the city police department to request access to facial recognition technology from the Louisiana State Analytical and Fusion Exchange, which analyzes data for police, to investigate certain kinds of crimes, including rape, murder, carjacking, robbery, and “purse snatching.” 

The ordinance passed amid a surge in violent crime in New Orleans not seen since the mid-1990s. In early July, just weeks before the city council approved the policy, New Orleans reportedly had the highest murder rate in the nation. Supporters of the measure, including the city’s mayor, claimed that it would help police rein in crime by helping officers track down perpetrators more effectively. 

This raises a critical question: Do these tools actually help reduce or solve crimes? As one city council member who voted against the New Orleans policy pointed out, the argument was not backed up by empirical evidence. 

During a hearing on the vote, an official with the police department admitted that he had no information about how frequently the department used facial recognition before it was banned in 2020 and whether its use had led to any arrests or convictions. “You have no data, sitting here today, telling me that this actually works, that it leads to arrests, admissions or clearances,” the councilmember Lesli Harris said. 

The Louisiana chapter of the ACLU blasted the council’s decision to “expand racist technologies,” highlighting research that has found that facial recognition disproportionately misidentifies women and people of color. A 2019 federal study found that the majority of facial recognition systems are biased, misidentifying Black and Asian faces at significantly higher rates than their white counterparts. 

These flawed matches have real-world consequences: At least three Black men in the U.S. have been wrongfully arrested after facial recognition software incorrectly identified them for crimes they did not commit.

Elsewhere, cities are embracing a controversial gunshot detection surveillance technology that a study from the Northwestern School of Law found to be “inaccurate, expensive, and dangerous,” sending police on “unfounded deployments” in predominantly Black and Latino neighborhoods. The technology, ShotSpotter, uses a system of discrete acoustic sensors to identify the location of gunshots and then send an alert to the police, who can then decide to send an officer to the scene of the alleged crime.

The firm has contracts in over 120 cities nationally, some of which have come under fire for pouring millions into a technology that critics say is error-prone and ‌ineffective. ShotSpotter contests claims of inaccuracy, saying the technology has a 97% accuracy rate. But a 2021 analysis of the Chicago Police Department’s use of ShotSpotter by the city’s Office of Inspector General found that just ​​9% of alerts were linked to gun-related crimes.

A recent class action lawsuit, filed by the MacArthur Justice Center at Northwestern University, alleges that the city “has intentionally deployed ShotSpotter along stark racial lines and uses ShotSpotter to target Black and Latinx people.”

Despite such criticisms about the technology and its impact on policing, cities are still using it. Earlier this month, the Detroit City Council ended a months-long, divisive debate about whether to expand ShotSpotter when it approved a $7 million contract to deploy the system to 10 new neighborhoods in the city. Detroit’s decision came just days after Cleveland’s City Council voted to quadruple the size of ShotSpotter’s current use area. Other cities that have recently moved to expand or renew contracts include Sacramento, Houston and Chicago.  

Meanwhile, in New York, Mayor Eric Adams, whose ‘90s-style “tough on crime” rhetoric has been a hallmark of his campaign and time in office, has been a vocal proponent of high-tech policing, including facial recognition and gunshot detecting technology like ShotSpotter. Adams, a former New York City police officer, has sought to dramatically expand the use of facial recognition within the police department and has expressed interest in installing metal detectors in city subway stations and replacing school metal detectors with new technology that would scan students for weapons. 

The overall picture, says Albert Fox Cahn, the founder and executive director of the Surveillance Technology Oversight Project in New York, is one of “surveillance opportunism” in which technology companies are pitching surveillance systems to lawmakers and law enforcement agencies seeking to quell concerns about public safety. To promote these technologies, Fox Cahn added, some public officials have positioned the expansion of surveillance in cities as a more humane alternative to traditional policing.

Guariglia of the Electronic Frontier Foundation explained, “Surveillance doesn’t come without the iron fist of the police department. Because even if they capture something on surveillance and they want to arrest a person, that person is not going to be arrested by a camera. They're going to be arrested by a person with a nightstick and handcuffs and a gun.” At the end of the day, this trend pushes them towards a vision of citywide surveillance favored by some of the world’s most authoritarian regimes.

For now, San Francisco’s facial recognition ban remains intact. But some civil liberties advocates worry that the decision by the city’s Board of Supervisors to grant the police wider surveillance powers could give license to other cities and jurisdictions to follow suit. 

“I think that's one of the most disturbing parts of what happened in San Francisco,” explained Oakland Privacy’s Rosenberg. “Because when you don't have those facial recognition bans in place, the green light from a big city, a progressive city, a city that's been famous for innovations in surveillance and looking at things with a critical lens — I think it provides a sort of implicit invitation to other cities that don't have these bans in place to jump on the bandwagon.”

Still, as many privacy experts are quick to point out, it’s unclear if this trend will have staying power. They point to the general ebbs and flows of crime — at its peak, a sense of public insecurity tends to garner more support for policing and a willingness to erode civil liberties than it may when citizens feel safer — as well as the strength of the growing anti-surveillance movement. 

“Five years ago, it was unimaginable that there could have been a ban on any type of surveillance technology,” ​​Matt Cagle, a senior staff attorney for the Technology and Civil Liberties Program at the ACLU of Northern California, remarked. “When we started talking about this at the ACLU, we got laughed at by folks in political spaces when we proposed the idea of banning facial recognition.” Now, though, he adds, there are “more groups who are opposed to government surveillance at the local level…by an order of magnitude over what that was five or ten years ago. And I think that’s an important trend even though on the policy itself, the votes didn’t swing the right way this time.” 

In the next five years, we will see if those groups have the power to put the genie back in the bottle.

Related Articles

‘I felt like I was a prisoner’: The rapid rise of US immigration authorities’ electronic surveillance programs

Erica Hellerstein

Between the US and Mexico, a corridor of surveillance becomes lethal

Erica Hellerstein

How San Francisco police surveillance closed in on Black Lives Matter protests

Caitlin Thompson

The post As anxiety about crime peaks, US cities look to surveillance tech. But does it actually work? appeared first on Coda Story.

Ostensibly a photobook on the small Macedonian town of Veles, which made international headlines in 2016 as the unlikely factory of pro-Trump fake news, The Book of Veles created a furore after Bendiksen revealed that the project’s images were synthetically generated using 3D software and the book’s text was written entirely by artificial intelligence. 

Fascinated with the story of Veles as well as with developments in synthetic imagery, Bendiksen set out to see just how “real” he could make his images. Although he left breadcrumbs throughout the text, to his surprise the book was published in April 2021 to “nice, positive echo-chamber feedback,” said Bendiksen. No one questioned the authenticity of the images or text. He stepped up the game by then submitting his manipulated photographs to the world’s most prestigious photojournalism festival Visa pour l'image which screened his images in September.

“I thought, what could be a higher threshold for fooling someone with junk, synthetic images than this crowd?,” said Bendiksen. “I gave it 24 hours for someone to come with some questions about the work. It didn’t happen.” The photographer’s final attempt to out himself involved buying a squadron of Facebook and Twitter bots to attack him online. The bots posted dozens of messages claiming that his work was fraudulent, only to have Bendiksen’s colleagues and supporters rush to his defense.

Bendiksen finally came clean in an interview on September 17 with Magnum Photos. We spoke with Bendiksen about what he’s come away with from the experience and how the project has continued to take unexpected turns even after his revelation last month.

This conversation has been edited for length and clarity.

What were the main goals that you set out with?

I wanted to create some discussion and awareness around this technology but I didn’t go into this project wanting it to be just a technical demonstration of something. I get into my projects because it’s a good story to tell. If it hadn’t been for all these exciting layers of Veles and mythology I wouldn’t have done this. All of these parts of the puzzle fell miraculously into place.

Yes, tell us more about how the puzzle fell into place, there are so many bizarre layers to this project.

One was the discovery of the historical layers to the story. The town Veles is named after this pre-Christian Slavic god who was a kind of sneaky guy, a shapeshifter who turns into a bear, a god of chaos and deception and magic. I thought he would probably be super happy about all this deception going on in Veles.

The introduction written by AI is still a bit clumsy. Some of your images, especially the one of a bear stomping through town, feel like they should have set off some alarm bells. Yet it was all “real” enough that no one questioned it.

That was also something that frightened me. The technology in the field has developed a lot even from when I started using it to when the project was done. It’s clear to me that within a few years 95% or more of people will have a hard time decoding whether an article was written by a New York Times journalist or a bot. I wanted this to be a look into what I think is the near future of our information landscape.

How are our current concerns about automation or AI different from all the previous technology scares?

In photography at every step when there is a new technology people say it’s the collapse of truth. Whether that was when digital cameras came or when Photoshop showed up. People always cried wolf like that. Maybe I’m the same and this is a bunch of wolf crying and this will sort itself out nicely.

But I think the difference is: automation and synthetics. We will always have good journalism but I think it will be mixed in with so much synthetic junk or half synthetic junk that it will just be very, very chaotic and difficult to navigate. The difference is that the automation of it gives it such a bigger potential for spread and makes it so difficult to contain.

You’ve repeatedly compared your work to a penetration test that hackers run searching for vulnerabilities in their code. These tests then allow companies to better fortify their software and find solutions for loopholes in the code. What did your stress test reveal and what solutions did it reveal to you?

I’m not trying to pretend I have all the answers but I think there are many levels to it. I think the content verification business will be an industry unto itself. This is also a call out to social media platforms who I believe have failed us in many ways on this front thus far to step up the game. This is a call out to our education system. As a father of four, I’m wondering how my children are going to handle this. To be a functional citizen in the next 50 years, navigating the information space should be at least as important a subject as mathematics in schools.

This story continues to go in unexpected directions. Tell us about the latest.

A junk information site pretending to be a newspaper in Texas called Texas News Today came out from a very similar story that was stolen from Wired about my project. I think it’s automated: they suck stuff into their system, rewrite it and then it gets blasted out again on all these websites which look very similar to what the fake news websites out of Veles were doing. It’s the same business model. I looked into it and it turns out these people at Texas News Today are a junk news site based in Pune, India running a bunch of sites from this location in different languages.

Now, these fake news sites are stealing stories about my fake news project. And there are people quoting the fake writer who supposedly wrote the piece as some credible source. It’s full circle. There you can see the whole mechanism of the chaos.

The post A photographer and artist walk into a fake news factory appeared first on Coda Story.

However, the technology has attracted significant criticism, citing biases inherent to its algorithms and alleging that its use contributes to the over-policing of marginalized communities. 

Now, following a number of high-profile examples, including the 2020 murder of George Floyd by Minneapolis police officer Derek Chauvin, the conversation is turning to how the same methods can be used to combat police brutality.

Enter Future Wake, an interactive website that has received the Mozilla 2021 Creative Media Award. The project uses artificial intelligence to analyze data on fatal police encounters in the U.S. and predict future incidents. It then creates computer-generated avatars to tell the stories of each composite victim.

We sat down with two of its creators — Oz, based in New York, and Tim in the Netherlands — to talk about the motivations behind their work. Both asked to be referred to by their given names only.

This conversation was edited for length and clarity.

Tell us about Future Wake. What was your goal with this project?

Oz: Future Wake focuses on using the principles of predictive policing to predict when the next fatal encounter with the police will occur. The tactics of predictive policing and the way it’s implemented are relatively unknown. Most of the time, whenever we mention it, people bring up “Minority Report.” They only have a fictionalized understanding of the technology. A lot of people don't realize that it's actually in their own cities. So, in order to bring attention to it, we thought about just flipping its application.

When someone enters the website, what do they see?

Tim: At first, you see a warning. We are very much aware of trauma for people who've lived through police violence. You don't see any data immediately. You see the five faces. I thought it was the most important thing to show that we're talking about humans here, not numbers. 

Oz: Each face is a computer-generated image of the next victim from one of the five most populous cities in the U.S. — Chicago, Houston, Los Angeles, New York and Phoenix. Below each victim, you see a countdown that refers to the moment we predict that they're going to die. The people are animated to bring this awareness that they are still alive. Once the countdown ends, that will be the end of their lives. We want to breathe life into the people we predicted. 

Tim: When you click on a person, you enter their space. We predicted the location of the fatal encounter with police. We have a Google Streetview in the background, and it's like you're having a call with them. Then this person tells the story of their own demise. 

The project has two elements. You ran data about fatal encounters with police in the U.S. through predictive algorithms to determine the details of future victims. Then you use deepfake technology to create avatars that represent them. Why did you feel like you needed both? 

Oz: Our data set starts from 2000 until now. We wanted to highlight these recurring patterns of police brutality in the U.S and to hone in on the fact that a prediction still has consequences for a human being. 

Tell us about the databases you worked with. 

Oz: We used two main ones, called Fatal Encounters and Mapping Police Violence. These are citizen-initiated projects that try to capture every fatal encounter with police officers in the U.S. We did try to find police databases. The FBI has one that it started in 2018, but they're mainly relying on self-reported outcomes from police agencies. It’s actually under-representative of what's going on. We put that data through algorithms to predict who — which consists of the gender and ethnicity of the victim — where and when the next fatal encounter would occur.

Each potential victim has a backstory that describes the circumstances in which they were killed. When you click on their face, they tell their story. You used AI text-generation software to do this, right?

Oz: In the databases we used, there were two or three sentences detailing if it was a car chase, if somebody was wielding a weapon or how they were shot. We use this algorithm called GPT-2 to learn the aesthetic of all of these media reports. GTP-2 would then generate future police-related media reports. We then edited the text slightly, to make it in the future tense and the first person.

Why did you use the future tense?

Oz: I see more talk about the horrific events that happened to victims of police brutality after the fact. Occasionally, there are little bubbles of conversation about how we can prevent this in the future. By predicting future victims and showing that this is an ongoing issue, we're asking, ”How can we protect this person from being a future victim?”

Let’s talk about the countdown clock…

Oz: In traditional predictive policing, they use spatial-temporal models to predict where and when a crime will occur. We replicate that. We want to say that this person is going to die at this specific moment. I used a time series algorithm to predict recurring police-related fatal encounters, and was able to predict an estimated day of when someone would die. The clock is supposed to generate a sense of urgency. 

Was there anything that surprised you in the data?

Oz: We looked at the average time between each fatal encounter for each city and for each demographic. It was pretty creepy. Based on the data that we had, in Chicago, Black males had the shortest time in between each incident. It was an average of 34 days. It was quite shocking. Minorities are overrepresented in the database, but I was still surprised by the fact that everyone is represented.

The post Future Wake: the AI art project that predicts police violence appeared first on Coda Story.

From 2014 to 2019, he worked on the acclaimed project and photographic book How to Secure a Country, focusing on the tensions between security and control that exist in Switzerland, where he now lives. His most recent project, Persuasive System, is an interactive installation focused on the use of CCTV in public spaces. We caught up with him for a chat about his work.

This conversation has been edited for length and clarity.

Coda Story: First of all, let’s talk about your most successful project, “How to Secure a Country.” Why did you decide to work on surveillance systems, and why in Switzerland?

Salvatore Vitale: The idea for “How to Secure a Country” came to me in 2014, after Switzerland held a referendum to change the constitution, in favor of reducing mass immigration. I had been living in an Italian-speaking Swiss canton for 10 years, but I had never really asked myself what it meant to be an immigrant there. After that moment, I started noticing how many contradictions were at play in the country, torn between progressivism and conservatism, between a need for safety and an almost all-controlling surveillance system. 

Switzerland was not only my country of residence, but also a valuable case study. As one of the safest countries in the world, I knew that its surveillance system would be a sort of heightened version of those in place elsewhere.

Surveillance takes so many different shapes that understanding them all seems almost impossible. How did you tackle this incredibly broad subject?

I didn’t really envision a project about security and surveillance in the beginning. What I wanted was to understand where the need for safety in Switzerland comes from. It was an interesting angle to tackle, because safety is such an abstract concept that I didn’t know where to look. I did a lot of research, but I didn’t really talk to normal people about it. I partnered with academics and members of law enforcement, the insiders. 

Many of the themes your work deals with are very abstract. How do you represent ideas like surveillance and control visually?

I took an analytical approach, trying to make the abstract idea of surveillance tangible through its actors and its tools. I tried to be creative in the use of my medium. I knew that my subjects were going to be extremely varied, and I tailored my approach accordingly. It’s one thing to photograph police forces patrolling the border, a completely different thing to photograph malware. 

I took a risk. I decided not to capture actions, because surveillance is a process — it’s constantly evolving. I decided to use my images as triggers. I wanted to convey a sense of oppression, in a way that is cynical and clinical.

“How to Secure a Country” took five years to complete. Did you notice the development of surveillance systems over this period of time?

I definitely noticed how the dividing line between the people who control and the people who are being controlled became thinner and more fluid. As algorithms become more sophisticated and biometric surveillance more widespread, the system becomes circular. The people who control are also being controlled at the same time. 

“How to Secure a Country” is closely intertwined with your current project. Can you tell us more about that?

Yes, the premise for both projects is similar: understanding surveillance. But “Persuasive System” is more universal, not focused on a single country, and it’s not a work of photography, but an interactive installation, almost like a performance. 

I essentially built a system with three CCTV cameras. When the installation is working, it constantly captures images and information about people who interact with it, and people can play together, collecting data about each other.

What is its aim?

I’m particularly interested in how video surveillance influences behavior. I wanted to understand what being a data subject means. We, as humans, have a physical body, but we also have an online persona, built on the basis of all the data that is collected about us. Algorithms that collect data from us online operate like black boxes and so do CCTV cameras. When we see them around, we know that they’re filming us and collecting data, but we don’t know which kind. With “Persuasive System,” people can see how the data is collected, and how much of it there is. It’s a lot. Through it, I’m hoping to foster a shock reaction in my audience.

That’s a huge problem of our time. The amount of data that is collected about us online is unfathomable. How do you think your projects help to shed light on this?

I don’t want to give solutions. I think rushing to solve these issues can be dangerous. My idea is to give my audience more awareness, the “weapons” to navigate our reality. By visualizing the technology, I hope to push people to think critically about their existence as data subjects.

All images copyright to Salvatore Vitale. Follow Salvatore on Instagram or on his website.

The post ‘The people who control are also being controlled’ appeared first on Coda Story.

Twitter’s headquarters are a 15-minute walk away, the cloud-based software giant Salesforce is 20 minutes in the opposite direction. But right here — in the epicenter of a chronic housing crisis, recently exacerbated by high tech industry salaries — you would barely be able to guess that this city is home to some of the wealthiest and most powerful people in the world. According to the latest count in 2019, there are around 8,035 people experiencing homelessness in San Francisco, a per capita rate comparable to much larger cities like New York.

For the thousands experiencing homelessness in the Tenderloin, their chance of getting off the streets comes down to a single number, generated by an algorithm. It is meant to assess each person’s unique vulnerabilities and allocate assistance accordingly. But now, even the designers of such systems say that, far from solving the problem of homelessness in the United States, these algorithms are used by local governments to deny assistance to large numbers of people in need.

In 2013, Iain De Jong and his colleagues at OrgCode, a consulting firm specializing in issues relevant to the homeless, created the Vulnerability Index — Service Prioritization Decision Assistance Tool, a scoring algorithm to help address America’s homelessness crisis. They were so successful that versions were adopted by authorities in at least 40 states. Other local governments, like San Francisco, have followed suit and created their own similar tools. Eight years later, De Jong and OrgCode say that cities are misusing their system — and that this has to stop. 

VI-SPDAT was meant to help local social service providers assess what type of housing assistance might best suit a homeless person’s needs. Instead, resource-strapped cities are relying solely on tools such as VI-SPDAT to make a binary choice: who gets housing and who doesn’t. 

“One of the gross misunderstandings and misuses of the tool was making housing decisions based upon the outputs of it,” De Jong told me. “It was never designed to do that.”  In December, OrgCode announced that it would begin phasing out VI-SPDAT and will no longer provide support for cities using the most common version of it. 

But it might be too late. The tool was rolled out one year after the Department of Housing and Urban Development required all cities receiving federal funding for programs that house people who are homeless to adopt centralized assessment processes known as “coordinated entry systems," built on tools such as VI-SPDAT, to allocate accommodation. 

Today, VI-SPDAT offers a cautionary tale of how an algorithm meant to help people resolve a thorny societal dilemma replaced human decision making entirely, with devastating effects for its intended beneficiaries. Yet, it is not the classic story of viewing a complex human crisis through a reductive tech-bro lens. Instead, it is an example of the wilful misuse of a well-intentioned tool by city administers, who have turned to such systems to deflect attention from a persistent and multi-layered problem, rather than attempting to marshal the resources and political will to solve it.

VI-SPDAT grew out of a collaboration between OrgCode and Common Ground, a national organization working to house homeless people. The system generates a vulnerability score out of 17 based on a set of questions about mental health, physical health and risk factors for chronic homelessness, with the aim of aiding case managers to triage people to appropriate resources.

“It was intended to try and help frontline staff better understand, across multiple dimensions, what people's vulnerabilities were, what their risks to housing stability were, so that you can work with the individual to guide a plan of support,” said De Jong, explaining that the score was not meant to be the ultimate criteria for deciding whether or not a person should receive housing assistance. 

But that, unfortunately, is exactly what happened. City administrators simply ignored the fact that VI-SPDAT was intended to provide a starting point for offering assistance and facilitating further conversations between homeless individuals and case workers.

“People just skipped over that step,” he said. “We even heard things like, ‘Well, we just don't have time,’ or, ‘It's inconvenient,” or, ‘Following up with people to get that sort of information is hard work and the survey isn't.’”

Prior to the widespread adoption of coordinated entry systems, the process of providing housing for homeless people was built on relationships between case workers and the individuals in question. Social workers were supposed to develop an understanding of their clients’ needs and assist them accordingly. 

Coordinated entry replaced individualized case management with a standardized system that is — depending on where you stand or who you talk to — either objective and less prone to favoritism, or cold, rigid and brutally mechanical.

“The goal of coordinated entry is to promote equity, and to ensure everyone has the equal ability to access resources,” wrote Denny Machuca-Grebe, the public information officer for the Department of Homelessness and Supportive Housing (HSH) in San Francisco. 

Service providers working with people experiencing homelessness say that deemphasizing human relationships has created an unbending and soulless system that actively impedes the provision of individualized help. Tools such as VI-SPDAT or San Francisco’s similar Primary Assessment algorithm, critics say, have become a way to quickly eliminate people from housing eligibility under the guise of fairness and efficiency.

“You can propose a solution to meet the scale of the problem. Or you can shrink the problem to meet the available solution,” said Joe Wilson, the executive director of Hospitality House, a community-based organization in the Tenderloin that provides services for people experiencing homelessness and runs a shelter. “Coordinated entry reduces the scale of the problem.”

The reality is there simply isn’t enough housing. As of February, 225 of the roughly 7,755 supportive housing units allocated for people experiencing homelessness in San Francisco were vacant and ready for occupancy, according to reporting by the San Francisco Public Press. While over 10,800 people lived in permanent supportive housing as of June of last year, 5,180 people are sleeping on the streets and another 2,855 are living in cars, on couches or in temporary shelters, according to the latest count in 2019.

Wilson, who was homeless in San Francisco himself in the early 1980s, has been a vocal critic of coordinated entry since its launch. “It’s designed not to help people get in, but to keep them out,” he said. 

“I think we went from one extreme to the other,” De Jong said. “We went from a system of care that had really come down to luck, self-advocacy or first come, first served, in terms of how people got housing, to a very hands-off numeric-based approach that was very dehumanizing, in which people were not, in my opinion, always seen as people with potential, strengths and resiliency. They were resigned to a number on a waiting list. And that was just overwhelmingly disheartening.” 

Across the park from San Francisco City Hall, a few blocks south of the Tenderloin, rows of tents are hidden behind a concrete barricade. It’s one of half a dozen city-sanctioned areas with 24/7 security and access to food, water and sanitation. These “safe sleep” sites are the city’s latest experiment in controlling sprawling sidewalk encampments and a strategy to limit the spread of Covid-19 among the homeless population. But, for many, permanent housing remains an unattainable dream.

Primary Assessment, the city’s scoring tool, was created by the local government with community input, but it functions in a similar way to VI-SPDAT. Its questions are phrased and weighted differently, but the fundamental principle of an algorithm assessing vulnerability is exactly the same.

If a person’s score meets or exceeds a certain threshold, they enter the queue to be allocated a housing placement. If their score is too low, they are put into what is known as “problem-solving status,” to be matched with programs that provide assistance other than permanent housing. 

The questions asked of applicants are deeply personal: information on drug or alcohol use, mental health issues, developmental disorders and experiences with sexual assault or domestic abuse. Single adults and people aged 18 to 24 are asked if they have visited detox centers or called into suicide hotlines, or if they have traded sex for a place to sleep.

“You're not my therapist. You don't need to know how many times I've been sexually assaulted in my life,” said Roxie, who had been homeless for most of the past four years, before they were finally housed in a subsidized unit. “I hate the fact that the coordinated entry system is based on how traumatizing your life has been, essentially.” 

Roxie is based in San Francisco, but has traveled the country from Nashville to St. Louis. They have been through the Primary Assessment process on four occasions, and each time, it reopened old wounds. 

"The Department is transparent and respectful about the reality of trauma for the people we serve, and we strive to minimize such impacts through training and standardization across the Homelessness Response System," Deborah Bouck, communications lead at HSH, wrote in response to questions for this article. 

Applicants are advised that they don’t need to go into detail during the assessment, but not answering a question can adversely affect their score, which leads to people like Roxie feeling compelled to share far more than they are comfortable with. 

Xander, whose mother was homeless and grandmother lived in supportive housing, has taken the assessment twice and experienced panic attacks both times. 

“It really sucks trying to bring up these topics,” Xander said. “When I did my housing assessment, I was fresh out of an abusive situation with my family, so it was all raw. When I tried talking about it, I'm like, `Hey, can we skip this question?’”

Xander walked out without finishing their first assessment or getting a score and was forced to return to a bad situation with their family.

“Stepping back from the complex system and how it all works, fundamentally, that is a really intense thing to ask anyone to do, to offer up even a piece of their vulnerability to somebody,” said Kenn Sutto, who conducts Primary Assessments at the Homeless Youth Alliance, a grassroots harm reduction coalition which is contracted by the city to serve as an access point for young people experiencing homelessness. Because these access points are contracted by the city, they are required to use the Primary Assessment. People have told Sutto that the coordinated entry assessment feels like the “trauma Olympics.”

“If you need to know to be able to determine how much support someone is going to need in housing, that's a legitimate case,” said Wilson, explaining that it is necessary to know if an individual has a disability that makes it difficult to use stairs or a medical condition that requires a private bathroom. But he believes that questions about substance use or sexual violence should not be relevant to determining whether someone qualifies for housing. 

“If you're asking it to determine the severity of one's homelessness, why do you need to know that? Everybody that comes needs a roof over their head,” he said.

For De Jong, the designer of VI-SPDAT, these intrusive questions conflate two very different concepts — eligibility, which determines the specific housing program from which they will gain assistance, such as veteran or HIV-positive housing, and “depth of need,” which simply refers to a person’s unique risks and vulnerabilities.

“That's where I'll say the failure of implementation of VI-SPDAT to its true intention is remarkably pronounced,” he said. 

San Francisco’s Department of Homelessness and Supportive Housing doesn’t share this view. "The assessment is used to prioritize those most vulnerable,” said Bouck at HSH. But in practice, many people don’t score high enough to be considered for housing assistance.

Roxie got lucky and recently moved into an apartment in San Francisco’s Mission district, but the trauma of the repeated assessments remains with them. 

“It’s such bullshit that we have to be willing to expose ourselves and be that vulnerable in order to get housing. It’s exploitative,” they said.

If you believe the strategic five-year plan, issued by San Francisco’s Department of Homelessness and Supportive Housing in 2017, coordinated entry is supposed to “make homelessness a rare, brief, and one-time event.” 

On the other hand, you could listen to Jamale, whose experiences suggest an uncomfortable truth: coordinated entry systems will never fix homelessness. Instead, they will keep kicking people to the back of the queue, until the system deems them homeless enough for housing assistance.

Jamele, who asked that his name be changed for reasons of privacy, has taken the assessment twice. Both times he didn’t make the cut for housing priority status. 

“I felt like being honest and trustworthy gets me nowhere in that system, because I didn't qualify,” he said. “And I opened myself up to every little incident that I've had, just to be told. ‘Nope, not now.’ It really drains you. It makes you wonder, ‘Why am I still here, or what's my purpose in society, if I can't access help?’”

Jamale, who is originally from Los Angeles, moved north about 15 years ago. A quiet, kind man in his early 30s, he enjoys building model rockets. In college, his favorite course was on legal terminology. “I wanted to be an attorney that could help people get out and stay out of jail,” he said. 

But Jamale didn’t know anyone in San Francisco when he first arrived, and he couldn’t afford rent. Without any idea where to turn for help, he ended up on the streets, where he has stayed for most of the past 10 years.

By focusing on a specific definition of vulnerability, San Francisco is inadvertently excluding people like Jamale from housing services. With a roof over his head and support to process the trauma caused by years of homelessness, he could be one of the success stories that many service providers hope to find. Ironically, that could be part of the reason why he hasn’t been allocated housing yet. 

Like the vast majority of people who go through the coordinated entry system in San Francisco, Jamale wasn’t determined to be vulnerable enough for permanent supportive housing. According to data from HSH, only 17% of the 18,327 assessments conducted from January 2019 through May 2021 resulted in people moving into a new home.

Of the 11,979 single adults who were assessed between July 2018 and June 2021,  a staggering 69% did not meet the threshold for housing priority status, according to HSH data obtained by Coda Story via a public records request. The numbers are better for other demographics — 88% of families and 58% of youth applicants receive housing referrals. 

“We're basically saying, ‘Wait until you're sick enough, until you've been impacted to a point that's very detrimental to a person who is experiencing homelessness, or until that point when this algorithm is going to score you very high,’” said Laura Valdez, executive director of Dolores Street Community Services, a nonprofit which will become a coordinated entry access point for adults later this year. Currently, there are only two such offices in the whole city. Valdez hopes that operating a third will give her team greater insight into how the system works. 

It is hard to say why Jamale has not yet been housed. Coordinated entry’s scoring system is deliberately opaque. The people conducting the assessments are not even told how the questions are weighted. According to Jeff Kositsky, who served as director of HSH from 2016 to March 2020 and led the department throughout the implementation of coordinated entry, this is to avoid service providers coaching their clients. 

“We want to make sure we’re getting an accurate picture, and we don’t want case managers to game the system to get their people to the top of the list. It’s a best practice,” Kositsky told the local online newsroom The Frisc in 2017. 

“Any large, complex system that renders critical social services is subject to potential manipulation. While HSH understands there is a possibility for coaching, we also are aware such behavior occurs because staff care about those they serve and want to assist community members in getting housed,” said Bouck, communications lead at HSH, in a statement to Coda Story.

Still, people have guesses as to how it works. Joe Wilson believes that “chronicity of homelessness” — how long an individual has been unhoused — makes up a significant part of the score. He’s right, according to a breakdown of how questions are weighted reviewed by Coda Story, which has not been made public before and was obtained via a public records request by the Coalition on Homelessness. People who have been homeless for more than 15 years get 15 points added to their score. Someone who needs help carrying out daily activities or maintaining housing receives nine points. Experiencing sexual or physical violence in their current living situation can count for 12 points, but only applies if the person is 24 years old or younger. 

Sometimes, things don’t add up. One of Wilson’s clients at Hospitality House is an 87-year-old woman, who has been homeless for 40 years. After going through the coordinated entry process, she didn’t qualify for a housing referral. 

For many, the algorithm’s word is final. Case managers and the people conducting assessments can’t change a score. Individuals placed in problem-solving status can challenge the decision via a clinical case review, but that route is time consuming and requires clients to hand over even more personal information. Many don’t pursue it and instead opt to take the assessment again in six months, remaining homeless in the meantime. Others, like Jamale, just walk away for good. He doesn’t plan on trying a third time.

Without permanent housing, it can be harder to stay engaged with social services like child care, mental health and addiction treatment. Megan Geary is the program director at Central City Access Point, which is contracted by the city to conduct coordinated entry assessments for families. She told me that a significant number of people turn away from other avenues of help after being told that they do not qualify for housing. “We see people cycling through the system much longer. It just feels counterintuitive.” 

For Jamale, it feels like the system is telling him that things need to get much worse before he is deemed worthy of a home.

“Why do I need to be in a hospital bed fighting for my life in order to get housed? If that's the case, what do I have to do? Walking in off the street and asking for help, it seems like a dead road,” he said.

Joe Wilson has been working in homeless services for the better part of the 20 years, starting while he was sleeping at the Hospitality House shelter. He’s furious that human judgement has been replaced by an algorithm. 

“To cede that kind of decision-making authority to a computer, is that what we want to do in our field?” he asked. “We can’t do any better than that? That’s not what I came here to do. I came here to bring me to this mix.”

Because of the rigidity of the coordinated entry system, service providers now can’t get their clients into their own housing programs, even when they can clearly see that doing so is the right course of action. Mary Kate Bacalao, the policy director at Compass Family Services, said her team worked with a client who needed a place to live after giving birth to a baby with special needs. Compass runs a housing program for pregnant women and new mothers, but because the scoring system put the woman in problem-solving status, they couldn’t place her. Social workers ultimately lost track of her. 

HSH disagrees with criticism of the assessment. “Human decision-making is still very much a part of the process. The pandemic has shown that coordinated entry still very much incorporates human decision-making into the model,” wrote public information officer Denny Machuca-Grebe in response to questions for this story. 

He went on to say that the coordinated entry system “strives to center client choice.” To a degree, it does. Families who have made it through the process and receive a temporary rapid rehousing rental subsidy can request a housing case review when a spot in permanent supportive housing becomes available. People who qualify for permanent supportive housing can also be offered up to three units, in the hope of finding one that matches their needs. But many people don’t get that far. 

Service providers highlight that not all types of housing intervention will work for everyone. For example, a rapid rehousing subsidy may not be a good fit for people who need more support.

“The housing resolutions at the end of the assessment are a very cookie cutter approach,” said Geary at Central City Access Point. “We can't really pivot to triaging them to a housing intervention that would maybe better meet those needs and be more helpful to them to be able to achieve longer-term stability. We kind of just have to refer them to whatever is available.” 

The first time Roxie went through coordinated entry several years ago, they were placed in a single-room occupancy hotel (SRO) in the Mission district. It was right after they were sexually assaulted.

“It was one of the worst places that you could put somebody that had just recently been assaulted. It was like a crack den SRO,” they said. 

“I ended up leaving after two weeks. I just said, fuck it, I’m done.”

Meanwhile, people like Kenn Sutto are doing their best to operate within the system. When he conducts a primary assessment, he tries to make people feel as comfortable as possible. Even though he is supposed to ask the questions exactly as written, he is allowed to clarify and steers clear of terms like “substance abuse,” in favor of less stigmatizing phrasing. 

“I'm not here to be a person that's reading off a computer screen,” he said. “My job is to be there for people experiencing homelessness.”

For Wilson, handing a decision as monumental as whether someone gets housing over to an algorithm just isn’t a sustainable or moral option. 

“There were people in my life, particularly when I became homeless, who refused to turn away from me,” he said. “I refuse to do less than that. No computer is going to help me decide the worth of another human being, and who gets what, when and how much.”

The post Who’s homeless enough for housing? In San Francisco, an algorithm decides appeared first on Coda Story.

Clearview says that it is changing the way police investigations operate by providing a searchable database of billions of faces. The company has been used by law enforcement agencies all over the world to track down potential criminals. However, it has now been hit by legal complaints in five countries for violating citizen privacy. 

In January, the Hamburg data protection authority ordered Clearview to delete the code that identified Marx’s face, saying that the technology violated European data protection rules. His campaign has been at the forefront of an international push by privacy activists, condemning the company and calling for more stringent controls on facial recognition tech. We spoke to Marx about his odyssey to get his face back. 

Clearview AI did not respond to a request for comment for this story.

This conversation has been edited for length and clarity

For anyone who missed the Clearview AI story, could you sum it up for us?

On most search engines, you can upload a photo and a search engine would show you similar photos. But Clearview AI is different because it lets you search for specific faces. If I took a photo of you and uploaded it to Clearview, it would look for the same face on the internet. So Clearview AI has trawled the internet, looked for photos, identified all faces in those photos, and built a huge database. 

How did your quest to get your face back from Clearview start?

The whole trip started in January 2020, when I read a New York Times article about Clearview AI. They’d crawled more than three billion images on the internet, so I had reason to believe that photos of me might be among them. There are a few images of my face around, so I just asked them if they had any.

Were you conscious about maintaining privacy online at that time?

I care about my privacy online, so I usually don’t upload pictures of my face on the internet. I don’t use Facebook. But I did appear on the internet a few times, because I participated in student projects.

You’re part of a campaign called “Reclaim your Face.” What does that mean?

At the moment we don’t really own our faces. There are already lots of biometric experiments out there that use, say, CCTV to process our faces without our consent. We need to do something if we want to claim our faces back, because at the moment, companies just could use our face to identify us. 

What do Clearview AI say they’re doing?

They say they’re just looking for public images on the internet. But the Clearview AI search engine is a bigger risk to everyone’s privacy. They make it impossible to remain anonymous in the offline world.

Why is remaining anonymous in the offline world important?

I think it should be important to everyone. Under surveillance, we change our behavior. If I want to attend a protest, but I know it’s easy to be identified, I might decide not to go, even if it was completely legal to do so. Likewise, I might not want to go to the psychologist’s office, if I knew that I was being identified wherever I went. 

Is there any way to disguise yourself from the algorithm?

That doesn't work. The algorithms are just too good at identifying faces now. I would have to change my nose, ears, and eyes to trick the algorithms. 

What happened after you sent the request to Clearview?

I didn’t expect them to respond. But after a month, they told me they’d found pictures of my face twice on the internet. I was surprised. I didn't know those images even existed. It was scary to be part of this database because anyone could use Clearview AI to identify me, just based on my photo. 

Did the technology work perfectly?

Actually, no. Clearview later sent photos of eight people from different parts of the world. It’s a good example that those algorithms fail from time to time, and it's dangerous to blindly believe their results. 

What happened at the end of the process?

It took more than 12 months. Eventually, the Hamburg data protection authority ordered Clearview to delete the biometric mathematical hash value that describes my face.

What's your message to someone who doesn't care that their face is part of the system? 

Maybe it's not a danger to them, but it could be a danger for their friends, family, to minorities. This tech may not be that dangerous when democracy is perfectly functioning. But times change, countries change, and this technology in the hands of a dictator is very dangerous. 

The post The hacker who spent a year reclaiming his face from Clearview AI appeared first on Coda Story.

The study, published in April by the Institute for Strategic Dialogue, a UK-based think tank researching extremism, analyzed Amazon’s book recommendation algorithms for literature about white nationalist and far-right content, coronavirus denialism, and conspiracies about QAnon and fraud in the 2020 U.S. presidential election. 

The analysis found that the platform’s recommendation algorithms, which suggest titles and authors to people browsing through books on the platform — while relatively harmless for most users — can create a disturbing pipeline into extremist literature, directing customers toward conspiracy theories and white nationalist literature.

The study found Amazon’s algorithmic recommendations “could serve as a gateway into a broader universe of conspiracy theories and misinformation, or to increasingly radical far-right and white nationalist content."

Amazon’s recommendation algorithm suggests books to users by telling them what other customers who clicked on the book bought and viewed. This algorithm, according to the ISD, creates a feedback loop. People who click on a book about a conspiracy are fed recommendations for the same and other conspiracies, including books about QAnon, debunked claims about vaccines and fraud in the U.S. 2020 presidential election.

A descent into this algorithmic dystopia could take you to the book page for The Hammer is the Key to the Coup: How Obama, Brennan, Clapper, and the CIA spied on President Trump, General Flynn ... and everyone else, written by the proponents of a debunked election fraud conspiracy about the 2020 U.S. presidential election. The page suggests links to books about the Illuminati, the Rothschilds, the “Scamdemic,” Pizzagate, and aliens.

“I was shocked by the cornucopia of weirdness in the recommendations,” Elise Thomas, the author of the report and an ISD analyst, told me. She was especially alarmed to see books on the platform by the founder of the Order of Nine Angles (O9A), a UK-based Nazi satanic group that has been tied to a number of terror offences, prompting calls to outlaw it as a terrorist organization. 

“These beliefs are very extreme, and I was genuinely quite taken aback to see them just sitting on Amazon,” she said. “As a result of doing this research, I learned about other 09A texts I didn’t previously know about because of Amazon’s recommendations. I think that’s an example of the potential harm here.”

Other examples directing readers to extremist literature abound. Click on the book Whiteness: The Original Sin, and recommendations include titles about European ethnostates, “white identity politics,” and New World Order conspiracies.

Amazon recently removed the 1978 white supremacist novel The Turner Diaries, but a search for the book will now direct people to The Anarchist’s Cookbook, a bomb-making guide published in 1971 by William Powell that has been linked to acts of violence including the Columbine shooting and the Oklahoma City bombing.

An Amazon spokesperson said the company takes “concerns from the Institute for Strategic Dialogue seriously and are committed to providing a positive experience for our customers. Similar to other stores that sell books, we provide our customers with access to a variety of viewpoints and our shopping and discovery tools are not designed to generate results oriented to a specific point of view."

While any conversation about banning books is complex, Thomas says an overhaul of Amazon’s algorithm for extremist books could offer one solution. This would involve turning off the automatic recommendation of books promoting conspiracy theories, disinformation, or extremist views. 

“The books are still on the platform, you can find them if you search for them, but if you search for one and go to that page it’s not going to recommend you 20 others,” she said. 

The ISD report marks the most recent investigation into Amazon’s search algorithms. Previous analyses have also examined the role played by the digital giant's search algorithms in leading customers into conspiracies and misinformation. A January 2021 study by researchers at the University of Washington, for example, revealed that nearly 10.5% of searches involving the term “vaccine” promote books containing anti-vaccine conspiracies and health misinformation.

The post How Amazon’s algorithms push people towards extremist content appeared first on Coda Story.

Last week, the Utah Attorney General’s office suspended a $20.7 million contract with Banjo — a technology firm using government surveillance data to develop crime detection software — following revelations of the founder’s past membership of a white supremacist group.

Damien Patton, who serves as CEO of the SoftBank-backed company, was reportedly an active member of the Ku Klux Klan as a teenager, and participated in a 1990 drive-by shooting of a synagogue in suburban Nashville, according to the tech blog OneZero.

In a statement, a spokesperson for Utah Attorney General Sean Reyes said the office would be moving forward an already planned third-party audit of the software to “address issues like data privacy and possible bias.” Reyes recommended that other state agencies do the same.

“The Utah Attorney General’s office is shocked and dismayed at reports that Banjo’s founder had any affiliation with any hate group or groups in his youth,” said the statement. “Neither the AG nor anyone in the AG’s office were aware of these affiliations or actions. They are indefensible.”

According to documents obtained by Coda Story under Utah’s Government Records Access and Management Act, Banjo’s contract with the state gave the company live access to an unprecedented number of government data streams, including 911 calls, traffic and CCTV cameras, and location data for state vehicles.

Banjo’s real-time access to this vast amount of information employs artificial intelligence to alert first responders in dozens of agencies across Utah to crimes and other public safety threats as they happen. Before the suspension of the contract, the company’s technology was already in use by all of the state’s 29 counties, at least 23 cities and even campus police at the University of Utah, Motherboard reported last month.

The Utah Department of Public Safety said in a statement that it is also conducting a review of Banjo’s technology, though it and other agencies could not unilaterally suspend their agreements with the company. 

“We have not suspended the current contract because the contract is with the Department of Administrative Services Division of Purchasing. We are discussing and reviewing the matter with them,” a spokesperson wrote.

Banjo announced last Wednesday that it had suspended all of the company’s data collection operations in Utah.

The revelations about Patton’s past ties to the white supremacist movement have prompted new calls for transparency in how artificial intelligence is used by law enforcement. Several prominent Utah lawmakers called for a public audit of Banjo’s contract long before this week’s news. They have now renewed their push for increased oversight of state data sharing with private companies.

“I’ve been one of the only people asking questions about this for years now, and nobody seemed to know what the answers are — for so long I thought I was crazy,” said Democratic Utah Representative Angela Romero, who sits on the state’s Law Enforcement and Criminal Justice Committee. “A lot of really powerful people seem to have a vested interest in keeping the details of this program a secret.” 

Until this week’s revelations, Patton’s colorful past was a selling point for the company in an industry noted for its culture of founder-worship. Known for his long beard and eccentric dress sense, Patton’s story starts as a homeless teenage runaway. His roundabout career also includes stints in the U.S. Navy, as a NASCAR pit mechanic and a crime scene investigator.

This tale has been told in magazines, newspapers and in pitch meetings and introductions at technology conferences for nearly a decade.

But there is also a darker side to it. OneZero, which reviewed thousands of pages of public court documents, wrote that Patton was also involved with white supremacist groups, and, at 17 years old, party to a synagogue shooting.

According to court records, Patton was driving the vehicle on the day a Klan leader shot out the windows of the West End Synagogue in Nashville, Tennessee. No one was injured in the incident. OneZero reported that two Klansmen were later convicted of crimes connected to the shooting and Patton pleaded guilty to acts of related juvenile delinquency. 

Patton also testified at the trial about his beliefs at the time. “We believe that the blacks and the Jews are taking over America, and it’s our job to take America back for the white race,” he said.

Artificial intelligence and bias

The details of Patton’s past highlight the importance of oversight in the writing of surveillance algorithms so as to ensure that they are as unbiased as possible, says Michael German, a fellow at the Brennan Center for Justice’s Liberty & National Security Program, in a telephone interview.

“When information like this is brought to light, it also brings reform,” he said. “Unfortunately, law enforcement agencies often hide how their systems work, and the only opportunity for reform is a good deal later, when we have the data to back it up.”

Ties to white nationalist ideology and alt-right groups have marred the reputations of several law enforcement technology firms recently. The founder of Clearview AI, the facial recognition startup used by more than 600 U.S. law enforcement agencies, including Immigration and Customs Enforcement, Customs and Border Protection, was recently found to have links to far-right figures, including Mike Cernovich, who spearheaded the “Pizzagate” conspiracy theory, and Andrew “weev” Auernheimer, webmaster of the neo-Nazi website “The Daily Stormer.” Several Clearview AI employees were also exposed for sharing extremist content and attending events with other white nationalists, including alt-right figurehead Richard Spencer.

Peter Thiel, who sits on the board of Facebook and runs the data mining giant Palantir, is also reported to be linked to several alt-right figures, including the former Breitbart writer Milo Yiannopoulos.

While these companies and individuals have written off criticism of their connections to far-right and white nationalist groups as partisan attacks, such exposures point to a need to examine biases in AI systems, says German.

“Any system made by human beings will mirror or even amplify the biases that already exist in both law enforcement and society,” he said.

The post Tech company’s ties to white supremacism trigger debate on surveillance algorithms appeared first on Coda Story.

The Senate hearing had been called to examine the reasons behind the recent resurgence of preventable diseases. Across the United States, American physicians are battling the worst measles outbreaks in decades. And globally, polio – which had almost been on the brink of extinction – is making a deadly comeback. 

Responding to mounting pressure in the wake of Lindenberger’s speech, Facebook pledged to stop recommending vaccine disinformation on its platforms and make it harder to find anti-vax content in searches. But eight months on, evidence gathered by Coda shows that anti-vaxxer movement is flocking to Facebook-owned Instagram, gaming the app’s algorithm to spread anti-vaccine disinformation.

Until the spring of this year, anti-vaxxers enjoyed near total freedom on Instagram. “They had a white flag where they could do anything they wanted,” said Anne-Julie Dionne, 22, a Quebec-based pro-vaccine campaigner who runs an account called @queenofvaccines.

In May, Instagram shut down some of the most popular anti-vax hashtags, including #vaccinescauseAIDS, #vaccineskill and #vaccinescauseautism. But anti-vaccine Instagram users have been getting around the controls by employing more than 40 cryptic hashtags such as #learntherisk and #justasking.

“It doesn’t sound as straightforward and can be misleading for families who are looking up that information,” said Alice, 29, a California pediatrician who runs the pro-vaccine account @vaccinesinvogue and wished to remain anonymous for fear of online abuse.

Tactics like spelling vaccines with a cedilla (vaççines) or using a bracket (va((ines) to try to avoid detection by Instagram also proliferate on the platform. “The same amount of misinformation is floating around, if not more,” Alice said. “It kind of forms this almost exclusive group of people who are using the same hashtag to distribute their information.”

Another popular tactic has been simply to take over hashtags that were initially used by other Instagrammers campaigning for abortion rights and sexual consent, such as pro-choice hashtags #righttochoose, #mybodymychoice and #bodilyautonomy. Many of these hashtags are now saturated with anti-vaccine propaganda. 

A consent hashtag, #idonotconsent, has been particularly heavily co-opted — around three quarters of last month’s posts had an anti-vaccine agenda.

Holly, a Canadian anti-vaxxer in New York with the handle @novaccinesnoworries, simply latched on to hashtags used by vaccine advocacy groups. “What accounts like mine have done is actually started using the pro-vax hashtags. I use #vaxwithme and #provax just to try and get the information out there, since our hashtags are being so heavily censored,” the 24-year-old told Coda on condition of anonymity. 

Anti-vax users have increasingly turned to Instagram as they felt the effects of Facebook’s effort to deprioritize anti-vaccine content and stop recommending it to its users. “I find it a lot harder to share information on Facebook. Posts on Facebook get taken down a lot faster and it seems to be more monitored,” said Holly. “But with Instagram, while it’s still censored, it’s not as heavily done. Posts don’t really get removed that easily – even if someone reports them it’s really rare.”

A scroll within any of the hashtags currently used by anti-vaxxers quickly brings up widely-debunked pseudoscience, such as the non-existent link between vaccines and autism, or the false claim that the flu shot causes fetal death. “Hundreds of babies were murdered in the name of vaccines,” falsely claims a particularly prevalent post from last month.

“Don’t tell me you [sic] vaccinated kids turned out “fine” if they have asthma, autism, epilepsy, depression, diabetes, cancer, food allergies, ADHD, anxiety, eczema,” another popular and much-reposted disinformation meme claimed on the #idonotconsent hashtag in November. 

Together, the hashtags contain more than 240,000 posts. Instagram would not comment on what benchmark was needed to shut down a particular anti-vax hashtag, citing that doing so would weaken their anti-misinformation defenses. 

Instagram clarified that anti-vaxxers themselves did not violate the company’s guidelines. “Anti-vaccine content isn’t against our policies and we won’t take action on a hashtag because it contains this type of content,” said Stephanie Otway, a Facebook company spokesperson in an email. “If a hashtag contains a certain amount of vaccine misinformation, we may restrict it or block it altogether.”

After Coda shared a list of hashtags with Instagram that contained significant volumes of anti-vax content, the #vaççineskill, #noflushot, #learntherisk, #researchdontregret and #stoppoisoningyourkids hashtags were taken down. Dozens of anti-vax hashtags remain, and more are created and followed every week.

For Instagram, “the challenge is, they’re trying to find a balance between bad information and freedom of expression,” said Renee di Resta, a disinformation expert and co-founder of advocacy group Vaccinate California. 

Another issue Instagram is dealing is what di Resta terms “the asymmetry of passion.” “There’s a real asymmetry of passion on who involves themselves in the issue. The overwhelming majority of people who vaccinate have a perfectly positive experience,” she said, explaining that this meant they were unlikely to begin campaigning about vaccines. “It’s the equivalent of saying, I’m going to tweet about how the earth is round.”

Pro-vaccine accounts like Dionne and Alice’s tend to have a much harder time gaining followers – Alice gets around this by using anti-vax hashtags to post her vaccine advocacy content. Her posts, distinctively colored “millennial pink” stand out amid the sea of anti-vaccine memes. “My goal is not to get lots of followers, my goal is just to put that information out there.” 

Otavio Freire, President and co-founder at U.S.-based cybersecurity company Safeguard Cyber,  said anti-vaccine content is a favored pressure point in disinformation campaigns run by foreign bots and trolls. “There are some topics that are perfect to create that divisiveness and anger. That’s what these memes are trying to do,” he said.

“The overwhelming majority of these accounts make Instagram behave in a certain way,” he added. “The algorithm’s being spoofed.”

Freire calls this practice as “memetic warfare,” a term used to describe modern information warfare waged through the confines of a meme. “It’s digestible, it’s easy to look at,” he said. “It’s great for amplification of disinformation, and lo and behold – what platform is great for images – Instagram.”

Another Instagram feature that proves useful to anti-vaxxers is the fact that the app does not allow users to post links except in their bios, making it easy to post unfounded information without citations and accountability.

 “What I like about Facebook is you can actually post articles and links,” said Dionne, the pro-vax campaigner. “Instagram keeps people from opening an article and reading it. I’ll post my links anyway,” she said, but added that it was a source of frustration – anti-vaxxers would frequently ask her for “proof” that vaccines were safe. 

In September, Facebook and Instagram introduced a further measure to limit disinformation and increase awareness around vaccines: when a user searches for vaccines, they are greeted with a pop-up asking if they would like to visit the Center for Disease Control or the World Health Organization instead. In a statement, WHO Director-General Dr Tedros Adhamnom Ghebreyesus welcomed the feature. “Major digital organizations have a responsibility to their users – to ensure that they can access facts about vaccines,” he said.

Those studying the day-to-day vaccine movement were less optimistic. “Now Instagram is picking the winners and losers in society — I don’t know if that’s a better freedom of speech approach to the actual problem,” said Freire, who believes the app’s algorithms allowing falsely-amplified content to reign supreme should be addressed as a priority.

Instead of visiting the WHO websites, users can opt to “see posts anyway,” upon which they’re greeted with a rash of the most popular anti-vaccine accounts. “It’s like ‘we can’t deal with the underlying symptom, so let’s try to mediate it by giving you a pop-up’ – it’s ultimately not attacking the cause but a symptom of disinformation.”

Anastasia Gviniashvili contributed research. 

Correction: In the original version of this article, Otavio Freire's name was misspelled, and his title was misidentified.

The post How anti-vaxxers get around Instagram’s new hashtag controls appeared first on Coda Story.