Who’s homeless enough for housing? In San Francisco, an algorithm decides
Replacing human decision making with a computerized scoring system is hurting California’s most vulnerable residents
San Francisco’s Tenderloin district is chaotic. Sirens, music and loud conversations generate almost constant noise. In every direction, dozens of people are camped out, crouched outside tents or sleeping in the open. Planters filled with soil but no flowers line a sidewalk, nails sticking out to discourage sitting. A broken drinking fountain, installed to give people access to clean water during the pandemic, gushes into the street.
Twitter’s headquarters are a 15-minute walk away, the cloud-based software giant Salesforce is 20 minutes in the opposite direction. But right here — in the epicenter of a chronic housing crisis, recently exacerbated by high tech industry salaries — you would barely be able to guess that this city is home to some of the wealthiest and most powerful people in the world. According to the latest count in 2019, there are around 8,035 people experiencing homelessness in San Francisco, a per capita rate comparable to much larger cities like New York.
For the thousands experiencing homelessness in the Tenderloin, their chance of getting off the streets comes down to a single number, generated by an algorithm. It is meant to assess each person’s unique vulnerabilities and allocate assistance accordingly. But now, even the designers of such systems say that, far from solving the problem of homelessness in the United States, these algorithms are used by local governments to deny assistance to large numbers of people in need.
In 2013, Iain De Jong and his colleagues at OrgCode, a consulting firm specializing in issues relevant to the homeless, created the Vulnerability Index — Service Prioritization Decision Assistance Tool, a scoring algorithm to help address America’s homelessness crisis. They were so successful that versions were adopted by authorities in at least 40 states. Other local governments, like San Francisco, have followed suit and created their own similar tools. Eight years later, De Jong and OrgCode say that cities are misusing their system — and that this has to stop.
VI-SPDAT was meant to help local social service providers assess what type of housing assistance might best suit a homeless person’s needs. Instead, resource-strapped cities are relying solely on tools such as VI-SPDAT to make a binary choice: who gets housing and who doesn’t.
“One of the gross misunderstandings and misuses of the tool was making housing decisions based upon the outputs of it,” De Jong told me. “It was never designed to do that.” In December, OrgCode announced that it would begin phasing out VI-SPDAT and will no longer provide support for cities using the most common version of it.
But it might be too late. The tool was rolled out one year after the Department of Housing and Urban Development required all cities receiving federal funding for programs that house people who are homeless to adopt centralized assessment processes known as “coordinated entry systems,” built on tools such as VI-SPDAT, to allocate accommodation.
Today, VI-SPDAT offers a cautionary tale of how an algorithm meant to help people resolve a thorny societal dilemma replaced human decision making entirely, with devastating effects for its intended beneficiaries. Yet, it is not the classic story of viewing a complex human crisis through a reductive tech-bro lens. Instead, it is an example of the wilful misuse of a well-intentioned tool by city administers, who have turned to such systems to deflect attention from a persistent and multi-layered problem, rather than attempting to marshal the resources and political will to solve it.
VI-SPDAT grew out of a collaboration between OrgCode and Common Ground, a national organization working to house homeless people. The system generates a vulnerability score out of 17 based on a set of questions about mental health, physical health and risk factors for chronic homelessness, with the aim of aiding case managers to triage people to appropriate resources.
“It was intended to try and help frontline staff better understand, across multiple dimensions, what people’s vulnerabilities were, what their risks to housing stability were, so that you can work with the individual to guide a plan of support,” said De Jong, explaining that the score was not meant to be the ultimate criteria for deciding whether or not a person should receive housing assistance.
But that, unfortunately, is exactly what happened. City administrators simply ignored the fact that VI-SPDAT was intended to provide a starting point for offering assistance and facilitating further conversations between homeless individuals and case workers.
“People just skipped over that step,” he said. “We even heard things like, ‘Well, we just don’t have time,’ or, ‘It’s inconvenient,” or, ‘Following up with people to get that sort of information is hard work and the survey isn’t.’”
Prior to the widespread adoption of coordinated entry systems, the process of providing housing for homeless people was built on relationships between case workers and the individuals in question. Social workers were supposed to develop an understanding of their clients’ needs and assist them accordingly.
Coordinated entry replaced individualized case management with a standardized system that is — depending on where you stand or who you talk to — either objective and less prone to favoritism, or cold, rigid and brutally mechanical.
“The goal of coordinated entry is to promote equity, and to ensure everyone has the equal ability to access resources,” wrote Denny Machuca-Grebe, the public information officer for the Department of Homelessness and Supportive Housing (HSH) in San Francisco.
Service providers working with people experiencing homelessness say that deemphasizing human relationships has created an unbending and soulless system that actively impedes the provision of individualized help. Tools such as VI-SPDAT or San Francisco’s similar Primary Assessment algorithm, critics say, have become a way to quickly eliminate people from housing eligibility under the guise of fairness and efficiency.
“You can propose a solution to meet the scale of the problem. Or you can shrink the problem to meet the available solution,” said Joe Wilson, the executive director of Hospitality House, a community-based organization in the Tenderloin that provides services for people experiencing homelessness and runs a shelter. “Coordinated entry reduces the scale of the problem.”
The reality is there simply isn’t enough housing. As of February, 225 of the roughly 7,755 supportive housing units allocated for people experiencing homelessness in San Francisco were vacant and ready for occupancy, according to reporting by the San Francisco Public Press. While over 10,800 people lived in permanent supportive housing as of June of last year, 5,180 people are sleeping on the streets and another 2,855 are living in cars, on couches or in temporary shelters, according to the latest count in 2019.
Wilson, who was homeless in San Francisco himself in the early 1980s, has been a vocal critic of coordinated entry since its launch. “It’s designed not to help people get in, but to keep them out,” he said.
“I think we went from one extreme to the other,” De Jong said. “We went from a system of care that had really come down to luck, self-advocacy or first come, first served, in terms of how people got housing, to a very hands-off numeric-based approach that was very dehumanizing, in which people were not, in my opinion, always seen as people with potential, strengths and resiliency. They were resigned to a number on a waiting list. And that was just overwhelmingly disheartening.”
Across the park from San Francisco City Hall, a few blocks south of the Tenderloin, rows of tents are hidden behind a concrete barricade. It’s one of half a dozen city-sanctioned areas with 24/7 security and access to food, water and sanitation. These “safe sleep” sites are the city’s latest experiment in controlling sprawling sidewalk encampments and a strategy to limit the spread of Covid-19 among the homeless population. But, for many, permanent housing remains an unattainable dream.
Primary Assessment, the city’s scoring tool, was created by the local government with community input, but it functions in a similar way to VI-SPDAT. Its questions are phrased and weighted differently, but the fundamental principle of an algorithm assessing vulnerability is exactly the same.
If a person’s score meets or exceeds a certain threshold, they enter the queue to be allocated a housing placement. If their score is too low, they are put into what is known as “problem-solving status,” to be matched with programs that provide assistance other than permanent housing.
The questions asked of applicants are deeply personal: information on drug or alcohol use, mental health issues, developmental disorders and experiences with sexual assault or domestic abuse. Single adults and people aged 18 to 24 are asked if they have visited detox centers or called into suicide hotlines, or if they have traded sex for a place to sleep.
“You’re not my therapist. You don’t need to know how many times I’ve been sexually assaulted in my life,” said Roxie, who had been homeless for most of the past four years, before they were finally housed in a subsidized unit. “I hate the fact that the coordinated entry system is based on how traumatizing your life has been, essentially.”
Roxie is based in San Francisco, but has traveled the country from Nashville to St. Louis. They have been through the Primary Assessment process on four occasions, and each time, it reopened old wounds.
“The Department is transparent and respectful about the reality of trauma for the people we serve, and we strive to minimize such impacts through training and standardization across the Homelessness Response System,” Deborah Bouck, communications lead at HSH, wrote in response to questions for this article.
Applicants are advised that they don’t need to go into detail during the assessment, but not answering a question can adversely affect their score, which leads to people like Roxie feeling compelled to share far more than they are comfortable with.
Xander, whose mother was homeless and grandmother lived in supportive housing, has taken the assessment twice and experienced panic attacks both times.
“It really sucks trying to bring up these topics,” Xander said. “When I did my housing assessment, I was fresh out of an abusive situation with my family, so it was all raw. When I tried talking about it, I’m like, `Hey, can we skip this question?’”
Xander walked out without finishing their first assessment or getting a score and was forced to return to a bad situation with their family.
“Stepping back from the complex system and how it all works, fundamentally, that is a really intense thing to ask anyone to do, to offer up even a piece of their vulnerability to somebody,” said Kenn Sutto, who conducts Primary Assessments at the Homeless Youth Alliance, a grassroots harm reduction coalition which is contracted by the city to serve as an access point for young people experiencing homelessness. Because these access points are contracted by the city, they are required to use the Primary Assessment. People have told Sutto that the coordinated entry assessment feels like the “trauma Olympics.”
“If you need to know to be able to determine how much support someone is going to need in housing, that’s a legitimate case,” said Wilson, explaining that it is necessary to know if an individual has a disability that makes it difficult to use stairs or a medical condition that requires a private bathroom. But he believes that questions about substance use or sexual violence should not be relevant to determining whether someone qualifies for housing.
“If you’re asking it to determine the severity of one’s homelessness, why do you need to know that? Everybody that comes needs a roof over their head,” he said.
For De Jong, the designer of VI-SPDAT, these intrusive questions conflate two very different concepts — eligibility, which determines the specific housing program from which they will gain assistance, such as veteran or HIV-positive housing, and “depth of need,” which simply refers to a person’s unique risks and vulnerabilities.
“That’s where I’ll say the failure of implementation of VI-SPDAT to its true intention is remarkably pronounced,” he said.
San Francisco’s Department of Homelessness and Supportive Housing doesn’t share this view. “The assessment is used to prioritize those most vulnerable,” said Bouck at HSH. But in practice, many people don’t score high enough to be considered for housing assistance.
Roxie got lucky and recently moved into an apartment in San Francisco’s Mission district, but the trauma of the repeated assessments remains with them.
“It’s such bullshit that we have to be willing to expose ourselves and be that vulnerable in order to get housing. It’s exploitative,” they said.
If you believe the strategic five-year plan, issued by San Francisco’s Department of Homelessness and Supportive Housing in 2017, coordinated entry is supposed to “make homelessness a rare, brief, and one-time event.”
On the other hand, you could listen to Jamale, whose experiences suggest an uncomfortable truth: coordinated entry systems will never fix homelessness. Instead, they will keep kicking people to the back of the queue, until the system deems them homeless enough for housing assistance.
Jamele, who asked that his name be changed for reasons of privacy, has taken the assessment twice. Both times he didn’t make the cut for housing priority status.
“I felt like being honest and trustworthy gets me nowhere in that system, because I didn’t qualify,” he said. “And I opened myself up to every little incident that I’ve had, just to be told. ‘Nope, not now.’ It really drains you. It makes you wonder, ‘Why am I still here, or what’s my purpose in society, if I can’t access help?’”
Jamale, who is originally from Los Angeles, moved north about 15 years ago. A quiet, kind man in his early 30s, he enjoys building model rockets. In college, his favorite course was on legal terminology. “I wanted to be an attorney that could help people get out and stay out of jail,” he said.
But Jamale didn’t know anyone in San Francisco when he first arrived, and he couldn’t afford rent. Without any idea where to turn for help, he ended up on the streets, where he has stayed for most of the past 10 years.
By focusing on a specific definition of vulnerability, San Francisco is inadvertently excluding people like Jamale from housing services. With a roof over his head and support to process the trauma caused by years of homelessness, he could be one of the success stories that many service providers hope to find. Ironically, that could be part of the reason why he hasn’t been allocated housing yet.
Like the vast majority of people who go through the coordinated entry system in San Francisco, Jamale wasn’t determined to be vulnerable enough for permanent supportive housing. According to data from HSH, only 17% of the 18,327 assessments conducted from January 2019 through May 2021 resulted in people moving into a new home.
Of the 11,979 single adults who were assessed between July 2018 and June 2021, a staggering 69% did not meet the threshold for housing priority status, according to HSH data obtained by Coda Story via a public records request. The numbers are better for other demographics — 88% of families and 58% of youth applicants receive housing referrals.
“We’re basically saying, ‘Wait until you’re sick enough, until you’ve been impacted to a point that’s very detrimental to a person who is experiencing homelessness, or until that point when this algorithm is going to score you very high,’” said Laura Valdez, executive director of Dolores Street Community Services, a nonprofit which will become a coordinated entry access point for adults later this year. Currently, there are only two such offices in the whole city. Valdez hopes that operating a third will give her team greater insight into how the system works.
It is hard to say why Jamale has not yet been housed. Coordinated entry’s scoring system is deliberately opaque. The people conducting the assessments are not even told how the questions are weighted. According to Jeff Kositsky, who served as director of HSH from 2016 to March 2020 and led the department throughout the implementation of coordinated entry, this is to avoid service providers coaching their clients.
“We want to make sure we’re getting an accurate picture, and we don’t want case managers to game the system to get their people to the top of the list. It’s a best practice,” Kositsky told the local online newsroom The Frisc in 2017.
“Any large, complex system that renders critical social services is subject to potential manipulation. While HSH understands there is a possibility for coaching, we also are aware such behavior occurs because staff care about those they serve and want to assist community members in getting housed,” said Bouck, communications lead at HSH, in a statement to Coda Story.
Still, people have guesses as to how it works. Joe Wilson believes that “chronicity of homelessness” — how long an individual has been unhoused — makes up a significant part of the score. He’s right, according to a breakdown of how questions are weighted reviewed by Coda Story, which has not been made public before and was obtained via a public records request by the Coalition on Homelessness. People who have been homeless for more than 15 years get 15 points added to their score. Someone who needs help carrying out daily activities or maintaining housing receives nine points. Experiencing sexual or physical violence in their current living situation can count for 12 points, but only applies if the person is 24 years old or younger.
Sometimes, things don’t add up. One of Wilson’s clients at Hospitality House is an 87-year-old woman, who has been homeless for 40 years. After going through the coordinated entry process, she didn’t qualify for a housing referral.
For many, the algorithm’s word is final. Case managers and the people conducting assessments can’t change a score. Individuals placed in problem-solving status can challenge the decision via a clinical case review, but that route is time consuming and requires clients to hand over even more personal information. Many don’t pursue it and instead opt to take the assessment again in six months, remaining homeless in the meantime. Others, like Jamale, just walk away for good. He doesn’t plan on trying a third time.
Without permanent housing, it can be harder to stay engaged with social services like child care, mental health and addiction treatment. Megan Geary is the program director at Central City Access Point, which is contracted by the city to conduct coordinated entry assessments for families. She told me that a significant number of people turn away from other avenues of help after being told that they do not qualify for housing. “We see people cycling through the system much longer. It just feels counterintuitive.”
For Jamale, it feels like the system is telling him that things need to get much worse before he is deemed worthy of a home.
“Why do I need to be in a hospital bed fighting for my life in order to get housed? If that’s the case, what do I have to do? Walking in off the street and asking for help, it seems like a dead road,” he said.
Joe Wilson has been working in homeless services for the better part of the 20 years, starting while he was sleeping at the Hospitality House shelter. He’s furious that human judgement has been replaced by an algorithm.
“To cede that kind of decision-making authority to a computer, is that what we want to do in our field?” he asked. “We can’t do any better than that? That’s not what I came here to do. I came here to bring me to this mix.”
Because of the rigidity of the coordinated entry system, service providers now can’t get their clients into their own housing programs, even when they can clearly see that doing so is the right course of action. Mary Kate Bacalao, the policy director at Compass Family Services, said her team worked with a client who needed a place to live after giving birth to a baby with special needs. Compass runs a housing program for pregnant women and new mothers, but because the scoring system put the woman in problem-solving status, they couldn’t place her. Social workers ultimately lost track of her.
HSH disagrees with criticism of the assessment. “Human decision-making is still very much a part of the process. The pandemic has shown that coordinated entry still very much incorporates human decision-making into the model,” wrote public information officer Denny Machuca-Grebe in response to questions for this story.
He went on to say that the coordinated entry system “strives to center client choice.” To a degree, it does. Families who have made it through the process and receive a temporary rapid rehousing rental subsidy can request a housing case review when a spot in permanent supportive housing becomes available. People who qualify for permanent supportive housing can also be offered up to three units, in the hope of finding one that matches their needs. But many people don’t get that far.
Service providers highlight that not all types of housing intervention will work for everyone. For example, a rapid rehousing subsidy may not be a good fit for people who need more support.
“The housing resolutions at the end of the assessment are a very cookie cutter approach,” said Geary at Central City Access Point. “We can’t really pivot to triaging them to a housing intervention that would maybe better meet those needs and be more helpful to them to be able to achieve longer-term stability. We kind of just have to refer them to whatever is available.”
The first time Roxie went through coordinated entry several years ago, they were placed in a single-room occupancy hotel (SRO) in the Mission district. It was right after they were sexually assaulted.
“It was one of the worst places that you could put somebody that had just recently been assaulted. It was like a crack den SRO,” they said.
“I ended up leaving after two weeks. I just said, fuck it, I’m done.”
Meanwhile, people like Kenn Sutto are doing their best to operate within the system. When he conducts a primary assessment, he tries to make people feel as comfortable as possible. Even though he is supposed to ask the questions exactly as written, he is allowed to clarify and steers clear of terms like “substance abuse,” in favor of less stigmatizing phrasing.
“I’m not here to be a person that’s reading off a computer screen,” he said. “My job is to be there for people experiencing homelessness.”
For Wilson, handing a decision as monumental as whether someone gets housing over to an algorithm just isn’t a sustainable or moral option.
“There were people in my life, particularly when I became homeless, who refused to turn away from me,” he said. “I refuse to do less than that. No computer is going to help me decide the worth of another human being, and who gets what, when and how much.”
The story you just read is a small piece of a complex and an ever-changing storyline that Coda covers relentlessly and with singular focus. But we can’t do it without your help. Show your support for journalism that stays on the story by becoming a member today. Coda Story is a 501(c)3 U.S. non-profit. Your contribution to Coda Story is tax deductible.