Vaccine website frustrations point to larger government tech problems

New Yorkers have waited almost 10 months for a COVID-19 vaccine. Now that it’s finally here, all that stands between them and what will hopefully be the pandemic’s antidote is a maddening, sometimes hours-long online registration process.

As eligibility for the COVID-19 vaccine expanded on Monday to phase 1B – a cohort of more than four million people statewide, including some essential workers and people ages 75 and over – a number of issues quickly surfaced with the systems used to find info about where to get vaccinated and sign up for appointments. On Tuesday, Gov. Andrew Cuomo announced that eligibility would be expanded to people 65 years old and up and people who are immunocompromised. However, he said it was unclear which categories of immunocompromised people were covered by the Centers for Disease Control guidance. 

Miriam Grossman, a Connecticut resident who has been trying to sign up her father – who is over 75 years old and lives in Queens – for a vaccine appointment, estimated that she spent about four hours on Sunday and Monday trying to secure a slot. The Health + Hospitals vaccine scheduler would show her no available appointments and then direct her to find COVID-19 testing availability, which she wasn’t looking for. After filling out the many fields of questions that the city’s vaccine hub asked her, Grossman finished the long process only to find no appointments available. The city also has a vaccine finder to connect users to providers administering the vaccine, but Grossman encountered problems here too. A number she tried for a provider listed on the city’s website told her to schedule an appointment online, but the provider’s website had no online scheduling function. (On Tuesday, the New York City vaccination sign-up websites allowed New Yorkers 65 and older to sign up, along with various essential workers, but not immunocompromised people under 65.) 

“I thought, maybe I’m doing something wrong,” Grossman recalled. “But then when I saw on Twitter that other people were noting the same thing, I felt kind of vindicated. It’s not just me.” Many of the troubles New Yorkers reported with signing up for vaccines this week can be found in replies to a Sunday tweet from New York City Comptroller Scott Stringer, calling attention to what he said were “twin failures of outreach and technology” on the part of the city. “This is an impossible tech bureaucracy that is going to hurt seniors, hurt people without internet access or people who are just not digitally literate, and people who have language barriers,” Stringer told City & State on Monday. 

Many of the technical issues New Yorkers had signing up for vaccines have to do with design elements of the websites and a lack of adequate planning to make the experience user-friendly, technologists said. More broadly, issues with online sign-ups for vaccine appointments reflect a greater problem in New York: A failure to tap the vast technology resources at the city and state’s disposal to offer accessible essential services to New Yorkers. “It’s a lack of attention to the fact that digital technology is a critical tool in how government services are delivered in the 21st Century,” said Noel Hidalgo, co-founder of the technology nonprofit BetaNYC. That’s especially true during a pandemic, he said.

“Like our entire public health apparatus, our vaccine roll out is a massive undertaking requiring the resources of multiple agencies working together,” New York City Department of Health spokesperson Patrick Gallahue wrote in an email on Tuesday. Representatives for Cuomo did not respond before publication to a request for comment about problems reported with the state’s website. 

Despite New York City Mayor Bill de Blasio’s administration being the one to create new technology roles, including the city’s chief technology officer and New York City Cyber Command, Hidalgo said the administration has come up short in coordinating to create innovative tools that make the process of signing up for vaccines, for example, as easy as possible. “It’s missing that additive component where there’s very clear leadership and coordination on how technology, data and design can be used to address the real, present needs during the pandemic,” he said.

New York City isn’t alone in this though. The state has had its own high-profile tech issues during the pandemic, including the state Department of Labor’s unemployment insurance application – an outdated and bug-filled system that made the process of getting unemployment checks unnecessarily difficult for the more than one million suddenly jobless New Yorkers this spring. The state’s vaccination website appeared to fail some users on Monday too, with the site reportedly timing out for some. New York City and New York state have now both launched phone lines where eligible people can schedule an appointment to get the vaccine. It’s an important alternative to online scheduling, but callers may experience wait times there too. 

New York’s vaccine rollout has been the subject of non-technology related criticism too. Before eligibility was opened up to include non-health care essential workers and older New Yorkers, vaccine doses were being thrown away because government couldn’t connect eligible New Yorkers to providers. And leading up to this week’s expansion to phase 1B, there was still confusion about which essential workers would be eligible, as many types of occupations – grocery store clerks as well as firefighters – have been classified as “essential” throughout the pandemic.

Tracey, a social worker in Brooklyn who asked to be identified by first name only to preserve her privacy with clients, said that she has received conflicting messages about whether she’s eligible as someone working remotely who would hopefully be able to eventually transition back to in-person work after being vaccinated. The state and city websites haven’t provided much help in clarifying that for her. Instead, she’s resorted to seeking information on a Facebook page for New York City social workers. When the state’s unemployment insurance system failed to work this spring and summer, unemployed New Yorkers took to a Facebook group with over 47,000 members to seek answers and commiserate too.

A well-functioning government website or application needs to not only provide users with all the relevant information and functions, but be easy to use and intuitive. That means that seniors can use it without needing help to upload a photo of their insurance card or knowing to clear their browser’s cookies to circumvent error messages. The tools need to be as accessible to non-native English speakers as they are to English speakers. A website needs to be as easy to use on a mobile phone as it is on a desktop computer, so that people with only mobile internet access can use it. Nearly 30% of New York City households don’t have a home broadband subscription, according to the city’s own figures. (Offline options also need to be available for the 18% of city residents who lack both home and mobile broadband.)

For New York City’s two main vaccine scheduling websites – the one run by the Department of Health and the other run by Health + Hospitals – user experience and design is a big shortcoming, critics said. The Health Department website asks many questions – Stringer reported as many as 51 questions and fields – before an appointment can be made. Not only does that take a long time, but one crash or interruption along the way can force users to start the long process from scratch.

Sign-up websites should be asking for a minimal amount of data to remove as many hurdles that could trip people up – like remembering their insurance ID number – said Emil Skandul, founder of Capitol Foundry, a digital innovation firm. Rather than asking users to answer dozens of questions, the site could ask for a few main data points – name, age, eligibility, health risks and contact info, for example. Insurance and other information could be collected later, in person. There is concern within the city, however, that this would lead to bottlenecks at the vaccination sites, when the goal is to get as many people in and out of the sites quickly.

Others questioned why frustrations and technical problems with the websites weren’t foreseen before people started signing up for appointments en masse this week. It’s not clear how much user testing was done of New York City’s vaccine websites, but when designing a well-functioning site, testing it out on every kind of user has to be a part of the planning. “Did they run usability tests? Did they run load tests on these websites?” Skandul asked. The design process should include testing the product with different user “personas” – people who only have access to a smartphone, Spanish speakers and seniors, as just a few examples – he said. 

In a press conference on Monday, de Blasio addressed complaints about the city’s website, saying that the city planned to continue to make improvements. “Folks who are going through a lot, we want them to have as easy an experience as possible making their appointments,” he said. “What we do find as we put any system into play (is) that we improve it as we go along based on customer feedback, so I'm certain that's going to happen here.”

The Health + Hospitals website, while more streamlined, also experienced some slowness on Monday because of a high volume of users. Hidalgo said that the website also uses a web design element called an “iframe” for its scheduling framework, which he said makes it difficult for a translator to work with the site. “If somebody is using a language translator, the content in the actual scheduling app doesn’t get translated,” he said. That means that the Health + Hospitals scheduler is currently only offered in English. 

“Our scheduling system was able to connect New Yorkers yesterday to an appointment,” Health + Hospitals spokesperson Christopher Miller wrote in an email on Tuesday. “In the coming days more appointments will become available – we urge those eligible to check back frequently. As vaccine supply increases more appointments will become readily available."

There are easier ways to sign New Yorkers up to get the vaccine, experts said. Some lawmakers have called for New York City to have a single sign-up system to avoid confusion over which of the current options to choose from. Skandul said that New York state should be working with a single site through which residents can pre-register for a vaccine. He cited New Jersey’s pre-registration system as a model to follow here. Users could register their basic information with the state and be notified by text or other means when they are eligible and make an appointment on a system that integrated the different scheduling mechanisms used for public hospitals, pharmacies and all other vaccine providers. Instead of New Yorkers having to seek out a vaccine appointment, the opportunity to schedule an appointment would come to them. 

While a low number of available vaccine appointments on these websites is a sign that vaccination is speeding up, some are nervous that technical issues could mean wasted vaccines. In addition to the problems she encountered with the government-run websites, Grossman had problems getting through to private providers that the city and state’s vaccine sites directed her to. “I think New York City and New York state need to be not only on top of their own sites and web scheduling, but the private places that they’ve entrusted the vaccine to,” Grossman wrote in a message later on Monday. “If New York City says a clinic is a vaccine site but I can’t get through to make an appointment, where is the vaccine going, knowing (the) vaccine expires when not used in time?” By Monday night, Grossman had finally secured an appointment for her father at a Health + Hospitals site, though not in Queens, but Manhattan. 

While experts said that designing websites that could integrate all of these elements is no easy task, it’s work that could have been done long in advance of a vaccine actually arriving. “You’ve had nine months to prepare for this particular moment,” Hidalgo said. “It is a massive shortcoming that was preventable.”

It’s also not as if New York doesn’t have the brainpower to accomplish a system like that. Hidalgo cited several divisions of city agencies that already exist that could be a part of creating a more user-friendly and accessible system, including a civic service design studio within the Office for Economic Opportunity and user experience unit within the Department of Information Technology and Telecommunications. A spokesperson for the city did not comment on what agencies and resources were involved in designing its vaccine website or who led the effort, so it’s not clear if any of these resources were tapped. 

“This city needs government to succeed in this pandemic, because it is the only entity that is going to get us out of this pandemic,” Hidalgo said. “It’s frustrating as a technologist to see these things happen when you know there are good people inside of government who want to get us there.”

Correction: The vaccine finder website Grossman used is run by New York City and she did not have to upload photos of an insurance card.