Why did progressives outperform the polls?

All eyes in New York politics were on the Siena College poll when it came out on Sept. 10, just days before the 2018 primary election. On the Democratic side, it told a story of a too-close-to-call attorney general race, a cushy lead for Lt. Gov. Kathy Hochul’s re-election, and a 41-point spread in the gubernatorial race in favor of incumbent Gov. Andrew Cuomo.

The results didn’t exactly match those findings. Letitia James won the attorney general nomination by a 19-point margin. The candidate who came in second, Zephyr Teachout, was in third in the Siena poll. Hochul only bested Jumaane Williams six points, while Cuomo beat opponent Cynthia Nixon by 30 points. While that might sound like the poll was at least right about the governor’s race, a 10-point difference is an unusually large swing for a poll taken so late in the race.

The discrepancy between the poll’s findings and the final results followed a pattern: candidates who lean further left than their opponents did better in the actual results than they had in the poll. So, is there a structural flaw in Siena’s model? And, given the dearth of other statewide polls, might every poll underestimate progressive candidates?

Election or no election, Siena conducts these surveys once a month “as a public service to keep people informed to show where New Yorkers stand.” It’s proven itself a well-respected polling organization, recently named one of the best in the country by political website FiveThirtyEight.com. Siena also managed to accurately call the final outcomes for two out of the three statewide races. (The results only differed on the AG race, putting James in second a point behind U.S. Rep. Sean Patrick Maloney.)

So what happened when it came down to the details? The large number of undecided voters and the tendency of those to break towards challengers may be part of the story. “The poll is not a predictor,” Steven Greenberg a spokesperson for the Siena College Polling Institute told City & State. “It’s a snapshot in time.”

Even so, the reputation of Siena and other polls like it – Quinnipiac University and Monmouth University to name a few – means constituents and candidates alike look to the polls leading up to crucial elections. And it’s not clear what changed in the six days between polling of the 509 “likely voters” and when the ballots were cast.

Perhaps it’s time to update the polling process to reflect modern communication technology. The way polling is conducted today is a bit antiquated. Live callers contact voters by phone to ask them a series of questions about who they’re voting for and how they think candidates stand with regard to things like corruption and policies. The responses are weighted using a point-based system that takes into account things like how many times the person has voted before and how zealous they are about one of the contenders.

The result is often skewed demographically or geographically. Siena said that its own breakdown was fairly close to the actual, but voter turnout wound up being a lot different than anyone could have predicted.

There were more than two times the number of voters on Sept. 13 than there were in 2014’s primary, which was much more than anyone, including pollsters, predicted. That increase was especially prominent downstate. Siena’s poll, for example, was 50 percent New York City residents, but Thursday’s Democratic electorate was 57 percent from the five boroughs. That made all of the difference in the lieutenant governor race. Hochul is an upstate New York resident while her opponent, Jumaane Williams, is a Brooklyn native, and New York City showed up for Williams a lot more than it did for the incumbent.

For the other two races, Greenberg said the difference between the polling results and Thursday’s election results was largely about the undecideds, who “clearly broke in the direction of James,” and probably did the same for Nixon. Historically, if a voter isn’t planning on supporting the incumbent six days before the election, they probably won’t on the day of, but that isn’t necessarily true of the challenger, according to Siena. So probably most of the 18 percent of people it polled who said “I don’t know” or “I don’t have an opinion,” probably went with Nixon.

Pollsters call roughly twice as many landlines as they call cell phones. “We have to move towards cell phones as more and more houses are getting rid of landlines,” said Greenberg. Even so, when the electorate is clearly being reshaped by the influx of tech-savvy younger voters – none of whom should be assumed to use landlines and many of whom don’t even talk on their phones, but use them for communicating only via text message and social media – pollsters should probably find new ways of reaching them.