Preview: Three comparisons to watch in New York state's test scores

With this year’s state test scores expected to be released Wednesday, all sorts of comparisons are sure to follow.

How do this year’s results stack up against last year’s? How about New York City’s scores versus the state’s? And what about different groups of students: Did the vast racial achievement gap narrow at all?

Here’s a guide to the most important comparisons, along with some caveats about placing certain numbers side by side.

2014 vs. 2015: A more stable measurement

Until recently, comparing New York state test results over time offered little insight into what students were learning.

For most of New York’s nine-year history during which all students in grades three through eight took state tests, year-over-year comparisons were erratic. From 2006 to 2009, rising test scores were later discredited. The state then changed what it took to pass the test, sending scores tumbling in 2010 and 2011. Scores recovered slightly in 2012, but the state introduced new, Common Core-aligned assessments in 2013. That led to another precipitous drop in scores.

This year, for the second time, city students took tests that looked similar (and were scored similarly) to the tests they took last year. That relative stability means year-over-year comparisons will offer a more reasonable glimpse at progress.

Jennifer Jennings, a professor of sociology at New York University whose analysis of test scores in 2010 revealed evidence of test-score inflation, said increases in proficiency rates tend to follow the introduction of any new set of tests. That improvement can be attributed to a host of factors, she said, including greater familiarity with the test’s content.

Still, a third year of data under the same testing regime offers “at least some sense of trajectory and some progress over time,” Jennings said.

State vs. city: A more suspect measurement

When the city’s pass rates are stood next to the state’s, the state always comes out on top.

That’s no surprise, since the city has more typically lower-scoring poor and non-white students than the state as a whole. But one measure of the city’s academic progress is whether the city can narrow that performance gap with the state.

This year, it could be much harder to gauge that gap because of an unusual testing scenario: thousands of students across the state opted out of this year’s tests — by some estimates, as many as one-fifth of all test takers.

“Opt out,” said Robert Lowry, deputy director of the New York State Council of School Superintendents, “makes everything murky.”

The state has yet to say how many students refused to take this year’s math and English exams, but a pro-opt out advocacy group put the number at more than 200,000 students out of about 1.1 million eligible test-takers. In New York City, the number is far smaller, with advocates estimating that less than 1 percent of test-takers boycotted.

Officials said Monday that opt-out figures will be released at the same time as the scores. Until then, it’s impossible to know exactly how all the opt-outers will skew the statewide pass rates. Still, advocacy groups and news reports have suggested that many of the boycotters hailed from typically high-scoring, wealthier districts. If that turns out to be true, the state’s overall pass rates could wind up lower than expected, bringing the city’s rates even closer to state averages.

“It’s hard to know who the opt-out students were without additional data,” said New York University researcher Sean Corcoran. “But presuming they’re higher-achieving students than in New York City, leaving those kids out of the state calculation will tend to look good for the city.”

Last year, the gap between city and state proficiency rates in English was just three points, its smallest margin since 2006. Unfortunately for the city, experts may find city-state comparisons suspect this year.

“When this year’s data from the state is so marked by incomplete information,” said Teachers College education professor Aaron Pallas, any city-state comparison “is going to be hard to make.”

Student group vs. student group: A persistent problem

There’s also huge variation among the city’s different student groups.

For instance, about 56 percent of white students passed last year’s English tests, compared to less than 19 percent of black students. In general, white and Asian students continued to fare far better on the tests than their black and Hispanic peers — a longstanding pattern in New York and across the country.

In fact, black and Hispanic students saw their scores rise last year. But Asian and white students made even larger strides, causing the gaps between those groups to widen.

“All groups making progress is certainly something to cheer about,” said Pallas, the Teachers College professor. Still, non-white students would have to make “accelerated gains” to close the gap, he added.

Parallel progress among student groups is unlikely to satisfy critics who want the city to more aggressively address racial disparities. In its annual pre-score release analysis, the city teachers union predicted that critics will seize on any growing gaps. In its own memo, StudentsFirstNY, a pro-charter school group that has been one of the de Blasio administration’s loudest critics, promised to do just that.

This story originally ran on Chalkbeat.org.