Open data

New York City Department of Information Technology and Telecommunications’ open data portal provides public information for New Yorkers across a variety of topics and agencies, including 311 service requests and the types of trees in any neighborhood. As director of open data, Al Webber oversees the portal and coordinates information with other city agencies. He spoke with City & State about the evolution of open data and how it can be used by average New Yorkers.

RELATED: An interview with DoITT Commissioner Anne Roest

C&S: What do you do as director of open data for DoITT?

AW: What I do is work with all of the city’s agencies to make city government as transparent as possible. I like to look at it as really a portrait of the city – a portrait of the city in numbers and in information. So we have roughly 1,600 data sets on the (open data) portal, about 80 agencies that are contributing a lot of that data. Really what open data is, is telling people what’s going on in terms of operations in city agencies. It’s a great tool to allow users to sort of make informed decisions and to just understand what’s going on in the city that they’re living in, working in or playing in.

C&S: Is there a coordination between different city agencies to kind of oversee open data?

AW: Absolutely. I think that’s one of the main roles that I have. One of our main goals is to make agencies completely understand what they need to do when it comes to open data. Whether that means publishing data sets, creating a data dictionary, geocoding the address information that you can find. What we have to do is make sure that they know what their responsibilities are and we also provide suggestions.

C&S: Could you talk a little bit about the origin of open data? How was this project conceived?

AW: I guess it started six or seven years ago. It was part of a large transparency effort. When open data launched, the site was actually called the NYC Data Mine at that point in time. It was an effort to release city data sets and it was done in conjunction with the big apps competition. We released about 170 data sets, and we also had an application development competition where we invited developers to take the data that we released from the open data portal and create applications were going to be useful to New Yorkers. It was twofold. It was releasing data and then I guess also just to make it sort of noteworthy, that competition that allows people to see the data and make the data a whole lot more user friendly with the applications.

C&S: The open data site recently released a report on its progress. What was in the most recent report?

AW: What we’re supposed to do every year is make an update to the annual compliance plan. So the earlier versions of the report was, it didn’t include narratives, so it was strictly a schedule of data sets that all city agencies went and released between that point in time and 2018. But we also saw an opportunity to speak to New Yorkers about things that we’ve done over the last years. So we tried to engage users a whole lot more. The things that they’ve always asked of us is to highlight the things that we have going on. So what we did this year was we talked about a lot of the users’ stories, use cases where people were using open data. We listed some of our core values, which is also in the report. It’s a really complete report with a lot of interesting stories that we’ve done over the past year.

C&S: What are your goals for open data?

AW: For a number of years – and this wasn’t done intentionally – data scientists, analysts, those are the primary users going to the open data portal. We want to do is make the open data site a place where anyone can go and have an understanding. It’s a really great resource and we don’t want to limit it only to the more technical folks. So open data for all is really what we’re working towards. You can see some of that in some of the changes that we’ve made in terms of, you know, redoing our website earlier in the year. We really wanted to just make the site accessible for anyone, no matter what your level of data knowledge was.

C&S: When was open data launched?

AW: Open data as a program started in 2009, where we sort of had one site where you could find data sets from all city agencies. I will say that, even prior to the open data program online, there were other tools where you could see transparency efforts from the city, like the citywide performance report and a few agencies had their own sort of microsites when it came to open data. But 2009 was when the citywide initiative started; 2015 is when we started expanding on that, when we went towards open data for all.

C&S: How can average New Yorkers benefit from open data?

AW: I look at it as a real chance to make informed decisions. We have data that goes across so many different subject areas. We’ve released a lot of data sets. There are about 1,600 on the portal right now, but we’re always looking to make improvements to the site. People always want to know the things like how frequently we’re updating data or what sorts of data sets are available. Over this last year, one interesting thing we did is we’re now able to refresh data on the portal, sort of close to real time. During last year’s winter, we released data from the department of sanitation from PlowNYC. Every snowplow has a GPS tracker on it, and we’re picking up that sensor information and we’re releasing it as open data every 15 minutes. So both of these data sets were actually available before, but PlowNYC was only available during a snow event. So when it wasn’t snowing, you couldn’t see the PlowNYC information. But now, via open data, that data is always available. So whether it’s winter, spring, summer or fall you have access to that PlowNYC data. The same thing with the DOT traffic speed, the data was available but it wasn’t in the most user friendly format. So what we wanted to do was break it down to that columns and rows format so the average person could look at it and understand it. We released that maybe two weeks ago, and it’s updated every five minutes. So you could go to the portal and maybe look at the sensors that are all around the city and you can look at how fast cars are going. It can help you predict your commute.

Return to DoITT Agency Focus