The USDOT Joint Program Office Intelligent Transportation (ITS) Connected Vehicle (CV) Pilot Deployment Program has led the way in providing live CV data from its ongoing projects to the public. For the past eight months, the Wyoming CV Pilot project has automatically provided sanitized Basic Safety Messages (BSM) and Traveler Information Messages (TIM) to the public through the ITS Public Data Hub. The pilot team will continue to provide this information through the remainder of the project. Data files produced by Wyoming have already been downloaded more than 350,000 times.
The CV Pilot program pioneers the deployment of connected vehicle technologies in three U.S. locations: New York City, Tampa, FL and Wyoming. One of the key goals of the CV Pilot program is to produce and provide Open Data from the pilots to the public in a quick and useful manner for use by third party researchers and developers.
Wyoming is the first CV Pilot project site to complete their data pipeline and send continuous data deliveries to the ITS Public Data Hub. The ITS Public Data Hub (available at https://its.dot.gov/data) provides a single point of entry to discover publicly available USDOT ITS research data. By providing access to these data, the USDOT aims to enable third-party research into the effectiveness of emerging ITS technologies, preliminary development of third-party applications, and harmonization of data across similar collections.
Wyoming leveraged the USDOT ITS JPO’s Operational Data Environment (ODE) to quickly receive data from the field, validate the data, sanitize it of Personally Identifiable Information and deliver it to the ITS Public Data Hub. The Wyoming CV Pilot project produces many different types of data, however due to privacy and data protection concerns only sanitized BSMs and TIMs are made available to the public through the ITS Public Data Hub.
Once this data is delivered to the ITS Public Data Hub it’s made available to the public in two ways. The first way is through the ITS Public Data Hub Data Sandbox, which uses a public Amazon Web Services S3 bucket to store all sanitized data provided by the CV Pilot projects. Users can access the Data Sandbox through a web interface that allows users to search through the Sandbox to see the data available and through the Amazon Web Services Command Line Interface. The ITS Public Data Hub also provides instructions and examples for advanced users on GitHub on how to access the data programmatically.
The second way is through the live updating data samples provided on data.transportation.gov. Separate datasets for the BSMs and TIMs are designed to store up to 3 million records, estimated to be about a day’s worth of messages as Wyoming ramps up data production. These datasets are updated every five minutes (BSM) or every hour (TIM) with the newest messages delivered to the ITS Public Data Hub. These are provided as a sample to help users get a better feel for the data available as well as providing the opportunity to use the advanced data analysis and visualization tools available through Socrata. Socrata is a data as a service platform designed for government agencies to provide open data to the public alongside visualization, filtering and other data processing tools.
(Data.transportation.gov BSM sample)
The Wyoming CV Pilot BSM One Day Sample is the second most-downloaded and third most-viewed ITS JPO dataset on data.transportation.gov since inception. Over 350,000 file downloads of Wyoming CV Pilot data have been completed on the Data Sandbox, with 43,383 downloads of sanitized BSM files and 318,572 downloads of sanitized TIM files.
Wyoming’s work in preparing and building out a data pipeline will create a model for ongoing data delivery that will be further refined by the other CV Pilot sites and used by future USDOT ITS JPO projects.
