Audit Reports

-A A +A
skip-to-content

Inadequate Data and Analysis Undermine NHTSA’s Efforts To Identify and Investigate Vehicle Safety Concerns

Requested by the Secretary of Transportation
Project ID: 
ST2015063

Since February 2014, the General Motors Corporation (GM) has recalled 8.7 million vehicles in the United States due to an ignition switch defect that can unexpectedly shut down the engine and disable power steering, power brakes, and air bags. The defective ignition switches have been linked to more than 110 fatalities and 220 injuries. The National Highway Traffic Safety Administration’s (NHTSA) Office of Defects Investigation (ODI) identified the GM air bag non-deployments as a potential safety issue but ultimately decided not to investigate the problem. The Secretary requested that we assess NHTSA’s vehicle safety procedures and determine whether NHTSA had information on GM’s ignition switch issues.

ODI’s processes for collecting vehicle safety data are insufficient to ensure complete and accurate data. Deficiencies in ODI’s vehicle safety data are due in part to the Agency’s lack of detailed guidance on what information manufacturers and consumers should report—resulting in inconsistent data that ODI investigative chiefs consider to be of little use. Weaknesses in ODI’s processes for analyzing vehicle safety data further undermine ODI’s efforts to identify safety defects. Specifically, ODI does not follow standard statistical practices when analyzing early warning reporting data, and ODI does not thoroughly screen consumer complaints or adequately train or supervise its staff. Collectively, these weaknesses have resulted in significant safety concerns being overlooked. Finally, ODI’s process for determining when to investigate potential safety defects is insufficient to prompt needed recalls and other corrective actions. While ODI has identified factors for deciding whether an investigation is warranted, it has not developed sufficient guidance or reached consensus on how these factors should be applied. ODI’s investigation decisions also lack transparency and accountability.

NHTSA concurred with all 17 of our recommendations to improve ODI’s processes for collecting and analyzing vehicle safety data and for determining which potential safety issues warrant investigation.

Recommendations

Open

Closed

Closed on 08.22.2017
No. 1 to NHTSA

Develop and implement a method for assessing and improving the quality of early warning reporting data.

Closed on 04.14.2016
No. 2 to NHTSA

Issue guidance or best practices on the format and information that should be included in non-dealer field reports to improve consistency and usefulness.

Closed on 03.30.2016
No. 3 to NHTSA

Require manufacturers to develop and adhere to procedures for complying with early warning reporting requirements; and require ODI to review these procedures periodically.

Closed on 06.27.2016
No. 4 to NHTSA

Expand current data verification processes to assess manufacturers' compliance with regulations to submit complete and accurate early warning reporting data. At minimum, this process should assess how manufacturers assign vehicle codes to specific incidents and how they determine which incidents are reportable.

Closed on 04.29.2016
No. 5 to NHTSA

Develop and implement internal guidance that identifies when and how to use oversight tools to enforce manufacturers' compliance with early warning reporting data requirements.

Closed on 09.30.2015
No. 6 to NHTSA

Provide detailed and specific guidance to consumers on the information they should include in their complaints, as well as the records they should retain (such as police reports and photographs) in the event that ODI contacts them for more information.

Closed on 08.08.2017
No. 7 to NHTSA

Develop an approach that will determine which early warning reporting test scores provide statistically significant indications of potential safety defects.

Closed on 03.14.2017
No. 8 to NHTSA

Periodically assess the performance of the early warning reporting data tests using out-of-sample testing.

Closed on 03.14.2017
No. 9 to NHTSA

Institute periodic external expert reviews of the statistical tests used to analyze early warning reporting data to ensure that these methods are up-to-date and in keeping with best practices.

Closed on 04.18.2016
No. 10 to NHTSA

Implement a supervisory review process to ensure that all early warning reporting data are analyzed according to ODI policies and procedures.

Closed on 03.30.2017
No. 11 to NHTSA

Develop and implement a quality control process to help ensure complaints are reviewed thoroughly and within a specified timeframe.

Closed on 06.20.2016
No. 12 to NHTSA

Update standardized procedures for identifying, researching, and documenting safety defect trends that consider additional sources of information beyond consumer complaints, such as special crash investigation reports and early warning data.

Closed on 12.02.2015
No. 13 to NHTSA

Document supervisory review throughout the pre-investigative process including data screening.

Closed on 06.20.2016
No. 14 to NHTSA

Evaluate the training needed by pre-investigative staff to identify safety defect trends; and develop and implement a plan for meeting identified needs.

Closed on 01.12.2016
No. 15 to NHTSA

Develop and implement guidance on the amount and type of information needed to determine whether a potential safety defect warrants an investigation proposal and investigation.

Closed on 06.27.2016
No. 16 to NHTSA

Develop a process for prioritizing, assigning responsibility, and establishing periodic reviews of potential safety defects that ODI determines should be monitored.

Closed on 02.05.2016
No. 17 to NHTSA

Document and establish procedures for enforcing timeframes for deciding whether to open investigations; and establish a process for documenting justifications for these decisions.