Follow-up Audit on NHTSA's Office of Defects Investigation
We found that NHTSA successfully implemented 20 of the 22 TREAD Act requirements, and completed development of a new safety defects information system, called ARTEMIS, in July 2004. However, the ARTEMIS development effort experienced significant cost increases and schedule delays. For example, development cost estimates increased 76 percent from $5.35 million in June 2001 to $9.4 million in March 2004, and schedule estimates increased from 21 to 42 months during the same time period. We also found that NHTSA had identified, but could not verify $17.12 million in future operations and maintenance costs for ARTEMIS. After we questioned how these costs were derived, NHTSA reduced the amount to $11.46 million, thus creating an opportunity to put $5.66 million to better use.
In addition, we found that ARTEMIS does not have the analytical capabilities originally envisioned to help point analysts toward potential safety defects warranting further investigation. NHTSA plans to separately acquire these capabilities, but has not finished defining the capabilities needed, identified the software it will purchase to analyze the early warning reporting information, outlined associated costs, or established a schedule for implementing these capabilities. Nevertheless, NHTSA now has much more information from which to identify potential safety defects. Ensuring that the early warning reporting information is thoroughly and consistently analyzed to identify potential safety defects is especially critical because in July 2003 NHTSA announced it would publicly release only a portion of the early warning reporting information reported by manufacturers, unless a defects investigation is opened. Because only NHTSA will have access to the majority of the early warning reporting information, it is critical that NHTSA complete screening procedures to address congressional concerns expressed in September 2000 about its ability to use the data it possessed to spot trends related to failures in Firestone tires.
NHTSA concurred with our recommendations, stating that it will ensure the $11.46 million in estimated ARTEMIS operations and maintenance costs is adequately supported. NHTSA will also better define its plans for acquiring the advanced analytical capabilities, including identifying the cost of purchasing additional commercial-off-the-shelf software and the milestones for obtaining the capabilities. Further, NHTSA agreed to set time frames for completing procedures to incorporate early warning reporting information into the defects screening process and to train defect analysts on the new procedures. However, because NHTSA's comments did not explain what those actions were or when they would be implemented, we requested more details on the specific steps that NHTSA will take.