Toyota and Runaway Acceleration A Case Study Some Background • From 2009-2011 Many of Toyota’s vehicles experienced an event called “runaway acceleration” or “unintended acceleration”. This involves a car accelerating without given input or with unintentional input. Needless the results of runaway acceleration can be dangerous and often fatal. Inciting Event • Toyota would receive the occasional complain about unintended acceleration however, it was not uncommon for these types of errors to be at the fault of the driver (hitting the gas pedal as opposed to the break by accident). As such, many of these events were ignored. However, in 2009, a highway patrol officer named Mark Saylor was tragically killed by one of these instances of runaway acceleration, prompting more official inquiry Reyes, Alvin. “Deadly Defect: The Story behind Toyota’s Biggest Controversy.” SlashGear, SlashGear, 3 Sept. 2023, www.slashgear.com/1380568/toyota-biggestcontroversy-accelerator/. Official Story and Investigation • According to Toyota, many of these issues were the fault of drivers and all-weather floor mats which would cause the pedal to get stuck in certain vehicles. • An investigation held by the NHTSA (national highway transport safety administration) with the collaboration with NASA revealed that there were more factors at play. This included pedals that could get stuck in an accelerating position, potential electrical failures caused by frayed wires (called “tin whiskers”), but these were combated with several failsafe measures to ensure safety of the vehicle. No software issues were discovered. U.S. Department of Transportation National Highway Traffic Safety Administration, "Technical Assessment of Toyota Electronic Throttle Control (ETC) Systems". February 2011, https://www.nhtsa.gov/sites/nhtsa.gov/files/nhtsa-ua_report.pdf NASA Engineering and Safety Center, "National Highway Traffic Safety Administration Toyota Unintended Acceleration Investigation". 18 January 2011, https://static.nhtsa.gov/odi/inv/2014/INRP-DP14003-61483.pdf Case Closed • Toyota paid $1.2 billion in order to avoid criminal prosecution • Recalls made • Payouts to Toyota Owners • Runaway Toyotas put to an end ABC News. “Toyota to Pay $1.2B for Hiding Deadly ‘Unintended Acceleration.’” ABC News, ABC News Network, 14 Mar. 2014, https://abcnews.go.com/Blotter/toyota-pay-12b-hiding-deadly-unintendedacceleration/story?id=22972214/. Ifill, Gwen. “What Toyota’s $1.2 Billion Settlement Means for the Auto Industry.” PBS, 19 Mar. 2014, https://www.pbs.org/newshour/show/toyotas-1-2-billion-settlement-means-auto-industry Spaghetti Spaghetti Spaghetti Testing Toyota’s Software • As previously mentioned, NASA had investigated the vehicles and occurrences of runaway acceleration themselves. They had concluded that the electronics did not play a major in the malfunctions, particularly in terms of the software. • However, they admit themselves to not testing the software thoroughly. • “it is not realistic to attempt to ‘prove’ that the [vehicle’s computer] cannot cause UAs. Today’s vehicles are sufficiently complex that no reasonable amount of analysis or testing can prove electronics and software have no errors” (NASA, 20). NASA Engineering and Safety Center, "National Highway Traffic Safety Administration Toyota Unintended Acceleration Investigation". 18 January 2011, https://static.nhtsa.gov/odi/inv/2014/INRP-DP14003-61483.pdf Why Was There So Little Effort Put Into Investigating The Software? “Toyota Unintended Acceleration and the Big Bowl of ‘Spaghetti’ Code.” Safety Research & Strategies, Inc., 7 Nov. 2013, https://www.safetyresearch.net/toyota-unintended-acceleration-and-the-big-bowl-of-spaghetti-code/ . Dunn, Michael. “Toyota’s Killer Firmware: Bad Design and Its Consequences.” EDN, 28 Oct. 2013, https://www.edn.com/toyotas-killer-firmware-bad-design-and-itsconsequences/. How Bad Was It Really? According to Micheal Barr, an embedded systems expert who spent 18 months investigating the source code, it was truly “spaghetti code” Over 10,000 instances of global variables, including variables that were responsible for a vehicles acceleration Prevalent Bugs Improper Memory protection, and protections against bit flips Program responsible for monitoring this software for errors didn’t work properly Monitoring program threw away error codes raised Barr, Micheal. "Bookout V. Toyota.“ Barr Group, 2013, https://www.safetyresearch.net/Library/BarrSlides_FINAL_SCRUBBED.pdf “Toyota Unintended Acceleration and the Big Bowl of ‘Spaghetti’ Code.” Safety Research & Strategies, Inc., 7 Nov. 2013, https://www.safetyresearch.net/toyota-unintended-acceleration-and-the-big-bowl-of-spaghetti-code/ . The Issues Continue • 67 of the source code’s functions were so poorly designed that they were untestable • The function responsible for angle throttle control, which controls vehicle acceleration, was so poorly designed that it wasn’t even maintainable • Breaking of industry standards with recursion, failed to do runtime stack monitoring • Failed to follow OSEK, an international standard API for automotive software • Over 80,000 violations of MISRA-C, a set of rules within the automotive industry for software reliability. • Failed to follow even their own set of rules for designing software, with over 30% of their own rules being violated within the source code. Barr, Micheal. "Bookout V. Toyota.“ Barr Group, 2013, https://www.safetyresearch.net/Library/BarrSlides_FINAL_SCRUBBED.pdf “Toyota Unintended Acceleration and the Big Bowl of ‘Spaghetti’ Code.” Safety Research & Strategies, Inc., 7 Nov. 2013, https://www.safetyresearch.net/toyota-unintended-acceleration-and-the-big-bowl-of-spaghetti-code/ . It Gets Even Worse • Toyota Failed to mirror several critical variables in order to protect them from corruption. Variables responsible for vehicle acceleration were found amongst these. • All fail safes for the vechicle along with throttle control were run under the same task, meaning if task death were to occur the vehicle would not be able to use any of its failsafes properly. Most diagnostics code needed this single task as well. Barr calls this “Task X” • Monitoring software didn’t work properly, so if Task X were to die and the vehicle be put into an unsafe state, the software itself would not be able to detect this • Multiple single points of failure Barr, Micheal. "Bookout V. Toyota.“ Barr Group, 2013, https://www.safetyresearch.net/Library/BarrSlides_FINAL_SCRUBBED.pdf Alternative Verdict • Barr and Safety Research, an independent organization who also investigated Toyota concluded the same thing; that the software itself were a very likely cause of the unintended acceleration issues. And should have had much more thorough investigation. Click, Felix, et al. "An Examination of the National Highway Traffic Safety Administration and the National Aeronautics and Space Administration Engineering Safety Center Assessment and Technical Evaluation of Toyota Electronic Throttle Control (ETC) Systems and Unintended Acceleration" Safety Research & Strategies Inc., 23 May 2011, https://www.safetyresearch.net/Library/NHTSA-NASA_Response_Final_052311.pdf “Toyota Unintended Acceleration and the Big Bowl of ‘Spaghetti’ Code.” Safety Research & Strategies, Inc., 7 Nov. 2013, https://www.safetyresearch.net/toyota-unintendedacceleration-and-the-big-bowl-of-spaghetti-code/ . Takeaways • Undeniably Toyota, but it’s ineffective to be more specific than that, a mixture between the engineers and company management. • Management should not have tried to hide facts and the reality of the situation. They should have also been stricter on quality assurance Who’s At Fault? • Engineers failed to meet most industry standards, best practices, and likely failed to run any true quality assurance on their code, going by Burr’s reports. Their code got the point were even a first year computer science student could easily question their design decisions. How To Do Better • A very quick way to improve would be, as Burr pointed out, to actually follow industry standards set in place. They do, after all, exist for a reason. Failure to follow these standards and any form of quality insurance likely lead to many people being injured and the deaths of a few. • Need to also follow best safety practices for software such as redundancy, error handling, stack monitoring, etc. • Professor Phil Koopman from Carnegie Mellon University investigated this case himself and derived a few more helpful additions • He conceded that testing alone is not enough to guarantee safety, so proposed adopting a Safety Integrity Level (SIL) Approach, in which issues are addressed in order of failure severity. • He also pointed out that, at least in the US, there are no certification standards when it comes to the safety of automotive software, automotive companies aren’t even legally required to follow MISRA, the industry standard for safety. Koopman, Phil. "A Case Study of Toyota Unintended Acceleration and Software Safety" Carnegie Mellon University. 18 Sept. 2014, https://users.ece.cmu.edu/~koopman/pubs/koopman14_toyota_ua_slides.pdf Takeaways • Software quality can quite literally become a matter of life and death. • The developer themselves are the first line of defense against this and should always opt towards best practices and trying to ensure code is as bug-free, concise, testable, and maintainable as possible. • Third party quality assurance is almost always a good idea • While NASA themselves didn’t find any issues with the software this was mostly because they used automated tools and failed to do a deep dive. Barr identified these issues and, had they been brought to Toyota’s attention sooner, they likely could have been addressed before people got injured • Standards exist for a reason and we should try to follow them. • Had Toyota been following industry standards such as MISRA or even their own standards its likely the quality of the code could have been improved immensely and even avoided many of its own issues, again saving people. • When In Doubt, Blow The Whistle • There have been several former employees of Toyota who blew the whistle about issues like a problematic culture around safety, if nobody is ensuring that is safe for the public, it might be worth blowing the whistle in order to help others. In this case, if the quality of the engineering was made public sooner lives may have been saved. Bibliography Barr, Micheal. "Bookout V. Toyota.“ Barr Group, 2013, https://www.safetyresearch.net/Library/BarrSlides_FINAL_SCRUBBED.pdf. Accessed 12 January 2024. Click, Felix, et al. "An Examination of the National Highway Traffic Safety Administration and the National Aeronautics and Space Administration Engineering Safety Center Assessment and Technical Evaluation of Toyota Electronic Throttle Control (ETC) Systems and Unintended Acceleration" Safety Research & Strategies Inc., 23 May 2011, https://www.safetyresearch.net/Library/NHTSA-NASA_Response_Final_052311.pdf. Accessed 12 January 2024. Dunn, Michael. “Toyota’s Killer Firmware: Bad Design and Its Consequences.” EDN, 28 Oct. 2013, https://www.edn.com/toyotas-killer-firmware-bad-design-and-its-consequences/.. Accessed 13 January 2024. Ifill, Gwen. “What Toyota’s $1.2 Billion Settlement Means for the Auto Industry.” PBS, 19 Mar. 2014, https://www.pbs.org/newshour/show/toyotas-1-2-billion-settlement-meansauto-industry. Accessed 13 January 2024. Koopman, Phil. "A Case Study of Toyota Unintended Acceleration and Software Safety" Carnegie Mellon University, 18 Sept. 2014, https://users.ece.cmu.edu/~koopman/pubs/koopman14_toyota_ua_slides.pdf.. Accessed 11 January 2024. NASA Engineering and Safety Center, "National Highway Traffic Safety Administration Toyota Unintended Acceleration Investigation". 18 January 2011, https://static.nhtsa.gov/odi/inv/2014/INRP-DP14003-61483.pdf. Accessed 11 January 2024. Reyes, Alvin. “Deadly Defect: The Story behind Toyota’s Biggest Controversy.” SlashGear, 3 Sept. 2023, www.slashgear.com/1380568/toyota-biggest-controversy-accelerator/. Accessed 11 January 2024.. “Toyota to Pay $1.2B for Hiding Deadly ‘Unintended Acceleration.’” ABC News, ABC News Network, 14 Mar. 2014, https://abcnews.go.com/Blotter/toyota-pay-12b-hiding-deadlyunintended-acceleration/story?id=22972214/.=. Accessed 13 January 2024. “Toyota Unintended Acceleration and the Big Bowl of ‘Spaghetti’ Code.” Safety Research & Strategies, Inc., 7 Nov. 2013, https://www.safetyresearch.net/toyota-unintendedacceleration-and-the-big-bowl-of-spaghetti-code/. Accessed 12 January 2024. U.S. Department of Transportation National Highway Traffic Safety Administration, "Technical Assessment of Toyota Electronic Throttle Control (ETC) Systems". February 2011, https://www.nhtsa.gov/sites/nhtsa.gov/files/nhtsa-ua_report.pdf. Accessed 11 January 2024.
