Boeing MCAS Crashes Highlight Automation Design Flaw
A Boeing MCAS system was at the center of two plane accidents that killed 346 people. The system, which was not disclosed in the 737 MAX crew manual or training materials before the Lion Air Flight 610 accident on October 29, 2018, relied solely on one angle-of-attack sensor to trigger MCAS.
Boeing publicly named MCAS twelve days after the first accident and four months before the Ethiopian Airlines Flight 302 crash in March 2019. The system had no mechanism for treating a single sensor's input as possibly faulty, leading to the accidents.
According to investor Bill Gurley, the failure is not just about disclosure but also about the system's inability to represent uncertainty about its own inputs. MCAS acted on a false reading with the same confidence it brought to a true one, highlighting a general property of automation acting on sensed input.