PulseUX Blog

Theory, Analysis and Reviews on UX User Experience Research and Design

HomeMissionAbout Charles L. Mauro CHFP

The Human Factors Science Behind The 737MAX MCAS Problem and Why The MAX Should Be Scrapped

The Spin There has been a great deal of back and forth over the reasons two Boeing 737 MAX airliners took the lives of 346 innocent souls. At the core of the problem is the classic duality of man vs. machine. Were the crashes the result of incompetent pilots and failure to follow procedures (The Man) or was the root cause a gross failure on the part of Boeing to employ rigorous and professional human factors analysis, design and testing in the development of the 737 MAX (The Machine)? In the beginning, the Boeing mediated media spin pushed the needle in the direction of human error, focusing on poor training and pilot inexperience. But soon enough, it became clear based on the excellent reporting of The New York Times that apparently, there was a lot for Boeing to account for both structurally and conceptually with respect to the design of the 737 MAX.

The FAA There were also rather staggering accusations leveled at the FAA concerning lack of oversight during development of the MAX, including criticism for failure to oversee basic function allocation design, automation design, root cause analysis, human factors design criteria development, flight domain modeling, training regimen development, negative transfer analysis and all manner of other straight-up human factors science problems. Now that the dust has settled on the basic inquiry of the MAX, it is clear that Boeing, and by association, the FAA not only dropped the ball but drop kicked the pilots into a flight regimen so terrifying and unforgiving that one can only imagine their pain when they realized they had no control over their massive chunk of aluminum and jet fuel as it porpoised into Tera Firma at terminal speed. This was not part of the pilot’s mental model and that frankly is exactly the problem.

What Mental Model? At the core of all complex man/machine problems, such as the design of the flight deck of a major commercial aircraft, is the user’s (pilot’s) ability or inability to create a working mental model of the functional state and interaction modes of the system. The user’s mental model should have sufficient detail to allow the formation of a sort of dialogue with the machine that results in net positive benefits for those who engage with such technology and society at large. A massive and complex piece of technology that ends up in the hands of users (pilots) without providing the benefit of a robust mental model is not only highly incompetent engineering, but, as one can now see, ethically and professionally bankrupt corporate behavior.

This idea of mental model development is more than an arcane bit of cognitive science. It turns out that ALL technology we deal with requires our minds to develop, internalize and operationalize an understanding of how a given product actually works. This is the same for your smartphone, your new Tesla or even the TV remote. Usability starts and ends with mental model development. However, the creation of a robust mental model is much more science than art.

But let us be clear here, the issues with the FAA and Boeing are much more than simply piling on with yet another negative analysis. The question now becomes exactly how does one design a complex flight control system, like that of the 737 MAX, so that pilots do in fact develop and operate on the basis of a validated and productive mental model? Recently, there has been an attempt on the part of Boeing and the FAA to imply that creation of a MAX 737 MCAS system devoid of such human factors problems was somehow a problem too complex, too costly, or too time-consuming, given the demands of the global marketplace for commercial aircraft. This point of view is patently false and here is why.

The Science of Human Factors is NOT Design There is a well understood and widely utilized expertise domain known as professional human factors science which can and does produce solutions to complex man/machine problems, such as those found in the MAX and related flight control systems. This field of expertise is based on graduate-level academic training in human factors science, cognitive engineering, and ergonomics. One can obtain either a master’s degree or a Ph.D. in human factors science from scores of US institutions of higher learning. There is a formal certification process for such professionals.

To be clear, this is science, not design. Human Factors Science is not industrial design, or UX design or User Experience Design. It is very common for corporate executives to think that design, in its many flavors, delivers human factors solutions to complex problems. This is not the case. Today there are a few thousand qualified human factors scientists working behind the scenes on the truly complex man/machine problems. They are for the most part quiet and thoughtful research professionals who maintain a low profile. There are no TED Talks specifically on human factors science.

The Human Operating System (HOS) The methods employed by such professionals have been in place for decades and utilized to solve some of the most vexing and complex man/machine problems. This is a field where the science and functional limitations of the human operating system (HOS) are integrated with the design and engineering of advanced technology-based systems. It is the science that solves deeply complex man/machine problems based on validated research methods and a working knowledge of the HOS.

HFES The primary professional organization for those who practice this discipline in the US is the Human Factors and Ergonomics Society. The HFES has a large body of peer-reviewed professional research literature, annual conferences, major awards, and deep penetration into some industries. However, today the highly qualified majority of HFES members work on DOD-related systems and not in the broader commercial environment. This is changing rapidly.

But There Is A Problem The important point is that this discipline has in place methodologies, research protocols and testing frameworks that could have and can in the future ensure that a Boeing 737 MAX never takes the lives of passengers based on gross human factors science oversight. The problems with the 737 MAX MCAS system were both knowable and entirely avoidable. It is impossible for the Boeing 737 MAX development teams to have even considered an MCAS system design as delivered to the marketplace had human factors science methods been utilized and validated during development. But there is a problem with this relatively new field of human factors science. In fact a big problem.

The problem is that, as a professional discipline, human factors science it is not well known within industry at large. More importantly, it is completely unknown in the halls of Congress, where laws are developed that can compel federal agencies such as the FAA, as well as those who develop technology like Boeing to adhere to the science of human factors in much the same way pharmaceutical companies must adhere to FDA Guidance during development and validation of drugs for human use. Lack of legislative awareness of human factors science is the death keel for changes in legal frameworks and actual public policy related to the design and production of complex man/machine systems like the MAX.

The MAX Is Not A Smartphone These complex systems like the 737 MAX are exactly the types of man/machines systems that will define how productive we are as a society going forward. The development of an easy to use smartphone is trivial compared to the design of far more complex systems like autonomous vehicles, online voting systems, AI-based decision support products, large scale city planning, environmental control systems, specialized military systems, medical delivery and clinical outcome optimization, distributed education and large-scale financial market optimization. These are the problems of scale, as is the design of a commercial aircraft like the MAX which was designed to be produced by the thousands, flown by pilots of wide-ranging expertise over highly populated areas with massive shareholder pressure for airline and Boeing profitability. These are not the problems of another smartphone, internet-enabled toothbrush or the next generation of the home voice assistant. If the Boeing 737 MAX MCAS problem has proven anything, it is that complexity demands science and science demands a body of validated methods and research on which to base both conceptual design and system validation. However, where do we turn for a solution that dramatically increases the awareness of human factors science and its related benefits?

Who Pays The Bills? It turns out that professional societies such as the Human Factors and Ergonomics Society are complex, highly political and always unwieldy entities. Within such organizations, groups and sub-groups form, break up, argue and even reject the research of their peers. The large scale and dynamic nature of such a professional society make it almost impossible for such institutions to speak with one voice. This is a highly unfortunate fact that limits the maximum possible impact of a professional society to push for broad social or professional change. Even though professional societies have the scale, they oftentimes do not act at scale by taking a collective position on topics that are broadly important to their professional membership. Professional societies are often held back by internal political agendas. Speaking out on important issues can cause serious problems as board members are often afraid to offend large corporate sponsors of their annual meetings or upset companies that routinely give large sums to social events. Sometimes board members are actually from companies impacted by events on which the society at large should have a point of view.

Taking the Risk This is another way of saying that professional societies rarely take a political position as an entire entity. HOWEVER, in the case of the failure of Boeing to employ robust human factors science in the design of the 737 MAX, the Human Factors and Ergonomics Society took an inspiring and laudatory step of stating, in no uncertain terms, why the 737 MAX was a massive failure. The HFES then went on to give the US Congress an object lesson in exactly how human factors science can be applied in the future to solve these types of problems.

Making The Point The HFES message to Congress was delivered by Ms. Mica Enderly, the former President of the HFES, in a session before the US House Committee on Transportation and Infrastructure. Her session spanned over 3.5 hours of direct testimony. Make no mistake, listening to Dr. Endsley’s testimony serves as an object lesson to all that there is a science that can and will help solve the most complex technology-based problems we will face in the future. We can only hope Congress was listening and the FAA was taking notes…copious notes. But of course, that is not the end of the story.

Should The MAX be Scrapped? What about Boeing and the future of the 737 MAX as a commercially viable product? That is a question which testimony to Congress cannot answer. The reported findings show a culture so broken in terms of the application of human factors science that one must wonder what else lurks in the millions of lines of computer code, faulty transducer logic design, training and simulation design, manufacturing SOP adherence and failure to define negative transfer from prior 737 aircraft. This raises the question, should the 737 MAX be scrapped? It that a question too far?

When one considers the systemic failures of Boeing, the question of scrapping the MAX is worth asking, if not pondering in detail. If the MAX is considered for recertification, it should never be allowed to carry passengers without a complete human factors audit of the entire man/machine system including, flight deck design, flight domain design, function allocation analysis, training regimen development and validation, training simulation design and validation and confirmation that all manufacturing SOPs were followed and validated. Such an effort should be followed by a complete and robust whole system human factors validation of the 737 MAX as a new aircraft, not linked in any way to prior 737 versions. From a human factors science point of view, the MAX is a new and arguably defective design compared to all prior generation of the 737 family going back decades. If there is an object lesson in the MAX, it is that both government and industry ignore human factors science, at the risk of damaging and/or destroying massive shareholder value and public policy integrity. Fix it right or scrap it…Without robust new FAA policy, scrapping seems like the best option.

Charles L. Mauro CHFP

President / Founder Mauro Usability Science


Twitter: @PulseUX

BLOG: https://www.mauronewmedia.com/blog/

For the complete 3.5 hour testimony of Dr. Endsley go here: https://transportation.house.gov/committee-activity/hearings/the-boeing-737-max-examining-the-federal-aviation-administrations-oversight-of-the-aircrafts-certification

Post a Comment

Your email address will not be published. Required fields are marked *