PulseUX Blog

Theory, Analysis and Reviews on UX User Experience Research and Design

Home Mission About Charles L. Mauro CHFP

A Critical Analysis of FDA Guidance for User Percentile Device Design Criteria Versus Currently Available Human Factors Engineering Data Sources and Industry Best Practices

A staggering report was recently released by the International Consortium of Investigative Journalists (ICIJ) indicating that 83,000 deaths and 1.7 million injuries were linked to medical device adverse events reported over the last decade. Furthermore, a 2013 report by McKinsey & Company suggested that such adverse events and associated quality issues cost the medical device industry between $2.5 billion and $5 billion per year on average. There is also indication that some aspects of both medical device human factors engineering (HFE) design and Food and Drug Administration (FDA) HFE validation criteria may not provide a proper methodology for selecting and testing patient populations. This may lead medical devices that pass FDA HFE validation to still exhibit significant use errors when utilized by actual patients in the real world.

Feature Article – Down to a Science: Charles Mauro’s Expertise in Neuroscience-Based Design Research Dates Back to His ArtCenter Education

Kim Dalton

An exceptional piece was just published in the ArtCenter DOT Magazine exploring the professional journey of Charles L. Mauro CHFP, our founder and president, from ArtCenter College of Design to the development of award-winning solutions for world-class clients utilizing advanced neuroscience-based methodologies.

Mauro’s experience in Russia led him to fulfilling a lifelong vision: starting a firm that combined robust science on human factors performance with the rigorous design expertise he learned at ArtCenter. He launched Mauro Usability Science in 1975 and quickly discovered there were three main industries interested in his firm’s expertise—the U.S. military, medical technology, and Wall Street. The latter brought Mauro one of his firm’s defining projects.

Read Article

Understanding Costs And Risks For Human Factors Engineering (HFE) Usability Studies – Part 2: Outsourcing HFE Usability Testing

Kim Dalton

MedDeviceOnline-Author-Image — *Image courtesy of MedDeviceOnline*

Many of you may have read Part 1 in our series on Understanding Costs And Risks For HFE Usability Studies, which discussed the factors that device development teams should consider when executing HFE studies internally. We recently published the second article in this series, which discusses how to optimize data quality while controlling for study costs and time constraints when employing an independent HFE testing agency.

The goal of a device development team should be to obtain the highest quality data possible, given budgetary and time constraints. High-quality data is a non-negotiable requirement for HFE testing because device development teams need it to make critical design decisions or even consider submission for FDA review and approval. Thus, teams must evaluate whether a given HFE testing agency will deliver high-quality data at the price proffered in a formal bid. Such evaluation involves two questions: whether the independent agency is capable of delivering quality data, and whether the agency’s proposed study design is based on a robust understanding of HFE best practices.

Read Full Article

Please share this article with any of your colleagues who may find it of interest, and feel free to reach out to us directly by email with any follow up questions.

Should McKinsey Design Retract Its “Business Value of Design” Report and Related MDI Index?

Kim Dalton

By Charles L. Mauro CHFP and Paul W. Thurman DBA, MS

Two weeks ago my research partner, Paul Thurman, and I published an analysis of the McKinsey Design “Business Value of Design” (BVD) report and MDI Index. That analysis articulated the numerous research problems with the report and put forward the opinion that its findings were, from a research perspective, both inaccurate and unsupportable. For any of you who have not read the original report and then the critical analysis, you should consider doing so as the combination is both insightful and, at the same time, deeply concerning. Should McKinsey retract its BVD report? Here are the issues…you decide.

Critical Analysis: Why McKinsey’s “The Business Value of Design” Report Ultimately Misstates the Value of UX Design and Design Overall

Kim Dalton

By Charles L. Mauro CHFP and Paul W. Thurman DBA, MS

Recently, there has been an ongoing fascination with McKinsey’s acquisition of design agencies and how the renowned management-consulting firm would turn a profit from such investments. One would expect such an integration of high design and management consulting by a world-class firm like McKinsey to produce industry changing insights and expertise options. However, if the recently published and highly publicized McKinsey Design industry report “The Business Value of Design” (hereafter referred to as BVD) and associated McKinsey Design Index (hereafter referred to as MDI) is any indication, the grand vision may need reconsideration. It appears, based on critical analysis, that the breathless excitement presented by McKinsey Design in the BVD release video, claiming to establish for the first time “individual design actions that leaders have taken…tied back to business performance” is not only inaccurate, but unsupportable. Sadly, given the business gravitas of McKinsey and its obvious financial resources, it is clear that this a major missed opportunity for McKinsey Design, and design overall.

Important Note: Prior to publication of this review, the authors attempted to contact McKinsey Design BVD and MDI authors, but received no response. It was our intention to seek clarification and elaboration on the findings and supporting data before publication of this review. We welcome a response to this analysis by the McKinsey Design research team. The business value of design is of value to all who engage in the theory and practice of design and design science.

Understanding Costs And Risks For HFE Usability Studies — Part 1: Testing In-House

Kim Dalton

Executing HFE studies internally is often far more complex and time-consuming than drug delivery device development teams realize. MUS recently published part one of a two-part article detailing the factors that need to be considered by device development teams in order to produce high-quality usability testing data appropriate for decision-making and HFE optimization while maximizing cost-effectiveness.

The U.S. Food and Drug Administration (FDA) is increasing demands^[i],[ii] for drug delivery device developers to demonstrate the human factors engineering (HFE) effectiveness of their devices and related labeling systems. Therefore, rigorous HFE usability testing is required to ensure a given medical device meets HFE standards prior to submission for FDA approval.

Read Full Article

Keep an eye out for part two of this series in which we will describe in detail how device development teams can best understand and manage the costs of high-quality usability testing, should they decide to retain an independent testing agency.

October 2018 Panels and Speeches by Charles L. Mauro

Kim Dalton

Charles L. Mauro CHFP Delivers Keynote Speech at HFES International Annual Meeting 2018 in Philadelphia. Charles L. Mauro CHFP was the invited keynote speaker at the recent Human Factors and Ergonomics Society (HFES) International Annual Meeting held in Philadelphia, PA on October 2^nd, 2018. The speech titled: “Human factors engineering innovations in the age of Apple v. Samsung: How HF/E practice can add major value to the intellectual property assets of corporations and academic institutions” was well-received by the audience made up of HFE professionals from both academia and industry. The keynote talk focused on a detailed analysis of the far-reaching impact of the recent Supreme Court decision and its impact on device design IP theory and practice. The talk explored the use of neuroscience-based testing methods to determine infringement and to provide rigorous research-based support for IP litigation.

Charles L. Mauro CHFP Invited to Speak on Prestigious Panel at Design Law 2018 Conference in Washington, DC. It is no secret that design patent law is in a major period of upset and reformation. The dramatic changes are the direct result of the recent Supreme Court decision in several intellectual property cases, including Apple v. Samsung. The prestigious panel, including Christopher J. Renk of Banner & Witcoff Ltd., John Froemming of Jones Day and Nika F. Aldrich of Schwabe, Williamson & Wyatt, was focused on the complex issues of damage calculations once infringement has been found by the jury. Mr. Mauro covered theory and practice applied to testing the impact of visual design on consumer decision making and related research.

(Photo by D. Barnes)

Recent Research: Real-World Usability Testing

Kim Dalton

MUS Executes Complex Multi-City Study to Explore Disparities in Actual Use of Complex Drug Delivery Devices vs. Expected HFE Performance Based on Summative FDA Study Data.

There is an interesting phenomenon occurring in the actual real-world use of more complex drug delivery devices by patients and HCPs. The problem is there is a wide disparity between how the device performs during summative HFE testing for FDA submission and the surprising number of reported usability problems recorded after the device actually gets into the hands of patients in real-world settings. This disparity is reason for concern because a dramatic decrease in HFE performance by a device can signal significant loss of clinical effectiveness. To address this question MUS has recently executed a complex multi-city HFE field analysis to determine the source of such disparities and to aid major clients in understanding the structure and basis for identifying and resolving such outcomes during formative and summative studies. The learnings are being applied to updated and more robust summative testing methods that will deliver to the FDA more reliable HFE performance data. Request more information.

Apple v. Samsung Update

Charles Mauro CHFP

Apple-Samsung-Court-Image

Back to Square One The landmark Apple v. Samsung matter is far from over. It is a story both tragic and hopeful. Here is my summary of the case as it currently stands on appeal before Judge Koh in California.

Involvement of The Professional Design Community and Related Amicus Brief Some of you may have signed the Supreme Court amicus brief in the Apple v. Samsung case that Charles Mauro co-wrote together with the legal team at Orrick. Thank you for your support.

Impact of Our Design Amicus Brief In October of 2016, the Supreme Court heard oral arguments from Apple, Samsung, and from the US Department of Justice as a neutral party. All justices read our amicus brief signed by numerous IDSA members and other world-class designers including Dieter Rams, Norman Foster, Terance Conran, fashion designer Calvin Klein and design heads of many leading design studios across the globe. The contents of the brief were referenced in the proceedings. The amicus received very wide exposure in the design and business press. In fact, the brief has become required critical reading for design students at leading design programs on a global basis.

What Happened At The Supreme Court Hearing During the actual proceedings in Washington DC, quite unexpectedly, Apple agreed with Samsung on the primary question before the court. This was a major surprise to everyone who worked to blunt the attack by Samsung. In short, the two parties agreed that it would be conceptually possible for damages in design patent cases to be calculated on less than the total profits obtained from the sale of the entire product. As you may recall, the total profits method for calculating damages has been in place since the late 1890s and was the only point which Samsung had asked the court to rule on.

The Not-So-Surprising Outcome Left with no meaningful dispute between the parties (a very rare event), the justices voted 3 months later in their final ruling that damages could be calculated based on the relevant component part of designs to which the design patent had been applied instead of to the total profits of the entire product. The ruling by the Supreme Court essentially gutted the prior damages legal framework and with it a good deal of design rights IP on a global basis. However, the ruling was no surprise given that both Apple and Samsung agreed on the key legal question. This ruling left the global design profession in disbelief.

Why Professional Usability Testing Matters

Charles Mauro CHFP

How Do Development Teams Produce World-Class Products? For consumer-facing hardware or software to have high levels of usability and UX quality, they must undergo rigorous professional usability testing before design freeze. Such professional testing must make use of a wide range of mockups, simulations, and functional prototypes. It turns out that such testing is a fundamental requirement of all consumer-facing hardware and software systems that will eventually deliver a high degree of consumer engagement. One can often predict the potential usability and UX performance of a hardware device or software interface simply by examining the development process and the degree to which professional human factors science and unbiased professional usability testing has been applied during development.

Investors Don’t Link Poor Usability With High-Risk Usability and feature/function engagement are criteria that are rarely investigated by institutional investors when assessing the relative risk of making an investment decision. Defective usability has played a major role in the failure of numerous recent high profile products including the now infamous SNAP Spectacles. A few simple questions by investors during SNAP’s recent IPO roadshow would have likely revealed a failure to employ human factors research and professional usability testing during development of the SNAP Spectacles. More generally, when management is asked whether or not their products have undergone professional user testing and feature/function engagement testing they will often respond in the affirmative. Many executives wrongly assume that running a survey, or obtaining feedback from a few employees or a small group of consumers constitutes professional usability testing. It does not.

Design Does Not Replace Professional Usability Testing Professional usability testing is not the same as UX Design, Industrial Design or Design Thinking which are concept generation frameworks. Professional usability testing is a concept verification methodology. In order to have confidence that what has been developed will be usable and engaging one must apply a meaningful measure of marketing science. Professional usability testing is, first and foremost, a science-based process that is designed to produce unbiased and robust data on products and software user experience performance. In professional usability testing programs, there is always a need for statistical confidence that can aid management in making objective decisions about the usability and feature/function engagement of hardware or software before design freeze. Continue Reading…