PulseUX Blog

Theory, Analysis and Reviews on UX User Experience Research and Design

Home Mission About Charles L. Mauro CHFP

A Note To Our Clients: Medical Device Usability Testing In The Age of COVID-19

Dear Research Colleague: We sincerely hope you and your family are safe and secure. Even though we have temporarily closed our offices in NYC, we remain fully productive in a remote capacity through our cloud-based mirror capability covering all projects and related research. This note is focused on helping our clients and professional colleagues navigate this complex and challenging time of COVID-19.

Clients Are Not Canceling Studies, But Reconfiguring To Meet New Risk Factors: Based on our direct experience and feedback from corporate research colleagues and clients, at this time, development teams are not canceling lab-based user testing studies, but are working to reconfigure studies into 3 primary categories (below is a link to our article that describes how to segment into 3 categories) based on product development priorities. Notably, today our firm has more requests for proposals (RFPs) for testing programs than prior to the onset of COVID-19 in the United States. We have also recently executed a complex lab-based usability study within the time of COVID-19 in NYC. Times have changed, but critical user testing will continue under a new set of risk mitigation standard operating procedures (SOPs) and utilization of only the most experienced user research teams. The following is a shortlist of 4 key steps we recommend for starting the process of dealing with user testing in the age of COVID-19.

Step 1: Segment Your User Study Pipeline Into 3 Categories: Your team should immediately review all planned user studies and segment them into 1 of 3 categories. We recently published an article describing how to proceed with such segmentation. The article describes segmentation and user testing reconfiguration in the context of medical device user testing; however, the same approaches can be applied to user testing for products, websites, or systems of any kind. Please feel free to send us questions based on that publication.

Step 2: Begin Updating Your User Testing SOPs Immediately: We highly recommend that all product development teams immediately update their SOPs to accommodate user testing in the age of COVID-19. There is no future for user testing without improved planning and risk mitigation processes. It is also important to generate a robust vendor selection criteria checklist to confirm that your user testing partners have in place updated risk mitigation SOPs as well. For those teams undertaking this process remotely, our team has found SLACK to be especially helpful in combination with the Google Docs collaboration framework. We also utilize an online database created using FileMaker Pro. Please feel free to send us questions based on this recommendation.

Step 3: Move Formative Studies to Professional HFE Heuristics Analysis: Today, we are working with our clients to help them replace lab-based formative research with professional heuristics analysis, a service for which we are well known. Professional heuristics analysis involves highly experienced human factors engineering (HFE) experts auditing a physical product, website, system, etc., for usability issues based on a validated HFE audit structure. Early and aggressive use of professional HFE heuristics analysis can identify HFE problems prior to user testing, which can significantly reduce HFE formative testing costs. We are also helping our colleagues and clients reconfigure lab-based user testing studies already in the pipeline based on the application of our SOPs for risk mitigation. This work includes collaboration with our IRB sites and testing facility vendors, who are also undertaking major updates of their SOPs. There are many moving parts to proceeding with lab-based HFE research in the age of COVID-19. Please feel free to send us questions based on this recommendation.

Step 4: Rethink How HFE Science Is Integrated Into Development Programs: Our clients tell us that this is a time to analyze how studies are executed and to determine if other methods, such as HFE heuristics and/or robust online data capture, can supplant early formative studies. It is clear, based on the execution of hundreds of usability testing studies over the past 30 years, that MANY usability problems with products, websites, systems, etc. uncovered in formative studies could have been identified and eliminated through early and aggressive use of HFE heuristics analysis. In a surprising way, a benefit of this very difficult time may be an improvement in how HFE science is utilized to improve usability AND reduce development costs. Please feel free to send us questions based on this recommendation.

Be safe and secure. We are all hoping for the best, but planning for less.

Charles L. Mauro CHFP
Cmauro@MauroUsabilityScience.Com

Professor Burstein Says The Sky is Not Falling…Really, Where is The Proof?

Charles Mauro CHFP

In her recent Patently-O article, The Sky is Not Falling, Professor Burstein goes to great length to argue that Columbia Sportswear v. Seirus is nothing to be concerned about, despite the vigorous protestation of many amicus briefs. However, it seems the esteemed professor has actually proven the reverse or at least has demonstrated a profound lack of self-awareness in terms of her conflicting prior opinions and inaccuracies.

DESIGN As A Corporate Asset Is About To Become Essentially Worthless. Yes, Lawyers Are Involved.

Charles Mauro CHFP

Designers ply their craft improving the aesthetic appeal and, to some extent, the functional performance of all manner of products ranging from automobiles to handbags to entire cities. Design is big business, with several million designers working worldwide in a staggering number of industry verticals. The impact that DESIGN has on business performance has been well-understood since the 1930s when Raymond Loewy worked his magic on the original Gestetner Duplication Machine and sales increased several hundred percent. Historically, DESIGN was understood to be of value in improving customer appeal, but not a primary factor in corporate performance. However, in recent years, DESIGN as a primary means of improving customer engagement has also been linked to the overall success of businesses who employ such expertise aggressively. Much of the support for increased impact on business success has come from less than rigorous studies and opinion-based research. But let’s be clear, aside from sketchy research, DESIGN is now perceived as having a major impact on corporate business success. There are designers in the C-Suite, and for truly talented industrial designers, entry-level salary now exceeds that of engineers with degrees in computer science. DESIGN is on a roll…but not so fast. The party is about to end in ways no one could have imagined. Enter the lawyers.

The Human Factors Science Behind The 737MAX MCAS Problem and Why The MAX Should Be Scrapped

Charles Mauro CHFP

The Spin There has been a great deal of back and forth over the reasons two Boeing 737 MAX airliners took the lives of 346 innocent souls. At the core of the problem is the classic duality of man vs. machine. Were the crashes the result of incompetent pilots and failure to follow procedures (The Man) or was the root cause a gross failure on the part of Boeing to employ rigorous and professional human factors analysis, design and testing in the development of the 737 MAX (The Machine)? In the beginning, the Boeing mediated media spin pushed the needle in the direction of human error, focusing on poor training and pilot inexperience. But soon enough, it became clear based on the excellent reporting of The New York Times that apparently, there was a lot for Boeing to account for both structurally and conceptually with respect to the design of the 737 MAX.

FAA Ignores Human Factors Science in Tests Related to New Reduced Size Seats

Charles Mauro CHFP

The long fight between the airline industry/FAA juggernaut and passengers regarding the Incredible Shrinking Seat Problem (ISSP) is being explored in a new study run by the FAA that apparently does not follow the rigorous science behind human body size variation (physical anthropometry) and study design. It turns out that there is a robust and complex science behind human body size and seat design which the FAA and airline industry has ignored for decades. The selection of a representative sample of passengers is at the core of reliable and scientifically valid human factors testing. This process involves screening and selection of passengers at the extreme ends of the user population based on percentile distribution analysis. Such individuals must be screened for physical and cognitive limitations in addition to physical body dimensions.

Mauro Usability Science Introduces First Neuroscience-Based Empirical Ordinary Observer Testing Methodology at IPO Annual Meeting

Charles Mauro CHFP

Mauro Usability Science (MUS) has developed and scientifically validated a system of advanced neuroscience-based consumer testing methodologies for applying the Ordinary Observer Test (OOT) in design patent infringement cases. Founder and President of MUS, Charles Mauro CHFP, recently introduced the system at the Intellectual Property Owners Association (IPO) Annual Meeting in Washington, D.C. on September 26, 2019.

To develop the system, MUS decomposed the legally mandated ordinary observer test (OOT) into its core components and designed individual testing methodologies for assessing a patented and accused infringing design according to each OOT core component. The system is designed to meet the 7-point federal guidelines related to the use of consumer research in patent litigation. The system, which was recently validated through a large-sample consumer study of 900 respondents, integrates data from high-performance eye-tracking, micro-facial expression analysis, advanced statistical analysis and robust online surveys using the iMotions platform.

The new system provides the finder of fact with objective and scientifically valid data related to critical legal questions surrounding the ordinary observer test to better inform the finder of fact’s design patent infringement determination. The system may also be adapted for copyright, trade dress, and utility patent cases. For more information direct message through LinkedIn or email Cmauro@MauroUsabilityScience.Com

What WeWork Got Right!

Charles Mauro CHFP

A PROFOUNDLY GOOD IDEA: In light of the recent Chernobyl-scale meltdown of WeWork, mostly related to lack of adult supervision and an S1 that read like Alice in Wonderland, it is important to note specifically what WeWork actually got right with the basic concept of UX-optimized shared office space. The initial vision, developed for the most part by co-founder Miguel McKelvey, could have changed the way small businesses entered into and expanded in the global economy. This is not to say that WeWork executed Mr. McKelvey’s initial vision correctly, but one should not forget that there was valid thinking behind the WeWork framework that sprang from real market conditions and the needs of small business owners. It is tragic to see a business solving real problems destroyed by the staggering malfeasance of its founder, VC investors, board members, lawyers, investment bankers and just about every other professional discipline that touched the management of the WeWork business model.

A Critical Analysis of FDA Guidance for User Percentile Device Design Criteria Versus Currently Available Human Factors Engineering Data Sources and Industry Best Practices

Charles Mauro CHFP

A staggering report was recently released by the International Consortium of Investigative Journalists (ICIJ) indicating that 83,000 deaths and 1.7 million injuries were linked to medical device adverse events reported over the last decade. Furthermore, a 2013 report by McKinsey & Company suggested that such adverse events and associated quality issues cost the medical device industry between $2.5 billion and $5 billion per year on average. There is also indication that some aspects of both medical device human factors engineering (HFE) design and Food and Drug Administration (FDA) HFE validation criteria may not provide a proper methodology for selecting and testing patient populations. This may lead medical devices that pass FDA HFE validation to still exhibit significant use errors when utilized by actual patients in the real world.

Feature Article – Down to a Science: Charles Mauro’s Expertise in Neuroscience-Based Design Research Dates Back to His ArtCenter Education

Charles Mauro CHFP

An exceptional piece was just published in the ArtCenter DOT Magazine exploring the professional journey of Charles L. Mauro CHFP, our founder and president, from ArtCenter College of Design to the development of award-winning solutions for world-class clients utilizing advanced neuroscience-based methodologies.

Mauro’s experience in Russia led him to fulfilling a lifelong vision: starting a firm that combined robust science on human factors performance with the rigorous design expertise he learned at ArtCenter. He launched Mauro Usability Science in 1975 and quickly discovered there were three main industries interested in his firm’s expertise—the U.S. military, medical technology, and Wall Street. The latter brought Mauro one of his firm’s defining projects.

Read Article

Understanding Costs And Risks For Human Factors Engineering (HFE) Usability Studies – Part 2: Outsourcing HFE Usability Testing

Charles Mauro CHFP

MedDeviceOnline-Author-Image — *Image courtesy of MedDeviceOnline*

Many of you may have read Part 1 in our series on Understanding Costs And Risks For HFE Usability Studies, which discussed the factors that device development teams should consider when executing HFE studies internally. We recently published the second article in this series, which discusses how to optimize data quality while controlling for study costs and time constraints when employing an independent HFE testing agency.

The goal of a device development team should be to obtain the highest quality data possible, given budgetary and time constraints. High-quality data is a non-negotiable requirement for HFE testing because device development teams need it to make critical design decisions or even consider submission for FDA review and approval. Thus, teams must evaluate whether a given HFE testing agency will deliver high-quality data at the price proffered in a formal bid. Such evaluation involves two questions: whether the independent agency is capable of delivering quality data, and whether the agency’s proposed study design is based on a robust understanding of HFE best practices.

Read Full Article

Please share this article with any of your colleagues who may find it of interest, and feel free to reach out to us directly by email with any follow up questions.