Lab-based Testing
Questions this method answers: Effective in exploring high-level usability of products and services. Provides visualization of user interactions based on a limited number of tasks. Development team can view entire process from behind one-way mirror. Good for identifying a small number of critical interactions that are having a universal negative impact on the user experience. Not useful for making complex business-critical development decisions.
Statistical reliability: none/low; small sample size (12-60 respondents)
Functional scope: 2-3 tasks or interaction sequences
Fees: Moderate cost per subject
Lead time: 3-8 weeks for top line report
Geographic reach: All major world continents
Deliverable: Video tape summary + report/recommendations
Major advantage: Confidential and compelling demonstration of user interaction with your product or service
Disadvantages: Small sample size; mostly subjective data; does not allow detailed analysis of data for statistical reliability
Quality of service issues: Professional study design and data analysis is a critical aspect of achieving reliable, actionable insights. Quality of the reported findings varies widely by vendor. Inexperienced firms often attempt to test too many tasks or functions, leading to highly unreliable results. Methodology frequently used to study inappropriate variables.
