Skip to main content

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Usability Test Quality Heuristics

The following indicators can help determine if a usability test will produce useful results.
This is not an exhaustive list, but it should be helpful in planning and assessing usability tests. Additions are welcome!

Study design

Indicator	Good sign	Bad sign
Study purpose	Researcher clearly articulates the purpose of the usability test (for example, as a specific question to answer, an area of inquiry, etc.).	Researcher does not specify a purpose for the test, or the purpose specified is very broad, like “testing the app” or “finding problems.”
Number of participants	Tests include a sufficient number of participants necessary to see patterns, and there’s only one participant per session.	Study includes only one participant, or includes more that two participants testing simultaneously (like a focus group).
Sampling bias	Participants are a diverse set of people who actually use the application (or do the task the application is supposed to support).	Participants are experts (vs. average users), friends, family, or close colleagues of the product team.

Moderator style

Indicator	Good sign	Bad sign
Question style	Researcher asks follow up questions that are open-ended and non-leading.	Questions are leading or subtly suggest potential solutions or hypotheses.
Task variability	Researcher asks participants to complete similar tasks related to the study’s purpose.	Researcher asks participants to complete different tasks, or tasks unrelated to the study’s purpose.
Framing	Researcher asks participants to complete tasks with the product or service that align with their work-related goals.	Researcher asks participants to complete tasks unrelated to their work-related goals. For example, asking a participant how they might send a fax when their job doesn’t call for that.
Priming	Moderator asks participants to complete tasks without indicating how that might be done. For example, “How would you view the status of your application?”	Researcher guides participants in completing tasks. For example, “Which of the links in the header would you click to login?”

Team participation

Indicator	Good sign	Bad sign
Number of teammates	The team designates a single moderator for the test, and at least one member of the product team observes the usability test.	A single person from the product team participates in and leads the test.
Observer role	Observers do not moderate. They are generally quiet, and ask open-ended questions after the test has concluded.	Observers interrupt the participant, or attempt to sell or explain the product. Observers debate the participant’s actions in their presence.

Sensemaking

Indicator	Good sign	Bad sign
Notetaking	Tests are recorded or notes are taken for sharing with absent stakeholders.	Tests are not recorded, or test results are not documented.
Synthesis	Moderator debriefs with teammates immediately after each interview. Researcher looks for patterns across multiple participants and surfaces problems that affected several people.	Moderator reports the most memorable problems without conducting affinity mapping or some other analysis technique.
Presentation of findings	Researcher reports findings to team and stakeholders in an easy to follow, well prioritized way.	Researcher presents the team a “basket of issues” or an unprioritized laundry list of potential changes.
Incorporation of findings	Product team translates findings into future user stories or product refinements.	Researcher reports do not affect the product backlog or ongoing development work.