How do you design tree-testing tasks?

ProcrustesPR

When designing tree-testing tasks, first choose the content you want to test, then ask participants to find it within a hierarchy of labeled categories. This helps you understand how easy that content is to find. Task wording should explain the need without revealing the correct category, and the correct location must be defined in advance for each task. Before collecting quantitative data, run a pilot to check that the wording is clear and participants understand it as intended. When analyzing results, look at paths and backtracking alongside success rates—not just the number of correct answers.

ProcrustesPRProcrustesPRDoes my website need a speed score of 100?

Decide what to test before choosing resources

Tree testing evaluates an existing or proposed navigation hierarchy; it does not ask participants to group items themselves. Once the full set of categories and labels is ready, you can test it with tasks without first completing the layout, visual design, or content. Choose tasks based on the following goals:

  • Key resources: Important products, services, or information related to business goals.
  • Suspected problems: Newly proposed categories or areas where participants disagreed during card-sorting research.
  • Labels and locations: Comparing alternative names or placements for the same category.

Infrequently used resources may also be worth testing. An NN/g author asked participants to find a company’s charitable donation matching program to assess whether HR-related category labels provided sufficient clues. What matters is not how popular the resource is, but which categorization issue it can help test.

Describe the need without putting category labels in the task

Each task should ask participants to find content within a category, rather than explicitly asking them to select the category. Take a business-startup scenario: if the category is called “Starting a business,” a task that says “Find information about starting a business” gives away the answer. “You are considering running a lawn care service. Find resources that could help you get started” is better suited to testing how users understand the categories.

Scenarios need only include the motivation and necessary conditions. Leave out irrelevant details, such as when or where someone moved, so the key need does not get buried in a long story. Specify the correct location in the tree for each task in advance. If a resource has multiple reasonable locations, you can define multiple correct answers.

Run moderated pilot sessions to check that tasks are not misleading

Before collecting data at scale, run a few pilot sessions with a moderator who can check that participants understand the tasks and ask why they made their choices. Small-scale qualitative research is useful for identifying confusion, but not for drawing statistical conclusions about success rates or task completion times.

Testing without a moderator does not allow real-time follow-up questions. After the test, you can list the category labels and ask participants to select any they found difficult to understand, then gather feedback through open-ended questions. This can help explain some of the reasons behind their behavior, but cannot replace follow-up questioning.

Look at answers and paths together

When reviewing results, look beyond success rates and time spent. Check whether participants reached the correct location directly, which category they clicked first, and where they ended up. When success rates are low or backtracking is frequent, first clicks can reveal where users initially expected to find the resource. If their first step was correct but they still ended up in the wrong place, it is worth checking whether lower-level categories overlap.

Tree testing deliberately excludes search, visual cues, and content cues, so its results cannot be treated as a direct measure of the full website’s performance. A well-executed final design may improve success rates, while poorly designed search or menus may make performance worse.

ProcrustesPRProcrustesPREight SEO essentials for beginners: Website speed

Designing tree-testing tasks: perspectives from across the web

Should you start with a warm-up task?

You can begin with a simple task to familiarize participants with the testing procedure before moving on to the tasks you want to study. In unmoderated testing, an incorrect answer to a warm-up task may also suggest that a participant is not paying attention, but a single wrong answer is not enough to determine the cause. You should still consider whether the task itself is easy to understand.

How should you interpret results when the same resource appears in multiple categories?

If the tree genuinely contains multiple correct locations, you can mark all of them as correct answers, then examine which final destination each participant chose. Tools typically report an overall success rate, so looking only at that total may hide differences between locations. If a particular correct location is rarely selected, you can generally consider removing that cross-listing.

Are tree-testing results still useful when a website uses mega menus?

Yes, but keep the difference in presentation in mind. Mega menus can show multiple category levels at once, giving users an opportunity to compare options, so the full design may achieve a higher success rate than the tree test—provided users enter the correct menu first. During analysis, pay particular attention to first clicks to see whether participants found the right entry point.

ProcrustesPRWhat Ten Years of SEO Taught Me About Its Real Purpose

References

  1. Tree Testing: Evaluate Menu Labels and Categorieswww.nngroup.com · 2023-08-06T18:00:00+0000 — Discusses: Should you start with a warm-up task?; Are tree-testing results still useful when a website uses mega menus?
  2. Tree Testing Part 2: Interpreting the Resultswww.nngroup.com · 2024-01-19T17:00:00+0000 — Discusses: How should you interpret results when the same resource appears in multiple categories?; Are tree-testing results still useful when a website uses mega menus?

Related articles

Does my website need a speed score of 100?Eight SEO essentials for beginners: Website speedWhat Ten Years of SEO Taught Me About Its Real PurposeHow Do RWD and AWD Work, and Which Should You Choose for Your Website?What are the benefits and development challenges of SPAs?How do UTF-8 and Big5 differ in their effects on search rankings and user experience?What are single-page websites best suited for, and what are their pros and cons?Keep different topics on one site or split them up? Start with audience overlap and maintenance capacity
ProcrustesPR
ProcrustesPR