How Many Users Are Enough for a Card-Sorting Study?

Abstract

study was conducted to assess the minimum number of participants needed for a card-sorting study. Similarity matrices and tree structures from various sample sizes were compared to those based on a set of 168 participants. Results indicate that reasonable structures are obtained from 20-30 participants.

Introduction

Card-sorting, either online or with actual cards, has become a very popular technique for helping to organize the elements of an information system in a way that makes sense to users of that system. It has become a standard tool in the toolbox for most usability professionals and information architects. Card-sorting has been used for designing mainframe menu systems (Tullis, 1985) and, more recently, for designing web sites (e.g., Frederickson-Mele, 1997; Tullis, 2003). A variety of computer-based tools are now available for conducting card-sorting exercises online and/or analyzing the data from manual or online card-sorting studies (e.g., EZSort, WebCAT, WebSort, Socratic CardSort, Classified, CardZort). However, one of the questions that has not been answered is how many users need to do the card-sorting exercise to get an accurate picture of how the information should be organized. The purpose of this study was to answer that question.