Thief 3 Usability Testing

Shannon Lucas and Denise Fulton present a brief summary of their usability testing project of Ion Storm's game Thief 3: Deadly Shadows in an article called "What We Learned Evaluating the Usability of a Game"

They made use of the "Rapid Iterative Testing and Evaluation" (RITE) methodology developed by Pagulayan et al. at the Microsoft Game User Research Group to guide designers and programmers during the early Alpha stages of the game.

Interesting observations are that while they didn't have time to review the videotaped footage, this was useful as a resource for programmers to observe their AI under real operating conditions. It doesn't sound like they made use of data visualisation or game metrics, but rather relied upon subjective evaluation from the players first-hand experience. For example, the conclusion that "8 out of 10 players had difficulty using the blackjack tool" appears to be based on player reports rather than measurements of the weapons use during gameplay.

"What We Learned Evaluating the Usability of a Game". Usability and User Experience Community of the Society for Technical Communication. October 2004 issue (Vol 11, No. 2).