I made a web page that allows playtest participants to mark specific moments while they play.
Each time the participant presses one of those buttons, the page logs the timestamp of the button press. Later, during a retrospective think-aloud, the participant will review each moment with a moderator.
1. Download the page here (right-click, save as...)
Note that the logs will be erased as soon as you leave the page. I did not include an export function due to time constraints, so for now you'll have to copy/paste the text into an excel sheet or a text file.
I like using John Cutler's model to describe my intentions.
I made this to give researchers a cost-efficient way to collect data during their playtest.
Target: games user researchers
Current behavior: When researchers cannot afford to have moderators in the room to watch the playtest participants, they tend to rely on post-session surveys and/or post-session interviews to collect data. This approach is valid but has a weakness: playtest participants are more likely to give feedback on events that happened late in the session, and/or on the event that impacted them the most (peak-end rule).
This can lead to playtest results that cover only large-scale problems (ex: "people did not like the story") and omitt small, easier-to-fix problems (ex: "people felt that having to go around the hill every time they wanted to talk to the mission giver was tedious").
Intervention:to collect data on smaller problems without moderators, we need to give researchers a tool to collect data throughout the playtest session.
Outcome: The number of early/mid game issues being reported should increase, as well as the number of "small" issues. In addition, those issues could be discovered earlier in the game's development.
We collected more feedback on smaller problems and on the early parts of the game. This helped us a great deal during the last months of a project, when some of the feedback could not be addressed anymore due to production schedule (ex: when the story is under assets-lock).
This was especially useful to the level-design team: we gave them a steady flow of issues over 3 months, and their workflow allowed them to fix most of them before the release of the game.
Unexpected bonus: the quality of those issues was higher than usual: participants frequently gave a context while describing problems during the retrospective think-aloud (ex: "...and this happened while I was collecting this resource for 30min already, which made it worse"), which made it easier for researchers to uncover the causes of those problems.
Copyright © 2017 Jean-Luc Potte, All rights reserved