MST-READI (Medical Simulation TRaining TEchnology EvAluation DesIgner) |
||||||||||||||||||
The Focus Group session involves: (1) observation of real-user performance followed by (2) group discussion and finally (3) a survey. Observation Participants should be presented with the training material (this may include the entire training content or may be limited to specific modules) individually or in small groups (depending upon the ratio of available simulations to participants) and asked to complete particular training tasks. Researchers have an opportunity to watch the user-simulation interaction and make observations about the usability of the system, accuracy of instructional content and instructional strategies. (Key differences between this stage in the evaluation and the pluralistic walkthrough are the more realistic pace of user interaction and the fact that developers are no longer participants.)
If direct interaction with the simulation based training system is not possible due to logistical constraints a video recording of the system being used could be used as a substitute. For instance, the following video demonstrates CLS training using Americas Army and could be shown to participants if the actual software were not available:
Group Discussion After becoming familiar with the medical simulation based training system, a group discussion will be conducted. This methodology is a qualitative approach commonly used in research where small groups of people are encouraged to disclose their attitudes and perceptions of a product through group interaction. In the following video recording of a group discussion session for a "Dream Bank" notice:
|
||||||||||||||||||
|
||||||||||||||||||
Survey Finally, each participant will be asked to complete a survey capturing self-reports of usability of the device and prediction of successful use in practice as a training system. Generic questions about a training simulation:
Simulation specific questions:
http://usability.gov/templates has example questionnaires including the System Usability Scale (SUS) from http://www.usabilitynet.org/trump/documents/Suschapt.doc. General considerations for forming well-formed questions are given by Walonick, D. (2004) Designing and Using Questionnaires in Survival Statistics. StatPac, Bloomington, MN. Available at: http://www.statpac.com/surveys/ |
||||||||||||||||||
The MST-READI methodology includes procedures for conducting a Focus Group session. References http://grouplab.cpsc.ucalgary.ca/saul/hci_topics/tcsd-book/chap-1_v-1.html Task-Centered User Interface Design: A Practical Introduction, by Clayton Lewis and John Rieman. Ericsson, K., & Simon, H. (May 1980). "Verbal reports as data". Psychological Review 87 (3): 215–251. doi:10.1037/0033-295X.87.3.215. |
||||||||||||||||||
MST-READI is a collaborative research effort among US Army RDECOM-STTC, OSDi and CWS, funded by RDECOM-STTC
|