MST-READI

(Medical Simulation TRaining TEchnology EvAluation DesIgner)

MST-READI recommends the use of a ‘true’ experimental design whenever possible so that you have the best chance to determine causality, i.e., to determine that the technology being evaluated is the cause of the outcomes being assessed.

The pre-post test control group design is commonly used to answer questions addressed in summative evaluation research. In this design all participants take a pretest, then a subset of participants receive the new training (experimental group) while the remainder receive traditional training (control group), and finally all participants take a post test. In general, pretest-posttest designs are the preferred method to compare participant groups and measure the degree of change occurring as a result of training.  Control groups are intended to provide a group against which to compare the experimental group to see examine changes related to the training.  

Regardless of the specific design you choose, conducting an experiment will typically follow the sequence of events below.

1. Determine Approach –As mentioned earlier, a true experiment is recommended, however may not always be possible. The evaluation team will gather necessary information to determine a true experiment is feasible.

2. Develop Protocol - For any design, an experimental plan, or protocol, should be developed defining important components of the research and data collection procedures.

3. Submit IRB - Once the protocol is finalized, it should be submitted to IRB if review and approval is required.

4. Logistics Planning - While the protocol is being reviewed, the evaluation team should work with the facility to support the logistics of the implementing experimental procedures.

5. IRB Approval - Data collection cannot begin until IRB approval is obtained.

6. Pilot Study - Data collection is time and resource intensive; therefore, once preliminary activities are complete and the protocol is approved by IRB, a pilot study should be conducted.

7. Tweaks to Protocol - The goal of the pilot study is to test experimental materials and procedures and make recommendations for tweaks or modifications to facilitate effective, efficient, and valid data collection.

8. Conduct Experiment - After tweaks to the protocol are made, the experimental data collection should be scheduled and may begin.

9. Analyze Data - Once all data is collected it will be analyzed

10. Report Results - Research results will be documented for presentation / reporting

       MST-READI is a collaborative research effort among US Army RDECOM-STTC, OSDi and CWS, funded by RDECOM-STTC