Step 2: How will implementation of the treatment be monitored?

You decide that implementation data and data on instruction will be collected in both intervention and comparison groups in each of the two years of the study, with more extensive efforts made to collect data on implementation challenges in year one. In addition, because the intervention is to be used as a supplement to regular instruction, you are also interested in monitoring whether students in the intervention group receive more time on math instruction than students in the comparison group. During the second year, you decide to continue to monitor the amount of instructional time in math and instructional practices used in the intervention and comparison groups. You also want to track implementation challenges. You base this design decision on the assumption that the teachers will have intensive challenges the first year and that by the second year implementation is likely to be more stable. Nevertheless, you anticipate continued implementation challenges for some teachers in the second year because they will have new students who may have different needs.

By systematically collecting these data in intervention and comparison classrooms, you will be able to better understand which aspects of implementation are proving to be most challenging and how variations in the ways teachers are implementing the program are potentially contributing to differences in the impacts on student learning in the intervention group. In addition, you will be able to examine which instructional practices may be associated with better student outcomes and whether increases in instructional time may be responsible for the differences in outcomes between the intervention and comparison group.