1. From what groups should the evaluator seek information?
2. How should the evaluator use instrument triangulation to investigate student engagement?
3. The evaluator decides to collect pre- and post-learning outcome data from the high school biology students in all four classes. How should the evaluator use instrument triangulation to investigate the student learning outcomes?
4.
All online biology students take standardized achievement tests in science. However, because the students come from four different states and each state administers its own standardized test, different students will be taking different standardized tests. Is it appropriate to compare the learning outcomes of the four online versions of the courses by comparing the results of the four standardized tests that the different students will be taking?
5. All four teachers of the virtual classes are at liberty to grade their students as they wish. Would it be appropriate to compare the grades across teachers as an outcome measure?
6. Does OERL contain learning assessments that would be useful and adaptable for the pre- and post-course assessments of learning outcomes in the four virtual classs?
7. Where else can the evaluator find potentially usable biology assessment items at the appropriate grade levels?
8. To try to explain differences in gain scores from pre- to post-course assessments among the different students enrolled in the four versions of the course, the evaluator wants to gather background information about each individual student. The evaluator is aware that archival demographic data are available on the Common Core of Data Web site of the National Center for Education Statistics (http://nces.ed.gov/ccd/bat/). Should the evaluator use these data for background information?
9. Assuming that the evaluator has found enough instruments from which to extract and adapt items for the implementation instruments and learning assessments, whom should the evaluator use as subjects for piloting the implementation instruments and learning assessments? How should the pilot be structured?
