go home

Select a Professional Development Module:
Key Topics Strategy Scenario Case Study References

Introduction  |  Step 1  |  Step 2  |  Step 3

Step 1. Conceptualize an instrument triangulation strategy.

Substep 1a. Identify groups from whom you need to obtain information.

You decide that you will need to collect data either from or about the following three populations of participants: (1) the workshop facilitators, (2) the high school teachers who took the workshop, and (3) the students of those teachers who took the workshop.

Substep 1b. Search for archives in which you can find information.

For student background data, you learn that the National Center for Education Statistics displays summary student demographic data on schools throughout the United States . This information is displayed on its Web site called Common Core of Data (http://www.nces.ed.gov/ccd/index.asp). You want to use these data to get an overall representation of the ethnic, linguistic, and socioeconomic makeup of the schools at which the teachers teach. Later on, when you conduct data analysis, you will be able to explore possible relationships between these school profile statistics and the student outcome data you collect. The rationale is to determine if demographic characteristics have an influence on the outcomes.

You also investigate whether data about the teachers' professional histories are available, but discover that schools keep this data confidential. The data are therefore not available to you. Hence, you decide to request this information from the teachers directly, using a questionnaire. The state administers a standardized achievement test in science that covers earth science topics. The state's science standards stipulate that students need to know these topics by the time the state test is administered. You decide to collect and analyze the test scores for student outcomes.

Substep 1c. Identify the types of instruments you will need for collecting new data.

You decide to collect questionnaire, interview, observational, and assessment data. Table 9 identifies the instruments you choose.

Table 9. Types of instruments for collecting the new data

Instrument type and focus Source of information
Questionnaires about the workshop Workshop facilitators and participating teachers
Questionnaire about classroom experience Participating teachers
Questionnaire about classroom experience Students of the participating teachers
Protocol for observing classrooms Teachers and students
Protocol for interviews about classroom experience Participating teachers
Rubric for scoring the quality of the teachers' units Participating teachers
Learning assessments Students of participating teachers

Substep 1d. Decide how you will triangulate to take optimal advantage of your instruments, and produce an instrument triangulation table.

As Table 10 shows, you may your instrument triangulation strategy on a table that broadly identifies what information you need and how you will triangulate your information sources and instruments.

Table 10. Instruments triangulation table

Evaluation question Information needed Instrument triangulation strategy
Has the workshop been implemented as planned? Facilitators' plans for the workshop Workshop facilitator questionnaire
Teachers' perceptions of the workshop's effectiveness Teacher participant questionnaire to be administered immediately after the workshop
Are teachers who have particpated in the workshop producing curriculum units that maintain fidelity to the approach? Quality of the instructional units in terms of fidelity to the approach Rubric that the workshop facilitators will use to rate the quality of the teachers' units according to the fidelity of the units' to the instructional approach
Peer review of each unit by other teacher workshop participants
Are the teachers maintaining fidelity to the instructional approach as they implement their new units? Changes in teacher instructional practices following participation in the workshop Teacher participant questionnaire to be administered in the middle and at the end of their implementation of the curriculum unit
Teacher interview protocol to be used with participating teachers in the middle of implementation and at the end of it.
Observation of teacher skill level with the new instructional methods, in their classrooms, during the middle of their implementation of the curriculum unit
Is there evidence of a relationship between the teachers' workshop participation and their students' earth science achievements? Student learning outcomes Relevant student standardized test scors
Student assessments specifically selected for the evaluation