The consultants might consider adding some higher-inference rating items to the end of the "topic" form. For example, they could ask the observer to rate the teacher’s lesson on several dimensions that describe the range from computation-only math instruction to math instruction that is inclusive of many conceptual strategies (e.g., the extent to which teachers and students treat a math problem as having one path to a correct answer vs. generating multiple strategies to solve the problem). Although it would take additional training time to get observers to reach agreement (and, thus, interrater reliability) on the meaning of these higher-inference rating items, the ratings would have the advantage of yielding immediate quantitative data that could be summarized and compared across teachers and over time. Thus, the quantitative data would supplement the greater body of qualitative data.

Close Window