The evaluators should train their staff in using the observation protocol. The staff should practice using the protocol in a small set of live field observations of student teams in the field or by watching videotapes of the teams. The evaluators should strive for no less than 80% inter-rater reliability. The evaluators should also train their staff in using interview protocols for students and instructors. In mock interviews, the staff could role-play being interviewers and interviewees and the interviews could be timed to ensure that the staff will be adept at staying within allotted time limits.

Close Window