Step 7: Pilot-test the interview protocol and make needed revisions.

Once your interview is developed, pilot-test it with people not participating in the evaluation. Pilot testers can provide valuable feedback about how to improve the wording of the interview protocol. Also, the interviewer may recognize the need for revisions in the structure and appearance of the interview protocol as he or she works through it in a true interview situation. Although there are no set rules for how many people should pilot an interview, a reasonable pilot-test process for an average-size evaluation is as follows:

1. Editing. Have several individuals (e.g., three to six) familiar with the evaluation (but not involved in the interview development) read through the interview protocol and make suggestions. Make revisions.


2. Early Pilot. Give the interview separately to two individuals similar to your target respondents. Tell each individual that the purpose of the pilot is to be sure that the questions are easily understood and ordered sensibly. Ask each individual to respond to each question as if he or she were in the evaluation and also to comment on the clarity of the questions. Record responses using the method(s) intended for the evaluation. At the end of the interview, ask each individual for feedback on the entire experience of taking the interview. Make revisions according to the respondents' feedback and your own experience of using the protocol and recording responses.


3. Full Pilot. Give the interview again to a slightly larger group of individuals (e.g., three or four) similar to your target respondents. Again, try to approximate the administration conditions you plan to use for the interview's full implementation. This time, ask the individuals to go through the entire interview with you and, except in cases where they don't understand a question, save their feedback until the end of the interview. As the interviewer, focus on the amount of time it takes to address each question and complete the entire interview. Make revisions if there are significant issues with wording or timing.

For larger, higher-stakes evaluations, testing with groups that include more individuals and more iterations is advised.