Observation 1. Because the purpose of this interview is to compare student experiences in using a systematic intervention that is well-understood, a fairly structured interview is needed. The nature of proposed questions is appropriate for this level of structure.

Observation 2. A quick overview of the questions indicates that some of them focus on basic information about use while others solicit opinions about program quality. This seems generally appropriate to the purpose of the interview. Examining the wording of the questions, you notice that several begin with auxiliary verbs (questions 3, 5, 7, 9 and 10). You decide you can word most of these items more effectively. In addition, you recognize that question 6 has a "built-in" response (the "How X was Y " structure) that can be fixed. You also notice that questions 3, 4, and 6 are phrased with a positive bias (or without a balanced counterpart). Thus, you plan to balance these questions.

Observation 3. It is difficult to know in advance whether you will have to probe for clarification, but you do not anticipate comprehension difficulties with this straightforward topic. Depending on the interviewee's willingness to talk, you may have to probe for greater depth on some questions if you do not get sufficient detail about the student's use and opinion of Homework Help. You decide to note a couple general probes at the bottom of the protocol in case you need them.

Observation 4. You notice that some questions seem out of order. The first two questions elicit basic time use information about Homework Help and can stay where they are. Questions 5, 6, and 7 concern certain features of Homework Help. They probably belong together after the time use questions. Questions 3, 4, and 9 elicit user opinions. The questions probably will generate the most complex responses, and it makes sense to ask them after the interviewee has warmed up with questions about time use and features. Question 8 is the only background question. It can remain toward the end of the interview. Question 10 (about suggestions for improvement) is a good concluding question, although you decide you should follow this up by asking interviewees whether there is anything else they want to comment on.

Observation 5. You were given a set of questions without any script for the interviewer. Because you want the interview to be administered consistently (especially if there is more than one interviewer), you decide to write a short introduction and conclusion. You also decide to add transition statements based on your reorganization of questions.

Observation 6. Even allowing for the fact that you may expand some questions or break them into parts, the number of questions seems fine for an interview limited to 1 hour. The questions about time use and features should elicit relatively short answers, allowing at least 5 minutes each for the questions that may elicit lengthier responses.

Observation 7. Because this interview will be used with small groups of students from four classes, a modest pilot test seems called for. You decide that after you make your edits, you will show the protocol to two members of the math department who are familiar with Homework Help, asking for their comments. Then you can line up two math students who are familiar with computers (but not Homework Help specifically) and ask them to talk their way through the interview with you. After making any necessary revisions, you plan to administer the interview again to four similar students, this time trying to approximate administration conditions closely to make sure the interview can be completed easily in the allotted time.