Step 8. Administer the procedures identified in Step 6.

Procedures that make training for data collectors, instrument administrators, and coders and raters effective resemble in many ways those you would use in other learning situations:

1. Schedule the sessions at convenient times. Without this procedure, trainees may not be able to attend, may need to arrive late or depart early, may be more easily distracted, or may be resentful.
   
2. Vary the agenda of the training to meet the different skill levels of the trainees yet raise all trainees to the level of skill required. Trainees may have various degrees of experience or skill. If so, too much uniformity in the agenda may make the more experienced feel they have "heard it all before" and may make the less experienced feel overwhelmed. Assigning trainees to proficiency-based sessions would help.
   
3. Employ hands-on exercises in the training that trainees can use to practice their skills before they perform them in the field. Give the trainees opportunities to practice their skills. Then, before you conclude that they have received enough training, assess their proficiency.

For example, when you are training raters and coders, a good way to begin the training process is to have the trainees examine a sample of artifacts that have already been rated or coded by experts. As much as possible, the artifacts selected for this exercise should capture the full range of codes and ratings that are likely to be applied to the full data set. As a culminating exercise, trainees should independently rate or code a set of additional artifacts that also have been prerated or coded by experts. Lack of agreement with the experts will signify that the trainees are not yet proficient enough in the application of the criteria and need more training. If lack of agreement happens repeatedly, the fault may lie less in trainee inexperience and more in defects in the interpretative criteria. Such defects may be the product of insufficient piloting of the instrument (see the Instrument Triangulation and Selection module for more on piloting).

When you are training observers of targeted settings such as classrooms, a comparable way to begin the training process would be to have the trainees watch a videotape of the type of setting they will be asked to observe. The trainees can individually practice using the protocol on observing the videotaped setting, then come together to compare their notes.

When you are training interviewers, the trainees can practice using the interview protocol by role-playing the interviewer and the interviewee, then giving each other feedback about how well they handled their roles. The trainer can help make this a more effective exercise by giving the interviewee a certain attitude or character profile to adhere to when responding to the interviewer's questions.

When you are training data collectors to go out into the field, training does not have to end with the training event. If you have both experienced and inexperienced data collectors, it makes sense to send them out together. This ensures that the inexperienced data collectors are mentored. It also ensures that you are not entrusting too much responsibility to the less experienced ones.