Step 8: Minimize the effects of other sources of bias besides sampling bias.

(R) = report example

There are other ways to bias an evaluation aside from faulty sampling strategy. The following are two common additional sources of bias (R) and what evaluators can do to avoid them:

The Hawthorne Effect. This happens when participants in the evaluation act differently than they would normally act because they know they are being observed. An example of this would be teachers trying harder to use a new instructional method because they know they are part of a study. One way to avoid the Hawthorne Effect is to make the non-intervention subjects feel the same way as the intervention subjects. Another way is to stretch out the time in which data are collected, on the assumption that self-consciousness about being a subject will dissipate over time.

Contamination. This occurs when members of the intervention sample are in contact with members of the non-intervention sample. The expectations of one can influence the expectations (and behavior) of the other. A good preventive strategy is to select members of your intervention and non-intervention groups such that they have as little exposure to each other as possible. For example, if your evaluation goal is to find out if teachers are teaching better with a new instructional approach, the risk of contamination would be reduced if the teachers were from different schools.

Both types of bias lessen the chance of finding an effect because they compromise the purity of the non-intervention group. If the biases cannot be eliminated, the evaluator can at least track their occurrence with implementation data gathered through questionnaires, interviews, and observations, and then triangulate it with the outcome data.