Designing An Evaluation: Methodological Approach and Sampling
By Daniel R. Zalles, Ph.D.
This is the first of two modules about how to design evaluations
for projects that intervene to change existing processes, and may
be either in progress or close to completion. Many projects in education
fall into this category. This module explains how to articulate
a methodological approach, then carry out appropriate sampling procedures.
The second design module focuses on data collection procedures and
scheduling (under development).
The major focus of this module is on evaluations that involve examining
outcomes, although there is some attention to examining implementation
as well (the difference between outcome and implementation evaluations
is discussed in Step 2). Steps 1-5 pertain to determining the methodology
of the evaluation. Steps 6-10 address sampling design. It will help
you to have some knowledge of statistics and the nature of scientific
inquiry, but that is not a prerequisite.
The objectives of this module are for you to understand:
- How decisions about what data to collect and whom to collect
it from need to proceed out of the evaluation's purposes.
- How conclusiveness in an evaluation is a relative concept that
has to do with the strength of accumulated evidence.
- How decision-making about methodology and sampling involves
trade-offs and value judgments that should be brought to the attention
of the key stakeholders.