Alignment Table for Report
Components
Analysis Process
The alignment table for sound project evaluation reports
can be viewed either as a whole, displaying all components,
or as six separate tables corresponding to report
components: (1) Executive Summary, (2) Project Description,
(3) Evaluation Overview, (4) Design, (5) Analysis Process,
and (6) Results & Recommendations. See the
alignment table overview for a
general description of what appears in the alignment
tables.
The glossary and
quality criteria entries for
report components are also available on their own.
Component |
Glossary Entry |
Quality Criteria |
Related Program Evaluation Standards |
Analysis Process
|
Describes the type or types of analyses conducted
(e.g.,
quantitative,
qualitative, mixed methods) and
procedures used for examining
results and ensuring
their trustworthiness, such as:
- training conducted to ensure
reliable coding and
scoring of data
- checks of the data to remove errors
- procedures for reducing and summarizing the
data
- descriptions of analyses, that identify a pattern
of results
This section also describes results
non-interpretively (e.g., without being subject to
values, perspectives, and conceptual frameworks).
|
|
|
|
Describes procedures taken to analyze numeric
data:
- organizing the data
- verifying it
- summarizing it
- presenting purely descriptive information about
the project (e.g., percentages of different
responses to a survey question; percentages of
different scores on a test item) that could lead to
patterns and trends
- examining relationships among variables (e.g.,
Pearson Product Moment correlations, multiple
regression, factor analyses)
- using inferential statistical techniques to test
for significant differences between comparison
groups (e.g., t-tests, analyses of variance,
analyses of covariance)
|
The
quantitative analysis procedures should be
appropriate to the evaluation questions being
addressed and the characteristics of the information
being analyzed. The practical significance (e.g.,
effect sizes) and replicability, as well as
statistical significance, should be considered when
drawing inferences and formulating
conclusions from
quantitative analyses. Analyses of effects for
identifiable subgroups should be considered, as
appropriate, because a program may have differential
effects for them.
In addition, the number of informants who actually
provided data should be reported. (Informants who fill
out a survey are called "respondents," and
the percent of those solicited who actually respond is
called the "response rate." This will help
reviewers determine the extent to which the informants
are representative of the total population.
Potential weaknesses in the quantitative data
analysis, along with their possible influence on
interpretations and conclusions, should be
described.
|
A8 Analysis of
Quantitative Information
Quantitative information in an evaluation
should be appropriately and systematically analyzed so
that evaluation questions are effectively
answered.
A7 Systematic Information
The information collected, processed, and reported in
an evaluation should be systematically reviewed, and
any errors found should be corrected.
|
|
Describes the
qualitative analysis procedures used to
compile, analyze, and interpret the data in order to
find themes, patterns, and trends.
|
The
qualitative analysis procedures should be
appropriate to the evaluation questions being
addressed and the characteristics of the information
being analyzed. As the evaluation progresses, the
accuracy of findings from qualitative data must be
confirmed by gathering evidence from more than one
source and by subjecting inferences to independent
verification.
Potential weaknesses in the qualitative data
analysis, along with their possible influence on
interpretations and
conclusions, should be
described.
|
A9 Analysis of
Qualitative Information
Qualitative information in an evaluation
should be appropriately and systematically analyzed so
that evaluation questions are effectively
answered.
A7 Systematic Information
The information collected, processed, and reported in
an evaluation should be systematically reviewed, and
any errors found should be corrected.
|
|
|