The university has completed the second year of a 4-year pilot implementation of "Learning by Doing," a pedagogical approach in which inquiry-related skills and strategies specific to a discipline are taught in parallel with its concepts and theories. Its primary advocate is the Dean of Academic Affairs, who adopted it at a different university in his prior tenure as a chemistry professor.

Learning by Doing requires that the students carry out an inquiry-based task throughout the length of the course. It can be a controlled experiment, a nonexperimental data collection and analysis effort, or a research proposal. That is up to the instructor.

The assumptions of change underlying Learning by Doing are that:

1. Students will become more interested in the subject matter when engaged in this approach.
2. Students will learn abstract scientific principles better if they are engaged from the start in applying the principles to a concrete, meaningful inquiry task.

Current Implementation of Learning by Doing

In the first 2 years, each of the eight departments got 10 instructors to participate in the pilot (out of a population of 20 to 22 instructors per department). Training was delivered to the participating instructors during the summer before the first year of the project. The goal was to build the instructors' capacities to apply and adapt the model to their own curricula and determine how to assess their students' outcomes. Now they are about to enter their third year.

Preliminary evidence gathered informally by the Dean has suggested that the wide latitude afforded to the instructors has led to large differences in how the intervention is being implemented. For example, some instructors follow the guidelines they learned in the training sessions meticulously, while others pick and choose among them.

Expectations

Now that the instructors have had 2 years of experience with the intervention, the university has hired a team to evaluate it. The Dean wants the evaluator to examine how much Learning by Doing is leading to increased learning of abstract scientific principles and increased interest in the course content. He wants to see if there are differences in impact of the intervention from department to department. Ultimately, he wants to use the results to decide which of the following actions to take:

a. Mandate Learning by Doing in all the introductory courses.
b. Mandate it for only certain departments.
c. Maintain it solely as a voluntary project.
d. Discontinue it.

The Dean expects that the intervention will have only a small effect on student learning and interest. However, he feels that even a small effect would justify continuing it, because it is relatively inexpensive to implement. Also, many faculty and administrators believe that its goals are important to address, even if they are difficult to achieve.

There is not enough money in the evaluation budget for collecting data from all the instructors and students participating in the project. Hence, the Dean requests that random samples of instructors and students be drawn.