Laboratory Improvement Annotated Plan Excerpts

Excerpt 1 [Anonymous]

Data Collection Procedures & Schedules:
Describes collection schedule

Information Sources & Sampling:
Describes multiple sources

Instruments:
Specifies two questionnaires and archival data

Evaluation will occur at various times throughout and at the end of the grant period. Since the first 6 months of the grant period will be largely devoted to purchase, set-up, and software development, the initial evaluation, in August of 2001, will simply draw upon brief written reports from the PIs and a 2-hour meeting of the PIs with the evaluation committee, allowing clarification of the development plan for the committee and assessment of progress relative to the development plan timeline. Since teaching activities utilizing the new equipment are expected to commence in the fall of 2001, evaluations will occur at the end of each semester for the remaining two years of the grant period. Each of the PIs will be responsible for seeking responses from the various participant groups with whom he works: on-campus students enrolled in one of the three main courses affected by the project, on-campus students taking other courses affected by the project, student assistants for the project, off-campus pre-college students, high school students in classes who are conducting one of the online experiments, teachers of these high school students. Students will fill out the standard course evaluation forms used for all courses at the college. They will also respond to questions pertaining to the experiments supported by this proposal. Since our intent is for these experiments to be experiences in scientific inquiry, we have based the questions for high school and beginning college students on the aspects of the National Science Education Standards pertaining to inquiry (National Research Council, 2000). These evaluations will be submitted to the evaluation committee. The evaluation committee will also obtain information on participating students from our Registrar's office regarding subsequent science courses taken, whether or not a major in science is declared, whether science teaching careers are planned, and the degree to which upper level science student participants undertake senior theses related to this project.

Excerpt 2 [University of Redlands]

Methodological Approach

Specifies using control group

B. Drafting tools for formative assessment. Quantitative, skills-based testing will be crucial to the evaluation plan and will be used to monitor progress towards each of the three goals. This skills-based test will meet each of the following requirements. First, it will be administered on both a pre- and post- course basis to allow for measurement of improvement for each of the three goals. Second, scoring of the test should be objective. Therefore the exam will be primarily multiple choice. Third, because the test will be given on a pre-course basis, neither the pre-course nor the post-course test will be part of the students' grades in the course. This also has the advantage that supplemental studying on the part of the students will not cloud the meaning of the results. Fourth, the test will be given to a control group so as to measure the improvement, if any, in the tested areas without the student experiencing the impact of the project.

Instruments

We will work with the outside evaluator to design a bank of test questions, so that a unique test that includes questions to address each of the objectives can be used each semester of the project. Each test question will be designed to clearly address one of the project objectives. In addition, the test will include data about the examinee such as student ID number, class rank and previous exposure to the project via another course. This additional information will be necessary for both formative and summative data analysis.

Describes multiple methods

In addition to the quantitative assessment that the tests will provide, there may also be objectives under a goal that are best measured by a student survey. Although we will rely heavily on the quantitative aspects of assessment it is important to consider that "...by focusing primarily on quantitative techniques, evaluators may miss important parts of a story." McKeachie also echoes these sentiments in his book Teaching Tips. Thus the surveys will be designed to assess the students' perception of their progress towards each of the three goals of the project and in what ways the course aided their progress. The combination of quantitative and qualitative data will not only indicate our progress toward the project goals, but will also indicate how the project can be altered to better meet these goals. As with the skills-based test, the design of the survey will be designed in conjunction with the outside evaluator.

Information Sources & Sampling

In addition to the quantitative assessment that the tests will provide, there may also be objectives under a goal that are best measured by a student survey. Although we will rely heavily on the quantitative aspects of assessment it is important to consider that "...by focusing primarily on quantitative techniques, evaluators may miss important parts of a story." McKeachie also echoes these sentiments in his book Teaching Tips. Thus the surveys will be designed to assess the students' perception of their progress towards each of the three goals of the project and in what ways the course aided their progress. The combination of quantitative and qualitative data will not only indicate our progress toward the project goals, but will also indicate how the project can be altered to better meet these goals. As with the skills-based test, the design of the survey will be designed in conjunction with the outside evaluator.