home
  : Reports : Curriculum Development





























home reports instruments plans
search

Curriculum Development Annotated Report Excerpts

Return to Curriculum Development Reports

Executive Summary

The table below contains report excerpts (right column) accompanied by annotations (left column) identifying how the excerpts represent the Executive Summary Criteria.

Annotations Report Excerpts
 

Excerpt 1 [Anonymous 1]

Provides evaluation overview

 

Describes sample

Multiple methods were used to evaluate a newly offered mathematics and physics course at the University of [anonymous]. The combination course was a section of Math 140 and Physics 150, offered back-to-back and team taught. Students in the course were interviewed during the fall 1996 semester, and were surveyed the following semester. The survey was administered to all sections of Math 141 (calculus II for the natural science), since all but four of the original combined course students were registered. Characteristics of the combined course participants were examined through administrative data and compared with their regular course counterparts. SAT scores on all students enrolled in Math 140 (both regular and combined sections) during the fall. There were no differences found in mean SAT scores among sections (all differences were well below one standard deviation). This is a crude index to establish the comparability of students in the courses with respect to their prior academic achievement. In another attempt to examine comparability, this time of combined students and their regular course counterparts in Math 141, demographic information was requested on the survey form. All respondents from both the regular and combination courses during the spring semester answered the questions asking gender and prior (high school) calculus experience. The student groups were roughly similar in demographic makeup, but the regular students had a higher proportion of females and prior calculus in high school than the combination students.

The survey was meant to capture attitudes toward mathematics and science, with a separate section of items reserved for those who had taken physics at the university. The domains of interest were:

Specifies areas addressed in the evaluation instrument

  • The amount of transfer of mathematics concepts into other courses or the real world
  • Attitudes toward use of technology and computers in mathematics and science
  • Enjoyment of mathematics
  • Enjoyment of physics
  • Knowledge of mathematics as a process
  • Confidence in mathematics ability

Responses were compared for the combination group and the rest, and in general, the combination group showed larger proportions of responses in the desired direction. That is, combination students tended to report more enjoyment of math, more confidence in their abilities, and more transfer of concepts into courses and situations outside the mathematics classroom than students who did not take the course. Combination students were also more likely to report using the Internet for mathematics and physics reference than regular students. Students did not differ on many items tapping mathematics processes.

Describes analysis design

In order to examine the integrity of the survey itself, an exploratory factor analysis was performed, resulting in a three-factor Varimax rotated solution. The first factor corresponded to transfer of mathematics into other situations. Factor II concerned attitudes toward technology, and Factor III corresponded to confidence in mathematics ability. (Physics items were excluded from the factor analysis.) This is an indication that the survey is tapping at least these three domains.

Describes limitations of evaluation results

Specifies response rate

In sum, the survey results are indicative of some differences among regular and combined students in their attitudes toward mathematics and physics. However promising, these results should be interpreted with caution since only 46% of the students enrolled in Math 141 were in attendance on the day the survey was distributed. This figure includes a slightly higher proportion of combination students. Initially a 48% response rate was obtained for the combined students, but e-mail announcements resulted in a few more responses, for a 55% response rate overall among combination students (23 of 43 students.)

 

Excerpt 2 [Anonymous 9]

Executive Summary

The treatment is a CD-based educational tool for use in fundamental science and engineering courses. Capable of being used with virtually any quantitative course of study, it is intended to help students learn the course material in a more effective manner and to make the administration and presentation of the course easier for the instructor. The National Science Foundation sponsored a program of testing and improving the software, which was conducted over a three-year period in engineering mechanics classes at University X and University Y. The software was also modified and implemented in a high school statistics course.

The objectives of the NSF study were to assess and improve the effectiveness of the software, particularly in the areas of implementation and administration, student use, instructor use, and subject matter modularity.

Implementation and Administration: The treatment was implemented at a small technological university (University Y), a large university (University X), and in a high school setting. Students had the option of using the software on an MS Windows® network platform or on individual PCs using a CD installation. This part of the software study was quite successful with a large majority of students as well as the participating instructors indicating a high level of satisfaction with the software's efficiency, navigational characteristics, and ease of use. Corroboration and discussion between University Y and the other participating institutions produced a wide array of improvements to the software throughout the course of the study.

Student Use: The treatment was tested on experimental and control groups of students at Universities X and Y. The approach for testing the treatment was to have the same instructor teach one or more experimental classes and one or more control classes. Both the experimental and control classes had identical formats except that the experimental classes fully utilized the treatment for submission of homework and the control classes followed a traditional homework approach (i.e., submission of problems on paper). Surveys and questionnaires gauged the students' level of satisfaction each semester. A statistical study at University Y indicated that use of the treatment resulted in some improvement in test scores for the study population as a whole; however, statistically significant improvement occurred for those students in the middle GPA range. Approximately 70% of the participating students indicated that they would prefer using the treatment over the traditional homework method.

Instructor Use: Six instructors (one primary and one secondary at each institution) used the software throughout the study. Input from each of these participants resulted in significant improvements to the software, particularly in the area of tracking student progress.

Subject Matter Modularity: One of the goals of the treatment was to be able to extract that portion of the code pertaining to a particular subject and efficiently replace it with material for another subject. For the NSF study of the treatment, course material for engineering mechanics was replaced with that for high school statistics. The study demonstrated that careful modular programming can result in efficiently adapting educational tools for differing subject areas.

The treatment performed well at the three institutions where it was implemented. A large majority of the students and all participating instructors were pleased with its performance. The study indicated that the major challenge for developers of educational software is to effectively keep pace with changes in both the publishing and the computer science industries and to adapt the educational software accordingly.