OERL: Reports: Teacher Education

An Evaluation Study of the Teaching of Hands-on Investigative Biology on a Shoestring

DAVID P. BUTTS

DAVID JACKSON

STEVE OLIVER

DOUGLAS P. BUTTS

The University of Georgia

MARY LOUISE BELLAMY

KATHY FRAME

National Association of Biology Teachers

This material is based on work supported by the National Science Foundation under Grant Number ES 1 9154112. Any opinions, findings, conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect views of the National Science Foundation.

INTRODUCTION

In this project BIOLOGY ON A SHOESTRING, the National Association of Biology Teachers recruited excellent, experienced high school biology teachers to develop and field test investigative laboratory activities that would require only limited resources to implement. The goal of the project was to increase the teaching of biology in high schools with an emphasis on how knowledge is generated rather than just a collection of what is known. There were two objectives for the project:

Thus, or in concert with the objective of the Division of Elementary, Secondary and Informal Education of the National Science Foundation, this project involved the development of new and improved instructional pre-college biology materials that involve students actively in scientific investigation. It further provided ways for teachers to involve all students, including women, minorities and those with physical disabilities in learning biology.

As noted in PROJECT 2061, SCIENCE FOR ALL AMERICANS (Rutherford & Ahlgren, 1989) and THE SCIENCE REPORT CARD: ELEMENTS OF RISK AND RECOVERY, (Mullis & Jenkins, 1988), there is a an urgent call for teaching the processes of how science is generated using hands-on investigative learning experiences:

… teaching related to scientific literacy needs to be consistent with the spirit and character of scientific inquiry… This suggest… starting with questions about phenomena rather than with answers to be learned; engaging students actively in the use of hypotheses, the collection and use of evidence, and the design of investigative processes… (Rutherford & Ahlgren 1989, p 5)

Merely calling for change does not always transform into altered experiences for students. Teaching investigative or hands-on science requires more than just a commitment from teachers. It requires resources both of time and of equipment. Asking teachers to upgrade their students laboratory experiences at a time when there is not enough money to purchase sufficient equipment and materials to implement these changes may indeed be a key reason for the lack of change in the classroom. While there are better and longer- term solutions for the lack of money for science instruction, an immediate and critical concern is to provide teachers with hands-on science activities that require little or no money. Such experiences would enable them to teach new biology labs "on a shoestring." If this innovation could be accomplished, then it is believed that students from all segments of our school populations would benefit.

The SHOESTRING BIOLOGY LABORATORY INVESTIGATIONS were developed by outstanding high school biology teachers and extensively reviewed by a panel of biologists, science educators and other high school biology teachers. Each laboratory investigation included information for the teachers, such as objectives, synopsis of the lab, preparation time required, materials needed, essential teacher background, teaching tips, instructional procedures including safety notes, sample hypotheses, procedures, data analysis and references for further readings. For the student, there were sections to help students design their own experiments, safety notes, questions for data analysis and opportunities for them to generate subsequent questions and design experiments to secure relevant data.

The final publications of NABT's SHOESTRING BIOLOGY LABORATORY INVESTIGATIONS consists of fifteen laboratories that have been field tested and found to meet the following four criteria, which are both teacher and student user-friendly.

EVALUATION DESIGN

Questions

In order to evaluate the success of this project, three questions comprised the basis of the evaluation design:

Does involvement in SHOESTRING BIOLOGY LABORATORIES make a difference in student problem-solving behavior?

What were the student problem solving skills at the end of individual laboratory topics?

What were the teacher perceptions of the usefulness of the SHOESTRING BIOLOGY LABORATORIES?

To answer these three questions, two data sources were used. Teachers provided their descriptions of the usefulness of the laboratories after using them in the classroom with their students. This information incorporated comments they had secured from their students. A second source was the problem solving performance of students which had been pre-tested in the fall of the school year, after the completion of each Shoestring Laboratory, and then post-tested in May.

Sample

Students in three contrasting urban/suburban settings were selected to provide ethnic diversity. Thirty-six teachers in three school districts were invited to participate in the study. In District A, nine teachers in its single high school of that small community participated. The students in their classes reflected the ethnic and socioeconomic diversities of a rural agricultural community. In District B, the nine teachers were in five of the six high schools of that suburban community. The students in their classes reflected the diversities characteristic of a rapidly growing community. In District C, the fifteen teachers were in nine high schools. In this most densely populated area of their state, these students illustrated both the diversities seen in urban populations and the influx of non-English speaking students. In all three school districts, there was about a 30% change in student populations on a given teacher's class roll during the school year. Appropriate permissions were secured from each school district for the participation of the teachers and students in this study. Part of this agreement included the understanding that all data would be reported in ways that there would be no identification of specific teachers or students.

Each teacher was asked to select at least three labs that would fit their curriculum expectations. While all teachers made their selections, their actual use of the labs ranged from none to six.

Instruments

Two instruments were used in this study. One was a generic measure of process skills which are part of problem solving behavior, the Test of Integrated Process Skills (TIPS) (Dillashaw & Okey, 1980). It includes five components of problem solving: a) interpreting data, b) identifying hypotheses, c) identifying variables, d) defining operationally, and e) designing experiments. Each item in this test is in a multiple choice format and is an independent sampling of the student's process skill performance. As reported by Dillashaw and Okey (1980) the Test of Integrated Process Skills has a reliability of .89 and an estimated readability of 9.2 grade level. The original instrument has two forms of 36 questions each. For this study, the test was shortened to 20 questions. As seen in Table 1, the five components of problem solving or process skills under study were equally represented on both forms of the test. Based on split half analysis, the reliability of this test was recalculated to be .78 for TIPS Form A (the pre-test) and .80 for TIPS Form B (the post-test.)

A second measure of the student problem solving behavior was a contextual test, the Shoestring Problem Solving Assessment (SPSA). This test measures student problem solving behavior with questions related to a single experimental context. It provides students with a set of information about an experiment, a data table of the results, and a graph of the data. In an open response format, students were required to describe the pattern they observed in those data and identify a possible hypothesis that was being studied. The student was provided with another hypothesis and an opportunity to describe the manipulated or independent variables and responding dependent variables of a study that they could design if they were investigating that hypothesis. Finally, in an open response format, students described how they would test the hypothesis. The instrument consists of fourteen items built around two problem contexts. As seen in Table 2, student responses were linked to the five components of problem solving or the process skills of interpreting data, identifying hypotheses, identifying variables, defining operationally and designing experiments. Split-half reliability for this instrument was determined to be .82.

TABLE ONE
TABLE OF SPECIFICATIONS FOR THE TEST OF INTEGRATED PROCESS SKILLS (AS MODIFIED) (TIPS)

Data Collection Tasks

Of the data collected, matched pre and post problem solving test data were obtained for 770 students and it is from these students that the majority of the findings of this study are based.

Data Analysis

To answer the first question to the evaluation of SHOESTRING BIOLOGY LABORATORY INVESTIGATIONS,

Does involvement in SHOESTRING BIOLOGY LABORATORIES make a difference in student problem solving behavior?

The pre-test and post-test scores of the TIPS and SPSA for 770 students were analyzed by computing score averages for each of the five integrated process skills. Their post-test scores were then compared using analysis of covariance with student pre-test scores and number of laboratories in which they participated as the covariants.

What were student problem solving skills at the end of individual laboratory topics?

A laboratory quiz for each topic was developed using a format similar to that of the SPSA. Since the investigations described in the quiz were related in content to that of the laboratory but were not taken directly from the lab activities, the lab quiz was not a test of factual recall. After the students had completed a specific SHOESTRING BIOLOGY LABORATORY INVESTIGATION, they completed the LAB QUIZ for that investigation. A total of 2279 lab quizzes from all students who participated in the labs was used for this analysis. Thus, for each of the topics, the students' success with problem solving or process skills is seen in both the lab quiz scores as well as the comparison with the averages for all the lab quiz scores which reflects changes in problem solving which occurred as the year progressed.

To answer the third question of the study - What were the teacher perceptions of the usefulness of the SHOESTRING BIOLOGY LABORATORY INVESTIGATIONS? -Focus group sessions were held in April and May in 1994 with the teachers participating in the evaluation study. At these sessions, all participating teachers had opportunity to share their reflections from the classroom experiences of using the labs. As described in the PACE evaluation model (Koballa, Butts & Riley, 1995) these open responses were then categorized on the four goals of the project as another indicator of the success of the project.

RESULTS

Does involvement in SHOESTRING BIOLOGY LABORATORIES make a difference in student problem solving behavior?

The students scores were grouped based on the number of laboratories that they had completed. The control group (N = 113) did not have the opportunity to do any of the Shoestring Laboratories. The Minimum Practice group (N = 247) were those students who had completed one or two of the Shoestring Laboratories. The Multi-Practice group (N = 441) were those students who had completed between 3 and 6 of the Shoestring Laboratories. By grouping the students this way, it was possible to retain the anonymity of the teachers involved. The average post-test scores for the five process skills as they relate to the number of laboratories that students completed are seen in Table 3 for the generic test of problem solving (TIPS) and Table 4 for the context measure of problem solving (SPSA). The percent of correct student responses show how successful students were in using the integrated process skills. It is clear that at the end of the school year, the pattern of the problem solving behavior who had no experience with the Shoestring Laboratories(control group students) was consistently lower than those who had opportunities for instruction with the Shoestring Laboratories.Students who had an opportunity to practice the Shoestring Laboratorieshad better post test performance than the control group of students in each of the integrated process skills in both the generic (TIPS) or contextual (SPSA) measurement of these skills. Those students in the Multi-Practice group had higher end of the year scores than the scores in the Minimum Practice group. Thus the conclusion is that participation in the Shoestring Laboratories does indeed strengthen students' problem solving performance.

While this finding could have been a function of experience with Shoestring Lab instruction, it could also have been a reflection of lower pre-test performance of students in some groups. Using the pre-test scores as the concomitant variable and the number of labs in which they had participated as the treatment, the results of an analysis of covariance are seen in Table 5 for TIPS and Table 6 for SPSA. The F ratios provide evidence for the conclusion that the greater the number of laboratories students experience, the higher their post test scores regardless of their pre-test scores. Since the R indicates how much of the variance is accounted for by the treatment, the higher R for the SPSA indicates that this test is a more sensitive indicator of the changes that occurred as a result of the treatment. The more experience they had with the SHOESTRING BIOLOGY LABORATORY INVESTIGATIONS, the higher their post-test scores.

What were student problem solving skills at the end of individual laboratory topics?

LAB QUIZ data from 2917 students are show in Table 8. After the field testing of the labs, the decision was made by NABT to exclude some of the topics based on the review of the science advisory committee and based on feedback from the field test teachers. These decisions were made independent of the test data. The SHOESTRING BIOLOGY INVESTIGATION LABORATORIES that have been published in the final product are indicated by (#) in Table 8.

The student quiz performance in each lab were collected and the averages computed for all students who participated in the specific Shoestring lab. Table 8 shows the average student performance on each of the integrated process skills from these lab quiz scores. While individual lab quiz scores were not matched to the pre-test and post-test scores of all students, it is interesting to note that the pattern of student performance of each of the integrated skills showed remarkable gain for three of the process skills, interpreting data, defining operationally,and designing experiments. Each of these skills required students to write in an open response format rather than a multiple choice format. These results provide further evidence that student problem solving skills were enhanced by their opportunity to practice them as part of the SHOESTRING BIOLOGY INVESTIGATION LABORATORY experiences.

TABLE 7
STUDENT PROBLEM SOLVING PERFORMANCE AFTER INSTRUCTION AS MEASURED BY LAB QUIZCOMPARED WITH END OF YEAR SPSA POST TEST

TABLE 8
STUDENT PROBLEM SOLVING PERFORMANCE ON LAB QUIZZES OF INDIVIDUAL SHOESTRING LAB TOPICS

What were the teacher perceptions of the usefulness of the SHOESTRING BIOLOGY LABORATORY INVESTIGATIONS?

In focus group sessions at three school district sites completed in April and May 1994, the teachers were asked to reflect on their students' reactions to the SHOESTRING BIOLOGY LABORATORY INVESTIGATIONS. Their reactions were then related to the four goals of the project.

For the first goal, "develop laboratory experiences that require a minimum of money to implement," teachers noted that:

Students liked the flexibility in the way they could make the lab fit their interests, time and resources.

Students really got into preparing, securing materials and teaching parts of the labs to the rest of the class.

The second goal was for "the labs to have a process emphasis and actively involve students". Most teachers clearly saw the labs having a process emphasis as they involved students in the design of the experiments.

Students liked asking questions such as what else? What happens now? What happens next-and knowing that they did not have the answer on the next page.

Students designed their own experiments but frequently did not have time for them to do what they had designed.

Teachers also described the labs as providing experiences not normally in their courses.

Students liked the graph interpretations - it was good practice with a skill not usually done in class.

Involvement of students in their own learning was frequently identified by teachers as a positive outcome of using the labs.

Students really enjoyed preparing and teaching parts of the labs to the rest of the class.

Students liked the group work once they were organized with assignment of tasks and responsibilities for work to be done.

However, involvement of students in their work together was not always seen as a positive outcome. Some students did not enjoy cooperative groups and teachers noted that this was probably because they did not know how to work together.

The third goal of the project was to "produce materials that reflected practices that experienced high school biology teachers know work in the classroom and include worthwhile science understandings". Related to this goal, teachers noted that

The fourth goal was to "include students' feedback in the development of the labs". Many teachers noted that students really enjoyed the labs but time limitations was a real concern.

Students designed their own experiments but on occasion did not have time for them to do what they had designed.

Students enjoyed the labs but sometimes they did not want to do the experiments they had designed because they had become bored with the topic or wanted to move on to another topic.

Other teachers noted that the involvement of the students as peer teachers was viewed very positively by students.

Students found that they liked having those who had the skill to have opportunities to teach those who did not yet have the skill.

Students enjoyed a new strategy for getting a lab grade - "Everyone in here will get a lab grade and all will get the lowest grade earned by anyone in the class.

But all students did not want to be involved in the creative phase of designing and conducting experiments

Students resented having to figure out what they were to do - they wanted to be given the steps and then they would do it.

Students wanted more directions in the labs and only one person assigned to do each task.

Thus, most teachers believed that the labs were most successful in achieving the goals of the project. It should be noted that the teachers participating in the evaluation study were not part of the field testing of the SHOESTRING BIOLOGY LABORATORY INVESTIGATIONS and thus their comments related to desired changes were much more limited. On several occasions, they did describe ways they found it useful to modify the student direction sheets or to use other organisms to accomplish the goals of the lab.

CONCLUSIONS

An evaluation study of this project, the SHOESTRING BIOLOGY LABORATORY INVESTIGATIONS, was designed to answer the question, Did these investigations make a difference and if so, in what way? Related to the two main goals of the project - to facilitate hands-on investigations in biology and to do it without the need for added expenses - it is clear that the project demonstrated its success. When given the option to use as many of the laboratory investigations as they desired, some teachers chose to use up to six of them. In no cases were additional funds required for the labs to be used in classrooms. The fact that the laboratory investigations are easily conducted with commonly found materials in students' neighborhoods further enhances the overall usefulness of the laboratories - and practically guarantees the widespread use of them if the teacher is willing.

In addition to the usefulness of the topics of the laboratory investigations, the students clearly enjoyed the opportunity to ask questions and to design their own experiments. The quiz results show that the more that students had practice finding evidence for their answers to questions they had posed, the better their problem solving skills became. As seen in the results from the control group students, lacking these kinds of laboratory experiences, exposure to biology does not itself help students to solve problems. If problem solving is a valued outcome of high school biology, then the SHOESTRING BIOLOGY LABORATORY INVESTIGATIONS provide evidence that learning how to think or problem solving can be achieved by high school students.

The results of this study show that students can be helped to learn how to solve problems using process skills, and outcome of biology instruction that is valued by teachers since success is thus accessible to all their students. The teacher is the key for the use of the labs. In this study, all teachers initially chose to participate by selecting labs that they believed fit their curriculum. Some teachers used the labs and some did not. Was this choice influenced by their belief or concern about the appropriateness of the labs for their students or their desire to incorporate more laboratory instruction into their biology courses? Knowing the teacher's perception of the need for lab instruction and the need for inclusion of more lab instruction in their courses clearly are two issues that will influence if the labs will be used. If they are used, students will show gains in their performance of problem solving or process skills.

The SHOESTRING BIOLOGY LABORATORY INVESTIGATIONS provided the teachers with a tool currently not available. This tool helped their students achieve what for many years has been described as the primary goal for teaching science - that is, learning how to solve problems and think critically. Reading about what science is and what scientists have learned differs greatly form doing science - asking questions, designing experiments, interpreting results and finding ways to design experiments could be improved to provide more credibility to their answers to scientific questions.

It is in the doing of the SHOESTRING BIOLOGY LABORATORY INVESTIGATIONS that possibly for the first time, students experience the joy, the excitement and the intellectual power of science.

Teacher Education Stand-Alone Report 2 (Final)

An Evaluation Study of the Teaching of Hands-on Investigative Biology on a Shoestring

EVALUATION DESIGN