Technology Annotated Report Excerpts

Excerpt 1 [Delaware]

Provides evaluation overview

In 1995, the U.S. Department of Education awarded a five-year Technology Innovation Challenge Grant (TICG) to the Capital School District in Dover, Delaware. This report details evaluation findings from the first four years of Delaware's TICG implementation. The Delaware Challenge project targets elementary school students and employs Lightspan^TM educational software in the classroom on desktop computers and at home on Sony Playstations^TM. The primary focus of this five-year evaluation is to provide information regarding how well the project has met its primary goals:

• generating more time for learning;
• increasing parent involvement in their child's education;
• providing professional development for teachers and other school staff;
• providing equitable access to technology and the information infrastructure; and
• improving student learning.

The evaluation of the Delaware Challenge project has proceeded along three lines of activity: 1) formative evaluation to provide relevant information to the project staff; 2) impact studies to assess the impact of the initiative on students and schools as it relates to teaching and learning; and 3) implementation assessment to determine how closely the project's actual implementation matches its intended implementation. Data to measure the progress towards project goals were collected using a variety of methods including surveys, interviews, self-report usage logs, achievement tests, and classroom observation. Selected evaluation results in the areas of classroom, home usage, students, parents and perceptions, staff development and perceptions, and student achievement are highlighted below.

Excerpt 2 [Delaware]

Summarizes findings

CLASSROOM USAGE

• The Lightspan software is used most often in the classroom as an individual activity, rather than as a group or whole class activity. A common model used is that of Centers, where the Lightspan software is the activity at one of several Centers. The use of computer Centers ranged from individualized, instructionally focused activity, to downtime activity with little or no connection to classroom instruction.
• On average, teachers reported spending about an hour a day (or 316 minutes per week) using the Lightspan software in the classroom.
• The Internet is becoming an increasingly important part of classroom component of the Lightspan project. While the project began primarily as software available on CD-ROM, the Lightspan Network (an Internet site available to schools participating in the project) has provided teachers with a variety of Internet activities and tools. In light of this, over half of teachers (55.6%) reported they use the Internet in the classroom at least one day a week.

HOME USAGE

• Like classroom implementation, the home to school connection varied by classroom. In many classrooms, the distribution of Lightspan CDs for home use was routinized; in some classrooms, it was erratic.
• When students used the Lightspan software at home, they tended to use it for a half hour or more, most often by themselves. Over two-thirds of students indicated they sometimes or always use Lightspan at home with a grown-up (usually a parent).
• Almost three-quarters of the students surveyed said they would (sometimes or always) rather use the programs than watch TV. For three consecutive years, the evaluation has found that students who use the Lightspan programs at home with a parent prefer the software to watching TV.

STUDENT AND PARENT PERCEPTIONS

• Nearly all students said they thought the Lightspan programs were fun to use, both at school and at home. Most parents said the project had been a positive experience and the Lightspan CD-ROMs were great learning tools for their child.
• Both parents and students indicated that they view the use of Lightspan software as a leisure activity. However, most parents thought the programs were a good use of student time outside of school.
• When asked about behavioral changes they had observed since their child's involvement with the project, many parents reported the amount of time their child spent … 1) watching television had decreased, 2) doing schoolwork had increased, and 3) participating in family activities had increased.

PROFESSIONAL DEVELOPMENT AND STAFF PERCEPTIONS

• Most school staff reported the Lightspan programs to be user-friendly and great learning tools for the students in their class.
• Over the past four years, project-related professional development has evolved from traditional training sessions to include classroom-based, job-embedded sessions. Implementation assessment revealed that professional development efforts have not successfully penetrated classroom curricula. That is, the Lightspan software has not been as closely integrated with classroom-level curriculum as is necessary for true curricular integration and implementation. Thus, classroom-based training activities, geared towards what to use and when to use (as opposed to simply how to use), will be increasingly important as the project strives towards sustainability in its final year.

STUDENT ACHIEVEMENT

• As would be expected in any given academic year, first and second grade student scaled scores on both the reading and mathematics achievement tests increased significantly from the pretest to the posttest. In relation to a national reference population, first grade students on average gained 5.9 percentile points in reading and 14.0 percentile points in mathematics. Second grade students on average gained 24.0 percentile points in reading and 16.2 percentile points in mathematics, as compared to the national reference population.
• No significant correlation was found between student home usage of the Lightspan software and student achievement gains. It is cautioned that readers do not interpret this as meaning that home usage does not influence achievement, but rather that these data do not provide conclusive evidence of a relationship.
• A significant relationship was found between classroom usage of the Lightspan software and student percentile gains in reading achievement (r= .202; p<.01). However, for female students, the more time spent in the classroom on the Lightspan software, the lesser the mathematics scaled score achievement gain (r= -.251; p<.001); this was especially true for female students who tested below the 50th percentile in mathematics on the fall test (r= -.378; p<.01). On the other hand, a positive relationship was found between classroom usage and percentile gains in mathematics for males who tested below the 50^th percentile in the fall (r= .240; p<.05).
• Students who tested in the lower two quartiles during the fall testing experienced much higher reading and mathematics gains than students whose fall achievement scores were above the 50^th percentile. While many students who tested above the median in the fall had no or moderate change in percentile, first graders who tested above the 50^th percentile in reading experienced significant percentile declines.