Technology Annotated Report Excerpts

Excerpt 1 [Delaware]

Evaluation Purposes:
Describes summative evaluation purposes

Finally, the implementation of the Delaware Challenge project was evaluated to determine, to the extent possible, whether the outcomes measured were indeed attributable to the project's implementation. As mentioned previously, CBAM's Levels of Use framework was used in conjunction with impact data to interpret outcomes in light of the underlying project theories.

The focus of the implementation study was to further examine factors that contributed to the findings from last year's evaluation report, using data from the current year's evaluation of target schools. The home-school connection was studied to learn more about how the connection works and what factors might have accounted for the promising results found last year in the areas of student behavior and student achievement. The implementation study addressed the following research topics:

Evaluation Questions:
Lists topics and questions behind formative evaluation

• 1. Classroom Component - Teacher Usage
  How are the Lightspan materials used in the classroom? Are Lighspan materials integrated with other classroom lessons? For what purposes?
• 2. Classroom Component - Home Usage
  How do teachers manage the home-school connection? How often are Lightspan materials sent home with students? How do the materials sent home relate to classroom curriculum?
• Home Component - Student Usage
  How do students use the Lightspan materials at home? How does the Lightspan software relate to other student homework activities? How often do students work on the Lightspan software for leisure versus for homework? Who do students like to work with and whom do students usually work with when they use the software at home?
• Home Component - Parent Role
  What is the role of the parent in working with students at home on Lightspan materials? How comfortable are parents with helping their children on program-related activities? What is the parent's relationship with the school?
  

Excerpt 2 [Delaware]

Evaluation Purposes

The evaluation of the Delaware Challenge Grant has proceeded along three lines of activity:

1. formative evaluation to provide relevant information to the project directors;
2. impact studies to assess the impact of the initiative on students and schools as it relates to teaching and learning;
3. and implementation assessment to determine how closely the project's actual implementation matches its intended implementation.

Excerpt 3 [Agents]

Stakeholder Involvement:
Describes ongoing process of getting input from stakeholders

A number of our Year Three evaluation activities have been a continuation of those begun in Years One and Two. In monthly activity reports and periodic conversations, we have continued to provide regular feedback to the project staff and offer suggestions for meeting both the incremental and larger goals of the project. We have also had ongoing discussions with the project staff and district leaders about management issues, district policies, and other activities and decisions that influence the project's status. These discussions have helped us place this project in a broader context and see how it affects and is affected by other school and district initiatives.

Evaluation Purposes:
Presents purposes behind various data collection activities

In addition to consulting with the staff, evaluators have continued to make monthly site visits to observe various aspects of implementation and attend important project events, such as students' multimedia presentations, staff development sessions, and summer Tech Camps. By documenting the project's progress we hope to provide a record of accomplishments to the Department of Education and other stakeholders and, ultimately, tell the project's story from beginning to end. Data on the many efforts made in staff and curriculum development, technology integration, and community partnerships should prove useful as project participants begin to scale up their efforts. In time, the data may also provide a model for other grants designed to use technology to improve education and the prospects of students, especially minority students, in urban areas.