OCEPT Annual Report, 1998 (Year 1)
This evaluation report is embedded in a larger progress
report prepared by Portland State University.
Table of Contents:
-
Project Summary
- Development of a Statewide
Collaborative
- Course, Curriculum, and Program
Development
- Faculty Professional
Development
- Recruiting, Retaining and
Supporting Future Teachers, and Increasing Teacher
Workforce Diversity
- General Strategies
for Change
-
Annual Report
(sections omitted)
A7. Evaluation Activities and
Formative Evaluation Results
- Organizing the OCEPT Evaluation
and Research Team
- Evaluation Overview: Stakeholder
Involvement
- Developing the Evaluation Plan
- Evaluation Overview: Evaluation Purposes,
Stakeholder Involvement
- Summer Institute Evaluation
- Evaluation Overview: Stakeholder
Involvement
- Results & Recommendations:
Interpretations & Conclusions
- Pre-Award and 1997 CETP Data
Reporting Process
- Design: Data Collection Procedures &
Schedule
- Entering Teacher Education
Student Survey
- Evaluation Overview: Evaluation Purposes,
Stakeholder Involvement
- Design: Methodological
Approach, Information Sources & Scheduling,
Instruments, Data Collection Procedures &
Schedule, Meta-Evaluation,
- Analysis Process: Quantitative
Analysis
- Mid-Year Interviews with CO-PI's,
Faculty Fellows and Mentor Team Members
- Design: Information Sources &
Sampling, Data Collection Procedures &
Schedule,
- Results & Recommendations:
Stakeholder Review & Utilization
A8. Changes in Evaluation Plans
- Design: Methodological Approach,
Meta-Evaluation
- Evaluation Overview: Evaluation Purposes,
Stakeholder Involvement
OCEPT Annual Report, 1998 (Year 1)
Return to Table of
Contents
The Oregon Collaborative for Excellence in the Preparation
of Teachers (OCEPT) was created to improve the math and science
preparation of future teachers in Oregon and increase the
state's teacher workforce diversity. While these goals are
consistent with those of the NSF CETP program in general,
certain conditions in Oregon's educational system require
different strategies for reaching these goals than is possible
in many other states. Some of these are:
- Most teacher education programs in Oregon are offered
at the graduate level only. Students preparing to teach
generally receive their bachelor's degree in another discipline
(not education) and then enter a graduate teacher licensure
program, often at another institution. Identifying students
as prospective teachers while they are still undergraduates
is problematic, particularly since they themselves may not
yet be considering teaching as a career. Advising for these
students at the undergraduate level is generally inadequate.
As a result of this disconnect between liberal arts and
education programs, many mathematics and science faculty
do not see themselves as teacher educators.
- To compound this situation, student mobility among institutions
in Oregon is high. Students may begin at a community college,
where they take their introductory math and science courses,
before moving on to a four-year college, or they may move
back and forth between two- and four-year schools or even
be enrolled simultaneously. Community college faculty, while
offering many excellent science and mathematics courses
that are highly appropriate for future teachers, are even
more disconnected from teacher education programs than four-year
college faculty.
- The organization that licenses teachers in Oregon, the
Teacher Standards and Practices Commission (TSPC), does
not require specific coursework for licensure, only passing
scores on standardized tests. It is up to teacher education
programs whether to require specific coursework in mathematics
and science, and many do not, partly because of market pressures
to attract students who might be deterred from applying
if they lack the required math and science background. Prospective
elementary teachers, if they have taken math or science,
would have done so only as part of their general education
requirements, which vary greatly from institution to institution
and may not include either math or science.
- While this may now be changing, there has not been a
shortage of teachers in recent years. According to a TSPC
report, in 1996 only half of newly licensed elementary teachers
were employed in Oregon public elementary schools by September
30; between one-half and two-thirds of newly licensed secondary
teachers in math and science had jobs in the fall. Where
a shortage does exist is in the diversity of the teacher
workforce; only 2-3% of teachers overall are from minority
groups, and even less in math and science. This shortage
is particularly evident in urban and rural areas where minority
populations are concentrated.
- Excellent programs to support students from underrepresented
groups interested in pursuing higher education in general
and science/mathematics in particular exist at both the
precollege and college level, but these programs often do
not communicate with one another, and many science/mathematics
faculty do not know of them or know how to connect their
students with them.
In spite of these challenges, Oregon also enjoys some real strengths
which make this an ideal place and time to achieve OCEPT's
goals:
- Science and mathematics faculty at many institutions
have a strong history of educational reform, including many
federally- and privately-funded course and curriculum development
projects. Nationally-recognized leaders in physics and biology
reform are among OCEPT's leaders, and a strong network of
collaboration among physics, biology, and mathematics faculty
is already in place. Faculty in other disciplines also offer
a wealth of knowledge and experience about college teaching
and learning. The community colleges' personal, student-centered
environment and their faculties' focus on teaching excellence
provide particularly appropriate experiences for future
teachers.
- Systemic school reform is well underway in Oregon, including
a move towards standards-based instruction using the Oregon
Benchmarks, performance-based assessments at 4th, 8th, and
12th grade levels, Certificates of Mastery rather than traditional
seat-time-based diplomas, and a new portfolio-based system
of college admissions. Teachers at many pilot schools are
receiving extensive professional development to help them
learn to teach effectively in this environment, and are
thoroughly involved in planning as well as implementing
reforms. Initial statewide assessments in mathematics have
shown a real need for educational improvement, and 1998's
science assessment is likely to be even more
telling.
- New teacher licensure requirements have recently been
instituted by TSPC, requiring students to be licensed in
a combination of two grade levels: early childhood-elementary
(age 3-grade 5); elementary-middle (grades K-8); and middle-secondary
(grades 5-12). As a result, teacher education programs are
in the midst of revising their programs, with particular
attention to the preparation of middle-level
teachers.
- Oregon is enjoying strong economic growth, particularly
in high-tech industries, so that the argument for a scientifically
and mathematically skilled workforce is particularly
persuasive.
In this context of advantages and challenges, OCEPT is employing
four general strategies to achieve its goals: the development
of a statewide collaborative network among mathematics, science,
engineering, and technology (SME&T) and education faculty in
two- and four-year, public and private colleges; the development
of courses, curricula, and programs which will strengthen future
teachers' preparation in mathematics and science; opportunities
for faculty professional development and pedagogical learning;
and programs to recruit, retain, and support excellent students
from their undergraduate years to their first years of teaching,
particularly those from underrepresented groups. The major achievements
of OCEPT in its first six months in each of these areas is summarized
below.
Return to Table of
Contents
Development of a statewide
collaborative
OCEPT will involve all 34 regionally-accredited institutions
of higher education in Oregon, including the 17 institutions
with teacher education programs. Involvement of all institutions
is critical due to the high level of student mobility among
institutions as mentioned earlier. At least two active OCEPT
participants or supporters have been identified in all but
three of the institutions: one small private college (which
does not have a teacher education program) and three small
community colleges. On most institutions, the OCEPT "team"
now consists of around three to six faculty and
administrators.
OCEPT's primary strategy for encouraging collaboration among
these institutions are the Faculty Mentor Teams. In Year 1,
eight Faculty Mentor Teams were established, in biology, chemistry,
physics, mathematics, integrated science (focusing on the
preparation of secondary teachers in these areas of licensure),
elementary/middle school science, elementary/middle school
mathematics, and teacher education. The permanent (year-to-year)
membership of each team consists of three faculty members,
generally one from a state university, one from a private
college, and one from a community college. These teams are
meeting frequently and communicating regularly to develop
ways of encouraging collaboration among faculty with teaching
interests in these areas, and to collaborate across teams
to achieve the overall goals of OCEPT. Team members have been
traveling extensively, attending meetings and making presentations
to support current participants and recruit additional participants
to OCEPT.
The Mentor Teams were created to address the problem of
the individual faculty member working in isolation from colleagues,
often in an environment where teaching is not valued as a
scholarly occupation. The teams provide other OCEPT participants
with access to a network of colleagues in their own discipline,
both within and outside their institution, which is providing
the energy and support needed to sustain their teaching improvement
efforts. In many cases, the teams are able to connect with
strong professional organizations at the state and regional
levels; in others, these organizations are themselves being
strengthened.
Each team leader is also a co-principle investigator at
an institution where it makes sense to target OCEPT influence
and support, primarily those institutions which enroll the
majority of teacher education students. The co-PIs are thus
facilitating inter-departmental collaboration and change on
their own campuses as well. In the case of the two largest
research universities, faculty from the sciences, mathematics,
and engineering, and faculty from education, are meeting and
collaborating in ways they rarely have before. The development
of a statewide collaborative is what will break down barriers
between SME&T and education faculty, provide support and recognition
for teaching improvement, and make the changes initiated by
OCEPT sustainable over the long term.
Return to Table of
Contents
Course, curriculum, and program
development
OCEPT funds Faculty Fellowships for faculty to develop courses,
curricula, and programs intended to strengthen the math and
science preparation of future teachers. Faculty Fellows join
the Mentor Team in the appropriate discipline, which provides
support and resources for their course and program development
projects. Twenty-one Faculty Fellowships were awarded in Year
1, and thirty-two have already been awarded in Year 2, with
two to three more to be added. Most Fellows' projects in Year
1 involved revising an undergraduate science or mathematics
course. In Fall term 1997 (the first academic term of the
project), a total of twenty-two courses were either revised
or newly developed, enrolling almost 700 students. Other Year
1 projects are being completed in winter/spring 1998. Many
Faculty Fellows have made significant revisions in other courses
they teach as well, not just the one they were funded by OCEPT
to revise. As the teams receive progress reports, however,
it has become evident that while these are generally highly
effective improvements, the connection to teacher preparation
is often weak. In recruiting Fellows for Year 2, we created
more specific requirements for Fellowship applicants to clearly
identify their project's impact on future teachers, and the
Mentor Teams will continue to emphasize this as they work
with the Fellows.
More Fellows' projects in Year 2 also address extracurricular
programs especially designed for future teachers, such as
early field experiences, teaching seminars, and "emerging
scholars" programs on the Treisman model. Some Faculty Mentors
and co-PIs are also facilitating the development of these
programs on their campuses.
OCEPT does not prescribe any particular approach to revising
courses or developing programs, so at first glance the Fellows'
and Mentors' projects may seem like an eclectic mix. Developing
a truly effective course or program requires more than just
release time or other funding; it requires the personal and
professional commitment of the developer. OCEPT participants
feel strongly about their projects, and are developing them
as potential models for others to adapt. The collaborative
networks and opportunities for professional development offered
by OCEPT will help project developers navigate the fine line
between "reinventing the wheel," and creating something to
which they are committed but which incorporates good ideas
developed by others.
Return to Table of
Contents
Faculty professional
development
The OCEPT leadership has agreed that any model for professional
development within OCEPT cannot be prescriptive or hierarchically
determined. Our institutions, programs, and faculty are too
diverse to attempt to prescribe any one set of methods of
teaching or assessment. For professional development to be
truly "professional," faculty need to make their own decisions
about teaching and how they determine its effectiveness, within
a context of collaboration, dissemination, and peer review.
Faculty Fellows are all expected to make these decisions about
their own course/program development projects and share their
results in a scholarly arena, with the continuing support
of their Mentor Team. The fact that so many Fellows in Year
1 made more changes in their teaching than those specifically
outlined in their project plan, supports the effectiveness
of OCEPT's focus on faculty change rather than course change.
Faculty who modify their beliefs about teaching and learning
will modify their teaching practice in all their courses and
other interactions with students, and can serve as catalysts
of change for their colleagues. Attempting to make courses
"teacher-proof" so that they can be taught by anyone, if the
original developer leaves or changes his/her teaching assignment,
is unrealistic, since the teacher's beliefs about teaching
and learning will impact his/her practice in ways that cannot
be prescribed by a syllabus or laboratory handbook.
One of OCEPT's major strategies for increasing the recognition
and reward faculty receive for their teaching scholarship
is to encourage the dissemination of participants' work in
scholarly ways, such as conference presentations and publications.
Three current Faculty Fellows, along with other team members,
presented progress reports on their projects at the Oregon
Academy of Sciences meeting in February 1997, and others will
present at regional professional meetings later this spring.
A mid-year Assessment Retreat was held in Bend this March,
and was attended by 15 Fellows and Mentors. We discussed options
for scholarly publication beyond conference presentations,
such as an edited volume of case study reports, and solidified
the Fellow's plans for collecting and analyzing both qualitative
and quantitative data on their course's effectiveness in ways
that would be effective in such a publication.
Most faculty also need to explore new models of teaching
and assessment, learn about the research on teaching and learning
in their discipline, and experience first-hand how new approaches
are likely to affect their students. OCEPT faculty also need
to learn more about issues of teacher preparation and licensure
and how their teaching can affect future teachers. These are
the objectives of the three-week OCEPT Summer Institute, attended
by the majority of OCEPT Fellows, Mentors, and co-PIs. The
first Summer Institute was held in July-August 1997; a complete
evaluation report is presented in Appendix D. The next Institute
is planned for 20-31 July at Central Oregon Community College
in Bend, and from 10-14 August in Portland. Two co-PIs are
chairing the planning committee, and Institute leaders will
be drawn almost solely from within the OCEPT group. The major
change in this year's program from last year's is that assessment
and "teaching as scholarship" will play a much more central
role.
OCEPT was able to fund the hiring of two Teachers-in-Residence
at Portland State University and Eastern Oregon University
in Year 1, who have been able to work with faculty on developing
courses and programs appropriate for prospective teachers,
and to help all OCEPT participants (through their participation
in the Summer Institute and the Mentor Teams) understand the
needs of future teachers. We are exploring additional means
of involving K-12 teachers in OCEPT, in order to give faculty
the benefit of their pedagogical expertise and first-hand
knowledge of the changing needs of teachers in this
state.
Return to Table of
Contents
Recruiting, retaining, and
supporting future teachers, and increasing teacher workforce
diversity
OCEPT is designed to recruit and support future teachers
at all levels, from their undergraduate and graduate programs,
through licensure and into their first years of teaching.
In Year 1, we have begun development of the Early Career Teacher
Mentor Program, which will provide mentors (master K-12 teachers
in an appropriate discipline and level) for teachers in their
first three years of their career. The teacher mentors will
take part in a summer institute which will provide guidance
and training in mentorship, and they will also be encouraged
to participate with the Faculty Mentor Teams in working with
Faculty Fellows. The two Teachers-in-Residence have taken
the lead on developing and implementing this program, modeled
after the highly successful program of the Montana Collaborative,
which they were able to visit last fall.
Making sure the best mathematics and science students seriously
consider a career in teaching has become a new priority for
OCEPT faculty. Once their experiences in OCEPT helped them
see the need for college science/mathematics faculty and departments
to take responsibility for teacher recruitment and preparation,
they began working on means of identifying and supporting
students interested in teaching, including teaching seminars,
early field experiences, and "emerging scholars" programs
for students in introductory mathematics and science
courses.
Many of these programs also address the need to attract
students from underrepresented groups into teaching. In addition,
we have developed and are updating a list of contacts in programs
specifically designed to support minority students, and have
held several meetings to gain their input into OCEPT strategies
for increasing diversity. One such meeting concentrated on
the development of our proposal for the NSF Teaching Scholars
program, for $100,000 per year in scholarships to recruit
outstanding mathematics and science students into teaching,
particularly those from underrepresented groups. Finally,
we have recruited two Faculty Fellows to work specifically
on the diversity issue, who will help to gather and synthesize
information on programs that work, how to develop better connections
between precollege and college-level programs, and provide
this information to OCEPT participants.
Return to Table of
Contents
General strategies for
change
In addition to these accomplishments, OCEPT has established
the Oregon Science Education Council as an overall strategy
for promoting policy change in the state. OSEC will support
science education reform in the way that the Oregon Mathematics
Education Council has supported mathematics education reform
for the past 20 years. OSEC and OMEC, both of which involve
OCEPT participants as leaders, will review the current state
of science education in Oregon, make recommendations for change,
and communicate those recommendations to appropriate
constituencies.
Return to Table of
Contents
II. Annual Report
(sections omitted)
This summary is organized into six sections to reflect the
work to date related to OCEPT evaluation activities and
results:
- Organizing the OCEPT Evaluation and Research Team
- Development of the Evaluation Plan
- Summer Institute Evaluation
- Pre-Award and 1997 CETP Data Reporting Process
- Entering Teacher Education Student Survey
- Mid-Year Interviews with Co-PIs, Faculty Fellows and
Mentor Team Members
Return to Table of
Contents
Organizing the OCEPT Evaluation
and Research Team
The PI, Project Coordinator and Evaluation Coordinator identified
individuals early on, before OCEPT received the award, who
had background in evaluation and research and who we thought
would be interested in serving on the Team. A series of meetings
took place between February 1997 and Fall 1997 to develop
the Evaluation Plan framework and proposed activities. By
late Fall, membership on the Team had stabilized at ten individuals,
and includes: the PI, Project Coordinator, Evaluation Coordinator,
the Science Specialist for the Oregon Department of Education,
four Mentor Team members, a Faculty Fellow and the director
of a Teaching and Learning Center at a local community college.
The backgrounds represented include two- and four year, private
and public institutions; and math, science and teacher
education.
At its February 1998 meeting, the Team began to organize
itself into several sub-groups:
- students
- Faculty Fellow and Mentor Team development, and change
in teaching practice
- Collaborative development, including institutional
change
- diversity
The group will continue to meet quarterly for half-days
but will conduct more of its work in sub-groups both at these
meetings and between meetings. The sub-groups may choose to
recruit others to work with their sub-groups. These sub-groups
will also recommend to the Team how a particular evaluation
activity should be carried out and which special research
studies should receive priority for funding from grant evaluation
funds.
An on-going concern of the Team is the scope of work proposed
and the feasibility of completing all of it within current
evaluation resources. The Team will continue to review the
scope of work and in consultation with the Management Team,
make needed adjustments.
Return to Table of
Contents
Developing the Evaluation
Plan
Development of the Plan has occupied a great deal of the
Team's time. The latest version, included in Appendix I, will
be reviewed by the Management Team in the Spring. A concern
throughout many of the discussions has been over what we believe
OCEPT can reasonably be expected to affect over the life of
the grant; and what will take a longer period of time. OCEPT
is a very large Collaborative, involving thirty-four different
colleges and universities in the State. Seventeen different
institutions offer Teacher Education programs, all organized
a bit differently; some only at the undergraduate level and
others only at the post-baccalaureate level. Student mobility
is considerable, with many teacher education programs serving
students who have completed some or most of their undergraduate
math and science course work at other in-state or out-of-state
institutions.
The Cooperative Agreement spells out a number of outcomes
that should be achievable. The Plan reflects the belief that
over the next 4 years OCEPT can have an impact on faculty,
their students and their colleagues; on Mentor Teams and the
development of more extended professional communities; on
the recruitment and retention of individuals currently underrepresented
as teachers of K-12 math and science; and on the collaboration
process itself, involving more communication across institutions
and disciplines. As the project continues to develop, the
Team will need to develop more specific ways to assess more
specific impacts or changes influenced by OCEPT.
In particular, we need to develop better ways to document
and assess change in our future teachers. Such change might
manifest itself as becoming interested in teaching because
of the encouragement of a particular faculty member; a change
in attitude about teaching math and science; a change in the
amount of math and science courses of particular kinds taken
by pre-service elementary students and secondary math and
science students; and a change in actual classroom or field-experience
practices.
Return to Table of
Contents
Summer Institute
Evaluation
The Summer Institute Evaluation was completed in September
1997 and is part of the OCEPT formative evaluation. Several
Team members who attended the Institute were instrumental
in helping to develop the survey instrument used. Several
Co-PIs and Mentor Team members also contributed ideas and
reviewed survey drafts. The Project Coordinator summarized
the statistical results and the Evaluation Coordinator summarized
the qualitative responses.
Overall, responses indicated that participants were very
positive about their Institute experience. Given the short
timeframe for planning and the fact that the Institute was
held before OCEPT was notified it would be funded (and before
it received any funds), this outcome is notable. Several staff
members were working for no pay during this period of
time.
The report includes a series of recommendations for those
planning the 1998 Summer Institute.
Return to Table of
Contents
Pre-Award and 1997 CETP Data
Reporting Process
One of the major challenges in completing both the pre-award
and the more recently due annual CETP data set has been identifying
key contact individuals for gathering student data at each
of the 34 institutions in the Collaborative. These contacts
were not adequately in place during the collection of the
pre-award data. We now have identified the "right" individuals
on each campus who can help us collect the needed student
demographic information. Also, while many were appreciative
of the ability to enter the data electronically, many others
did not have access to a current version of Netscape or Explorer.
Still others experienced considerable time delays in calling
up their file and then in moving from one reporting form to
another. Some of this delay may have to do with the particular
way in which the Website is organized, though some of it is
due to local communication systems.
We also did not have adequate support systems in place for
Faculty Fellows to gather required data on students in their
courses. At this summer's Institute, Fellows will each get
a packet with clear guidelines on the data needed and tools
they can use in gathering it, such as the brief Student Information
Survey, along with other resources for planning and assessment.
Return to Table of
Contents
Entering Teacher Education Student
Survey
With assistance from project staff and some Research and
Evaluation Team members, a Teacher Education Student Survey
was developed during Summer 1997 and administered to students
at 14 of the 16 OCEPT institutions with teacher education
programs. An effort was made to identify and review instruments
designed for a similar purpose from other CETP Collaboratives.
Information was received from the Rocky Mountain Collaborative
which was adapted for use on our survey.
The primary purpose of the survey was to collect information
from students entering teacher education programs about their
experiences in undergraduate math and science courses. The
survey was also designed to collect information about how
much undergraduate math and science courses these students
had taken, their attitudes about teaching math and science,
and how they assessed their science literacy skills.
We estimate that we have surveyed at least one-third of
the students who had recently entered (Summer or Fall 1997)
a teacher education program. We are still in the process of
entering the information from the 330+ surveys received and
expect to have a report available in May 1998. Before this,
however, we expect to have descriptive summary reports available
for each of the participating institutions.
The information gained from the survey will serve both as
baseline OCEPT information about the math and science course
experiences of students who enter teacher education programs.
Also, we hope the information will serve to heighten and extend
local and statewide conversations about the math and science
preparation of our future teachers. What do they think about
the results? How do they interpret them? as
"good"? "troubling"?
What do the OCEPT Management Team and the teacher education
leaders in the State think of the results?
Based on our experiences with the survey this first year,
some modifications in both the survey itself and in the way
it is administered are needed. More individuals from teacher
education programs at OCEPT institutions need to be involved
in helping to shape these modifications. The intention is
to administer the survey every year.
The survey administration process itself was very time consuming
and difficult. Local contacts had to be established at each
institution. An important by-product of this initial survey
experience has been to establish contact with the key teacher
education faculty at each of the institutions. We now have
a network with whom to communicate about the future survey
and the administration process. Also, individual faculty had
to be willing to take 25 minutes of class time to administer
the survey; and few had had any involvement in its development.
Even though the Human Subjects Committee at Portland State
University had reviewed the survey and the administration
and analysis plan and approved it, several other campuses
had to send the proposal through their own local committee,
thus delaying survey administration. Also, we thought we might
need student social security numbers for later follow-up purposes.
Students in several classes refused to participate in the
study because such information was called for. Although we
indicated to local contacts that having students participate
was the priority and to abandon asking for SSNs if necessary,
some students still would not participate. In the future,
SSNs will not be called for. A future instrument needs to
require less time to administer. And students and faculty
have to understand better the reasons for the survey and what
they may gain from participation. In the rush to collect some
data from students early in the life of OCEPT, we failed to
develop sufficient understanding about the survey and OCEPT
and developed insufficient ownership in the survey
process.
Even given the problems encountered, we are still optimistic
that the data set will provide added impetus for thoughtful
conversations about the preparation of our teacher educators
in math and science. We expect to develop an improved instrument
for use next year, with some carry-over of survey items.
Finally, we are analyzing a small data set to gauge instrument
stability over a two week period of time between a first and
second administration to the same group (n=14). Despite the
small number, the preliminary results look positive.
Return to Table of
Contents
Mid-Year Interviews
Interviews were conducted in February with all Co-Principal
Investigators and a sample of Faculty Fellows and Mentor Team
Members. A total of 25 individuals were interviewed by phone
by the Evaluation Coordinator. These 15-minute mid-year interviews
were designed to learn more about how Mentor Teams were functioning,
what was working especially well and where improvements were
needed; about how Fellows felt about the quality of support
they're receiving for their local projects; and about how
Co-PIs perceive the collaborative itself to be
developing.
A preliminary summary of some of the findings were reported
to Co-PIs and some Mentors who attended a recent all day OCEPT
Management Team retreat to help develop next year's strategic
plan. A final summary report will be made available to the
Management Team in April, 1998.
Return to Table of
Contents
Beginning in February 1997, a series of meetings were held
with individuals with an expressed interest in OCEPT evaluation
and research activities. Those attending included some Co-Principal
Investigators, some Mentor Team Members, several representatives
of the Oregon Department of Education, the Principal Investigator,
the Project Coordinator and the Evaluation Coordinator. By
early Fall 1997 the group had developed and reviewed a draft
Evaluation Plan for OCEPT. Copies of the draft were circulated
to all OCEPT Management Team members for their review and
comment. The draft was informed by the OCEPT grant proposal
as well as the Cooperative Agreement. The Research and Evaluation
Team then further scrutinized the draft for feasibility, given
the current resources available, and determined what should
receive priority. The Plan attached to the annual report will
undergo an additional review and discussion with the Management
Team at one of their Spring meetings.
Updates to the Plan may continue to be made in response
to any changes in focus or emphasis in OCEPT activities, as
determined by the Management Team. The overall evaluation
framework and major activities, however, are expected to remain
in place.
The Evaluation Plan describes both formative and summative
evaluation activities that address the major OCEPT goals.
The OCEPT Evaluation Plan places early emphasis on identifying
and describing how the project is contributing to faculty
development, Mentor Team development, early career teacher
development and the development of the Collaborative itself.
These are the major interventions intended over the longer
term to contribute to an improvement in the preparation of
teachers of math and science in elementary, middle and secondary
schools.
During this first year, however, we have collected some
baseline information about the undergraduate math and science
course experiences of students entering our teacher education
programs in Oregon. These data will be used to promote wider
discussion of how we are currently preparing our future teachers
in math and science and to assess changes over time in this
preparation. The initial Evaluation Plan called for the development
and administration of a survey of students exiting our teacher
education programs and a study over time of the actual classroom
practices of our graduates. Given an assessment of where OCEPT
might be expected to have the greatest influence over the
next five years, coupled with limited evaluation resources
and the variety of ways current teacher education programs
assess and follow-up with their own exiting students, these
two initiatives are on hold for now. More conversations are
needed with teacher education program leaders in the State
about both the initial instrument used this past Fall for
the first time to survey entering teacher education students
as well as how we might together begin to understand more
about the math and science preparation of students at the
time they exit our programs and the actual practices in classrooms
of our former students who have experienced OCEPT-influenced
faculty, classrooms and field-related experiences. These studies
need to be shaped by the teacher education community in the
State and have their endorsement and buy-in.
As mentioned previously, increased emphasis will be given
to how Faculty Fellows, Mentor Teams and the Collaborative
as a whole develop and extend their influence over the life
of the grant. More effort will be given to documenting Faculty
Fellow initiatives and the impact of their projects and other
OCEPT related experiences on the Fellows themselves, their
students, their colleagues and their institutions. Included
in evaluation activities related to the Fellows will be attention
to the connection these Fellows make between their role and
identity as faculty members and identifying, encouraging and
preparing future K-12 teachers.
The eight OCEPT Mentor Teams are one of the major ways the
project has organized itself to accomplish its objectives.
The development of these Teams over time will receive increased
formative evaluation attention. In addition, more attention
will be given to understanding how the Collaborative itself
is developing. Members of the Management Team will participate
in a special session this Spring designed to describe how
OCEPT defines collaboration and the specific forms of collaboration
that OCEPT values and is trying to promote (for instance,
intra-institutional collaboration; collaboration between faculty
in a discipline and those in colleges of education; inter-institutional
collaboration within the discipline; and collaboration on
behalf of the development of new kinds of professional communities;
etc.).
To document an "improvement", in a truly profound way, in
how we prepare future K-12 teachers of math and science in
Oregon, more than five years will be needed. While some evidence
of change that affects students will be possible over the
life of the grant, more time will be needed before we expect
to observe the kind of change that is the most important:
change in actual practices in K-12 classrooms. As a result,
the evaluation will attempt to document and assess OCEPT's
efforts to develop as a collaborative such that it can sustain
such efforts when the grant ends.
Return to Table of
Contents
|
 |
