Evaluation Proposal to Evaluate the Los Angeles
Collaborative for Teacher Excellence
This evaluation plan was prepared by an outside evaluator, Evaluation
& Training Institute, as a stand-alone proposal.
Table of Contents:
-
Introduction
- Project Description: Project Features
a. Evaluation of LACTE
- Evaluation Overview: Stakeholder
Involvement
- Design: Methodological Approach, Data
Collection Procedures & Schedule
b. Evaluation of Training Institute
- Evaluation Overview: Evaluator
Credibility, Stakeholder Involvement
-
Workplan
- Project Description: Project Features
- Design: Methodological Approach,
Information Sources & Sampling, Data Collection
Procedures & Schedule
- Analysis Process: Quantitative Analysis,
Qualitative Analysis
-
Bidder Qualifications
- Evaluation Overview: Evaluator
Credibility
Evaluation Proposal to Evaluate the Los Angeles
Collaborative for Teacher Excellence
Of California's nearly 32 million total population, half
of the adults lack the basic math skills required for today's
technologically based workforce. At the same time, the state
continues to be a national leader in fields of advanced technology,
telecommunications, robotics, geographic information systems,
and bioscience. Clearly the fit between labor market needs
and workforce skills requires that Californians develop greater
literacy in science and math. For the state's schools to educate
children to be productive, contributing adults, educators
must strengthen science and math curriculum and instruction
in public schools. Inadequate skills in these areas will prevent
even bright, motivated children from succeeding in the
workplace.
In addition, California's K-12 student population is increasing
at approximately twice the rate of population growth. Public
school enrollment is nearly twice that of the next-largest
state, New York. The proportion of non-white students in California's
public schools has increased from about 25 percent in the
late 1960's to more than 50 percent today. Hispanic students
now constitute one-third of all public school children in
the state. In urban areas, including Los Angeles County, the
figures are higher. In addition to challenges of basic math
and science education, these students often face language
and cultural barriers that negatively impact learning.
The National Science Foundation has funded the Los Angeles
Collaborative for Teacher Excellence (LACTE) to address the
issue of the inadequacy of science and math education. Specifically,
the program targets teacher preparation and the participation
of underrepresented students. LACTE brings together ten post-secondary
partners (five four-year institutions and five two year institutions)
in a consortium to address the shortage of both qualified
math and science teachers and quality instruction in elementary,
middle and senior high schools. LACTE's five-year program
specifically intends to:
- Build collaborative relationships and develop a network
of students, faculty and administrators;
- Recruit and train students in math and science, specifically
underrepresented students;
- Develop, revise and introduce new math and science
curricula;
- Provide faculty development opportunities that result
in improved classroom instruction and student
learning;
- Develop internships and other experiential activities
for students; and
- Disseminate program information to other undergraduate
institutions for replication.
Return to Table of
Contents
The proposed evaluation is designed to assess LACTE's effectiveness
in meeting these objectives. During Year One of the evaluation,
major emphasis will be placed on refining the evaluation design
and collecting baseline data against which to evaluate program
outcomes during Years Two through Five. In addition, rigorous
efforts will be made to provide on-going feedback to LACTE during
the course of the evaluation so that program adjustments may
be made as needed.
The evaluation is designed to fully incorporate the collaborative
nature of the program, with participation of students, faculty,
administrators, master teachers, mentors, employers, and
others.
The evaluation will utilize both qualitative and quantitative
research activities, with ETI providing on-going feedback
to all partners. During Year Five, in addition to documenting
program outcomes against baseline data, the evaluation is
designed to provide extensive documentation for program, replication
in other undergraduate institutions.
Return to Table of
Contents
The Evaluation and Training Institute (ETI) proposes to
undertake the five-year evaluation for LACTE. ETI has had
more than 21 years experience working with both secondary
and postsecondary institutions conducting evaluations in:
underrepresented student recruitment; teacher development;
curriculum reform; math and science education programs; longitudinal
studies; inter-segmental collaborations; and assessing performance
outcomes against baseline data. ETI has conducted numerous
studies for both The California State University and the Chancellor's
Office of the California Community Colleges, in addition to
evaluations for private California universities.
Specifically, ETI is conducting CSU's evaluation of the
California Academic Partnership Program's statewide efforts
to provide enriched curriculum for underrepresented students
by developing cooperative efforts between secondary and postsecondary
institutions in order to improve the academic preparation
of these students for higher education. In addition, ETI previously
evaluated four CSU programs whose objectives were the recruitment
and retention of underrepresented student in postsecondary
education. These included CSU's High School Outreach; Retention
Incentive; Faculty Mentoring; and Faculty/Student Mentoring
Programs.
Other examples of ETl's extensive evaluation experience
in the areas relevant to LACTE include:
- Current work with KCET and the Los Angeles Educational
Partnership in assessing the impact of the Math, Science
and Technology Professional Development Series on
teachers;
- A recently completed evaluation of the California Academy
of Physical Sciences program aimed at improving health and
physical sciences education in California;
- An award-winning, national study for the National Science
Foundation of academic employment patterns and trends of
women students and faculty in science and engineering;
and
- A current evaluation of Dow Chemical's US and Canadian
Hands-On Science Program for K-6 students and teachers,
including extensive teacher development and curriculum
components.
Additional qualifications of the Evaluation and Training
Institute as well as professional staff are presented in this
proposal.
Return to Table of
Contents
This section presents an overview of ETl's proposed evaluation
design for the Los Angeles Collaborative for Teaching Excellence.
Given the limited evaluation funds and the five-year duration
of the study, the activities are designed to optimize project
dollars. ETI has designed an evaluation which will give the
greatest return for the most cost-effective use of project
resources.
The Role of Evaluation Research Questions
Because ETI anticipates that the program will undergo changes,
we believe it is important to examine the basic evaluation
research questions each year.
ETI also believes that these questions will drive the specific
evaluation activities. For these reasons, we have developed
specific research questions that we believe should be addressed
in the evaluation. While we anticipate that this list will
be revised, it becomes a starting point at which to plan evaluation
activities.
Major Evaluation Activities
Overall, ETl's evaluation seeks to utilize both qualitative
and quantitative data to determine the extent to which
LACTE succeeds in meeting its major objectives during the
five-year course of the project. Specifically, the evaluation
is designed to determine to what extent is the project effective
in:
- Student recruitment and retention, including
underrepresented students;
- Faculty development, including opportunities and
outcomes;
- Student internships and other experiential
learning; and
- Support networks for students and faculty, both
across and between campuses?
The major activities that will occur in each of the five
years of the evaluation are:
- Review and revise evaluation research
questions
- Conduct on-site visits, focus groups and
interviews with program staff;
- Review program documents and
records;
- Design and distribute written surveys for
program participants*;
and
- Report findings, comparing project outcomes
against baseline data
Activities for each program year are outlined below. All
evaluation activities are designed to reflect the program
components that are emphasized for that year. These evaluation
activities will be under the direction of the LACTE project
director or designated monitor. In addition, ETI will work
closely with LACTE to revise or refine any evaluation
activity.
Year One: Building the Collaborative/Initial
Recruitment/Course Piloting
Major Program Activities:
- Student Recruitment
- Communication and Network Development (including ongoing
faculty recruitment)
- Faculty Development (including workshops)
- Course Piloting
- Collection of NSF Collaboratives for Excellence in Teacher
Preparation (CETP) Annual Information
Proposed Evaluation Activities:
The major evaluation activities during Year One will
be centered on refining the evaluation design and establishing
baseline data against which to compare subsequent program
outcomes.
- 1. Meet with LACTE and finalize evaluation design.
ETI will meet with LACTE staff to review and revise evaluation
activities outlined in this proposal. In addition, issues
such as deadlines, deliverables, coordination, etc. will
be discussed.
-
2. Refine evaluation research questions.
The evaluation research questions will drive the study. ETI
proposes to work closely with LACTE in developing the appropriate
questions for the evaluation. ETI anticipates that the research
questions will include questions such as:
- To what extent were faculty workshops effective
in presenting innovations in science and math courses?
Which components of the workshops were found to be most
effective? least effective?
- How were successful teacher recruitment strategies
identified? What strategies and procedures were developed
to identify and recruit math, science and liberal arts
students?
- Which recruiting strategies and procedures worked
most effectively for different types of students (based
on major, ethnicity, etc.)? What are the barriers to effectively
recruiting minority students? What are effective strategies
for overcoming these barriers?
- 3. Develop site visit interview guides that
reflect evaluation research questions. The interview
guides will be developed by ETI and reviewed by LACTE prior
to their use.
-
4. Develop and distribute workshop evaluation instruments
to:
- Assess pre-workshop attitudes and involvement; and
- Evaluate the impact and effectiveness of the
workshops.
ETI proposes to develop workshop evaluation instruments
and to analyze the findings. ETI proposes that LACTE be
responsible for distributing and collecting the survey
instruments.
- 5. Conduct on-site visits and interviews with program
participants using interview guides. ETI proposes
that all 10 sites be visited. ETI will attend workshops
and interview key staff, faculty participants, and students
individually and in focus groups. Student/mentor interactions
and meetings will also be observed and selected classes
will be observed.
- 6. Develop pilot course review instrument for faculty
and students. With LACTE, ETI will develop a course
review to be completed by both faculty and students at the
end of each pilot course.
- 7. Review student recruitment materials and strategy
with LACTE staff. ETI will review the recruitment materials
and their use at each site.
- 8. Analyze findings from workshop evaluation instruments
and course reviews. All surveys will be entered into
a computer database. ETI will analyze the data and document
findings.
- 9. Review program records of recruitment efforts and
outcomes for students, faculty, and master teachers.
ETI will collect and analyze recruitment activities and outcomes
at each campus, including data from the NSF Collaboratives
for Excellence in Teacher Preparation (CETP) annual information
survey. Also, recruitment vs. retention numbers will be
reviewed.
- 10. Identify where and why recruitment efforts have
been most effective. Based on the review of program
records, ETI will develop findings and recommendations as
to effective and ineffective recruitment and retention strategies
for students, faculty, and master teachers.
- 11. Present Year One findings to
LACTE. ETI will prepare for LACTE a summary of
findings at the end of Year One.
Year Two: Curriculum Component
Major Program Activities:
- Continue Student Recruitment
- Faculty Development Workshops and Technical
Coaching
- Course Development, Revision, and Piloting
- Continue Review of Elementary and Secondary
Programs
- Communication and Network Development (including continued
faculty recruitment and identification of Master
Teachers)
- Collection of NSF Collaboratives for Excellence in Teacher
Preparation (CETP) Annual Information
Proposed Evaluation Activities:
- 1. Meet with LACTE and discuss Year Two objectives
and activities. Based on Year One outcomes, ETI
will meet with LACTE staff to review and revise evaluation
activities.
-
2. Refine evaluation research questions.
ETI anticipates that the research questions will include
questions such as:
- How effectively did science and education faculty
work together in planning and teaching courses? What contributed
to effective working relationships? What were barriers?
How were barriers overcome?
- To what extent was the workshop teacher training
effective? What indicators exist of shared techniques
and goals across campuses? between campuses?
- To what extent were faculty and administrators
willing to work together to provide professional recognition
and rewards? Were rewards viewed as meaningful by teaching
faculty? Were institutional changes implemented that included
promotions and merit salary increases? Were these rewards
effectively conveyed to the faculty?
- 3. Develop site visit interview guides that reflect
evaluation research questions. The interview guides
will be developed by ETI and reviewed by LACTE prior to
their use.
- 4. Assess impact of collaborative participation on
untenured faculty involved in the program. With the
assistance of LACTE coordinators, ETI will identify a small
sample of junior faculty participating in the program in
order to track their professional trajectory over the following
3 years. ETI will deal with the sample on a case study basis
to evaluate whether program participation promotes or impedes
professional success.
- 5. Conduct on-site visits and interviews with program
staff using interview guides. All 10 sites will be visited.
ETI proposes to observe faculty training, interview program
staff, and observe classroom instruction. In addition, ETI
will conduct focus groups with students enrolled in pilot
courses.
- 6. Review faculty and student recruitment strategies
with LACTE staff. ETI will review the recruitment materials
and their use at each campus.
- 7. Administer pilot course review and workshop
surveys and analyze results.
- 8. Review program records of student and faculty recruitment
and retention efforts. ETI will collect and analyze
Year Two recruitment and retention activities and outcomes.
Among the records that ETI anticipates reviewing for the
students are changes in underrepresented students' enrollment
in higher level math and science courses (by program year),
course performance, and enrollment in next level course.
(These data will be tracked over the five years of the evaluation.)
ETI will also include data from the NSF Collaboratives for
Excellence in Teacher Preparation (CETP) annual information
survey.
- 9. Continue to identify where and why recruitment efforts
have been most effective. Based on the review of program
records, ETI will continue to develop findings and recommendations
as to effective and ineffective recruitment and retention
strategies for students, faculty, and master teachers.
- 10. Present Year Two findings to LACTE.
ETI will prepare for LACTE a summary of findings at the
end of Year Two.
Year Three: Course Dissemination/Student Work
Opportunities
Major Program Activities:
- Continue Student Recruitment
- Faculty Development Workshops and Technical
Coaching
- Course Development, Revision, and Transport
- Continue Review of Elementary and Secondary
Programs
- Communication and Network Development (including
continued faculty recruitment and identification of Master
Teachers)
- Initiate Student Intern Projects
- Submit Revised Elementary and Secondary Programs
- Course Dissemination (through videos, professional
papers/presentations, and teleconferences)
- Collection of NSF Collaboratives for Excellence in
Teacher Preparation (CETP) Annual Information
Proposed Evaluation Activities:
- 1. Meet with LACTE and discuss Year Three objectives
and activities. Based on Year Two outcomes, ETI will
meet with LACTE staff to review and revise evaluation
activities.
-
2. Meet with LACTE to refine Year Three evaluation
research questions. ETI anticipates working closely
with LACTE to determine the project's evaluation priorities
for assessing the pilot courses and faculty development
component. ETI anticipates that the research questions will
include questions such as:
- Was the Faculty Development Component effectively
planned, staffed, implemented? What indicators are there
of overall program effectiveness? How were faculty recruited
to the program? What factors contributed to their retention
in the program?
- How were faculty development leaders identified?
trained? To what degree were they effective in their roles?
What evidence exists of their effectiveness?
- To what extent were faculty development workshops
effective in reaching the target audience? How were faculty
recruited to the workshops? Did faculty members benefit
from the workshops? Was this reflected in students' classroom
experience?
-How did the students experience the teaching internships?
Were "employers" pleased with the preparation
given the students? How much of the students' teacher
training was directly employed in their intern
experience?
-How effectively were courses transported to the
other institutions? What differences, if any, were observed
in the implementation of courses to different student
populations?
- 3. Develop site visit interview guides that reflect
evaluation research questions. The interview guides
will be developed by ETI and reviewed by LACTE prior to
their use.
- 4. Continue to track sample of untenured faculty involved
in the program.
- 5. Conduct on-site visits and interviews with program
staff using interview guides. All 10 sites will be visited.
ETI proposes to observe faculty training, interview program
staff, and observe classroom instruction. Particular attention
will be given to the observation and discussion of pilot
courses transported to other institutions. In addition,
ETI will conduct focus groups with students. In Year Three,
ETI will specifically question students about their internship
experiences.
- 6. Administer pilot course review and workshop surveys
and analyze results.
- 7. Review program records of student and faculty recruitment
and retention efforts. ETI will collect and analyze
Year Three recruitment and retention activities and outcomes,
including data from the NSF Collaboratives for Excellence
in Teacher Preparation (CETP) annual information
survey.
- 8. Continue to identify where and why recruitment efforts
have been most effective. Based on the review of program
records, ETI will continue to develop findings and recommendations
as to effective and ineffective recruitment and retention
strategies for students, faculty, and master teachers.
- 9. Review program records of pilot course enrollment
and faculty workshop participation. ETI will collect
and analyze school records of course enrollment as well
as the attendance at faculty development workshops. These
will be analyzed by campus, student major, faculty teacher
area, and participant ethnicity, among other factors.
- 10. Identify and assess dissemination activities with
LACTE. ETI will discuss and evaluate the development
of materials for dissemination with program staff.
- 11. Present Year Three findings to
LACTE.ETI will prepare for LACTE a summary of
findings at the end of Year Three.
Year Four: Continued Course Dissemination/Student Work
Opportunities
Major Program Activities:
- Continue Student Recruitment
- Faculty Development Workshops and Technical
Coaching
- Course Development, Revision, and Transport
- Continue Review of Elementary and Secondary
Programs
- Communication and Network Development (including continued
faculty recruitment and identification of Master
Teachers)
- Initiate Student Intern Projects
- Submit Revised Elementary and Secondary Programs
- Course Dissemination (through videos, professional
papers/presentations, and teleconferences)
- Collection of NSF Collaboratives for Excellence in Teacher
Preparation (CETP) Annual Information
Proposed Evaluation Activities:
- 1. Meet with LACTE and discuss Year Four objectives
and activities. Based on Year Three outcomes, ETI will
meet with LACTE staff to review and revise evaluation
activities.
-
2. Refine evaluation research questions.
Again, appropriate evaluation research questions will drive all
evaluation activities. The evaluation research questions
may include questions such as:
- What opportunities were developed for student internships?
How were students recruited to participate in these opportunities?
Did the opportunities represent the diverse interests
of students? Were appropriate mentors identified at these
sites?
- What roles did master teachers play in mentoring
students? In what ways did master teachers enrich students'
learning? What types of learning experiences did master
teachers provide for students? What were the characteristics
of the most successful master teacher-student
relationships?
- What additional professional development opportunities
were identified and utilized? What type of follow-up training
was viewed as needed? was viewed as beneficial?
- 3. Develop site visit interview guides that reflect
evaluation research questions. The interview guides
will be developed by ETI and reviewed by LACTE prior to
their use.
- 4. Continue to track sample of untenured faculty involved
in the program.
- 5. Administer pilot course review and workshop surveys
and analyze results.
- 6. Conduct on-site visits and interviews with program
staff using interview guides. ETI proposes to interview
master teachers, campus faculty, LACTE program administrators
and students participating in internship. ETI will also
conduct focus groups of students, master teachers, mentors
and faculty.
- 7. Develop, implement and analyze student, mentor and
master teacher surveys. ETI will develop, implement
and analyze data from students, mentors and master teachers
participating in pilot/new courses and internships. ETI
will work with LACTE and faculty to determine the appropriate
content of surveys.
- 8. Develop, implement and analyze surveys from professional
development workshops and other faculty activities. All
surveys will be entered into a computer database. ETI will
analyze the findings and document findings.
- 9. Review program records of recruitment efforts and
outcomes. ETI will collect and analyze recruitment activities
and outcomes for new and pilot courses, student internships,
faculty development activities, etc. In addition, student
and faculty retention numbers will be reviewed. Data will
include information from the NSF Collaboratives for Excellence
in Teacher Preparation (CETP) annual survey.
- 10. Identify where and why student, faculty, mentor
and master teacher recruitment efforts have been most
effective. ETI staff will review program records
and develop findings and recommendations as to effective
and ineffective recruitment and retention
strategies.
- 11. Identify where and why student internships have
been most effective. ETI staff will review interviews,
surveys and program records and develop findings and recommendations
as to effective and ineffective internship strategies.
- 12. Present Year Four findings to LACTE.
ETI will prepare for LACTE a summary of findings at the
end of Year Four.
Year Five: Project Dissemination
Major Program Activities:
- Continue Student Recruitment
- Faculty Development Workshops and Technical
Coaching
- Course Development, Revision, and Transport
- Continue Review of Elementary and Secondary
Programs
- Communication and Network Development (including
continued faculty recruitment and identification of Master
Teachers)
- Initiate Student Intern Projects
- Revise Elementary and Secondary Programs Submitted in
Year 3 & 4
- Course Dissemination (through videos, professional
papers/presentations, and teleconferences)
- Collection of NSF Collaboratives for Excellence in Teacher
Preparation (CETP) Annual Information
Proposed Evaluation Activities:
The major evaluation activities during Year Five will
focus on: documenting program outcomes against Year One baseline
data; and providing extensive documentation for program replication
in other undergraduate institutions.
- 1. Meet with LACTE and discuss Year Five objectives
and activities. Based on Year Four outcomes, ETI will
meet with LACTE staff to review and revise evaluation
activities.
-
2. Refine evaluation research
questions.ETI staff anticipate that the
evaluation research questions will include questions
such as:
- To what extent was a project manual developed that
would allow other campuses to duplicate LACTE's activities?
To what extent were evaluation findings reflected in the
manual so as to assist other projects? How was the manual
publicized? disseminated?
- To what extent were teleconference videos developed
that would assist other campuses to implement a teacher
excellence program? How were the videos publicized? disseminated?
How were these videos evaluated by other academics? To
what extent was the teleconference series effective? What
indicators exist of its effectiveness?
-To what extent did LACTE participating faculty and
administrators utilize state, regional and national meetings
of professional organizations to provide information on
LACTE? To what extent did these meetings lead to increased
interest in the project? What indicators are there that
state, regional and national meetings are effective dissemination
outlets?
- What additional dissemination outlets were identified?
How were these used? What impact did they have?
- Overall, to what extent did LACTE succeed in meeting
major objectives during the five-year course of the
project? Including, to what extent was the project effective
in:
- Student recruitment and retention, including
underrepresented students;
- Faculty development, including opportunities and
outcomes;
- Student internships and other experiential learning;
and
- Support networks for students and faculty, both
across and between campuses?
- 3. Develop site visit interview guides that reflect
evaluation research questions.The interview guides
will be developed by ETI and reviewed by LACTE prior to
use. These guides will reflect the overall objectives
of the study and address follow-up issues related to
research questions from previous years.
- 4. Develop and distribute evaluation
instruments. ETI proposes to develop evaluation
instruments that reflect the overall objectives of the
project. These will be customized for each group of
participants: students, faculty, master teachers, mentors,
administrators, teleconference participants, manual users,
etc. To maximize evaluation resources, ETI
proposes that LACTE be responsible for distributing and
collecting the survey instruments. All surveys will be entered
into a computer database.
- 5. Conduct on-site visits and interviews with program
staff using interview guides. ETI proposes that all
10 sites be visited. ETI proposes to view videos, attend
the teleconference, interview key staff, faculty participants
and students, etc. ETI will conduct focus groups when
appropriate.
- 6. Analyze final student enrollment and internship
figures based on major, ethnicity, campus, course enrollment,
etc. ETI proposes to utilize campuses' databases as
well as program records. Specific analyzes will be conducted
comparing baseline and outcome data.
- 7. Review program records of faculty recruitment efforts,
new and pilot course offerings, and outcomes.
- 8. Identify where and why recruitment efforts have
been most effective. Based on the review of program
records, ETI will develop final findings and recommendations
as to effective and ineffective recruitment and retention
strategies for both faculty and student participation.
- 9. Identify effective student internship and other
experiential learning; opportunities and practices. ETI
will conduct interviews with and survey mentors, students,
faculty and others.
- 10. Analyze career path of sample of untenured faculty
involved in the program. ETI will assess the impact
of program participation on these faculty members.
- 11. Present Final Report to LACTE. ETI will prepare
for LACTE a final report of Years One through Five. This
report will address all research questions and present both
qualitative and quantitative data. Specific attention will
be given to comparing Year One baseline and Year Five outcome
data. The report will include both evaluation findings and
recommendations. The final report will be available in both
hard copy and electronic format. In addition, ETI will provide
LACTE with all data file, record layouts, and other project
documentation.
Return to Table of
Contents
Included in this section of the proposal are descriptions
of the Valuation and Training Institute and relevant consulting
experience. In addition, references, staff resumes and the
proposed organization chart for the LACTE evaluation are
presented.
Introduction to ETI
The Evaluation and Training Institute (ETI) is a full-service,
non-profit consulting firm headquartered in Los Angeles, with
a satellite office in San Francisco. ETI's staff is multidisciplinary
and multicultural, and includes professionals with backgrounds
in eduction, evaluation and research methodology, public administration,
management, statistics, psychology, sociology, political science,
and urban planning. In addition to its regular staff, ETI
employs consultants, specialists and research assistants on
both a continuing and supplemental basis. Advisory Board members
also serve as consultants in the development and conduct of
specific projects.
ETI is a 501(c)3 corporation whose policy direction is provided
by the officers of the corporation in consultation with eighteen-member
Board of Advisors. The Board is composed on national and international
leaders in education, social science and related fields.
Collectively and individually, ETI senior staff have directed
more than 500 studies and projects covering organizational
educational and social policy issues in such areas as program
evaluation, criminal justice, cost/benefit analyses, needs
assessment, management audits, and health management. Educational
projects have included all levels from projects in secondary
schools through adult, alternative and professional
education.
ETI's Consulting Approach
Our approach to consulting is "client-driven" and
unique in that it helps organizations produce concrete
evidence intended to:
- Document program effectiveness and success for both internal
and external review;
- Provide a systematic approach to reviewing and redirecting
activities;
- Identify potential problems as well as areas where it
is possible to build upon current or past success;
- Indicate underutilized resources that can be activated
to meet program goals;
- Integrate evaluation findings and recommendations in
strengthening organizational policies, practices and programs;
and
- Develop on-going procedures and tools that remain in
place after the consulting team leaves.
In addition, three central elements distinguish our
consulting services from traditional approaches:
- Every consulting project is developed specifically
for each client. There are no canned or formula approaches
utilized in any consulting project. This customized approach
incorporates the unique attributes of each client and the
environment in which the organization operates.
- Objectives and criteria for every project are
developed with professionals and internal staff working
together. These objectives and criteria
incorporate the practical concerns of managers and staff.
In additional, the outcome measures are designed to be
appropriate to the program's environment, and to the
particular action involved.
- Qualitative, as well as quantitative information,
is integrated both in design and implementation. By
integrating the most appropriate quantitative and qualitative
methods, clients obtain a more comprehensive and accurate
perspective of the organization's effectiveness.
Facilities
ETI's corporate headquarters in located at 12300 Wilshire
Blvd., Suite 420, Los Angeles, California. Facilities include
small and large conference rooms, a library, data processing
center, graphic design center, business office, focus group
and viewing rooms, and private offices for mangers, consultants,
and support staff.
ETI's computing capabilities include IBM and IBM-compatible
personal computers, including Intel chip-driven pentium computers,
along with laser and color graphics printers. The computer
network at ETI is capable of handling large databases and
undertaking complex statistical analyses. ETI hardware is
able to read and write to a multitude of storage media including:
5.25 and 3.5 inch floppy disks, DC2120 tape cartridges, 9-track
tapes (at 6250 and 1600 BPI), 4mm DAT cartridges, and CDs.
A dedicated "number cruncher" computer has 10 gigabytes (10,000
MB) of hard drive space. For ease of data transfer, ETI has
four dedicated modem lines and 28,800-baud modems, and operates
its own electronic bulletin Board System (BBS). In addition
to its highly sophisticated computing resources, ETI has programmers,
analysts, and designers on staff capable of complex programming,
data analysis, graphic design, etc.
ETI's computer system supports a variety of word processing,
graphic, and statistical software packages. The software utilized
by ETI allows the firm to produce reports and documents that
are "camera ready" for clients seeking to reprint and distribute
directly to board members, the media, internal staff, customers,
and others. ETI staff work closely with clients to ensure
that reports and documents utilize the graphs, charts, illustrations,
etc. that best suite the text and client's needs. These graphs,
charts and illustrations are available to clients in full
color.
ETI has 486 notebook computers for our use off-site, and
for presentations. Also in use are state-of-the-art presentation
systems, including a LED-data display panel and high intensity
projection system.
ETI has a state-of-art Novel network v 4.1) which allows
staff to provide clients with more efficient file and print
services. The network allows for dial-in and dial-out access
from every workstation, enabling access from off-site. This
allows both ETI staff and clients to share files when needed.
The system-wide electronic mail allows staff to transfer files
and correspondents both internally and externally. Faxes may
also be send and received directly from work stations.
ETI also operates a dedicated FAX machine which is on-line
24-hour-per day, 7-day-per week. The firm is therefore able
to respond immediately to clients' needs via FAX as well as
high-speed modem.
In the area of office management, ETI's billing and internal
management controls are all computerized. Project managers
received weekly printouts of expenditures, projected costs,
remaining budget, etc. Additional office procedures are also
computerized, e.g. employee worklogs, project timesheets,
expense reports, etc.
Overall Management Structure
ETI has considerable experience in the successful management
of large-scale projects. Specific management procedures have
been developed which keep our work on time, on budget, and of
high technical quality. ETI views the project management function
as having three objectives:
- Producing highly professional and high quality technical
work and deliverables;
- Completing contract work on time and on budget;
and
- Staffing projects so that the needs of the project
are met, including goals, objectives, budgets, timelines,
etc.
The project director is responsible for making sure that
these three management goals are met. For the LACTE evaluation,
ETI President Clare Rose will serve as the project director.
The project director is continually supported by experienced
senior colleagues for administrative, substantive and personnel
concerns. Assisting Dr. Rose in this evaluation will be project
manager Dr. Saskia Subramanian. (Additional key ETI staff are
indicated on the project organization chart. Resumes of all
key staff are included in this proposal.) A set of regular administrative
procedures take place which provide ample opportunity to review
progress and problems, plan strategies for on-going and future
tasks, and resolve conflicts among competing project
goals.
Administrative Management
Scheduled every week is a meeting of the project director,
project manager and all project staff. These meetings focus
on a variety of issues, both contractual and technical, and
involve discussions which lead to action. The emphasis in these
meetings is on problem solving and project success.
During these reviews, then topics are covered: 1) evaluation
research questions; 2) staffing; 3) research design issues;
4) policy concerns; 5) project schedule; 6) deliverables and
dissemination; 7) relations with contractor; 8) formal contract
compliance; 9) field activities and data collection; and 10)
project budget. Progress and activities over the last several
weeks are reviewed and plans for both the short term and long
tern are discussed.
ETI believes in maintaining close, collegial coordination
with all key members of the team as well as the project monitor.
We have developed a highly successful coordination
system.
Substantive Review
Each project has a Technical Quality Reviewer appointed
by the ETI President at the project's outset. The Technical
Quality Reviewer is a senior staff member who provides the project
director with internal feedback on the quality of both the substantive
planning and approach to the work and the deliverables produced
by the project. The Technical Quality Reviewer for the LACTE
evaluation will be Mary Kay Stout. No deliverables may be sent
to a client without Technical Quality Reviewer signoff.
At the beginning of each major project, the project manager
conducts a one-day technical colloquium for all ETI research
staff. In these technical colloquia, the overall plan for
the project is discussed, its objectives and activities are
reviewed, and staffing and budgeting details are outlined.
These sessions are specifically useful in the anticipation
of potential project problems. The Technical Quality Review
works with the project staff during start-up and assists in
the development of specific benchmarks and milestones for
the project.
Financial Accountability
ETI has a strong track record of sound financial accountability
both internally and externally. Internally, ETI senior managers
receive the weekly financial statements for each project as
well as the entire organization. These are reviewed at a weekly
staff meeting of senior managers. In addition, the ET office
manager, external accountant, and president closely scrutinize
the firm's financial statements on a quarterly basis.
Return to Table of
Contents
|
 |
