Evaluation Plan
Table of Contents:
-
Personnel
- Evaluation Overview: Stakeholder Involvement, Evaluator Credibility
-
For the Course and Participants
- Design: Methodological Approach, Information Sources, Data Collection Procedures and Schedule
-
For the Project
- Evaluation Overview: Stakeholder Involvement
- Design: Methodological Approach
Personnel
Since the project is a pilot test for an academic course at a coastal facility, it is important for the evaluation to be conducted by individuals who are stakeholders in project success - for the academic program, for the facility's reputation, and for the future use of the course in other areas. Dr. Chris Stanton, Assistant to the Director of Ohio Sea Grant and faculty of the Entomology program at Ohio State, is a new instructor at Stone Lab. He would use this opportunity as a participant observer to learn how 21st Century interdisciplinary courses look in a field setting, and to evaluate the science/pedagogy combination as it is developed and implemented by the project staff. Dr. Stanton would have three primary tasks:
- monitor the progress of course development and project objectives over the year,
- direct an Honors student (undergraduate in Natural Resources) in collection and analysis of quantitative data from student progress in the course, and
- collection and reporting of naturalistic (interview and observational) data from students, project staff, and invited observers of the course.
Return to Table of
Contents
For the Course and Participants
Nontraditional instruction requires nontraditional assessment - no paper/pencil exams are planned. Instead, a number of formative assessments will be designed to give feedback to students on their progress in meeting course objectives for systems learning and in use of the electronic and field data collection and manipulation tools. For example, each student will assemble a physical or (preferably) digital portfolio demonstrating skills in use of the data, including but not limited to concept maps of relationships among Earth systems components in a given study (REF), photos or drawings of organisms or landforms key to understanding of an issue or process, satellite images processed according to a plan for visualizing certain attributes of a scene (REF), graphs or tables of field data relationships, and analysis and synthesis of arguments in an environmental issue. Some portfolio elements will be assigned and others may be added by the student to demonstrate extended learning or skill development. Portfolios developed on a class internet site will be accessible to all participants, a technique that typically results in higher quality science and greater attention to communication skills (REF), with the additional benefits of providing outreach for the program as well as student resumé material when the course is finished.
In addition, students working in small groups will pursue daily assignments together and undertake a term project in which each group member has particular responsibility for applying data for solution-seeking and decision making. Group process will be evaluated both quantitatively using a performance rubric such as those in Environmental Communication (Fortner 1999), and qualitatively with anecdotes and observations. Data will be collected daily by the Honors undergraduate and at least weekly by the evaluator, with attention to course progress and formative evaluation. Evaluators are members of the project team, and concerns raised in mid-course will be reported to the PIs so that personal and group performance issues can be addressed before they affect the learning environment and academic outcomes.
Return to Table of
Contents
For the Project
The evaluator will be included in all planning meetings of the project staff during the academic year preceding the course implementation. He will monitor the staff's
- attention to responsibilities of the project
- attention to diversity and equity issues
- use of effective recruitment and outreach mechanisms
- use of personnel and academic resources
- efficiency and timeliness for Stone Lab planning
His regular interactions with and reports to the project staff will help to maximize the potential for project success, and his evaluation notes will become part of the background information supplied to others attempting later versions of the course.
Return to Table of
Contents
|
 |
