CCLI Annotated Report Excerpts

Excerpt 1 [Collaborative Research: Developing and Implementing Just-in-Time-Tracking (JiTT) Techniques in the Principles of Economic Course, North Carolina A &T State University]

Interpretation & Conclusions

Characteristics of students in the JiTT and non-JiTT groups for each exam indicated some differences between the two groups. One group (Group A) had a lower percentage of female students throughout the semester, as well as higher SAT scores, higher GPAs, and fewer credit hours completed. These factors may be responsible for Group A earning higher exam scores on each of the exams, even when Group A was not completing JiTT assignments; however, group A scored slightly lower than group B on the JiTT-related exam questions when group B was completing JiTT assignments (exam #2). The differences in student characteristics across groups highlight the need to control for these variables, in particular students' knowledge entering the course (as captured by SAT scores and GPA), when examining the learning benefits of JiTT during the semester.

When controlling for demographic and academic factors, regression analysis for each of the exams indicates that completing JiTT exercises prior to class has a weak positive effect on student exam performance, both on JiTT-related questions and overall. However, JiTT participation does not explain the whole story of student exam performance. Students in the non-JiTT group performed better than students in the JiTT group on one of the exams (exam #2), although there was still a weak positive JiTT-group effect when additional student characteristics were added to the regression model. One factor possibly contributing to the weakness of the JiTT effect on exam #2 is that students who participated in the JiTT group for exam #1 (group A) may have changed their study habits based on their JiTT participation, leading to a 'residual JiTT effect' on exam #2 (and exam #3) that weakened the JiTT-group effect on this exam. It is also possible that students in Group B would have done even worse if they had not been doing the JiTT exercises during the period leading up to exam #2. Because of this possible residual effect, there may be reason to focus more on the results from exam #1, which obviously contain no such effects.

Student Perception of Effort and Learning

To better understand the role of JiTT exercises from students' perspectives, students were asked in an end-of-semester questionnaire to rate a variety of course activities in terms of 'how helpful the activity was to learning in the course,' and 'how much effort the activity required' during the semester. Mean student responses (based on 38 students who finished the course) are listed in Table 3 (on a scale of 1 to 5, with 5 representing the 'most helpful' or 'most effort'). JiTT questions were rated as being important for students' learning in the course (on an absolute scale) but not quite as helpful as homework assignments and about as helpful as class lectures, various in-class activities, and overall use of web resources (including the Blackboard course management system) in the course. The numerical results are consistent with students' written comments - which were generally positive. Students indicated that the JiTT component of the course should be retained in future semesters.

In terms of effort required, students ranked JiTT exercises in the middle of the pack, relative to other activities in the course. The finding that students viewed JiTTs as not requiring a great deal of effort, coupled with their view that they were important to learning, also led us to think about the relationship between student effort (or at least students' perception of that effort) and activities that were Final Report: 0088303 Page 6 of 12 perceived to provide the most help in learning. The data clearly show a positive correlation between students' perceived effort for various course activities and students' perceptions of those activities' aid to learning, and further, the results suggest that JiTT exercises provide an efficient way, relative to other types of course-related assignments, of helping students to achieve greater learning.