
	: Reports : Curriculum Development

Curriculum Development Stand-Alone Report 2 (Final)

Return to Curriculum Development Reports

NSF CCLI-A&I Pilot Project Evaluation

Return to Table of Contents

Global Conflict — IR 326

Focus Groups

Eight of nine students enrolled in IR 326 in Fall 2000 participated in the discussion group. Five students (63%) specified the course was required for their International Relations degree, and no students knew about the module component prior to enrollment.

Module Relevance

Students in this course did not identify the Global Conflict events module as a unique Web-based tool, but instead referred to it as a semester-long set of research and analysis exercises. Students said they had been instructed to complete these data collection assignments on an international events topic of their choice using primarily the Lexis-Nexis world news search engine. Thus, according to students, whether the module was fully integrated, supported other components of the class or was relevant to overall course material depended largely on the area of conflict selected. However, although most students felt the exercises per se were not particularly integrated with the texts, lectures or exams, this lack of cohesion with other class components was not necessarily viewed as a drawback.

Most students (75%) felt the primary academic and professional benefit the exercises offered was to help them conduct basic research and analyze their topics in-depth. Additionally, students were aware of and expressed much interest in the instructor's proposal to make a Web-site available where students would then communicate their research findings to others. They felt this feature would have increased the cohesion between their individual work on module exercises and the course as a whole.

Content and Function

Students reiterated that the content of the events module exercises depended on students' chosen area of research. Although a few found the exercises "sometimes tedious" and very time consuming, about half indicated the level of work required both engaged and interested them. In general, students found the course to be overall "pretty challenging," and the events exercises, though time consuming, appealed to them conceptually. In large part, the area students chose to research determined the degree of challenge, and all agreed they were free to change topics at will. A few especially appreciated the "independent study" nature of the exercises, though one student felt more examples would have been helpful. One student offered, "I like that you could do it how you wanted to," and another agreed, "I like that you could choose your own country."

Overall, students agreed that the primary challenge and reward in the exercises involved gathering, assimilating and analyzing large quantities of data. No students felt their ability to communicate information had been enhanced. All agreed their prior computer skills were more than sufficient to perform the exercises; moreover, 63% stated they had had enough previous knowledge of Lexis-Nexis to perform the exercises without formal instruction. Two students suggested they would have benefited more from increased exposure to a wider variety of tools and resources, as merely using computers to access information was not novel enough to stimulate their interest.

Module Effectiveness Surveys

Characteristics of Participants

Surveys were administered to all nine students enrolled in IR 326 in Fall 2000. Participants ranged in age from 21 to 28 (M=24.0, SD=2.7), and 78% of the class was male. Of 7 participants who provided ethnicity data, 2 students (29%) identified as Asian, 2 as Filipino, 2 as White, and 1 (14%) as African American.

The majority of respondents (78%) were currently employed at least part-time and worked from 10 to 50 hours per week (M=24.4, SD=13.8). Five participants (56%) were seniors and four (44%) were juniors. Slightly over half of the class (56%) identified as International Relations majors, while 2 students (22%) were pursuing undergraduate degrees in computer science.

In general, students of IR 326 reported medium to high prior experience and interest in the Internet, research, international events and foreign policy, with no apparent differences based on sex, age, class level or ethnicity. However, one third of the class characterized their research experience as low and half the class reported low prior exposure to Lexis-Nexis.

Frequency of Use

Students' reported module use ranged from 0 to 15 or more times during the semester with an average use of 3-4 times. Although 3 of 9 students (33%) indicated they had not used the module at all, their pattern of responses to other survey items indicated they had completed the events module exercises as assigned. It may be some students interpreted the term "module use" as the number of times they had fulfilled events module assignments, while others referred to a more comprehensive web-based instructional tool the instructor had described to them. Thus, given the potential ambiguity of this measure as it relates to IR 326 responses, no further analysis was conducted on frequency of use.

Student Satisfaction

While students reported only moderate satisfaction with the overall quality of the module exercise (M=3.78 SD=.83), they strongly believed the exercise was applicable to the course (M=4.44 SD=.53) and that its purpose was clearly integrated with the goals of the course (M=4.22 SD=.44). Although students also felt they had received sufficient instruction to do the exercise effectively (M=4.00 SD=.71), only half the class agreed the events module was technically easy to perform. Overall satisfaction did not vary by sex, age, class level or ethnicity, but students who worked more hours per week tended to report higher satisfaction.

Those students who found the events module highly integrated with goals of the course also felt the exercise was not too difficult (r=.68 p<.05) and reported more overall satisfaction with the module's quality (r=.83 p<.01).

Impact on Learning

In general, students moderately agreed the module had a positive impact on their learning. For instance, the class reported they were somewhat motivated to learn more about international relations due to the module exercise (M=3.67 SD=.71), and they moderately believed their interest in social science inquiry and confidence in accessing information had been enhanced by the module exercise (M=3.56 SD=1.13). Impact on learning did not vary by age or ethnicity, but juniors and women tended to rate these dimensions somewhat higher.

Students' self-ratings of pre- and post-class experience on five course-related dimensions as measured by Q13 is illustrated in Figure 3. Students' perception of their experience and/or interest in the Internet, international events, research, and foreign policy rose only slightly after the course, as over half the class reported high pre-class experience on all four dimensions. More dramatic change was reported for student experience with Lexis-Nexis, as two thirds indicated high after-class experience on this dimension. A composite measure of self-rated experience, derived by summing Q13 scores on all five dimensions, increased slightly from 11.25 (before this class) to 13.33 (after this class) on a scale from 5 (low experience) to 15 (high experience).

Students who strongly agreed the module exercise had improved skills relevant to their careers also felt it had increased their confidence in accessing information (r=.96 p<.01) and enhanced their interest in social science inquiry (r=.86 p<.01). Those who felt the exercise had increased their confidence in accessing information strongly believed their interest in social science inquiry had been enhanced (r=.90 p<.01).

Strengths and Weaknesses

Three of nine respondents (33%) provided feedback on the strengths of the module exercises. One student felt the exercises made it possible to "learn a lot of information." Another felt they were organized, and a third appreciated the comprehensive chronological overview presented by the "first four rubics."

Four students (44%) commented on weaknesses. Two students indicated the exercises were somewhat tedious and required covering too much information in a short time period. One student felt Lexis-Nexis was too limited and suggested incorporating a broader range of online resources into the exercises. The fourth student felt some of the terms used to analyze the material gathered were confusing and were not relevant or applicable to all topic areas.