Under-Represented Populations Stand-Alone Report 1 (Progress)

1995 Program Evaluation of the Women in Science Project at Dartmouth College

Return to Table of Contents

Previous Page

Appendix A: Design of Evaluation

Three interrelated evaluation strands structured the collection of data:

• Effects assessment investigated the impact of WISP in both qualitative and quantitative terms.
• Internship tracking probed the nature and dynamics of the internship experience.
• Career histories explored women's experiences in science during their four years in college to develop a more comprehensive picture of the challenges that they face.

The 1994-1995 WISP evaluation activities consisted primarily of:

1. A statistical analysis of data from all members of the classes of 1992-1994 that investigated whether there had been any change in the number of women majoring in science, and identified significant predictors of whether a student majors in science.
2. A survey of senior women '94 was distributed to all senior '94 women who had participated in a WISP internship and a random sample of other senior science majors who had not held an internship. This instruments collected background data, asked students to reflect about their experiences with science at Dartmouth and WISP activities, and asked about future plans.
3. A survey of freshwomen interested in science was distributed to both the WISP '97 interns and a random sample of freshwomen interested in science who did not hold internships. These interviews primarily functioned to corroborate journal data and explore some of the questions in more depth.
4. Interviews with freshwomen WISP '97 interns that probed the questions included on the freshwomen survey in more detail.
5. Senior '94 focus groups in which groups of 3 or 4 senior science majors met to discuss their experiences in the sciences at Dartmouth.
6. Interviews with juniors '96 who had initially expressed an interest in majoring in science but had decided to leave the sciences. These interviews asked about the reasons for their switch, the science support structures that they utilized or that would have been helpful, the differences they saw between science and humanities majors and classes, and what about the sciences discouraged potential majors.
7. Pre and post-internship survey of WISP '98 interns that closely resembled the questionnaires administered to the WISP '97 interns.
8. Nine internship journal questions that WISP '98 interns wrote biweekly throughout the course of their internship.
9. Three site visits to Dartmouth to attend the poster symposium in 1994 and 1995, an introductory information session about WISP internships, a WISP luncheon, and to speak with students, project administrators, and external advisory committee both formally and informally.
10. Review of other WISP materials such as the newsletter, other WISP publications and information and previous evaluations.

I analyzed the student-generated data on two levels, by question and by student. Principally, I compiled all the responses to one particular question to distill major thematic categories of responses. Each of the categories were assigned a numerical code. Then the questions were reread and students' responses were coded, enabling the qualitative data to be quantified. In addition, especially interesting replies were noted. This level of analysis grounds the statistics included in this report. However, I was also concerned with developing a more complete understanding of the relationship between a student's background, experiences, attitudes, and choices in science. Therefore, after analyzing the instruments by question, I reviewed students' questionnaires as a whole.

Return to Table of Contents

Appendix B: Statistical Analysis

The variables investigated in the quantitative analysis were:

• whether the student had a dual major
• SAT math scores
• whether the student postponed graduating a year
• graduation date
• gender
• GPA
• work study eligibility (a measure of financial need).

The dependent measure in the following analyses, SCIENCEMAJOR, is TRUE if a student majored in a science. For students with dual majors, SCIENCEMAJOR = TRUE if either or both of their majors were in a science.

First analysis of variance: Variation in SCIENCEMAJOR (Main effects only)

Purpose: To identify demographic covariates with student decision to major in science

Dependent variable: SCIENCEMAJOR Student majors in science, TRUE or FALSE

Independent variables:

	SATMATH	-Student's math SAT score, range 300-800
	GENDER	-Student's gender, MALE or FEMALE
	YEAR	-Year of graduation, 1992-1994
	WORKSTUDY	-Student's workstudy eligibility, a measure of financial need, TRUE or FALSE

Second analysis of variance: YEAR * GENDER interaction

Purpose: To determine whether the effect of gender is stable over three years of data.

Dependent variable: SCIENCEMAJOR

Independent variables:

	SATMATH
	GENDER
	YEAR
	WORKSTUDY

Third analysis of variance: Main effects and interactions with YEAR

Purpose: To determine whether gender effect changes from 1992 to 1994 can be attributed to changes in other demographics.

Dependent variable: SCIENCEMAJOR

Independent variables:

	SATMATH
	GENDER
	YEAR
	WORKSTUDY

Return to Table of Contents

Appendix C: Journal Questions

1. What were your initial impressions (about any aspect of the internship)? What has struck you most about your internship so far? What was different than you envisioned or expected? What were your gut or emotional reactions--how do you feel at the moment about your internship (overwhelmed, excited, unprepared, challenged, intimidated, etc.)?
2. This week, I'd like you to tell me about your role, tasks, and responsibilities in your internship. Picture that a friend has asked you to explain how your work fits into the larger purpose or research question(s) being investigated? You may not know all of the terminology as yet (but neither will your friend); describe your work in whatever terms you feel work best. How comfortable do you feel asking for more background information about your work? What has helped or hindered you in understanding the bigger picture? {If you would like, feel free to use this question as a means to initiate a relevant conversation with your sponsor.}
3. This week's focus is on communication in your internship. What are the interactions like with different members of your project or research team? What is the nature of YOUR communication, e.g., do you ask to have your work assignments or plans established for the week, do you always ask for help when you need it, have you asked any questions about the lifestyle of an academic researcher, have you asked for any academic advice, do you have purely social conversations ? What kind of communication is typically started by your sponsor or assistant sponsor? How do they treat you? As part of the "team?" How do the members of your lab or project keep each other informed about their progress and findings or get feedback? What kind of interactions occur between your research group or project personnel and their larger community (how do they disseminate or receive information from their colleagues at other institutions?)
4. This week, I'm interested in learning about some of the sources of stress and frustration that you have encountered in your internship. What has been frustrating for you? Please think about this in broad terms--you may or may not have control over them, they may or may not be science-related. For example, they might include: a commute, the lab lingo that you need(ed) to become familiar with, never knowing whether a trial or experiment will be successful until it is done, finicky procedures or instruments, dealing with co-workers etc. What sorts frustrations are you facing at the moment? If there was one thing that you could change what would it be? (Go for it, no need to be realistic!)
5. This week I'd like you to reflect about some of the non-scientific aspects and understandings that you are gaining from your internship. Has your internship or conversations with coworkers provided you with any new insights about academic classes or careers (that you would either be interested or uninterested in pursuing)? What kind of advice or mentoring do you get (or seek out) from your coworkers? Does anyone serve as a role model? What have you learned about yourself, your interests, and your likes or dislikes from the experience? (For example, the internship may have improved your responsibility or time management skills, you might have realized that you love working without much supervision, or perhaps that your future career will need to bring you into contact with many more people.) This question is highly individualistic and broad; please reflect on whatever you feel has been valuable for you to learn or realize.
6. In this journal, I'd like you to reflect on the issue of women in science, math, and engineering both on a personal and institutional level. What are your experiences or feelings about the issue of women in these fields? Have you perceived or experienced any differences, difficulties, or sexism in your classes (high school or college), in your internship, or in larger society? If so, please describe. Have you discussed the issue of women in science, math, or engineering either in the context of your internship or with other people? Why do you think that WISP exists; what purpose does or should it serve? Do you think that internships should be open to men? Why or why not?
7. This week I'd like you to reflect on the "culture" of science, math, and engineering. What have you become aware of about science in the course of your internship that the average citizen wouldn't know? If you were to explain to a friend how science operates or describe the culture of your internship site what would you say?
8. This week I'm interested in your reflections about a few different topics. First, how has the WISP experience influenced your personal views or attitudes about science and its status? Second, how has your internship experience expanded on or contradicted your experiences in college labs? (I realize that for some internships this question may not be relevant.) Third, (how) has the internship influenced your confidence in your scientific abilities?
9. Please use this last journal to review your internship as a whole. What reflections do you have regarding your preparation for the poster symposium? In more general terms: What were the highs and lows, the most rewarding and most frustrating aspects or experiences of the experience? What would you do differently? What are your thoughts about the WISP internship program overall: Did you get sufficient support? What would have been helpful to know before or along the way? What worked well? How could the project be improved? Feel free to include any other end-of-the-year reflections.

Return to Table of Contents

Appendix D: Questionnaires

WISP Questionnaire-Seniors

Name:

Mailing address:

When you entered Dartmouth what major(s) were you considering?

What is your major?

When did you decide to major in this subject?

Why did you decide to major in this subject?

Would you chose your major again? (if no, why not)

What three insights or tidbits of advice would you offer to a freshwoman who was planning to major in a science?

Describe the three factors that most encouraged you to major in a science. (parents, high school experiences, hobbies, role model, interest, project, job, research experience, future career plans, etc.) Please rank them (#1 most encouraging).

Describe the three factors that have most discouraged your pursuit of a science major. (difficulty of courses, size of classes, teaching, competition, image of science, emphasis on facts, peer and parental pressure, grades, etc.) Please rank them (#1 most discouraging).

We are interested in your "history" as a science major at Dartmouth. What were the influential activities, experiences, individuals, decisions, stumbling blocks, and triumphs that you encountered in various years? How did the years compare or differ? (Which years or quarters were the most difficult, the most enjoyable etc?) For each year, please write about five sentences characterizing or summarizing your experiences and attitudes toward majoring in a science.

Freshman year:
Sophomore year:
Junior year:
Senior year:

Internship Questions:

What did you learn from your internship experience?

Did your WISP internship influence: (if yes, please elaborate)

1. your decision to major in science?
2. your science course-selection at Dartmouth?
3. your perceptions of research science?
4. your future career plans?
5. your comfort and confidence with science?

How much contact have you had with your mentor or other members of your lab group since your internship?

WISP programming

What WISP programs have you participated in?

When? (What year of college-- 1=freshman)

How would you rate them overall?

YEAR RATING ACTIVITY 1,2,3,4 Excellent Good Fair Poor Internship Visiting Speakers Read Newsletter Student Panels/ Workshops Peer Mentors Site Visits Study Groups/ Study Room

Why did you participate in these programs?

Other than the internship, how did WISP programs impact your course-selection, pursuit of a science major, perceptions about science, future plans? Different years of a science major present different types of challenges and questions. A few programs that could address these are listed below. These were not necessarily structured programs, but could have been influential. Please indicate which you used (place a U in the box) or would have been helpful for you (place an H in the box) during your college career. Also, please add any other experiences or contacts that you utilized.

Activity Freshman Sophmore Junior Senior Older peer science mentor (for example through Older and Wiser, or an upperclass friend) Role model/ faculty mentor Study groups (organized through ACS or informally convened) Informal contact with fellow women science majors Career counseling Laboratory research (non-WISP) Summer job (please describe) Other (please describe) Other (please describe)

How did the contacts or experiences in the table above impact your course selection, pursuit of a science major, perceptions of science, and future plans?

Are there any other kinds of programs or support that you would have found helpful during your college science career?

Do you have any other recommendations for the WISP program?

What are your plans for the coming year?

What are your longer-term plans?

WISP Pre-Internship Questionnaires ('98)

Name:

Dartmouth mailing address:

Please list the science, math, and computer courses you took in high school.

Describe any other science and math-related experiences and honors that you had during high school (extracurriculars, volunteer work, jobs, awards etc.)

Have you had any experience doing scientific or mathematical research outside of high school labs? If so, please describe.

How well-prepared did you feel in science when you entered Dartmouth?

How well-prepared did you feel in math when you entered Dartmouth?

Who or what has interested you in science, engineering, or math?

When you entered Dartmouth last fall what major(s) were you considering?

What major(s) are you considering now?

Why?

How sure are you about this choice?

If you were originally quite certain about a major in science, math, or engineering, but have since begun to reconsider a major in one of these areas, what prompted this shift?

Please list the science, math, computer science, and engineering courses that you have taken so far this year and those you plan to take this term.

Why did you decide to take these courses? What sources of advice (professors, ORC, freshman advisor, WISP peer mentor, UGA, &'98s, upperclass students, etc.) did you utilize, if any?

Please briefly describe your experiences in and impressions of science courses at Dartmouth.

What future careers are you presently contemplating?

WISP-Related Questions:

How did you find out about WISP and the internship program?

Did WISP affect your decision to attend Dartmouth?

What other WISP programs, if any, have you participated in? How would you rate them overall? (Put one "X" in each row.)

Why did you participate in these programs?

Why did you apply for an internship?

Besides for the WISP internship, what other activities do you plan to participate in this term?

WISP Post-Internship Questionnaire ('98)

What major(s) are you considering now?

Why?

How sure are you about this choice?

If you last fall you were strongly considering a major in science, math, or engineering, but have since begun to reconsider a major in one of these areas, what prompted this shift?

Please list the science and math courses that you took this year and your grades.

Have these courses influenced your decision to continue or major in science? Please explain.

What science-related activities have you participated in this year other than a WISP internship (clubs, jobs, hobbies etc.)?

What science, engineering, and math courses do you plan to take next year?

What three insights would you offer to an incoming freshwoman who was planning to enroll in science courses?

Why did you decide to apply for an internship?

What are the five most important things that you learned from your internship?

Did your WISP internship influence: (if yes, please elaborate)

1. your decision to major or continue in science?
2. your science course-selection for next year?
3. your perceptions of research science?
4. the probability that you will seek out another laboratory research experience? (summer job, senior thesis, Presidential scholars, lab work)
5. your future career plans?
6. your comfort and confidence with science?

Who was the primary person that you worked with or reported to during your internship? (a faculty member, grad student, post doc, upper class student, other)?

How much contact did you have with him/her?

Do you feel that you had adequate guidance during your internship from your mentor (or whomever worked with you most closely)?

Were there any other WISP or student interns that worked in the same lab as you? If so, please describe your interaction with them (did they work on the same project, what were the benefits or detriments to working with another student?)

What was the most rewarding aspect of the WISP internship?

What was the most frustrating aspect of the WISP internship?

What advice would you give to a student who was about to begin an internship?

Do you feel that the support that you received from the WISP office was adequate?

Who would you recommend should apply for an internship?

What recommendations do you have for improving the WISP internship experience?

What WISP programs have you participated in? How would you rate them overall? (Put one "X" in each row.)

Why did you participate in these programs?

Do you anticipate participating in any WISP programming in future years? If so, how?

Other than the internship, how did WISP programs impact your course-selection, pursuit of a science major, perceptions about science, future plans?

Were there any other programs or people that you found useful to your science study the past year (an upperclass "mentor", a faculty mentor, study groups, etc.)

Are there any other kinds of programs or support that you WOULD have found helpful for your science study this past year?

Are there programs or other support structures that you think would be helpful to you for your science study next year or in future years (such as career counseling, exposure to role models or older peer science mentors etc)?

Do you have any other recommendations for or comments about the WISP program?

At the moment, what are you considering for your future plans?

At Dartmouth:

After Dartmouth:

Return to Table of Contents

Appendix E: Interview Protocol for Semi-structured Junior Interviews

When you entered Dartmouth as a freshman what majors were you considering?

Why?

What you are majoring in now?

How did you become interested this?

When did you decide X as a major?

What were some critical factors that shaped your choice of major?

Describe sciences you took at Dartmouth?

What was the most difficult time or experience you have had in science at Dartmouth?

What might have helped?

What kind of programs, counseling, support networks would have been helpful?

Describe the differences between the two majors.

Where do you go for support and advice about your major?

Describe WISP internship.

What did you learn from the internship?

Were you involved in any other WISP activities?

Future plans?

Thoughts about how more women major might be encouraged to major in science at Dartmouth? What turns them off?

Return to Table of Contents