Printable Version
Web Link - http://www.cur.org/Publications/AIRE_RAIRE/michigan.asp 


 
Assessment and Evaluation of Innovative Programs: 
Measuring their Impact

Russel S. Hathaway, College of Literature, Science and the Arts
Sandra R. Gregerman, Director, Undergraduate Research Opportunity Program 
Cinda S. Davis, Director, Women in Science and Engineering

University of Michigan
Ann Arbor, MI 48109


The Undergraduate Research Opportunity Program (UROP) was developed in 1988 to increase the retention and improve the academic performance of underrepresented students of color at the University of Michigan through student/faculty research partnerships. Today, the program is open to all undergraduate students with a continued emphasis on underrepresented students and a focus on young women in the sciences. The program was opened to all students in 1992, in part because of increasing criticism of undergraduate education at the state and national level. The integration of research and undergraduate education through UROP addressed the mounting concerns of students and state legislators that undergraduate education was the poor stepchild of research. By engaging lower-division students at the University of Michigan more directly with faculty in the wealth of research activity taking place at the University, it was felt that all students and especially students of color, would become better integrated into research and develop skills that would transfer to greater success in academic coursework. From its inception, UROP had an assessment and evaluation program that focused primarily on gathering data on the program's effect on retention and academic achievement.
 
The retention study was initially funded by the came from the U.S. Department of Education’s Fund for the Improvement of Postsecondary Education. Funding from the NSF-RAIRE award enabled UROP to continue to conduct the additional assessments described below. NSF-RAIRE funds also supported the University of Michigan’s Women in Science and Engineering Residential Program that houses 150 women interested in the sciences and engineering in a co-ed residence hall on campus and offers them a supportive learning environment. This portion of the RAIRE award facilitated the development of some joint surveys and focus groups that are still underway and will not be discussed in this chapter.

Assessment and Evaluation

The UROP evaluation and assessment program is structured around multiple methods designed to identify direct benefits of the program such as retention and academic performance, and indirect benefits such as intellectual self-confidence and pursuit of graduate studies. The assessment program combines both qualitative and quantitative methods. At first, we relied heavily on quantitative research to collect the hard numbers that described whether UROP students had higher overall retention and academic achievement compared to students who were not enrolled in the program. As the assessment program matured, we have added focus groups and faculty interviews to expand our understanding of the program’s influence on undergraduate students. We have also added surveys to our research repertoire designed to identify the program’s influence on students’ academic self-confidence, perceptions of intellectual growth and feelings of belonging. 

Our multi-method approach has allowed us to develop a detailed understanding of UROP’s influence on its students. However, the various research projects also challenge us to identify and understand the effects we have found. What follows are descriptions of the UROP research projects, including brief descriptions of the sample, methods and results. 

UROP Focus Group Study

Separate group interviews are set up with UROP students, non-UROP students and students participating in the non-research-based Retention Program (RP). Interviews are conducted using a semi-structured interview protocol that allows the interview to be adjusted to the discussions generated by the students. Students are questioned about their experiences on campus, perceptions of faculty and fellow students in their disciplines, positive and negative academic experiences and plans for future academic and professional careers. The resulting data is analyzed using a grounded theory method that uses multiple readings of the interview transcripts to identify themes. 

Focus Group Study Findings: Students discuss their undergraduate experiences in three distinct manners. 

We have found that UROP students make 58% of the proactive comments. UROP students are more likely to discuss future events such as looking for jobs or going to graduate school, and they are more likely than non-UROP students to initiate activity with faculty and staff and see these individuals as positive influences on their academic experiences. Non-UROP students make 68% of the reactive/inactive comments. Non-UROP students are more likely than UROP students to see people as barriers to their learning experiences.

These findings provide indirect support for research and theory that indicates that students who are more integrated into campus life are more likely to be retained (Pascarella and Terenzini, 1991, Tinto, 1994). Our study shows that UROP students are more proactive in their interactions with the staff, faculty and administrative offices on campus. In general, UROP students think more about their futures, are more likely to feel that people are helping their educational experiences and are more likely than their non-UROP counterparts to seek out people instead of bureaucratic offices. Whether UROP actually prompts this type of behavior is difficult to determine from this qualitative study, and we cannot say that UROP caused students' proactive behavior. It is difficult in qualitative research to control for incoming characteristics that may relate to the outcome. The likelihood that UROP students come into the program with a proactive orientation, or that UROP attracts students with such an orientation, precludes us from specifically stating that UROP causes proactive behavior. However, we can say that UROP is associated with proactive behavior. Knowing that UROP students are more proactive allows us to better serve their needs in terms of programming and information. For example, we can provide role models who demonstrate the appropriate manner in which to contact a professor for a recommendation, suggestion, or academic advice.

Figure 1: Three student researchers make friends with their research subjects.

 
UROP Retention Study

Because the main goal of UROP is to increase undergraduate student retention and academic achievement, our retention study is central to UROP’s assessment and evaluation program. The sample (n=1280) consists of African American, Latino/a and White UROP students matched to non-UROP students who applied to the program but who were not accepted. We obtain retention data from the university’s Office of the Registrar that includes demographic information (race and gender), term and year of entry, term and year of most recent active enrollment, current enrollment status, grade-point average for each term, cumulative grade point average and enrollment status by term for each student. We define retention as students’ persistence through graduation and define attrition as students’ departure from the University of Michigan. 

Retention Study Findings: UROP participation increases retention rates for some students. 

This study proves that the program does have an influence on retention and degree completion for some students and that the effect is strongest for African-American males, who are a focus of the program, African American males. Nationwide, African American males have weak retention rates and appear to have difficulty adjusting to predominantly white campuses. We hope to use our focus groups and End-of-Year surveys to determine the reason for this positive effect on retention of African American males. One puzzling finding that we will attempt to unravel is why UROP does not appear to have an impact on Latino and Latina students. Research and literature suggest that the factors that determine the Latino/a retention may be beyond the reach of programs such as UROP. 

It is important to recognize that the control students in the retention study are students who applied to UROP and were not accepted. Specifically in the African American study, the controls have retention rates that are very similar to those of the general population of African American male students. Given Since these control students applied to UROP and were not accepted, their incoming characteristics should be similar to those who entered UROP. However, our study shows that without the UROP intervention, African-American male students are significantly less likely than their UROP counterparts to stay at the university.

Carrying out the retention study did not pose any significant challenges. We have an accurate record of students who participate in UROP for each year, and we have access to students’ academic data. The challenge we encountered, however, is However, determining retention at specific time periods was difficult.. For example, it was easy to determine degree completion for the 1990 cohort by identifying those who had or had not received a degree after 6 years. However, determining two-year retention was more difficult because some students “stopped” out or switched schools. Another challenge involved determining students’ field of interest during their four years. We could determine their degree field of interest, but students halfway through their undergraduate tenure would often be in the process of changing their field of interest and have said it was “undeclared” or “undetermined.” 

UROP Alumni Survey

The Alumni Survey is designed to identify UROP effects beyond graduation and to examine whether and to what degree UROP students differed on post-graduate education, experiences and career paths. The Alumni Survey sample includes UROP students matched to small control groups of students who had applied to UROP but who were rejected. These two groups are matched by high school grade-point-average, SAT scores, ACT scores, intended major, race/ethnicity and where possible, high school type. The sample consists of 291 students, with a 58.55% return rate. The survey asks students about their post-graduate experiences, including graduate school, career choice and satisfaction with current job.

Alumni Study Findings: 

The findings suggest that undergraduate research participation, either through UROP or other formats, is significantly related to pursuit of post-graduate education and further research activity (Hathaway, Nagda, and Gregerman, 2002). Because the controls are those who applied to UROP but were not accepted, they should be similar to UROP students in their intention to continue their education beyond the undergraduate degree. However, we did not anticipate that some of these control students had also been involved in undergraduate research. We only identified this issue when students who we identified as non-UROP alumni indicated on the survey that they had participated in some form of undergraduate research.1 However, these individuals allow us to compare UROP participants to non-UROP research participants. This comparison shows that, although both UROP and other undergraduate research participants are significantly more likely to graduate compared to controls, UROP students are significantly more likely to pursue Ph.D., medical and law degrees than the control students with research experience.2 All those with undergraduate research experience (UROP or non-UROP research) are significantly more likely to be engaged in research after graduation. These results suggest that UROP is fulfilling its goal of getting students interested in research for the long-term.

Though the alumni survey identifies several positive effects of UROP and undergraduate participation in research, we have found after analyzing the data that some of the questions could be more detailed and specific. In addition to asking whether students had asked for a job recommendation from a university faculty member, we could have also asked whether they requested a graduate school recommendation. The question that asks whether they requested a recommendation for a job, did not also ask if they actually received a job recommendation – a key distinction. We ask respondents whether their undergraduate research experiences contributed to eleven skill development areas, but the question does not ask how much they thought their research experiences contributed. Simple yes/no responses produced data that are not very useful in determining differences between UROP and non-UROP research participants or differences by race/ethnicity and gender. 

The size of the alumni sample is another significant challenge. The number of underrepresented students of color in the sample is not as large as that of the White/Asian students in the sample. Therefore, when we want to analyze differences in the type of post-graduate education pursued (e.g., Law, M.D., Ph.D. versus M.A.), the numbers of underrepresented students of color in each comparison cell became too small for meaningful comparison.3

As with any alumni related survey, acquiring accurate mailing addresses is an obstacle, but not an overly difficult one. The mailing addresses that we acquire from the university Alumni Records Office are mostly those of the alumni’s parents, who usually forwarded the surveys to their adult children. 

Figure 2: Students and faculty involved in a child study.

 
End-of-Year Survey

The End-of-Year Survey (EOYS) research project is designed to identify indirect influences of UROP. The EOYS measures students’ satisfaction, perceptions of the faculty and student environment, perceptions of growth and development in various personal and academic areas, development of overall academic confidence and development of research competence. The control group and the matching characteristics are the same as those outlined above for the Alumni Survey. The sample consisted of 342 students. We constructed several factors to measure whether and to what degree UROP participants differed from controls on key variables of research competence, likelihood of changing major, likelihood of dropping out and perceptions of the faculty and student environments.

End-of-Year Survey Findings: 

As expected, UROP does contribute significantly to students’ development of research competence and sense of self as a scientist or researcher. In fact, UROP participation and students’ agency level appear to have an additive effect on research development. Agency is defined as the degree to which students feel as if they are responsible for their academic experiences and successes. Those students with higher levels of agency are more likely, above and beyond the effect of UROP participation, to evidence significant research competence development. It was important for us to control for agency since UROP students have significantly higher agency levels than do their matched controls. However, even though there is an additive effect for UROP and agency, even for those students who do not feel in-charge of their experiences, UROP participation increases their identity as researchers. 

We also control for the amount of time students actually spend doing research. Time on research was a significant contributor to developing research competence. However, even when controlling for time spent on research, UROP participation still had a significant effect on research competence. The continued effect suggests that UROP programmatic elements such as peer group meetings, peer advisor meetings and poster presentations also contribute to students’ development as researchers. 

One of our challenges in the EOYS research is the return rate of the surveys. Each year, on average, we have a 55%-60% return rate. The return rate is high, in part, because we offer an incentive of a drawing for $25 gift certificates (50 gift certificates) for a local bookstore. However, the return rate for control students and for African Americans is much lower. We suspect that one of the reasons for the lower return rate for African American students is over-surveying. Given the relatively small number of African American first-year students on campus, we suspect that they are surveyed frequently and this contributes to their lack of interest in completing another survey. 

Figure 3: Faculty mentor and students working on a project in paleontology.

  
Experiential Sampling Study

The experiential sampling study, or “beeper study,” is an innovative research method designed to assess real-time student behavior. For this study, students wear a watch that beeps six times per day for one week. At the watch signal, the students complete a brief survey documenting where they are, what they are doing, and indicate what additional behaviors they had been engaged in since the previous signal. The purpose of this study is to determine whether and to what degree UROP students differ from a matched control group of non-UROP students in their day-to-day behaviors and activities. The beeper sample of 507 students consists of the same students selected for the End-of-Year Survey for 1995/96, 1997/98 and 1998/99. The beeper study is used to inform our analyses of the EOYS data.

Experiential Sampling Findings:

The experiential sampling study has provided us with a large amount of useful information on students’ day-to-day activities and behaviors. However, one of the challenges of this study is the amount of work and organization it takes to coordinate weekly samples of students. Because it is important for students to be detailed in their activity reports, we ask students to specify “studying chemistry for quiz” rather than “studying.” For each activity report, we ask students to rank how they were feeling on eleven affect emotions on a scale from 1-5. However, we find that students often leave these indicators blank for emotions that they are not feeling, or just check those they are feeling, instead of providing a 1-5 ranking indicator. The affect indicators are intended to determine whether and to what degree UROP students were more engaged or involved in their academic and day-to-day undergraduate experiences. We have yet to determine whether the data based on the incorrect completion of the affect indicators will preclude these analyses. 

Future Research

We have developed a more extensive survey that has been sent to all UROP alumni and to a matched control during Summer 2002. In addition, we are now analyzing EOYS data to investigate the connection, if any, between variables of psychological empowerment and students’ academic achievement. 

During Winter 2001 we also completed an intensive focus group study that investigates the issues facing science and engineering students. The focus groups consisted of students in UROP, UROP in Residence (these are UROP students who live together on one residence hall floor), Women in Science and Engineering Residence Program (WISE-RP) and matched control students. These interviews focused on students’ experiences as science and engineering majors with an analysis of whether students’ experiences differed by program. Students were interviewed in groups of 4-8 students, divided by science and engineering, race/ethnicity, gender and program participation. For example, we interviewed African American male engineers separately from African American female engineers and African American male science majors. The research design was structured to encourage free discussion among the student participants and avoid the group dynamics that may silence discussion between, for example, white male and white females, or between white male and African American females. The intent of this study is to better understand, from the perspective of students, their experiences in science and engineering at the University of Michigan. 

The second qualitative study is one on faculty.4 We have interviewed faculty UROP mentors as well as non-UROP faculty who work with undergraduate students involved in research in order to investigate faculty perceptions of undergraduate researchers. We are also interested in determining how faculty view their roles with undergraduate researchers – do they view themselves as mentors, or do they view their undergraduate students as research help? We have interviewed faculty from a wide variety of disciplines, tenure status, race/ethnicity and gender. From this research we hope to develop an understanding of the environment in which students do their research.

What We Have Learned

Very early, we realized the importance of including several types of individuals on the research team. It is important to not only have people with expertise in research design, implementation and analysis, but to include administrators and staff from the programs under investigation. Program administrators bring an understanding of the issues the program participants face. They also have a wealth of anecdotal evidence that is useful in developing questions and research. 

We have learned that it is best to include undergraduate students in several phases of the assessment and evaluation program. Undergraduates have been particularly useful in developing the interview protocols and helping us use appropriate language. For example, in one of our original interview protocols tested with UROP peer advisors, the peer advisors indicated that our use of ‘extracurricular activity’ was incorrect. We were using it to prompt student discussion of their involvement in academic programs, but for undergraduate students, ‘extracurricular’ relates to social or fun activities such as intramural athletics. 

In another instance, an undergraduate research assistant suggested that one of the reasons we found that UROP students study more is that these students work and attend UROP functions at times when other students generally study. Our experiential sampling study caught them studying more often, because UROP students study during the day. This same undergraduate research assistant helped us unravel issues related to students’ awareness of new developments in their fields of interest. She suggested that UROP might not influence such development because students’ research projects may not be directly related to their interests. She proved her hypothesis through a regression analysis that indicated that students’ personal motivations were more significant in their developing an awareness of their fields of interest.5 A humorous example occurred when we were coding the open-ended activities on students’ experiential study questionnaires. Many times students indicated that they were watching television and studying. The researchers coded this as studying, but the undergraduate research assistants said that when students indicate that they are studying and watching television, they are more likely doing the latter.

Figure 4: Student researchers who have made presentations at Michigan's Research Symposium.

 
One of the common challenges we encountered in our research was the small number of underrepresented students of color in the sample. The smaller numbers restricted our ability to perform some statistical analyses and therefore restricted our ability to determine the type and degree of programmatic influence on underrepresented students of color. For example, it is difficult to determine the influence of UROP on African American engineering students due to the fact that there are small numbers of such students on campus. When we want to compare the UROP effect on engineering students by race/ethnicity, it becomes problematic because there are far more white engineering students than African American engineering students and the statistical comparisons become weak. However, we continue to add new cohorts of UROP students to our database and in the future we will be able to fully investigate the influence of the program on all students. 

One has to acknowledge and accept that no research, evaluation, or assessment project will be perfect. Tape recorders malfunction, students do not show up for focus group interviews, there are not enough control students to match your sample, etc. For example, in our current focus group study, we were unable to get Asian American female science majors for a control group. And we found that no matter how many times we critiqued questions, once we started analyzing the data, we would realize that one question could have been worded better or could have had a better scale. 

Several of our projects are cross sectional or ‘one shot’ studies that look at students at one point in time (e.g., our End-of-Year Survey). We do not know, for example, whether the sense of research competence exhibited by UROP students at the end of their first year actually continues through to their sophomore year, or whether that development is tied specifically to other outcomes, such as science laboratory grades. The importance of conducting longitudinal studies is evident in the UROP retention findings. Our first study (Nagda et. al, 1998) followed UROP students and their matched controls through a specific semester, not to degree completion. The follow-up study investigated the same student sample six years following their first-year matriculation. There were minor differences in the findings. 

Social science research is quasi-experimental by nature. In most instances it is impossible to clearly identify the factors that create an effect or the factors that interfere with one’s ability to isolate the effect of a particular variable. In most of the UROP studies, we addressed this issue by creating control groups consisting of students who applied to UROP but who were rejected. These ‘rejected’ students, we theorized, would be similar to the accepted UROP students on various unmeasured characteristics. We have begun to utilize data gathered for first-year students from the Cooperative Institutional Research Program that measures students incoming beliefs, values and attitudes, including degree aspirations. Using this incoming data in the future will allow us to better control for other influences on our outcomes of interest. 

We have learned the importance of using both quantitative and qualitative research to develop a complete assessment and evaluation program. The quantitative research highlighted several influences of participating in the program, both on students’ undergraduate experiences and also on their post-graduate experiences. Yet, for each finding, we identified another question or wondered about other effects we had not identified via the quantitative methods. Although quantitative research provides the hard data on the influence of UROP on retention and specific areas of academic and intellectual development, such research does not reveal the students’ perceptions of the influence of UROP or other experiences on their progress. The two qualitative studies highlighted issues and effects that we had not identified beforehand in designing the quantitative studies, and each informs the other. 

While we believe it important to use mixed methods, we must be sensitive to the audience to which the research will be presented. Administrators are often more willing to accept findings from quantitative research than from qualitative research. Even presenting the hard data from the retention study before the data from qualitative studies can help the audience appreciate the results.

Obviously, it is important to disseminate findings to internal university constituencies who have an interest in the program’s success. In addition, it is also important to present findings to external audiences through conference presentations, refereed journals, or other publications. In our case, there are many other institutions implementing programs similar to UROP that can benefit from understanding how our program is working. By submitting our results to refereed journals, we receive feedback from external experts on the strengths and weaknesses of our research. This feedback informs future research and helps us improve the current work. External reviews also make the research more credible to university constituents. 


Discussion

Our assessment and evaluation program is an ongoing process. In some ways, we do not believe the process will ever be completed, for with each new finding or result, we find several more questions to be asked and additional research projects and analyses that can be performed. Each study provides insight into the influence of the UROP program and, as we continue our investigations, we have a better understanding of why and how the programs have their influence. We know, for example, that UROP students develop a better sense of the new ideas and new research in their fields of interests, yet it is highly likely that this development may differ by race/ethnicity and/or gender. What is it about UROP that influences African American male retention that may be helpful in prompting the same effect at another institution? As we analyze the UROP faculty interviews, we hope to identify how faculty construct or perceive their roles in the undergraduate research experience - do they see themselves as mentors or supervisors? If such a distinction exists, does it influence the students’ research experiences, learning and academic performance? We know that UROP does not have an influence on first-years students’ grade point averages, but we need to determine if there is an effect after their first-year and/or whether the effect leads to higher GPS in the students’ fields of interest. Perhaps the effects are not exhibited in higher academic performances, but on students’ feelings of accomplishment and commitment to their goals and objectives. 

While we were fortunate to receive significant external funds to do this research, we realize not all campuses will have access to the resources needed to conduct an evaluation of this scope. By joining forces with education, sociology, or psychology programs on your campus or an affiliated campus, you can identify faculty and/or graduate students who do research in this area. If these individuals are interested in a case study or dissertation topic related to your work, they may wish to collaborate with you on the assessment. We also encourage other campuses to use our results to demonstrate the impact of undergraduate research. While each campus is different, the lessons learned at Michigan are applicable to issues of student retention and diversity on many campuses.


The Institution

The University of Michigan is a large public research university located in Ann Arbor, Michigan, 45 minutes from Detroit, one of the nation’s largest predominantly African American cities. The University is a highly selective public institution and perennially leads the nation in annual research expenditures. The university’s Fall 2003 undergraduate student population was 24,517, of which 12.5% are underrepresented minority students (African American, Latino/a American, Native American). The majority of entering first year students enroll in the College of Literature, Science and the Arts. The College of Engineering enrolls approximately 4506 undergraduate students. In a typical year UROP enrolls 1000 first and second year students working with over 500 faculty from all of the University’s schools and colleges including the professional schools and graduate schools. Like many other large research institutions, we do not have a good mechanism to accurately count the number of undergraduate students engaged in research campuswide. Our best estimate is between 4000-5000 students employed either through work-study, doing independent research, volunteering, etc. Unlike UROP students, many of these students would be upper division students. Alumni of UROP often continue to work with their original research mentor throughout their undergraduate careers, a small but significant percentage go on to complete honors theses, and others pursue off-campus research opportunities during the summer. 

At the time UROP was developed, the University president had instituted a comprehensive campus-wide program to increase the numbers of faculty, students and staff from underrepresented groups and several initiatives to improve the retention rates for minority students. While still far higher than the national retention rates, the University of Michigan faced differences in the graduation and retention rates of diverse students. During the first three years of its existence, UROP enrolled underrepresented minority students exclusively. Since the 1992-1993 academic year it has been open to all first-year students and sophomores and recently, UROP has added a small program for juniors and seniors.

Key studies that looked at factors critical to the success of students of color on predominantly white campuses have made two critical observations. First, minority students at predominantly white institutions do not identify with the intellectual mission of the university as much as white students do, and this leads to higher attrition rates for minority students (Terenzini 1996). Secondly, faculty contact outside the classroom is critical to the retention of students of color (Astin 1998). Similar research points to the importance of early mentorship and hands-on learning to reduce attrition of young women interested in science who shift to non-science fields. Because these results suggested that faculty/student research partnerships might improve student retention and academic performance, UROP’s major goal was to broker intellectual relationships between faculty and undergraduate students, specifically first-year and sophomore undergraduates, through research partnerships. 

Research projects are available in most liberal arts departments, in the professional schools of medicine, law, social work and business, in environmental studies and in the hard sciences of biology, chemistry and engineering. Students are involved in bibliographic research and literature reviews, laboratory work and archival research. Students also construct, disseminate and analyze surveys, technology transfer and course development. Many UROP students have coauthored research presentations and journal articles with their faculty sponsors. 

Students in UROP participate in the faculty research project 9-12 hours per week, either for academic credit or as a work-study job. Students attend an orientation workshop early in their first semester that is designed to help them become familiar with UROP and with working on faculty research projects. Students then apply to research projects, develop a resume and interview with the project faculty and staff. Students attend bi-weekly peer group meetings that cover such topics as research ethics and research careers. During the meetings students share their research experiences with other students participating in similar academic disciplines. Each UROP student is assigned a peer advisor with whom they meet on a monthly basis. At the conclusion of the academic year experiences, students present their research orally or in poster format during a campus-wide research symposium.


Acknowledgements

UROP would like to acknowledge the support of the National Science Foundation, the University of Michigan's Office of the Vice President for Research who nominated us for the RAIRE award, the State of Michigan's Office of Equity and the U.S. Fund for the Improvement of Postsecondary Education which provided the original seed funding for our evaluation and assessment activities, Edie Goldenberg, former Dean of the College of Literature, Science and the Arts who supported the program in its infancy and John Jonides, Professor of Psychology who had the vision to build assessment and evaluation into the program's design and ensured the study's integrity.


References

Astin, A.W. (1993). What matters in college? Four critical years revisited. San Francisco, CA: Jossey-Bass Publishers.

Hathaway, R.S., Nagda, B.A., and Gregerman, S.R. (2002). The relationship of undergraduate research participation to graduate and professional education pursuit: An empirical study. Journal of College Student Development, 43(5), 614-631.

Nagda, B.A., Gregerman, S.R., Jonides, J., von Hippel, W., and Lerner, J.S. (1998). Undergraduate student-faculty partnerships affect student retention. The Review of Higher Education, 22(1), 55-72.

Pascarella, E.T., and Terenzini, P.T. (1991). How college affects students. Jossey-Bass: San Francisco.

Tinto, V. (1993). Leaving college: Rethinking the cause and cures of student attrition (2nd edition). Chicago: The University of Chicago Press.


Endnotes

  1. We did not classify all control students’ undergraduate research experiences as such. For example, one student indicated that they had participated as a subject in research studies as an undergraduate – this student was not classified as having participated in undergraduate research. 
     
  2. Non-UROP research participants did not differ from the controls on the type of education they pursued. 
     
  3. For example, when we compared African American UROP students to non-UROP African American students in an unmatched sample, significantly more African American UROP students attended medical school. This significant different disappeared when we use a matched sample in which each African American UROP students is compared to only their matched controls.
     
  4. The faculty interview research was for UROP assessment only.
     
  5. Paige Kuebler

 

Copyright © 2003 Council on Undergraduate Research. All rights reserved.