Summer Humanities Institute
UCLA Bunche Center for African American Studies
Evaluation Report
Summer 2005
SRM Evaluation Group
University of California, Los Angeles
Tanner LeBaron Wallace, Ravneet Tiwana & Artineh Samkian
1
INTRODUCTION
The program staff from the Summer Humanities Institute (SHI) contacted the SRM
Evaluation Group to collect and analyze data from focus group and interviews with all ten SHI
students.
In order to design the protocols for the focus group and interviews, the SRM
Evaluation Group developed a draft Theory of Action for the program. (See appendix A for the
draft Theory of Action). This Theory of Action is based upon document review, conversations
with program staff, and a review of the literature on underrepresented populations in graduate
school.
The Theory of Action is a graphical representation of how the program should
theoretically work. It attempts to capture the relationship between activities and outcomes. For
this program, SHI recruits undergraduate students from Historically Black Colleges and
Universities and provides the students with a summer institute experience in order to achieve an
ultimate goal of more underrepresented students successfully enrolling and receiving fellowships
in humanities doctoral programs. What happens between Point A, the recruitment, and Point B,
enrollment in a humanities doctoral program, is important to understand. Hence, the program
contracted with the SRM Evaluation group to begin looking at this question.
The key evaluation question to answer in this report is, “What tools, attitudes, and skills
did the SHI program provide student participants?” An important follow-up question, not covered
by this annual evaluation work, would be, “Once provided, do those tools, attitudes, and skills
assist students in achieving the ultimate program goal?”
METHODS AND DATA COLLECTION PROCEDURES
The data collected for this evaluation was primarily qualitative in nature with an added
survey that included both quantifiable and open-ended questions. The sources of information for
the qualitative component included focus group and individual interviews with SHI participants.
2
The one-hour focus group was conducted first, and all ten participants were present. After a
short break, individual interviews that lasted approximately 25 minutes each were conducted
independently by all three of the evaluation team members. These individual interviews were
previously scheduled and participants were given the time and location of their interviews. The
focus group and all the individual interviews were audio-recorded for the purposes of
subsequent data analysis and participants were assured that any and all comments would be
kept confidential and anonymous. After completing the individual interview, each participant was
asked to fill out the anonymous short survey. (See Appendices B, C and D for copies of focus
group and interview protocols and the survey instrument.)
The main reasons for using primarily qualitative methods of data collection were that
there was a strict time constraint, which prevented us from obtaining a wide variety information
to quantify, and as well, there were only ten participants in this year’s SHI program, thus limiting
the sample size. In addition, the choice to use a focus group and individual interviews provided
us with a more rich and in-depth examination of students’ self-reported perceptions of their
program experience. Had we had more time, it would have been ideal to supplement this
interview data with the following qualitative methods: 1) observations, 2) interviews with
administrators and staff of SHI to better understand the program and 3) analysis of SHI’s
marketing materials.
ANALYSIS OF DATA
The data collected from the focus group, interviews and surveys were analyzed using the
draft SHI Theory of Action as a framework. Where possible, the evaluation team looked for
evidence supporting the achievement of the immediate outcomes and intermediate outcomes.
Two of the evaluation team members reviewed the audio-recorded interviews, looking
specifically for comments that supported observed trends in the data corpus. The other team
3
member was responsible for the analysis of the survey information. The following data analysis
section is organized by SHI program components.
The analysis integrates focus group,
interview and survey data sources.
Faculty Seminars
Some students identified the faculty seminars as the most memorable and useful
component of the program. The seminars played a key role in developing their critical thinking
skills and taught them to identify and use primary sources that are recognized by the academic
community. The students found the discussion format and the type of facilitation by professors
to be different from that of their home institutions — where professors primarily conduct lectures.
One student identified the seminars at SHI to be different from those at the student’s home
institution by being, “more participant discussion than professor lecturing … our professors
would put an idea out there and we would discuss it amongst ourselves…” This discussion
format appeared to play a pivotal role in developing students’ critical thinking skills because, “we
had to analyze two works that seemed very dissimilar, but delve deeper into both of them and
you found similarities and if you had to come up with something to correlate the two … I had to
use my critical thinking skills there.” The active participation structure in the seminar discussions
helped students gain a critical understanding of how to engage various texts in graduate school.
Being able to differentiate between types of scholarly works and their perceived validity in
the academic community was another key skill the faculty-conducted seminars taught student
participants. One student said, “one of things that was different for me was not necessarily the
tools that they gave us, but making sure that we found primary sources and the type of
information that was considered valid.” Students repeatedly cited gaining more knowledge about
4
the type, use, and validity of scholarly works to support their arguments, rather than relying on
more simple web searches1 and any piece of literature.
The two primary critiques of the faculty seminar component were their underlying purpose
in the program and the manner in which they were presented in SHI’s marketing material.
Students believed the large amount of time allocated for the seminars made them seem, at
times, as more of a monitoring and tracking tool than a source of academic engagement.
Several of the students thought the current SHI schedule did not permit enough time to conduct
independent research and focus on writing their final papers. One student said, “ … we meet
everyday from like 9 to 2 or something. That’s something we did when we were in high school
and we’re all either graduates or seniors in college and I kind of felt like this was done so they
could keep tabs on us, they’ll know where we are at all times, which is kind of understandable
because you have all these people from other places but at the same time, it took away from our
papers.”
Students felt as though the large amount of time allocated to the seminars
underestimated the level of maturity and capability to undertake independent work the students
possessed. One student suggested that, “if they want to keep tabs on us, have like a time when
we have to be in the library doing research”, rather than being a classroom listening to the “same
people talking.”
Lastly, students felt as though the lack of time and resources2 given to them to conduct
their independent research undermined the primary purpose of the program as advertised in
SHI’s literature. Before attending SHI, students believed the key purpose of the program would
be independent research, rather than be in “summer school.” One student said, “ … when I filled
out my application, I didn’t know that we were doing classes every day. I thought it was going to
be more of research...” Another student followed this comment with the statement that, “initially, I
1
2
The students referred primarily to “www.google.com” web searches.
Addressed more in-depth in a later section of the report.
5
would not have even came if I knew it was set-up the way…because I looking for an intense
summer research and writing a paper.” Therefore, some students believed they had been
misinformed of the primary academic purpose of the program, even though they understood that
SHI’s overarching goal was to prepare them for graduate school.
When asked on the survey to report how useful the faculty conducted seminars were as a
program component, 4 of the students reported “most useful” and all students thought the
seminars were useful to some degree. (See Table 1 below for the frequencies of responses.)
Table 1: Faculty Conducted Seminars
Faculty Conducted Seminars
4
4
2
Most useful
Very useful
Useful
0
0
Somew hat
useful
Not useful
On the survey, student participants were asked, “Which program component contributed the
most to your current understanding of what graduate school is like? Why?” Some noteworthy
answers related to the seminars are listed below:
”They showed that a lot of information will be assigned, but the student must differentiate
between what is most important.”
“The seminars allowed me to compact a lot of reading material in a short amount of time.”
“They helped me to understand that one must be well read in order to engage in ‘scholarly’
conversations.”
“The seminars contributed the most to my current understanding of what graduate school is
like because we had to think critically and have productive class discussions.”
6
Course Materials
Overall, the students appreciated the content of the course materials. Some, however,
were unhappy with the amount of time reading the materials took and felt this time could have
been better spent conducting their independent research. Several students acknowledged the
material, in the neatly complied, compacted, and spiral notebook, was relevant to their research
on the African American community in terms of both their personal and academic goals;
however, they would have appreciated getting a reading list prior to coming to the program. This
reading list would have helped them save time from rereading in-depth works that they had read
before and/or prioritize those works most relevant to their individual research focus.
Opinions regarding the value of the course materials varied.
Some of the students
believed the readings were “busy work” that took time away from their independent research.
Other students, however, felt the course materials were fundamental to their overall experience.
One student believed that it was necessary to the read the course materials in order to
personally develop as a scholar in African American studies. The student said, “sometimes felt
that is was busy work, but necessary. It was necessary to read works on the African American
experience so we could reiterate the things we already knew, regroup and rethink these
experiences so to make it a personal experience so we could be passionate about the work we
are doing here and the relationships we are building here and to stay in contact with people.”
Another student found the course materials to be intellectually challenging because at first the
student was unaware of how the materials would be relevant to the research the student was
planning to conduct. Later in the program, when the student better understood the material, in
part, by having access to some of the authors who wrote the works, the student reported
continuously citing and applying the concepts from the course materials to his/her research.
7
When asked on the survey to report how useful the course materials were, 2 of the
students reported “most useful.” All student participants reported the course materials were
useful to some degree. (See Table 2 below for the frequencies of responses.)
Table 2: Course Materials
Course Materials
4
4
2
Most useful
Very useful
Useful
0
0
Somew hat
useful
Not useful
Mentoring Experience
The mentoring component of SHI was consistently cited as the most beneficial and
important part of the program. Students who found their mentors to be skillful, reliable, and
personable were glad to have had such a good experience.
Whereas students who were
unhappy with their mentoring experience, felt an extreme sense of disappointment for not having
effective and productive guidance.
Many students included the graduate student mentors as an important addition to their
overall mentoring experience, because they received first hand perspectives about being
currently enrolled in a graduate program from them. One student said, “I have gotten to speak
to people in the process … that has been wonderful to get first hand experience with people who
8
are getting Masters degrees … graduate mentors are pursuing PhDs.…they are great resources
… getting information as far as financial awards and navigating the bureaucracy…”
Students’ reflections regarding the mentoring experience centered on three main themes.
These reported themes were: academic guidance, personal connection, and feedback. The
faculty mentors played a key role in students’ academic and personal development. However,
there were inconsistent mentoring experiences across the student population. A robust and
productive mentoring experience consisted of mentors being particularly helpful in focusing
students’ research ideas, developing critical connections across potential research topics, and
assisting in the development of more confident and skilled writers. One student said, “I have
very, very few educators that I really look back at and say they had an impact … and he [the
mentor] was one of them … he took me from being cocky about my research thinking that ‘okay I
know how to write’ to being confident in saying I put out a scholarly piece of work … and I still
have a long ways to go now… I still can improve upon [this work].” This type of mentoring
helped students develop personal and intellectual self-confidence. This dual development was
strongly linked to some mentors who were sincere about their students’ future as young African
Americans.
Contrasting these positive experiences, however, some students reported feeling a sense
of abandonment by their mentors. One student said, “I only met her once or twice … I felt really
disappointed because I felt like I did not really have a mentor … every time I went to her she did
not tell me that she was canceling or when I went to her she did not have time for me.” Another
type of disappointing mentoring experience transpired when the mentor was “very validating of
my scholarship, but he did not give me the type of critical analysis on [the] paper and go line for
line on my actual content … this experience was supposed to be about boosting my level of
scholarship.”
9
All students were very cognizant about how the variance in mentoring approaches and
relationships influenced the quality of their final work. One student said, “I see that the people
who did have those faculty mentors that were just really into their research, scheduling to meet
them two times a week on a consistent basis, their research came out to be so much better. I
know professors have to travel and everything … but I think if we have to be here for eight
weeks, the faculty mentor should too… corresponding through the Internet is different from when
you have that face to face contact.”
The issue of feedback was an important theme that emerged in students’ reflections. During
the interviews many students referred to mentors providing detailed feedback on their academic
writing. For some students, this was their first experience with one-on-one writing assistance.
Some students questioned the degree of professionalism displayed by the graduate mentors
when giving feedback—especially those students who relied more on this type mentoring when
their faculty mentors were not present. One student said, “I only got frustrated once in the
program when they conducted constructive criticism because they stood up and basically as a
group attacked our work for the summer and I remember one comment was, ‘when I read this it
makes me think why am I here’ and to get that from a graduate mentor who is looking over your
work I think that was a bit disheartening and then to get my paper back and it was like excellent
job and I can’t wait till you finish this paper...” This relaying of contradicting information, for some
student conveyed the lack of professionalism because the negative and positive feedback was
intended for specific individuals but directed to the entire group. One student dealt with this
situation by, “ …just let[ing] it roll off my
back … I felt like it was kind of an attack and when
you put all of that out there and put everyone in a room you are directing it to all of them … you
cannot separate and say I meant it for these four people and then there are ten in the room.”
10
When asked on the survey to report how useful the mentoring component was to the
program, 5 of the students reported “most useful.”
One student reported the mentoring
component as “not useful.” (See Table 3 below for the frequencies of responses.)
Table 3: Mentoring
Mentoring
5
2
2
1
0
Most useful
Very useful
Useful
Somew hat
useful
Not useful
On the survey, student participants were asked, “which program component contributed
the most to your current understanding of what graduate school is like? Why?”
Some
noteworthy answers related to the mentoring experience are listed below:
“My mentor showed me what grad school is-having a close relationship with that person the
next 3 to 4 years.”
“I learned the most from talking one-on-one with my mentor…His experiences helped me
through the program.”
“The mentoring helped me understand the importance of having someone who fully
supported my research.”
Workshops
Many of the students felt the workshops were more of a scheduling technique to “keep tabs”
on them rather than beneficial exchanges of important information. One student asserted, “I
think the only thing that the workshops did was give us a set schedule … for getting up everyday
11
and that is about it.” Despite the negative perceptions of the workshops in general, some
positive reviews were given for a funding graduate education workshop. The students who
attended the funding workshop found it to be helpful, but rushed. One student said, “there was a
workshop on funding graduate education and financial funding, but it was rushed … there
needed to be a part two and part three”. Furthermore, some students cited funding graduate
education as a major obstacle for them, which led one student to comment, “I was hoping they
would give us more information on graduate school and maybe talking about fellowships…”
Students found the SHI workshops to be rather technical in nature and felt the content could
have been easily conveyed in a shorter period of time with the use of handouts. Many students
commonly cited the most useful workshop material as the dedicated to creating reference
sections papers — footnoting, word choice, and paraphrasing. Therefore, one student believed
that SPURR workshops were more beneficial than SHI workshops because “more came from
SPURR than SHI. They actually went in to-depth of this is what your graduate school application
will take.”
When asked on the survey to report how useful the workshop component was to the
program, the majority of students responded “useful” and only 1 student reported “most useful.”
(See Table 4 below for the frequencies of responses.)
12
Table 4: Workshops
Workshops
4
2
2
1
Most useful
1
Very useful
Useful
Somew hat
useful
Not useful
Extracurricular Activities
Overall, the students enjoyed one of the two extracurricular activities and overwhelmingly
reported a desire for more extracurricular activities in the Los Angeles area. Many students felt
as though the lack of extracurricular activities in the program prevented them from benefiting
from the great amount of academic and leisure activities available in the Los Angeles area —
particularly those that concentrated on African American history.
The students mentioned that they enjoyed attending the Golden State Life Insurance
African American Art Museum more so than the King Tut exhibit, which they paid to attend. One
student said, “we went to Golden State Life Insurance to see African American Art … we paid for
King Tut and it was not worth it.” However, students’ reasoning for attending the activities varied
across the group. One student reflected that the museum trip was beneficial because it related
to the discussions on African American Art they were having in the faculty seminars.
Another student commented one of the main reasons she enjoyed the extracurricular
activities was, “to me it was fun activity because I got out of the building … just to be out and
13
about in city and not be cooped up in the Bunche Center Library…” This student’s opinion is
supported by many other students’ suggestions that the SHI program include more
extracurricular activities. As one student asserted, “ … Monday through Friday, 9 to 12 we were
in the room the whole time… it doesn’t cost anything to plan for students to go on public
transportation and go to a museum and half the time museums are free in the summer or they
cost five dollars… it would be good just to get out of the classroom setting.” Hence, there was a
deep desire among students to learn about Los Angeles, its connection to African American
history, and to vary the program in terms of activities and location.
When asked on the survey to report how useful the extracurricular component was to the
program, no students reported “most useful” and two students reported “not useful.” (See Table
5 below for the frequencies of responses.)
Table 5: Extracurricular Activities
Extracurricular Activities
7
2
1
0
0
Most useful
Very useful
Useful
Somew hat
useful
Not useful
On the survey, student participants were asked, “Which program component contributed
the least to your current understanding of what graduate school is like? Why?”
Some
noteworthy answers related to the extracurricular activities are listed below:
14
“We didn't do anything extracurricular that connected with graduate school.”
“We only had one activity that applied to our class discussion.”
“The opportunities were not as meaningful.”
“The extracurricular activities did not teach me anything.”
Presentations
Some of the students found the presentations to be helpful in their growth as scholars.
One student believed that it made her feel more intellectually confident and gave her additional
experience to feel prepared for graduate school. Another student explained, “I think it was very
good experience to be in a room with people who are experienced in various fields and stand
before them and present in a scholarly manner.” Other students in the group who had had
previous experiences with presentations said that this was the first time they had been involved
in a panel. These students were expecting an individual question and answer session following
their presentation. Although they were not critical about the panel format per se, one student felt
that the panels should have been arranged around relevant topics rather than conveniently
grouping different research projects.
For example, one panel could represent all research
projects having something to do with black-on-black violence, etc. Another student suggested
inviting more people from outside SHI, thus representing different departments.
Some of the students noted that not all of the SHI program staff were present during the
event. One student explained that “in the beginning our research was so important and they
were investing all this in us, but yesterday some of the head people were not even there… some
of them did not come in on time. It’s like they were talking us up like this is so important and
everything was mandatory but they were not even there.” Following this comment, a discussion
ensued and one of the students said that she would have appreciated some explanation for why
certain people had not come to the event. She said some excuses are truly understandable, but
no effort was made to relay these reasons to the presenters.
15
When asked on the survey to report how useful the writing of a research paper and
presenting the research paper components were to the program, 8 students reported writing the
paper “most useful, while 7 students reported presenting the paper “most useful.” (See Tables 6
& 7 below for the frequencies of responses.)
Table 6: Writing Research Paper
Writing a Research Paper
8
2
0
Most useful Very useful
Useful
0
0
Somewhat
useful
Not useful
Table 7: Presenting Research Paper
Presenting a Research Paper
7
2
1
Most useful Very useful
Useful
0
0
Somewhat
useful
Not useful
16
Facilities and resources
One aspect of the program about which we did not ask any questions but was brought up
by the students themselves was the issue of facilities and resources. Many of the students
expressed in the focus group that there was a lack of communication about the ability to access
resources such as laptop computers. Given that the libraries on campus close early during the
summer, those students who did not have computers felt at a loss as to how to work on their
research during the evening hours. One student explained, “these are things that I thought
should be considered when people are here doing research… I work at night.” Although there
was a computer lab in their residence hall, students explained that these labs were often
reserved for other groups and they were unable to make use of this resource because
“consistently throughout the summer, it was not available.”
Some of the students expressed frustration about the fact that they felt they were at a
disadvantage because some of their colleagues owned and had brought their own personal
laptops. One student said, “I was one of the people that did not have a laptop and it did put me
at a disadvantage… my colleagues were very understanding by letting me borrow their
resources when they weren’t using them, but it was difficult.” Another student explained that had
they had access to a laptop and the library for more hours in the day, “we might have all did
extremely better had we had these resources… the possibilities are endless when you give
people the resources.”
Not all students were aware that laptops could be rented.
They
suggested the rentals be arranged for student participants ahead of time by program staff.
Institutional Sense of Belonging
Drawing on the literature on underrepresented minorities in higher education, we
identified two other themes we wanted to explore in our interviews with students. Given that the
program attempts to encourage students to apply and enroll in graduate study, we tried to gauge
17
how successful the program was in instilling both an institutional sense of belonging and an
intellectual self-confidence in program participants.
In regards to institutional sense of
belonging, we found two main themes in participants’ responses: 1) whether their topics of
interest would be considered scholarly and 2) whether their minority identity would result in
added obstacles and challenges.
In our individual interviews with students, we asked them whether or not they foresee any
obstacles and challenges upon entering graduate school. Some of the students explained that
the program has afforded them the ability to see that not all research is considered scholarly.
They learned though, that knowledge and power are intertwined and that they must question,
“Who’s to say what’s scholarly?” Although these students saw their topics of interest as being
non-canonical, they were optimistic that “someone will find it scholarly and feel that my topic
should deserve scholarly sensibility.”
The program contributed to this optimism for those
students who knew their topic of interest before coming to SHI, but the program served also to
help some participants develop their topics while here. One student said she focused her topic
and explained that “[SHI] definitely has changed what I initially thought I wanted to do.”
Another concern voiced by program participants about possible challenges to attending
graduate school was their minority identity. One student said that she feels she “may have to
constantly prove that I deserve to be here, that I’m worthy of being a grad school student
because [she is both] a woman AND an African American.” In addition, at least two students
were concerned that their HBCU education may be seen as inferior: “I will face obstacles coming
from a historically black university and going to a predominantly white institution for graduate
school. A lot of times people feel as though your education was sub par.” A related issue that
was brought up by one student about the diversity she would face was the concern that HBCU
students were ill equipped to handle a predominantly non-black student and faculty student
18
body. For example, she said, “this program was only African American, which I appreciated but
the world is not just African American and then I go to an HBC also so… I kind of feel like ‘ok,
you need to learn how to interact with other people.’” Although she did not explicitly suggest
changing the demographics of the program, she did suggest that it might be useful to place
participants in contexts in which they interact more frequently with other, more diverse groups of
students in order “to open our eyes to see that the worlds is not just black and white.”
A second theme extracted from the literature on minority experiences in higher education
was that of intellectual self-confidence. Although most of the program participants did not think
this self-confidence was learned at SHI, they did give the program credit for sharpening their
skills and level of self-confidence. Different students learned different skills while at SHI. Most
of the students improved in their writing. One student said, “in school, I felt like everything was
brushed over. They pretty much allowed us to get away, just turn in a paper and that’s it. But
with the Summer Humanities Program, I was able to write a research paper and get feedback…
I know personally where my weakness was so I was really looking forward to the criticism. It
didn’t hurt my feelings because I needed it to make me a better writer.” Others learned how to
read more strategically and critically, while a few learned about what it means to be a student in
a large research institution.
Most of the students also expressed a heightened level of self-confidence as a result of
the program. One student said “when I came here I wasn’t really sure if I wanted to go to
graduate school or not but now, going through this program, I’ve kind of made my mind that yes,
[I am] capable of going to grad school.” For some students, the mentors were important in
developing this sense of self-confidence. Besides developing participants’ self confidence, SHI
also affirmed already existing beliefs. One student explained that although she has always been
confident in her intellectual abilities, it was still beneficial to receive positive feedback from
19
program staff and other participants: “I have always been told that I was a good writer but just
actually being able to go someplace where people don’t really know me, they just can only make
a decision on the work that I produce, and then for me to get the same positive feedback” was
very affirming.”
Miscellaneous
Although, overall, participants saw the benefits of a program such as SHI, they voiced
some concerns and criticisms about the program that we feel should not be overlooked. In
addition to those mentioned in the body of the report above, there were a few others that did not
as neatly fit into the program components and relevant literature review topics, which organize
this report. One frustration on the part of students was the way they were treated by program
staff.
Some of the students complained that staff discouraged them by being rude to the
participants: “you know, we’re working hard… I was starting to get discouraged. I was just ready
to go [home].” Others felt that they were being treated like children: “I think in some respects we
were treated like children and I think they need to understand we are college seniors and college
graduates… and we were treated as middle school students when we are not.” It seems this
comment stems not only from the regimented schedule with which they had to abide, but as well
with the way they were treated and spoken to by program staff.
Another complaint by at least two different participants involved the space available for
true dialogue and the voicing of different opinions. These students believed that there was a
lack of acceptance of others’ opinions during discussions. For example, one participant said that
“when one of the students said something I mean the program advisors really took [him/her] to
task for even having an opinion of that magnitude and that changes the group dynamic because
you have some people who will say ‘okay, I am not going to say anything period because I do
not wanna feel like that’ and then you have other people who say ‘okay that was not right you
20
should not have done that to him/her’ and they speak up and they are victimized like [the other
student] was.” In these comments, it seemed students were hoping the program did a better job
in fostering more open dialogue, no matter what perspectives were shared.
CONCLUSIONS
SHI was successful at attracting a motivated and committed group of undergraduate
students from around the United States to participate in their graduate school “boot camp.”
When asked on the survey whether they intend to pursue graduate studies after SHI, all ten
student participants responded positively. All but one student responded that they would
recommend the SHI program to a friend.
When asked why they would make that
recommendation, students had various reasons. The table below highlights some responses.
Reasons for recommending SHI to a friend…
“…Because it helps increase academic level. I feel like a much better student than before I
came.”
“…This program informed me on how to apply to graduate school and what personal statements
and CVs should be created.”
“…For the experience and to see if they actually want to go to graduate school, because
graduate school is not for everyone.”
“I would recommend this program, because it allows one the opportunity to develop as a
scholar.”
“…Great experience, looks good on resume, and offers great seminars…”
“…The experience was challenging yet fulfilling.”
“It is an enriching program, LA is a great place to intern, you make a lot of connections…”
“I gained confidence in knowing that I can go out of state and attend school somewhere else.”
This year’s student participants are supportive of the program’s ultimate goal and saw the
value in such a program that targets African American students from HBCUs. Through this
evaluation, three main areas were identified for program consideration and improvement.
21

More time and resources are needed to support independent and intensive
research.

To further engage students in both academic and social development, more
opportunities to participate in extracurricular activities should be provided.

As perhaps the most beneficial part of the program, intensive mentorship
opportunities should be further supported and developed.
Students who applied and participated in SHI came with the assumption that there would
be sufficient time and resources allocated for intensive, independent research. The program
should work to provide all needed resources to the student participants (i.e. rented laptops, Bruin
Cards, copy cards, etc.)
Although students did not suggest eliminating any of the other program components, it
was clear the amount of time allocated to workshops and seminars should be carefully
considered as the students expressed desire for more time spent on independent research and
extracurricular activities. Students want to take advantage of the resources available to them in
the urban Los Angeles area.
As stated in the report and well supported by the evidence in the survey results, students
had varied experiences with their mentors, but all students recognized the critical importance of
a good experience with their mentor for their development as a scholar during SHI. Those
students who found their mentors to be helpful and continuously engaged in their work were a
strong contrast to those students who felt neglected by their mentors. As such, we suggest that
the mentorship aspect of SHI be further strengthened and made consistent across participants.
22
FURTHER THOUGHTS
Collecting data from the student participants one time, at the end of a program limits the
types of questions one can answer about the SHI program. We could only gather student selfreported answers regarding their perceptions of gain in academic skills and overall personal
growth. The addition of multiple data collection opportunities throughout the program and other
sources of data would greatly enhance knowledge regarding the program’s effect. For example,
interviewing or surveying home institution advisors and faculty members could provide additional
insights into student growth that could be used to support student self-reports. Observations
during program activities could help corroborate or contradict students’ self reports. Content
analysis of student writing samples could provide evidence of academic skill gain. Longitudinal
data would also be extraordinarily beneficial to get a sense of the path students take after SHI.
23
APPENDIX A: Draft Theory of Action
Experiential
Learning Activities
Faculty Conducted
Seminars
Course Materials
Workshops
Mentoring
Experiences
Extracurricular
Activities
Writing and
Presenting
Research Paper
Academic Skill Set Expansion
and
Personal Growth (Immediate
Outcomes)
Preparation
Active participation
Knowledge of:
Conceptual approaches to
scholarship in a range of
humanities disciplines
Theoretical paradigms/
interpretative approaches
Discussion techniques
Analytical comprehension
Break down arguments by
recognizing the themes and their
organization in the materials.
Incorporation of published works
into student’s own scholarly work
Inter-relationship between Ideas
Comprehensive knowledge of the
technical aspects of the graduate
school/fellowship application
process.
Writing and research techniques.
One-on-one technical coaching
Social support
Personal and professional role
models
Embodied Experience (Bridging
the knowledge gained in the
classroom with practical
application)
Recognizing the difference between
primary and secondary sources.
Socializing with colleagues (i.e.
social network development)
[Within SHI Program & UCLA
SPUR Program]
Use of Citations
Development of arguments based
on critical and personal reflection
Organization and cohesiveness of
writing
Answer questions orally related to
work
Orientation & Socialization
(Intermediate Outcomes)

Intellectual Self
Confidence

Student Motivation to
Achieve

Academic Ability

Sense of Institutional
Belonging

Climate for Diversity
Ultimate Goal
Underrepresented Students
Earn Fellowships and Doctoral
Degrees in the Humanities
24
APPENDIX B: Focus Group Protocol




Thank them
Let them know the purpose of the focus group interview, who the evaluators are, and
how the personal interview will be an opportunity to elaborate more on personal
experiences. In addition, in both interviews, we want the real story. We will not be
judging your remarks because this evaluation is of the program and not you.
Furthermore, the program staff wants you to give your honest opinion about the
program.
Explain the purpose and instructions about the use of the “Experiential Learning
Activities” cards.
Discuss confidentiality and how they may stop at anytime if they feel uncomfortable.
1. O.K. I am going to start off with a little ice break to get everyone warmed up and talking. I
know it is a little weird to all start talking. My first question is an easy one, what activity
have you found to be the most useful at Summer Humanities Institute (SHI)? Look through
your cards and think about all of your experiences. Which one will you remember the most?
Why?
2. How were the seminars at SHI different than classes back at your home institution? Do you
feel as though the Faculty Seminars guided you to become more prepared and active
participants in scholarly discussions? Why or Why not?
 How do you prepare for the faculty conducted seminars?
 Conceptual frameworks
 Theoretical paradigms
 What kinds of research methods have you learned about?
 Interpretative approaches
 Analytical strategies
 Presentation techniques
3. Tell me about the course materials? What did you read? What was your favorite reading?
Why? How did you use the readings in your writing?
-How did you analyze the writing?
4. What techniques in applying for graduate school/fellowships did you learn in the Workshop?
Out of curiosity, how many of you have attended prior graduate school preparation
workshops? Do you think the SHI Workshops are different from other “Graduate School
Preparation” workshops, if so, why?
5. Did you use the writing and research techniques you learned in the workshops in writing
your research paper? Describe some of the techniques. Was this new information to you?
6. Explain how the process of writing a research paper was different in this program versus in
your other courses at your respective colleges/universities?
25
7. Describe the experience of presenting your research. What did you learn?
8.
What do you think was the purpose of the extracurricular activities? What kind of
knowledge did you gain from these experiences that you did not have in the Seminars?
9. On a scale of 1-10, with 1 being the least amount of frustration and 10 being the highest
amount of frustration, what has been your frustration or challenge level during SHI? Do you
feel like there was a purpose to this frustration? Did you learn from the pain and
frustration? Like the saying “grow through you pain,” how have you benefited or not from
this frustration level?
10. Name the most important thing you learned this summer.
11. Are you more likely to apply to graduate school after this summer experience? Why?
26





APPENDIX C: Interview Protocol
Thank them
Remind them the purpose of the individual interview
25 minutes
Confidentiality.
We want the real story. We will not be judging your remarks because this evaluation is
of the program and not you. Furthermore, the program staff wants you to give your
honest opinion about the program.
 Any questions before we start?
First, I will ask you questions regarding your academic and personal background and then move into
questions regarding your personal experience at SHI.
1. Where are you from? What was the community like? What college/university do you
attend? What was the main reason you chose this school?
2. Tell me about your intellectual growth over the last several weeks. Overall, do you feel more
confident in your intellectual abilities since participating in SHI?
3. Describe what the term “critical thinking” means to you. Did you use those skills this
summer? When?
4. Describe your relationship with your SHI mentor. How is the relationship similar or
different to the relationship you have with your advisor at your home institution?
5. As an underrepresented minority, can you see yourself in a graduate program?
 Can you foresee any obstacles or challenges?
 Do you think SHI addressed the obstacles and challenges you may face as an
underrepresented minority in a higher education? Why or why not?
6. If you were to pursue graduate studies, what would your contribution be to the academic
community? Did those ideas develop while here this summer?
7.
Any final thoughts?
27
APPENDIX D: Survey Instrument
SUMMER HUMANITIES INSTITUTE
Student Survey
1.
How useful were the following program components in helping you understand what graduate school is
like?
Program Component:
Most
useful
Very
useful
Useful
Somewhat
useful
Not
useful
Faculty Conducted Seminars
1
2
3
4
5
Course materials
1
2
3
4
5
Workshops
1
2
3
4
5
Mentoring
1
2
3
4
5
Extracurricular Activities
1
2
3
4
5
Writing Research Paper
1
2
3
4
5
Presenting Research Paper
1
2
3
4
5
2.
Which program component (listed above) contributed the most to your current understanding of what
graduate school is like? Why?
_______________________________________________________________________________
_________________________________________________________________
3.
Which program component (listed above) contributed the least to your current understanding of what
graduate school is like? Why?
_______________________________________________________________________________
_________________________________________________________________
4.
Do you intend to pursue graduate studies after SHI?
 Yes
5.
 No
 Undecided
Would you recommend SHI to a friend? Why or why not?
_____________________________________________________________________________________
_____________________________________________________________________________________
________________________________________________________________
Thank you very much for taking the time to complete this survey.
Your feedback plays an invaluable role in our efforts to strengthen the SHI program.
28