SHRL Assessment Plan and Report 2013-14
Department Name:
Assessment Plan for Academic Year:
Student Housing & Residential Life
2013-2014
Don Yackley
Mark Vitek, Assistant Director for Assessment and Student Conduct or Rebecca Szwarc, Administrative Assistant
for Assessment
We are a diverse community of students, staff and faculty who foster student success in living-learning
Department Mission Statement:
environments by building relationships, promoting engagement and leading with an ethic of care.
Director:
Assessment Contact(s)
(if not the director):
Department Vision Statement: Student Housing & Residential Life is the unrivaled experience in Student Success.
Department Goals:
1. Develop comprehensive First and Second Year Residential Experience programs inclusive of sequential, intentional and
structured co-curricular involvement opportunities.
2. Provide the highest quality customer service experience utilizing technology, training and resources to improve user
satisfaction.
3. Assess student employment opportunities and explore options for enhancing and improving such opportunities.
4. Establish and enhance a supportive, intentional and advocacy based program for residents that focuses on the provision
of services; cultivation of study and life skills; and the development of a vibrant campus life.
Program or Services Being Assessed: RA-Resident Interactions - Resident Connections with Staff and UH
Learning Outcome(s) and/or
Timeline
Method
Frequency
Program Objective(s)
(Month)
1
Purpose
Goal(s) Supported
SHRL Assessment Plan and Report 2013-14
Objective - Assess the Intentional
Interaction 'Programming' Model (how
RAs are interacting with their residents).
Learning Outcomes:
-Residents will identify and commit to
personal goals in the areas of (a)
Academic, emphasizing academic skills
and career exploration and
development; (b) Community
Involvement and Leadership, and (c)
Time Management.
--Residents will be able to describe how
their involvement in at least one
curricular or co-curricular experience
impacted their college experience.
Information from EBI survey,
The EBI survey will be
specifically Factor 1 - Satisfaction:
administered once in the
Hall/Apt Student Staff and Factor 2 Spring semester, and the
Satisfaction: Hall/Apt
Programming, and 4-6 institution- Campus Labs survey will be
specific questions, will be used to administered during the
rate satisfaction with RA staff and
Fall semester.
community involvement as a result
of RA interaction. In addition,
Factor 15: Learning: Mange Time,
Study, Solve Problems will be used
to rate time management. Goal is
to maintain current 5.80 mean
average on Factor 1; raise Factor 2
to average to at least 5.25; and
raise Factor 15 goal to at least 5.5.
In addition, actual interaction
specific involvement will be
measured by survey administered
through Campus Labs.
The Campus Labs survey
will be administered in
November, and the EBI in
mid Spring 2014 (mid
March to early April). This
will allow adjustments to
be made to RA
performance if necessary.
The purpose of this survey Department Goal #4 and
DSAES Strategic Initiatives
is to gauge the level of
residents involvement on #1 and #2.
campus and the
connection/ relationship
they have with their RA.
Data received will be
incorporated into
enhancements/ changes of
the Intentional Interaction
'Programming' Model.
Results:
We had originally intended to administer a survey specifically on resident connections with staff and UH, but because of fear of survey fatigue
(several other large surveys were being done), we decided not to do that survey. Therefore, we rated RA/Resident Interactions by using
questions/factors from the Resident EBI, which was administered at the end of the Spring semester. On the EBI, there were six factors related to
the intentional RA interaction that SHRL tried to foster: Satisfaction: Hall/Apt. Student Staff; Satisfaction (5.60): Hall/Apt Programming (4.91);
Learning: Personal Interactions (4.94); Learning: Faculty/Staff Interactions (4.53); Learning: Life Skills (5.05); and On-Campus Living Improved
Integration to College (5.32). Of these, only Hall/Apt Student Staff exceeded the 5.5 goal, but it also showed a statistically significant decline from
last year (-.16). Three others showed a statistically significant decline in score and are in the “issue” range: Hall/Apt Programming (-.15); Learning:
Personal Interactions (-.37); and Learning: Life Skills (-.16). Faculty/Staff Interaction is also in the “issue” range, and it is a new factor, so there is
no comparison to last year. On-Campus Living Improved Integration to College is likewise a new factor, and it was scored in the “needs work”
range.
Additionally, we asked three institution specific questions regarding RA interaction. On whether the RA had helped the resident increase their
interactions with other people, 53% answered in the “mildly disagree” to “mildly agree” range (3, 4 or 5 on a 7-point Likert scale). Asked if the RA
had helped the resident connect to campus resources and organizations, 49.6% were in that middle range. However, when we asked how often
they spoke with their RA, 54.9% indicated that they spoke with the RA at least once a week or more often, with “more than once a week” being
the most popular answer (21.9%).
2
SHRL Assessment Plan and Report 2013-14
Action:
We will explore changes to RA training that may include and ways to better evaluate the quality of RA/resident interactions.
Program or Services Being Assessed: Student Housing & Residential Life Department (SH&RL) General Services
Learning Outcome(s) and/or
Timeline
Method
Frequency
Purpose
Program Objective(s)
(Month)
Objective - Assess the general satisfaction
and perceived learning that occurs from
residents who live in residence halls.
Satisfaction Measurements: Hall/Apt Student
Staff; Hall/Apt Programming; Room/Floor
Environment; Facilities; Services Provided;
Room Assignment or Change Process; Safety
and Security; Dining Services. Learning
Outcomes: Judge Tolerance of Fellow
Residents; Judge Respectfulness of Fellow
Residents; Sense of Community; Personal
Interactions; Diverse Interactions; Manage
Time, Study, Solve Problems; Personal
Growth.
An assessment instrument The survey will be
distributed to all residents administered once a year
by EBI (Educational
in the Spring.
Benchmarking, Inc.). Goal
is to maintain 5.5 EBI goal
for 4 factors already above
it; increase to 5.5 or better
for at least 6 of 7 factors
currently at intermediate
level; and raise to
intermediate (5.25 or
better) of at least 7 of
remaining 8 factors.
Results:
3
The EBI will be
administered in early to
mid March, with results
available in April. In
addition, data will also be
compared to data from
other universities, which
becomes available in June
2014.
Goal(s) Supported
This survey measures resident Department Goals #2 & #4
satisfaction with SHRL services and DSAES Strategic
and personnel; as well as with Initiatives #1 & #2.
students' satisfaction with
their college experience. FY
2013-2014 results will be
compared to data from 2011
and 2012, and used to
enhance programs and
services in order to increase
overall resident satisfaction.
SHRL Assessment Plan and Report 2013-14
The ACUHO-I EBI Resident Survey was sent to all residents living in UH-managed housing except Resident Advisors, live-on staff and faculty. There
were 4,570 student residents, of which 1,620 residents responded, for a 35.5% response rate. The survey was administered later than we originally
planned, sent out in the latter part of April.
EBI changed its survey since last year, upping the number of factors from nineteen to twenty-two. Twelve factors remained essentially the same;
they combined or reworked and/or renamed five factors; and added five new factors. As in past years, questions are scored on a 7-point Likert
scale. The goal is to get a score of 5.5 or above, which represents a 75% satisfaction rate, and counted as “good”. Scores in the 5.25 to 5.49 range
are classified as “needs work”, and scores below 5.25 are classified as “issue”.
Of the three major indicator factors, Overall Satisfaction fell in the “needs work” range (5.42), and Overall Learning (4.94) and Overall Program
Effectiveness (4.94) fell in the “issue” range. SHRL had three scores in the “good” range: Satisfaction: Roommates (5.70), Satisfaction: Safety and
Security (5.64), and Satisfaction: Hall/Apt Student Staff (5.60). Five factors fell between 5.25 and 5.49: On-Campus Living Enhanced Retention and
Graduation (5.46), Overall Satisfaction (5.42), Satisfaction: Room/Floor Environment (5.36), Satisfaction: Facilities (5.35), and On-Campus Living
Improved Integration to College (5.32). The rest of the factors fell below 5.25: Satisfaction: Room Assignment or Change Process (5.23), Learning:
Diversity and Social Justice (5.09), Learning: Intrapersonal Development (5.07), Learning: Life Skills (5.05), Satisfaction: Services Provided (5.04),
Overall Learning (4.94), Overall Program Effectiveness (4.94), Learning: Personal Interactions (4.94), Satisfaction: Hall/Apt Programming (4.91),
Learning: LLC Connections and Support (4.89), Learning: Risk Behaviors (4.85), Learning: Stewardship (4.66), Satisfaction: Dining Services (4.54),
and Learning: Faculty/Staff Interactions (4.53).
We did not show the kind of improvement from last year we had projected. Last year, we showed statistically higher improvement in fifteen of
nineteen factors; improvement, but not statistical, in one measure; no change in two measures; and lower, but not statistical in one measure. The
two measures where there was no change were both above the 5.5 goal. This year, five factors showed statistically significant improvement over
last year: Satisfaction: Roommates (+.74), Satisfaction: Safety and Security (+.26), Satisfaction: Facilities (+.26), Overall Satisfaction (+.21), and
Satisfaction: Room/Floor Environment (+.20).
4
SHRL Assessment Plan and Report 2013-14
One factor showed improvement, but it was not statistically significant: Satisfaction: Room Assignment or Change Process (+.03). Three factors
showed declines in scores, but they were not statistically significant: Overall Program Effectiveness (-.07), Satisfaction: Services Provided (-.07),
and Learning: Risk Behaviors (-.09). Eight factors showed a statistically significant decline in scores: Satisfaction: Hall/Apt Programming (-.15),
Learning: Life Skills (-.16), Satisfaction: Hall/Apt Student Staff (-.16), Overall Learning (-.34), Learning: Personal Interactions (-.37), Satisfaction:
Dining Services (-.48), Learning: Diversity and Social Justice (-.67), and On-Campus Living Enhanced Retention and Graduation (-.82). The last five
factors did not appear on previous EBIs, so there was no comparison: Learning: Faculty/Staff Interactions, Learning: Intrapersonal Development,
Learning: Stewardship, Learning: LLC Connections and Support, On-Campus Living Improved Integration to College.
One particular high spot was the dramatic increase in satisfaction with roommates. This year, we implemented RoomSync, a roommate matching
service. The factor went from a fairly significant “issue” to well above the goal. The significant increase in this would seem to indicate that the
RoomSync program is a success. The decision to implement a roommate matching service was based on the results of previous assessments,
including previous EBI results and comments and focus groups’ input.
EBI provides specific recommendations for improvement based on which factors have the highest impact on our particular residents and showed
low performance. EBI’s recommendations as top priorities for improvement are: On-Campus Living Improved Integration to College, Satisfaction:
Dining Services, Learning: Personal Interactions, Learning: Diversity and Social Justice.
Action:
The results are currently under review by SHRL administrators. The results have not yet been shared with the rest of SHRL staff, but will be soon.
One result is that the RA Committee model currently in place is being re-examined, and based on these EBI results, will likely undergo some
restructuring of the hall programming part of the Interaction Model. In addition, MAP-Works will be better integrated into the model. No other
changes have been made so far.
Program or Services Being Assessed: Student Housing & Residential Life Department Desk Assistants
Learning Outcome(s) and/or
Timeline
Method
Frequency
Program Objective(s)
(Month)
5
Purpose
Goal(s) Supported
SHRL Assessment Plan and Report 2013-14
Objective - Assess effectiveness of the Desk
Assistants and the Desk Assistant Program.
In particular, specific objectives include:
- consistency of interpretation of job duties;
-effectiveness of supervision;
-customer Service;
-effectiveness of training.
Multiple Methods, including
review of all written materials
regarding desk assistant
program (training materials,
job description, etc.); focus
groups conducted with Desk
Assistants (DAs), Resident
Assistants (RAs), Residence
Life Coordinators (RLCs),
Customer Service
Representatives (CSRs); and
interviews with Housing Staff.
Separate focus groups will be Focus groups and interviews
held with each group (DAs,
will occur in September and
RAs, RLCs, CSRs), with at least October.
two focus groups for DAs and
RAs. Housing Staff Members
will be interviewed separately.
Materials review will occur
both before and after focus
groups.
The purpose is to evaluate the Department Goals #2 & #3
effectiveness of the desk
and DSAES Strategic
assistant program and the
Initiative #2.
level of customer service of
the Desk Assistants in terms
of promptness of service,
helpfulness, politeness and
problem resolving. Results
will be used to make any
adjustments to office training
and expectations.
Results:
The methodology used to evaluate the Desk Assistant program included a review of training materials and job descriptions by the SHRL
Assessment Committee. We then conducted five focus groups; one each with CSRs (Customer Service Representatives), RLCs (Residence Life
Coordinators) and RAs (Resident Advisors); and two groups with DAs (Desk Assistants). The CSR and RLC focus groups were well-attended; the
student worker focus groups were relatively small. Based on the focus groups, the program outcome, an effective DA program, was generally
achieved. However, participants agreed there were opportunities for program growth. This is known because opinions regarding the DA position
were fairly consistent across the groups.
Specific suggestions for improvement included:
 More in-depth training, including area-specific training.

Clearer communication of job duties, responsibilities, requirements and expectations.

More resource materials, such as a desk manual.

Better after-hours supervision (nights and weekends).

Clear and consistent disciplinary processes for no-shows, etc.

Consistent routine procedures (e.g., equipment, lock-outs, etc.) from area to area.

Increased profile or prestige for DA job.

More open communication between RLCs and SCRs.
Action:
A number of recommendations were made as a result of these focus groups and are on file for review. At this time, no changes to the program,
service content, method of provision, funding or personnel have been made. Because the DA and RA focus groups were small, one suggestion
made is that SHRL implement the EBI Student Worker Survey during the Spring Semester to all SHRL student workers to gather more complete
information.
6
SHRL Assessment Plan and Report 2013-14
Program or Services Being Assessed: First Year Experience Program (FYRE)
Learning Outcome(s) and/or
Method
Frequency
Program Objective(s)
Objective -To provide support to first year
residents by providing information on class
registration and financial aid; enhancing
study skills; and fostering the development of
social networks. Through these interactions,
FYRE students will acquire the knowledge
and confidence to persist to their second year
and beyond. Goal is to have a higher
retention rate among FYRE residents than UH
overall retention rate.
Learning Outcomes:
- Residents will persist in their studies and
return the following year.
-Residents will identify campus resources to
help them with their pursuits.
-Residents will participate in community
activities.
The "Map-Works" First
Year Transition Survey will
be sent to all freshmen
residents. A "Map-Works"
Fall Check-up Survey and
Spring Check-up Survey will
also be sent. GPAs and
retention rates will be
checked. In addition, short
surveys will be
administered after each
program.
Timeline
(Month)
The First Year Transition
Survey, Fall Check-up and
Spring Check-up will each
be administered once.
Program surveys will be
administered after each
program. GPA's will be
checked at the end of Fall
and Spring semesters.
Retention will be checked
in Fall 2014.
Results:
7
Purpose
Goal(s) Supported
The First Year Transition To gauge the extent to
Department Goal #1 & #4
Survey will be
which FYRE residents feel and DSAES Strategic
administered in
supported in their choices; Initiative #1.
September. The Fall Check- engage in university life;
and persist in their studies.
up will go out in mid
Results will be used to
October. The Spring
offer intentional
Check-up will go out in
intervention to at-risk
February. Program
residents, Results will also
surveys will be
be used to enhance the
administered after each
FYRE programming for the
program. GPA's will be
coming year.
checked in January and
June. Retention will be
checked in September
2014.
SHRL Assessment Plan and Report 2013-14
Originally, we had proposed to evaluate FYRE using MAP-Works, a comprehensive survey administered to all first-year students to evaluate their
risk of failing as a student. This was the first year we used MAP-Works, and we had limited knowledge about its use. Once we began to use it, it
became apparent that it would not be useful as an assessment of FYRE. However, we also administered the Resident EBI Survey to all residents,
and decided to compare First Year responses to the responses of other residents on campus as a measure of FYRE effectiveness.
The Resident EBI Survey was sent to all residents in UH-managed on-campus housing, 2,947 people, with a 35.5% response rate. Of those
respondents, 629, or 38.8% of the total sample, were first year students, and so part of the FYRE program. That represents 51.7% of the FYRE
program.
When comparing the responses of the first-year residents to the responses of the entire sample (which included the first-year residents), FYRE
students scored higher on 21 out of 22 factors (the last factor was equal to the entire sample). The scores were statistically significant in 17 of the
factors, including the overall indicators of Overall Satisfaction, Overall Learning and Overall Program Effectiveness. For Overall Satisfaction, the
FYRE score was 5.65, above the 5.5 goal (equivalent to 75%), as compared to the entire sample score of 5.46, which was in the “needs work”
range. In fact, FYRE residents had six factors that exceeded the 5.5 goal (compared to three for the total sample); six factors in the “needs work”
range (compared to five for the total sample); and ten in the “issue” range (compared to fourteen for the total sample). It should be noted that
given the higher scores of the FYRE residents, if their scores were removed from the total sample and a comparison was made between FYRE and
other residents not FYRE, the differences would be even more significant.
Regarding the program learning outcomes, “Residents will persist in their studies and return the following year” is best measured by actual
retention rates, which will not be available until September. However, the EBI factor “On-Campus Living Enhanced Retention and Graduation”,
which asked about students’ intensions, was 5.65, above the 5.5 goal, and above the total sample score of 5.46 (in the “needs work” range).
Additionally, in answer to an institution-specific question asking if they intended to continue to live on campus, 75.8% said yes (compared to 67.7%
overall). This indicates that FYRE residents perceive themselves as persisting in their studies, at least partially achieving the goal.
To measure “Residents will identify campus resources to help them with their pursuits”, we looked at the “LLC [Living Learning Community]
Connections and Support” factor. The factor score for the FYRE residents was 5.39, in the “needs work” range, but higher than the overall sample
score of 4.89 (in the “issue” range). However, when we looked at specific questions that made up this factor, FYRE respondents indicated that “As
8
SHRL Assessment Plan and Report 2013-14
(
g )
,
p
q
p
,
p
a result of my living-learning community, I am better able to:
• Use campus resources as needed” (5.79)
• Be academically successful” (5.61)
• Connect with fellow students within my living-learning community” (5.80)
These are all above the 5.5 goal, which would indicate that the learning objective, as stated, was met. The areas under this factor that had lower
scores were “Form study groups” (5.40 – “needs work”); “Connect with peer advisors/mentors” (5.11 – “issue”); and “Connect with
faculty/instructors” (4.70 – “issue”).
Since the purpose of the learning objective “Residents will participate in community activities” is to involve or integrate residents in college, we
looked at the “On Campus Living Improved Integration to College” factor to assess that goal. The FYRE score on this measure was 5.49, just 0.01
point below the 5.5 goal. The score for the entire sample was 5.32. On the other hand, we included institution specific questions asking how
involved they were with their resident hall community. Only 11.9% answered “involved” or “highly involved”. The answer with the highest
response was “somewhat involved” (25.2%), followed by “not necessarily involved, but not disengaged” (21.2%). We asked if they would like to be
more involved, and the highest response was “neutral” (31.6%), followed by “mildly agree” (22.3%). Highest reason why they were not involved
was “no time” (33.9%), followed by “activities not at a time when I am available” (21.8%). It would appear that we need to be clearer about what
we want to accomplish with our learning outcomes.
We will look at GPAs in mid-June and persistence rates in September as further evaluations of the FYRE program.
Action:
Since there were factors below the goal, there is room for improvement in the FYRE program. However, the data needs to be evaluated further to
determine if changes need to be made to the program. We are reevaluating our assessment method to see if we can find a better measure of the
program.
Program or Services Being Assessed: Desk Assistants (DAs)
Learning Outcome(s) and/or
Method
Program Objective(s)
Timeline
(Month)
Frequency
9
Purpose
Goal(s) Supported
SHRL Assessment Plan and Report 2013-14
Students would be satisfied with their
experience with their job demonstrated
by a 5.5 or better average mean on the
Student Affairs Student Employees
Assessment EBI.
The EBI Student Affairs
It will be administered
Student Employees
once a year at the end of
Assessment will be
the Spring Semester.
administered to the Desk
Assistants (DAs) by the
Division of Student Affairs
and Enrollment Services.
The survey was sent out to
all Student Affairs student
employees on the UH
payroll, but we just looked
at the data from SHRL
employees and compared
it to the general DSAES
data. Most questions are
measured on a 7-point
Likert scale. The goal is to
score 5.5 or better, which
is equivalent to a 75%
satisfaction rate.
Results:
10
It was administered in late To measure student
April/early May.
worker satisfaction with
their training, supervision
and job skill acquisition.
Department Goal #3 and
DSAES Strategic Initiative
#2c.
SHRL Assessment Plan and Report 2013-14
DSAES conducted the EBI Student Affairs Student Employee Assessment (SASEA), which was sent out to all DSAES student employees listed in
Human Resources, which included Desk Assistants, Sixty-eight (68) DAs responded to the SASEA, for a 29.3% response rate.
Most of the questions on the EBI are scored on a 7-point Likert scale. A score of 5.5 or better is considered “good” and is the goal, equaling a 75%
satisfaction rate. A score of 5.25 to 5.49 indicates a factor that “needs work”. Any factor that falls below 5.25 is considered an “issue”.
The SASEA had eleven factors on it: Support & Training, Quality of Supervision, Collaboration with Co-Workers, Co-Workers are Respectful,
Empathy, Self–Knowledge & Skills, Personal Competence, Diverse Interactions, Overall Satisfaction; Overall Learning, and Overall Program
Effectiveness. The DAs made the goal on four factors: Collaboration with Co-Workers (5.52), Empathy (5.67), Self–Knowledge & Skills (5.65), and
Diverse Interactions (6.04); the “needs work” range on six factors: Quality of Supervision (5.44), Co-Workers are Respectful (5.47), Personal
Competence (5.27), Overall Satisfaction (5.46); Overall Learning (5.29) and Overall Program Effectiveness (5.42); and the “issue” range on one
factor: Support & Training (5.18).
By contrast, the total DSAES student employees population made the goal on seven factors; four were in the “need work range, and none were
below that range. DAs scored higher than the DSAES population on three factors: Empathy (DSAES-5.49), Self–Knowledge & Skills (DSAES-5.58),
and Diverse Interactions (DSAES-5.92); and lower on eight factors: Support & Training (DSAES-5.45), Quality of Supervision (DSAES-5.68),
Collaboration with Co-Workers (DSAES-5.73), Co-Workers are Respectful (DSAES-5.49), Personal Competence (DSAES-5.48), Overall Satisfaction
(DSAES-5.71); Overall Learning (DSAES-5.51), and Overall Program Effectiveness (DSAES-5.68).
Since the SASEA was administered by DSAES, I do not have access to national comparison data on it, but will try to get it from DSAES.
These surveys would seem to indicate that we need to further explore these areas: training, staff support, quality of supervision, personal
competence, and respectfulness of co-workers. In particular, EBI recommends increasing efforts in Support and Training and Personal
Competence, because these areas have high impact on our DAs, but performance was low.
Action:
As a result of these surveys, SHRL will review the DA program, including how the assessment results might be incorporated into the training.
Program or Services Being Assessed: Resident Advisors (RAs)
Learning Outcome(s) and/or
Method
Frequency
Program Objective(s)
Timeline
(Month)
11
Purpose
Goal(s) Supported
SHRL Assessment Plan and Report 2013-14
Students would be satisfied with their
experience with their job demonstrated
by a 5.5 or better average mean on the
ACUHO-I Student Staff Assessment EBI.
The EBI ACUHO-I Student It will be administered
Staff Assessment will be
once a year at the end of
administered to the
the Spring Semester.
Resident Advisors (RAs).
Most questions are
measured on a 7-point
Likert scale. The goal is to
score 5.5 or better, which
is equivalent to a 75%
satisfaction rate.
Results:
12
It was administered in late To measure student
April/early May.
worker satisfaction with
their training, supervision
and job skill acquisition.
Department Goal #3 and
DSAES Strategic Initiative
#2c.
SHRL Assessment Plan and Report 2013-14
SHRL conducted the EBI ACUHO-I Student Staff Assessment (ASSA) and administered it to all Resident Advisors (RAs). One hundred-fourteen (114)
RAs responded to the ASSA, for a 70.8% response rate.
Most of the questions on the EBI are scored on a 7-point Likert scale. A score of 5.5 or better is considered “good” and is the goal, equaling a 75%
satisfaction rate. A score of 5.25 to 5.49 indicates a factor that “needs work”. Any factor that falls below 5.25 is considered an “issue”.
The ASSA had eighteen factors, and was specifically designed for student workers in residence halls. The RAs made or exceeded the 5.5 goal on
five factors: Empathy (5.56), Residents are Respectful (5.77); Self-Knowledge & Skills (5.50), Personal Competence (5.51), and Diverse Interactions
(5.91); scored in the “needs work” range on six factors: Supervisor Supporting Student Staff (5.28), Management Skills of Supervisor (5.41),
Practical Competence (5.43), Overall Satisfaction (5.37), Overall Learning (5.40), and Overall Program Effectiveness (5.34); and were in the “issue”
range on seven factors: Student Staff Selection Process (4.81), Job Expectations (5.17), Job Demands and Compensation (4.93), Types of Training
(4.84), Training (4.87), Collaboration within Staff Team (4.93), and Residents are Tolerant (5.23).
When comparing UH scores to other universities (Select 6, Carnegie Class and All Institutions), only one factor scored above the respective means.
“Residents are Respectful” (5.77) scored slightly above the Select 6 mean of 5.74 and ranked 5 of 7 (Carnegie Class was 12 of 20; All Institutions
was64 of 87). “Types of Training” ranked somewhat near the middle of the groups (4 of 6, 14 of 20 and 68 of 87, respectively). The overall scores
rankings were Overall Satisfaction: 5 of 7, 15 of 20, and 72 of 87; Overall Learning: 5 of 7, 15 of 20, and 70 of 87; and Overall Program
Effectiveness: 5 of 7, 14 of 20, and 72 of 87. The other factors rank at or near the bottom: Student Staff Selection Process: 7 of 7, 20 of 20, 82 of
87; Job Expectations: 7 of 7, 20 of 20, 85 of 87; Job Demands and Compensation; 6 of 7, 17 of 20, 75 of 87; Supervisor Supporting Student Staff: 6
of 7, 19 of 20, 80 of 87; Training: 5 of 7; 18 of 20, 80 of 87; Empathy: 7 of 7, 19 of 20, 84 of 87; Collaboration within Staff Team: 7 of 7; 20 of 20, 84
of 87; Residents are Tolerant: 7 of 7, 19 of 20, 80 of 87; Self Knowledge and Skills: 6 of 7; 18 of 20, 82 of 87; Personal Competence: 6 of 7, 16 of
20, 79 of 87; Practical Competence: 6 of 7, 19 of 20, 82 of 87; and Diverse Interactions: 7 of 7; 15 of 20, 72 of 87.
These surveys would seem to indicate that we need to further explore these areas: training, student staff selection process, job expectations, job
demands and compensation, collaboration and resident tolerance. In particular, EBI recommends increasing efforts in Job Demands and
Compensation and Training, because these two areas have high impact on our RAs, and there was low performance.
Action:
As a result of these surveys, SHRL will review the RA program, including how the assessment results might be incorporated into the training.
Program or Services Being Assessed: On-Campus Partnership Properties
13
SHRL Assessment Plan and Report 2013-14
Learning Outcome(s) and/or
Program Objective(s)
Method
Timeline
(Month)
Frequency
Residents of the partnership properties The EBI Apartment
The Apartment EBI will be The Apartment EBI was
will score at or above the 5.5 goal on a 7- Assessment will be
given one time at the end given at the end of
point Likert scale, which is equivalent to administered to the
of the Spring semester.
April/beginning of May.
a 75% satisfaction rate
residents of the privatelymanaged apartment
complexes on campus
(Cambridge Oaks and
Cullen Oaks). Most
questions are measured on
a 7-point Likert scale. The
goal is to score 5.5 or
better, which is equivalent
to a 75% satisfaction rate.
Scores will be compared to
comparison university
groups and to scores from
Resident Assessment EBI
administered to residents
of SHRL-managed resident
facilities.
Results:
14
Purpose
To evaluate the
satisfaction and program
effectiveness of the
public/private partnerships
that provide on-campus
housing.
Goal(s) Supported
Departmental Goals #2
and #4, and DSAES
Strategic Initiatives #1d,
#2e and #5c.
SHRL Assessment Plan and Report 2013-14
SHRL administered the EBI Apartment Assessment to the Partnership Properties, Cambridge Oaks and Cullen Oaks. The survey was sent to 304
people in Cambridge Oaks and 767 in Cullen Oaks, for a total of 1,071. Cambridge Oaks had a 26.9% response rate and Cullen Oaks had a 12.6%
response rate, for an overall response rate of 16.5%. The assessment is scored on a 7-point Likert scale, with the goal being to score an average
mean of 5.5 or better, which represents a 75% satisfaction rate. Factors that score in the 5.25 to 5.49 range are considered to “need work,” and
factors that score below 5.25 are considered an “issue.”
There were sixteen factors on the Apartment Assessment, and the apartments as a whole scored in the “issues” range (below 5.25) on all of them:
Apartment Selection Criteria (5.11), Satisfaction: Contract and Lease (3.51), Satisfaction: Apartment Condition (3.85); Satisfaction: Apartment
Environment (4.20), Satisfaction: Apartment Staff and Policies (4.02), Satisfaction: Services and Facilities Provided (3.73), Satisfaction: Safety and
Security (3.86), Satisfaction: Apartment Programming (3.75), Satisfaction: Dining Services (Residents who have meal plans) (5.18), Learning:
Diversity and Social Justice (4.34), Learning; Intrapersonal Development (4.07), Learning: Life Skills (4.39), Learning: Interpersonal Interactions
(3.88), Overall Satisfaction (3.69), Overall Learning (3.21), and Overall Program Effectiveness (3.02).
Separately, Cullen Oaks scored in the “needs work” range on one factor, “Apartment Selection Criteria” (5.26).
When compared to the other institutions, the partnerships ranked last of all Institutions on five factors (#70 of 70); last of the Carnegie Class
institutions on an additional five factors (#28 of 28); and last in the Select 6 on an additional two factors (#7 of 7). There were four factors in which
they were not ranked last in at least one group. The one area where they did better than some means was Dining Services, where their mean
(5.18) was better than the Carnegie Class (5.17) and All Institutions (4.85) (Select 6 mean was 5.31). Their rank was 3 of 7 for Select 6, 9 of 28 for
Carnegie Class, and 13 of 70 for All Institutions. Although their mean was lower than the comparison means for Intrapersonal Development, they
ranked near the middle (4 of 7, 17 of 28, and 36 of 70, respectively). The other factors in which they were not last were Diversity and Social Justice
(6 of 7, 26 of 28, and 67 of 70, respectively) and Overall Learning (6 of 7, 27 of 28, and 69 of 70, respectively). Of the Overall indicators, they
ranked 69 of 70 on Overall Satisfaction and Overall Learning, and 70 of 70 on Overall Program Effectiveness.
When compared to analogous scores on the Resident Assessment, both Cullen Oaks and Cambridge Oaks, both separately and together, scored
statistically significant lower on all factors except Dining Services (Cambridge Oaks/Total-5.18; SHRL-4.54). Also, Cullen Oaks’ “Apartment
Selection Criteria score (5.26) was slightly higher than SHRL’s “Room Assignment or Change Process” score (5.23), but it was not statistically
significant.
Action:
EBI’s specific recommendations suggest increasing efforts for Satisfaction: Contract and Lease; Learning: Life Skills; and Satisfaction: Apartment
Programming as areas where impact is high and performance is low. We will present these results to the partnership properties, and will offer
assistance in determining possible action steps to improve their respective programs.
Program or Services Being Assessed: SHRL Marketing
15
SHRL Assessment Plan and Report 2013-14
Learning Outcome(s) and/or
Program Objective(s)
Marketing would be able to identify
ways that might make them more
effective.
Method
In order to avoid sending
out yet another survey, ten
questions will be added to
the institution specific
section of the Resident EBI.
The scope of the questions
is to gather information
rather than evaluate
services.
Timeline
(Month)
Frequency
Purpose
The Resident EBI, with the The survey was
To identify ways to make
institution specific
administered at the end of our marketing more
questions, will be sent out April/beginning of May.
effective.
once at the end of the
Spring semester.
Results:
16
Goal(s) Supported
Department Goal #2 and
DSAES Strategic Initiatives
#5b and #5c.
SHRL Assessment Plan and Report 2013-14
We added ten institution-specific questions on the EBI Resident Assessment related to marketing. The Resident EBI was sent to all residents
(4,567) in UH-managed facilities, with 1,620 people responding for a 35.5% response rate. The purpose of the questions was to elicit information,
rather than evaluate services.
When asked how they first heard about SHRL, the most frequent answer was Orientation (34.6%), distantly followed by the SHRL website (18.4%)
and Email (15.0%). When asked for the most important reason why they chose to live on campus, the three most frequent answers were
“Family/Home too far away” (26.6%), “Convenience” (24.5%) and “Did not Want to Commute” (20.6%), with Improve Grades a distant fourth at
13.8%. However, when asked for other factors that influenced their choice to live on campus, “Convenience” was number one (18.2%), followed
by “Did not Want to Commute” (17.3%) and “Improve Grades” (14.0%). It should be noted that while “having fun” and “sleeping late” are often
promoted by Marketing as important reasons for living on campus, they were much farther down both lists. The next question was open-ended,
and asked if there were other reasons not listed above. The two answers that were cited most often were “wanted independence”, sometimes
expressing the idea that they liked the “safety net” of living in residence halls (29 out of 129 actual comments); and that they wanted the “full
college experience” (19 out of 129).
When asked about the SHRL website, 61.7% of the respondents rated it in the middle (Somewhat Difficult, Average, or Somewhat Easy to
navigate). “Average” was the most frequent answer (35.5%), The “Easy” and “Very Easy” responses accounted for 30.2% of the answers, with
“Easy” being 19.2%. Those who ranked it “Difficult” or ‘Very Difficult” accounted for 8.2% of the responses. When asked what they would like to
see on the website, the most frequent response was “Chart comparison of all residence halls” (15.9%), “Living expenses calculator” (14.1%),
“Calendar of events for each area and overall” (10.7%), “Safety and security information (10.6%), and “Information about improvements made to
facilities or programs (10.5%). When asked an open-ended question about what they would like to see on the website, common responses
included better/more accurate pictures of rooms, links to support services, better information on how to get things done/handle problems, and
just do a better job with the website.
When asked if they planned to live on campus next year, 67.7% said yes, 22.8% said no, and 9.5% were unsure. For those who answered “no”,
when asked why they were not returning, 22.0% said that it was for financial considerations, 20.2% were graduating, and 20.0% wanted to live off
campus. Another 15.2% said that they were unhappy with on-campus living arrangements.
Action:
The results of this survey will be shared with the Marketing Team to help refine some of their strategies. They may wish to revise their list of
reasons to live on campus to include independence in a supportive setting and the full college experience. They may also wish to de-emphasize
having fun and sleeping late as reasons to live on campus. The Marketing Team is currently working on a redesign of the SHRL website, and can
use this information to inform their work on the website.
17