MSAP PRC 2015 pg.1
2015
PRC Self-Study Report
MSAP PRC 2015 pg.2
Program Director Self-Study Report
Master of Science Applied Psychology
Submitted by:
Libby Smith, M.S.
Interim Program Director
Submitted to:
2015-2016
Planning and Review Committee
Table of Contents
1.
UW-STOUT’S STRATEGIC PLAN ........................................................................................................................ 4
1.1
2.
3.
4.
5.
UW-Stout's Strategic Plan ........................................................................................................................ 4
DESCRIPTION OF THE PROGRAM .................................................................................................................... 6
2.1
Curriculum Design .................................................................................................................................... 6
2.2
Faculty/Academic Staff Expertise............................................................................................................. 9
2.3
Facilities .................................................................................................................................................. 10
2.4
Resources for the Program .................................................................................................................... 11
QUALITY OF THE PROGRAM GRADUATES ..................................................................................................... 12
3.1
Program Demand ................................................................................................................................... 12
3.2
Alumni Follow-up Surveys ...................................................................................................................... 13
3.3
PRC Surveys ............................................................................................................................................ 14
PROGRAM EVIDENCE OF CONTINUOUS IMPROVEMENT ............................................................................. 18
4.1
Comparative Strengths & Weaknesses .................................................................................................. 18
4.2
Response to Past Recommendations ..................................................................................................... 20
Attachments .................................................................................................................................................. 24
5.1 Links to specific program information ........................................................................................................ 24
MSAP PRC 2015 pg.3
Summary of Tables
Table 1: Application & Enrollment Data 2011-2015 ............................................................................................... 4
Table 2: Retention & Graduation Data 2009-2015 ................................................................................................. 5
Table 3: Research & Experiential Learning in MSAP Courses ................................................................................. 8
Table 4: Key MSAP Faculty ...................................................................................................................................... 9
Table 5: Placement Data for 2011-2013 ............................................................................................................... 12
Table 6: Occupational Outlook for Select MSAP Job Titles................................................................................... 13
Table 7: PARQ Alumni Follow-Up Survey Data ..................................................................................................... 13
Table 8: PRC Student Survey - Program Quality ................................................................................................... 14
Table 9: PRC Student Survey - Program Strengths ............................................................................................... 14
Table 10: PRC Student Survey - Areas of Opportunity ......................................................................................... 15
Table 11: PRC Student Survey - Program Details .................................................................................................. 16
Table 12: PRC Faculty Survey - Program Quality .................................................................................................. 17
Table 13: Comparison of Similar Programs........................................................................................................... 19
MSAP PRC 2015 pg.4
1. UW-STOUT’S STRATEGIC PLAN
1.1
UW-Stout's Strategic Plan
1.1.1 Describe early and ongoing experiential learning opportunities to students within the program.
The foundation of the MS Applied Psychology (MSAP) program is progressive, experiential learning.
Students engage in real-world research projects in their first semester, working with clients to develop
research or evaluation plans. In their second semester the students execute those plans in small work
groups and present their results to clients.
Into their second year in the program, they begin to work independently, with faculty supervision, on
research or evaluation practicums. In some cases, students may have several different applied projects in
their third semester depending on course selection.
Finally, the students engage in a required internship experience in which they get to apply the range of
skills they have developed throughout the program. Additionally, they engage in a capstone independent
research project, completing either a thesis or a Plan B field problem.
1.1.2 Describe program initiatives employed to support and/or increase student enrollment,
retention and graduation rates.
Enrollment: The MSAP was an early adopter of the Hobson’s CRM system, implementing a prospective
student communication plan in fall 2013. We have found that this has improved our communication with
students during the application stages. Prior to our use of Hobson’s we had as many as 20 students who
were accepted, but did not respond to the letter of acceptance. This number has gone down to less than
5 since we began using Hobson’s. Our acceptance rate has exceeded expectations the past two years and
the show rate (62% in 2014, 65% in 2015) has exceeded our historical rate of 51%. Table 1 summarizes
data about applications received and the resulting cohort size.
Table 1: Application & Enrollment Data 2011-2015
Cohort
2011
2012
2013
2014
2015
Applications Received
43
30
48
39
38
Students Accepted
31
23
33
34
31
Cohort Size
23
15
13
21
20
Retention: Early applied projects and engagement with real-world research has been the primary factor
leading to our strong retention rates. Since 2011, 94% of students have been retained through the
completion of their required coursework. Table 2 summarizes data regarding course completion
(retention) and graduation (thesis & internship completion).
MSAP PRC 2015 pg.5
Table 2: Retention & Graduation Data 2009-2015
Cohort
2009
2010
2011
2012
2013
2014
2015
Cohort Size
11
18
23
15
13
21
20
Course Completion
11 (100%)
18 (100%)
21 (91%)
15 (100%)
11 (85%)
21 (retained to fall)
NA
Cohort Graduation*
10 (91%)
12 (67%)
17 (81%)
8 (53%)
5 (38%)
NA
NA
*Percentages based on cohort size, calculated differently than the 7-year graduation rate
Graduation: Efforts to increase the graduation rate have primarily focused on integrating independent
research project requirements into the three Applied Psychology Seminar courses (PSYC-710, 711, & 712).
These courses were first offered in fall 2011. Research milestones such as topic selection, literature
review drafts, & proposal completion have been integrated as deliverables in these courses. This has
helped students to manage their timeline and be closer to completion by the time they complete their
coursework. Ongoing efforts to improve the thesis completion rate, and help all students complete in a
more timely manner, include development of a Thesis Guide, as well as a move to on campus (rather than
online) courses during the summer between the first and second year. Students who make thesis
progress during this summer are much more likely to complete in a timely manner.
1.1.3 Describe, provide examples, and explain how the program intentionally integrates diversity efforts,
functions and contributes to the program in support of Inclusive Excellence: “UW-Stout’s plan to
intentionally integrate diversity efforts into the core aspects of everything we do. Diversity is broadly
defined and includes, but is not limited to, race/ethnicity, gender, sexual orientation, age, and
disability status.”
Inclusiveness and diversity are integrated into the course objectives for each of the Applied Psychology
Seminar courses (see course objectives below). The assignments and application vary each semester
depending on activities on campus and other external opportunities. Every effort is made to meet this
objective through an applied learning experience. The Tents of Witness event on campus is a good
example of this. This campus-wide event was co-sponsored by the Psychology Department and included
interactive displays on six different historical genocides. The experience asked students to confront issues
such as religious oppression, human-trafficking, & violent conflict. MSAP students were asked to
independently engage in the experience, develop a reflection of their experience, and finally a class
discussion was held to debrief.
MSAP PRC 2015 pg.6
Figure 1: PSYC 711 - Applied Psychology Seminar II Course Objectives
Additionally, there is significant focus throughout the program on diversity of perspective and cultural
competence as it applies to conducting research and evaluation. Understanding the needs of diverse
stakeholders is central to conducting high quality evaluations.
Students have also elected to do practicums and internships with the Office of International Education,
Multi-cultural Student Services, and multiple off-campus organizations that serve the needs of diverse
populations. This allows some students the opportunity to further apply their learning in this area.
1.1.4 Describe environmental sustainability initiatives embedded and supported by the program: “UWStout’s attempt to make students, faculty, and staff more aware of the importance of sustaining our
environment through energy conservation, waste reduction, and other measures that will not bring
harm to the environment, and to provide students with innovative research opportunities in these
areas.”
Currently, there are no embedded elements related to environmental sustainability. However, the
program has a strong relationship with the Sustainability Office and a number of students have done
practicums, internships, and held GA positions there. Administratively, the program has encouraged
sustainability through discouraging students from printing documents and requiring nearly all program
deliverables to be submitted electronically. In 2014-2015, the MSAP converted to entirely digital files for
student records. Further, we strongly encourage students to carpool to off campus events and
conferences.
2. DESCRIPTION OF THE PROGRAM
2.1 Curriculum Design
2.1.1 State the approved program objectives.
Objective 1: Applied Research and Assessment Skills
Proficiency in designing, conducting, evaluating, and communicating research in applied organizational
settings. Competence in this domain encompasses the entire applied research process and is
MSAP PRC 2015 pg.7
demonstrated through expertise in defining the purpose of the investigation, identifying key constructs
and mapping their relationships, selecting or designing valid and reliable means of measurement,
capturing and maintaining data, performing analyses, interpreting results and creating suitable reports for
a variety of audiences.
Objective 2: Systemic Thinking Skills
Demonstrate skills in the application of psychological concepts, theories, and methods towards effecting
behavioral change and/or organizational improvement. Competence in this domain is achieved and
demonstrated through studying and applying systems-based theories and methods related to group and
organizational dynamics within courses, applied projects, internships and other opportunities for
experiential learning.
Objective 3: Technical Proficiencies and Communication Competence
Demonstrate appropriate technological and communication proficiency required to support excellence in
professional practice. Competence in this domain is exemplified through demonstrated abilities in
employing appropriate software to ensure data integrity; communicating with various stakeholders
through preparation of written documents, and oral presentations to stakeholder audiences, including
both informal/small group and formal large audience venues.
Objective 4: Interpersonal and Teamwork Skills
Work effectively with and within groups to establish and achieve individual, group, and organizational
goals. Competence in this domain is established and shown through studying and applying theories of
interpersonal effectiveness, group dynamics, formal and personal ethical principles as well as appreciation
for diversity in its many forms towards contributing to productive partnerships with others.
Objective 5: Professional Development
Define personal strengths, opportunities for improvement, values, and ethics, and apply these towards a
process of lifelong learning. Competence in this domain is indicated by the ability to perform critical selfassessment, define specific goals, and employ means-ends problem solving skills toward achieving
personal and professional growth.
2.1.2 Describe processes and initiatives employed in determining the need for program revision.








Findings from the yearly Assessment in the Major reports
Faculty assessment of learning and competence in each of the core objectives via a portfolio and
formal mid-program review process
Student’s self-assessment of learning and competence in each of the core objectives via a portfolio and
a KSA survey
MSAP faculty expertise regarding the continual development of their respective professional areas
Regular review and comparison of competing masters programs
Student and alumni feedback regarding professional readiness and skill set needs
Program Advisory Committee feedback and suggestions, especially regarding Applied Psychology
market needs/demands
Regular communication with employers and review of open positions in the field
MSAP PRC 2015 pg.8

Enrollment trends and assessment of the demand for Applied Psychologists in the professional
marketplace
2.1.3 Check all that apply regarding the program:
_X Traditional, on campus program
___ Offsite location
___ Online program
2.1.4 Briefly describe the components of your program where students participate in scholarly activity
such as: research, scholarship, experiential learning, and creative endeavor. “programs are
presented through an approach to learning which involves combining theory, practice and
experimentation” (UW-Stout’s Mission Statement)
Students in the MSAP engage in research and experiential learning in nearly every course in the program.
Forty-two of the 45 credits offered through the psychology department provide these types of learning
opportunities. Table 3 describes how each course provides these activities.
Table 3: Research & Experiential Learning in MSAP Courses
Course
PSYC 790 – Research Core
Research Design & Analysis I
PSYC 792 – Research Core
Research Design & Analysis II
PSYC 793 – Research Core
Psychometrics of Test Construction
PSYC 707 - Research Core
Applied Social Psychology
PSYC 781 – I/O Concentration
Personnel Psychology
PSYC 782 – I/O Concentration
Organizational Psychology
PSYC 750 – Evaluation Concentration
Foundations of Evaluation Research
PSYC 751 – Evaluation Concentration
Applications of Evaluation Research
PSYC 771 – Health Concentration
Foundations of Health Promotion
PSYC 772 – Health Concentration
Applications of Health Promotion
PSYC 752 – Professional Core
Practicum in Applied Psychology
PSYC 770 – Research Core
Thesis Applied Psychology
PSCY 890 – Professional Core
Applied Psychology Internship
Activity
Data collection, data analysis, report development
Data collection, data analysis, report development
Survey development, data analysis, report development
Literature review, research proposal
Literature review, research proposal
Literature review, research proposal
Evaluation plan development for client, instrument development
Conduct evaluation, data collection, analysis, reporting
Literature review
Needs Assessment, program development for client
Independent consulting with client, may include research plan &
instrument development, data collection & analysis, reporting
Independent research project (Plan A or B), proposal
development, data collection & analysis, scholarly paper
Application of MSAP related skills in a work-based learning
opportunity, minimum of 240 hours
MSAP PRC 2015 pg.9
2.1.5 Does your program currently have an accreditation or certification agency that reviews the
program? If so, which agency and to what extent does it influence the structure of the curriculum?
The program is not accredited or certified by an outside agency. There are not currently any accrediting
bodies for the fields of psychology in which we provide training. Organizations that provide accreditation
for Psychology programs are typically focused on programs that have a clinical or counseling component.
2.2 Faculty/Academic Staff Expertise
2.2.1 List key instructors in the program. A key instructor is one who teaches at least one required
professional course in your program (this should be the combined faculty of Key A and Key B who
were surveyed by the PRC).
Table 4 provides an overview of current faculty, what they can teach, and the number of credits taught in
the 14-15 academic year. The MSAP program has lost three key faculty members over the past three years.
Needs are discussed in the section below.
Table 4: Key MSAP Faculty
Dr. Sarah Wood
Credits taught
in ‘14-15
MSAP Courses
Research Design & Analysis
9
Industrial/Organizational courses
Practicum & Internship
6
Health Promotion courses
Evaluation courses
6
Internship
Applied Social Psychology (every other year)
6
Psychometrics
Dr. Chelsea Lovejoy
0
Applied Social Psychology (every other year)
Dr. Mike Mensink (2015)
0
Research Design & Analysis (Spring 2016)
Cori Beskow, M.S.
3
Libby Smith, M.S.
3
Practicum & Internship (as needed)
Program Director
Applied Psychology Seminars
Faculty Member
Dr. Alicia Stachowski
Dr. Kiki Gorbatenko-Roth
Dr. Tiffany Smith
2.2.2 What additional faculty/academic staff expertise is needed?
Our greatest area of need is an additional faculty member in Industrial/Organizational Psychology. The
majority of our students select the I/O concentration. Ideally we would have two instructors in this area in
order to give the students both a broader perspective on the discipline. Additionally, many of the students
elect to do an I/O specific topic for their thesis and having another faculty member in this discipline would
help support those students. Multiple searches have been conducted, but we have not been successful in
retaining or hiring faculty from this discipline.
MSAP PRC 2015 pg.10
Further, we are currently without a primary faculty member to teach in the Health Promotion & Disease
Prevention concentration. Dr. Cam Weaver left the University in May 2015 and currently there are no
plans to fill this spot. The health concentration has been discussed at length over the past two years (and
prior). It underwent a revision in 2010 that sparked strong enrollment in 2011, but has since lacked a
faculty person to champion its worth. Since 2011, applications for this area have been down dramatically
and the decision was made in 2014 to not enroll any new students who elected Health as their primary
concentration. Course offerings have been altered accordingly, PSYC 771 Psychological Foundations of
Health Problems will be offered every other year (this course is shared with the MS in Food & Nutrition
program). PSYC 772 Psychological Applications of Health Promotion will not be offered unless there is
appropriate student demand.
2.3 Facilities
2.2.3 Describe facilities and or capital equipment currently used and how it supports or strengthens the
program? What program specific facilities (unique classrooms, labs, additional space involving minor
construction) have been requested and provided?
The program currently uses very little in the way of specialized facilities or capital equipment. The current
difficulties with the available classroom space and work space were highlighted in both the PRC Student
Survey and the Faculty Survey. The classrooms that have been used over the past two years have often
been too small and poorly suited to the collaborative nature of our courses.
There is an office in Jeter Hall with one computer available for student use. It is rarely used by the students
due to location and small space. Students often express frustration at the difficulty of finding appropriate
space for group work, often conducting group meetings in public spaces such as our local coffee shops &
restaurants.
In 2012, the previous MSAP program director requested the inclusion of an applied research experience
laboratory in the Harvey Hall renovation, with the purpose of providing students with opportunities to
engage in applied research, particularly research that benefits regional organizations. Equipment and
supplies for this laboratory included computers and software (e.g., SAS Statistics and Sequel software) for
data analysis and reporting, facilities and furniture to conduct focus groups and telephone interviews (e.g.,
conference room, telephone banks, and recording devices), and psychological assessment and test library
to gain access to common tests used in psychological research in applied settings. This request was denied.
However, a space for conducting focus groups was included as part of the renovation. This space will be
shared with several other programs.
2.2.4 What added facility needs (if any) such as unique classrooms, labs, additional space involving minor
construction exist in the program?
The Harvey Hall improvements will expand the available space for student group work and provide a more
suitable location for classes. An MSAP-dedicated graduate student office on the 4th floor will include space
for group work which is very much needed. However, it is likely that the space will be in high demand and
will not fully serve the needs of 40+ students.
The most pressing need moving forward will be access to collaborative work spaces. There will be two
classrooms in Harvey Hall that are set up in this fashion (see below). There will be multiple programs that
MSAP PRC 2015 pg.11
will compete for space in these classrooms and it remains to be determined if the MSAP students will
benefit from the use of these spaces. The majority of our classes involve group work on applied projects;
learning and collaboration would be greatly enhanced through use of these classrooms.
2.4 Resources for the Program
2.4.1 Evaluate the quality, relevance, and quantity of the library resources to support the program. Include
a brief statement as to how these needs have been met by the library.
The needs of the MSAP program are met by both the library staff and the available resources. There are a
handful of journals that are not directly accessible, but when students work directly with a research
librarian it is rare for them not to be able to obtain an article they are searching for. The library does make
regular requests for suggestions and a more thorough evaluation of our requests is needed and will be
conducted in 2015-2016. Library staff also provide invaluable services, including thesis writing support,
literature review support, & online resources such as citation managers.
2.4.2 List any special resources used to meet program and/or student needs such as: Learning
Technology Services for curriculum materials development, ASPIRE, Research Services, Advisement
Center, Disability Services, Multicultural Student Services, etc.
Planning, Assessment, Research, & Quality & Applied Research Center: A primary source of real-world
research experience for MSAP students. Many PARQ/ARC staff are alumni of the MSAP program and
often serve as unofficial advisers and mentors to the MSAP students. Over the past several years with
faculty shortages, they have been called upon to teach courses within the program. Many students have
the opportunity to do graduate assistantships in this campus office. They have also offered practicum and
internship opportunities to students.
Discovery Center: Another primary source of real-world research experience for MSAP students. Many
students take advantage of the opportunity to work on research projects. The center staff has supported
students doing practicums, internships, graduate assistantships, thesis projects, and other extracurricular
research opportunities.
Research Services: Given the applied nature of the program, students and faculty work with Research
Services across their time in the program. Specific services needed are Human Subjects Training,
Institutional Review Board review, acquisition of research funds such as the Student Research Grant, and
establishment of contracts and agreements. They have also offered graduate assistantships to our
students.
Career Services: They provide job search, interviewing, and resume writing workshops to the MSAP
students as part of the Applied Psychology Seminar courses. Some students have met individually with
career services counselors. Career Services has also assisted students with practicums and thesis projects
and have offered research-based graduate assistantships to past students.
Writing Center: Each year, the few MSAP students needing extra assistance in writing skill training are
referred. A representative from the Writing Center is also invited to the first semester seminar course to
discuss their services with students.
MSAP PRC 2015 pg.12
Counseling Center: Students who have encountered difficulties coping with school and/or life issues have
been referred to the Counseling Center. Anecdotally, students have found the services to be helpful. They
have also offered graduate assistantships to our health concentration students.
Office of International Education: Each cohort typically has at least one international student. OIE has
provided assistance directly to students and to the program director. They have been especially helpful in
navigating the paperwork needed for international students to remain in the country through their
internship experience and thesis completion.
2.4.3 Describe other resources (if any) needed to meet the program objectives?
Continued access to high quality graduate assistantships is critical to the success of our students. The
extracurricular research opportunities set us apart from our competitors in regard to recruitment.
Additionally, these applied experiences allow students to build their resumes while they complete their
coursework. Access to these paid positions also allows students to work on-campus which research shows
leads to more engaged students and improved student outcomes.
Graduate assistant funding has been restricted in the 2015-16 due to budget cuts and puts constraints on
our ability to successfully recruit new students. It also puts a financial strain on current students who had
previously had sufficient funding for their graduate education.
3. QUALITY OF THE PROGRAM GRADUATES
3.1
Program Demand
3.1.1 Describe program graduate demand and/or anticipated changes or trends impacting the future
demand.
Anecdotally, we know that our graduates are in demand. The program director receives regular (monthly,
if not weekly) requests for upcoming or recent graduates who are available to fill open positions. We
regularly receive feedback via client and supervisor evaluations that indicate that our students are strong
candidates for employment. Anecdotally, we also know that the strongest students obtain employment
easily. Table 5 presents employment data for the three cohorts for which we have complete records.
Table 5: Placement Data for 2011-2013
Cohort (N)
2011 (21)
2012 (15)
2013 (11)
Prior to Graduation
38% (8)
47% (7)
36% (4)
Within 6 mo of Course Completion
81% (17)
87% (13)
82% (9)
Within 1 Year of Course Comp.
95% (20)
87% (13)
NA
A 2012 survey of MSAP graduates indicates that new graduates spend 3.3 months on average searching
for their first professional position. New graduates stay in this first job an average of 2 years and spend
just 2.2 months search for their second professional position.
National demand for applied psychologists remains strong. The following table presents data from the US
Department of Labor Bureau of labor Statistics Occupational Outlook Handbook
(http://www.bls.gov/ooh/home.htm). The selected job titles are all within the scope of the jobs that are
MSAP PRC 2015 pg.13
held by MSAP alumni.
Table 6: Occupational Outlook for Select MSAP Job Titles
Selected Job Titles
I/O Psychologists
Market Research Analyst
Operations Analyst
Management Analyst
Survey Researcher
Human Resources Specialists
Compensation & Benefit Managers
All Other Psychologists
3.2
National 2012
Median Pay
$83,580
$60,300
$72,100
$78,600
$45,050
$55, 640
$95,250
$90,020
2012 – 22 Job
Outlook
+53%
+32%
+27%
+19%
+18%
+7%
+3%
+11%
Growth Pace
Much Faster
Much Faster
Much Faster
Faster
Faster
Slower
Slower
Average
Alumni Follow-up Surveys
3.2.1 Interpret the data provided by the Planning, Assessment, Research, and Quality (PARQ) office of the
alumni follow-up surveys.
The most problematic issue with Alumni Follow-up survey results is the low response rate. Seven is the
highest number of respondents on any of the given surveys over the past three administrations. These
small samples make it difficult to place much emphasis on the quantitative results. Key questions were
analyzed for the AIM report and are reported here; the results are favorable, but provide limited evidence
due to the small sample.
Results for four questions from the Alumni Follow-Up survey are presented here. Full results can be found
in Image Now. Alumni were asked the following questions in each of the past three survey
administrations:
 What is your perception of your occupational preparedness compared to that of co-workers
trained at other colleges/universities?
 How would you rate the overall effectiveness of your program/major?
 If you had to do it over again, would you enroll in the same program?
 Please rate the degree to which you were challenged by the curriculum.
Alumni responded on a four-point scale where, 1 = a negative response and 4 = a positive response. It is
important to note that while the overall response rate for the Alumni Follow-Up survey is typically above
35%, there are few responses from MSAP alumni. Each survey group below has seven or fewer
respondents. Table 7 presents a summary of these responses, including averages scores and the standard
deviation (in parentheses).
Table 7: PARQ Alumni Follow-Up Survey Data
2010 Survey
2004 5-year 2008 1-year
Occupational
Preparedness
Overall
Effectiveness
2012 Survey
2006 5-year 2010 1-year
2014 Survey
2008 5-year 2012 1-year
3.00 (.00)
3.00 (.00)
3.00 (.00)
0
3.29 (.49)
3.00 (.00)
3.33 (.50)
3.00 (.00)
3.25 (.50)
0
3.29 (.49)
4.00 (.00)
MSAP PRC 2015 pg.14
Enroll in Prg
Again
Challenged
by Program
3.67 (.23)
3.00 (.00)
3.25 (.28)
0
3.14 (.23)
2.75 (.23)
3.00 (.00)
3.20 (.45)
3.00 (.00)
0
3.60 (.55)
3.00 (.00)
Moving forward, emphasis has been placed on maintaining current email addresses for program alumni.
Primary email addresses will be collected from students prior to graduation and maintained in the
MSAP Assessment Database. The Alumni Follow-up survey will be administered by PARQ in spring 2016.
3.3
PRC Surveys
3.3.1 Student Survey
Forty students were surveyed in the spring of 2015. Twelve were from the 2013 cohort, 21 from the
2014 cohort and seven were students from the 2012 cohort who have not yet finished their thesis.
Thirty-one of the forty students completed the survey for a response rate of 78%. Students were first
asked to responded to a series of four questions, to which they could respond on a scale where 1 =
strongly disagree and 5 = strongly agree. The table below presents the individual responses and mean
scores for each question.
Table 8: PRC Student Survey - Program Quality
Question
Strongly
Disagree
Disagree
Neutral
Agree
Strongly
Agree
Objectives are clear & met
0
2
3
20
6
3.97
Prepared for Profession
0
0
5
23
3
3.94
Quality Program
0
1
2
19
9
4.16
Choose Again
0
2
4
13
12
4.13
Mean
Students were then asked a series of three qualitative questions about program strengths and areas of
improvement, as well as a question about ideas for improvement. The tables (9 & 10) below summarize
the key themes for each question. Complete responses are included as an appendix. The majority (71%)
of students identified the hands-on applied learning throughout the program as a key strength. The
most common theme that emerged from the questions about areas for improvement was the perceived
over-emphasis of SPSS training, combined with the desire to learn other software applications for
statistical analysis.
Table 9: PRC Student Survey - Program Strengths
Theme
Hands-on applied learning,
projects, working with realworld clients
Percent of
Respondents
71% (22)
Examples



It teaches the practical applications of the theories
that we learn.
Experience working with clients
Internship and Practicum gives us real-world
experience
MSAP PRC 2015 pg.15

Faculty & staff provide
expertise, support, care
about students, help us
prepare
29% (9)
Professional development
opportunities, networking,
guest speakers
Prepares to work in the
real-world
19% (6)


13% (4)




Cohort Model, peer support 13% (4)


Multiple concentrations

10% (3)
The people involved care. They care about us and
want to see us succeed.
Knowledgeable and approachable faculty/staff.
The professors were also very kind and willing to
help.
The guest speakers added value
Networking opportunities are some of the programs
biggest strengths.
I think it does a great job preparing students for the
actual workforce.
This program is all about planning, scheduling,
meeting objectives, completing tasks, just like in a
job setting
Having a supportive cohort
Group work help us to develop our interpersonal
skills and team spirit
The three emphases--You have the ability to tailor
your degree and employers love that I have
multiple emphases
Table 10: PRC Student Survey - Areas of Opportunity
Theme
Need to focus on other
stats software
Percent of
Respondents
29% (9)
Examples


Too much focus on SPSS
26% (8)

Psychometrics should be
offered in a different
format
13% (4)



Not enough professional
development, networking,
need funding
13% (4)
Program is poorly
13% (4)



Learn more applications for stats (R, SAS, Excel,
etc.)
Incorporate other software tools for data analysis
(Excel, R, SQL/Access)
SPSS is the main focus of analysis in the MSAP
program. I think this needs to change.
Only working with SPSS
While having a summer class was nice, I think that
there are some class that would be better suited for
the online format than Psychometrics
Psychometrics should be offered in person and
social psychology should be conducted online over
the summer
We should get funding to attend related
conferences such as MPPAW or SIOP.
I would also prefer more professional networking
facilitated in the classroom. I love working with
professionals to set up my practicum experience,
getting guest speakers in class, and doing projects
with professionals. The program does well with that
now, but I wish there was more.
The thesis was not stressed enough at the
MSAP PRC 2015 pg.16
organized, need more
advising, thesis help
beginning of the program; it came up so quickly
that picking a topic at the end of the program
seemed like a daunting task that needed to be done
too quickly.

It needs to be better organized for the students. I
would have loved to have had a suggested course
matrix that told me what courses were offered each
semester for the 2 years I am in the program.
While there were many helpful comments offered for the question about ideas for improvement, just
one theme emerged. The quality of the classrooms was mentioned by 16% of respondents, which brings
us to the next set of survey questions. The lowest mean rating (3.03) was for the quality of the
classrooms, highlighting the problems experienced during the Harvey Hall renovation.
The highest mean rating (4.65) was in response to the question about faculty making themselves
available to students. Our small number of program faculty are highly dedicated and are deeply involved
in student learning both in and out of the classroom. High mean ratings for “faculty provide current and
relevant information” (4.26) is also indicative of their involvement with the program. Consequently,
student learning outcomes (critical thinking skills (4.29), problem solving skills (4.39), written
communication skills (4.32)) are also strong, a further tribute to the quality of our program faculty.
Table 11: PRC Student Survey - Program Details
Question
Strongly
Disagree
Disagree
Advisement meets my needs
0
4
3
15
9
3.94
0
1
0
20
10
4.26
0
0
0
11
20
4.65
0
0
2
19
10
4.26
0
0
4
19
8
4.13
0
2
9
14
5
3.73
2
9
6
14
0
3.03
0
0
18
11
0
3.38
0
3
4
15
9
3.97
0
1
0
18
12
4.32
Faculty provide current and
relevant information
Instructors make themselves
available
Courses achieve stated
objectives
Evaluation procedures
appropriately measure
learning
Little or no unnecessary
repetition or overlap
Classrooms provide a good
environment
Laboratory and equipment
good environment.
Library resources are useful
and up-to-date.
Written communication skills
enhanced
Neutral
Agree
Strongly
Agree
Mean
MSAP PRC 2015 pg.17
Question
Oral communication skills
enhanced
My critical thinking skills have
been enhanced
My problem solving skills
have been enhanced
My attitude towards racism
and ethical diversity have
been matured
My attitudes about having a
global perspective have
matured
Strongly
Disagree
Disagree
0
1
3
14
13
4.26
1
0
0
18
12
4.29
0
0
1
17
13
4.39
0
1
12
12
6
3.74
0
1
9
16
5
3.81
Neutral
Strongly
Agree
Agree
Mean
3.3.2 Faculty Survey
Fourteen faculty members were asked to respond to a survey in spring 2015. Seven faculty members
completed the survey for a response rate of 50%. Similar to the students, the mean rating (3.71) for
classroom quality was the lowest. Leadership (4.83) and communication (4.83) from the program
director were rated as strengths.
Table 12: PRC Faculty Survey - Program Quality
Strongly
Disagree
Disagree
Neutral
Agree
Strongly
Agree
NA
Classroom Quality
0
1
2
2
2
0
3.71
Lab Quality
0
0
1
2
1
3
4.00
Instructional Technology
0
0
1
3
3
0
4.29
D2L Training
0
0
1
3
3
0
4.29
Students Prepared
0
0
0
5
2
0
4.29
PD Communication
0
0
0
1
5
1
4.83
PD Leadership
0
0
0
1
5
1
4.83
Decision Participation
0
0
0
1
4
2
4.80
Question
Mean
Faculty were also asked about the program’s strengths and areas of opportunities. There were not
enough responses to provide a summary of themes. There were no areas of opportunity mentioned
more than once. A few strengths mentioned more than once were:



The applied nature of the program
Active role of the program director
Students are well-prepared at enrollment
A report inclusive of all comments is included in the appendix.
MSAP PRC 2015 pg.18
3.3.3 Program Advisory Committee Survey
Fifteen members of the Program Advisory Committee (PAC) were surveyed in spring 2015. Ten surveys
were completed for a response rate of 66%. Two PAC meetings are scheduled each year and nine of the
10 respondents indicated their attendance at two yearly meetings. All 10 respondents were satisfied
with the frequency of the meetings. Further all 10 members indicated that they receive data about the
program, have the opportunity to provide feedback, and believe that their recommendations impact
program decisions.
Like the students and faculty, PAC members were asked to share what they viewed as the strengths and
areas of opportunity for the program. Again, there were not enough responses to provide a summary of
themes. The most commonly expressed strength was the applied nature of the program and the number
of hands-on research experiences provided to students. There were no common themes among the
areas of improvement, but faculty turnover and the weakness of the Health concentration were
mentioned. A report inclusive of all comments is included in the appendix.
4. PROGRAM EVIDENCE OF CONTINUOUS IMPROVEMENT
4.1
Comparative Strengths & Weaknesses
4.1.1 Describe program strengths distinguishing it from similar programs. Describe and explain program
weaknesses.
After nearly 15 years, the MSAP remains a unique offering due to amount of applied work integrated into
the program, as well as the multiple concentrations offered. The table below offers a comparison with the
other programs that MSAP applicants most frequently submit applications, Mankato & St. Cloud. The
Masters offering at University of Minnesota-Duluth is relatively new (first offered in 2014), but shares
similar characteristics with the MSAP. It remains to be seen if they become a competitor. Again, the key
feature which distinguish us from these similar programs is the amount and nature of applied
experiences, both within the program and across the campus (ARC, Discovery Center). Secondly, the fact
that we are an Applied Psychology program, rather than solely focused on I/O Psychology, is seen as a
strength because we prepare students for a diversity of professional settings. Further, it would appear on
the surface to be more rigorous than these other programs. The number of similarities makes it difficult
to identify weaknesses as compared to these other programs. One key weakness that we appear to share
is the low number of I/O specific faculty.
MSAP PRC 2015 pg.19
Table 13: Comparison of Similar Programs
UW-Stout
Master of Science
in Applied
Psychology
I/O
Concentrations Evaluation
Available
Research
Health Promotion
Degree
Offered
MN State - Mankato
Master of Arts
Degree in I/O
Psychology
MN State –
St. Cloud
UM-Duluth
MS in I/O
Psychology
Master of Arts in
Psychological
Science
None
None
I/O
Experimental
Clinical/Counseling
Duration
2 years; Full-time
2 years; Full-time
2 years; Full-time
2 years; Full-time
# Credits
42 credits
44 credits
41 credits
36 credits
Independent
Research
Requirement
6 credit Plan A
Thesis or Plan B
Field Problem
Majority of
courses, 240
Internship
required, ARC,
Discovery Center,
Multiple GA
positions
3.0 UG GPA
No GRE
Psychology Bkgrd
3 credit thesis
6 credit Plan A
Thesis or Plan C
Portfolio
6 credit Plan B
Research Project in
Psychology
Many classroom
projects, 3 month
internship,
Consulting Center
(OERG)
Not emphasized
Not emphasized
3.0 UG GPA
GRE Required
Psychology Bkgrd
3.0 UG GPA
GRE Required
Psychology Bkgrd
3.0 UG GPA
GRE Required
Psychology Bkgrd
I/O: 2
Experimental: 5
Clinical: 5
Opportunity
for Applied
Experience
Admission
Requirements
# Faculty
Aligned with
Program
3 Core
5 Additional
4 Core
4 Additional
3 Core
Program Cost
(approximate)
In-state: $17.8 K
Out-State: $35K
In-state: $17.8K
Out-state: $24K
In-state: $17.8K
Out-state: $24K
In-state: $46K
Out-state: $71K
Program
Delivery
Format
Traditional oncampus; No
distance education
Traditional oncampus; No
distance education
Traditional oncampus; No
distance education
Traditional oncampus; No
distance education
MSAP PRC 2015 pg.20
4.2
Response to Past Recommendations
4.2.1 Submit evidence of program response to the concerns and recommendations from previous
program review.
Recommendation #1: Work with the Department Chair and Dean to successfully fill vacant faculty positions.
Multiple searches have been conducted since this recommendation was made in 2009. It has been
difficult to both hire and retain faculty. Two key faculty members left the University within two years of
their hire. Other searches for MSAP specific faculty have failed. Candidate pools have been generally
weak. The department successfully hired a key faculty member for the Evaluation Research
concentration, but continues to search for an additional I/O faculty person (anticipated search in 2015).
There are currently no plans to search for someone for the Health promotion concentration due to the
low number of applicants and the limited number of searches that can be conducted in any given year.
Recommendation #2: Work with the Dean and the Advisory Council to find additional funds for more graduate
assistantships.
This has been an area of strength for the program since this recommendation was made. Generally
speaking, every student who wants a GA position has had one over the past five years. The Psychology
Department is able to offer between 10 and 12 positions (typically 8 to 13 hours a week) each semester
depending on course offerings. These positions are funded through a variety of sources, including the
Provost, CEHHS, and Access to Learning funds. Further, MSAP students are frequently employed in GA
positons across campus including, PARQ, ARC, Discovery Center, Counseling Center, Ethics Center,
Research Services, & Career Services. However, the recent state-level budget cuts have led to restrictions
and fewer opportunities for GA work, so this will remain a point of emphasis moving forward.
Recommendation #3: Work with the Dean and Advisory Committee to review and revise the admission
requirements.
The current admissions requirements have been in place since I became Program Director in fall 2013and
there has been no discussion about changing them. Currently, the program requires a 3.0 GPA, a
minimum of 15 semester credits in Psychology, including methods and statistics, evidence of research
experience, and demonstration that they would be a good fit for the program (career goals aligned,
strong interpersonal skills). The preparation and quality of our incoming students was cited as a strength
of our program by our faculty. The lack of a GRE requirement is the primary factor that separates us from
our regional competitors, but there is currently no identified need to make this a requirement for our
applicants.
Recommendation #4: Work with the Dean and Advisory Committee on student and faculty recruitment
strategies.
Student recruitment has been improved through several of the previously identified methods, including
website development and the purchase of GRE lists and other graduate applicant lists. However, the key
factor that has increased the show rate has been the level of personal contact between the program and
the applicant. This has been done both through the hard work of the MSAP Program Director, Assistant
and Graduate Assistant, as well as the Hobson’s CRM system.
Faculty recruitment and retention remains problematic. The Psychology Department has experienced the
departure of ten tenured or tenure-track faculty since 2011. During that same time frame nine searches
were conducted. Six of those searches were successful, the remaining three failed. Of the six that were
MSAP PRC 2015 pg.21
successful, three of those people have left the University. Two of those three were MSAP faculty. All three
of the failed searches were for faculty that would have taught in the MSAP program.
Two searches are planned for the 2015-16 academic year, but due to budget constraints they have not
yet received final approval from the Provost. There is significant concern that the searches will not begin
in time to lead to successful hires. In past years, administrative delays are also believed to have
contributed to the inability to hire a qualified candidate. The timing of the offers was such that candidates
had already accepted other positions.
Recommendation #5: Work to improve student access to computers, statistical software, and technical support
and to provide training as needed.
The actions proposed by the 2009 program director in response to this recommendation have been
largely completed and those specific needs continue to be satisfied. All MSAP students come prepared
with a laptop and are required to install SPSS. The statistical software is offered at a discounted and
affordable price (about $50 a year) through WISC Software. SPSS is also available on the computer in the
Graduate Student office in Jeter Hall (this will be more accessible once Harvey Hall opens again) and is
also available on the computers in the library. Further, they are largely satisfied with the technical
support provided through ASK5000 and the training provided through My Lynda.
However, this remains an area of focus as students continue to express the desire to learn additional
statistical software packages (Excel, SAS, R, and SQL).
Recommendation #6: Work with the Chairs of the Psychology and Math, Stat and C.S. Departments to develop
and/or revise courses in Statistical Analysis and Health Psychology.
Psychology 790 & 792, Research Design & Analysis I, II, were revised in 2010 and have significantly
increased the amount of statistical training that the students receive. PSYC 771 Psychological Foundations
of Health Problems & PSYC 772 Psychological Applications of Health Promotion, the two Health
Promotion concentration courses, were also revised and offered for the first time in 2010.
Recommendation #7: Work to clarify program requirements and expectations to students.
The actions proposed in 2009 have been implemented and remain a key part of the success of the
program. Students attend a mandatory summer orientation prior to program start, revised seminar
courses ensure that students receive regular communication about program requirements and have an
opportunity to ask questions. Students also receive regular feedback about progress and expectations via
the portfolio process and the mid-program review process.
Recommendation #8: Work on increasing your availability to students.
There have been several program director changes since this recommendation was made, but availability
of the program director has been largely satisfied through the addition of the Applied Psychology Seminar
courses (PSYC 710, 711, & 712) in 2011. Students see the program director every week for at least one
hour. Additional advising meetings are scheduled each semester, as well as individual meetings at the
mid-program review. PRC Student Survey results indicate that the students are very satisfied with the
availability of the current program director.
Recommendation #9: Work with the Dean and Advisory Committee to review the viability of the Health
Psychology concentration.
The action plan developed in 2009 was completed, leading to a revision and re-naming of the Health
MSAP PRC 2015 pg.22
concentration in 2011. Following the revision the concentration experienced a strong uptick in
enrollment, but this dropped off sharply in subsequent years with the lack of a faculty champion. The
decision was made in 2014 to not enroll new students who sought Health Promotion as a primary
concentration. Current students are still able to obtain this concentration however. The Psychological
Foundations of Health Promotion class (PSYC 771) will be offered every other fall as it is shared with
another program. Students can fulfill the second course requirement through an appropriate elective
from another department, including Health Education, Health, Wellness, & Fitness, and Counseling.
Currently, there is not enough demand to offer PSYC 772, Psychological Applications of Health Promotion.
4.3 Planned Program Improvements
4.3.1 In the next seven years, what major improvements or changes are planned for implementation to
improve program quality?
 A minor program revision is needed to clarify and improve the MSAP Core Objectives. The 2014-15
PAC Committee, including key MSAP faculty, provided input and approved an update of the core
objectives. Objective 4 (Technical & Communication Proficiency) needs to be divided into two
separate competencies. Additional updates to the language will help clarify the objectives and the
ways in which students demonstrate that they have met the objective.
 Evaluate and act on other program revisions.
o Between 2012 and 2014 it became challenging for students to enroll in INMGT-750
Organizational Development. Prior to this time the course was routinely offered in the fall
semester, which is the most viable time for MSAP students to take this course. When the
course moved to only being offered in the spring, we began to evaluate other courses that
would be suitable for this concentration. MSAP Faculty are interested in substituting PSYC570 Interpersonal Effectiveness.
o As was mentioned earlier, no new students were enrolled in 2015-16 with Health Promotion
as a primary concentration. Currently, there is no faculty support for the two Health
Promotion courses. Two key faculty had been evaluating a change to the concentration that
would better align with their areas of expertise, but one of them has since left the University.
The Health concentration is still a marketable concentration and viable pathway to successful
careers, but without faculty support it will be difficult to maintain the concentration. A final
decision on this concentration will be needed within the next two academic years.
o With the prospect of going down to two concentrations, we must evaluate the
program plan and whether or not having concentrations makes sense. The
concentrations are a unique and marketable aspect of the program, less so if there are
just two concentrations that everyone takes (90% of students elect two
concentrations). The addition of a new, third concentration comprised of classes
already offered is another possibility that will be evaluated.
 As highlighted in the feedback received in the PRC Student Survey, students have been asking to get
additional training in statistical software other than SPSS. A survey was conducted with a convenience
sample of recent MSAP alumni to determine what software was most typically used in their MSAP-
MSAP PRC 2015 pg.23
related careers. The results of that survey indicated that MS Excel was most commonly used, and that
more complex packages like SAS and R were rarely used despite the fact that they often show up in
position descriptions. These results are consistent with a survey conducted with MSAP alumni in 2012
that indicated that advanced inferential statistics was the least used MSAP-related skill. Consequently,
key MSAP faculty are developing plans to integrate the use of MS Excel into the Research Design &
Analysis courses.
 Students have reported the need to learn about qualitative research and analysis earlier in their
training. During the past two years, the section on qualitative analysis has come at the end of the
second semester. However, students are finding that they are increasingly using qualitative methods
in their applied projects in other courses and need to have this information and experience sooner.
Further, the students have asked to have qualitative analysis software (NVivio) incorporated into the
training. MSAP faculty will evaluate the appropriate course and timing for qualitative analysis and
make course revisions as needed.
 Due to the amount of faculty turnover in recent years, we have experienced some issues with
foundational topics (e.g. validity, reliability, survey use, ethics) being covered in different courses at
the same time (same day, same week). These topics are covered in multiple classes due to the need to
apply them to specific situations or projects. Redundancy at some levels is important, but when the
topic is covered on the same day, it can be difficult for students to see the need and the lack of
engagement that results is detrimental to the student and the course outcomes. An analysis of course
content by semester to identify potential overlap will be conducted. Further, faculty will be
encouraged to share and discuss course schedules in order to avoid this undesirable situation.
 While students have been overwhelmingly successful in obtaining internships over the past several
years, there is a desire to further develop key relationships with industry partners in order to create
yearly internship opportunities for MSAP students. These standing internships would both strengthen
the relationship between the industry partner and the MSAP, and offer the student a stronger
developmental experience.
 In recent years, students have more actively engaged with external networking opportunities
including professional events in the Twin Cities and attendance at national conferences. While funding
is available through the University for students who are presenting research, funding for simply
attending is not available. This often serves as a barrier for greater participation in these events. The
goal would be the development of funding stream for MSAP student professional development
opportunities. This will be the focus of the work for the PAC committee this year.
 Our strong connection to our program alumni has served us well in many capacities, including
mentoring, guest speaking, and job opportunities. However, these connections remain informal and
there is no regular communication with program alumni. A goal is to develop formalized
communication with alumni and to further integrate their involvement into the seminar courses.
MSAP PRC 2015 pg.24
5. Attachments
5.1 Links to specific program information



Current assessment in the major
 https://goo.gl/Y6PS0b
Program plan sheet
 http://goo.gl/GJVPyO
Individual program facts

https://goo.gl/BddjIi

Current program advisory committee

Appendix of qualitative comments from PRC surveys


https://goo.gl/RxB5Yd
https://goo.gl/V4GQb8