SEARCHING FOR EXCELLENCE AND DIVERSITY: EVALUATION OF WORKSHOP SESSION FOR CROSS‐COLLEGES PRESENTED OCTOBER 4, 2012 AND DECEMBER 6, 2012 Submitted to: Eve Fine Researcher and Workshop Coordinator, WISELI Submitted by: Julia Nelson Savoy Associate Research Specialist, WISELI Christine Maidl Pribbenow Evaluation Director, WISELI January 30, 2013 Preparation of this document was made possible by grants from the National Science Foundation (NSF #0619979). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
Table of Contents I. DEMOGRAPHICS OF SURVEY RESPONDENTS ........................................................................... 1 II. RATINGS AND COMMENTS ON WORKSHOP CONTENT ........................................................ 2 III. OUTCOMES: GAINED KNOWLEDGE AND RESOURCES THAT PARTICIPANTS WILL APPLY IN THEIR ROLE ON A SEARCH COMMITTEE .................................................................... 3 IV. SUGGESTIONS AND RECOMMENDATIONS GIVEN BY SURVEY RESPONDENTS ........... 4 A. IMPROVING THE WORKSHOP EXPERIENCE ........................................................................................................ 4 B. TOPICS THAT PARTICIPANTS HOPED WOULD BE COVERED IN THE WORKSHOP, YET WERE NOT ............... 4 C. WOULD YOU RECOMMEND THIS WORKSHOP TO OTHERS? .............................................................................. 4 V. GENERAL COMMENTS...................................................................................................................... 4 APPENDIX I. SURVEY INSTRUMENT ................................................................................................ 5 I. DEMOGRAPHICS OF SURVEY RESPONDENTS Out of 10 invitees, 3 people responded to this survey for a response rate of 30%. Title/Role on campus % (N) Associate Professor 33% (1) Professor 33% (1) Human Resources Professional 33% (1) Role on Search Committee % (N) Member of Search Committee 33% (1) Search Committee Chair 33% (1) Human Resources or Technical Support 33% (1) The source that informed them of the workshop offering % (N) Human Resources Office 100% (1) 1
II. RATINGS AND COMMENTS ON WORKSHOP CONTENT (Note: Written comments in this document are verbatim responses from workshop participants, altered in some cases to remove identifying information.) Rating Overall rating of workshop % (N) Average Very Useful 100% (3) 3.0 Somewhat Useful 0% (0) ‐‐ Not at all Useful 0% (0) ‐‐ Not at all Somewhat Very Rating Workshop Component Valuable Valuable Valuable NA Average Session 1: Introduction 0% (0) 0% (0) 100% (3) 0% (0) 3.0 (Fine) Session 1: Run an Effective and Efficient Search 0% (0) 0% (0) 100% (3) 0% (0) 3.0 Committee (Fine) Session 1: Actively Recruit an Excellent and Diverse Pool of 0% (0) 0% (0) 100% (3) 0% (0) 3.0 Candidates (Stern, Fine) Session 1: Evaluating the 0% (0) 0% (0) 100% (3) 0% (0) 3.0 Pool of Applicants (Carnes) Session 1: Ensure a Fair and Thorough Review of 0% (0) 0% (0) 100% (3) 0% (0) 3.0 Candidates (Fine) Session 2: Discussion of 0% (0) 0% (0) 67% (2) 33% (1) 3.0 recruitment efforts Session 2: Discussion of 0% (0) 0% (0) 67% (2) 33% (1) 3.0 evaluation of applicants Session 2: Develop and Implement an Effective 0% (0) 0% (0) 67% (2) 33% (1) 3.0 Interview Process (Piñero) Session 2: Dual‐Career Hiring 0% (0) 0% (0) 67% (2) 33% (1) 3.0 (Mayberry) Two respondents provided comments about the workshop’s components.  Establishing required qualifications and evaluation criteria prior to the position posting should be enforced a little bit more rather than recommending it.  This should all be done earlier if you want to impact pending or ongoing searches.... it is all too late in the fall to impact many searches. 2
III. OUTCOMES: GAINED KNOWLEDGE, ACTIONS PLANNED OR TAKEN, AND RESOURCES APPLIED ON PARTICIPANTS’ SEARCH COMMITTEES Knowledge, Action, or Resource Have Plan to Do Response Done (%/N) Count (%/N) Considered the membership of the search committee 100% (1) 0% (0) 1 and adjusted accordingly. Consulted with the workshop presenters or others on campus about conducting an effective search 0% (0) 100% (2) 2 process. Discussed and/or established ground rules for the search committee (e.g., about decision‐making, 67% (2) 33% (1) 3 attendance, expectations). Publicized the position in different venues 100% (1) 0% (0) 1 (compared to previous searches). Used networking and other means to recruit a 100% (2) 0% (0) 2 diverse pool of candidates. 0% (0) 100% (1) 1 Used "Recruiting Resources" on WISELI's website. 100% (1) 0% (0) 1 Used resources in the HR Recruitment Toolkit. Used resources provided by the Provost's Faculty Diversity Initiative (Strategic Pipeline and 0% (0) 100% (2) 2 Recruitment Fund). Distributed the brochure, "Reviewing Applicants: 50% (1) 50% (1) 2 Research on Bias and Assumptions." 67% (2) 67% (2) 3 Shared information about biases and assumptions. Developed and prioritized specific criteria for 100% (1) 0% (0) 1 evaluation of candidates. 100% (1) 100% (1) 1 Spent more time reviewing applications. Shared information about inappropriate questions 50% (1) 50% (1) 2 for on‐campus interviews and events. Considered needs and comfort of candidates when 50% (1) 50% (1) 2 planning on‐campus interviews. Relied on advice/resources in WISELI's Guidebook 0% (0) 100% (1) 1 for Search Committees. 50% (1) 50% (1) 2 Referred another person to this workshop. One respondent shared additional activities that they had done or planned to do. This included:  I plan to develop more specific and consistent Search Committee procedures. 3
IV. SUGGESTIONS AND RECOMMENDATIONS GIVEN BY SURVEY RESPONDENTS A. Improving the workshop experience (2 responses)  Discussions among ourselves were good, but I was expecting the presenters to go over the specific processes of how to do certain things.  Do it earlier in the fall semester or even summer to impact future searches. B. Topics that participants hoped would be covered in the workshop, yet were not (1 response)  Developing announcement that provides more specific information about the application/evaluation procedures other than just asking applicants to submit a cover letter and CV. If the announcement does not tell applicants what the Hiring Unit is looking for, they may not provide all the information that would make them qualified candidates. C. Would you recommend this workshop to others? Three of three (100 percent) evaluation respondents reported they would recommend the workshop to others. No reasons for the recommendation were provided. V. GENERAL COMMENTS One general comment was provided about the workshop.  This workshop should be mandatory or highly recommended to managers, supervisors, Dept Admins, HR reps, or anyone involved in the recruitment and selection process. 4
Appendix I: Survey Instrument
Searching for Excellence and Diversity: A Workshop for Search Committees
Thank you! As part of the workshop, you signed a consent form to participate in the evaluation of this series. Please take a few minutes to click on the button below and complete a survey. Results from you and your colleagues are reported in aggregate and are used to improve the workshop and to identify any outcomes from your participation. Thank you, in advance, for the time it takes you to complete this and for your candid feedback. Any questions? Please contact: Christine Pribbenow, (608) 263­4256; cmpribbenow@wisc.edu Searching for Excellence and Diversity: A Workshop for Search Committees
1. Your title or role on campus:
2. Your role on the search committee or in the search process:
5
6 3. Please rate the value of each of the aspects of the workshop (both Session 1 and
Session 2) using the scale below (Not at all valuable, Somewhat valuable, Very valuable).
Also, feel free to include additional comments in Question 4.
Session 1: Not at all valuable
Somewhat valuable
Very valuable
N/A
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
Introduction (Fine)
Session 1: Run an Effective and Efficient Search Committee (Fine)
Session 1: Actively Recruit an Excellent and Diverse Pool of Candidates (Stern, Fine)
Session 1: Evaluating the Pool of Applicants (Carnes)
Session 1: Ensure a Fair and Thorough Review of Candidates (Fine)
Session 2: Discussion of recruitment efforts
Session 2: Discussion of evaluation of applicants
Session 2: Develop and Implement an Effective Interview Process (Piñero)
Session 2: Dual­Career Hiring (Mayberry)
4. Please use this space for comments about any of the workshop's components:
5
6 Searching for Excellence and Diversity: A Workshop for Search Committees
5. Which of the following have you done (or plan to do) because of attending this
workshop? (Check either or both.)
Considered the Have done
Plan to do
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
membership of the search committee and adjusted accordingly.
Consulted with the workshop presenters or others on campus about conducting an effective search process.
Discussed and/or established ground rules for the search committee (e.g., about decision­making, attendance, expectations).
Publicized the position in different venues (compared to previous searches).
Used networking and other means to recruit a diverse pool of candidates.
Used "Recruiting Resources" on WISELI's website.
Used resources in the HR Recruitment Toolkit.
Used resources provided by the Provost's Faculty Diversity Initiative (Strategic Pipeline and Recruitment Fund).
Distributed the brochure, "Reviewing Applicants: Research on Bias and Assumptions."
Shared information about biases and assumptions.
Developed and prioritized specific criteria for evaluation of candidates.
Spent more time reviewing applications.
Shared information about inappropriate questions for on­campus interviews and events.
Considered needs and comfort of candidates when planning on­campus Searching for Excellence and Diversity: A Workshop for Search Committees
interviews.
Relied on advice/resources c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
c
d
e
f
g
in WISELI's Guidebook for Search Committees.
Referred another person to this workshop.
6. Please describe other activities you have done, or plan to do, because of this workshop:
5
6 7. Please provide us with ideas or suggestions that would have improved your experience
in this workshop:
5
6 8. What topics did you hope would be covered in this workshop, yet were not?
5
6 9. Please provide an overall rating for this session.
Not at all useful
Somewhat useful
Very useful
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
10. How did you hear about this workshop?
5
6 11. Would you recommend this workshop to others?
c Yes
d
e
f
g
c No
d
e
f
g
c Why or why not?
d
e
f
g
12. Any other comments?
5
6 Please click on the SUBMIT button below. You will know that your results have been recorded if you see WISELI's website about Searching for Excellence & Diversity. Feel free to browse through these resources. Thank you for completing this survey!