Just Do It:
Data Collection
March 24, 2011
Shults Center Forum
Student Development Divisional Assessment Team
Agenda





Part 1: Welcome
Part 2: Review of the assessment process
Part 3: Data collection and measurement tools
Part 4: Connecting to the Division Strategic Plan
Part 5: Q & A
Welcome
Participants will be able to…
 Describe the differences between the following terms:
qualitative, quantitative, direct, indirect and
performance based/authentic.
 Identify the appropriate data collection method to
assess their department’s student learning outcome.
 Make connections between their department’s student
learning outcome and the Divisional Strategic Plan.
The Assessment Process:
A Continuous Cycle
1. Establish
Outcomes or
Goals
4. Use the
Results
2. Provide Learning and
Development Opportunities
3. Assess Student
Learning and Development
Suskie, 2009, p.4
Data Collection

What do you want to measure?

What data do you need to collect to see if you
are achieving your learning outcomes?


Qualitative/Quantitative
How can you collect that data?

Examples of tools and data sources:
 Focus Groups
 Surveys/Questionnaires
 Student work: bulletin boards, portfolios, resumes
Definitions






Quantitative – numerical data
Qualitative – words, objects or pictures
Direct – based on actual student work that demonstrates
that learning has taken place
Indirect – based on a report of perceived student learning
Performance based/authentic - systematic observation
and rating that requires a person to demonstrate a specific
skill and/or competencies
Rubric – a guide containing criteria arranged in levels
indicating the degree to which a standard has been met
Examples of Types of Data and Measurement
Quantitative
• Test scores
Direct
Measure
• Rubric scores for writing, oral
presentations, performances
Indirect
Measure
• Counts, such as # hours
spent in activities
• Surveys
Performance • Rubric scores
Based
(Authentic)
Qualitative
• Observations - field work,
internships, service learning
• Student publications or
conference presentations
• Program evaluations
• Questionnaires
• Interviews
• Focus Groups
• Portfolios
• Service projects
• Performances/creations
Example of a Rubric
3
2
1
Preparation
Professional dress. Clear knowledge of
career/job interests. Has well
developed 30 second elevator pitch as
way of introduction. Has professional
business cards, if appropriate.
Questionable dress (such as top shirt
buttons undone, mismatched
tops/bottoms, wearing backpack). General
awareness of career/job interests. Has
basic introduction (beyond name and
major) prepared to share.
Inappropriate dress (such as jeans,
untucked shirts, clothes not ironed,
skirts cut too high). Lack of
knowledge of career/job interests.
No introduction prepared beyond
name and major. Brings parent,
friend, significant other, child along.
Approach
Displays enthusiasm for speaking with
Alumni even while waiting their turn by
standing quietly, engaging in current
conversation occurring when
appropriate, and making eye contact
with Alum if possible. Well developed
approach, including smile, outstretched
hand for handshake, and
implementation of short elevator pitch.
Cautiously approaches Alumni. Does
not integrate self into current
conversation. Handshake not fully
developed, or overzealous. Introduces
self with just name.
Not acting professional while waiting
to speak to Alumni (such as
talking/texting on cell phone).
Hesitation when making approach,
or making self center of discussion
immediately as they join. Weak (or
absence of) handshake.
Interaction
Makes and maintains strong eye
contact, and uses non-verbals to
indicate interest and understanding.
Engages in full interaction with Alumni,
answering and asking questions in
normal flow of conversation. Provides
detailed examples to support answers
to questions. Integrates multiple
questions that show interest and seek
to gain insight into knowledge Alum
has to share.
Makes eye contact, but does not maintain
throughout conversation. May smile and
nod to show interest. Uses filler words
(such as "umm" or "like") while formulating
answers or transitioning between topics.
Exhibits some confidence by adding detail
when answering questions. Has at least
one question to ask of Alumni.
Not maintaining eye contact with
Alumni. Waiting for Alum to ask
questions or guide conversation.
Not fully answering questions posed
to them. Lack of non-verbal
communication. Bad posture (such
as standing with arms crossed).
Lack of emotion/not displaying
interest.
Closing
Asks Alumni for business cards, and
makes clear plan of follow up. Smiles
as they shake Alum’s hand and thank
them for their time.
May or may not ask for business card. No
clear plan of follow up made. Smiles as
they thank Alumni. No parting handshake.
Not thanking Alumni for time. No
clear sign off or parting handshake.
Turning to walk away while still
talking/saying goodbye.
Overall
Highly Competent
Competent
Not Competent
Reflective Conversation
Based on your SLO:

What kind of data/evidence does your
department need to collect?

How will this help you measure your outcome?
Data Collection
Campus Safety Example:
“Students will learn the skills to effectively evaluate situations and
make appropriate judgments regarding their own health and safety,
and the health and safety of others.”


Want to know if students will make a good decision
based on a set of circumstances
Use a rubric to evaluate written descriptions of past
incidences (qualitative, indirect)
Data Collection
Athletics Example:
“Student athletes will exhibit good sportsmanship.”


Want to know number of standard and ethical
violations in a given academic year
Review reports of game statistics
(quantitative, indirect)
Data Collection
Child Care Center Example:
“Student workers will demonstrate the skills necessary to
provide care for children @ PCCCC.”


Want to know how well student workers
demonstrate skills to care for children
Use a rubric to measure performance of student
workers (qualitative, performance based)
Data Collection
Student Activities Example:
“Students will apply effective decision making skills in group
settings.”


Want to know to what degree students are applying
effective decision making skills
During observation, use a rubric to determine
degree of effectiveness (quantitative, direct)
Reflective Conversation
Based on the type of data/evidence you have
identified:

What appropriate tools can you use to collect
your data/evidence?
Connecting to the Strategic Plan

All departments will identify/have identified at least one specific learning
outcome for their areas from the following 6 learning domains:

Knowledge Acquisition, Construction, Integration, and
Application – understanding ideas from multiple disciplines,
linking concepts and experiences, relating knowledge to daily life

Cognitive Complexity – critical and/or reflective thinking,
effective reasoning, creativity

Intrapersonal Development – realistic self-appraisal, selfunderstanding, self-respect, identity development
Connecting to the Strategic Plan

All departments will identify/have identified at least one specific
learning outcome for their areas from the following 6 learning domains:

Interpersonal Competence – initiate, maintain and manage
positive social relationships

Humanitarianism and Civic Engagement – recognition of self
as a member of a larger social fabric; acquiring knowledge, skills,
values and motivation to promote the quality of life in a
community

Practical Competence – ability to translate theoretical
knowledge, skills and abilities to real world applications
Reflective Conversation
Based on the 6 domains in the Divisional Strategic
Plan:

What domain(s) does your SLO connect to?
Questions?
Thank you for
participating!
Resources







Assessment terms and definitions [PDF document]. Retrieved from
http://www.wcupa.edu/tlac/documents/More%20on%20Measures--Definitions.pdf
Council for the Advancement of Standards in Higher Education. (2009). CAS
professional standards for higher education (7th ed.). Washington, DC: Author.
Direct and indirect measurement result [Word document]. Retrieved from
http://www.googlesyndicatedsearch.com/u/SPSU?q=direct+and+indirect&domains
=spsu.edu&sitesearch=spsu.edu&x=0&y=0
Kelly, M. (n.d.). The definition of a rubric. Retrieved from
http://712educators.about.com/od/rubrics/g/rubric_definition.htm
Rubric (academic). (n.d.). Retrieved from Wikipedia:
http://en.wikipedia.org/wiki/Rubric_(academic)
Schuh, J., & Upcraft, L. (2001). Assessment Practice in Student Affairs: An Applications
Manual. San Francisco, CA: Jossey Bass.
Suskie, L. (2009). Assessing student learning: A common sense guide. (2nd ed.). San Francisco,
CA: Jossey Bass.