Assessment
David Kniola, Ph.D.
Assistant Director
Office of Academic Assessment
http://www.youtube.com/watch_popup?v=6Cf7IL_eZ38&vq=medium
Rising tension within and between
Internal and External
Where might we be headed?
Within the next 2-3 years
Continue current practices.
• LEAP essential learning outcomes and authentic assessment.
• New Leadership Alliance (http://www.newleadershipalliance.org/)
Shift from quantitative to a balance with qualitative measures. LEAP
report from employers wanting “context and substance.”
Within the next 2-3 years
Keeping eye on emerging trends:
• Discerning Learning from Lumina
(http://www.luminafoundation.org/publications/)
• Degree Qualifications Profile
(http://www.luminafoundation.org/publications/The_Degree_Qualific
ations_Profile.pdf)
• ETS e-rater, automated essay evaluation
(http://www.ets.org/erater/about)
• Targeted analytics defined by NRC doctoral program survey
Transparency Framework
National Institute for Learning Outcomes Assessment (NILOA)
http://www.learningoutcomeassessment.org/TransparencyFrameworkIntro.htm
Transformation in Assessment
Diana Chapman Walsh who served as president of Wellesley College
from 1993 to 2007. Essay in Inside HE:
•
•
•
•
Toward a Science of Learning
Advances in learning sciences and fast-changing technology
Transcending new and better measures of SLOs
Assessing WHAT students have learned is less valuable than finding
out HOW they learn.
• Polanyi’s "Learning about" involves explicit knowledge, "learning to
be" is more tacit…combine with “socially constructed understanding”
in digital age
Transformation in Assessment
Define higher education and learning in traditional university. Walsh
calls for a “science of improvement.” Creation of “highly intentional
learning”
•
•
•
•
What is quality?
What is “value added?”
Do we need improvement in educational output?
We have seen quality of life improvements (res halls, dining, fitness
center). These have not led to educational improvements.
• What is the role of land-grant?
• What do these mean to VT? How do we invent our future?
• Data, honest conversation, systematic research.
Analytics
Learning Analytics
Rise of learning and business analytics
•
•
•
•
How do we know if a student is struggling?
Do we know what makes a student successful?
What data do we have?
What data do we need?
Learning Analytics
Learning analytics:
• Utilizing performance data (grades, quizzes, tests) captured by LMS.
• Purdue Signals (http://www.itap.purdue.edu/tlt/signals/)
…but what if this data was augmented with data from “outside” the
LMS?
What if we could see patterns of data that has been captured at every
interaction a student has with the university?
What if this could then be used by faculty as well as students to better
understand learning?
Learning Analytics
Identify
Risks
Intervention
Identify
Learning
Awareness
and growth
Assessment
Within the next 5-10 years
• Convergence of data sources (implementation of learning analytics)
• Portable and personalized assessment tools. Student or professor
can deploy bot to retrieve data. Call up on mobile device. Where
does student fit in relation to others?
• Modeling ontologies? Simulations?
• Networked assessment. Move from department/discipline to
university wide. Include PK-12? A social network for assessment?
• Is privacy an issue for the Facebook generation?
Within the next 15 years
Within the next 15 years
Artificial intelligence and augmented intelligence
• Assessment hinges on how we define learning?
• Computers readily retrieve “answers” (e.g., Watson, which will be
commercialized by IBM).
• Human capabilities enhanced by interaction with computer (e.g.,
pattern recognition, devices as extension of our senses,
decisions based on data)
• Asking questions (humans) not answering them (computers)
Within the next 15 years
Where does that leave assessment? Possibility to explore deeper
questions about student learning.
If learning is individualized, assessment will need to be.
Different questions important to different users:
•
•
•
•
Faculty—how/what are students learning in my class?
Advisors—which students need attention and guidance?
Students—what am I learning and how do I compare to others?
Administrators—where do we need to focus resources to support
learning?
• Public—are they doing what they say they are doing?
Within the next 15 years
Next Generation Learning Challenges (http://nextgenlearning.org/)
Goal: Scale the real-time use of learning analytics by students,
instructors, and academic advisors to improve student success.
“That’s our challenge to you: Develop a model that identifies, improves,
and scales existing solutions of learner analytics.”
Within the next 15 years
Think back to the Corning video.
• Imagine this as a university campus.
• Imagine this is what we could do with our data.
• Can we build this at VT?
• An interdisciplinary “meeting of the minds”: instructional design,
learning technologies, brain science, education, psychology,
computer science, systems engineering, OAA, CIDER, others…