Reducing digestible tidbits
from meaty stock:
Satisfying varied tastes with an attractive
instructional assessment menu
Erika Bennett, Jennie Simning, & Kim Staley
Capella University Library
LOEX 2011
How are you turning your data into
persuasion?
1) The Hoarder
How are you turning your data into
persuasion?
2) The Philosopher
Meanwhile we have hungry people
waiting...
How are you turning your data into
persuasion?
3) Data informs your Actions
• You pose questions that data can answer
• You gather
• You analyze statistically
• You propose actions
• You carry out and test the actions
• Everybody knows what important measures
you collect and why.
So this session is for you if…
• You collect streams of data that never see the
light of day.
• There is data you wish you had quickly at
hand.
• You struggle reconciling decision-makers’
demands with instructional best practices.
Objectives
1. Prepping the Menu: Who is our audience and
what are trying to give them?
2. Equipment: Why scorecards and dashboards?
3. From the Pantry: What do we have already?
4. Preparing the meal: Gathering measures &
metrics and creating scorecards and dashboards.
5. Plating and Garnish: Using technology for
attractive dashboard displays & infographics.
About Us
• Capella is a
regionally accredited
online university
• 39,900+ students
• Avg. age is 39
• Instruction Librarian
Team
• In-house library,
2007
Education
Business
Prepping Our Menu
The Big Trends
• In IL assessment
– Moving toward larger scale value assessments
• In business assessment
– Balanced assessments of business performance
• Significant parallels!
Similar development cycles
IL Assessment (ILIAC)
1.
2.
3.
4.
5.
6.
7.
Review learning goals
Identify learning outcomes
Create learning activities
Enact learning activities
Gather data
Interpret & analyze data
Communicate data, enact
decisions, & seek
improvement
Balanced Scorecards
1. Identify perspectives & strategic
themes
2. Identify objectives & causal links
3. Create metrics & measures
4. Develop targets & goals
5. Gather data
6. Interpret & analyze data
7. Communicate data, enact
decisions, & seek improvement
Equipment
Assessment Tools
• Scorecards
• Dashboards
• Often confused…
Scorecards
What are Scorecards?
- Tool for managing performance
- Report card on the organization’s
performance.
- Key objectives
- Measures & Metrics
- Goals or targets
Scorecard example
From Dagan, 2007
Scorecard example
http://www.metricus.jp/index.php/itil/metricus-modules/modules-for-itilbest-practices/request-fulfillment-management.html
Scorecard example
http://tandyjacque.narod.ru/balanced-scorecard-example.html
Dashboards
What are Dashboards?
- Tool for monitoring
performance
- Summarize scorecard data
- Snap-shot view of most
important metrics.
- Often graphical, like gauges on
the dashboard of your car.
Dashboard example
Dashboard example
http://www.spiderstrategies.com/blog/index.php/2009/10/stunning-dashboards-from-scott/
Beyond the Stereotype
• Proactive tools
• Flexible tools
From the Pantry
Shared Responsibility
Narrow “Old world” Library Instruction
1.
2.
3.
4.
5.
6.
7.
8.
9.
Focusing a Topic
Sources: Background
Searching the Catalog
Search Terms
Searching Periodicals
ID-ing Reference Sources
Citation & References
Library Classification
Etc.
Shared Responsibility
Contemporary IL Instruction
1.
2.
3.
4.
5.
6.
7.
Determine Info Need
Research Methods
Scholarly Communications
Information Technology
Evaluating Info Sources
ID-ing Reference Sources
Economic, political, legal and
cultural contexts
8. Evaluating info content
9. Etc.
Our Data Network
Strategy Alignment
Strategy Map
A clear illustration of
long-term strategy
Strategy Map
A clear illustration of
long-term strategy
Strategy Mapping
http://www.moe.edu.bn/image/image_gallery?img_id=31478
Strategy Mapping
Classic Perspectives:
1. Customer
2. Financial
3. Internal Processes
4. Learning & Growth
Strategy Mapping
Our Perspectives:
1.
2.
3.
4.
Learner
Financial
Internal Processes
Learning & Growth
Strategy Mapping
Our Perspectives:
1.
2.
3.
4.
Learner
National Standards
Internal Processes
Learning & Growth
Strategy Mapping
Our Perspectives:
1.
2.
3.
4.
Learner
National Standards
Internal Processes
Learning & Growth
Our Strategy Map
Our Perspectives:
1.
2.
3.
4.
Learner
National Standards
Internal Stakeholders
Learning & Growth
Our Strategy Map
Perspectives
•
•
•
•
Learner
National
Internal
Growth
Our Strategy Map
Strategic Themes
•
•
•
•
Learner
National
Internal
Growth
Strategic Themes
• Memorable and motivational names
of 3-5 words.
• Defensible with good execution.
Our Strategy Map
Lifelong Learning Program Integration
Great Teaching
Assessment as Learning
•
•
•
•
Learner
National
Internal
Growth
Perspectives
Strategic Themes
Initiatives
WE
Preparing the Meal:
Scorecards & Metrics
Instruction Strategic Scorecard
• Key Performance Indicators (KPIs)
• Targets and Benchmarks
• SWAG approach
Academic Scorecard Example
From: http://www.osu.edu/academicplan/2006scorecard.pdf
Metric Review Questions
• What action will I take if I see significant change
in this metric?
flatlines and spikes
• Will that action control the process in the way
needed?
• Will measuring this metric change the behavior
and action (learning) as I want?
Raw Data
Dealing with the Raw Data
Our Solution:
• Operational
Scorecards with
Dashboard
components
Ramekins of Data
• Built a relevant
2011 sheet for
each category/
venue:
– Webinars
– Self-directed
Tutorials
– LibGuides
– First Course
Experience
– Collaboratives
– Residencies
Plating &
Garnish
Plating & Garnish
• Executive
Dashboards
– Revisit
definition/tips/sam
ples?
Dashboard example
http://www.spiderstrategies.com/blog/index.php/2009/10/stunning-dashboards-from-scott/
Analytical Add-ons
• Bissant SparkMaker
• BonaVista Microcharts
Add-ons
• Columns
• Minipies
• Bullet Graphs
http://www.vertex42.com/ExcelArticles/sparklines.html
Add-ons
• Sparklines
• Scatter Plots
• Whiskers
What if, for YOU…
EXCEL
EXCELLENCE
Plating & Garnish
• New Technologies
– Cloud/Freeware options
• Bittle: Reporting and dashboards
• Style Scope free edition: Dashboards
• Active Dashboard: Dashboards
– Infographics for specific situations
• Creately: Diagrams and collaboration
• Stat Planet: Interactive maps and graphs
• Tufte’s VUE: Visual Understanding Environment
http://blog.socrato.com/wpcontent/uploads/2011/04/staggering_education_infographic1.jpg
Wrap Up/Q&A
Bibliography from Handout
Balanced Scorecards for Libraries & Higher Education:
•
Matthews, J. R. (2008). Scorecards for results: A guide for developing a library
balanced scorecard. Westport, Conn: Libraries Unlimited.
•
Lyddon, J., & McComb, B. (2008). Strategic reporting tool: Balanced scorecards in
higher education. Journal of Applied Research in the Community College, 15(2),
163. Retrieved from http://www.ncccrp.org/
•
Bielavitz, T. (2010). The balanced scorecard: A systemic model for evaluation and
assessment of learning outcomes? Evidence Based Library & Information Practice,
5(2), 35-46. Retrieved from http://ejournals.library.ualberta.ca/index.php/EBLIP
•
Lyddon, J., & McComb, B. (2008). Strategic reporting tool: Balanced scorecards in
higher education. Journal of Applied Research in the Community College, 15(2),
163. Retrieved from http://www.ncccrp.org/
Bibliography from Handout
Scorecards and Dashboards:
•
Person, R. (2009). Balanced scorecards and operational dashboards with Microsoft
Excel. Indianapolis, IN: Wiley.
•
Dagan, B. (2007). Dashboards and scorecards aid in performance management and
monitoring. Natural Gas & Electricity, 24(2), 23-27. Retrieved from
http://www.wiley.com/WileyCDA/WileyTitle/productCd-GAS.html
•
Hursman, A. (2010). Diagnosis: Dashboredom. Information Management, 20(6),
14-17. Retrieved from http://www.sourcemedia.com
•
Schiff, C. (2008). Three things you should know about dashboards. DM Review,
18(6), 29-29. Retrieved from http://www.information-management.com
•
Butler, L. M. (2007). Warning lights. Connection: The Journal of the New England
Board of Higher Education, 21(5), 31. Retrieved from http://www.nebhe.org/
Bibliography from Handout
Information Literacy Assessment:
•
Gilchrist, D., & Zald, A. (2008). Instruction & program design through assessment. In C. N. Cox, & E. B. Lindsay
(Eds.), Information Literacy Instruction Handbook (pp. 164-192). Chicago: Association of College and Research
Libraries.
•
Oakleaf, M. (2009). The information literacy instruction assessment cycle: A guide for increasing student learning
and improving librarian instructional skills. Journal of Documentation, 65(4), 539-560.
doi:10.1108/00220410910970249
•
Oakleaf, M. (2011). Are they learning? Are we? Learning outcomes and the academic library. Library Quarterly,
81(1), 61-82. Retrieved from http://www.journals.uchicago.edu/toc/lq/current
•
Oakleaf, M., & Kaske, N. (2009). Guiding questions for assessing information literacy in higher education. Portal:
Libraries and the Academy, 9(2), 273-286. Retrieved from doi:10.1353/pla.0.0046
•
Poll, R., & Payne, P. (2006). Impact measures for libraries and information services. Library Hi Tech, 24(4), 547-562.
doi:10.1108/07378830610715419
•
Oakleaf, M. (2009). Writing Information Literacy Assessment Plans: A guide to best practice. Communications in
Information Literacy, 3(2), 80-89. Retrieved from http://www.comminfolit.org/index.php/cil
•
Orcutt, D. (Ed.). (2010). Library data: Empowering practice and persuasion. Santa Barbara, CA: Libraries Unlimited.
•
Thompson, G. B. (2002). Information literacy accreditation mandates: What they mean for faculty and librarians.
Library Trends, 51(2), 218. Retrieved from http://www.press.jhu.edu/journals/library_trends/
Bibliography from Handout
Free Infographic creation software:
• Infographic software links compiled by Melodie Brewer:
http://melodiebrewer.com/data%20visualization.html
• Creately: http://creately.com/
• Tableau: http://www.tableausoftware.com/public/
• Hohli: http://charts.hohli.com/
• Stat Planet: http://www.sacmeq.org/statplanet/
• Visual Understanding Environment (VUE): http://vue.tufts.edu/
Free statistical software:
• List of links: http://www.freestatistics.info/index.php
• Many Eyes: http://www958.ibm.com/software/data/cognos/manyeyes/
Additional Capella program-focused Pubs & Presentations
• Staley, K., Brothen, E. & Bennett, E. ACRL Virtual – 2011; “When
Nontraditional is the Norm: Shifting the Instruction Paradigm for Adult
Students.” Link 1 & Link 2.
• Bennett, E. & Brothen, E (2010). Citation Analyses as a Prioritization Tool
for Instruction Program Development. Journal of Library Administration.
50(5/6).
• Bennett, E., & Simning, J. (2010). Embedded Librarians and Reference
Traffic: A Quantitative Analysis. Journal of Library Administration. 50(5/6).
• Sollien, J., Pohlman, J. & Waitz, E. ACRL 2009 “Challenges for Distance
Students and Distance Librarians: Taking Advantage of the Online
Environment” Link 2.
• ARLD Day 2009 “Reference Data in Action: Explore the Possibilities.”
• Brothen, E., Berg, S., & Bennett, E. (2009). Maximizing the value of
reference data: A case study. In D. Orcutt (Ed.), Library data: Empowering
practice and persuasion (Chapter 11). Santa Barbara, CA: Libraries
Unlimited.
• Veal, R., & Bennett, E. (2009). The virtual library liaison: A case study at an
online university. Journal of Library Administration, 49(1/2), 161-170.
• Bennett, E. (July 2008). eLearning: Libraries increasingly help steer the
ship. Elsevier’s Library’s Connect Newsletter: v6(3).