Classroom Walls That Talk: Using
Online Course Activity of Successful
Students to Raise Awareness of
Underperforming Peers
University of Maryland, Baltimore County
AACRAO Technology Conference
July 19, 2010

• About UMBC
• Problem—Why did we do this?
• A Solution, Not THE Solution
• • Lessons Learned
• Future Plans
• Effective Practices
• Resources
• Q & A

• Founded in 1966
• “Research extensive university” Carnegie classification
• Fall 2009 Stats
– 12,870 Students
• 9,947 undergrad, 2,923grad
– 730 Faculty
• 480 FT, 250 PT
• Selected Brags
– #1 “Up and Coming National University”
U.S. News America’s Best Colleges, 2010
– 1 st in undergrad chemistry degrees awarded to African Americans
– One of 50 Best Colleges for Women
– 7-time National College Chess
Champions

• Blackboard Learn 9.1
• As of Spring 2010 (began using in SP200)
– 95% of all students
– 75% of all instructors
– 65% f ll (1 645 FA2010)
– 356 Communities
• Includes all student, faculty and staff senates
• Support Staff:
– 2 FTE (Admin & Support)
– 1 Server Admin

“So, is Blackboard making a difference?”
PROBLEM
HOW DO YOU ANSWER?

• Functional
– What is the relationship between Blackboard use and teaching and learning?
– What tools can we give users to shed light on (and improve) their own performance within the system?
• Technical
– How do we query the system without breaking it?
– How do we scale and maintain the process?
• Bb – Core product?
• Community – Building Blocks?
• Other?

A SOLUTION

www.umbc.edu/blackboard/reports

• Show faculty what peers are doing through publicly available reports of student use.
• “Average hits per student” course rankings don’t
• New user tools build on “activity” as an indicator
(not a cause) of student success.
• We are NOT interested in if Blackboard makes good students, but how good students use
Blackboard.

Blackboard
(Static Replica)
Blackboard
(Production)
Queries
(PHP Scripts)
Complete copy of database made daily at 1 a.m. for reports
Cached Reports www.umbc.edu/blackboard/reports

• Code Download
– http://www.umbc.edu/oit/newmedia/blackboard/ stats/getthecode.php
• Video Show & Tell “Walkthrough” (same as above)

LESSONS LEARNED

• Based on voluntary participation by instructors in 131 courses, students earning a D or F tend to use Bb 39 percent less than students earning higher grades .
– SP2010 21 courses | 37 percent less
– FA2009 | l
– SP2009 11 courses | 47 percent less
– FA2008 13 courses | 40 percent less
– SU2008 7 courses | 33 percent less
– SP2008 26 courses | 32 percent less
– FA2007 15 courses | 36 percent less
• Does the pattern hold true during the semester?
• What if students knew this information sooner?

• How would you describe the CMA’s view of your Bb activity compared to your peers?
– 28% “I i d b h t it h d ”
– 12% “It confirmed what I already knew”
– 42% “I’d have to use it more to see”
– 16% “I haven’t used it.”
– 2% did not respond to this question

• If your instructor published a GDR for past assignments, would you be more or less inclined to use the CMA before future
– 54% “More inclined”
– 10% “Less inclined
– 36% “Not sure”

• Quantitative
– Expand the sample of UMBC courses being studied.
– Study the demographic backgrounds of students.
• Qualitative
– Why do UMBC students use the CMA?
– Why do they return?
• Literature Review
– CMS activity as predictor vs. indicator of success.
– How do others use the CMS for intervention?

EFFECTIVE PRACTICES

Examples of other CMS “Data Mining” projects
` 5/30/08, Chronicle of Higher Education
` Argosy University
` Purdue University
` Slippery Rock University of Pennsylvania
` South Texas College
` SUNY Buffalo
` Tiffin University
` University of Alabama
` University of Central Florida
` University System of Georgia
` Blackboard Greenhouse Grant - Project ASTRO
` OSCELOT.org, Advanced System Tracking & Reporting tool
` Hofstra University

“Colleges Mine Data to
Predict Dropouts”
“At the University System of
Georgia , researchers monitored how frequently students viewed discussion posts and content pages on course Web sites for three different courses to find connections between online success. In the graph below, students who were
"successful" received an A, B, or C in the class, and students who were "unsuccessful" received a D, F, or an incomplete.”
- 5/30/08 Chron of Higher Ed .

Educause Center for Applied Research

Educational Technology Framework
Exploratory Supported Strategic Mission Critical Transformative
Transition 4
Transition 2 Transition 3
Phase I
Transition 1
Phase 2
Time
Phase 3 Phase 4 Phase 5

• Campbell, J.P., DeBlois, P.B. & Oblinger, D.G. (2007, July/August) Academic analytics: A new tool for a new era . EDUCAUSE Review, 42 4): pp. 41-57.
Retrieved March 3, 2009 from http://connect.educause.edu/Library/EDUCAUSE+Review/AcademicAnalyt icsANewTool/44594
• Rampell, C. (2008). Colleges Mine Data to Predict Dropouts . The Chronicle of Higher Education , 5/30/08. Retrieved March 6, 2009 from http://chronicle.com/weekly/v54/i38/38a00103.htm#web-course (login required)
• Young, J. (2009). College 2.0: A wired way to rate professors—and connect teachers. The Chronicle of Higher Education , January 8, 2009. Retrieved
April 23, 2009 from http://chronicle.com/free/2009/01/9311n.htm

RESOURCES

"Project ASTRO" Blackboard Greenhouse Grant
Eric Kunnen
Coordinator of Instructional Technologies
Grand Rapids Community College ekunnen@grcc.edu
Santo Nucifora
Manager of Systems Development and Innovation santo.nucifora@senecac.on.ca

• Identify & Detect
– Manual Flags
– Automatic Flags
– Attendance
– Instructor
– Advisor
& T k
More info: http://www.starfishsolutions.com
– Groups of Courses and Students
• Improve & Retain
– Student Communication and 360 Close Loop

STARFISH EXAMPLE
AUTOMATIC FLAGS BASED ON BLACKBOARD
GRADEBOOK/COURSE ACCESS
Administrators can set up flags to be raised that are autogenerated. Flags can be raised by the system by grades and average scores and specific gradebook columns in
Blackboard. Flags can also be raised based on students’ access to their courses in Blackboard. Additional customization is available through API’s.

• Project ASTRO http://projects.oscelot.org/gf/project/astro/
• Starfish Early Alert Project Site http://www.starfishsolutions.com
P d U i it Si l P j t Sit http://www.itap.purdue.edu/tlt/signals/
• UMBC’s Blackboard Reports & CMA http://www.umbc.edu/blackboard/reports

Thanks fritz@umbc.edu
www.umbc.edu/blackboard/reports