Technology Enhanced Learning: Best Practices and Data Sharing in
Higher Education
Example/Case Study Template
(Version 2: December 2015)
The following template should be used by those wishing to submit case studies exemplifying the
recommendations of the Global Learning Council for best practices in technology-enhanced learning (TEL)
and educational data sharing. Information submitted may be collected and posted on the GLC website and
referenced in best practices documents issued by the GLC.
Heading
Description/Questions to be addressed
Investigators
Contact information for investigators including email address of
corresponding author.
Date Submitted
Date of submission
TEL intervention type
Is it a platform, instructional activity or resource, piece of software,
process, other?
Brief description
In a few sentences, what is the goal of the TEL intervention and how is
it intended to be used (e.g., by learners, teachers, designers, etc)
Recommendations
Implemented
Which recommendations from the GLC document are reflected in this
intervention?
Evidence for the design
On what teaching/learning principles, theoretical framework, and/or
empirical results was the intervention's design based? In what way(s)
do you expect the intervention to enhance learning, performance,
engagement, or design, analysis, improvement of TEL? (Please include
references where possible.)
Context(s) for application
In what teaching, learning, or design context is the intervention
intended to be used? As appropriate, please describe common use
cases; target user population, discipline/topic/concept addressed.
Study design and context
How was the intervention evaluated? Describe the context and study
design used. This should include the standard of comparison is used,
when & where the study was completed, how users were
selected/sampled/assigned to participate, and duration of study.
Datasets collected
What data were collected to assess intervention's effectiveness? Please
describe the instruments or measures (and how they were validated).
Ideally, data should include direct measures of learner outcomes; if not
the connection to learner outcomes should be discussed. Include link to
dataset if possible.
Summary of results
Statement of
generalizability
Lessons
learned/considerations for
future
Find out More [Links to
References & Other
relevant Publications]
To what other contexts would this TEL intervention easily apply and
likely show similar results?
EXAMPLE:
UC Davis iAMSTEM Case Study
Heading
Description/Questions to be addressed
Investigators
Marco Molinaro (UC Davis: Assistant Vice Provost for Undergraduate
Education), marco.molinaro@iamstem.ucdavis.edu
Chris Pagliaro (Director of Instruction & Assessment),
chris.pagliarulo@iamstem.ucdavis.edu
Date
16 March 2015
TEL intervention type
A combination of software and courseware, leveraged to support
specific kinds of instructional activities
Brief description
The UC Davis iAMSTEM team’s goal is to improve learner outcomes
in introductory STEM courses and improve STEM retention rates,
especially among vulnerable student populations. To achieve this goal,
they are working to change instructional practices in large (1000
student) lecture sections, shifting practice to more active case-driven
approach that supports higher order outcomes. They are supporting this
change through a combination of technologies -- courseware, clickers,
data capture -- and professional development for educators. This
approach is iteratively approved through analysis of multiple data
sources.
Recommendations
The UC Davis approach reflects many of the recommendations in the
Implemented
GLC Document, providing a combination of support and incentives for
faculty to engage in deliberate instruction, along with the necessary
resources and expertise to systematically implement a number of TEL
interventions to drive changes in practice, capture data and use that data
to support teaching and improved design.
Evidence for the design
The design of the project focuses heavily on Bloom’s 2Sigma work
(Bloom 1984; Burke 1984; Anania 1982, 1983), hypothesising that in
lieu of Bloom’s 1:3 educator to student ratio, active learning (Freeman,
2013) supported by evidence-informed, adaptive teaching practices and
continuous improvement in course design, can provide similar results
in learner achievement. The project design also took advantage of
work in adaptive learning (Lovett, ; Hattie), Process Oriented Guided
Inquiry Learning (POGIL) (Farrell, 1999; Wolfskill, 2000) and the
Classroom Observation Protocol for Undergraduate STEM (COPUS)
(Smith, 2013).
Context(s) for application
As a part of the University of California system, UC Davis serves a
broad array of learners. This diverse population is well represented in
STEM majors, where introductory Biology and Chemistry courses have
traditionally been structured as gatekeeper courses, delivered in large
(1000 student) lecture sections supported by relatively static review
activities in recitation. Up to 42% of science majors in UC Davis come
from vulnerable or disadvantaged learner populations
In what teaching, learning, or design context is the intervention
intended to be used? As appropriate, please describe common use
cases; target user population, discipline/topic/concept addressed.
Study design and context
From Fall 2013-Winter 2014, students were randomly assigned to 500student treatment and 500-student control sections. Control students
engaged in traditional lecture and review-style recitation activities.
Treatment students also participated in traditional lecture, but were
assigned the use of adaptive courseware that provided learning outcome
mastery predictions to trained TAs. TAs used this predictions to
identify common areas of challenge for learners and address these
challenges using active learning and POGIL techniques. Beginning in
Summer 2014, additional training in active instruction was provided to
faculty teaching the large lecture settings. Through the experiment,
classroom activities were observed and captured using COPUS.
Datasets collected
Data collected includes
●
traditional learner demographic and achievement information
(final grade, progress)
●
classroom activities measures using COPUS tool
●
learner interaction data from adaptive courseware
●
final exam
●
pre/post test
Some elements of the dataset are publicly available at: ALMAP fall
2013 and ALMAP spring 2014
Summary of results
This approach has had a significant impact in changing the mix of
classroom activities based on COPUS reviews. Experimental group
shows improvement in exam performance; students in this group are
1.655 times more likely (90%) to pass the course with a C or better.
This effect is most pronounced among the more vulnerable student
populations. Students perception of the new approach was also
positive, with reflecting an understanding and appreciation for the
instructional techniques.
Statement of
This combination technology and practice is likely to be successful in a
generalizability
wide range of classes and for a wide array for learners; it’s specific
application may make the most economic sense in large sections
Lessons
The study provided some core lesson in understanding barriers to
learned/considerations for
adopting these types of approaches. A broad consideration of
future
incentives and motivation on the part of the faculty are essential for
these types of initiatives to succeed. Data sharing, within the institution
and with collaborators, continues to be a challenge; a broader, shared
understanding of data sharing norms and best practices is necessary to
scale this type of work. Although pre-existing open, adaptive
courseware has proven successful with learners, the ability for faculty
to customize this courseware is critical for adoption and long-term
success.
Link to Publications
http://www.nytimes.com/2014/12/27/us/college-science-classes-failurerates-soar-go-back-to-drawing-board.html?smid=fbnytimes&smtyp=cur&bicmp=AD&bicmlukp=WT.mc_id&bicmst=140
9232722000&bicmet=1419773522000&_r=0
https://www.insidehighered.com/blogs/technology-and-learning/mostimportant-higher-ed-story-2015
http://www.educause.edu/annual-conference/2014/adaptive-learningand-quest-improve-undergraduate-education