“Whenever a thing is done
for the first time,
it releases a little demon.”
Pilot Strategies for Blended Learning
Programs
#aln14
#demonpilot
Emory University ~ L. Roxanne Russell ~ Timothy D.
Harfield
Interactive Session Prompt
On Twitter use
#aln14 #aln7717 & #demonpilot
How have you used traditional data collection
methods or learning analytics to inform course
design and delivery?
What is the relative importance of student
perception and student behavior in informing course
design and delivery?
#aln14
#demonpilot
D Min Pilot Overview
#aln14
#demonpilot
Process
• One D Min faculty member and 10 accepted D Min students
• One typical course week of activities with primary tools:
Blackboard, Adobe Connect, Echo360 & Voicethread
• Data Collection
– Survey
– Individual Interviews
– Focus Group Interviews
– Learning Analytics
• Data analyzed for design-related program implications and
to inform programmatic student support infrastructure,
student orientation, administrative training and faculty
development
#aln14
#demonpilot
Student Perceptions
#aln14
#demonpilot
Student Perceptions
Wanted to work with
the named
Professors
Needed convenience
of online format and
no long residency
Knew if Candler
was doing it, they
would do it well.
#aln14
#demonpilot
Really interested
in the focus of
tracks
Had met and
been
impressed with
high quality
Candler Alums
Student Perceptions
Live and die by my
calendar
Felt so alone
when audio
wasn’t working
in the session
#aln14
#demonpilot
How long are the
synchronous sessions?
Made myself use the
digital version of the
readings
Insight into fulfilling student
expectations
• Caliber of faculty
– Originality of perspective- scholarship
– Personal exposure
– Charisma/personality
• Prestige of institution
– Quality of environment (facilities/interface/tools)
– Quality of service & support
– Caliber of and access to peer network
• Challenge of curriculum
– Relevance and level of content
– Worthwhile course activities
– Scaffolded project
#aln14
#demonpilot
Student Behavior
Access Summary
• Students tend to leave assignments to last day possible
Monday
5-May-14
Course Accesses
Unique Users
Total Item Accesses
Total Minutes in Course
Avg Minutes in Course (per user)
1
1
4
1.57
1.57
Tuesday
6-May-14
6
6
68
622.43
103.74
DAY 1
Wednesday
7-May-14
DAY 2
Thursday
8-May-14
DAY 3
Friday
9-May-14
DAY 4
DAY 5
Saturday
Sunday
10-May-14 11-May-14
DAY 6
Monday
12-May-14
DAY 7
Tuesday
13-May-14
Read Miller-McLemore,
"Introduction"
Read Moschella,
"Ethnography"
Review Materials as needed
Review Materials as needed Review Materials as needed
Create a Blackboard Profile
View Lecture: Cultivating
Practical Theological
Sensibilities
Post your replies to
classmates' DQs
Attend (ordinarily studentled) Synchronous Session:
Colloquy
Post your response on the
Voicethread Discussion
Post Introduction in
"Getting to Know You"
Discussion
Post your Course Roundtable
Discussion Question (DQ)
Response
Begin "Designing a Research
Question" Assignment
Continue working on
"Designing a Research
Question" Assignment
Submit "Designing a
Research Question"
Assignment
Attend faulty-led
Sunchonous Session: What
is a Practical Theologian?
Begin "Designing a Research
Question" Assignment
10
8
93
1139.35
142.42
2
1
6
10.37
10.37
11
7
170
937.45
133.92
#aln14
#demonpilot
1
1
3
2.50
2.50
14
3
58
121.28
40.43
11
9
94
513.90
57.10
7
5
147
514.12
102.82
Wednesday
14-May-14
2
5
514
273.77
54.75
Student Behavior
Access Summary
• Students who access the course the most, are not necessarily the most
active
Accesses
30
25
20
15
10
5
0
Student A
Student B
Student C
Student D
Student E
Student F
Student G
Student H
Student I
Student J
#aln14
#demonpilot
Student Behavior
Discussion Forum
• Standard personal introductions do not foster community, but
decentralized assignments may lead to differences in integration
DMin Pilot
DMin 500
#aln14
#demonpilot
Student Behavior
Discussion Forum
Course Roundtable
 INFLUENCERS
generate more posts from others than
they contribute
 CONNECTORS
may generate relatively few posts, but
serve to integrate others into the
community
#aln14
#demonpilot
Student Behavior
• Opportunity to advise students on effective
strategies for interacting with asynchronous
video content
#aln14
#demonpilot
Insight into doctoral level
program evaluation
• Acknowledge that students are advanced level
professionals- typical approaches to measuring
student success or performance may not be
appropriate
• Recognize the assessment value of student
expectations and perceptions for their learning
experiences in doctoral level graduate programs
• Prompted holistic examination of context,
purpose, and questions for integrating learning
analytics into program evaluation
• http://www.slideshare.net/rockirussell/d-min-programeval
#aln14
#demonpilot
Next time
• Authentic sample useful for gathering
data but also for preparing 1/3 cohort
prior to orientation
• False sense of security from remote
students- technology dependency and
anxiety much more palpable when on
campus
• Refine framework for data collectionsurvey questions, interview questions,
and learning analytics
#aln14
#demonpilot