Action Project Final Report
Faculty Learning Community for Action Research on
Active and Collaborative Learning
October 2015
Submitted by Rich James
Project Summary: What, Why and How
This project supported a faculty learning community in the study, practice, and evaluation of Active and
Collaborative Learning (ACL) instructional methods. Many years of research literature has shown a
strong correlation between ACL and improved student outcomes (learning, retention, persistence and
graduation) with a greater positive effect on students who, historically, have lower achievement
outcomes (eg. African-American and first generation). Student responses to our CCSSE surveys from
2010 and 2013 showed we lagged behind cohort schools in the category of Active and Collaborative
Learning more than any other benchmark. There has also been a growing conversation within higher
education about the need to bring the findings from research on learning into classroom practice and to
foster the Scholarship of Teaching and Learning (SoTL) to assess and improve instruction. These
opportunities lead us to investigate, apply and measure the practice of ACL in the community college
setting.
We did so by forming a faculty learning community to engage in action research on ACL. The community
was comprised of nine faculty in eight disciplines: English, Biology, Respiratory Care, Developmental
Math, History, Psychology, Humanities, Sociology (see Table 1). Faculty spent one semester studying
literature on ACL and designing a course ACL intervention intended to improve a targeted learning
outcome. The second semester was spent applying the method, adjusting as necessary, and supporting
learning community peers in reflecting on and improving their intervention. The final semester was
spent assessing the effect of their interventions and developing recommendations for faculty practice
and institutional processes.
The three major objectives for the work of the learning community were:



To gain a deeper understanding of how, when and why ACL is it effective.
Determine if and how ACL can be widely adopted practice and the key considerations,
institutional and cultural, effecting adoption.
Identity the benefits and challenges of faculty action research and SoTL and the key
considerations, institutional and cultural, effecting adoption.
Measures
In our original proposal, we identified several indicators to assess:
1.
2.
3.
4.
5.
Faculty attitude, efficacy and practice
Student learning
Student success outcomes that can be measured during the duration of the project
Adoption of institutional practices and resources to support ACL and action research
CCSSE scores in our next survey cycle
In practice, items 3 and 4 were not possible to assess during the timeframe of the project. Faculty had to
work through implementation challenges and did not have time to collect longitudinal data. We offer
related recommendations under “Lessons Learned”.
Student learning, item 2, is summarized in Table 1. While each project began with the expectation of
improved student learning, many faculty encountered challenges using a new method that often ran
counter to the expectations of students. This experience generated some recommendations that can be
found under “Lessons Learned.”
While we recognized that our nine-person community was unlikely to change our CCSEE benchmark
score (item 5), our project coincided with several other professional development programs on ACL that
collectively, could affect our score. The 2015 CCSSE benchmark for ACL, however, continued to show our
college falling behind the cohort by a significant margin.
2010
46.5
2013
44.8
2015
42.5
This continued decline indicates that the factors behind the scores are persistent and structural aspects
of the student experience at Columbus State and it will take more than episodic professional
development to change our scores. See “Lessons Learned” for some recommended actions.
We were able to complete item 1, “attitude and performance of faculty.” Learning community faculty
competed pre- and post surveys assessing confidence in implementing key ACL competencies. Selfreported data from faculty showed their increased frequency and confidence in using ACL activities. In
response to the statement “Overall, how do you rate your confidence in applying ACL activities in your
classes?” the group average increased from 2.25 (on a five point confidence scale) to 4.11. Also
improving by a whole scale interval are proficiency in “Mapping course content to ACL activities” (2.88
to 3.88), student frequency of “Discuss[ing] lecture material with each other in class” (2.38 to 3.44) and
“[students] practice professional oral communication in class” (1.75 to 2.75). This last aligns with the
CCSSE survey item measuring frequency of classroom presentation, one of our lowest items in the entire
CCSSE survey.
“I've adjusted my classroom around ACL activities and can already see a difference in
the way my students approach class. Even outside of my project, I'm trying the
activities and finding them helpful in lesson planning and concept introduction. It
makes class more fun.” – Faculty Learning Community Participant
Confidence dropped slightly on two items related to the scholarship of teaching and learning (SoTL):
“Studying and evaluating academic research on teaching and learning?” (4 to 3.67) and “applying
findings from research on teaching and learning?” (3.88 to 3.78). This mirrors group conversations about
how increased exposure to the large body of educational research is challenging to assess and assimilate
and can lead one to recognize “how much we don’t know.” Nevertheless, the group favors continued
engagement with SoTL (see recommendation #9).
“I understand the research at a broad-view and applied level, and the ACL faculty
group has been very effective at helping me understand challenges to preparation
and implementation.” – Faculty Learning Community Participant
See Table 2 for a summary of faculty survey responses.
Lessons Learned
ACL benefits
One of our objectives was to confirm, through our own study, the value of the CCSSE emphasis on ACL
as a significant factor in student engagement. CCSSE does not offer a concise definition ACL but
describes a collection of student behaviors that are indicative (see Recommendation #8). Academic
literature is also inconsistent in the terminology and definitions for ACL. For understanding our own
work we settled on the following definitions:



Active Learning: Students are cognitively active by explaining, applying, comparing, or
generating ideas relative to the learning goal.
Collaborative learning: Cognitively active while working with others to produce a “product”
Cooperative Learning: Cognitively active while in interdependent roles to “accomplish shared
learning goals and maximize their own and their groupmates’ achievement” (Johnson and
Johnson, 2006)
Pedagogies that utilize the above are effective in engaging students cognitively and emotionally and lead
to improved: learning, social skills, meta-cognition, self-management, self-efficacy and interdependence.
Additionally, many learning community faculty noted that student satisfaction and enjoyment improved.
They also noted that ACL methods better simulate workplace, family and community interactions
leading to better student preparation for success in career and life.
Recommendation 1: Because the various pedagogies under the umbrella of Active and
Collaborative Learning have demonstrated positive benefits for student outcomes, the
college should continue to support faculty acquisition of knowledge and skills in ACL
practices.
Faculty Adoption
Among the insights derived from the experiments of learning community participants is that effective
use of ACL is skill and, like any complex skill, requires regular practice, mentoring and feedback to
master. A common pitfall for many was the mistake of “trying to do too much.” ACL activities can range
from quick in-class activities to long-term projects. The correct choice depends on many factors
including the subject and type of learning outcome; faculty strengths and personality; time available to
prepare, refine and assess ACL methods; and the expectations of students, colleagues and institutions.
To support faculty adoption, our learning community has several recommendations for faculty and the
college (see “Institutional Support”). For faculty:
Recommendation 2: Promote principles and examples, not prescribe methods. Teachers must
play to their own strengths when using collaborative learning in their classroom and
therefore focus on taking advantage of the principles that promote learning and express
them through one’s own style.
Recommendation 3: Allow significant time to research, plan, implement, revise and reimplement.
Because ACL activities depend on the presence and engagement of students, well-designed activities
can unravel when students are absent, late or unprepared. Faculty need to be able to adjust on the fly
by modifying the activity or transitioning to a “Plan B.” Until faculty accumulate experience in adjusting
to intervening factors, preparing for the unexpected can demand more time for class preparation. See
Recommendation 6 regarding faculty workload.
Recommendation 4: Faculty should always be transparent about their pedagogical methods
Most all learning community participants reported at least some level of student resistance to ACL
pedagogies. Most students have had only intermittent exposure to ACL throughout their schooling and
likely only in younger grades. Many hold the view that they are paying someone “to teach them” and
their own role is that of passive listener. ACL, however, is built on the scientifically backed premise that
student cognitive effort – through elaboration and application with others -- is a necessary ingredient for
deep and lasting learning. Faculty, therefore, should always explain to students the purpose for learning
activities including ACL. Stress to students that, in addition to subject mastery, ACL develops social and
self-monitoring skills that transfer to career, civic and family contexts. By being transparent we are
helping students learn what constitutes effective learning while they learn subject matter (See
recommendation 5).
Institutional Support
To have a significant impact -- such as moving our scores on CCSSE benchmarks and improving learning
and outcomes for students -- a critical mass of faculty mass must practice ACL pedagogies within an
institutional culture that values and emphasizes ACL. This requires institutional support.
Recommendation 5: Re-shape student expectations about what learning looks like.
To reinforce our recommendation that faculty be transparent about the aims and benefits pedagogical
methods (#4), we must communicate complimentary messages through all of our student support
strategies. Through orientations, COLS, tutoring and other student support services we teach success
skills to students. We need to strive for consistency in highlighting practices that are consistent with the
principles and methods of ACL such as those that comprise the CCSSE benchmarks (See
Recommendation #8). By seeking out and emulating these activities, students become more effective
directors of their own learning.
Recommendation 6: When supporting the adoption of new teaching methods, the college
must adjust faculty workload so faculty can give appropriate time to preparation and
assessment.
Consideration should be given to release time or reallocation of responsibilities so faculty can invest the
appropriate amount of time in developing new lessons and subsequent assessments that better align
with ACL experiences.
Recommendation 7: Invest in and support comprehensive ACL training and skill development.
ACL is a complex skill that requires practice, mentoring and feedback to master. Episodic or once a year
training is not sufficient to achieve broad adoption and deep mastery of ACL pedagogies. The CCSSE
validation study showed that, of all the engagement benchmarks, ACL pedagogies have the strongest
correlation with student success outcomes. Training in ACL, therefore, merits a high level of institutional
commitment like we have given to student pathways, student support and business processes. Some
key steps to support this faculty development can include:




Development of regular home-grown and externally led learning opportunities
Faculty time to participate in training (as leaders and learners)
An easy to use resource directory with model learning activities
A support network and regularly meeting support groups (See recommendation 10)
Recommendation 8: Develop complimentary assessments that identify effective practice of
active, collaborative and cooperative pedagogies and assess immediate and long term
student outcomes.
CCSSE items in the ACL benchmark are consistent with, but do not require application of, the pedagogies
defined under “ACL Benefits.” The benchmark items are:
1.
2.
3.
4.
5.
6.
7.
Worked with classmates outside of class
Made a class presentation
Participated in a community based project
Worked with other students in class
Discussed ideas from class with others (family, co-workers, etc.) outside of class
Tutored or taught other students
Asked questions in class or contributed to class discussions
Developing and administering home-grown surveys is one method of assessing ACL practices. A more
organic approach, however, would be to incorporate ACL indicators into faculty feedback and formative
assessment tools such as classroom observation forms and student evaluations. Additionally, evidence
of ACL pedagogies can be included in evaluative criteria for faculty awards and recognition. This signals
the importance of ACL then captures and recognizes effective practice.
For Further Study: Key considerations for adoption of action research
and SoTL
There are other positive aspects of the learning community experience that merit further exploration.
These can lead to additional institutional support and faculty professional development opportunities.
Recommendation 9: Support practicing the Scholarship of Teaching and Learning (SoTL) and
consistently assessing instructional effectiveness
Learning community participants attempted to assess their instructional practices by using scientific
methods. They compared student outcomes between an experimental group – students experiencing an
ACL practice – and a control group -- students in current or past sections without the ACL activity. Doing
so posed a number of challenges ranging from identifying proper measurements, to unforeseen changes
in enrollment to lack of experience in scientific methods. Also, it is difficult for faculty to be
dispassionate experimenters when faculty want to provide what they believe is the best learning
experience for all students.
Despite these limitations, participants felt the precision and rigor required of a sound experiment led
them to closely re-examine their teaching methods, what they consider indicators of learning, and how
well they assess learning. The group recommends creating other opportunities to support faculty in
studying and applying evidenced-based practices and assessing outcomes whether they be ACL
pedagogies or other methods backed by experimental evidence.
Recommendation 10: Support more faculty learning communities
Learning community participants valued the faculty learning community model for professional
development. With a research focus, rotating roles, and regular support meetings, the members
benefited from peer support and mentoring. Some reported increased motivation and energy when
adopting the role of “teacher-scientist.” It was a consensus opinion that all college faculty have multiple
opportunities during their college career to participate in a learning community focused on pedagogy.
This could occur in conjunction with one or more phases of the promotion process. A model for this, and
for practicing SoTL, is provided by Valencia Community College. Valencia requires instructors seeking
tenure to investigate and test a pedagogical practice. While such deep integration could be a long-term
goal, in the near term the college could convene and support learning communities that can vet
promising practices and recommend appropriate paths to adoption.
Recommendation 11: Design learning spaces conducive to ACL
Learning community participants observed that many college classrooms are not conducive to ACL.
Faculty who want to use ACL sometimes find themselves in this difficult rooms while lecture dominant
classes are in less congested spaces. Long, heavy tables, tables fixed to the wall or floor, converted labs
with tables facing walls and a default arrangement of rows facing front all, by design, discourage ACL.
Room design is not an insurmountable obstacle but it should be acknowledged that physical space
shapes expectations and behavior. We should be deliberate about making our learning spaces more
flexible.
Table 1: Summary of Learning Community Projects
Project Summary
# Students
in study
Impact of intervention on
learning
Continuing changes for faculty
Criminal Justice 1025
(Web) Use of
cooperative “Jigsaw”
learning activity to
improve student
understating and
interpretation of crime
statistics. Section with
intervention compared
to section where
students did assignment
alone.
20
The intervention group scored 2
percentage points higher than
control group on assessment.
Students provided unprompted
positive feedback about the
activity on end of term surveys
Self-Regulated
Learning (Human
Growth and
Development Across
the Life-Span Psy2340)
Use of a self-regulated
learning (SRL) classroom
with a feedback look
and collaborative
engagement both
online and in the
classroom. A SRL class
was compared to a nonSRL class in Blended
format.
27
History 2224 (Af-Am
History since 1877) –
Individual and group
active learning modules
using primary source
material from the web;
student presentations
and projects replaced
instructor lecture.
ENGL 1100 –
Collaborative reading
project. Students were
asked to compile a joint
reading of several
assignments, looking
most at annotation
methods, and analyze
why each of them
annotated things
differently or in a
similar manner. I was
looking for an
improvement on
homework scores.
10
Both groups had similar selfesteem numbers at the start of
the project and similar grades at
the end of the project. The
metacognition scores of SRL
group in increased (from 4.7 to
5.4) but this was within the
standard deviation. The SRL
group was able to lower their
test anxiety scores (from 4.7 to
3.3), also within the standard
deviation. Qualitative reports
indicate students enjoyed the
self-regulated learning style after
initial concerns about the nontraditional introduction of the
course.
2 students who took the
traditional History 2224 class
served as observers in the active
learning class; individual
projects showed improvement in
student engagement and grades.
Group project did have the same
success
Exploring further jigsaw research, with a
specific expansion of the variables to include
student feedback, with an emphasis on
efficacy and student confidence. Providing
test questions that apply the research from
the jigsaw could present additional
validation. Future research could include
expanding the data longitudinally. Currently
I am not using a jigsaw due to the transition
of CJ 1025 to SOC 2410. This was recently
developed into web format, but I will add a
similar jigsaw project into my spring 2016
web courses.
Results of the study indicate that students do
respond well to a self-regulated classroom;
however, proper introduction and support is
needed for the students with instructor
interaction and guidance needed throughout
the semester. In Spring 2016, I will attempt
to revise and do the Self-Regulated
classroom again but with more specific
instructions related to how to begin the
project, and increased feedback in between
class sessions via Blackboard.
25
This intervention was not
successful because of logistic
issues. Students were not
prepared enough for class to
make the project work, and the
software I’d hoped to use
depended on all students having
access, which was impossible.
Implement individual active learning
projects in future classes, reconfigure group
project
I am re-examining ways to tie more points to
the activity (make it higher stakes) and to
logistically do it without software but also
without taking up as much class time as it
did.
BIO 2232 (Traditional)
Use of cooperative
activity to improve
understating on key
concepts in neural
physiology. Lecture was
designed in light of
“Primacy-Recency
Effect” whereby
students retain the
material learned at the
start and end of lecture
with not much retained
from the “down time” in
between. Cooperative
learning was employed
during the “down-time”
to encourage active
processing of learning.
DEV 0115
(Traditional)
Assessed ACL activities
and their Impact on
success and retention in
developmental
mathematics courses.
24 students
in each
term:
AU 14 and
SP
Intervention group scored higher
consistently throughout all four
lecture exams and specifically
on 6 neural physiology
=objectives instructed using
cooperative learning in SP 15
compared to AU 14. Students
fostered positive learning
experiences during laboratory
and lecture activities and
reported greater overall
confidence with the course
content. Additionally, students
scored more consistently and
with less deviation in overall
grade for every exam.
Strategy continuing to be used to teach
neural physiology and added for additional
concepts in endocrine and renal physiology.
22
The course in our department has changed,
so I need to reevaluate how to use some of
the same activities in the new course. One of
the challenges with the new course is the
wide range of student abilities.
ENGL 1100/1101 –
Visual text analysis
“rehearsal essay”.
Students watched two
videos with the common
theme of portrayal of
the military,
individually answered
questions about the
videos, then worked in
small groups to write
collaborative analyses
of the videos. The
assignment mirrors the
structure and purpose of
the major writing
assignment that students
struggle with the most
in this basic
composition class.
History 1111
This study measured
whether or not historical
reasoning is enhanced
when knowledge is coconstructed by assigning
students to collaborate
on writing evaluative
20
Informally, class grades
appeared better than most
classes with all students who
stayed passing. Retention was
better than previous classes
(only lost one student after the
first two weeks). Results of
ATMI (Attitudes Towards
Mathematics Inventory) show
overall student self-esteem
increased with the use of group
work.
I created this assignment before
using it in this research project,
but for the ACL research group,
I refined some aspects including
a more deliberate plan for
organizing students into groups,
assignments about the article
used as a critical framework for
the analysis, and more focused
questions on the “rehearsal essay
form”. For 1100 traditional and
blended courses, there were
more passing final grades for the
assignment (C and above) for
Spring 2015 as compared with
previous semesters.
The experimental group
exceeded the score of the control
group by a 5% margin. I did not
measure differences in each
kinds of statement, only the
total.
The initial run suggests that the study should
be repeated with tighter controls and
repeated to establish reliability.
Approxim
ately 115
based om
initial
enrollment
, 69
completed
Continued study comparing SP semesters
and AU semesters noting statistically
significant differences.
I have also decided that I prefer to use active
lecturing along with some group activities
rather than asking students to “self-teach.”
Continue to refine questions and provide
model answers for the individual worksheet
to elicit more specific examples from the
visual text to support their analysis.
essays. Students in the
control class wrote a
brief evaluative essay
addressing the issue
while students in the
experimental class cocreated the essay with
their partner. The final
exam essay measured
the number of
statements in which
students used of
historical reasoning
statements as an
indicator of their
disposition to use
historical reasoning
strategies.
RESP 1360 –
Assessed ACL activities
to improve clinical skill
competency evaluations
all course
work.
More striking though was that
retention in the experimental
group, which was not part of
what was being tested, was
much stronger. 60% in the
control group compared to 83%
in the experimental group.
16
Out of the 5 competencies
compared between two cohorts,
only 1 competency resulted in
significant improvement with the
use of ACL activities. However
there were too many
inconsistencies between the two
cohorts to determine if the ACL
activities were effective in
improving competencies. The
current students who participated
in ACL activities had fewer
competencies to complete during
the semester than the ones that
did not participate in ACL
activities. The two cohorts had
different teachers for the lecture
component of the course and the
two cohorts also had different
entrance criteria to enter the
program. All technical courses
are only offered once per year
which limits the ability to make
a definitive determination of
whether ACL activities made an
impact or not within a year
study.
The next time this course is offered – SP 16,
students will have the same criteria as the
group who participated in ACL activities last
year. I will continue to use the ACL
activities and then compare the two groups.
There will be more similarity between the
two groups to make better comparisons and
conclusions.
Table 2. Learning Community Self-Assessment
Please rate YOUR proficiency in doing the following items using this rating scale:
1 Poor -- 2 Fair -- 3 Good -- 4 Excellent
Designing ACL activities
Assigning students to roles in ACL activities
Pre
Post
3
3.5
2.88
3.5
2.25
3.44
2.5
3.57
2.88
3.88
2.63
3.63
2.2
2.67
3
3.56
1.75
2.75
2
3
2.38
3.44
1.88
2.11
2.5
2.67
1.63
2.33
2.25
4.11
2.75
3.34
3.25
3.34
4
3.67
3.88
3.78
3.25
3.67
Building individual accountability into ACL activities
Evaluating learning in ACL activities
Mapping course content to ACL activities
Coaching students in ACL activities
Please rate the frequency in which your STUDENTS do the following in your
class: 1 Never -- 2 Sometimes -- 3 Often -- 4 Very Often
Discuss their study habits and academic progress in class.
Practice teamwork and collaboration in class
Practice professional oral communication in class
Work on projects where they are accountable to each other for successful
completion
Discuss lecture material with each other in class
Work with each other on assignments outside of class
Engage in peer instruction or in-class “tutoring.”
Make a class presentation.
Overall, how do you rate your confidence in applying ACL activities in your
classes? (From 1 [None] to 5 [complete])
Please rate the frequency in which YOU do the following in your class:
1 Never -- 2 Sometimes -- 3 Often -- 4 Very Often
How often do YOU: Apply a finding from learning research to your teaching
Evaluate the effectiveness of your instructional methods?
How confident do you feel in studying and evaluating academic research on
teaching and learning?
How confident do you feel in applying findings from research on teaching and
learning?
How confident are you in evaluating the impact of your instructional methods?