Learning from errors and
Feedback: Statistics Efficacy
Dissertation Proposal
CUI Department
Simin Hall
Organization




Background
Statement of the
problem
Statement of
purpose
Previous Research





Significance of this
research
Research Design
Questions of the
study
Analysis
Utility/Results
Background






Increase of 62% between 1976 to 2004 in
enrollment in graduate programs
www.NCES.ed.gov,
career development and personal enrichment UNCLong_range_planning,
These graduate students are required to take
statistics & quantitative research courses,
Their background may be far removed from the field
of statistics,
They experience anxiety,
What interventions are needed?
What does this intervention intend to
accomplish?


To reduce anxiety in learning statistics.
Cultivate self-efficacy in graduate
students in using statistics and research
methods.
Environmental Scanning

The population of the study

The nature of the problem
Characteristics of adult learners






problem-centered
results –oriented
self-directed
seek education that relates or applies directly
to their perceived needs
accept responsibility for their own learning, if
learning is perceived timely and appropriate
Time/Distance are issues of concern
Nature of the problem



students may experience anxiety due to
internal factors such as low efficacy
perceptions in statistics or external factors
such as poor instruction.
Instruction fragments theory, statistical
analysis, and empirical reality.
Anxiety due to lack of background in
quantitative reasoning or problem solving
skills.
Statement of Purpose



Explore how students’ learning can be
enhanced through instruction that builds selfefficacy.
Test the social cognitive theoretical prediction
that self efficacy is enhanced by feedback
that fosters problem solving skills.
Examine these predictions using a networkportfolio or a web-based feedback technology
such as a wiki.
Previous Research








Wang and Lin (2006): Social
Cognitive theory
Technology
Liu et al. (2001): Web-based peer
review
Seifert and Hutchins (1992): errors
as opportunity for instruction &
learning Navigation
Mejias (2006):Network Portfolio
system & practical research skills
Teamwork
Network-technology
Lovette et al. (2000): cognitive
theory, statistics instruction,
individual practice, real-time
feedback
Lin et al. (2001):Web-based peer
assessment NetPeas





McKendree (1990): feedback
content
Mayer, R.E. (2002, 1996) Cognitive
theory; Examples & Statistics word
Problems, Models for understanding
Fredrickson et al. (2005) websupported vs lecture-based: course
in statistics & research methodology
for graduate students in education
Larreamendy-Joerns et al.
(2005)Six on-line Statistics courses
Penn State
Features of this
Research/Significance of this Study






Monitoring errors (Recovery Strategy)
Explicit Demonstration of how to provide
feedback
Pattern recognition:Problems,Mathematicalsteps, Soft-ware
Decision tree Map
Use of Published research articles research
Control & treatment groups
Social Cognitive Theory


Social learning theory explains human
behavior in terms of a continuous reciprocal
interaction between cognitive, behavioral and
environmental determinants (Bandura,
1986).
Students are capable to self-regulate, selfreflect, to plan alternative strategies, to learn
by doing, and by modeling.
Social Cognitive Model of
learning
Network Technology




Allows continuous reciprocal
interaction between cognitive,
behavioral and environmental.
Reduces individual cognitive loads
Allows students to monitor and integrate the
information with the help of peers.
interactive technology accommodates
students’ differences in regard to their
information processing rate.
Feedback/Efficacy


Hypothesis: four sources of feedback
impact students’ efficacy positively
through their direct link to
metacognitive strategies of planning,
monitoring and regulating.
Planning, monitoring, and regulating
are steps to problem solving strategies.
Sources of Efficacy
Theoretical Prediction

According to social cognitive model of
learning, students monitor their steps and
errors, they note their progress toward the
solution, and progress indicators convey to
students that they are capable of solving
statistical problems. This reciprocity enhances
self-efficacy for continued learning and
impacts students’ emotion positively (Pintrich
& Degroot, 1990; Pintrich & Schunk, 2002).
Research Design

Quasi-experimental design with two
groups, one control and one treatment
group. The control group will receive all
instructions via Blackboard system
Course, and the treatment group will
use the Web-Based feedback system
(WBF- wiki).
Intervention

(a) The teacher posts the homework assignment. (b)
Each student prepares the homework and uploads it
to the wiki. (c) For each assignment, the teacher
assigns randomly three reviewers for each student.
(d) Each student views other works and must
comment on the steps of how the solution is derived.
(e) Each student must revise the original assignment
in line with the comments received. The final solution
has a brief description from each student to
summarize their initial understanding of the problem
and their final understanding.
What do students achieve
during the feedback process?

In completing the assignments, a student may plan
the steps to the solution, the procedures to be shown
in the solution, and finally execute the plan. In
reviewing peer homework, student must read,
compare, or question ideas, suggest modification, or
even reflect how well student’s work is compared
with others. These cognitive processes involve
monitoring the adequacy of steps adopted. However,
if student receives a message that a step is not
adequate, then the student must regulate the
cognitive function and employ other alternatives.
Control Group

Students receive the same assignment as the
treatment group; however, it is in the assignment
format of Black Board for which students can not
view each other’s works. Moreover, students can not
retrieve and correct errors since they only receive
feedback from the teacher on their final submissions.
Students are encouraged to use the Discussion Board
and emails to interact among themselves and with
the teacher. They are assigned into groups to
facilitate communication and learning. However, there
is no grade associated with these activities.
Question-1


Is there a significant difference in the means of the
two achievement grades across the groups (WFB vs.
BB)? Students receive scores on assignments with
peer feedback and the final is without the peer
feedback. The assignment score will be compared
with the final score using a repeated measure
analysis. The dependent variables are the assignment
and review scores combined and the final score and
the independent variable is the method which is the
web-based feedback system (treatment) and the
Blackboard (control). The covariates are the students’
characteristics. It is hypothesized that students’
scores are higher with peer feedback.
H0: μt = μc ; Ha: μt > μc.
Question-2


Is there a significant difference in the means of
pre/post of efficacy scores across the groups? The
pre and post test scores of students on a
questionnaire that measures students’
confidence/competence with statistics and feedback
system are compared using a repeated measure
analysis. The dependent variables are the pre and
post efficacy scores and the independent variable is
the method with students’ characteristics as the
covariates.
H0: μt= μc, the null hypothesis states that the
difference in efficacy scores is the same for both
groups. Ha: μt> μc
Question-3


Is there a significant difference in the means of
pre/post of anxiety scores across the groups? The pre
and post test scores on statistics anxiety for both
control and treatment groups are analyzed using a
repeated measure analysis with the test scores as
within subject effect (dependent variable) and the
intervention system as the between subject effect
(independent variable) with students’ characteristics
as covariates.
H0: μt= μc, the null hypothesis states that the
difference in anxiety scores is the same for both
groups. The alternative hypothesis is that the
treatment group manage to reduce anxiety more
than the control group. Ha: μt < μc
Question-4

How do students cluster based on the
variables in the study? In cluster analysis,
there is no distinction between dependent
variables and independent variables. A cluster
analysis is performed to determine the
grouping of students that have similar
characteristics in terms of the variables
measured in this study. By looking at the
combination of variables in each cluster, I
hope to define each cluster.
Question-5


Is there a significant difference in the means
of satisfaction between the two groups? A
one-way analysis of variance with satisfaction
scores as dependent variable and the method
as the independent variable with students’
characteristics as covariates is analyzed.
H0: μt= μc, the null hypothesis states that
the mean of satisfaction scores is the same
for both groups and Ha: μt > μc.
Advantages & Disadvantages: web-based
feedback
Advantages
 Continuous interaction

Learning from modeling and
errors

Planning, monitoring,
regulating

nonintimidating
characteristics, such as
anonymity in asking
questions
Disadvantages
 potential losses due to free
riding, social loafing, and
diffusion of responsibility

to lack focus or willingness
to participate or the
discipline for the web-based
environment

some students experienced
greater peer pressures in the
peer assessment

dislike peer assessment
because raters were at the
same time competitors
What benefits/Who would use these
results?

The prevalence of
quantitative methods in
social science literature
(Vijverberg, 1997).




The decisions for doctoral
students to persist may vary
widely, however, the peer
mentoring in the area of
research were identified as a
predictor for retention.
Mastery of quantitative
techniques opens up topics
for thesis and dissertations
and could potentially impact
career choices.
Understanding efficacy beliefs
should be helpful for teaching
courses that traditionally have
produced anxiety in students
This research may provide
empirical evidence in regard to
the effect of sources of
feedback on efficacy in learning
quantitative courses and thus
research. Students’ satisfaction
with this intervention may lead
universities instructors to adopt
the more advanced form of this
technology, which provides
instructors with tools for
assessments (Wang & Lin,
2006).
Recommendation for Future
Research



Researchers and engineers who construct
web-based learning systems tend to overlook
the reciprocal interactions between three
influences in social cognitive theory.
Interactive web-based system provide
scaffolding to learning. Need more empirical
results to improve functions such as
assessment tools.
Design better Psychometric measures of
students’ learning.
Conclusion


In this study, it is hypothesized that monitoring and
learning from errors on the network technology lead to
successful problem solving skills. My main interest is not
just limited to determine whether providing a learning
assisted system promote active knowledge construction
but also the effective learning strategies supported by
the system is of high priority.
Finding the grouping of students who do not adapt well
in the feedback system (are they not effective team
players? What cognitive strategies do they lack?) are
important findings for further studies to examine the
effectiveness of web-based feedback system in different
fields.