Debating the next BIG thing
in teaching statistics
Allan Rossman, Beth Chance
Cal Poly – San Luis Obispo
Overview

Goals

Stimulate thought and discussion




Five propositions as to what the next BIG thing is
About undergraduate, introductory statistics
Set stage for breakout sessions, other plenaries
Inspiration

“Nothing tunes the neurons like disagreement.”
-- David Moore
Overview (cont.)

Disclaimers:




We’re not experts on any of these topics
We don’t have sufficient time to do justice to any
of these propositions
We’ll give some unsubstantiated opinions
We don’t even necessarily agree with some of the
positions we’ll espouse
THE NEXT BIG THING IN
TEACHING STATISTICS WILL BE
Removing the letters z and t from
introductory courses
Elimination of letters z and t

Not literally! We can’t advertise our discipline
as
S_A_IS_ICS

We mean the elimination of normal-based (zand t-) significance tests and confidence
intervals from the introductory course
Motivation
“Ptolemy’s cosmology was needlessly
complicated, because he put the earth at the
center of his system, instead of putting the
sun at the center. Our curriculum is
needlessly complicated because we put the
normal distribution, as an approximate
sampling distribution for the mean, at the
center of our curriculum, instead of putting
the core logic of inference at the center.”
– George Cobb (TISE, 2007)
Arguments for such a curriculum





Randomization model is simple and easily
grasped
Randomization model ties data collection
process to inference technique to scope of
conclusion
Easily generalizeable to other statistics, other
designs
Takes advantage of modern computing
Truer to Fisher’s vision of inference
Many have taken up Cobb’s challenge

NSF-funded curriculum development projects




Rossman, Chance, Holcomb, Cobb (CSI)
West and Woodard
Gould et al (UCLA)
Garfield, delMas, Zieffler, et al (CATALST)
More have taken up Cobb’s challenge

Full implementations

Tintle et al (Hope College)



Hamrick et al (Rhodes College)



March 2011 JSE article
Textbook project
2011 JSM panel discussion
Lock5 textbook project
Tabor and Franklin, Statistical Reasoning in
Sports
BUT …
BUT … Simple and easily grasped?!?


Our assessment results have been mixed
Many students struggle with reasoning
process even after multiple activities

Pre-requisite knowledge?


Model, distribution, “random,” simulation
Biggest sticking points



Seeing the big picture of why doing this
Realizing/appreciating that simulation assumes null
model to be true
Understanding why look beyond observed result
Granted …

Student performance may improve with full
integration throughout curriculum, complete
materials/textbook
BUT… This has been tried before …

Wardrop, Statistics: Learning in the Presence
of Variation (1994)




Simulation based
Early exposure to inference
Normal based methods don’t appear until last 1/3
This approach did not catch on


Ahead of its time?
Not viable for publishers?
BUT…

Students still want to learn z- and tprocedures


Many find comfort, familiarity in the (apparent)
exactness of normal probability calculations
Students still need to learn z- and tprocedures


Those procedures still dominate statistical
practice in other fields
And will continue to do so?
Although…

Randomization methods are become more
widely used and accepted not only in
statistics but also in client disciplines

Manly, Randomization, Bootstrap, and Monte
Carlo Methods in Biology, 3rd ed., 2006
More discussion: Randomization
curriculum

Breakout sessions




11am today (panel discussion on implementation)
3pm today (Lock and Lock: bootstrapping and
randomization)
11am tomorrow (Lock, Lock, and Lock:
technology demonstrations)
Technology demo

4:30pm today (West, StatCrunch)
THE NEXT BIG THING IN
TEACHING STATISTICS WILL BE
Students entering introductory
college courses with considerable
knowledge of statistics
Students will know lots of statistics

Common Core State Standards Initiative





State-led effort coordinated by National Governors
Association and Council of Chief State School
Officers, released 6/2/2010
Standards define the knowledge and skills
students should have within their K-12 education
careers
Currently adopted by 42 states
Two assessment consortia (testing in 2014-15)
www.corestandards.org
Common Core – Mathematical
Practice Standards

Foster reasoning and sense-making in
mathematics
Reason abstractly and quantitatively
 Construct viable arguments and critique the
reasoning of others
 Model with mathematics
 Use appropriate tools strategically
[technology]

Common Core – Statistical Concepts

6th grade:



7th grade:


Develop understanding of statistical variability
Summarize and describe distributions
Investigate chance processes and develop,
use, and evaluate probability models
High school:


Using probability to make decisions
Making inferences and justifying conclusions
Can you imagine students who?

Have already mastered





Variability
Distribution
Sampling, Experimentation
Statistical Inference
Have been consistently asked to




Critique
Reason
Model
Use technology
Jerry Moreno’s perfect world

“In 7 years or so, STATS 101 has been
revised so to excite the CC student by:


Beginning the course with several real world
projects/case studies that review/address/
challenge the content and mathematical practice
base of CC statistically literate students;
Continuing the course with topics such as: Normal
theory inference; risk analysis; design of
experiments/clinical trials; anova;….”
-- CAUSE webinar, May 2011
What could we do with such students?




Mean vs. median?
Risk analysis (e.g., Utts, 2010)
Multivariate modeling (e.g., Kaplan, 2009)
Large, complex data sets, data mining (e.g.,
Gould plenary talk)

Bayesian methods, decision theory (e.g.,
Stewart plenary talk)

Computing, visualization tools (e.g., Nolan and
Lang, 2010)

Data dialogues (e.g., Pfannkuch et al, 2010)
Essential (and cool!) skills …
“I keep saying that the sexy job
in the next 10 years will be
statisticians. And I’m not
kidding. Now we really do
have essentially free and
ubiquitous data. So the
complimentary scarce factor
is the ability to understand
that data and extract value
from it. -- Hal Varian, Chief
Economist, Google
BUT …
BUT … Alternative standards





Design and conduct statistical experiment,
interpret and communicate outcomes
Construct and draw inferences from graphs
Understand and apply measures of center,
variability, association
Use curve fitting for predictions
Apply transformations of data
BUT … Alternative standards (cont.)




Understand sampling and recognize its role
in statistical claims
Use simulation to estimate probabilities
Create and interpret discrete probability
distributions
Use properties of normal curve to answer
questions about relevant data
BUT … What’s the point?

These alternative standards are more modest
than Common Core



Perhaps more realistic to attain?
But could still require a fundamental change in
content of introductory college courses
1989 NCTM Curriculum and Evaluation
Standards for School Mathematics

Have we substantially changed content of Stat
101 in past 22 years based on students’ achieving
these standards?
Granted…

Common Core has a lot more political might,
buy-in from important stakeholders

Much higher probability of impact
BUT … Another big concern

Preparing current and future teachers to
implement such a curriculum is a big
challenge


Need considerable professional development for
current teachers
Need to substantially re-think teacher preparation
for prospective teachers
More discussion: Common Core

Breakouts


11am today (Starnes: AP Stats, Nspire CX, and
Common Core)
11am tomorrow (Scheaffer and Franklin: K-16
Common Core)
THE NEXT BIG THING IN
TEACHING STATISTICS WILL BE
The disappearance of print textbooks
Let’s acknowledge

Students don’t read textbooks




See textbooks as a (very expensive!) repository of
homework problems
Perhaps also skim examples hoping to mimic for
homework problems
Students don’t keep textbooks as reference
Today’s students are “digital natives”

Very comfortable looking to internet, Wikipedia as
reference
Example data

Students more highly value instructors’ notes,
instructor-driven decisions

How useful did you find the following learning
aids/materials in helping you understand
statistics? (77-78 responses)
1 = Not helpful, 5 = Most helpful, skip the question if
you did not use the resource consistently
More importantly

Print textbooks aren’t dynamic enough to
support learning
 Can’t evaluate a student response and
provide guiding comments
 Not conducive to allowing students to work
non-linearly

Can’t easily jump around to what they need
Examples can become outdated very quickly
 Can’t adapt to student interests on the fly

Instead?








Integration of hot-off-the-press case studies
Adaptable presentation
Interactive demonstrations
Optional drill and practice
Immediate individualized feedback
Flexibility in timing and presentation
Replayable podcasts
Interactive online surveys
Some examples




ActivStats, CyberStats, SOCR, HyperStat
Carnegie Mellon’s Open Learning Initiative
The Open University (U.K.)
Publisher learning systems

StatsPortal (Exhibitor Test-Drive), WileyPlus, …
BUT …

What technology innovation has had the
greatest impact on education?

Printing press!
BUT …



Books have had huge impact on education
Textbooks maintain firm hold on U.S. higher
education
College faculty members (as a group) are very
resistant to change



Some of these multimedia materials have been around for
a while and have not taken over the world
Even if the use of print textbooks lessens
considerably in the next few years …
Print textbooks are not going away!
Compromise?

What’s needed is access to plethora of
resources for instructor/student to pick and
choose from


Not one (extra large) size (print textbook) fits all
And then



Server-side database maintaining individualized
interactive student texts
Add notes to eBook in class
Submission of work for instructor-embedded feedback
THE NEXT BIG THING IN
TEACHING STATISTICS WILL BE
Online and hybrid courses replacing
face-to-face interactions among
students/students and instructor
No more face-to-face classes

With all of these multimedia materials, why
do we require students to




Sit in (uncomfortable) seats
At the same place at the same time
Often without access to any resources beyond
paper and pencil?
Why not let students work at their own pace,
using technology, when it’s convenient?

Students at Cal Poly typically avoid Friday classes
More interaction?


Some students interact better online,
overcome reluctance to participate in person
On-line office hours, whiteboards


e.g., elluminate
Calibrated-peer-review model
Growing popularity and importance

Class Differences: Online Education in the United
States 2010 (Sloan Consortium)



63% of reporting institutions said online learning was a
critical part of their long term strategy, compared to
59% in 2009
Nearly 30% of U.S. higher education students took at
least one online course in 2009, compared to 20% in
2006, 10% in 2002
Many more institutions reported seeing an increase in
demand for online courses and programs than for
face-to-face.
Economics!


Online courses do not compete for scarce
classroom space
“Across the country, traditional colleges are
struggling, but for-profit schools such as the
University of Phoenix are experiencing
tremendous growth.” Moneywatch (2010)


438,000 students in 2010
Largest private university in U.S.
Comparison of student performance

“On average, students in online learning
conditions performed better than those
receiving face-to-face instruction.”

Evaluation of Evidence-Based Practices in Online
Learning: A Meta-Analysis and Review of Online
Learning Studies, U.S. Department of Education,
September 2010
BUT
BUT 50 years ago …


Another exciting new technological marvel
was predicted to replace face-to-face class
meetings between instructor and students
Frederick Mosteller pioneered the teaching of
statistics via …
TELEVISION!
BUT 50 years ago…
“In the early and mid 1960s, television was the
great technological hope. Here is a quote
from Time magazine: ‘Not only is a taped
professor as informative as a live one, but he
seldom turns sour and never grows weary of
talking.’ There was actually a feeling that
taped teaching by master teachers would
replace live teachers on campus as well as
taking advantage of the reach of broadcast
television.” -- David Moore (1993)
BUT 50 years ago…
“It's very likely that a course taught on television,
because of the careful preparation, will be better
organized lecture by lecture than the usual lecture in
class, but it does have a lack of flexibility…. The idea
that certain materials can be expressed better in a tv
session seemed to me to be right, and still can be
right. I think that the expanded ability to produce
material that has more visual content than anything
we were able to put together adds a lot more interest
to the course.” -- Fred Mosteller (1993)
Granted …


Online courses have great potential for
interactivity that televised courses do not
But in some (many?) online courses the
instructor merely delivers information
passively to students
BUT …

“Social interaction plays a fundamental role in
the process of cognitive development”
(Vygotsky)



Granted, today’s students are very comfortable
with socializing online
But our sense, and our own experience, is that
(synchronous) face-to-face discussions can be
much more efficient and productive than working
(asynchronously) online
Is there something special about face-to-face
social interaction with regard to learning?
Compromise?

Different model for face-to-face classes



Students complete background reading/ podcast
with guided questions, drill and practice prior to
attending class (literacy)
Class time is spent working examples, presenting
solutions, asking questions (of other students and
instructor), teamwork, peer instruction
Examples


“Inverted Classroom” (e.g., Mazur; Lage, Platt, Treglia)
“Statistical Reasoning Learning Environment” (e.g.,
Garfield & Ben-Zvi, 2008)
More Discussion: Online Teaching

Breakouts


11am today (Fairborn and Zeitler: Transition to
Online Teaching)
11am tomorrow (Everson and Miller: Social
Media)
THE NEXT BIG THING IN
TEACHING STATISTICS WILL BE
Curriculum and pedagogy decisions
will be grounded in educational
research
Statistics Education Research

May still be in its infancy as a discipline


But has enjoyed a tremendous growth spurt!
Journal of Statistics Education




Founded at N.C. State in 1993
Nearing its 20th anniversary
Publishing high-quality, rigorously refereed
scholarship
Including more and more research articles
More statistics education research

Statistics Education Research Journal



Nearing its 10th anniversary (launched 2002)
Publishing exclusively research articles in statistics
education
Ph.D. Dissertations

IASE website lists 70 Ph.D. dissertations in statistics
education since 2000




Including many from researchers here today
Probably many more not listed there
U of Minnesota Ph.D. program in Statistics Education (8
students in fall)
Ph.D. program to be developed at U of Georgia
More statistics education research

Models of Qualitative and Quantitative
methods




Using statistics effectively in mathematics
education research (ASA, 2007)
SRTL Research forums
SERJ special issue (Nov, 2010)
Second Handbook of Research on Mathematics
Teaching and Learning (Lester, 2007)
More statistics education research

CAUSE

Research Advisory Board


Led by Joan Garfield since inception of CAUSE
Research Clusters



2007-09: 3 clusters with 11 participants
2009-11: 3 clusters with 12 participants
Grant proposals, journal articles and presentations at
national and international conferences
Connecting Research to Practice?

JSE has a new feature titled “From Research
to Practice”

Garfield and Ben-Zvi, Developing Students’
Statistical Reasoning: Connecting Research
to Practice, Springer, 2008.
Example – The Statistics Pathway
(Carnegie Foundation, Dana Center)


Development of one-year curriculum in
statistics, data analysis and quantitative
reasoning for developmental math students
equivalent to one-semester college course
Collaboration of representatives of several
professional organizations, statistics
educators (2 and 4 year), developmental
mathematics educators (2 year), researchers,
and designers, access to policy makers

Design of Statway curriculum, materials,
teaching routines is evidence-driven
Based on hypotheses grounded in ed, math ed
and stat ed research, practitioner experience
 Hypotheses tested and refined as Statway is
implemented by community college faculty
 Revisions guided by evidence of student
learning, experiences of faculty implementers

Eliciting diverse sources of expertise
 Building on open source materials

BUT…


Statistics education research can provide sound
principles, but think about how many decisions
instructors make on a daily basis
Example: Statistical significance for 2×2 tables
Learning Goals:





Understand concept
Apply relevant procedure to real data
Interpret results
Draw appropriate scope of conclusions
Explain impact of various factors such as group sizes
BUT I have to decide…

Which method to present first? Which to
present at all?



Simulate randomization test, Fisher’s exact
test, Two-proportion z-test, Chi-square test
Describe method first, or try to ask questions
to lead students to suggest method?
Present example through lecture, or guided
activity, or on-their-own activity or …?
OK, simulation.
So now have to decide:

Start with tactile simulation or technology?



Choice of dataset


Which technology to use?
Should students design own simulations or press
buttons?
Real or realistic? Randomized experiment or
independent random samples or neither? Significant
difference or non-significant?
Choice of test statistic

Difference in success proportions or number of
successes in group A or relative risk or odds ratio or …?
Still more decisions


How many examples to present? With what
characteristics?
How to assess student learning to guide
learning?

Group quiz, individual quiz, homework
assignment, mini-project, multiple choice
questions, …?
BUT …

Not many research studies in statistics
education compare several options and try to
identify the most effective


With sufficient replication for results to be
generalizable
Not feasible to ask for research studies in
such a young field to address all of these
small decisions

Decisions instructors make every single day
BUT … Another big hurdle

College faculty members as a group are very
resistant to change


College faculty members as a group do not
like to be told what to do


Yes, we’ve said this before
Even when that advice is based on rigorous
educational research
College faculty members are often skeptical
of education research

Especially qualitative research
Compromise?

Research can continue to establish general
principles


Instructors can be trained to use their
judgment on how to apply them in their
particular setting


For example, active is better than passive learning
And given the freedom to do so
Develop and support more teacher-scholars
in statistics education
More Discussion: Statistics Education
Research



Statway: Kristen Bishop, Dana Center
Plenary: Bob delMas
Breakouts:



11am today (Zieffler & Mvududu: Qualitative
methods)
3pm today (Lovett: Qualitative data)
11am tomorrow (Hilton and Enders: Conceptual
framework)
Let’s Review

Eliminating z and t has potential


Future students will know more statistics
before college


But not a magic bullet
So we need to get prepared
Textbooks aren’t going away

But instructors need better access to plethora of
open-source, collaborative resources
Let’s Review

Online learning, multimedia resources will
continue to gain in popularity & accessibility


Research can lead to more effective
curriculum and pedagogy


Opportunity to change classroom experience
Needs to be closely tied to teaching practice
For more debate

Breakout 11am today (Peck)
So…





Focus more logic of inference
Students will come in knowing statistics
Textbooks need to change
Have more interactive class sessions
Learn from the research
Many of these ideas are not so new…
Why BIG now and not before?




Improved technology and understanding of
how to use technology for good
More availability and appreciation of data
Students are changing
Better understanding of student learning



Including specific to statistics
More buy in, alignment of stars
More insights: Pearl dinner presentation
Take Home Message

Engage students

Persist in face of resistance

Break shackles

Enjoy the conference!
Thanks very much!


arossman@calpoly.edu
bchance@calpoly.edu