"Can computers help faculty
teach and students learn?"
Outline
I.
Background
II. Defining the problem
III. Computer Assisted Learning Method
IV. Initial Results
V. Summary and Outlook
Acknowledgments
•Students who used CALM over the past 6 years.
•Faculty in chemistry (both IU and elsewhere) and physics who incorporated CALM into their courses.
•CALM development team: G. Alexander, T. Bredeweg, T.M. Hamilton, D. Felker, J. Anderson, A.
Dobie-Galuska, S. Wietstock
•R. Althauser, Dept. of Sociology, Indiana University
Romualdo T. de Souza, Dept. of Chemistry, IUB
Purdue University Feb. 19th, 2003
http://calm.indiana.edu
"Can computers help faculty
teach and students learn?"
Background
For the first 8 years I was at Indiana University, each Spring I taught the second semester
of our main freshmen chemistry sequence (450-600 students).
How does one assess the understanding of the students on a regular basis?
It is not just the problem of a large class, one has the same problem with a class of 20. Of
course in a small class one claims that by asking the class questions in-class one can
assess their understanding.
What about the students who refuse to answer questions in class?
Assertion –
The moment you sacrifice the one-on-one interaction you have lost
something. That is why students participating in undergraduate research have
such an unique experience.
Romualdo T. de Souza, Dept. of Chemistry, IUB
Purdue University Feb. 19th, 2003
http://calm.indiana.edu
"Can computers help faculty
teach and students learn?"
Problems in teaching a group of any size –
• many of the best students are not sufficiently challenged
• culling the “best” students into an honors section doesn’t solve the problem.
• In-class exams do not provide an environment to fully challenge the capabilities of every
student.
Part of the solution is to increase the accountability of students in
their out of classroom time.

Homework/Exercises with feedback
Romualdo T. de Souza, Dept. of Chemistry, IUB
Purdue University Feb. 19th, 2003
http://calm.indiana.edu
"Can computers help faculty
teach and students learn?"
Third International Mathematics and Science Study (1995)
Mathematics
Fractions and number sense, Algebra, Geometry, Data representation, analysis, and
probability Measurement, Proportionality
Science
Earth science, Life science, Physics, Chemistry, Environmental and resource issues,
Scientific inquiry and the nature of science
The following are the content areas included in the advanced mathematics and
physics assessments.
Advanced Mathematics
Numbers and equations, Calculus, Geometry
Physics
Mechanics, Electricity and magnetism, Heat, Wave phenomena, Modern physics
In the final year of secondary school (twelfth grade in the U.S.), U.S.
performance was among the lowest in both science and mathematics,
including among our most advanced students.
Romualdo T. de Souza, Dept. of Chemistry, IUB
Purdue University Feb. 19th, 2003
http://calm.indiana.edu
"Can computers help faculty
teach and students learn?"
WWW based instruction has increased dramatically in the past 5-7 years.
•
Faculty placing notes, exercises on the Web (static, no interaction).
•
Distribution of faculty notes by commercial enterprises (e.g. versity.com )
•
Interactive tutorial and mastery programs (among the best)
a) CyberProf, Univ. of Illinois ; physics and chemistry
b) CAPA, Michigan State University; physics and chemistry
c) OWL, University of Massachusetts, chemistry
• Initiatives for systemic change in chemistry (NSF funded)
a) CPR (Calibrated Peer Review) : UCLA -- intelligent writing in science classes
The educational materials developed by publishers is of substantially
poorer quality (intellectual content). The initiative must be taken by the
universities!
Accessibility-a rapidly changing landscape
• A recent study showed that “more than half of Americans now had access to the
Internet from home.”
• Release of Internet access network devices e.g. WebTV, iPAQ, etc.
Romualdo T. de Souza, Dept. of Chemistry, IUB
Purdue University Feb. 19th, 2003
http://calm.indiana.edu
"Can computers help faculty
teach and students learn?"
Motivation behind C.A.L.M.
• Math and science are problem solving subjects  students need to solve
questions on a regular (sustained) basis to help internalize the concepts.
• The best feedback is immediate. Not only is this a statement about our instant
gratification society, but education research demonstrates this fairly
conclusively.
• students should have individualized questions to encourage symbolic
communication (enhances cooperative learning).
• the faculty member should have “instant” feedback about the performance of
a class or individual on a particular topic.
Romualdo T. de Souza, Dept. of Chemistry, IUB
Purdue University Feb. 19th, 2003
http://calm.indiana.edu
"Can computers help faculty
teach and students learn?"
Basis for C.A.L.M.
“Computers are great for three things: Storing data, performing calculations, and
making decisions.” - RdS
1.
WWW based

a) allows widespread dissemination of material
b) facilitates accumulation of group/class statistics.
2. Questions are
individualized (semirandom, algorithmically
generated).
3. Based on a Socratic
pedagogy


a)
Encourages symbolic communication
b)
reduces copying/cheating.
Encourages logical thinking
The computer allows us to “simulate” one-on-one interaction.
Romualdo T. de Souza, Dept. of Chemistry, IUB
Purdue University Feb. 19th, 2003
http://calm.indiana.edu
"Can computers help faculty
teach and students learn?"
Features of the program
• Each student is presented with an individualized, semi-randomly generated
question. All questions are algorithmically generated which allows the
student immediate feedback about the correctness of their response.
• A student can make repeated attempts to solve the question without penalty
– which encourages learning.
• A student can request a new question to ensure comprehension of a
particular topic.
• All interactions of students with the program are logged so a teacher (or
parent) could examine the performance of a student or a group of students.
Romualdo T. de Souza, Dept. of Chemistry, IUB
Purdue University Feb. 19th, 2003
http://calm.indiana.edu
"Can computers help faculty
teach and students learn?"
C.A.L.M. incorporates several different types of problems:
1.
Numerical Problems
2.
Multiple choice
3.
Multiple selection – several of the provided choices are correct answers to
the question
4.
Graphical – Based on a graphical picture the student must answer a question
or select a region of the graph.
5.
Textual – The student must provide a textual answer to a question (e.g.
“Write the products of the following chemical reaction”).
6.
Multistage – The student is directed through a question. Upon incorrectly
answering a question the student is presented with a new “leading” question.
http://calm.indiana.edu
Romualdo T. de Souza, Dept. of Chemistry, IUB
Purdue University Feb. 19th, 2003
http://calm.indiana.edu
Correct
Incorrect
Is the following reaction spontaneous
under standard conditions?
CH4(g) + O2(g) ==> CO2(g) + H2O(l)
Yes
No
CH4(g) + O2(g) ==> CO2(g) + H2O(l)
Your answer is correct !
What is Go for the reaction?
Go =
kJ
Romualdo T. de Souza, Dept. of Chemistry, IUB
Purdue University Feb. 19th, 2003
http://calm.indiana.edu
Correct
Incorrect
Is the following reaction spontaneous
under standard conditions?
CH4(g) + O2(g) ==> CO2(g) + H2O(l)
Yes
No
A reaction is
spontaneous if and
only if Go < 0.
What is Go for the
reaction?
Go =
kJ
CH4(g) + O2(g) ==> CO2(g) + H2O(l)
Your answer is correct !
What is Go for the reaction?
Go =
kJ
Romualdo T. de Souza, Dept. of Chemistry, IUB
Purdue University Feb. 19th, 2003
http://calm.indiana.edu
Your answer is
incorrect! Go is
the sum over the
products’ Gibbs
Free Energy of
formation minus
minus the sum
over the
reactants’ Gibbs
Free Energy of
formation. What
is Go for the
reaction?
Go =
kJ
Your answer is
correct! What is
the value of ln(K)
for this reaction?
ln(K) =
Romualdo T. de Souza, Dept. of Chemistry, IUB
Purdue University Feb. 19th, 2003
http://calm.indiana.edu
Your answer is
incorrect! Go is
the sum over the
products’ Gibbs
Free Energy of
formation minus
minus the sum
over the
reactants’ Gibbs
Free Energy of
formation. What
is Go for the
reaction?
Go =
kJ
Your answer is
correct! What is
the value of ln(K)
for this reaction?
ln(K) =
Romualdo T. de Souza, Dept. of Chemistry, IUB
Purdue University Feb. 19th, 2003
http://calm.indiana.edu
Your answer is
incorrect.
Go = -RTlnK
• R = 8.314 J/
(mol.K)
ln(K) =
Good job!
Your answer
is correct. You
have finished
answering this
question.
Romualdo T. de Souza, Dept. of Chemistry, IUB
Purdue University Feb. 19th, 2003
http://calm.indiana.edu
Your answer is
incorrect.
Go = -RTlnK
• R = 8.314 J/
(mol.K)
ln(K) =
Good job!
Your answer
is correct. You
have finished
answering this
question.
Romualdo T. de Souza, Dept. of Chemistry, IUB
Purdue University Feb. 19th, 2003
http://calm.indiana.edu
IV. Initial Results
“Fuzzy-Wuzzy” results (anonymous survey following Spring 1997)
1.
Students increased time spent studying for course from <
2hr/week to ~ 4 hrs/week.
2.
Do you find CALM useful in helping you learn the material
presented in the course?
a) extremely useful
37%
b) very useful
35%
c) moderately useful
19%
d) slightly useful
5%
e) not useful at all
3%
3.
76% said that it was a “good” or “very valuable” tool in a
differently worded or arranged question.
"Can computers help faculty
teach and students learn?"
These initial self-perceived benefits were encouraging!
“The most positive aspect of CALM is that you can immediately find out whether
or not you got your question correct. This enables a person to know whether or
not she is approaching the problem in a correct way.” – Anonymous student
Romualdo T. de Souza, Dept. of Chemistry, IUB
Purdue University Feb. 19th, 2003
http://calm.indiana.edu
"Can computers help faculty
teach and students learn?"
Analysis following Fall 1997 (R. Althauser)
Multivariable linear regression analysis:
1.
2.
CALMPTS = f(SATACT, CHEM PLACEMENT)
EXAM SCORE = f(SATACT, CALMPTS)
3.
EXAM SCORE = f(SATACT, CALMPTS, CHEM PLACEMENT)
SATACT : measure of math skill prior to course.
CHEM PLACEMENT : Chemistry placement exam assesses prior knowledge of
chemistry.
EXAM SCORE : 4 in-term exams (25 pts each.) + 1 Final (50 pts.)
CALMPTS : Points accumulated in CALM, 1 pt./question correctly answered; 100
questions normalized to 10 pts.
Sample :
Students who missed no more than 2 exams
511/681 (77%)
Romualdo T. de Souza, Dept. of Chemistry, IUB
Purdue University Feb. 19th, 2003
http://calm.indiana.edu
"Can computers help faculty
teach and students learn?"
OLS Regression analysis
Dep Var: SCORE N: 498 Multiple R: 0.726 Squared multiple R: 0.527
Adjusted squared multiple R: 0.525 Standard error of estimate: 15.250
Effect
Coefficient
Std Error
Std Coef
Tolerance
t
P(2 Tail)
CONSTANT
-9.785
6.118
0.000
.
-1.599
0.110
CALMPTS
3.787
0.246
0.495
0.925
15.398
0.000
SATACT
0.113
0.010
0.360
0.900
11.052
0.000
CHEMPLACE
1.349
0.272
0.156
0.967
4.963
0.000
After correcting for differences in math ability and prior chemistry knowledge,
CALM performance is a strong predictor of Exam scores.
• Each additional CALM point corresponds to almost 5 additional exam points.
•
Distribution of CALM points within 1 std. Deviation of mean is 4-10  This
difference of 6 CALM points corresponds to 24 points in EXAM SCORE.
This difference exceeds a letter grade!
Romualdo T. de Souza, Dept. of Chemistry, IUB
Purdue University Feb. 19th, 2003
http://calm.indiana.edu
Expanding the impact of CALM
"Can computers help faculty
teach and students learn?"
CALM is presently being used at:
• I.U.: C105,C125,C106,C126,C101, C103, and P125
• Morehead State University C101
• High Schools
Bloomington North
Bloomington South
Plymouth
Westfield
Zionsville
Romualdo T. de Souza, Dept. of Chemistry, IUB
Purdue University Feb. 19th, 2003
http://calm.indiana.edu
"Can computers help faculty
teach and students learn?"
V. Summary
• We have developed and implemented in a large class environment a WWW based
learning tool (C.A.L.M) which provides algorithmic generation of individualized
questions for students based upon a Socratic pedagogy.
• Student performance with this tool correlates strongly with exam performance.
VI. Outlook
• Adaptive learning systems with artificial intelligence can significantly enhance
the learning process.
Need to develop “concept” questions in CALM which assess
student understanding
Present questions in a progressive manner which responds to students
success. Only allow progression to next concept if sufficient mastery is
achieved.
Develop better assessment tools
Romualdo T. de Souza, Dept. of Chemistry, IUB
Purdue University Feb. 19th, 2003
http://calm.indiana.edu