Overview of Alternative
Assessment Methods
John Holcomb
Cleveland State University
http://academic.csuohio.edu/holcombj
j.p.holcomb@csuohio.edu
Outline
• Writing Assignments/Projects
• Data Analysis Projects
–
–
–
–
–
–
Term Projects
Class Projects
Team Projects
Individual Projects
In-Class Projects
Creating your own Projects
• Other Rubrics
• Student Journals
Writing Assignments/Projects
• Joy Jordan’s ARTIST Roundtable Presentation
• https://data.gen.umn.edu/artist/publications.html
• First Day/Last Day Informal Writing
– On the first day and last day of class, students
informally wrote for 5 minutes on the following
question:
– What is statistics and how is it relevant?
– They turned in their writing, but did not include their
names.
Formal Example Assignment (not well
thought out)
• Select a newspaper, magazine, or Internet
article that involves statistics, as you
currently know it (be sure to include the
source). Type a short (2 – 3 paragraphs)
yet complete explanation of how and why
the statistics are used in the article. If you
think the statistics are used well (e.g.,
illuminated a point), then explain why. Or if
you think the statistics are misused, then
explain why.
Revamped Example Assignment
Carefully read the newspaper article included
below. Type a 1-page, single-spaced letter to the
editor, responding to the article. In the letter,
clearly explain if the study allows for the
conclusion that a causal relationship exists
between [insert variables from any article].
Because you are writing to all readers of the
newspaper, do not assume that your audience
has taken a statistics course (i.e., do not use
technical terms).
Sources of Articles
•
http://news.yahoo.com/
•
http://www.nytimes.com/pages/health/index.html
•
•
Girls abused by dates at higher STD risk
Can make Quiz based on these kinds of
articles
1.
2.
3.
4.
What is the population of interest?
What kind of study was completed?
What is a major finding of the study?
Does the article claim or imply a casual connection?
Is it warranted?
Second Example of Formal Writing
Hey Kiddo,
I am worried about Grandma. Remember that she was diagnosed with high
blood pressure? Well, she’s currently taking the medication Makemewell to
lower her blood pressure. At the time of Grandma’s diagnosis, her doctor
said that a randomized, double-blind experiment had been conducted and
that Makemewell was shown more effective in lowering blood pressure than
a placebo. To be honest, I have no idea what any of that means, but I
believed and trusted the doctor. Now I’ve heard two stories that make me
think differently. Larry, our next-door-neighbor, was taking Makemewell and
he got a terrible fever that put him in the hospital. Also, my coworker, Sally,
actually had her blood pressure go up while she was taking Makemewell!
I’m now very suspicious of this medication.
I know that you’re taking a statistics course at college. Based on the
information I’ve given you, do you think Grandma should stop taking her
medication? Whatever your opinion, will you please explain yourself
thoroughly and clearly? (I will draw on your responses when I talk with the
doctor.) And please don’t use any statistics mumbojumbo that I won’t
understand. I really appreciate your help with this.
Love, Dad
Your assignment is to type a 1-2 page letter (single-spaced, 12-pt. font)
responding to your dad.
Grading Criteria
• Assignment #1
• Grading Criteria (30 points possible)
• ____ The explanation to your dad convinces me (your
teacher) that you understand the statistical concepts
involved in the assignment. (14 points)
• ____ The explanation to your dad is thorough, well
organized, and clear. (6 points)
• ____ The explanation to your dad is presented in nontechnical terms that he will understand. (5 points)
• ____ You successfully paid attention to accepted
conventions of language use (syntax, spelling, grammar,
readability, etc.) (5 points)
• Can have Peer Review of Letter to Dad
• Follow up Assignment on explaining the
meaning of the p-value from the clinical
trial that approved Makemewell.
Data Analysis Projects
• Information at
https://data.gen.umn.edu/artist/projects.html
• Data Collection Projects
• Students collect data and work on project all
term long
• Sandra Fillebrown, “Using Projects in an
Elementary Statistics Course for NonScience Majors,” Journal of Statistics
Education v.2, n.2 (1994)
Brown: Schedule of Activities
1.
2.
3.
4.
5.
6.
Description of Topic
Method of collecting data
Data in rough form
Data in spreadsheet form
Rough draft
Final drafts
Chance: Checklist
1. Introduction
2. Summary of data collection (no such
thing as “too much detail” in this section!)
3. Summarize and interpret the data
(concise, well-labeled, easy to read)
4. Statistical inference (with justification
for procedures chosen)
5. Interpretation/Explanation of Results
6. Overall Conclusions
Beth Chance Rubric
• Grading Criteria for Final Report:
• 10%: Quality of written report
• 20%: Design of survey/experiment – was data collection
adequately explained, were the appropriate data
collected to answer the questions posed. Points here will
include originality of topic chosen.
• 25%: Correctness of statistical analysis and checks of
technical conditions
• 20%: Appropriateness of interpretations of the results of
the statistical calculations and conclusions (is it a cause
and effect relationship? what is a reasonable
population?)
• Presentations: (other 25%)
• Smith, G. (1998)
– "Learning Statistics By Doing Statistics"
Journal of Statistics Education [Online], 6(3)
www.amstat.org/publications/jse/v6n3/smith.html
– Appendix has 20 project ideas
• Challenges to Term Projects
– Small Sample Sizes
– Students don’t know enough to complete
analysis
Class Projects
• Nathan Wetzel, Class Data Collection and
Analysis – Assessment STATS, Spring 2005
•
http://www.rossmanchance.com/artist/Proctoc.html
• Assignment 1
– (10 points) We want to design a study of fast food.
We have an imaginary client who has asked us to get
answers to the following questions.
1. How many ounces are in the average McDonald's Large
French Fry? Small French Fry?
2. How often does the average UWSP student eat fast food? Is
there a difference between men and women?
– If you have any questions for our client, Professor
Wetzel will play the role of the client.
Grading Rubric
(5 points) Basics: Does the description include all of the
required parts including
• accurate and appropriate use of terminology?
• restatement
• measurements - including specifics
• questions to ask - including specifics
• extra demographic question
• how to get randomize - including specifics
• other data
(2 points) Organization: Is the description organized and
neatly presented?
Great answers also include some pluses.
• Pluses:
– Does the description include any significant extras?
– Significant improvements to a basic data collection design.
– Extra thought into the specifics of this context - recognizing
potential problems and giving solutions.
• Minuses:
– Does the description indicate that the student is mimicking a
book answer and not considering the context?
– Does the description include a design that would be extremely
impractical?
– Does the randomization described introduce a significant
confounding variable that was not identified?
Assignment II
• (22 points) Analyze all interesting portions of the data and write a
short report (at least 2 pages, double spaced, typed, not including
graphs) to our client summarizing our data collection, your analysis
and your interpretation. Write the report to a client who understands
p-values, but is not interested in the details of the computations.
Your client understands all of the graphical methods that we have
looked at. In other words, don't include MINITAB hypothesis test
output, but do include graphical summaries.
In addition, include a short letter that we could send to McDonalds.
Notice that this part is the main part of the assignment. 'Good'
answers will receive 14 points, you must do a great job to receive
full credit. In other words, I want YOU to ANALYZE the data. This
part is open ended and in order to do a great job, you will need to
look at MINITAB output that I did not explicitly tell you to get. You will
also need to THINK AND CARE about the data. Answer the
questions that the client should have asked or that you saw
interesting results.
Goals of Assessment:
1. The students know from the beginning that this
problem will be assessed differently. I have
found that many are very satisfied to get a
score of 70% because they knew that their
solution was average.
2. The students have access to solution to similar
problems.
3. The assessment is done and the problem is
handed back in a short time period.
4. The students who did not receive full credit
have a chance to see what kinds of solution did
receive full credit.
Past Topics
• www.uwsp.edu/math/wetzel/Artist
• Other topics:
– Surveys: Atomic bomb, campus parking, cell
phone use, credit cards use, and Y2K worries.
– Experiments: pulse rates, backpack weights,
volume in Coke products, readability of Time
magazine, chocolate chip cookie weights.
Structured Data Analysis Projects
with Existing Data
• Holcomb and Ruffer, (2000), “Using a
term- long project sequence in introductory
statistics,” The American Statistician, 54,
49-53.
• Use 2 data sets that are random samples
from the North Carolina Birth Registry
(www.irss.unc.edu/ncvital/bfd1down.html)
http://academic.csuohio.edu/holcombj/
Login: holcomb pwd: projects22
Project Sequence
•
•
•
•
•
Summary Analysis
Probability Analysis
CI and Hypothesis Testing
Regression
https://data.gen.umn.edu/artist/resources.
html
• http://academic.csuohio.edu/holcombj/eku/
Grading Rubric
Organization
Clarity of Exposition
2 4 6 8 10
Layout
2 4 6 8 10
Mechanics
Grammar, Spelling & Punctuation
3 6 9 12 15
Thoroughness
2 4 6 8 10
Professionalism & Style
1234 5
Mathematical and Statistical Accuracy (50 points)
Total Points(100 points maximum)
Individual Projects
• 18 Unique Take Home Exams
• Topics
– Taste Testing Emu Meat (emu)
– College Student Nutrition Survey (nutri)
– Bone Screening (osteo)
• Same format as Team Projects
• Graded with Same Rubric
• Given at Midterm and Final
Data Cleaning Project
• Published in November, 2005 JSE
• http://www.amstat.org/publications/jse/cont
ents_2005.html
• 34 problem values, 4 of which stay
missing
• http://academic.csuohio.edu/holcombj/eku/index.html
• This data can be used for projects with:
– 2 sample t-tests
– ANOVA (One-way, two-way)
– regression
In-Class Project
• Domestic Violence Project
• http://academic.csuohio.edu/holcombj/eku/index.html
• Prevalence Ratios, Relative Risk, and Odds
– Disease is X times more prevalent with risk factor than without
risk factor
•
•
•
•
Prevalence Ratio – Cross Sectional Study
Relative Risk – Prospective Study
Odds – Case/Control Study
Odds is an estimate of PR and RR if probability of
disease is less than 10%
abuse d * sxlhe ad Crosstabula tion
Count
sx lhead
No
abused
Total
No
Yes
Yes
10
11
21
25
95
120
Total
35
106
141
Creating Your Own Projects
• National Opinion Research Center (NORC)
•
•
•
•
•
•
General Social Survey (GSS)
1008 variables, 2765 observations
Latest data available is 2002
Snapshot of American Population
Categorical and Continuous Variables
http://academic.csuohio.edu/holcombj/eku/
Other Rubrics
• Crannell, “Assessing Expository Mathematics: Grading
Journals, Essays, and Other Vagaries”, Assessment
Practices in Undergraduate Mathematics, MAA Notes
49, p. 113–115, (1999).
• Emenaker, C. E., “Assessing Modeling Projects in
Calculus and Precalculus: Two Approaches,”
Assessment Practices in Undergraduate Mathematics,
MAA Notes 49, p. 116-119, (1999).
• Emert, J., “Assessing Modeling Projects in Calculus and
Precalculus: Two Approaches,” Assessment Practices in
Undergraduate Mathematics, MAA Notes 49.
Sample Scoring Rubric - Crannell
Does this paper:
1.
2.
3.
4.
5.
6.
7.
8.
clearly (re)state the problem to be solved?
state the answer in a complete sentence which stands on its own?
clearly state the assumptions which underlie the formulas?
provide a paragraph which explains how the problem will be
approached?
clearly label diagrams, tables, graphs, or other visual representations of
the math (if these are indeed used)?
define all variables used?
explain how each formula is derived, or where it can be found?
give acknowledgment where it is due?
In this paper,
1. are the spelling, grammar, and punctuation correct?
2. is the mathematics correct?
3. did the writer solve the question that was originally asked?
Total number of yeses = grade on paper.
Sample Holistic Project Rubric - Emenaker
•
4 points: Exemplary Response, All of the following characteristics must be present.
a.
b.
c.
•
3 points: Good Response, Exactly one of the following characteristics is present.
a.
b.
c.
•
c.
The answer is incorrect due to a major flaw in the plan.
Explanation lacks clarity or is incomplete but does indicate some correct and relevant
reasoning.
A plan is partially implemented and no solution is provided.
1 point: Poor Response, All of the following characteristics must be present.
a.
b.
c.
•
The answer is correct due to a minor flaw in plan or an algebraic error.
The explanation lacks clarity
The explanation is complete
2 points: Inadequate Response, Exactly two of the characteristics in the 3-point
section are present OR One or more of the following characteristics are present.
a.
b.
•
The answer is correct.
The explanation is clear and complete.
The explanation includes complete implementation of a mathematically correct plan.
The answer is incorrect.
The explanation, if any, uses irrelevant arguments.
No plan for solution is attempted beyond just copying data given in the problem
statement.
0 points: No Response
a.
The student’s paper is blank or contains only work that appears to have no relevance
to the problem.
Group Project Evaluation Form - Emert
1. Overall, how effectively did your group work together on
the project? (Poorly, Adequately,Well, Extremely Well)
2. In an effective working group, each person should be an
effective participant. How well did your group meet this
goal? (Poorly, Adequately, Well, Extremely Well)
3. Give one specific example of something you learned
from a group that you probably would not have learned
alone.
4. Give one specific example of something another group
member learned from you that he or she probably would
not have learned alone.
5. What is the biggest challenge to group projects? How
could this challenge be overcome?
Student Journals
• ARTIST resources from the CHANCE
(Dartmouth) course
– https://data.gen.umn.edu/artist/journals.html
• Beth Chance, (2002)
– "Components of Statistical Thinking and
Implications for Instruction and Assessment"
Journal of Statistics Education [Online], 10(3)
www.amstat.org/publications/jse/v10n3/chance.html
Chance: Journals
• Students reflect on activities from Activity
Based Statistics by Scheaffer,
Gnanadesikan, Watkins, and Witmer
– Explanation of activity
– Answer directed questions
– Helps get beyond the entertainment to the
concept the activity communicates
• Write Chapter Summaries
• Worth 7.5% of final grade