Next Generation
Math Assessments
Getting Ready for the Next Gen
(and This Gen)
February 2010
Mike Gallagher
Math Test and Measurement Consultant
Accountability Services, NCDPI mgallagher@dpi.state.nc.us
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION
1

Readiness & The Beehive
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION
2

• Present and Review Schedules for new generation math tests.
• Highlight what’s new and different.
• Suggest preparations, some of which pertain specifically to the new generation, others which apply across math generations.
3
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

• Encourage teachers and math specialists to take the on-line moodle class in item writing and review and participate in test development.
• http://www.dpi.state.nc.us/accountability/testing/ scroll down to:
Test Item Writer/Reviewer Online Training
4
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

• Things will be happening quickly!
• Be sure that most teachers are on the math curriculum listserv to get the regular, fun, and informative messages.
• Just send a request and indicate the level: elementary, middle, or high shart@dpi.state.nc.us
5
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

1. Several step algebra problem.
2. A non-routine problem.
What are some starting strategies for the second question?
These are similar to SAT questions.
Note that students need to be able to apply both procedures and strategies.
Check out the SAT question of the day: http://sat.collegeboard.com/practice/sat-question-ofthe-day
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION
6

• This conference’s Global Positioning
Students / 21 st Century theme implies strong math problem solving. e.g., see next slide: Common Core Readiness
Standard for Mathematical Practice .
7
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

Mathematical Practice
Proficient students expect mathematics to make sense. They take an active stance in solving mathematical problems. When faced with a nonroutine problem, they have the courage to plunge in and try something, and they have the procedural and conceptual tools to carry through. They are experimenters and inventors, and can adapt known strategies to new problems. They think strategically. See … www.corestandards.org
From … Common Core State Standards Initiative:
College and Career Readiness Standards
8
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

• Essential Standard plan from Sept.09
• Sept. 2009, Standards adopted NCSBE
• Spring 2010, Item Tryouts (small sample)
• 2010-2011, Stand alone field test
• 2011-2012, 1 st Operational year
See: http://www.ncpublicschools.org/acre/
9
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

• January 2010. North Carolina submitted a RttT application. http://racetothetop.nc.gov/
• RttT involves state work in four areas, including standards and assessments.
http://www2.ed.gov/programs/racetothetop/
10
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

• February 2010. K-12 math draft standards posted for public comment www.corestandards.org/
• April 2010. NC to release common core plus possible additional 15% for public comment.*
• July 2010. Standards to State Board*
*From NC RttT application, January 2010, p. 48.
11
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

• Alert the math people in your area to review, discuss, and comment on these common core standards, both in
February and in April.
12
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

The Essential Standards and RttT call for a Comprehensive Assessment
System, including:
• Formative Assessment
• Benchmark Assessment
• Summative Assessment
• Diagnostic Assessment
13
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

You cant win Song from The Wiz:
“You can’t win, chile, you can’t break even, and you can’t get out of the game.”
A fine book on motivation and attribution is Learned Optimism by Martin
Seligman (check it on youtube also  )
14
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

When it comes to learning, students are the key assessment users: “Am I succeeding?
Am I improving over time? What should I do next to succeed? Does my teacher think I am capable of success? Do I think I am capable of success?”
…from Student-Involved Classroom Assessment, 3 rd
Edition. R ichard J. Stiggins, 2001, Prentice-Hall.
15
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

[Diagnostic Assessment] requires treating assessment as an ongoing process of evaluating student knowledge and ability, not through the administration of more standardized tests but through meaningful analysis of student work. p.184.
• The Trouble with Black Boys …and other
Reflections on Race, Equity and the Future of Public Education by Pedro A. Noguera,
Jossey-Bass, 2008.
16
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

Formative Diagnostic Assessment can be a BIG part of closing the gap.
• Attend the NC Falcon session at this conference on a great professional development tool for formative assessment (FA).
• Share the information so that teachers can become more proficient at FA
17
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

• Using Student-Involved Classroom
Assessment to Close Achievement
Gaps , Richard Stiggins and Jan
Chappuis. Theory into Practice, 44,
1,11-18.
• A very short and readable article.
At www.assessmentinst.com
….(Point to “Resources” and click on “Papers.”)
18
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

• Go to a Quantiles ® Session.
• Alert math people that students will receive a Quantile score on their ISR and that the Qtaxon map can help teachers understand skill progressions.
e.g., If a student does not have an understanding of slope, what comes prior? What resources are there?
19
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

• Math Level A and Level B/C
• Discrete math and graph theory in the standards for Level A and Level B/C.
• Constructed Response Questions –
“Explain …” These are especially good for Formative Assessment.
• More tests with calculator inactive sections: Grade 8 & Levels A and B/C
20
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

• Target math understanding, math procedural skills, and problem solving.
• Regularity, e.g., biweekly
• Short size, e.g., 10 questions
• Mixture, e.g., 7 multiple choice and 3 constructed response.
21
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

• Content: start with previous grade questions and mix in current grade standards
• Source: Released and sample items.
• Scoring: Nope. Teacher usually gives one to many feedback, students grow in self and peer assessment. Use Polya’s problem solving strategy: http://www.ncpublicschools.org/docs/accountability/tes ting/eog/ProblemSolvingStrategy.pdf
22
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

• Quantiles, Wed. at 2:15
• Math Formative – Wed. at 3:45
• Tryouts – Thurs. at 8:00
• Quantiles – Thurs. at 9:15
• ACRE – Thurs. at 10:45
• Next Generation - Thurs. at 2:15
• NC FALCON – Formative at 5:00
23
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

“Toto, we’re not in Kansas anymore.”
24
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

• The Strategy is a one-page sheet of bullet items on the DPI web site, www.ncpublicschools.org/accountability/testing/eog/math/
Scroll down to Formative Assessments for Problem Solving
• The strategy sheet can be used to guide Formative Assessment descriptive feedback for problem solving throughout the year.
25
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

• PROBLEM SOLVING
G. Polya ... How to Solve It, the book is from the 1950s, but it’s stood the test of time.
• FORMATIVE ASSESSMENT
R. Stiggins … Student-Based
Classroom Assessment, Very readable textbook.
26
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

“I’d unravel any riddle, for any individ’le
In trouble or in pain .. Oh, I would tell you why, The ocean’s near the shore. I could think of things I never thunk before …” If I only had a brain.
Scarecrow in The Wizard of Oz.
27
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

Strong Emphasis in the NC 2003SCS
“Problem solving and reasoning are stressed throughout the goals at each grade level and in every course. …
Experiences in problem solving processes should permeate instruction. Problem solving should be integrated early and continuously into each student’s mathematics education .”
28
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

Two especially useful tactics.
(1) Sketching a drawing and
(2) Making a table are two problem solving tactics that are extremely useful.
Students need ongoing experience and assessment with these and other problem solving tactics.
29
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

Definition: “Formative assessment is a process used by teachers and students during instruction that provides feedback to adjust teaching and learning to improve students achievement of intended instructional outcomes.”
CCSSO 2008, adopted by N.C. Board of
Education
30
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

Help students answer three questions:
• Where am I going? (Target)
• Where am I now?
• How do I get there?
Formative Assessment helps students take more responsibility for their learning.
31
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

When it comes to learning, students are the key assessment users: “Am I succeeding?
Am I improving over time? What should I do next to succeed? Does my teacher think I am capable of success? Do I think I am capable of success?”
…from Student-Involved Classroom Assessment, 3 rd
Edition. R ichard J. Stiggins, 2001, Prentice-Hall.
32
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

“NCTM’s Assessment Principle indicates that assessment should not be done to students; rather, assessments are for students and should be used to guide and enhance their learning …”
(continued)
*
Francis (Skip) Fennell, President of the National
Council of Mathematics Teachers (NCTM)
33
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

“Formative assessment involves using classroom-based assessments to collect feedback that can be used to improve teaching and learning.”
From NCTM News Bulletin (December
2006) Complete message on the web at http://www.nctm.org/ … Go to the
“President’s Corner” -- Just one page and well worth reading!
34
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION
35

1. Problem Solving is learnable.
2. Formative Assessment is Effective for Complex Reasoning.
3. Descriptive Feedback without grades is key to formative assessment.
4. Formative assessment emphasizes self and peer assessment, not more teacher grading.
36
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

• Released test questions
• Problem Solving Strategy Sheet
• Math Problem Solving Worksheet.
• Other problem sources, NAEP, PISA, textbook, sample questions, etc.
37
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

1. Choose a problem for class discussion. e.g., NAEP, Estela.
2.
Introduce Polya’s strategy sheet and connect to the problem in Step 1.
3. Talk about problems as puzzles.
38
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

• Tell students what the target is: to become great math problem solvers!
• Have a class discussion on what great problem solving looks like. Use examples.
• Ask students to critique the quality of problem solving of some examples.
39
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

• Explain that these puzzles will provide feedback, not grades.
• BUT, similar problems will be graded on quizzes or tests.
40
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

Introduce “Problem Solving Sheet Worksheet”
• Work through a problem as a class, having students answer the questions on a sheet.
• Have students try on their own to solve a problem and provide the answers on the sheet.
• Students should not hurry through the problems. Stress understanding.
41
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

• Students will build a non-graded portfolio with their notes on some of the problems.
• Students will choose problems to include in the portfolio to keep track of their work, to help plan further action, and to communicate their progress to teacher and parents.
42
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

• What do I need to find?
• What is given?
• Make a sketch or table.
• What facts are needed?
• What procedures are needed?
• Stuck? What tactics did I try?
43
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

• Work the problem. Be sure to check each step.
• Did I get a reasonable answer?
• Is there another way to solve it?
• Are there any choices that are clearly wrong?
• How did I check my answer by solving the problem in a different way?
44
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

• What mistakes could cause errors in solving this problem?
• What can I learn to help with future problem solving?
• What will I work on next to become a better problem solver?
45
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

Select problems from the “grade below” released test which are cognitively complex. Call them “puzzles!”
Assign a puzzle regularly.
Discuss the next day. Use self assessment and peer assessment.
Mix in constructed response questions with the multiple choice.
46
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

Action Plan – Outcomes
– Short Term
Include one or more cognitively challenging problem on your regular quiz or test.
In discussing the quiz results, highlight common mistakes which are related to
Polya’s strategies.
Have students self assess while you are going over the problem.
47
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

One to many. Teachers provide feedback to the class on common strengths and weaknesses.
Many to many. Students provide feedback to each other and themselves. For self and peer assessment, students are providing feedback, not grading.
48
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

Action Plan – Outcomes –
Medium Term
• Have students explain to you their progress and difficulty in become better problem solvers.
• Have students explain at parent visits, how they are improving as problem solvers, using examples from their portfolios.
• Students grow in responsibility for their own learning, but not overnight!
49
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

Second semester – start using questions from the “on level” released test. The questions should be cognitively complex and use concepts and procedures you have already covered.
Help the students to use non-graded portfolios to track their growth.
Efficacy: “Achievement results from effort.”
50
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

What does the student believe is the cause of his/her success (or failure) on a summative test?
• “I passed because I’m smart.” “I failed because I’m not smart.”
• “I passed because I worked hard.” “I failed because I didn’t work.”
Learned Optimism by Seligman
51
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

You cant win Song from The Wiz:
“You can’t win, chile, you can’t break even, and you can’t get out of the game.”
52
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

There are lots of wonderful techniques to learn in formative assessment.
Professional Development modules are being added to the web this fall at: http://www.dpi.state.nc.us/accountability/educators/
Scroll way down and click on “Learn
More about Formative Assessment.”
53
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION
54

Trend in Average Scale Score
250
245
240
235
230
225
220
215
210
1990 1995
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION
Cut for Proficient = 249
Cut for Basic = 214
2000 2005 2010
55
NC
US

Trend in Average Scale Score
270
265
260
255
250
290
285
280
275
Cut for Proficient = 299
Cut for Basic = 262
1990 1992 1996 2000 2003 2005 2007
56
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION
NC
US

STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION
57

North Carolina’s rapid progress from the
1980s through the 1990s may have been due to better teaching of procedures.
To get off the plateau, we need to get better at problem solving.
58
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

from The Wiz
• There are lots of choices and details to tie math problem solving and formative assessment into your classroom.
• Please let me know what you try and what works (or doesn’t) for you: mgallagher@dpi.state.nc.us
• Just put YB Road in the subject line 
59
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

Type
•Exercises
• Problems
• Puzzles
• Projects
Goals
•Assessment for
Learning
•Assessment of
Learning
60
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

“
Ever since I can remember, my father has given us little problems or puzzles. … These puzzles challenged us and encouraged our curiosity … they taught us how to reason and think for ourselves.” Sarah Flannery In
Code – a Young Woman’s Mathematical
Journey. Algonquin Books of Chapel Hill,
2002. Ms. Flannery’ high school project on cryptography won Ireland’s Young Scientist award in 1999.
61
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

Formative Assessment for
Higher-Level Thinking
“Students who participate in the thoughtful analysis of quality work, so as to identify its critical elements or to internalize valued achievement targets, become better performers.
When students learn to apply those standards so thoroughly that they can confidently and competently … more
62
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

… evaluate their own and each others’ work, they are well down the road to becoming better performers in their own right (Stiggins, p. 25)
Good papers to read and other resources at Stiggins’ ATI website: http://www.assessmentinst.com/
63
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION

www.ncpublicschools.org/ accountability/testing/
• Details about EOGs and EOCs
• Sample questions
• Take the on-line writer/reviewer training
• Link to NAEP results
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION
64

o Balance
Assessment of
Learning with
Assessment for
Learning.
o Balance emphasis on algorithms with emphasis on heuristics.
STATE BOARD OF EDUCATION
DEPARTMENT OF PUBLIC INSTRUCTION
65