September 24, 2008
Dear Colleagues:
The work contained within this report represents a significant step for the state of Washington. As
required by the 2008 Legislature, the Office of Superintendent of Public Instruction (OSPI) is
required to present recommendations of no more than three basic mathematics curricula at the
elementary (K-5), middle (6-8) and high school (9-12) grade spans to the State Board of
Education for their review and comment. In determining the curricula to recommend, OSPI
conducted an instructional materials review of core/comprehensive programs. Since the K-8
mathematics standards were approved in April 2008, K-8 programs were reviewed first. High
school programs and K-12 supplemental programs will be reviewed in November and December
2008. This report details the recommendations for K-8 programs.
The recommendations for elementary and middle school are as follows:
Elementary (K-5):
Math Connects (elementary)
Bridges in Mathematics
Middle (6-8):
Holt Mathematics
Math Connects (middle)
I deeply appreciate the many individuals who have contributed hours of their time and energy to
review materials, and to provide thoughtful comment and insights on the Preliminary Draft
Report that was completed in August 2008. I commend all who have been involved for their
support in ensuring that this has been an inclusive and rigorous process.
The next step in the process for K-8 will be for me to present the elementary and middle school
recommendations to the State Board of Education on September 24, 2008. Per guiding legislation,
the SBE will have up to two months to comment on the recommendations. Following SBE
comment, OSPI will issue a final report that outlines the final recommendations for grades K-8 in
mid-November.
Sincerely,
Dr. Terry Bergeson
State Superintendent of Public Instruction
(This page intentionally blank)
2008 Mathematics Core
Comprehensive Materials
Review &
Recommendations Report
Final Draft
Grades K-8
September 24, 2008
Office of Superintendent of Public Instruction
Old Capitol Building
PO Box 47200
Olympia, WA 98504-7200
(This page intentionally blank)
Table of Contents
1
Executive Summary..................................................................................................... 9
1.1 Introduction/Purpose ........................................................................................... 9
1.2 Scope and Background ....................................................................................... 10
1.3 Contributing Stakeholders ................................................................................. 10
1.4 Process Overview ............................................................................................... 11
1.5 Findings .............................................................................................................. 13
1.5.1 Data ............................................................................................................. 13
1.5.2 Online Availability ....................................................................................... 19
1.5.3 Comments ................................................................................................... 20
1.6 Recommendations ............................................................................................. 22
1.6.1 Elementary School (K-5) Recommendations .............................................. 22
1.6.2 Middle School (6-8) Recommendations ..................................................... 23
2 Project Process .......................................................................................................... 24
2.1 Review Instrument Development ...................................................................... 24
2.1.1 Content/Standards Alignment Threshold ................................................... 25
2.1.2 Scale Definitions.......................................................................................... 25
2.1.3 Measurement Criteria................................................................................. 27
2.2 Reviewer Selection Process ................................................................................ 32
2.3 Publisher Involvement ....................................................................................... 32
2.4 Review Week Process......................................................................................... 33
2.5 Data Analysis Process/Methodology ................................................................. 34
3 Results ....................................................................................................................... 38
3.1 K-5 Results .......................................................................................................... 38
3.1.1 Content/Standards Alignment .................................................................... 38
3.1.2 Part 2: Other Factors for Grades K-5 .......................................................... 46
3.1.3 Results of Individual Publisher Series ......................................................... 48
3.2 6-8 Results .......................................................................................................... 62
3.2.1 Content/Standards Alignment .................................................................... 62
3.2.2 Part 2: Other Factors for Grades 6-8 .......................................................... 66
3.2.3 Individual Publisher Series .......................................................................... 69
4 Data Analysis Methodology ...................................................................................... 84
4.1 Approach ............................................................................................................ 84
4.2 Response Scales ................................................................................................. 84
4.3 Distributions of Scores by Grade Level .............................................................. 85
4.4 Reviewer Bias ..................................................................................................... 86
4.5 Content/Standards Alignment ........................................................................... 93
4.6 Threshold Tests .................................................................................................. 93
4.7 Calculation of Program Means and Standard Errors ......................................... 95
4.8 Program Comparison ......................................................................................... 95
4.9 Standard Error Calculations ............................................................................... 98
2008 Mathematics Instructional Materials Review Draft Report
Page 5
4.9.1 Recommended Approach ........................................................................... 98
4.9.2 Without assuming independence of scales .............................................. 102
4.9.3 Without assuming that each item is an identically distributed measure of
the true scale mean for a program ......................................................................... 105
4.9.4 Without assuming scale independence or identical distributions ........... 107
Appendix A.
Programs Reviewed............................................................................... 111
Appendix B.
Review Instruments .............................................................................. 113
Appendix C.
Acknowledgements ............................................................................... 141
2008 Mathematics Instructional Materials Review Draft Report
Page 6
Revision History
Date
8/14/2008
8/15/2008
9/23/2008
Version Notes
Preliminary Draft 1 completed. All results subject
to change and verification. Certain sections are not
included to save space, (e.g. reviewer comments),
but will be included in the final version.
Performance Element Alignment charts scale and
threshold changed based upon feedback from initial
reviewers. Adjusted tables 1 & 2 to remove
extraneous information. Added clarifying
comments and cautions about standard error
calculations. Added introductory letter. Inserted
footnote about parent involvement subscale. Minor
language edits. Added review instruments to
appendix.
Added Data Methodology Appendix with technical
details on statistical assumptions and calculations.
Added additional graphs for individual programs
that showed individual program results in each core
content area by grade.
Updated text (particularly Findings section) to
address comments from reviewers.
Performed minor edits throughout document.
Added section that includes recommendations and
rationale from OSPI.
2008 Mathematics Instructional Materials Review Draft Report
Updated By
Porsche
Everson
Porsche
Everson
Porsche
Everson
Page 7
(This page intentionally blank)
2008 Mathematics Instructional Materials Review Draft Report
Page 8
1 Executive Summary
1.1 Introduction/Purpose
The purpose of this document is to describe the process and outcomes from the 2008
Mathematics Core/Comprehensive Instructional Material Review for grades K-8. The
report contains information about the entire process, as well as statistical results from the
review. Finally, it contains the recommendations presented to the State Board of
Education on September 24, 2008 of the core/comprehensive curricula at the elementary
(K-5) and middle school (6-8) grade spans, as required by the 2008 Legislature.
It is important to note that successful mathematics programs may exist with virtually all
of the reviewed curricula. While instructional materials matter, other factors contribute to
the success of students in Washington state learning mathematics. Those factors include
quality of instruction, parent involvement, available supports and myriad other aspects.
While the recommended curricula will receive the bulk of attention within this report, it
also provides other key results as well. These results include:
Information on all curricula materials reviewed: Districts who currently use
instructional material not in the top three recommendations will find this report
valuable. It contains detailed, specific information on how all programs reviewed
meet the newly revised 2008 K-8 Washington Mathematics Standards.
Instructors, coaches, curriculum specialists and administrators can easily see how
their materials line up against the standards, grade by grade, and identify areas
where supplementation may be needed. No one set of instructional materials
matches the new standards completely; each one will need some augmentation,
even those that are recommended.
Support to districts in evaluating instructional materials: Finally, local
districts can use the rich set of information contained within to evaluate a wide
variety of textbooks based upon factors they deem important, to help them make
decisions in future regarding mathematics textbook adoptions.
Some words of caution are necessary. Reviews of instructional materials represent a point
in time, in a continuously evolving process. New versions will rapidly supplant those
reviewed herein.
In general, there are multiple versions of instructional materials in use by districts across
the state. This review process examined only one version of each program; typically the
most recently copyrighted version. Readers should be aware that older versions of the
programs would likely have different results. Many districts are using older versions of
these programs.
Further, the existing programs were evaluated against newly revised standards. No
publisher has had the chance to update their material to produce a new version since the
2008 Mathematics Instructional Materials Review Draft Report
Page 9
K-8 standards were released in April 2008. This review simply provides a baseline
comparison, from which publishers can adapt their material to be more closely aligned
with the revised Washington standards.
1.2 Scope and Background
As per 2007 and 2008 Legislation, OSPI is required to recommend no more than three
basic mathematics curricula at the elementary, middle and high school grade spans to the
State Board of Education (SBE) within six months of the adoption of the revised
standards for their “review and comment”. 1 The K-8 standards were adopted on April
28, 2008. This document provides recommendations for the elementary (K-5) and middle
school (6-8) grade spans. In undertaking the process for making the recommendations,
OSPI elected to conduct an instructional materials review that evaluated published
core/comprehensive K-8 mathematics instructional materials using the 2008 Revised
Washington State Mathematics Standards and other factors. The resultant data was used
to inform the selection process for the recommendations
In addition, 2008 Second Substitute House Bill (2SHB) 2598 indicates that appropriate
diagnostic and supplemental materials “shall be identified as necessary to support each
curricula.” OSPI is engaging in a Mathematics Supplemental Materials Review to meet
this objective for grades K-12. The results from the K-12 Supplemental Review will be
released in a separate report. To address providing support for the selection of
mathematics diagnostic materials, OSPI has developed a Diagnostic Assessment Guide
that will be available to school districts in late fall 2008 and will provide information on
diagnostic assessment materials available in mathematics, reading, writing, and science.
This work began in 2007 in response to 2007 Senate Bill 6023.
The high school standards were adopted in late July 2008. OSPI will release a subsequent
similar report for the high school instructional materials review and recommendations.
1.3 Contributing Stakeholders
Many individuals and groups contributed to the development of the instructional
materials review process, instrument design, materials review, data analysis and
development of the report.
Instructional Materials Review (IMR) Advisory Group – A group of 22
curriculum specialists, mathematics educators, mathematicians, math coaches,
educational service district math coordinators, and district administrators from all
over the state who have experience in curriculum reviews.
State Board of Education Math Panel – Educators, mathematicians, parents,
university faculty, advocacy group and business representatives who were
actively involved in providing input on the revised mathematics standards and
have key knowledge on effective, research-based mathematics instruction.
K-8 IMR Committee – 42 individuals from around the state representing a diverse
coalition of professionals and lay people, including math educators, math coaches,
1
See 2008 Second Substitute House Bill (2SHB) 2598.
2008 Mathematics Instructional Materials Review Draft Report
Page 10
curriculum specialists, parents, business people, advocacy groups, district
administrators and mathematicians.
OSPI Staff – educational leaders, mathematics specialists, and support staff.
National Experts and External Leaders – Individuals who shared their background
and experience with state-level materials review and adoption processes. It is
important to note that these individuals contributed information about their statelevel materials review and adoption processes. Some but not all of their ideas
were incorporated into the Washington process. Inclusion of their names does not
imply that they endorsed the results contained within this report.
o Charlene Tate-Nicols (Connecticut)
o Jonathan Weins, Drew Hinds (Oregon)
o James Milgram (California)
o Jane Cooney (Indiana)
o Charlotte Hughes (North Carolina)
o Skip Fennell (Maryland)
o George Bright (Washington)
1.4 Process Overview
The 2008 K-8 Core/Comprehensive Mathematics Instructional Materials Review
involved very high stakes outcomes, particularly the selection of no more than three basic
curricula recommendations in the elementary and middle grade spans (K-5 and 6-8).
Thus, the project processes and controls were designed to be rigorous, transparent,
inclusive and reliable. Hundreds of professionals contributed to the success of the project
during its multiple phases.
Phase
Design Review
Instrument and
Process
Solicit Publisher
Involvement
Process Steps
Sought input from multiple stakeholder groups, including
IMR Advisory Group and SBE Math Panel
- Iterative development process with two full
cycles of feedback
Research-based foundational resource materials included
2008 Washington Revised Math Standards, National
Mathematics Advisory Panel Foundations for Success
(NMAP), and National Council of Teachers of
Mathematics (NCTM) Curriculum Focal Points
Used process feedback from other states which have
successfully completed curriculum reviews to design
instrument and review process.
Outcomes included:
o Two review instruments (Content/Standards
Alignment and Other Factors)
o Proposed threshold process for deriving final
recommendations
o Proposed weighting for instrument scales
All publishers invited to submit materials
Publisher’s conference held to address questions and
2008 Mathematics Instructional Materials Review Draft Report
Page 11
Phase
Select IMR Review
Committee
Review Instructional
Materials
Analyze Data
Present Preliminary
Results
Process Steps
clarify submission process
Question and Answer document disseminated widely and
updated throughout period prior to the review
Publishers provided alignment worksheet to show where
their materials aligned to revised state standards
Publishers submitted multiple sets of materials for review
week
Application materials widely distributed statewide to
school districts and education stakeholder groups,
including math educators, curriculum specialists,
advocacy groups
Objective review and scoring of each application by two
independent reviewers using a common review instrument
Selections based upon score and having sufficient
variation in expertise among reviewers (for example: K-2
educators, 6-8 educators, mathematicians, community
representatives, curriculum specialists, administrators,
parents, etc.)
Rigorous process for controlling inventory, during
publisher check-in, reviewer check-in/out, and publisher
check-out
Reviewers received full-day training in K-8 standards
Trained reviewers in how to use the scoring instruments
Performed real-time data entry
Performed variance checks and corrective training to
reduce variance and increase inter-rater reliability
Independent reviews of materials
Four or more reads on 98% of submitted material
Random assignment of materials to reviewers
Twice-daily progress monitoring
Process improvement checks daily
Exploratory data analysis by two independent statisticians
Quality control checks comparing random 10% of score
sheets to electronic data to ensure accuracy of data entry
and extract processes
Rigorous design of statistical tests, validated by expert
statistician
Presentation of results in easy to read tabular and
graphical format
Followed legislatively mandated protocol and timeline
Presented preliminary results to State Board of Education
Math Panel
Sought advice from SBE Math Panel on the analysis,
2008 Mathematics Instructional Materials Review Draft Report
Page 12
Phase
Select
Recommendations
Provide Support to
Districts
Process Steps
recommendations and process
Presented preliminary results to legislators, districts,
publishers, review participants, and public
Sought advice from the SBE Math Panel
Used process and resultant data to inform the
recommendation selections
Communicated with districts about what information they
need, and included that information in the preliminary
report
Provided key information on how all mathematics
curricula reviewed aligns to 2008 revised K-8
Mathematics Standards
Will provide information about supplemental programs (in
a separate report) designed to augment reviewed curricula
to better meet Washington standards.
1.5 Findings
1.5.1 Data
The following tables show the overall ranking for all core comprehensive programs
submitted for review. The scaled category score is the rating value expressed as a
proportion of all possible points in the category. The scale value is calculated by
averaging the raw scores in a category, then dividing by the maximum scale value to
obtain a scaled average. Each category was assigned a weight, as described elsewhere in
this report. The weights were used to derive a final composite score.
The final composite score is calculated using the formula:
2008 Mathematics Instructional Materials Review Draft Report
Page 13
Table 1. Ranked list of all core/comprehensive elementary (K-5) programs reviewed, ordered by final composite score.
Overall Ranking for All Comprehensive Elementary Programs
Instructional
Content/
Program
Balance of
Planning and
Standards
Organization
Student
Professional
Alignment
and Design
Experience
Assessment
Support
Program
Scale Weights
Equity
and
Access
Final
Composite
Score
0.700
0.090
0.075
0.050
0.045
0.040
1.000
Math Connects (Elem)
0.734
0.703
0.681
0.676
0.750
0.715
0.724
Bridges in Mathematics
0.715
0.647
0.657
0.625
0.681
0.440
0.687
Investigations
0.628
0.703
0.626
0.583
0.674
0.655
0.635
Math Expressions
0.624
0.628
0.588
0.620
0.587
0.652
0.621
Everyday Mathematics
0.592
0.608
0.560
0.656
0.568
0.586
0.593
Saxon Math (Elem)
0.604
0.492
0.519
0.580
0.486
0.606
0.581
Growing with Mathematics
0.560
0.633
0.594
0.632
0.620
0.532
0.575
enVision
0.575
0.560
0.479
0.602
0.536
0.618
0.568
Progress in Mathematics
0.575
0.492
0.481
0.552
0.497
0.509
0.553
Math Out of the Box
0.529
0.532
0.512
0.542
0.493
0.378
0.521
Math Trailblazers
0.517
0.530
0.509
0.560
0.578
0.478
0.521
Singapore Math Standards
0.355
0.425
0.404
0.370
0.420
0.264
0.365
0.584
0.579
0.550
0.583
0.573
0.535
0.578
Average
2008 Mathematics Instructional Materials Review Draft Report
Page 14
Table 2. Ranked list of all core/comprehensive middle school (6-8) programs reviewed, ordered by final composite score.
Overall Ranking for All Comprehensive Middle School Programs
Content/
Standards
Alignment
Program
Scale Weights
Program
Organization
and Design
Balance of
Student
Experience
Assessment
Instructional
Planning
and
Professional
Support
Equity and
Access
Final
Composite
Score
0.700
0.090
0.075
0.050
0.045
0.040
1.000
Holt Mathematics
0.855
0.847
0.778
0.708
0.795
0.815
0.837
Math Connects (Middle)
0.741
0.722
0.675
0.670
0.656
0.657
0.723
Prentice Hall Mathematics
0.736
0.636
0.631
0.615
0.628
0.713
0.707
Math Thematics
0.714
0.647
0.636
0.653
0.632
0.583
0.690
McDougal Littell Math Course
0.676
0.597
0.606
0.642
0.576
0.681
0.658
Connected Mathematics 2
0.653
0.561
0.531
0.639
0.611
0.454
0.625
Impact
0.604
0.667
0.612
0.504
0.614
0.601
0.605
Cognitive Tutor/Carnegie Algebra
0.589
0.619
0.639
0.660
0.569
0.440
0.592
Saxon Math Intermediate (Middle)
0.570
0.517
0.564
0.538
0.576
0.542
0.562
CPM Middle Grades Program
0.512
0.533
0.558
0.566
0.427
0.375
0.511
Everyday Math/Transition
0.391
0.546
0.426
0.564
0.417
0.444
0.419
Mathscape
0.368
0.558
0.517
0.556
0.580
0.384
0.416
ALEKS, Online Mathematics Solution
0.386
0.367
0.447
0.521
0.139
0.574
0.392
0.598
0.601
0.585
0.603
0.554
0.558
0.594
Average
2008 Mathematics Instructional Materials Review Draft Report
Page 15
Table 3 shows the 95% confidence intervals for all comprehensive elementary programs.
The final score represents the sum of the weighted scaled averages for each scale.
See Section 4.9 Standard Error Calculations for additional detail.
Table 3. Confidence interval values for all core/comprehensive elementary (K-5) programs reviewed.
Values
95% Confidence Interval
Final
Program Name
Score
Std Error
Min
Max
Math Connects (Elem)
0.724
0.009
0.707
0.742
Bridges in Mathematics
0.687
0.008
0.671
0.704
Investigations
0.635
0.009
0.617
0.654
Math Expressions
0.621
0.010
0.602
0.640
Everyday Mathematics
0.593
0.010
0.574
0.612
Saxon Math (Elem)
0.581
0.010
0.562
0.600
Growing with Mathematics
0.575
0.009
0.557
0.593
envision
0.568
0.002
0.564
0.572
Progress in Mathematics
0.547
0.011
0.527
0.568
Math Out of the Box
0.521
0.009
0.503
0.540
Math Trailblazers
0.521
0.010
0.502
0.540
Singapore Math Standards
0.365
0.009
0.347
0.382
Grand Total
0.578
0.003
0.573
0.584
2008 Mathematics Instructional Materials Review Draft Report
Page 16
Elementary Composite Scores with 95% Confidence Intervals
0.8
0.75
0.7
0.65
0.6
0.55
0.5
0.45
0.4
0.35
0.3
Figure 1. 95% confidence intervals for core/comprehensive elementary programs.
The middle school results are presented below.
Table 4. Confidence interval values for all core/comprehensive middle school (6-8) programs reviewed.
Values
95% Confidence Interval
Final
Program Name
Score
Std Error
Min
Max
Holt Mathematics
0.837
0.009
0.818
0.855
Math Connects (Middle)
0.723
0.012
0.700
0.747
Prentice Hall Mathematics
0.707
0.012
0.684
0.730
Math Thematics
0.690
0.012
0.666
0.714
McDougal Littell Math Course
0.658
0.013
0.632
0.684
Connected Mathematics 2
0.625
0.014
0.598
0.652
Impact
0.605
0.015
0.577
0.634
Cognitive Tutor/Carnegie Algebra
0.592
0.014
0.564
0.620
Saxon Math Intermediate (Middle)
0.562
0.015
0.534
0.591
CPM Middle Grades Program
0.511
0.013
0.485
0.536
2008 Mathematics Instructional Materials Review Draft Report
Page 17
Values
95% Confidence Interval
Final
Program Name
Score
Std Error
Min
Max
Everyday Math/Transition
0.419
0.012
0.396
0.443
Mathscape
0.416
0.014
0.390
0.443
ALEKS, Online Mathematics Solution
0.392
0.014
0.364
0.420
Grand Total
0.594
0.004
0.586
0.602
Middle School Composite Scores with 95% Confidence Intervals
0.9
0.8
0.7
0.6
0.5
0.4
0.3
Figure 2. 95% confidence intervals for core/comprehensive middle school programs.
Figure 1 and Figure 2 display the program final composite scores and their confidence
intervals. Programs with overlapping confidence intervals should be considered as not
being significantly different. Programs with non-overlapping confidence intervals can
generally be considered to be statistically different in their ratings. However, when
multiple tests are performed and we adjust for multiple comparisons, some nonoverlapping intervals may be found to be not statistically different. Thus, the visual chart
provides a quick check, but readers should rely on the specific test outcomes to determine
statistical significance.
2008 Mathematics Instructional Materials Review Draft Report
Page 18
1.5.2 Online Availability
One of the further requirements of HB 2598 was for at least one of the recommended
curricula at each level to be available online. As part of finalizing this report and
determining the recommendations, we checked the online availability of the four topranked programs at both the elementary and middle school levels. We specifically
inquired as to the online availability of the materials that were reviewed in hard copy for
their alignment as part of our review. The online availability of instructional materials
typically takes the form of access by teachers, students, and parents to a PDF version of
the applicable materials.
For K-5, three of the four programs were fully available online, with student and teacher
editions and supplemental material available. Bridges in Mathematics was the only
program that did not have student and teacher editions of the text available online,
although it had supplemental material online.
All four of the top-ranked middle school programs were fully available online.
Districts typically negotiate costs of licenses to access the online materials during the
purchasing process. Most of the licenses were for a renewable six year period, and
offered seats based upon the number of student textbooks purchased. Once purchased,
most products had significant flexibility in assigning access rights to the online material.
The table below denotes detail about the materials available online for each of the top
ranked elementary and middle school programs. Please note that while supplemental
materials may be available, OSPI may not have reviewed them for their alignment with
the revised standards.
Table 5. Online availability for four top-ranked elementary programs.
Program Name
Math Connects
(Elem)
Bridges in
Mathematics
Core Materials
Available Online
Investigations
Yes. Available Fall
2008.
No. There are no
plans in place to
make the TE/SE
available online
Yes
Math Expressions
Yes
Supplemental
Materials
Available Online
Yes
www.macmillanmh.com
Yes
www.gotomlc.org/wa
Yes
www.pearsonsuccess.net
Yes
www.eduplace.com/eservices
2008 Mathematics Instructional Materials Review Draft Report
URL
Page 19
Table 6.. Online availability for four top-ranked middle school programs.
Supplemental
Materials
Available Online
Yes
Program Name
Core Materials
Available Online
Holt Mathematics
Yes
Math Connects
(Middle)
Prentice Hall
Mathematics
Math Thematics
Yes. Available Fall
2008.
Yes
Yes
Yes
www.glencoe.com (browse to WA
state, then mathematics)
www.pearsonsuccess.net
Yes
Yes
www.classzone.com
URL
http://go.hrw.com
1.5.3 Comments
Reviewers had the opportunity to provide optional comments on each of the programs
they reviewed. Their comments are included in a separate companion document,
available on the OSPI web site.
Many individuals commented on the preliminary draft report. A summary of the most
common comments and responses are presented below.
Comment
Response
Will districts be required No. These are recommendations only. Districts are free to
to adopt these materials? select any program they feel best meets the needs of their
students. Districts may find this report particularly helpful,
along with the accompanying data set as they make their
curriculum decisions. The State Board of Education is
considering a proposal that would mandate use of one of the
recommended programs if the district is consistently failing to
meet expectations.
There are other ways to
We agree that there are many methods that could have been
analyze the data. Why
used to analyze the data. Prior to collecting data, during the
didn’t you use method
design of the process, we considered several possibilities and
______?
selected t-tests with multiple comparisons for our primary test
statistic. Post hoc changes in methodology are risky; and lead
to concerns that the analyst is seeking specific results. Thus,
we continued to present results with our planned analysis
approach.
What happens if some
The legislation mandates that OSPI select no more than three
programs are tied with
programs at each level. Thus, if there are ties, OSPI must still
the top three?
select no more than three. We will note in the report where
ties exist.
My district is using
OSPI will be providing a report on available supplemental
program ______, which material and how well the materials align to state standards. In
is not in the top three.
addition there are several tables and charts that show how
2008 Mathematics Instructional Materials Review Draft Report
Page 20
Comment
What will OSPI do to
help us out?
I think OSPI should pick
a good traditional,
balance, and reform
program, not necessarily
the top three.
Will the state be funding
textbook purchases,
based on these results?
I believe some standards
are more important than
others, why are they all
weighted the same?
Why are programs that
address content a grade
early given the same
score as when they meet
a standard a grade later
than expected?
There is some concern
about program
placement in the rank
order, where individuals
thought a program
should have appeared
higher or lower than it
did.
Response
each program performs, for specific Performance
Expectations and mathematics Core Content within the
standards. This information will help districts identify areas
where supplementation is needed in existing programs.
The legislation asked OSPI to identify which materials most
closely align with state standards. OSPI took a neutral stance
on the type of program, preferring to let the data drive the
process. There is no clear agreement on which programs fall
into each category; most have evolved over time to use a
blend of pedagogies.
At this point, there is no funding identified for textbook
purchases based on these results.
Most individuals feel that some standards are more important
than others. However, there is no agreement among
stakeholder groups about which are the most important. OSPI
elected to take a neutral stance, and weigh all the standards
the same for the purposes of collecting and analyzing the data.
The legislation directed OSPI to measure alignment of the
material to WA state standards. The standards have been
carefully developed for each grade level based upon
significant analysis from many experts. There is a risk in
weighing standards that are met early, in that they may not be
developmentally appropriate for typical children at that age.
Instead, OSPI took a neutral position and gave partial credit
for a standard that was fully met at +/- one grade level from
the target grade. The rationale for partial credit was that it was
reasonable for an educator to select material from a textbook
series one grade above or below the target grade.
There were 400 instances where a scored grade dip occurred,
representing about 3% of the content/standards alignment data
collected. The content was found in the grade below the target
192 times, and 208 times in the subsequent grade. Singapore
had the vast majority of grade dips from the prior grade (103
out of 839 total data points), indicating a possible difference
in degree of rigor compared to other programs.
It should be noted that the vast majority of the reviewed
programs had a very reasonable correlation to the newly
revised state standards and the other factors measured. Each
program-grade had four independent reads (in 98% of the
cases). Overall, the scores are good, and just because a
program falls in the middle of the pack doesn’t mean it isn’t a
viable choice, depending upon the district’s needs. Most states
have a textbook evaluation process that sets a basic threshold
2008 Mathematics Instructional Materials Review Draft Report
Page 21
Comment
Response
and all programs that meet or exceed that basic threshold can
be considered for purchase. Washington state is unique in
providing no more than three recommendations. If this review
had been conducted in a more traditional manner, almost
every single program would likely be in the pool of approved
materials.
1.6 Recommendations
This report contains the initial recommendations by OSPI for no more than three
programs at the elementary (K-5) and middle school (6-8) grade spans. The State Board
of Education (SBE) has two months to provide comments about the recommendations.
OSPI will issue final recommendations in mid-November, following the comment period
by SBE at their meeting in early November.
1.6.1 Elementary School (K-5) Recommendations
Based upon a careful review of the results, OSPI recommends two programs, Math
Connects(elementary) and Bridges in Mathematics at the elementary school level. Both
of these programs met the initially proposed content threshold, and perform well on all of
the factors measured. Math Connects is fully available online, while Bridges in
Mathematics has only supplemental material available online.
While Math Connects was the top ranked elementary school program, it should be noted
that it is brand new and is currently untested in Washington State. It was developed after
the NCTM Curriculum Focal Points were produced in 2007. The relative newness of the
program and its strong alignment to the Focal Points may explain its strong performance
in the OSPI review, given that the revised Washington standards align closely with the
NCTM Focal Points. Despite the newness and current limited use in Washington, it
should be noted that ratings across all scales were consistently high, and the program
provides a good balance of conceptual and skills development.
It is a brand new program, untested in Washington State. It was developed after the
NCTM Curriculum Focal Points were produced, and that may explain how well it aligns
to state standards, given that our standards drew heavily from the curriculum focal points.
However, ratings across all scales were consistently high, and the program provides a
good balance of conceptual and skills development.
While the third and fourth ranked programs (Investigations and Math Expressions) are
statistically tied, neither met the content threshold level (a score of 0.70 or greater) for
consideration as a recommended program, and as a result are not included in the initial
recommendations. Nonetheless, both programs are viable choices, and many districts
have had success with both of them. Investigations and Math Expressions are fully
available online.
2008 Mathematics Instructional Materials Review Draft Report
Page 22
1.6.2 Middle School (6-8) Recommendations
OSPI recommends two programs, Holt Mathematics and Math Connects (middle) at the
middle school level.
In middle school, Holt Mathematics clearly stands out. It performed consistently well
across all core content areas and other factors.
Math Connects is a brand new program, published in 2008, after the NCTM Curriculum
Focal Points were produced in 2007. Washington standards drew heavily from that work,
and that may influence why it had such a high content alignment score. OSPI selected
Math Connects among the ties for second rank because it sees significant value in having
a product that spans grades K-8 in the set of recommended programs.
Both programs are fully available online, and include additional supplemental material as
well.
It should be noted that Math Connects, Prentice Hall Mathematics and Math Thematics
are statistically tied for second, third and fourth rank, respectively, based upon final
composite scores. As stated above, Math Connects was selected as the recommendation
due to its continuity over the elementary and middle school grade spans. All three of
these programs are viable choices for school districts to consider, and all met the
minimum content threshold established as part of the review design process. Prentice
Hall Mathematics and Math Thematics are also fully available online.
2008 Mathematics Instructional Materials Review Draft Report
Page 23
2 Project Process
2.1 Review Instrument Development
This section describes the process by which the review instrument and weights were
developed. It also includes the scoring rubric for Content/Standards Alignment and Other
Factors.
To develop the review instruments, OSPI engaged two groups in two full cycles of
development and revision. The IMR Advisory Group and SBE Math Panel were the two
primary groups contributing to the development of the instruments. Their work was
research based, and used the following primary sources:
2008 Washington Revised Math Standards
NMAP Foundations for Success
NCTM Curriculum Focal Points
Additionally, the groups also referenced the following secondary sources as resources.
Please note that in some instances, the secondary sources were used to compare and
contrast effective and ineffective instrument design.
Math Educators’ Summary of Effective Programs
Park City Mathematics Standards Study Group Report
Framework for 21st Century Learning
How People Learn: Brain, Mind, Experience and School
How Students Learn: Mathematics in the Classroom
NCTM Principles and Standards for School Mathematics
Choosing a Standards-Based Mathematics Curriculum – Chapter 6: Developing
and Applying Selection Criteria
Choosing a Standards-Based Mathematics Curriculum – Appendix: Sample
Selection Criteria
In addition to seeking advice and guidance from the IMR Advisory Group and the SBE
Math Panel, several national and/or external experts were consulted and provided
important recommendations for both the process and the review instruments. Several of
the external experts provided valuable advice about their state processes where they have
successfully completed comprehensive mathematics curriculum reviews.
The outcomes from the review instrument design phase included:
Two review instruments (Content/Standards Alignment and Other Factors), which
are described below
Proposed threshold and weighting process for final recommendations. Both
groups recommended that in order for programs to be considered for the final
three recommendations, they must first meet a minimum threshold in
content/standards alignment. A scaled score of 0.70 was proposed as this
2008 Mathematics Instructional Materials Review Draft Report
Page 24
threshold with a recommendation that the threshold be adjusted if necessary if a
sufficient number of materials failed to reach the threshold. In addition, both
groups proposed weighting percentages for the Other Factors.
2.1.1 Content/Standards Alignment Threshold
Part 1 of the review measured the alignment of the core/comprehensive instructional
materials to the revised 2008 K-8 Washington Mathematics standards. Materials that met
a minimum threshold of alignment with state standards were considered for inclusion in
the list of recommended mathematics curricula.
Reviewers looked for evidence that each Washington state standard Core Process,
Content, and Additional Key Information was met in the expected grade level.
An additional goal of the Content/Standards Alignment evaluation was to identify the
areas where existing materials need supplementation to meet state standards. See sections
3.1.1 and 3.2.1 for charts that show how well each program meets specific Performance
Expectations at each grade level.
2.1.2 Scale Definitions
Scale
Content/Standards
Alignment
Program Organization and
Design
Balance of Student
Experience
Assessment
Description
The Content/Standards Alignment (Part 1 of the
review process) determined to what degree the
mathematical concepts, skills and processes were in
alignment with revised state mathematics standards.
The materials reviewed were accurate, with no errors
of fact or interpretation. Adherence to standards
implies quality and rigor. It is a fundamental
assumption that if the program matches a standard
well, the math is accurate, rigorous, and high quality.
Overall program and design. Includes scope and
sequence, appropriate use of technology. Content is
presented in strands, with definitive beginnings and
endings. The program grounds ideas in a bigger
framework. The material is logically organized, and
includes text-based tools like tables of contents and
indexes.
Tasks lead to the development of core content and
process understanding. They present opportunities for
students to think about their thinking, develop both
skills and understanding, and apply multiple strategies
to solve real world problems. Tasks will provide a
balance of activities to develop computational fluency
and number sense, problem solving skills and
conceptual understanding.
Tools for teachers and students to formally and
informally evaluate learning and guide instruction.
2008 Mathematics Instructional Materials Review Draft Report
Page 25
Scale
Instructional Planning and
Professional Support
Description
Support for teachers that is embedded in the
instructional materials to assist them in teaching the
content and standards. Instructional materials provide
suggestions for teachers in initiating and orchestrating
mathematical discourse. Includes key information
about content knowledge to help teachers understand
the underlying mathematics. Materials help surface
typical student misconceptions and provide ideas for
helping address them.
Support for ELL, unbiased materials, support for
gifted and talented students, support for students with
disabilities, differentiated instruction, diversity of role
models, parent involvement, intervention strategies,
quality website, and community involvement ideas.
Equity and Access
Assessment, 5.0%
Equity and Access, 4.0%
Instructional Planning &
Professional Support,
4.5%
Student Experience,
7.5%
Program Organization &
Design, 9.0%
Content/Standards
Alignment, 70.0%
Figure 3. Category weights for the Mathematics Instructional Materials Review. Note that
Content/Standards Alignment is both a weighted category and a threshold category, meaning that
curricula must meet a minimum average score on content/standards alignment before the material can
be considered for possible inclusion in the three recommended core/comprehensive curricula.
2008 Mathematics Instructional Materials Review Draft Report
Page 26
Table 7. Measurement scales and weights for/Content Standards Alignment and Other Factors.
Scale
Content/Standards Alignment
Program Organization and Design
Balance of Student Experience
Assessment
Instructional Planning and
Professional Support
Equity and Access
Scale
Weight
70.0%
9.0%
7.5%
5.0%
4.5%
4.0%
2.1.3 Measurement Criteria
Part 1: Content/Standards Alignment criteria measured how well the Washington state
K-8 revised mathematics standards were addressed within the materials submitted for
review. Reviewers ensured that the mathematics content within the program was rigorous
and accurate, with few errors of fact or interpretation. In scoring Part 1, reviewers used a
3 point scale (corresponding with No, Partial, Yes) for each performance expectation.
This scale uses interval data to represent ordinal data (No, Partial, Yes). The criteria are
the Washington Revised Mathematics Standards (4/08). Each raw score was adjusted to a
scaled score of [0, 0.5, 1] for this tool.
2008 Mathematics Instructional Materials Review Draft Report
Page 27
A sample rating form for Part 1 is shown below. Note that the raw scores were adjusted
to a range of [0, 0.5, 1] for analysis and display.
Figure 4. Sample rating form for Content/Standards Alignment Review.
Reviewers used the following rubric to evaluate and score the Content/Standards
Alignment worksheets that were completed by each publisher. During the review week,
we posted variance reports that showed the rare instances where two or more independent
reviewers had selected “Not Met” and “Fully Met” on a particular Performance
Expectation for a specified program. With clear scoring guidelines this type of variance
should not occur; although in the process of collecting 36,000+ data elements some
anomalies are expected. In practical terms, if one reviewer selected “Not Met” on a
performance expectation for a specific program and another reviewer selected “Fully
Met”, there are some possible reasons, including that the initial reviewer might have
missed the evidence that shows the performance expectation was fully met. In each case
of a variance gap, the discrepancy was highlighted, and reviewers were asked to go back
and check their work and/or discuss the differences among each other to understand the
reason for the difference. They were given the opportunity to correct their scores or to
leave them as-is.
2008 Mathematics Instructional Materials Review Draft Report
Page 28
Table 8. Scoring rubric for Content/Standards Alignment instrument.
Not Met
•
The standard is not
covered at grade level
or +/- one grade level.
•
The standard is briefly
mentioned, but a
typical student would
not be able to achieve
mastery with the
available content.
•
The standard is
partially met at +/one grade level.
Partially Met
•
The standard is fully
developed at +/- one
grade level from the
expected grade.
•
50% or more of the
standard is met at
grade level, but some
aspect of the standard
is not present. The
standard is fully
developed, but limited
in practice or
reinforcement.
Fully Met
•
The standard is fully
developed at the
expected grade level.
A typical student
would be able to
achieve mastery with
the available content.
We collected additional grade level data when the reviewer indicated that the standard
was fully met at +/- one grade level from the expected grade. This allows us to
differentiate whether “Partially Met” was due to a grade level dip (rigor) or complete
development of the standard (comprehensiveness).
Part 2: Other Factors contributed 30% of the final composite score for each program.
There were five scales, with 6-10 elements per scale. In scoring Part 2, reviewers used a
consistent, 4-point Likert measurement scale measurement scale for each item (strongly
disagree, disagree, agree, strongly agree). A sample instrument form is shown below.
2008 Mathematics Instructional Materials Review Draft Report
Page 29
Figure 5. Other Factors sample instrument form.
In addition, for each Part 2 category (described above in the Scale Definitions section),
stakeholders identified 6-10 criteria, which are shown below.
Program Organization and Design
1. The content has a coherent and well-developed sequence (organized to promote
student learning, links facts and concepts in a way that supports retrieval, builds
from & extends concepts previously developed, strongly connects concepts to
overarching framework)
2. Program includes a balance of skill-building, conceptual understanding, and
application
3. Tasks are varied: some have one correct and verifiable answer; some are of an
open nature with multiple solutions
4. The materials help promote classroom discourse
5. The program is organized into units, modules or other structure so that students
have sufficient time to develop in-depth major mathematical ideas
6. The instructional materials provide for the use of technology with reflects 21st
century ideals for a future-ready student
7. Instructional materials include mathematically accurate and complete indexes and
tables of contents to locate specific topics or lessons
8. The materials have pictures that match the text in close proximity, with few
unrelated images
9. Materials are concise and balance contextual learning with brevity
2008 Mathematics Instructional Materials Review Draft Report
Page 30
10. Content is developed for conceptual understanding: (limited number of key
concepts, in-depth development at appropriate age level)
Balance of Student Experience
1. Tasks2 lead to conceptual development of core content, procedural fluency, and
core processes abilities including solving non-routine problems
2. Tasks build upon prior knowledge
3. Tasks lead to problem solving for abstract, real-world and non-routine problems
4. Tasks encourage students to think about their own thinking3
5. The program provides opportunities to develop students’ computational fluency
using brain power without use of calculators
6. Tasks occasionally use technology to deal with messier numbers or help the
students see the math with graphical displays
7. The program promotes understanding and fluency in number sense and operations
8. The program leads students to mastery of rigorous multiple-step word problems
9. The materials develop students’ use of standard mathematics
terminology/vocabulary
10. Objectives are written for students
Instructional Planning and Professional Support
1. The instructional materials provide suggestions to teachers on how to help
students access prior learning as a foundation for further math learning
2. The instructional materials provide suggestions to teachers on how to help
students learn to conjecture, reason, generalize and solve problems
3. The instructional materials provide suggestions to teachers on how to help
students connect mathematics ideas and applications to other math topics, other
disciplines and real world context
4. Background mathematics information is included so that the concept is explicit in
the teacher guide
5. Instructional materials help teachers anticipate and surface common student
misconceptions in the moment
6. The materials support a balanced methodology
7. Math concepts are addressed in a context-rich setting (giving examples in context,
for instance)
8. Teacher’s guides are clear and concise with easy to understand instructions
Assessment
1.
2.
3.
4.
The program provides regular assessments to guide student learning
There are opportunities for student self-assessment of learning
Assessments reflect content, procedural, and process goals and objectives
The program includes assessments with multiple purposes (formative, summative
and diagnostic)
2
Tasks can include homework, lessons, in-class group or individual activities, assessments, etc.
Students are expected to be able to analyze their thinking process to understand how they came to a
conclusion.
3
2008 Mathematics Instructional Materials Review Draft Report
Page 31
5. Assessments include multiple choice, short answer and extended response
formats.
6. Recommended rubrics or scoring guidelines accurately reflect learning objectives
7. Recommended rubrics or scoring guidelines identify possible student responses
both correct & incorrect
8. Accurate answer keys are provided
Equity and Access
1. The program provides methods and materials for differentiating instruction
(students with disabilities, gifted/talented, English Language Learners (ELL),
disadvantaged)
2. Materials support intervention strategies
3. Materials, including assessments are unbiased and relevant to diverse cultures
4. Materials are available in a variety of languages
5. The program includes easily accessible materials which help families to become
active participants in their students’ math education (e.g. “How You Can Help at
Home” letters with explanations, key ideas & vocabulary for each unit, free or
inexpensive activities which can be done at home, ideas for community
involvement4)
6. The program includes guidance and examples to allow students with little home
support to be self-sufficient and successful
2.2 Reviewer Selection Process
OSPI issued a statewide invitation to solicit applications from individuals interested in
serving as mathematics Professional Development Facilitators (trainers on the revised
standards) and/or to participate as Instructional Materials Reviewers Committee
members. Over 400 applications were received for both roles. Using a common review
instrument and criteria, a committee reviewed and scored the over 100 applications for
the instructional materials review and selected 42 individuals. The IMR Committee was
selected first based on the score of their application (primarily based on experience).
Next, it was important to have a balanced number of reviewers qualified to review
primary, intermediate and middle school levels. In addition, OSPI sought balance on the
review team, ensuring that math educators, curriculum specialists, parents, advocacy
group members, mathematicians and math coaches were represented in the final group.
Parent recommendations were solicited from the Washington State Parent Teacher
Association and Where’s the Math.
2.3 Publisher Involvement
All publishers were invited to submit core/comprehensive K-8 mathematics material for
review. The materials did not have to be in widespread use in Washington in order to be
considered. Information about the review was disseminated widely by the Washington
Oregon Alaska Textbook Representatives Association (WOATRA), the American
Association of Publisher (AAP) and available on the OSPI Publisher Notice web site.
4
Community involvement means ideas where students can apply math concepts they are learning in the
context of business, environment or public service for example.
2008 Mathematics Instructional Materials Review Draft Report
Page 32
In addition, OSPI hosted a Publisher’s Meeting to address questions prior to the review.
As a result, OSPI maintained a web-based Question and Answer document for the
publishers, so they had up-to-date information regarding the submission and review
process.
In addition to providing curricular materials for review, publishers were asked to review
their materials and compare them to the 2008 WA Revised Mathematics Standards. For
each program submitted for review, publishers completed a Program Alignment
Worksheet that provided between one and five references to locations in their materials
where the standard was presented.
Publishers also submitted a Professional Development plan that outlined what standard
professional development was available with the purchase of materials, and the optimal,
recommended amount and type of professional development.
Publishers delivered materials to the review site the day before the review. They were
escorted into the library repository, and participated in an inventory check with OSPI
staff. After the review week was completed, they collected their material. Publishers did
not meet with or present to the IMR Review Committee.
2.4 Review Week Process
The K-8 core/comprehensive mathematics review week took place in Spokane,
Washington from June 22-27, 2008.
On Sunday, June 21, the review team participated in an eight-hour mathematics standards
training, led by Dr. George Bright from OSPI. The purpose of this training was to
familiarize the reviewers with the standards at the grade levels they would be reviewing.
Dr. Bright provided clarity on the meaning of each standard, and example evidence that
shows how the standard could be developed in instructional materials.
Reviewers participated in another four-hour training on Monday morning that focused on
the review instruments (Content/Standards Alignment and Other Factors), how to score
the elements, and expectations for reviewers, such as independent assessments, bias-free
professional judgments, consistent scoring and productivity expectations.
Between Monday afternoon and Friday morning, reviewers read and evaluated all
materials submitted. They checked out programs (and ancillary materials, if submitted)
from the library, and spent on average about 3.5 hours per program-grade evaluating and
scoring the material.
Staff entered data from the instruments in near real-time. Twice per day, the group
gathered for progress updates, variance checks and process improvement changes.
The initial expectation was for each program-grade to receive three independent reviews..
However, the reviewers ended up working both before and after the standard day (The
2008 Mathematics Instructional Materials Review Draft Report
Page 33
review room was open between 6 a.m. and 9 p.m. daily) and were able to complete four
reviews per program-grade for over 98% of instructional materials reviewed.
2.5 Data Analysis Process/Methodology
The purpose of this section is to describe in easy to understand terms how the data were
analyzed. For example, it describes the process by which programs met a threshold level
and how the comprehensive score is calculated (with weights).
Professional data entry staff entered the data into an Access database in near real-time.
Once the review week was complete, we extracted the scores into a flat-file Excel
worksheet for graphics publication and also text file format for statistical processing
using the statistical package R.
Two statisticians worked independently with the data, first doing exploratory data
analysis, looking for any anomalies or outliers (like a score value of 11, when the max
score value should have been a 1). The statisticians checked counts of data, ranges,
distributions and variance, as examples. No entry or extract errors were apparent, which
was expected given the input constraints on the data entry application.
Some data cleaning and recoding ensued. Several program names were shortened or
clarified to prepare the data for final graphic presentation. Some sets of instructional
materials series had different names for different grade levels; a succinct composite name
replaced the individual names. For example, Wright Group/McGraw-Hill submitted
Everyday Mathematics 6, UCSMP Pre-Transition and UCSMP Transition to Algebra as a
comprehensive middle school program. For readability and visual presentation, the series
was renamed “Everyday Math/Transition”.
The data for the Other Factors scales had an original range of [1,4] and the
Content/Standards Alignment scale had a range of [0,2]. Before scaling the data and
converting it to a common [0,1] range, the Other Factors range was adjusted to [0,3]. This
was done to prevent an inflation of the Other Factors after the data was adjusted. (If a
range of [1,4] is divided by 4, it becomes [0.25,1], which cannot be directly compared to
the scaled content score at [0,1].)
After exploratory data analysis (EDA) and the data cleaning/recoding were completed,
we re-checked the accuracy of the data elements by randomly sampling10% of the
original data entry forms and comparing them to the values in the electronic data set.
There were no coding errors discovered during the EDA and cleaning process.
The final composite score was calculated by multiplying the scaled average values by the
scale weights and summing the values. Confidence intervals were set at 95% and
calculated for each instructional materials series.
One important consideration in ranking the data is to identify where statistical ties might
occur. The tables and graphs that show confidence intervals for each instructional
materials series are critical for understanding that small differences in composite scores
2008 Mathematics Instructional Materials Review Draft Report
Page 34
may be due to sampling or other error (including measurement error) rather than a true
difference in means.
The most critical statistical tie in the ranked list of composite scores involves the
recommended programs and subsequent lower ranked instructional materials series. For
example, if the third, fourth and fifth ranked series are statistical ties, then the simple
ranking is not sufficient justification alone to select and recommend the third ranked set
of instructional materials.
To test for statistical ties, we used a one-tailed t-test and accounted for multiple tests.
Prior to collecting the data, the statistical team considered several statistical tests, and
decided to use the one-tailed t-test for three reasons: 1) the expected number of data
elements, the expected distribution of the averages and the data type (ordinal converted to
interval) made the t-test a good fit; 2) the t-test is one of the most commonly used and
most easily understood statistical tests to use; and 3) it provides a very robust mechanism
for measuring differences of means.
We want to identify any statistical ties with the recommended curricula in each grade
band. To do so, it is sufficient to ascertain if any curriculum has a statistically equivalent
rating to the last rated program in the set of recommendations. The following example
assumes the selection of the top three ranked programs, and a comparison of the thirdranked program to lower ranked (4th, 5th, etc.) programs.
First, we perform hypothesis tests comparing the ratings of all lower ranked materials to
the third.
HO: rating 3 = rating [4…n]
HA: rating 3 > rating [4…n]
The test is a one-sided two-sample t-test. To allow for differences in the variances of the
means across materials, we used an unequal variance statistic:
Where the standard error of the difference is calculated by:
See Section 4.9 for the degrees of freedom calculations for the following tables.
Table 9 and Table 10 give the adjusted significance levels for elementary and middle
school curricula respectively, calculated using the Holm-Bonferroni method. Since we
2008 Mathematics Instructional Materials Review Draft Report
Page 35
are performing several comparisons for each grade level, we need to correct for multiple
testing. Rather than comparing each p-value to 0.05, we order the p-values from smallest
to largest and then compare them, in order, to the nominal significance level (0.05)
divided by the number of tests remaining. When we reach a p-value that is deemed
insignificant, we then say that all remaining values are also insignificant.
Table 9. t-test results comparing lower-scoring programs to the third-highest scoring elementary school
program.
Degrees
Program
Mean
t
of
score
statistic
freedom
p-value
# tests
Significance
remaining
cutoff
Math Connects (Elem)
0.724
Bridges in Mathematics
0.687
Investigations
0.635
Singapore Math Standards
0.365
-20.70
628
2.79E-73
9
0.006
Math Out of the Box
0.521
-8.55
681
4.03E-17
8
0.006
Math Trailblazers
0.521
-8.51
647
5.99E-17
7
0.007
Progress in Mathematics
0.547
-6.23
659
4.06E-10
6
0.008
enVision
0.568
-4.78
705
1.06E-06
5
0.010
Growing with Mathematics
0.575
-4.60
643
2.57E-06
4
0.013
Saxon Math (Elem)
0.581
-3.99
636
3.69E-05
3
0.017
Everyday Mathematics
0.593
-3.15
687
8.55E-04
2
0.025
0.6212
-1.06
709
0.146
1
0.050
Math Expressions
Table 10. t-test results comparing lower-scoring programs to the third-highest scoring middle school
program.
Program
Mean
t
Degrees of
score
statistic
freedom
# tests
Significance
p-value
remaining
cutoff
Holt Mathematics
0.837
Math Connects (Middle)
0.723
Prentice Hall Mathematics
0.707
Everyday Math/Transition
0.419
-16.96
305
3.55E-46
10
0.005
ALEKS, Online Mathematics Solution
0.392
-17.04
279
2.10E-45
9
0.006
Mathscape
0.416
-16.06
270
1.67E-41
8
0.006
CPM Middle Grades Program
0.511
-11.25
316
3.19E-25
7
0.007
Saxon Math Intermediate (Middle)
0.562
-7.72
291
9.31E-14
6
0.008
Cognitive Tutor/Carnegie Algebra
0.592
-6.17
322
1.04E-09
5
0.010
Impact
0.605
-5.44
308
5.53E-08
4
0.013
Connected Mathematics 2
0.625
-4.56
270
3.88E-06
3
0.017
McDougal Littell Math Course
0.658
-2.77
322
0.003
2
0.025
2008 Mathematics Instructional Materials Review Draft Report
Page 36
Program
Math Thematics
Mean
t
Degrees of
score
statistic
freedom
0.690
-1.01
330
# tests
Significance
p-value
remaining
cutoff
0.155
1
0.050
Of the elementary programs, only Math Expressions is not statistically different from the
third-ranked program, Investigations. For middle school materials, the fourth-ranked text,
Math Thematics is not statistically different from the third-highest ranked program,
Prentice Hall Mathematics. However, all remaining curricula are significantly different
from the third-highest rated program.
2008 Mathematics Instructional Materials Review Draft Report
Page 37
3 Results
3.1 K-5 Results
3.1.1 Content/Standards Alignment
The following graph shows ranked results for the content/standards alignment scale for
all comprehensive elementary programs that were reviewed.
Content/Standards Alignment
Math Connects (Elem)
Bridges in Mathematics
Investigations
Math Expressions
Saxon Math (Elem)
Everyday Mathematics
enVision
Progress in Mathematics
Growing with Mathematics
Math Out of the Box
Math Trailblazers
Singapore Math Standards
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 38
Math Expressions
Bridges in Mathematics
Everyday Mathematics
Investigations
Growing with Mathematics
Math Out of the Box
Saxon Math (Elem)
enVision
Math Trailblazers
Singapore Math Standards
Progress in Mathematics
Overall Average
PE
K.1.A
K.1.B
K.1.C
K.1.D
K.1.E
K.1.F
K.1.G
K.1.H
K.2.A
K.2.B
K.2.C
K.2.D
K.3.A
K.3.B
K.3.C
K.4.A
K.5.A
K.5.B
K.5.C
K.5.D
K.5.E
K.5.F
K.5.G
Overall
Average
Math Connects (Elem)
Table 11. Performance Expectations alignment for all submitted Kindergarten programs.
0.63
0.50
0.88
1.00
0.63
1.00
0.63
0.25
1.00
0.75
1.00
1.00
1.00
0.75
1.00
1.00
1.00
1.00
0.75
1.00
1.00
1.00
1.00
1.00
0.88
1.00
1.00
0.75
1.00
0.25
1.00
1.00
0.88
0.88
1.00
1.00
0.75
0.63
0.50
0.75
0.75
0.50
0.75
1.00
1.00
0.75
1.00
1.00
0.63
0.25
0.63
0.75
0.88
0.88
0.88
1.00
1.00
1.00
0.75
1.00
0.88
0.88
0.88
0.88
0.50
0.88
1.00
0.75
0.50
0.88
0.88
0.63
1.00
0.63
0.63
0.75
0.38
1.00
0.75
0.75
0.75
0.75
0.75
0.13
0.88
0.50
0.63
0.38
0.63
0.75
0.63
0.50
0.25
1.00
1.00
0.75
0.75
0.88
0.63
0.75
1.00
0.25
0.63
0.50
1.00
0.75
0.38
0.38
0.50
1.00
0.25
0.88
1.00
0.88
0.13
0.88
0.75
0.38
0.88
0.75
0.63
0.25
0.25
1.00
1.00
1.00
0.88
0.88
0.88
0.63
0.88
0.50
0.50
0.13
0.13
1.00
0.63
0.50
0.38
0.38
0.38
0.25
0.25
0.25
0.00
0.25
0.75
0.88
0.88
0.88
0.88
0.88
0.75
0.75
1.00
1.00
0.38
1.00
1.00
1.00
0.25
0.50
0.50
0.25
1.00
0.38
0.38
0.38
0.00
1.00
0.75
0.75
0.63
0.75
0.63
0.38
0.88
0.38
0.38
0.25
0.25
0.63
0.38
0.25
0.50
0.38
0.50
0.13
0.63
0.63
0.13
0.38
1.00
0.50
0.75
0.63
0.75
0.50
0.50
0.63
0.38
0.38
0.25
0.00
0.75
0.50
0.38
0.13
0.13
0.50
0.38
0.63
0.38
0.00
0.50
0.88
0.75
0.25
0.50
0.38
0.50
0.88
0.75
0.25
0.38
0.13
0.63
1.00
0.75
0.38
0.50
0.50
0.40
0.90
0.60
0.80
0.10
0.00
0.50
0.20
0.70
0.50
0.70
0.40
0.10
1.00
0.40
0.40
0.00
0.20
0.90
0.30
0.10
0.38
0.88
0.13
0.38
0.63
0.38
0.38
0.00
0.75
0.25
0.75
0.13
0.63
0.63
0.63
0.75
0.25
0.38
0.00
0.25
0.75
0.13
0.00
0.58
0.64
0.55
0.66
0.60
0.64
0.36
0.38
0.89
0.65
0.78
0.69
0.79
0.69
0.56
0.78
0.56
0.63
0.29
0.54
0.90
0.65
0.39
0.86
0.83
0.82
0.67
0.67
0.66
0.63
0.51
0.48
0.48
0.44
0.41
0.62
Table 11 shows the degree in which the kindergarten materials reviewed meet each K-8
Performance Expectation. The dashboard view shows a green up arrow if the scaled
average score from the four reviewers is ≥ .6255 (on a 1.0 scale); a yellow arrow if the
scale is between 0.5 and 0.6254 inclusive, and a red down arrow if the average score is
below 0.5.
The programs are listed in rank order from left to right based on the average score across
all kindergarten performance expectations. For example, Math Connects, with an overall
average kindergarten rating on content/standards alignment of 0.86 is shown first.
There are a couple of key conjectures that could be drawn from this chart. The standards
are organized into sections, (K.1.A through K.1.D for example). Some programs are very
strong in some sections while weak across other sections. See for instance, Math Out of
the Box, which performs well in K.2 Patterns and Operations, K.3 Objects and their
Locations and K.4 Additional Key Content, but is very weak in K.1 Whole Numbers
(Numbers, Operations). Thus, it may be that certain instructional materials need to be
heavily supplemented in some key content areas. It might also be noted that some areas
are easier to supplement than others. For example, given the large volume of
2008 Mathematics Instructional Materials Review Draft Report
Page 39
computational fluency programs available, it might be easier to supplement numbers and
operations than reasoning and problem solving.
Additionally, the far right column shows how all programs performed overall for each
specific performance expectation. For example, standard K.5.E (Answer the question(s)
asked in the problem) is well covered in all reviewed programs, but standard K.5.C
(Recognize when additional information is required to solve a problem.) is well covered
in only one program. This data may provide valuable feedback in understanding which of
the revised math standards may need supplementation to support a majority of the
students in the state.
The following tables show detailed results for grades 1-5, similarly organized as the
kindergarten table.
2008 Mathematics Instructional Materials Review Draft Report
Page 40
Investigations
Math Out of the Box
Growing with Mathematics
Bridges in Mathematics
Everyday Mathematics
Math Connects (Elem)
Math Trailblazers
Math Expressions
enVision
Progress in Mathematics
Singapore Math Standards
Overall Average
PE
1.1.A
1.1.B
1.1.C
1.1.D
1.1.E
1.1.F
1.1.G
1.1.H
1.1.I
1.2.A
1.2.B
1.2.C
1.2.D
1.2.E
1.2.F
1.2.G
1.2.H
1.2.I
1.3.A
1.3.B
1.3.C
1.4.A
1.4.B
1.4.C
1.4.D
1.4.E
1.4.F
1.5.A
1.5.B
1.6.A
1.6.B
1.6.C
1.6.D
1.6.E
1.6.F
1.6.G
1.6.H
Overall
Saxon Math (Elem)
Table 12. Performance Expectations alignment for all submitted 1st grade programs.
0.88
0.75
0.63
1.00
0.88
0.63
0.63
0.75
1.00
0.88
0.88
0.25
0.88
0.63
0.75
0.75
0.88
0.75
0.63
0.75
0.75
0.75
1.00
0.00
1.00
0.88
1.00
1.00
0.88
0.88
0.88
0.63
0.75
0.88
0.88
0.88
0.75
0.78
0.75
0.75
0.13
0.13
0.75
1.00
0.63
1.00
0.38
1.00
0.88
0.75
0.75
0.63
0.88
0.75
0.88
1.00
0.88
0.88
1.00
0.63
1.00
0.38
0.00
0.88
1.00
1.00
0.75
0.75
1.00
0.13
1.00
1.00
1.00
1.00
0.63
0.75
0.50
0.38
0.25
0.88
0.63
0.75
0.38
0.88
0.88
1.00
0.75
1.00
0.88
0.00
1.00
0.88
0.88
0.63
1.00
0.88
0.50
0.50
1.00
0.38
1.00
0.75
1.00
0.88
0.75
0.88
0.88
0.75
0.75
0.88
0.88
0.88
0.50
0.74
0.40
0.50
0.20
0.60
0.50
1.00
0.80
0.80
0.60
1.00
0.70
0.20
0.90
0.50
0.90
0.70
0.80
0.90
0.80
0.90
0.90
0.70
1.00
0.10
0.90
0.90
0.60
1.00
0.70
0.70
0.70
0.50
0.80
0.90
0.90
0.80
0.70
0.72
0.75
0.63
0.38
0.50
0.50
0.63
0.63
0.75
0.75
1.00
0.75
0.75
0.38
0.38
0.75
0.63
0.50
0.63
0.63
0.50
1.00
0.50
0.88
0.50
0.88
0.63
1.00
0.38
0.88
0.75
0.88
0.63
0.75
0.88
0.75
0.88
0.50
0.68
0.75
0.88
0.75
0.50
0.75
0.88
0.75
0.88
1.00
0.75
0.88
0.75
0.88
0.13
0.63
0.88
0.75
0.88
0.63
0.75
0.50
0.50
0.63
0.25
0.38
0.50
0.88
0.50
0.63
0.50
0.50
0.50
0.88
0.75
0.75
0.75
0.63
0.68
0.50
0.13
0.25
0.50
0.38
0.75
0.63
0.75
0.50
0.63
0.63
0.75
0.75
0.13
0.75
0.88
0.75
1.00
0.75
0.63
0.63
0.38
0.88
0.13
0.50
0.13
0.50
0.88
0.88
1.00
1.00
0.63
0.88
1.00
0.75
1.00
1.00
0.65
0.67
0.83
0.17
0.17
0.50
0.83
0.83
1.00
0.67
1.00
0.50
0.17
0.50
0.33
0.50
0.67
0.83
0.67
0.50
0.67
0.67
0.33
0.50
0.00
0.33
0.50
0.33
1.00
1.00
0.83
0.83
0.50
0.17
0.83
0.67
0.50
0.33
0.58
0.75
0.75
0.00
0.50
0.38
1.00
0.38
0.63
0.50
1.00
0.50
0.13
0.38
0.63
0.50
0.38
0.88
0.50
0.38
0.75
0.38
0.25
0.38
0.00
0.38
0.13
1.00
0.63
0.75
0.50
0.88
0.63
0.88
1.00
0.75
0.88
0.88
0.57
0.50
0.63
0.00
0.88
0.50
0.88
1.00
0.38
1.00
0.88
0.38
0.13
1.00
0.63
0.75
0.75
0.25
0.88
0.88
0.88
1.00
0.38
0.63
0.25
0.25
0.25
0.25
0.25
0.75
0.63
0.38
0.13
0.50
0.50
0.38
0.38
0.38
0.55
0.50
0.13
0.13
0.38
0.25
0.38
0.13
0.25
0.38
0.75
0.50
0.63
0.63
0.50
0.75
0.88
0.50
0.50
0.13
0.00
0.00
0.13
0.13
0.00
0.13
0.13
0.13
0.63
0.38
0.63
0.63
0.63
0.63
0.88
0.75
0.63
0.38
0.41
0.38
0.38
0.00
0.50
0.38
0.75
0.38
0.25
0.00
0.88
0.75
0.13
0.63
0.50
0.63
0.75
0.63
0.38
0.38
0.13
0.25
0.50
0.63
0.13
0.63
0.13
0.13
0.63
0.63
0.13
0.13
0.00
0.00
0.13
0.00
0.00
0.00
0.34
0.60
0.55
0.24
0.55
0.53
0.79
0.59
0.69
0.64
0.90
0.68
0.47
0.72
0.42
0.74
0.74
0.71
0.73
0.64
0.65
0.64
0.47
0.73
0.18
0.54
0.49
0.66
0.73
0.74
0.68
0.72
0.47
0.68
0.80
0.71
0.72
0.56
0.62
2008 Mathematics Instructional Materials Review Draft Report
Page 41
Growing with Mathematics
Math Connects (Elem)
Everyday Mathematics
Progress in Mathematics
Math Trailblazers
Math Expressions
Saxon Math (Elem)
Math Out of the Box
Investigations
Singapore Math Standards
enVision
Overall Average
PE
2.1.A
2.1.B
2.1.C
2.1.D
2.1.E
2.1.F
2.2.A
2.2.B
2.2.C
2.2.D
2.2.E
2.2.F
2.2.G
2.2.H
2.2.I
2.3.A
2.3.B
2.3.C
2.3.D
2.3.E
2.4.A
2.4.B
2.4.C
2.4.D
2.4.E
2.5.A
2.5.B
2.5.C
2.5.D
2.5.E
2.5.F
2.5.G
Overall
Average
Bridges in Mathematics
Table 13. Performance Expectations alignment for all submitted 2nd grade programs.
1.00
1.00
0.88
0.75
0.75
0.88
1.00
0.88
0.75
0.88
0.63
0.63
0.75
1.00
1.00
1.00
1.00
0.75
0.88
0.88
0.75
1.00
0.63
0.63
0.88
1.00
1.00
0.75
1.00
0.88
1.00
0.88
0.83
1.00
1.00
0.83
0.83
1.00
1.00
0.83
0.83
0.83
0.67
0.83
1.00
0.83
1.00
1.00
1.00
1.00
0.50
0.83
0.83
1.00
1.00
1.00
1.00
0.83
0.83
0.33
0.83
0.50
1.00
0.50
0.50
0.88
1.00
1.00
0.38
0.88
0.88
1.00
0.75
0.25
0.63
0.38
0.88
0.88
1.00
0.88
0.63
0.38
0.25
0.63
0.88
0.63
0.88
0.75
0.88
1.00
1.00
0.50
0.88
1.00
1.00
0.88
0.63
0.88
0.88
0.13
0.38
0.63
0.88
0.75
1.00
0.63
0.63
1.00
0.75
0.75
1.00
0.63
0.50
0.75
0.75
0.50
0.50
0.88
1.00
1.00
1.00
0.63
0.63
0.38
0.88
1.00
0.88
0.50
1.00
0.88
1.00
0.88
0.63
0.88
1.00
0.38
0.63
0.38
0.75
0.63
0.63
0.88
1.00
0.00
0.38
0.50
0.38
0.63
0.75
0.63
0.75
0.75
0.75
0.88
0.88
0.88
0.88
0.75
0.63
0.75
0.13
0.50
0.88
0.38
0.75
0.25
0.75
0.63
0.88
0.63
0.63
0.50
0.75
0.88
0.63
0.88
0.63
0.63
0.38
0.63
1.00
0.88
0.63
0.63
0.88
1.00
1.00
0.50
0.63
0.88
1.00
1.00
0.38
0.88
0.88
0.75
0.38
0.13
0.63
1.00
0.88
0.25
0.63
0.75
1.00
1.00
1.00
0.25
0.63
0.88
0.50
0.63
0.50
0.88
0.88
0.25
0.63
0.88
0.88
0.88
0.88
1.00
1.00
0.13
1.00
0.88
1.00
0.75
0.13
0.63
1.00
0.88
0.63
0.63
0.63
0.75
0.63
0.88
0.88
0.50
0.25
0.63
0.50
0.88
0.50
0.50
0.88
0.88
0.88
0.75
0.75
0.38
0.75
0.75
0.38
0.50
0.38
0.75
0.75
0.88
0.50
0.75
0.75
0.75
0.75
0.50
0.38
0.63
0.75
0.50
0.75
0.38
0.50
0.75
0.13
0.25
0.88
0.63
0.75
0.75
0.50
1.00
1.00
0.63
0.88
0.88
1.00
0.75
0.25
0.25
0.38
0.38
0.25
0.38
0.75
0.88
0.75
0.63
0.25
1.00
0.75
0.75
0.88
1.00
0.50
0.88
0.50
0.50
0.88
0.88
0.25
0.38
0.88
0.75
0.75
0.13
0.88
0.75
0.88
0.25
0.83
0.83
0.50
0.67
0.50
0.83
0.50
0.67
0.50
0.50
0.50
0.50
0.67
0.33
0.33
0.50
1.00
1.00
0.50
0.33
0.50
1.00
0.67
0.67
0.83
0.17
0.00
0.17
0.33
0.67
0.33
0.17
0.75
0.75
1.00
0.25
0.00
0.88
0.63
0.50
0.75
0.50
0.25
0.25
0.63
0.50
0.75
0.63
0.50
0.38
0.25
0.50
0.75
0.63
0.75
0.75
0.75
0.38
0.38
0.38
0.38
0.75
0.38
0.38
0.63
0.78
0.85
0.63
0.45
0.66
0.82
0.76
0.76
0.54
0.54
0.65
0.76
0.77
0.86
0.63
0.61
0.70
0.46
0.60
0.73
0.78
0.75
0.70
0.82
0.78
0.77
0.50
0.77
0.83
0.79
0.57
0.86
0.85
0.76
0.73
0.71
0.70
0.69
0.68
0.67
0.61
0.55
0.54
0.69
2008 Mathematics Instructional Materials Review Draft Report
Page 42
Math Expressions
Bridges in Mathematics
enVision
Investigations
Progress in Mathematics
Everyday Mathematics
Saxon Math Intermediate
Saxon Math (Elem)
Math Trailblazers
Math Out of the Box
Growing with Mathematics
Singapore Math Standards
Overall Average
PE
3.1.A
3.1.B
3.1.C
3.1.D
3.1.E
3.2.A
3.2.B
3.2.C
3.2.D
3.2.E
3.2.F
3.2.G
3.2.H
3.3.A
3.3.B
3.3.C
3.3.D
3.4.A
3.4.B
3.4.C
3.4.D
3.4.E
3.5.A
3.5.B
3.5.C
3.5.D
3.5.E
3.6.A
3.6.B
3.6.C
3.6.D
3.6.E
3.6.F
3.6.G
3.6.H
3.6.I
3.6.J
Overall
Math Connects (Elem)
Table 14. Performance Expectations alignment for all submitted 3rd grade programs.
1.00
0.83
1.00
1.00
0.83
1.00
1.00
1.00
1.00
1.00
0.83
0.67
0.83
0.83
0.83
0.83
0.50
0.33
0.33
0.33
0.50
0.33
0.50
0.33
1.00
1.00
1.00
1.00
0.83
0.67
0.83
0.83
0.83
0.83
1.00
0.33
0.33
0.76
0.75
0.75
1.00
0.50
0.88
0.88
0.50
1.00
0.75
1.00
0.63
0.50
0.75
0.75
0.50
0.88
0.25
0.75
0.75
1.00
1.00
0.63
0.88
0.88
0.75
0.88
0.63
0.75
0.88
0.88
0.63
0.88
1.00
0.63
0.50
0.50
0.50
0.74
0.88
1.00
0.75
0.88
0.63
0.75
0.63
0.63
0.63
0.25
0.75
0.75
0.88
0.75
0.50
0.38
0.13
0.88
0.75
0.88
0.88
0.00
0.50
1.00
0.75
0.88
0.75
0.50
0.13
0.13
0.75
0.75
0.63
0.88
0.63
0.63
0.75
0.65
0.88
0.50
0.88
0.88
0.88
0.75
0.63
0.75
1.00
0.75
0.50
0.50
0.75
1.00
0.63
0.75
0.38
0.63
0.50
0.50
0.63
0.13
1.00
0.63
0.38
0.38
0.88
0.25
0.75
0.63
0.75
0.63
0.75
0.25
0.88
0.75
0.25
0.65
0.38
0.25
0.50
0.63
0.75
0.75
0.50
0.63
0.88
0.25
0.88
0.38
0.75
0.50
0.38
0.63
0.50
0.00
0.88
0.50
0.88
0.63
0.88
0.88
0.13
0.13
0.38
0.88
0.75
1.00
0.88
1.00
0.88
1.00
0.88
0.88
0.63
0.64
0.88
0.50
1.00
0.88
0.88
0.50
0.50
0.50
0.25
0.63
0.38
0.25
0.50
0.38
0.38
0.38
0.38
0.38
0.38
0.38
0.38
0.25
0.38
0.38
0.38
0.38
0.63
1.00
0.75
0.75
1.00
1.00
0.75
0.88
0.75
0.63
0.50
0.57
1.00
0.25
0.50
0.50
0.75
0.38
0.50
1.00
0.63
0.75
0.63
0.38
0.50
0.63
0.63
0.38
0.38
0.63
1.00
1.00
0.75
0.13
0.50
0.38
0.63
0.50
0.38
1.00
0.63
0.25
0.25
0.63
0.75
0.25
0.88
0.25
0.38
0.56
0.75
0.38
0.88
0.75
0.75
0.63
0.38
0.50
0.50
0.88
0.50
0.13
0.75
0.50
0.38
0.38
0.50
0.25
0.38
0.13
0.63
0.50
0.00
0.75
0.50
0.50
0.50
0.75
0.63
0.75
0.75
0.88
0.88
0.13
0.75
0.63
0.25
0.54
0.63
0.38
1.00
0.75
0.38
0.88
0.13
0.50
0.75
1.00
0.38
0.00
0.38
0.75
0.50
0.13
0.13
0.63
0.13
0.38
0.63
0.25
0.25
0.88
0.50
0.50
0.38
0.38
0.00
0.13
0.50
0.63
0.75
0.63
0.50
0.63
0.38
0.48
0.33
0.50
0.67
0.33
0.67
0.83
0.33
0.50
0.67
0.33
0.50
0.50
0.50
0.67
0.50
0.50
0.33
0.50
0.50
0.00
0.50
0.33
0.17
0.00
0.50
0.17
0.50
0.33
0.17
0.17
0.50
0.67
0.67
0.50
0.50
0.83
0.83
0.46
0.90
0.30
0.70
0.40
0.70
0.30
0.20
0.50
0.50
0.50
0.40
0.20
0.70
0.60
0.20
0.20
0.00
0.60
0.30
0.70
0.90
0.10
0.00
0.70
0.50
0.40
0.60
0.30
0.10
0.20
0.50
0.70
0.90
0.80
0.20
0.60
0.50
0.46
0.50
0.13
0.50
0.25
0.38
0.50
0.50
0.88
0.75
0.38
0.75
0.13
0.63
0.50
0.38
0.38
0.38
0.00
0.38
0.00
0.50
0.13
0.00
0.38
0.13
0.38
0.38
0.38
0.13
0.25
0.13
0.50
0.63
0.50
0.38
0.38
0.13
0.36
0.88
0.88
0.50
0.50
0.63
0.38
0.38
0.25
0.25
0.25
0.50
0.63
0.25
0.38
0.38
0.38
0.00
0.50
0.38
0.38
0.25
0.00
0.25
0.00
0.50
0.50
0.50
0.25
0.25
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.30
0.75
0.50
0.75
0.63
0.70
0.64
0.46
0.66
0.65
0.61
0.58
0.37
0.63
0.63
0.46
0.46
0.28
0.47
0.51
0.49
0.66
0.25
0.40
0.57
0.50
0.50
0.57
0.59
0.45
0.44
0.57
0.70
0.73
0.56
0.59
0.54
0.41
0.55
2008 Mathematics Instructional Materials Review Draft Report
Page 43
PE
4.1.A
4.1.B
4.1.C
4.1.D
4.1.E
4.1.F
4.1.G
4.1.H
4.1.I
4.1.J
4.2.A
4.2.B
4.2.C
4.2.D
4.2.E
4.2.F
4.2.G
4.2.H
4.2.I
4.3.A
4.3.B
4.3.C
4.3.D
4.3.E
4.3.F
4.4.A
4.4.B
4.4.C
4.4.D
4.4.E
4.4.F
4.4.G
4.4.H
4.5.A
4.5.B
4.5.C
4.5.D
4.5.E
4.5.F
4.5.G
4.5.H
4.5.I
4.5.J
Math Connects (Elem)
enVision
Progress in Mathematics
Bridges in Mathematics
Saxon Math (Elem)
Math Expressions
Math Trailblazers
Investigations
Everyday Mathematics
Growing with Mathematics
Math Out of the Box
Singapore Math Standards
Overall Average
Table 15. Performance Expectations alignment for all submitted 4th grade programs.
0.75
0.88
1.00
1.00
0.75
0.88
0.88
0.88
0.63
0.63
1.00
1.00
0.75
0.88
1.00
0.88
0.38
0.75
0.63
1.00
0.88
0.75
0.25
0.25
0.63
1.00
0.75
0.88
1.00
0.50
0.88
0.88
0.63
0.75
0.88
0.50
0.75
0.75
0.75
0.75
0.75
0.63
0.38
0.63
0.75
0.88
0.75
0.88
0.75
0.63
1.00
0.88
0.88
1.00
0.88
0.88
0.88
1.00
0.88
0.75
0.38
0.75
0.75
1.00
1.00
0.88
0.88
0.88
1.00
0.88
0.63
0.75
0.63
0.63
0.63
0.38
0.50
0.75
0.88
0.38
0.38
0.63
0.63
0.75
0.50
0.38
0.83
1.00
0.17
0.83
0.00
1.00
0.67
0.67
0.83
0.83
1.00
0.83
0.50
0.83
1.00
1.00
1.00
0.50
0.67
0.33
0.17
0.50
0.67
0.50
0.33
1.00
0.83
0.67
1.00
0.17
0.50
0.67
0.67
1.00
0.83
0.83
0.83
0.83
0.50
1.00
0.83
0.50
0.50
1.00
1.00
0.75
0.50
0.63
0.13
0.63
0.25
0.63
0.25
1.00
1.00
0.38
0.63
0.50
0.88
0.13
0.13
0.50
0.63
0.63
0.50
0.88
0.75
0.50
0.38
0.63
0.38
0.75
0.88
0.88
0.88
1.00
0.75
0.13
0.25
0.63
0.88
0.88
0.75
0.50
0.63
0.75
0.88
0.50
0.25
0.50
0.00
0.75
0.75
0.75
0.63
0.63
0.63
0.38
0.88
0.50
0.50
0.50
0.50
0.50
0.38
0.75
0.50
0.38
0.25
0.13
0.38
0.63
0.50
0.50
0.25
0.38
0.25
0.25
0.25
1.00
0.75
0.38
0.63
0.75
0.88
0.88
1.00
0.75
0.50
1.00
0.75
1.00
0.63
0.00
0.75
0.50
0.88
0.75
0.63
0.50
0.63
0.75
0.38
0.63
1.00
0.75
0.75
0.38
0.88
0.63
0.88
0.75
0.13
0.38
0.50
0.50
0.38
1.00
0.25
0.00
0.38
0.00
0.63
0.63
0.50
0.25
0.38
0.50
0.25
0.25
0.00
0.13
0.75
0.63
0.50
0.75
0.38
0.63
0.38
0.63
0.63
0.88
0.75
0.50
0.25
0.38
0.50
0.63
0.13
0.50
0.38
0.13
0.38
0.38
0.13
0.13
0.25
0.50
0.38
0.25
0.13
0.63
0.88
0.88
1.00
0.63
0.25
0.38
0.63
0.63
0.75
0.75
0.75
0.88
0.75
0.38
1.00
0.88
0.75
0.50
0.25
0.63
0.63
0.88
0.88
0.50
0.63
0.38
0.25
0.25
0.63
0.13
0.25
0.00
0.25
0.88
0.13
0.63
0.38
0.38
0.38
0.00
0.25
0.00
0.75
0.88
0.50
0.63
0.25
0.25
0.38
0.38
0.63
1.00
0.75
0.88
0.88
0.63
0.75
0.38
0.88
0.63
0.25
0.25
0.63
0.63
0.25
0.38
0.88
1.00
0.25
0.75
0.75
0.88
0.13
0.00
0.50
0.00
1.00
0.75
0.38
0.00
0.25
0.75
0.13
0.13
1.00
1.00
1.00
0.63
0.50
0.75
0.38
0.25
0.00
0.63
0.63
0.38
0.50
0.25
0.25
0.25
0.63
0.75
0.63
0.50
0.38
0.63
0.50
0.63
0.63
0.88
0.75
0.13
0.88
0.38
0.50
0.13
0.25
0.63
0.88
0.63
0.63
0.88
0.38
0.75
0.13
0.63
0.63
0.25
0.13
0.63
0.00
0.38
0.38
0.25
0.13
0.25
0.50
0.38
0.50
0.25
0.50
0.13
0.63
0.13
0.63
0.38
0.50
0.88
0.25
0.25
0.50
0.50
0.88
0.75
0.38
0.88
0.63
0.50
0.13
0.25
0.13
0.25
0.38
0.38
0.38
0.13
0.25
0.38
0.63
0.75
0.50
0.50
0.63
0.25
0.50
0.38
0.25
0.25
0.50
0.63
0.63
0.63
0.50
0.63
0.50
0.13
0.63
0.25
0.25
0.50
0.25
0.25
0.50
0.25
0.50
0.63
0.63
0.63
0.25
0.38
0.38
0.25
0.25
0.13
0.88
0.38
0.38
0.88
0.50
0.25
0.25
0.50
0.25
0.50
0.13
0.38
0.25
0.25
0.25
0.25
0.00
0.13
0.00
0.25
0.13
0.13
0.13
0.13
0.66
0.68
0.67
0.63
0.41
0.56
0.56
0.63
0.62
0.63
0.80
0.74
0.51
0.62
0.62
0.71
0.35
0.37
0.41
0.56
0.63
0.55
0.57
0.34
0.44
0.56
0.52
0.47
0.59
0.50
0.63
0.51
0.51
0.60
0.46
0.38
0.44
0.57
0.65
0.61
0.59
0.52
0.41
Overall
Average
0.76
0.74
0.69
0.62
0.54
0.53
0.52
0.51
0.50
0.47
0.46
0.33
0.55
2008 Mathematics Instructional Materials Review Draft Report
Page 44
PE
5.1.A
5.1.B
5.1.C
5.1.D
5.1.E
5.1.F
5.2.A
5.2.B
5.2.C
5.2.D
5.2.E
5.2.F
5.2.G
5.2.H
5.3.A
5.3.B
5.3.C
5.3.D
5.3.E
5.3.F
5.3.G
5.3.H
5.3.I
5.4.A
5.4.B
5.4.C
5.4.D
5.5.A
5.5.B
5.5.C
5.6.A
5.6.B
5.6.C
5.6.D
5.6.E
5.6.F
5.6.G
5.6.H
5.6.I
5.6.J
Bridges in Mathematics
Math Connects (Elem)
Investigations
Progress in Mathematics
Saxon Math (Elem)
Everyday Mathematics
Math Expressions
Cognitive Tutor/Carnegie Algebra
enVision
Growing with Mathematics
Math Trailblazers
Math Out of the Box
Singapore Math Standards
Overall Average
Table 16. Performance Expectations alignment for all submitted 5th grade programs.
1.00
0.88
0.63
0.88
0.88
0.88
1.00
0.38
0.88
0.38
0.63
0.50
0.50
0.75
0.88
0.88
0.75
0.88
0.88
0.63
0.75
0.00
0.38
0.88
0.88
0.63
0.50
0.75
0.63
0.63
1.00
0.63
0.38
0.75
1.00
1.00
1.00
0.88
0.88
0.88
0.75
0.88
0.88
1.00
0.63
0.88
0.88
0.75
1.00
0.88
0.88
0.75
0.63
0.50
0.63
0.38
0.63
0.50
0.25
0.50
0.13
0.00
0.25
0.75
0.88
1.00
0.50
0.75
0.50
0.75
0.88
0.63
0.50
0.75
1.00
0.88
0.75
0.88
0.75
0.38
0.63
0.50
0.25
0.63
0.50
0.88
0.75
0.75
0.25
0.50
0.63
0.63
0.63
0.75
1.00
0.88
0.88
0.38
0.38
0.38
0.25
0.25
0.63
0.88
0.75
0.63
0.88
0.50
0.00
0.88
0.63
0.38
0.38
0.88
0.88
0.88
0.88
0.63
0.88
0.88
0.10
0.70
1.00
0.60
0.30
0.60
0.60
0.60
0.70
0.80
1.00
0.90
0.60
0.70
0.60
0.80
0.70
0.50
0.50
0.40
0.10
0.50
0.60
0.80
0.30
1.00
0.50
0.80
0.40
0.50
1.00
0.70
0.70
0.70
0.80
0.60
0.50
0.70
0.50
0.50
0.00
0.63
0.88
0.75
0.50
0.50
0.88
0.38
0.38
0.75
0.88
0.88
0.25
0.88
0.63
0.75
0.63
0.13
0.00
0.63
0.75
0.63
0.50
0.50
0.38
1.00
0.00
0.88
0.75
0.50
1.00
0.88
0.38
0.88
0.88
0.88
0.88
0.88
0.88
0.25
0.00
0.25
0.50
0.25
0.63
0.38
0.88
0.25
0.75
0.38
0.88
0.63
0.00
0.25
0.25
0.88
0.63
0.88
1.00
0.13
0.13
0.13
0.50
0.38
0.75
0.63
0.88
0.75
0.75
1.00
0.50
0.25
0.50
0.25
0.38
0.63
0.25
0.25
0.25
0.38
0.33
0.17
0.50
0.33
0.33
0.33
0.50
0.00
0.17
0.50
1.00
0.67
0.00
0.33
0.33
1.00
0.50
0.83
0.83
1.00
0.50
0.17
0.33
0.33
0.83
0.67
0.83
0.17
0.17
0.67
0.33
0.17
0.17
0.33
0.50
0.67
0.33
0.50
0.33
0.50
0.00
0.00
0.25
0.25
0.00
0.00
0.13
0.13
0.50
0.63
0.75
0.50
0.25
0.50
0.13
0.50
0.50
0.25
0.25
0.38
0.13
0.00
0.63
0.13
0.50
1.00
0.75
0.63
0.50
0.25
0.88
0.50
0.50
0.63
0.88
0.50
0.88
0.63
1.00
0.75
0.50
0.88
0.88
0.88
0.38
0.25
0.50
0.00
0.50
0.63
0.75
0.38
0.25
0.13
0.50
0.38
0.38
0.50
0.50
0.38
0.13
0.38
0.25
0.50
0.38
0.88
0.38
0.63
0.25
0.63
0.38
0.63
0.13
0.25
0.25
0.50
0.25
0.50
0.25
0.38
0.25
0.75
0.63
0.38
0.25
0.50
0.88
0.50
0.63
0.25
0.50
0.63
0.25
0.63
0.00
0.38
0.50
0.50
0.50
0.50
0.25
0.00
0.38
0.63
0.50
0.00
0.88
0.00
0.75
0.38
0.38
0.25
0.13
0.38
0.50
0.50
0.38
0.50
0.50
0.38
0.30
0.50
0.30
0.60
0.30
0.40
0.50
0.70
0.70
0.10
0.40
0.40
0.40
0.30
0.30
0.40
0.30
0.20
0.30
0.50
0.30
0.10
0.50
0.20
0.30
0.30
0.60
0.50
0.90
0.70
0.40
0.20
0.20
0.30
0.60
0.60
0.50
0.40
0.60
0.50
0.25
0.00
0.25
0.13
0.25
0.13
0.13
0.25
0.25
0.63
0.50
0.38
0.25
0.25
0.88
0.63
0.63
0.50
0.38
0.38
0.13
0.63
0.13
0.50
0.38
0.38
0.38
0.88
0.13
0.63
0.25
0.13
0.13
0.13
0.13
0.50
0.13
0.00
0.25
0.13
0.13
0.38
0.63
0.38
0.13
0.25
0.25
0.25
0.25
0.25
0.63
0.50
0.25
0.38
0.25
0.50
0.38
0.50
0.63
0.38
0.38
0.13
0.25
0.13
0.25
0.38
0.38
0.38
0.13
0.13
0.13
0.00
0.00
0.00
0.00
0.38
0.13
0.13
0.00
0.00
0.32
0.51
0.58
0.55
0.39
0.46
0.60
0.40
0.55
0.51
0.72
0.59
0.34
0.49
0.49
0.63
0.57
0.49
0.48
0.46
0.29
0.23
0.42
0.51
0.53
0.65
0.57
0.59
0.46
0.58
0.60
0.42
0.32
0.48
0.60
0.65
0.53
0.53
0.55
0.45
Overall
Average
0.73
0.68
0.63
0.62
0.62
0.48
0.45
0.44
0.43
0.43
0.42
0.32
0.26
0.50
2008 Mathematics Instructional Materials Review Draft Report
Page 45
3.1.2 Part 2: Other Factors for Grades K-5
The following graphs show results for each scale in Other Factors, in ranked order.
Program Organization and Design
Math Connects (Elem)
Investigations
Bridges in Mathematics
Growing with Mathematics
Math Expressions
Everyday Mathematics
enVision
Math Out of the Box
Math Trailblazers
Saxon Math (Elem)
Progress in Mathematics
Singapore Math Standards
0.00
0.25
0.50
0.75
1.00
0.75
1.00
Student Experience
Math Connects (Elem)
Bridges in Mathematics
Investigations
Growing with Mathematics
Math Expressions
Everyday Mathematics
Saxon Math (Elem)
Math Out of the Box
Math Trailblazers
Progress in Mathematics
enVision
Singapore Math Standards
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
Page 46
Assessment
Math Connects (Elem)
Everyday Mathematics
Growing with Mathematics
Bridges in Mathematics
Math Expressions
enVision
Investigations
Saxon Math (Elem)
Math Trailblazers
Progress in Mathematics
Math Out of the Box
Singapore Math Standards
0.00
0.25
0.50
0.75
1.00
0.75
1.00
Instructional Planning and Professional Support
Math Connects (Elem)
Bridges in Mathematics
Investigations
Growing with Mathematics
Math Expressions
Math Trailblazers
Everyday Mathematics
enVision
Progress in Mathematics
Math Out of the Box
Saxon Math (Elem)
Singapore Math Standards
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
Page 47
Equity and Access
Math Connects (Elem)
Investigations
Math Expressions
enVision
Saxon Math (Elem)
Everyday Mathematics
Growing with Mathematics
Progress in Mathematics
Math Trailblazers
Bridges in Mathematics
Math Out of the Box
Singapore Math Standards
0.00
0.25
0.50
0.75
1.00
3.1.3 Results of Individual Publisher Series
This section presents individual graphs and narrative that describe how the particular
publisher series did in the review process. It includes scaled values for each scale, for all
grades submitted for review. Note that this section includes results from all programs
presented alphabetically, including materials that did not cover the comprehensive range
K-5.
2008 Mathematics Instructional Materials Review Draft Report
Page 48
3.1.3.1 Bridges in Mathematics
Content/Standards Alignment
Bridges in Mathematics
Program Organization and Design
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
K
0.528
0.656
0.479
0.625
0.608
0.815
1
0.528
0.667
0.448
0.658
0.725
0.676
2
0.486
0.708
0.521
0.642
0.642
0.863
3
0.306
0.573
0.625
0.583
0.525
0.652
4
0.458
0.792
0.833
0.725
0.692
0.619
5
0.333
0.688
0.844
0.708
0.692
0.734
This graph and chart combination shows each of the scales on the vertical axis, and
displays the scaled average score for each grade on the horizontal axis. The intent is to
see a complete picture of how the program performed at all grade levels and all scales.
2008 Mathematics Instructional Materials Review Draft Report
Page 49
K
1
2
3
Bridges in Mathematics
4
5
Patterns and Operations
Additional Key Content
Objects and their Locations
Reasoning, Problem Solving, and Communication
Whole Numbers
Reasoning, Problem Solving, and Communication
Concepts of Measurement
Geometric Attributes
Addition and Subtraction
Additional Key Content
Whole Number Relationships
Reasoning, Problem Solving, and Communication
Measurement
Place Value and the Base 10 System
Addition and Subtraction
Additional Key Content
Addition, Subtraction and Place Value
Additional Key Content
Geometry
Concepts of Multiplication and Division
Reasoning, Problem Solving, and Communication
Fraction Concepts
Additional Key Content
Concept of Area
Reasoning, Problem Solving, and Communication
Multi-digit Multiplication
Fractions, Decimals and Mixed Numbers
Multi-digit Division
Reasoning, Problem Solving, and Communication
Representations of Algebraic Relationships
Additional Key Content
Triangles and Quadrilaterals
Addition and Subtraction of Fractions and Decimals
0.00
0.25
0.50
0.75
1.00
This graph shows the Core Content Areas of the 2008 Revised Washington Standards,
organized by grade for the program Bridges in Mathematics. Within each grade, the core
content areas are organized by average score, from highest to lowest. This graph gives
school districts valuable information on broad categories of areas where the text does
well, or where it might need to be supplemented.
2008 Mathematics Instructional Materials Review Draft Report
Page 50
3.1.3.2 enVision
Content/Standards Alignment
Program Organization and Design
enVision
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
0.569
0.583
0.708
0.458
0.533
0.484
0.653
0.552
0.563
0.467
0.508
0.551
2
0.514
0.521
0.583
0.442
0.467
0.539
3
0.681
0.552
0.615
0.492
0.592
0.645
4
0.694
0.521
0.563
0.500
0.633
0.744
5
0.597
0.490
0.583
0.517
0.625
0.434
6
0.681
0.635
0.563
0.558
0.600
0.625
3
6
5
4
enVision
2
1
K
K
1
Patterns and Operations
Additional Key Content
Objects and their Locations
Whole Numbers
Reasoning, Problem Solving, and Communication
Geometric Attributes
Whole Number Relationships
Addition and Subtraction
Additional Key Content
Reasoning, Problem Solving, and Communication
Concepts of Measurement
Additional Key Content
Place Value and the Base 10 System
Addition and Subtraction
Measurement
Reasoning, Problem Solving, and Communication
Addition, Subtraction and Place Value
Concepts of Multiplication and Division
Fraction Concepts
Additional Key Content
Reasoning, Problem Solving, and Communication
Geometry
Concept of Area
Fractions, Decimals and Mixed Numbers
Multi-digit Multiplication
Additional Key Content
Reasoning, Problem Solving, and Communication
Multi-digit Division
Representations of Algebraic Relationships
Additional Key Content
Addition and Subtraction of Fractions and Decimals
Triangles and Quadrilaterals
Reasoning, Problem Solving, and Communication
Mathematical Expressions and Equations
Ratios, Rates and Percents
Two- and Three-Dimensional Figures
Reasoning, Problem Solving, and Communication
Multiplication & Division of Fractions and Decimals
Additional Key Content
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 51
3.1.3.3 Everyday Mathematics
Content/Standards Alignment
Everyday Mathematics
Program Organization and Design
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
0.542
0.656
0.479
0.608
0.675
0.674
0.736
0.635
0.781
0.667
0.633
0.676
2
0.597
0.635
0.688
0.567
0.658
0.727
3
0.472
0.406
0.625
0.483
0.558
0.564
4
0.556
0.510
0.688
0.483
0.508
0.503
5
0.611
0.563
0.677
0.550
0.617
0.481
2
3
5
4
Everyday Mathematics
1
K
K
1
Additional Key Content
Patterns and Operations
Whole Numbers
Reasoning, Problem Solving, and Communication
Objects and their Locations
Whole Number Relationships
Addition and Subtraction
Reasoning, Problem Solving, and Communication
Geometric Attributes
Additional Key Content
Concepts of Measurement
Additional Key Content
Addition and Subtraction
Reasoning, Problem Solving, and Communication
Measurement
Place Value and the Base 10 System
Geometry
Addition, Subtraction and Place Value
Concepts of Multiplication and Division
Reasoning, Problem Solving, and Communication
Fraction Concepts
Additional Key Content
Additional Key Content
Fractions, Decimals and Mixed Numbers
Multi-digit Multiplication
Reasoning, Problem Solving, and Communication
Concept of Area
Additional Key Content
Representations of Algebraic Relationships
Triangles and Quadrilaterals
Addition and Subtraction of Fractions and Decimals
Reasoning, Problem Solving, and Communication
Multi-digit Division
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 52
3.1.3.4 Growing with Mathematics
Content/Standards Alignment
Growing with Mathematics
Program Organization and Design
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
0.625
0.667
0.635
0.617
0.775
0.663
0.589
0.733
0.642
0.753
0.800
0.716
2
0.556
0.667
0.625
0.722
0.744
0.854
3
0.486
0.594
0.604
0.508
0.525
0.365
4
0.417
0.521
0.646
0.475
0.425
0.468
5
0.514
0.521
0.635
0.483
0.517
0.425
2
3
5
4
Growing with Mathematics
1
K
K
1
Patterns and Operations
Additional Key Content
Objects and their Locations
Whole Numbers
Reasoning, Problem Solving, and Communication
Geometric Attributes
Additional Key Content
Reasoning, Problem Solving, and Communication
Addition and Subtraction
Concepts of Measurement
Whole Number Relationships
Additional Key Content
Place Value and the Base 10 System
Addition and Subtraction
Measurement
Reasoning, Problem Solving, and Communication
Concepts of Multiplication and Division
Fraction Concepts
Addition, Subtraction and Place Value
Reasoning, Problem Solving, and Communication
Additional Key Content
Geometry
Concept of Area
Multi-digit Multiplication
Fractions, Decimals and Mixed Numbers
Additional Key Content
Reasoning, Problem Solving, and Communication
Addition and Subtraction of Fractions and Decimals
Representations of Algebraic Relationships
Multi-digit Division
Reasoning, Problem Solving, and Communication
Additional Key Content
Triangles and Quadrilaterals
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 53
3.1.3.5 Investigations
Content/Standards Alignment
Program Organization and Design
Investigations
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
0.569
0.708
0.500
0.608
0.692
0.674
0.639
0.698
0.635
0.667
0.767
0.753
2
0.569
0.594
0.583
0.567
0.575
0.609
3
0.750
0.656
0.552
0.617
0.742
0.635
4
0.694
0.677
0.667
0.667
0.708
0.506
5
0.708
0.708
0.563
0.633
0.733
0.625
5
4
3
Investigations
2
1
K
K
1
Whole Numbers
Objects and their Locations
Reasoning, Problem Solving, and Communication
Patterns and Operations
Additional Key Content
Geometric Attributes
Additional Key Content
Addition and Subtraction
Reasoning, Problem Solving, and Communication
Concepts of Measurement
Whole Number Relationships
Addition and Subtraction
Measurement
Additional Key Content
Reasoning, Problem Solving, and Communication
Place Value and the Base 10 System
Reasoning, Problem Solving, and Communication
Concepts of Multiplication and Division
Geometry
Fraction Concepts
Addition, Subtraction and Place Value
Additional Key Content
Multi-digit Multiplication
Reasoning, Problem Solving, and Communication
Concept of Area
Additional Key Content
Fractions, Decimals and Mixed Numbers
Representations of Algebraic Relationships
Reasoning, Problem Solving, and Communication
Addition and Subtraction of Fractions and Decimals
Multi-digit Division
Triangles and Quadrilaterals
Additional Key Content
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 54
3.1.3.6 Math Connects (Elem)
Content/Standards Alignment
Math Connects (Elem)
Program Organization and Design
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
0.708
0.802
0.698
0.675
0.708
0.859
0.750
0.760
0.688
0.617
0.625
0.652
2
0.653
0.688
0.615
0.608
0.633
0.758
3
0.759
0.806
0.694
0.789
0.800
0.761
4
0.778
0.740
0.729
0.767
0.775
0.759
5
0.653
0.719
0.635
0.658
0.700
0.675
2
3
5
4
Math Connects (Elem)
1
K
K
1
Additional Key Content
Reasoning, Problem Solving, and Communication
Patterns and Operations
Objects and their Locations
Whole Numbers
Reasoning, Problem Solving, and Communication
Additional Key Content
Addition and Subtraction
Geometric Attributes
Whole Number Relationships
Concepts of Measurement
Reasoning, Problem Solving, and Communication
Additional Key Content
Place Value and the Base 10 System
Addition and Subtraction
Measurement
Addition, Subtraction and Place Value
Concepts of Multiplication and Division
Additional Key Content
Fraction Concepts
Reasoning, Problem Solving, and Communication
Geometry
Multi-digit Multiplication
Additional Key Content
Fractions, Decimals and Mixed Numbers
Reasoning, Problem Solving, and Communication
Concept of Area
Multi-digit Division
Representations of Algebraic Relationships
Addition and Subtraction of Fractions and Decimals
Reasoning, Problem Solving, and Communication
Additional Key Content
Triangles and Quadrilaterals
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 55
3.1.3.7 Math Expressions
Content/Standards Alignment
Math Expressions
Program Organization and Design
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
0.653
0.698
0.646
0.667
0.683
0.826
0.611
0.552
0.615
0.567
0.650
0.571
2
0.694
0.615
0.729
0.617
0.692
0.691
3
0.681
0.563
0.604
0.567
0.625
0.743
4
0.639
0.552
0.563
0.558
0.600
0.529
5
0.630
0.528
0.542
0.544
0.478
0.454
Math Expressions
5
4
3
2
1
K
K
1
Patterns and Operations
Whole Numbers
Objects and their Locations
Reasoning, Problem Solving, and Communication
Additional Key Content
Reasoning, Problem Solving, and Communication
Additional Key Content
Addition and Subtraction
Whole Number Relationships
Geometric Attributes
Concepts of Measurement
Reasoning, Problem Solving, and Communication
Addition and Subtraction
Additional Key Content
Measurement
Place Value and the Base 10 System
Geometry
Additional Key Content
Addition, Subtraction and Place Value
Concepts of Multiplication and Division
Reasoning, Problem Solving, and Communication
Fraction Concepts
Multi-digit Multiplication
Fractions, Decimals and Mixed Numbers
Concept of Area
Additional Key Content
Reasoning, Problem Solving, and Communication
Representations of Algebraic Relationships
Triangles and Quadrilaterals
Addition and Subtraction of Fractions and Decimals
Reasoning, Problem Solving, and Communication
Additional Key Content
Multi-digit Division
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 56
3.1.3.8 Math Out of the Box
Content/Standards Alignment
Math Out of the Box
Program Organization and Design
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
0.417
0.552
0.615
0.550
0.542
0.625
0.486
0.635
0.635
0.650
0.667
0.743
2
0.389
0.500
0.531
0.667
0.675
0.668
3
0.244
0.475
0.517
0.400
0.500
0.457
4
0.389
0.427
0.521
0.475
0.433
0.462
5
0.375
0.375
0.438
0.358
0.383
0.322
2
3
5
4
Math Out of the Box
1
K
K
1
Patterns and Operations
Objects and their Locations
Reasoning, Problem Solving, and Communication
Additional Key Content
Whole Numbers
Additional Key Content
Reasoning, Problem Solving, and Communication
Geometric Attributes
Addition and Subtraction
Concepts of Measurement
Whole Number Relationships
Reasoning, Problem Solving, and Communication
Additional Key Content
Place Value and the Base 10 System
Addition and Subtraction
Measurement
Addition, Subtraction and Place Value
Geometry
Reasoning, Problem Solving, and Communication
Additional Key Content
Concepts of Multiplication and Division
Fraction Concepts
Additional Key Content
Fractions, Decimals and Mixed Numbers
Reasoning, Problem Solving, and Communication
Multi-digit Multiplication
Concept of Area
Additional Key Content
Triangles and Quadrilaterals
Representations of Algebraic Relationships
Addition and Subtraction of Fractions and Decimals
Reasoning, Problem Solving, and Communication
Multi-digit Division
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 57
3.1.3.9 Math Trailblazers
Content/Standards Alignment
Math Trailblazers
Program Organization and Design
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
0.319
0.479
0.354
0.333
0.458
0.478
0.481
0.486
0.486
0.444
0.411
0.577
2
0.542
0.646
0.677
0.617
0.600
0.695
3
0.574
0.583
0.556
0.533
0.567
0.459
4
0.542
0.625
0.635
0.575
0.592
0.523
5
0.444
0.617
0.617
0.533
0.533
0.415
Math Trailblazers
5
4
3
2
1
K
K
1
Additional Key Content
Patterns and Operations
Objects and their Locations
Reasoning, Problem Solving, and Communication
Whole Numbers
Additional Key Content
Whole Number Relationships
Geometric Attributes
Reasoning, Problem Solving, and Communication
Addition and Subtraction
Concepts of Measurement
Reasoning, Problem Solving, and Communication
Additional Key Content
Addition and Subtraction
Measurement
Place Value and the Base 10 System
Concepts of Multiplication and Division
Reasoning, Problem Solving, and Communication
Fraction Concepts
Addition, Subtraction and Place Value
Geometry
Additional Key Content
Reasoning, Problem Solving, and Communication
Multi-digit Multiplication
Additional Key Content
Fractions, Decimals and Mixed Numbers
Concept of Area
Additional Key Content
Addition and Subtraction of Fractions and Decimals
Reasoning, Problem Solving, and Communication
Multi-digit Division
Representations of Algebraic Relationships
Triangles and Quadrilaterals
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 58
3.1.3.10
Progress in Mathematics
Content/Standards Alignment
Progress in Mathematics
Program Organization and Design
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
0.444
0.448
0.458
0.400
0.417
0.408
0.569
0.448
0.531
0.408
0.392
0.405
2
0.528
0.542
0.656
0.567
0.567
0.707
3
0.458
0.552
0.531
0.558
0.525
0.568
4
0.519
0.486
0.569
0.467
0.522
0.694
5
0.533
0.500
0.567
0.480
0.527
0.623
6
0.514
0.542
0.521
0.475
0.433
0.574
3
6
5
4
Progress in Mathematics
2
1
K
K
1
Additional Key Content
Objects and their Locations
Patterns and Operations
Whole Numbers
Reasoning, Problem Solving, and Communication
Reasoning, Problem Solving, and Communication
Addition and Subtraction
Additional Key Content
Whole Number Relationships
Concepts of Measurement
Geometric Attributes
Place Value and the Base 10 System
Reasoning, Problem Solving, and Communication
Additional Key Content
Addition and Subtraction
Measurement
Addition, Subtraction and Place Value
Reasoning, Problem Solving, and Communication
Concepts of Multiplication and Division
Additional Key Content
Fraction Concepts
Geometry
Fractions, Decimals and Mixed Numbers
Reasoning, Problem Solving, and Communication
Additional Key Content
Multi-digit Multiplication
Concept of Area
Addition and Subtraction of Fractions and Decimals
Reasoning, Problem Solving, and Communication
Representations of Algebraic Relationships
Additional Key Content
Multi-digit Division
Triangles and Quadrilaterals
Mathematical Expressions and Equations
Ratios, Rates and Percents
Reasoning, Problem Solving, and Communication
Multiplication & Division of Fractions and Decimals
Additional Key Content
Two- and Three-Dimensional Figures
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 59
3.1.3.11
Saxon Math (Elem)
Content/Standards Alignment
Saxon Math (Elem)
Program Organization and Design
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
0.597
0.448
0.563
0.483
0.442
0.505
0.625
0.542
0.594
0.575
0.625
0.780
2
0.569
0.510
0.667
0.608
0.600
0.684
3
0.722
0.427
0.615
0.458
0.408
0.476
4
0.500
0.406
0.438
0.417
0.342
0.541
5
0.625
0.583
0.604
0.575
0.533
0.622
2
3
5
4
Saxon Math (Elem)
1
K
K
1
Additional Key Content
Patterns and Operations
Objects and their Locations
Whole Numbers
Reasoning, Problem Solving, and Communication
Additional Key Content
Reasoning, Problem Solving, and Communication
Whole Number Relationships
Concepts of Measurement
Addition and Subtraction
Geometric Attributes
Addition and Subtraction
Place Value and the Base 10 System
Additional Key Content
Reasoning, Problem Solving, and Communication
Measurement
Addition, Subtraction and Place Value
Additional Key Content
Concepts of Multiplication and Division
Reasoning, Problem Solving, and Communication
Geometry
Fraction Concepts
Reasoning, Problem Solving, and Communication
Multi-digit Multiplication
Fractions, Decimals and Mixed Numbers
Concept of Area
Additional Key Content
Reasoning, Problem Solving, and Communication
Additional Key Content
Addition and Subtraction of Fractions and Decimals
Multi-digit Division
Triangles and Quadrilaterals
Representations of Algebraic Relationships
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 60
3.1.3.12
Singapore Math Standards
Content/Standards Alignment
Singapore Math Standards
Program Organization and Design
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
0.300
0.392
0.292
0.433
0.447
0.443
0.292
0.396
0.365
0.433
0.417
0.345
2
0.333
0.431
0.417
0.478
0.456
0.547
3
0.292
0.438
0.396
0.333
0.450
0.301
4
0.153
0.469
0.385
0.367
0.400
0.328
5
0.222
0.406
0.396
0.392
0.383
0.263
2
3
5
4
Singapore Math Standards
1
K
K
1
Additional Key Content
Whole Numbers
Patterns and Operations
Objects and their Locations
Reasoning, Problem Solving, and Communication
Additional Key Content
Addition and Subtraction
Concepts of Measurement
Whole Number Relationships
Geometric Attributes
Reasoning, Problem Solving, and Communication
Additional Key Content
Place Value and the Base 10 System
Measurement
Addition and Subtraction
Reasoning, Problem Solving, and Communication
Addition, Subtraction and Place Value
Concepts of Multiplication and Division
Additional Key Content
Geometry
Fraction Concepts
Reasoning, Problem Solving, and Communication
Concept of Area
Fractions, Decimals and Mixed Numbers
Multi-digit Multiplication
Additional Key Content
Reasoning, Problem Solving, and Communication
Triangles and Quadrilaterals
Addition and Subtraction of Fractions and Decimals
Multi-digit Division
Representations of Algebraic Relationships
Additional Key Content
Reasoning, Problem Solving, and Communication
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 61
3.2 6-8 Results
See Section 3.1 for detail and narrative explaining the graphs and charts. This section is
organized similarly to Section 3.1 K-5 Results.
3.2.1 Content/Standards Alignment
Content/Standards Alignment
Holt Mathematics
Math Connects (Middle)
Prentice Hall Mathematics
Math Thematics
McDougal Littell Math Course
Connected Mathematics 2
Impact
Cognitive Tutor/Carnegie Algebra
Saxon Math Intermediate (Middle)
CPM Middle Grades Program
Everyday Math/Transition
ALEKS, Online Mathematics Solution
Mathscape
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 62
Prentice Hall Mathematics
Connected Mathematics 2
Math Thematics
Math Connects (Middle)
McDougal Littell Math Course
enVision
Impact
Progress in Mathematics
Cognitive Tutor/Carnegie Algebra
Saxon Math Intermediate (Middle)
Mathscape
CPM Middle Grades Program
ALEKS, Online Mathematics Solution
Everyday Math/Transition
Overall Average
PE
6.1.A
6.1.B
6.1.C
6.1.D
6.1.E
6.1.F
6.1.G
6.1.H
6.2.A
6.2.B
6.2.C
6.2.D
6.2.E
6.2.F
6.3.A
6.3.B
6.3.C
6.3.D
6.3.E
6.3.F
6.3.G
6.4.A
6.4.B
6.4.C
6.4.D
6.4.E
6.4.F
6.4.G
6.5.A
6.5.B
6.5.C
6.6.A
6.6.B
6.6.C
6.6.D
6.6.E
6.6.F
6.6.G
6.6.H
Overall
Average
Holt Mathematics
Table 17. Performance Expectations alignment for all submitted 6th grade programs.
1.00
0.88
0.50
1.00
0.50
1.00
0.50
1.00
1.00
1.00
1.00
0.88
1.00
0.88
1.00
0.88
1.00
1.00
0.88
1.00
1.00
1.00
0.75
0.88
0.25
0.88
1.00
0.63
0.63
1.00
1.00
0.88
0.63
1.00
1.00
1.00
1.00
0.75
1.00
0.75
0.50
0.63
0.88
0.63
1.00
0.00
0.88
1.00
1.00
1.00
0.88
1.00
0.88
0.88
1.00
0.88
0.88
0.88
1.00
0.88
0.88
0.63
0.63
0.88
0.88
0.38
0.38
0.63
0.88
1.00
1.00
0.38
1.00
1.00
1.00
1.00
0.88
0.75
1.00
0.75
0.88
0.88
0.88
0.75
0.50
0.88
0.75
0.75
0.63
0.38
0.38
0.88
0.75
0.75
0.88
0.75
0.50
1.00
1.00
0.75
0.75
0.63
0.88
0.63
0.63
0.13
0.38
0.75
0.88
0.88
0.63
0.88
0.75
0.88
0.88
0.75
0.88
0.88
0.63
0.63
0.63
0.25
0.63
0.25
0.63
1.00
0.88
0.50
0.75
1.00
0.88
0.63
0.88
0.88
0.50
0.75
1.00
0.88
1.00
0.38
0.25
0.63
0.38
0.00
0.50
0.88
1.00
0.88
1.00
0.88
1.00
0.75
0.88
0.88
0.63
0.63
0.88
0.75
0.75
0.88
0.50
1.00
0.50
0.63
0.75
0.88
1.00
0.50
0.63
0.50
0.75
0.88
0.75
0.50
0.38
0.75
0.88
0.63
0.13
0.13
0.38
0.75
0.38
0.13
0.00
0.75
0.75
0.75
0.63
0.88
0.63
0.75
0.88
0.88
1.00
0.75
0.63
0.50
1.00
0.63
1.00
0.25
0.63
0.75
0.75
1.00
1.00
0.88
0.63
0.25
0.63
0.88
0.50
0.75
0.50
0.88
1.00
0.63
0.25
0.13
0.50
0.00
0.25
0.38
0.63
0.38
0.50
0.13
1.00
1.00
1.00
1.00
0.88
0.38
0.63
0.63
0.75
0.63
0.38
0.63
0.13
0.75
0.75
0.75
0.75
0.88
0.75
0.63
0.75
0.75
0.75
0.63
0.63
0.63
0.75
0.88
0.38
0.50
0.63
0.63
0.63
0.63
0.38
0.50
0.63
0.75
0.25
0.88
0.50
0.50
0.63
0.63
0.63
0.88
0.50
0.25
0.75
0.88
0.88
0.38
0.88
1.00
1.00
0.88
1.00
0.25
0.75
1.00
1.00
1.00
0.75
0.63
1.00
0.88
0.25
0.00
0.38
0.00
0.00
0.00
0.00
0.63
0.38
0.38
0.75
0.25
0.38
0.63
0.63
0.75
0.38
0.75
0.88
0.25
0.63
0.50
0.75
0.75
0.13
0.63
0.88
0.75
0.75
0.63
0.75
0.75
0.63
0.75
0.63
0.50
0.75
0.38
0.75
0.75
0.75
0.25
0.13
0.38
0.50
0.13
0.50
0.50
0.63
0.63
0.63
0.63
0.63
0.63
0.63
0.50
0.25
0.88
0.25
0.00
0.63
0.50
0.50
0.00
0.75
0.75
0.63
0.50
0.13
0.63
0.63
0.38
0.88
0.63
0.63
0.38
0.63
0.50
0.38
0.63
0.38
0.63
0.38
0.00
0.25
0.13
0.88
0.50
0.63
0.38
0.38
0.63
0.63
0.75
0.50
0.38
0.75
0.38
0.38
0.63
0.25
0.88
0.13
0.50
0.50
0.38
0.00
0.50
0.63
0.13
0.38
0.63
0.75
0.38
0.75
0.75
0.63
0.75
0.13
0.38
0.13
0.38
0.00
0.13
0.75
0.63
0.63
0.88
0.38
0.63
0.63
0.75
0.50
0.50
0.25
0.75
0.50
0.75
0.75
0.88
1.00
0.25
0.50
0.88
0.88
0.25
0.13
0.00
0.25
0.13
0.25
0.63
0.38
0.13
0.75
0.25
0.25
0.00
0.13
0.13
0.00
0.00
0.75
0.00
0.25
0.38
0.88
0.13
0.63
0.75
0.88
0.88
0.75
0.75
0.13
0.13
0.13
0.63
0.00
0.50
0.13
0.50
0.63
0.88
0.50
0.50
0.38
0.25
0.13
0.63
0.75
0.50
0.50
0.88
0.88
0.88
0.50
0.63
0.25
0.75
0.38
0.13
0.13
0.75
0.13
0.75
0.38
0.50
0.50
0.38
0.63
0.25
0.38
0.50
0.00
0.00
0.50
0.63
0.88
0.00
0.88
0.75
0.63
0.88
0.38
0.38
0.25
0.63
0.63
0.63
0.50
0.00
0.13
0.38
0.88
0.88
0.13
0.25
0.25
0.00
0.00
0.13
0.88
0.38
0.13
0.13
0.00
0.00
0.00
0.00
0.00
0.00
0.30
0.30
0.20
0.40
0.60
0.50
0.10
0.30
0.30
0.80
0.20
0.10
0.40
0.30
0.40
0.40
0.40
0.50
0.00
0.30
0.20
0.00
0.20
0.10
0.20
0.20
0.00
0.00
0.30
0.20
0.00
0.40
0.20
0.40
0.40
0.40
0.30
0.60
0.40
0.72
0.47
0.46
0.70
0.55
0.79
0.21
0.68
0.77
0.80
0.65
0.57
0.60
0.57
0.57
0.72
0.75
0.59
0.52
0.70
0.70
0.67
0.44
0.37
0.36
0.46
0.25
0.26
0.39
0.66
0.56
0.71
0.39
0.67
0.65
0.68
0.70
0.59
0.56
0.88
0.81
0.74
0.71
0.65
0.63
0.63
0.59
0.57
0.49
0.48
0.46
0.46
0.35
0.29
0.58
2008 Mathematics Instructional Materials Review Draft Report
Page 63
Math Connects (Middle)
McDougal Littell Math Course
Math Thematics
Cognitive Tutor/Carnegie Algebra
McDougal Littell Pre-/Algebra 1
Connected Mathematics 2
Prentice Hall Mathematics
Impact
Holt Pre-/Algebra 1
CPM Middle Grades Program
Saxon Math Intermediate (Middle)
Everyday Math/Transition
Mathscape
ALEKS, Online Mathematics Solution
Overall Average
PE
7.1.A
7.1.B
7.1.C
7.1.D
7.1.E
7.1.F
7.1.G
7.2.A
7.2.B
7.2.C
7.2.D
7.2.E
7.2.F
7.2.G
7.2.H
7.2.I
7.3.A
7.3.B
7.3.C
7.3.D
7.4.A
7.4.B
7.4.C
7.4.D
7.4.E
7.5.A
7.5.B
7.6.A
7.6.B
7.6.C
7.6.D
7.6.E
7.6.F
7.6.G
7.6.H
Overall
Average
Holt Mathematics
Table 18.Performance Expectations alignment for all submitted 7th grade programs.
1.00
1.00
1.00
0.75
1.00
0.75
1.00
0.13
0.88
0.88
0.63
0.50
0.25
0.50
0.75
0.63
0.88
1.00
1.00
0.88
1.00
1.00
0.75
1.00
1.00
1.00
0.88
1.00
1.00
1.00
0.88
0.88
1.00
0.88
1.00
0.75
0.88
0.63
0.63
0.88
0.63
0.63
0.25
0.88
0.75
0.75
0.75
0.25
0.50
0.75
0.88
0.88
0.38
0.25
0.88
1.00
1.00
0.88
1.00
0.75
0.88
0.88
0.88
0.75
0.88
0.88
0.88
0.88
1.00
0.75
0.50
0.75
1.00
1.00
0.75
0.50
1.00
0.50
0.75
1.00
0.75
0.25
0.25
0.38
0.38
0.63
0.88
0.38
0.38
1.00
1.00
1.00
0.88
1.00
0.88
0.75
0.75
0.50
0.38
0.88
0.88
1.00
1.00
1.00
0.88
0.83
0.67
1.00
1.00
1.00
0.67
0.50
0.50
0.50
0.17
0.50
0.50
0.33
0.50
0.17
0.67
0.67
1.00
0.17
0.67
0.67
0.83
0.67
0.83
1.00
0.83
1.00
1.00
0.83
1.00
1.00
1.00
1.00
1.00
1.00
0.75
0.75
0.88
0.75
1.00
0.75
0.88
0.63
0.88
0.88
0.88
0.50
0.50
0.50
0.50
0.50
0.88
1.00
0.75
0.75
0.50
0.75
0.88
1.00
0.13
1.00
1.00
0.88
0.63
0.38
0.88
0.88
0.63
0.88
0.25
0.88
0.88
1.00
0.63
1.00
0.75
1.00
0.13
0.75
0.63
0.88
0.25
0.63
0.50
0.13
0.25
0.75
0.88
0.13
0.75
0.63
0.63
0.13
0.75
1.00
0.88
0.88
0.75
0.63
0.63
0.75
0.88
0.63
0.88
0.63
0.88
0.88
0.63
0.25
0.75
1.00
0.50
0.25
0.63
0.88
0.88
0.88
0.38
0.38
0.75
0.00
0.38
0.50
0.75
0.75
0.75
0.75
0.88
0.38
0.13
1.00
0.38
0.75
0.75
0.88
0.63
0.88
1.00
0.88
1.00
1.00
0.88
1.00
0.88
0.88
0.63
0.50
0.75
0.75
0.50
0.75
0.88
0.50
0.38
0.50
0.63
0.63
0.00
0.13
0.50
0.75
0.38
0.50
0.88
0.75
1.00
0.75
0.75
0.63
0.63
0.63
0.50
0.63
0.50
0.50
0.63
0.75
0.75
0.25
0.75
0.75
0.75
0.00
0.88
1.00
0.75
1.00
0.63
0.88
0.88
0.88
0.50
0.00
0.00
0.50
0.25
0.25
0.38
0.63
0.75
0.38
0.25
0.75
0.63
0.63
0.88
0.88
0.50
0.63
0.75
0.25
0.75
1.00
0.75
1.00
0.38
0.63
0.00
0.50
0.75
0.75
0.38
0.50
0.00
0.25
0.75
0.50
0.75
0.63
0.50
0.63
0.75
0.50
0.38
0.63
0.75
0.25
0.75
0.50
0.63
0.25
0.50
0.75
0.75
0.88
0.63
0.88
0.75
0.88
0.88
0.50
0.50
0.25
0.50
0.63
0.63
0.13
0.25
0.38
0.38
0.38
0.88
0.63
0.38
0.50
0.50
0.88
0.63
0.38
0.25
0.75
0.38
0.63
0.50
0.75
0.50
0.63
0.75
0.63
0.63
0.75
0.50
0.88
0.63
0.50
0.50
0.38
1.00
0.63
0.63
0.38
0.13
0.13
0.38
0.13
0.50
0.50
0.50
0.38
0.50
0.88
0.25
0.13
0.63
0.25
0.63
0.38
0.63
0.63
0.88
0.38
0.63
0.75
0.63
0.50
0.75
0.88
0.75
0.50
0.25
0.25
0.75
0.50
0.75
0.38
0.38
0.38
0.13
0.13
0.38
0.38
0.38
0.13
0.25
0.63
0.88
0.63
0.75
1.00
0.25
1.00
0.50
0.38
0.25
0.50
0.50
0.75
0.50
0.50
0.38
0.13
0.63
0.50
0.00
0.38
0.75
0.25
0.00
0.50
0.38
0.50
0.13
0.13
0.13
0.75
0.00
0.25
0.25
0.00
0.50
0.63
0.63
0.13
0.50
0.00
0.38
0.25
0.88
0.50
0.88
0.88
1.00
1.00
0.63
0.88
0.75
0.38
0.75
0.88
0.75
0.25
0.63
0.38
0.63
0.63
0.00
0.00
0.00
0.13
0.00
0.13
0.75
0.00
0.00
0.75
0.13
0.25
0.13
0.50
0.00
1.00
0.25
0.00
0.13
0.00
0.00
0.25
0.25
0.00
0.00
0.69
0.76
0.83
0.64
0.78
0.60
0.66
0.35
0.69
0.68
0.63
0.44
0.32
0.37
0.45
0.47
0.64
0.48
0.35
0.67
0.68
0.66
0.54
0.72
0.51
0.81
0.58
0.69
0.58
0.69
0.65
0.76
0.75
0.71
0.66
0.85
0.75
0.74
0.73
0.73
0.67
0.66
0.64
0.60
0.57
0.56
0.51
0.50
0.44
0.30
0.62
2008 Mathematics Instructional Materials Review Draft Report
Page 64
Math Connects (Middle)
Holt Pre-/Algebra 1
Prentice Hall Mathematics
Saxon Math Intermediate (Middle)
McDougal Littell Pre-/Algebra 1
Math Thematics
McDougal Littell Math Course
Impact
Cognitive Tutor/Carnegie Algebra
Connected Mathematics 2
CPM Middle Grades Program
ALEKS, Online Mathematics Solution
Everyday Math/Transition
Mathscape
Overall Average
PE
8.1.A
8.1.B
8.1.C
8.1.D
8.1.E
8.1.F
8.1.G
8.2.A
8.2.B
8.2.C
8.2.D
8.2.E
8.2.F
8.2.G
8.3.A
8.3.B
8.3.C
8.3.D
8.3.E
8.3.F
8.3.G
8.4.A
8.4.B
8.4.C
8.4.D
8.5.A
8.5.B
8.5.C
8.5.D
8.5.E
8.5.F
8.5.G
8.5.H
Overall
Average
Holt Mathematics
Table 19. Performance Expectations alignment for all submitted 8th grade programs.
1.00
1.00
0.75
0.88
0.75
0.75
0.88
0.75
0.88
0.88
0.88
0.75
0.88
0.50
0.75
1.00
0.88
0.88
0.88
1.00
0.75
0.88
0.63
0.88
1.00
1.00
0.63
0.75
0.88
1.00
0.88
0.75
0.88
1.00
0.88
0.75
1.00
1.00
0.75
0.75
0.75
0.88
0.88
0.88
0.88
0.88
1.00
0.88
0.88
1.00
0.88
0.38
1.00
0.75
1.00
0.38
0.88
1.00
0.88
0.38
0.88
0.88
0.88
0.75
0.75
0.88
1.00
1.00
1.00
1.00
1.00
0.63
1.00
0.38
0.25
0.38
0.25
0.75
0.38
0.13
0.88
0.63
1.00
0.75
1.00
1.00
0.50
1.00
0.88
1.00
0.88
1.00
1.00
0.88
1.00
1.00
0.88
0.88
0.88
1.00
0.88
1.00
1.00
0.50
0.75
0.13
0.88
0.88
0.88
0.88
0.88
0.88
0.50
0.88
0.88
0.75
0.50
0.13
0.75
0.88
0.88
0.88
0.75
0.75
1.00
0.50
0.88
0.75
0.75
0.75
0.63
0.63
1.00
1.00
0.88
0.88
0.50
0.75
0.88
0.88
0.88
0.25
0.75
0.50
1.00
0.63
0.88
0.38
0.88
0.38
0.13
0.88
0.25
1.00
0.75
1.00
1.00
0.75
0.75
1.00
0.88
0.88
0.63
1.00
0.25
1.00
1.00
1.00
1.00
0.80
1.00
0.80
0.10
0.10
0.10
0.40
0.80
1.00
1.00
0.70
0.70
1.00
0.30
0.20
1.00
0.80
1.00
0.80
0.80
0.80
0.70
0.60
0.80
0.90
0.80
0.80
0.80
0.70
0.90
0.90
0.70
0.90
0.70
0.80
0.40
0.50
0.70
0.50
0.60
0.70
0.90
0.70
0.60
0.90
0.90
0.80
0.60
0.50
1.00
0.80
0.30
0.70
0.50
0.80
0.40
0.80
0.80
0.80
0.80
0.70
0.80
1.00
1.00
0.75
0.75
0.75
0.75
0.25
0.88
0.63
0.25
1.00
0.50
1.00
0.25
0.50
0.50
0.63
0.50
0.00
0.75
0.63
1.00
0.75
0.88
0.88
0.63
0.63
0.63
0.75
0.88
0.75
0.63
0.25
0.33
0.50
1.00
0.83
0.33
0.33
1.00
0.67
1.00
0.17
0.83
0.50
0.83
0.83
0.17
0.83
0.67
0.50
0.00
0.17
0.50
0.83
0.83
0.83
0.17
1.00
0.83
0.67
0.83
0.83
1.00
0.83
0.17
1.00
1.00
1.00
1.00
1.00
0.88
0.25
0.00
0.00
0.00
0.00
0.13
0.63
0.63
0.75
0.50
0.63
0.75
0.25
0.50
0.38
0.63
0.25
0.38
0.63
0.75
0.38
0.63
0.75
0.75
0.63
0.63
0.75
0.63
0.63
0.75
0.25
0.50
0.63
0.63
0.00
0.00
0.13
0.25
0.25
0.88
0.00
0.63
1.00
0.75
0.88
0.75
0.25
0.13
0.25
0.00
0.38
0.50
0.88
0.63
0.88
1.00
0.75
0.88
1.00
0.88
0.88
0.88
0.88
0.88
0.50
0.88
0.63
0.13
0.13
0.13
0.13
0.50
0.38
0.25
0.25
0.25
0.63
0.13
0.13
0.13
0.13
0.63
0.38
0.38
0.38
0.88
0.63
0.88
1.00
0.88
0.88
0.88
0.88
0.88
0.75
0.75
0.88
0.38
0.63
0.13
1.00
0.75
0.75
0.63
0.50
0.25
0.38
0.25
0.75
0.75
0.00
0.63
0.63
0.63
0.50
0.63
0.88
0.75
0.13
0.25
0.25
0.13
0.38
0.50
0.50
0.00
1.00
0.75
0.63
0.13
0.13
0.38
0.00
0.88
1.00
0.50
0.88
0.00
0.63
0.63
0.75
0.38
0.13
0.00
0.13
0.38
0.13
0.88
0.25
0.38
0.50
0.38
0.25
0.25
0.25
0.63
0.25
0.25
0.25
0.75
0.25
0.25
0.75
0.00
0.00
0.13
0.00
0.00
0.13
0.00
0.00
0.50
0.00
0.25
0.00
0.63
0.00
0.13
0.00
0.25
0.25
0.00
0.13
0.00
0.25
0.13
0.13
0.25
0.25
0.25
0.50
0.25
0.90
0.84
0.80
0.81
0.60
0.67
0.52
0.51
0.52
0.39
0.55
0.52
0.74
0.50
0.61
0.64
0.75
0.48
0.36
0.61
0.52
0.77
0.51
0.68
0.66
0.73
0.52
0.69
0.74
0.76
0.70
0.71
0.57
0.84
0.83
0.79
0.75
0.74
0.74
0.71
0.66
0.63
0.56
0.54
0.53
0.52
0.42
0.19
0.63
2008 Mathematics Instructional Materials Review Draft Report
Page 65
3.2.2 Part 2: Other Factors for Grades 6-8
Program Organization and Design
Holt Mathematics
Math Connects (Middle)
Impact
Math Thematics
Prentice Hall Mathematics
Cognitive Tutor/Carnegie Algebra
McDougal Littell Math Course
Connected Mathematics 2
Mathscape
Everyday Math/Transition
CPM Middle Grades Program
Saxon Math Intermediate (Middle)
ALEKS, Online Mathematics Solution
0.00
0.25
0.50
0.75
1.00
0.75
1.00
Student Experience
Holt Mathematics
Math Connects (Middle)
Cognitive Tutor/Carnegie Algebra
Math Thematics
Prentice Hall Mathematics
Impact
McDougal Littell Math Course
Saxon Math Intermediate (Middle)
CPM Middle Grades Program
Connected Mathematics 2
Mathscape
ALEKS, Online Mathematics Solution
Everyday Math/Transition
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
Page 66
Assessment
Holt Mathematics
Math Connects (Middle)
Cognitive Tutor/Carnegie Algebra
Math Thematics
McDougal Littell Math Course
Connected Mathematics 2
Prentice Hall Mathematics
CPM Middle Grades Program
Everyday Math/Transition
Mathscape
Saxon Math Intermediate (Middle)
ALEKS, Online Mathematics Solution
Impact
0.00
0.25
0.50
0.75
1.00
0.75
1.00
Instructional Planning and Professional Support
Holt Mathematics
Math Connects (Middle)
Math Thematics
Prentice Hall Mathematics
Impact
Connected Mathematics 2
Mathscape
McDougal Littell Math Course
Saxon Math Intermediate (Middle)
Cognitive Tutor/Carnegie Algebra
CPM Middle Grades Program
Everyday Math/Transition
ALEKS, Online Mathematics Solution
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
Page 67
Equity and Access
Holt Mathematics
Prentice Hall Mathematics
McDougal Littell Math Course
Math Connects (Middle)
Impact
Math Thematics
ALEKS, Online Mathematics Solution
Saxon Math Intermediate (Middle)
Connected Mathematics 2
Everyday Math/Transition
Cognitive Tutor/Carnegie Algebra
Mathscape
CPM Middle Grades Program
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 68
3.2.3 Individual Publisher Series
3.2.3.1 ALEKS Online Mathematics Solution
Content/Standards Alignment
ALEKS, Online Mathematics Solution
Program Organization and Design
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
6
0.500
0.146
0.500
0.392
0.325
0.346
7
0.597
0.115
0.469
0.392
0.325
0.304
8
0.625
0.156
0.594
0.558
0.450
0.519
Mathematical Expressions and Equations
Additional Key Content
6
Multiplication & Division of Fractions and Decimals
Ratios, Rates and Percents
Reasoning, Problem Solving, and Communication
Rational Numbers and Linear Equations
Additional Key Content
Surface Area and Volume
7
ALEKS, Online Mathematics Solution
Two- and Three-Dimensional Figures
Proportionality and Similarity
Probability and Data
Reasoning, Problem Solving, and Communication
Additional Key Content
8
Linear Functions and Equations
Properties of Geometric Figures
Summary and Analysis of Data Sets
Reasoning, Problem Solving, and Communication
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 69
3.2.3.2 Cognitive Tutor/Carnegie Algebra
Content/Standards Alignment
Cognitive Tutor/Carnegie Algebra
Program Organization and Design
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
5
0.264
0.594
0.573
0.658
0.508
0.438
6
0.361
0.490
0.604
0.600
0.575
0.490
7
0.542
0.604
0.698
0.650
0.650
0.729
8
0.417
0.615
0.677
0.667
0.633
0.557
Reasoning, Problem Solving, and Communication
Representations of Algebraic Relationships
5
Additional Key Content
Addition and Subtraction of Fractions and Decimals
Triangles and Quadrilaterals
Multi-digit Division
Ratios, Rates and Percents
6
Additional Key Content
Multiplication & Division of Fractions and Decimals
Two- and Three-Dimensional Figures
Additional Key Content
Surface Area and Volume
Rational Numbers and Linear Equations
7
Cognitive Tutor/Carnegie Algebra
Mathematical Expressions and Equations
Reasoning, Problem Solving, and Communication
Reasoning, Problem Solving, and Communication
Probability and Data
Proportionality and Similarity
Linear Functions and Equations
8
Reasoning, Problem Solving, and Communication
Summary and Analysis of Data Sets
Additional Key Content
Properties of Geometric Figures
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 70
3.2.3.3 Connected Mathematics 2
Content/Standards Alignment
Connected Mathematics 2
Program Organization and Design
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
6
0.556
0.708
0.677
0.617
0.625
0.737
7
0.403
0.604
0.646
0.492
0.550
0.664
8
0.403
0.521
0.594
0.483
0.508
0.542
Reasoning, Problem Solving, and Communication
Multiplication & Division of Fractions and Decimals
6
Ratios, Rates and Percents
Additional Key Content
Two- and Three-Dimensional Figures
Reasoning, Problem Solving, and Communication
Rational Numbers and Linear Equations
Additional Key Content
7
Connected Mathematics 2
Mathematical Expressions and Equations
Surface Area and Volume
Probability and Data
Proportionality and Similarity
Reasoning, Problem Solving, and Communication
8
Summary and Analysis of Data Sets
Linear Functions and Equations
Additional Key Content
Properties of Geometric Figures
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 71
3.2.3.4 CPM Middle Grades Program
Content/Standards Alignment
CPM Middle Grades Program
Program Organization and Design
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
6
0.389
0.365
0.573
0.508
0.483
0.455
7
0.306
0.385
0.500
0.508
0.492
0.561
8
0.431
0.531
0.625
0.658
0.625
0.527
Ratios, Rates and Percents
Mathematical Expressions and Equations
6
Two- and Three-Dimensional Figures
Reasoning, Problem Solving, and Communication
Additional Key Content
Rational Numbers and Linear Equations
Reasoning, Problem Solving, and Communication
Surface Area and Volume
7
CPM Middle Grades Program
Multiplication & Division of Fractions and Decimals
Additional Key Content
Probability and Data
Proportionality and Similarity
Reasoning, Problem Solving, and Communication
8
Linear Functions and Equations
Additional Key Content
Summary and Analysis of Data Sets
Properties of Geometric Figures
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 72
3.2.3.5 Everyday Math/Transition
Content/Standards Alignment
Everyday Math/Transition
Program Organization and Design
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
6
0.489
0.375
0.683
0.393
0.493
0.290
7
0.417
0.479
0.448
0.508
0.592
0.504
8
0.417
0.406
0.531
0.383
0.567
0.420
Reasoning, Problem Solving, and Communication
Mathematical Expressions and Equations
6
Multiplication & Division of Fractions and Decimals
Ratios, Rates and Percents
Additional Key Content
Additional Key Content
Probability and Data
Rational Numbers and Linear Equations
7
Everyday Math/Transition
Two- and Three-Dimensional Figures
Reasoning, Problem Solving, and Communication
Proportionality and Similarity
Surface Area and Volume
Properties of Geometric Figures
8
Additional Key Content
Linear Functions and Equations
Reasoning, Problem Solving, and Communication
Summary and Analysis of Data Sets
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 73
3.2.3.6 Holt Mathematics
Content/Standards Alignment
Holt Mathematics
Program Organization and Design
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
6
0.861
0.865
0.615
0.825
0.917
0.875
7
0.806
0.740
0.750
0.733
0.800
0.846
8
0.778
0.781
0.760
0.775
0.825
0.841
Ratios, Rates and Percents
Mathematical Expressions and Equations
6
Reasoning, Problem Solving, and Communication
Additional Key Content
Multiplication & Division of Fractions and Decimals
Two- and Three-Dimensional Figures
Probability and Data
Additional Key Content
7
Holt Mathematics
Reasoning, Problem Solving, and Communication
Surface Area and Volume
Rational Numbers and Linear Equations
Proportionality and Similarity
Summary and Analysis of Data Sets
8
Linear Functions and Equations
Reasoning, Problem Solving, and Communication
Additional Key Content
Properties of Geometric Figures
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 74
3.2.3.7 Holt Pre-/Algebra 1
Content/Standards Alignment
Holt Pre-/Algebra 1
Program Organization and Design
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
7
0.722
0.667
0.646
0.642
0.617
0.568
8
0.792
0.771
0.750
0.800
0.875
0.792
Rational Numbers and Linear Equations
Reasoning, Problem Solving, and Communication
7
Surface Area and Volume
Probability and Data
Holt Pre-/Algebra 1
Additional Key Content
Proportionality and Similarity
Linear Functions and Equations
8
Reasoning, Problem Solving, and Communication
Additional Key Content
Summary and Analysis of Data Sets
Properties of Geometric Figures
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 75
3.2.3.8 Impact
Content/Standards Alignment
Program Organization and Design
Impact
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
6
0.681
0.635
0.604
0.633
0.750
0.590
7
0.569
0.698
0.469
0.625
0.692
0.600
8
0.537
0.472
0.417
0.567
0.522
0.631
Ratios, Rates and Percents
Mathematical Expressions and Equations
6
Multiplication & Division of Fractions and Decimals
Reasoning, Problem Solving, and Communication
Additional Key Content
Two- and Three-Dimensional Figures
Proportionality and Similarity
Rational Numbers and Linear Equations
7
Impact
Reasoning, Problem Solving, and Communication
Probability and Data
Additional Key Content
Surface Area and Volume
Reasoning, Problem Solving, and Communication
8
Properties of Geometric Figures
Additional Key Content
Linear Functions and Equations
Summary and Analysis of Data Sets
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 76
3.2.3.9 Math Connects (Middle)
Content/Standards Alignment
Math Connects (Middle)
Program Organization and Design
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
6
0.625
0.719
0.656
0.650
0.742
0.651
7
0.667
0.667
0.708
0.692
0.733
0.754
8
0.681
0.583
0.646
0.683
0.692
0.833
Reasoning, Problem Solving, and Communication
Multiplication & Division of Fractions and Decimals
6
Mathematical Expressions and Equations
Ratios, Rates and Percents
Additional Key Content
Probability and Data
Additional Key Content
Reasoning, Problem Solving, and Communication
7
Math Connects (Middle)
Two- and Three-Dimensional Figures
Rational Numbers and Linear Equations
Proportionality and Similarity
Surface Area and Volume
Linear Functions and Equations
8
Properties of Geometric Figures
Summary and Analysis of Data Sets
Additional Key Content
Reasoning, Problem Solving, and Communication
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 77
3.2.3.10
Math Thematics
Content/Standards Alignment
Program Organization and Design
Math Thematics
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
6
0.597
0.615
0.646
0.642
0.642
0.705
7
0.593
0.625
0.625
0.611
0.578
0.733
8
0.567
0.650
0.675
0.647
0.693
0.709
Additional Key Content
Mathematical Expressions and Equations
6
Reasoning, Problem Solving, and Communication
Ratios, Rates and Percents
Multiplication & Division of Fractions and Decimals
Two- and Three-Dimensional Figures
Additional Key Content
Rational Numbers and Linear Equations
7
Math Thematics
Reasoning, Problem Solving, and Communication
Probability and Data
Surface Area and Volume
Proportionality and Similarity
Summary and Analysis of Data Sets
8
Linear Functions and Equations
Reasoning, Problem Solving, and Communication
Properties of Geometric Figures
Additional Key Content
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 78
3.2.3.11
Mathscape
Content/Standards Alignment
Program Organization and Design
Mathscape
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
6
0.417
0.625
0.552
0.592
0.683
0.455
7
0.431
0.656
0.729
0.583
0.608
0.436
8
0.306
0.458
0.385
0.375
0.383
0.193
Reasoning, Problem Solving, and Communication
Multiplication & Division of Fractions and Decimals
6
Mathematical Expressions and Equations
Ratios, Rates and Percents
Additional Key Content
Two- and Three-Dimensional Figures
Probability and Data
Rational Numbers and Linear Equations
7
Mathscape
Reasoning, Problem Solving, and Communication
Additional Key Content
Proportionality and Similarity
Surface Area and Volume
Linear Functions and Equations
8
Reasoning, Problem Solving, and Communication
Summary and Analysis of Data Sets
Additional Key Content
Properties of Geometric Figures
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 79
3.2.3.12
McDougal Littell Math Course
Content/Standards Alignment
McDougal Littell Math Course
Program Organization and Design
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
6
0.653
0.604
0.563
0.600
0.625
0.635
7
0.722
0.594
0.719
0.650
0.625
0.736
8
0.667
0.531
0.646
0.567
0.542
0.663
Mathematical Expressions and Equations
Reasoning, Problem Solving, and Communication
6
Multiplication & Division of Fractions and Decimals
Ratios, Rates and Percents
Additional Key Content
Probability and Data
Reasoning, Problem Solving, and Communication
Rational Numbers and Linear Equations
7
McDougal Littell Math Course
Two- and Three-Dimensional Figures
Additional Key Content
Surface Area and Volume
Proportionality and Similarity
Additional Key Content
8
Linear Functions and Equations
Properties of Geometric Figures
Reasoning, Problem Solving, and Communication
Summary and Analysis of Data Sets
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 80
3.2.3.13
McDougal Littell Pre-/Algebra I
Content/Standards Alignment
McDougal Littell Pre-/Algebra 1
Program Organization and Design
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
7
0.722
0.719
0.677
0.625
0.667
0.668
8
0.767
0.667
0.675
0.740
0.740
0.736
Additional Key Content
Rational Numbers and Linear Equations
7
Reasoning, Problem Solving, and Communication
McDougal Littell Pre-/Algebra 1
Surface Area and Volume
Probability and Data
Proportionality and Similarity
Linear Functions and Equations
8
Additional Key Content
Reasoning, Problem Solving, and Communication
Summary and Analysis of Data Sets
Properties of Geometric Figures
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 81
3.2.3.14
Prentice Hall Mathematics
Content/Standards Alignment
Prentice Hall Mathematics
Program Organization and Design
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
6
0.833
0.688
0.719
0.725
0.733
0.808
7
0.611
0.542
0.542
0.550
0.517
0.639
8
0.694
0.656
0.583
0.617
0.658
0.754
Mathematical Expressions and Equations
Ratios, Rates and Percents
6
Reasoning, Problem Solving, and Communication
Additional Key Content
Two- and Three-Dimensional Figures
Additional Key Content
Rational Numbers and Linear Equations
Probability and Data
7
Prentice Hall Mathematics
Multiplication & Division of Fractions and Decimals
Proportionality and Similarity
Reasoning, Problem Solving, and Communication
Surface Area and Volume
Properties of Geometric Figures
8
Additional Key Content
Linear Functions and Equations
Reasoning, Problem Solving, and Communication
Summary and Analysis of Data Sets
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 82
3.2.3.15
Saxon Math Intermediate (Middle)
Content/Standards Alignment
Saxon Math Intermediate (Middle)
Program Organization and Design
Student Experience
Assessment
Instructional Planning and Professional Support
Equity and Access
0.00
0.25
0.50
0.75
1.00
Equity and Access
Instructional Planning
and Professional Support
Assessment
Student Experience
Program Organization
and Design
Content/Standards
Alignment
6
0.514
0.479
0.427
0.433
0.342
0.478
7
0.528
0.542
0.510
0.517
0.483
0.514
8
0.583
0.708
0.677
0.742
0.725
0.739
Additional Key Content
Ratios, Rates and Percents
6
Reasoning, Problem Solving, and Communication
Multiplication & Division of Fractions and Decimals
Mathematical Expressions and Equations
Reasoning, Problem Solving, and Communication
Rational Numbers and Linear Equations
Additional Key Content
7
Saxon Math Intermediate (Middle)
Two- and Three-Dimensional Figures
Surface Area and Volume
Proportionality and Similarity
Probability and Data
Additional Key Content
8
Linear Functions and Equations
Reasoning, Problem Solving, and Communication
Properties of Geometric Figures
Summary and Analysis of Data Sets
0.00
0.25
2008 Mathematics Instructional Materials Review Draft Report
0.50
0.75
1.00
Page 83
4 Data Analysis Methodology
4.1 Approach
Prior to data collection, we developed an analysis plan consisting of five main steps:
1. divide the data by grade level (elementary or middle school);
2. calculate the average score on standards items;
3. compare those scores to a threshold of 0.7;
4. calculate weighted average scores across all factors for those that surpass the
threshold; and
5. compare these remaining programs to determine the top 3 (or fewer).
In calculating both the standards score and overall weighted scores, we considered using
a linear mixed effects model to control for possible reviewer bias by including a random
intercept for reviewer. However, since the design is not complete – i.e., only some
reviewers review each program – we cannot fully separate reviewer effects and program
effects. Thus, if a particular reviewer happened to see only the strongest aligned
programs, their overall average score would be high, not because they were biased, but
because they scored strong programs. Adjusting for this would effectively be punishing
the programs that were seen by that reviewer. Thus, we chose to test for reviewer bias
first, and only use the adjusted model if there was evidence of severe bias. If not a simple
average or weighted average was to be used.
There are a number of legitimate ways to then compare the program scores, both to the
threshold of 0.7 and to each other. We hoped to keep the analysis relatively clear and
simple, to facilitate transparency of the report. To this end, we opted to use t-tests to
compare programs, a widely used and well understood method. In this study, we are
comparing averages of many scores for each program, which allows us to use a t-test
even though the data are not normally distributed. The results, threshold tests and
program comparisons for elementary and middle school, were kept to the traditional 0.05
significance level.
A significance level of 0.05 is meant to imply that we are willing to accept a 5% chance
that we will reach the wrong conclusions based on the data we collect. There are
theoretical results that show that this significance level is maintained when doing one or
more tests (controlling for multiple comparisons in the latter case) when the analysis plan
is constructed without looking at the data. Once analysis decisions are made based on
what we see in the data itself, we no longer can make the assumptions necessary to know
the distribution of outcomes. In this case, p-values no longer carry the meaning they did
when we planned our analysis in advance; we cannot make rigorous conclusions about
the statistical significance of a result.
4.2 Response Scales
In the data collection, Content/Standards Alignment (hereafter “content”) questions were
rated on a Not met/Partial/Met scale, while other factors (Assessment, Equity and Access,
Instructional Development and Professional Support, Program Organization and Design,
2008 Mathematics Instructional Materials Review Draft Report
Page 84
and Balance of Student Experience) were rated on a 4 point Likert scale. These are
ordinal variables, and not inherently numeric. In the analysis that follows, we assume
that the “distance” between two consecutive levels is the same across a scale. That is, the
value added by moving from “Not met” to “Partial” is the same as moving from “Partial”
to “Met” in the standards. Similarly, the value added moving from “Strongly disagree” to
“Disagree” is the same as from “Disagree” to “Agree” on the Likert Scale.
4.3 Distributions of Scores by Grade Level
In Table 20 and Table 21 we see characteristics of the distribution of scores for
elementary and middle school programs broken down by the two scales. The unweighted
average scores are similar across the two scales, with somewhat more variability in the
content scores. We can assess the normality of the distributions, and important
assumption for hypothesis tests, by considering the skewness and kurtosis. Both should
be about zero if the distribution is normal. The distributions for content deviate more
seriously from normality than do the other factors. This can be seen more clearly in
Figure 6.
Table 20. Score distribution characteristics for elementary programs by Content/Standards Alignment
and Other Factors.
Statistic
Mean (unweighted)
Standard deviation
Skewness
Kurtosis
Content
0.581
0.385
-0.285
-1.266
Other Factors
0.564
0.261
-0.323
-0.217
Table 21. Score distribution characteristics for middle school programs by Content/Standards Alignment
and Other Factors.
Statistic
Mean (unweighted)
Standard deviation
Skewness
Kurtosis
Content
0.606
0.402
-0.403
-1.342
2008 Mathematics Instructional Materials Review Draft Report
Other factors
0.593
0.263
-0.388
-0.154
Page 85
Figure 6. Score distributions for Content/Standards Alignment and Other Factors.
While the distributions are not normal, we will be comparing averages over hundreds of
scores, which should make assumptions of normality not unreasonable.
4.4 Reviewer Bias
As described in detail within this section, we have concluded that there is no evidence of
reviewer bias. Table 22 gives the distribution of scores (Met/Partial/Not met) by reviewer
on content items. There appear to be a few reviewers (438, 439 and 440) who tended to
assign higher scores, but given the overall tendency for more standards to be rated at
“Met” or “Partial”, this likely does not indicate bias on the part of these reviewers. We
2008 Mathematics Instructional Materials Review Draft Report
Page 86
might reasonably expect a few reviewers to review only primarily strongly aligned
programs merely by chance. Moreover, these three reviewers reviewed almost all of the
same programs, so we would expect that their score distributions would be similar.
The following detailed analysis shows that there is no evidence of review bias.
Table 22. Distribution of scores by level on Content/Standards Alignment by reviewer.
Reviewer
11
25
36
44
47
76
86
88
91
98
105
119
125
136
151
157
191
199
206
211
212
217
221
224
232
278
282
297
300
317
320
331
345
354
355
376
406
426
431
438
Not met
35.7%
12.2%
22.3%
20.5%
13.8%
29.2%
10.2%
33.4%
32.3%
22.8%
32.8%
30.5%
21.5%
19.7%
18.8%
11.0%
17.7%
13.6%
8.0%
37.8%
36.0%
21.9%
37.8%
12.5%
15.0%
26.1%
26.4%
14.6%
52.0%
22.3%
15.1%
41.2%
28.7%
31.1%
17.0%
19.6%
32.5%
16.3%
26.0%
6.9%
2008 Mathematics Instructional Materials Review Draft Report
Score
Partial
29.5%
33.0%
26.0%
23.3%
22.1%
43.2%
27.2%
35.9%
43.0%
33.8%
40.8%
31.2%
56.5%
53.1%
39.9%
47.6%
47.7%
27.2%
54.7%
41.2%
36.8%
49.4%
33.0%
33.4%
43.9%
32.9%
40.5%
36.8%
14.6%
48.8%
26.8%
30.7%
30.5%
34.4%
26.5%
28.7%
37.1%
52.0%
29.5%
13.8%
Met
34.9%
54.8%
51.7%
56.2%
64.1%
27.6%
62.6%
30.6%
24.6%
43.5%
26.4%
38.3%
22.0%
27.2%
41.3%
41.4%
34.6%
59.2%
37.2%
21.1%
27.3%
28.7%
29.2%
54.1%
41.0%
41.0%
33.0%
48.6%
33.3%
28.9%
58.2%
28.1%
40.7%
34.4%
56.4%
51.7%
30.4%
31.6%
44.5%
79.3%
Page 87
Reviewer
439
440
Total
Not met
9.5%
9.4%
23.3%
Score
Partial
21.1%
26.3%
35.4%
Met
69.5%
64.4%
41.4%
Table 23 gives the distribution of scores (strongly disagree, disagree, agree, strongly
agree) by reviewer on Other Factors. There is substantial variability in the distribution of
scores across reviewers, but no single reviewer seems to stand out from the pack,
indicating that it is reasonable to assume that the variation results from reviewers
reviewing different subsets of texts, rather than a particular bias on a reviewer’s part.
Table 23. Distribution of scores by level on other factors by reviewer.
Score
Reviewer
11
25
36
44
47
76
86
88
91
98
105
119
125
136
151
157
191
199
206
211
212
217
221
224
232
278
282
297
300
317
320
331
345
Strongly
disagree
5.8%
3.3%
13.4%
5.4%
4.6%
2.2%
6.5%
8.9%
11.4%
5.6%
0.8%
11.5%
3.4%
13.0%
7.4%
1.5%
6.9%
4.8%
0.7%
10.4%
27.9%
14.0%
7.5%
4.8%
1.3%
14.3%
18.8%
0.3%
0.4%
7.4%
7.1%
11.4%
16.7%
Disagree
34.7%
17.4%
27.9%
19.8%
16.7%
38.9%
18.7%
29.9%
32.3%
13.9%
31.0%
22.0%
28.3%
27.3%
26.5%
20.4%
34.3%
26.6%
21.4%
21.6%
22.4%
36.9%
39.0%
37.2%
33.3%
18.4%
31.0%
29.4%
41.8%
25.4%
13.6%
55.7%
24.6%
2008 Mathematics Instructional Materials Review Draft Report
Agree
56.0%
63.1%
53.7%
60.6%
67.5%
52.2%
56.2%
50.0%
33.2%
59.3%
62.4%
48.5%
59.5%
40.9%
56.3%
70.5%
54.8%
64.5%
61.0%
39.4%
24.5%
38.8%
51.5%
57.7%
65.2%
41.8%
41.2%
63.8%
40.8%
57.9%
30.5%
32.4%
30.4%
Strongly
Agree
3.6%
16.2%
5.0%
14.2%
11.2%
6.7%
18.6%
11.3%
23.1%
21.2%
5.8%
18.0%
8.7%
18.8%
9.8%
7.6%
4.0%
4.1%
16.8%
28.6%
25.2%
10.2%
2.0%
0.3%
0.2%
25.5%
9.0%
6.6%
17.0%
9.3%
48.7%
0.5%
28.4%
Page 88
Score
Reviewer
354
355
376
406
426
431
438
439
440
Total
Strongly
disagree
1.9%
4.0%
0.0%
11.9%
0.7%
6.5%
3.4%
4.2%
2.8%
7.0%
Disagree
37.0%
27.9%
17.5%
32.7%
28.3%
44.8%
19.0%
20.1%
14.3%
27.3%
Agree
53.2%
54.8%
75.7%
42.5%
61.2%
45.7%
50.5%
55.3%
46.1%
51.8%
Strongly
Agree
7.9%
13.3%
6.9%
12.9%
9.8%
3.0%
27.0%
20.4%
36.8%
13.9%
We can confirm visually that no single reviewer stands apart from the rest from Figure 7,
which gives the average score by reviewer with bands of one standard deviation (SD)
indicating the variability for each reviewer.
2008 Mathematics Instructional Materials Review Draft Report
Page 89
Figure 7. Mean score by reviewer.
In order to test whether any reviewer had a tendency to over- or under-rate, we calculated
a standardized score within text for each reviewer, and performed a t-test comparing each
average standardized score to 0 to test whether the reviewer tended to score away from
the mean. The results are shown in Table 24 and Table 25 for Content/Standards
Alignment and Other Factors, respectively. Since we are performing tests for the 42
reviewers, it is important to adjust for multiple comparisons to avoid finding a difference
significant when it could have happened by chance when drawing 42 means from the
same distribution. The tables give the adjusted significance level, calculated using the
Holm-Bonferroni method, in which we compare the ordered p-values to the nominal
significance level (0.05) divided by the number of tests remaining. As soon as one test is
deemed insignificant, the rest are also. In this case, we see that even the smallest p-value
in each table does not reach the adjusted significance level of 0.05/42, so we can
conclude that there is no evidence of reviewer bias.
2008 Mathematics Instructional Materials Review Draft Report
Page 90
Table 24. Hypothesis tests for reviewer bias on content.
Reviewer
331
300
11
191
317
199
91
157
25
36
76
206
431
426
355
151
440
221
105
297
345
136
125
320
224
282
376
86
98
44
278
354
212
438
47
88
406
217
439
211
119
232
t-statistic
-4.082
-3.18
-2.94
-2.08
1.97
-1.90
-1.82
1.68
1.67
-1.60
-1.53
-1.46
-1.45
1.36
1.35
-1.34
1.20
-1.18
1.16
-1.04
-0.95
-0.95
0.88
0.87
0.84
-0.75
-0.72
-0.60
0.55
-0.53
-0.50
-0.49
0.43
0.39
0.32
-0.28
-0.28
-0.25
-0.18
0.14
-0.06
-0.02
df
10
13
12
12
9
11
14
16
10
11
12
13
11
10
10
9
11
13
9
9
12
11
9
11
8
10
9
15
11
18
11
9
10
9
17
11
12
10
9
11
16
11
p-value
0.002
0.007
0.012
0.059
0.080
0.084
0.091
0.112
0.125
0.139
0.151
0.167
0.175
0.203
0.206
0.213
0.255
0.260
0.275
0.324
0.358
0.365
0.399
0.402
0.423
0.473
0.491
0.556
0.593
0.604
0.629
0.636
0.680
0.707
0.755
0.782
0.784
0.805
0.858
0.894
0.951
0.987
2008 Mathematics Instructional Materials Review Draft Report
Tests
remaining
42
41
40
39
38
37
36
35
34
33
32
31
30
29
28
27
26
25
24
23
22
21
20
19
18
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
Significance
cutoff
0.0012
0.0012
0.0013
0.0013
0.0013
0.0014
0.0014
0.0014
0.0015
0.0015
0.0016
0.0016
0.0017
0.0017
0.0018
0.0019
0.0019
0.0020
0.0021
0.0022
0.0023
0.0024
0.0025
0.0026
0.0028
0.0029
0.0031
0.0033
0.0036
0.0038
0.0042
0.0045
0.0050
0.0056
0.0063
0.0071
0.0083
0.0100
0.0125
0.0167
0.0250
0.0500
Page 91
Table 25. Hypothesis tests for reviewer bias on Other Factors.
Reviewer
331
300
224
439
354
86
91
221
406
320
157
47
355
440
376
105
438
431
282
232
199
136
36
206
125
44
278
25
345
98
297
11
191
426
119
217
88
212
76
211
317
151
t-statistic
-4.26
-3.33
3.63
3.18
-3.09
2.63
-2.66
-2.57
-2.41
2.31
2.16
2.12
2.12
2.05
2.05
-1.77
1.56
-1.53
-1.46
1.44
1.37
-1.21
1.09
1.07
-1.07
0.96
0.89
0.88
-0.82
0.69
0.66
-0.64
0.60
0.57
-0.51
0.47
-0.45
-0.36
0.33
-0.29
0.26
0.25
df
10
13
8
9
9
15
14
13
12
11
16
17
10
11
9
9
9
11
10
11
11
11
11
13
9
18
11
10
12
11
9
12
12
10
16
10
11
10
12
11
9
9
p-value
0.002
0.005
0.007
0.011
0.013
0.019
0.019
0.023
0.033
0.041
0.047
0.049
0.060
0.065
0.071
0.110
0.152
0.155
0.175
0.178
0.197
0.252
0.299
0.303
0.312
0.351
0.391
0.397
0.427
0.504
0.525
0.533
0.560
0.580
0.618
0.649
0.662
0.729
0.750
0.780
0.797
0.807
2008 Mathematics Instructional Materials Review Draft Report
Tests
remaining
42
41
40
39
38
37
36
35
34
33
32
31
30
29
28
27
26
25
24
23
22
21
20
19
18
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
Significance
cutoff
0.0012
0.0012
0.0013
0.0013
0.0013
0.0014
0.0014
0.0014
0.0015
0.0015
0.0016
0.0016
0.0017
0.0017
0.0018
0.0019
0.0019
0.0020
0.0021
0.0022
0.0023
0.0024
0.0025
0.0026
0.0028
0.0029
0.0031
0.0033
0.0036
0.0038
0.0042
0.0045
0.0050
0.0056
0.0063
0.0071
0.0083
0.0100
0.0125
0.0167
0.0250
0.0500
Page 92
4.5 Content/Standards Alignment
The first step in our analysis was to evaluate the alignment of each program with the state
math standards. Table 26 gives the average score on Content/Standards Alignment items
for elementary programs, along with the 95% normal confidence interval for the mean.
Table 26. Summary of Content/Standards Alignment scores for elementary programs.
Program
Math Connects (Elem)
Bridges in Mathematics
Investigations
Math Expressions
Saxon Math (Elem)
Everyday Mathematics
envision
Progress in Mathematics
Growing with Mathematics
Math Out of the Box
Math Trailblazers
Singapore Math Standards
Mean
0.734
0.715
0.628
0.624
0.604
0.592
0.576
0.575
0.560
0.529
0.517
0.355
Std. dev.
0.347
0.331
0.384
0.385
0.387
0.389
0.378
0.385
0.371
0.387
0.376
0.358
N
811
848
848
808
848
848
848
845
853
885
814
839
Std. err.
0.012
0.011
0.013
0.014
0.013
0.013
0.013
0.013
0.013
0.013
0.013
0.012
95% CI
Lower
Upper
0.710
0.758
0.692
0.737
0.602
0.654
0.598
0.651
0.578
0.630
0.566
0.618
0.550
0.601
0.549
0.601
0.536
0.585
0.504
0.555
0.491
0.543
0.330
0.379
In Table 27 we see means and confidence intervals for the middle school programs.
Table 27. Summary of Content/Standards Alignment scores for middle school programs.
Program
Holt Mathematics
Math Connects (Middle)
Prentice Hall Mathematics
Math Thematics
McDougal Littell Math Course
Connected Mathematics 2
Impact
Cognitive Tutor/Carnegie Algebra
Saxon Math Intermediate (Middle)
CPM Middle Grades Program
Everyday Math/Transition
ALEKS, Online Mathematics Solution
Mathscape
Mean
0.855
0.741
0.736
0.714
0.676
0.653
0.604
0.589
0.570
0.512
0.391
0.386
0.368
Std. dev.
0.267
0.341
0.338
0.357
0.384
0.385
0.402
0.417
0.415
0.370
0.362
0.405
0.390
N
428
428
428
426
428
428
395
428
428
428
467
428
428
Std. err.
0.013
0.017
0.016
0.017
0.019
0.019
0.020
0.020
0.020
0.018
0.017
0.020
0.019
95% CI
Lower
Upper
0.830
0.881
0.708
0.773
0.700
0.768
0.680
0.748
0.640
0.713
0.617
0.690
0.564
0.644
0.549
0.628
0.531
0.609
0.477
0.547
0.358
0.424
0.347
0.424
0.331
0.405
4.6 Threshold Tests
The tables below give the results of t-tests comparing the average Content/Standards
Alignment scores for elementary and middle school programs, respectively, to the
threshold value of 0.7.
2008 Mathematics Instructional Materials Review Draft Report
Page 93
Of the elementary programs, only the top two, Math Connects (Elem) and Bridges in
Mathematics, make the cutoff, and the rest are statistically significantly less than 0.7.
Four middle school programs (Holt Mathematics, Math Connects (Middle), Prentice Hall
Mathematics and Math Thematics) have average scores above the 0.7 threshold. A fifth,
McDougal-Littell Math Course, is not significantly less than 0.7, and thus should be
considered to have made the cutoff.
Table 28. t-test results comparing Content/Standards Alignment scores of elementary programs to the
threshold value of 0.7.
Program Name
Singapore Math Standards
Math Trailblazers
Math Out of the Box
Growing with Mathematics
Envision
Progress in Mathematics
Everyday Mathematics
Saxon Math (Elem)
Math Expressions
Investigations
Bridges in Mathematics
Math Connects (Elem)
Mean
score
0.355
0.517
0.529
0.560
0.576
0.575
0.592
0.604
0.624
0.628
0.715
0.734
t
statistic
-27.85
-13.85
-13.12
-10.99
-9.58
-9.46
-8.06
-7.19
-5.56
-5.46
1.28
2.81
Degrees
of
freedom
838
813
884
852
847
844
847
847
807
847
847
810
p-value
1.09E-121
1.30E-39
2.25E-36
1.12E-26
5.34E-21
1.46E-20
1.30E-15
7.22E-13
1.85E-08
3.09E-08
0.900
0.997
# tests
remaining
12
11
10
9
8
7
6
5
4
3
2
1
Significance
cutoff
0.004
0.005
0.005
0.006
0.006
0.007
0.008
0.010
0.013
0.017
0.025
0.050
Table 29. t-test results comparing Content/Standards Alignment scores of middle school programs to
the threshold value of 0.7.
Program Name
Everyday Math/Transition
Mathscape
ALEKS, Online Mathematics Solution
CPM Middle Grades Program
Saxon Math Intermediate (Middle)
Cognitive Tutor/Carnegie Algebra
Impact
Connected Mathematics 2
McDougal Littell Math Course
Math Thematics
Prentice Hall Mathematics
Math Connects (Middle)
Holt Mathematics
Mean
score
0.391
0.368
0.386
0.512
0.570
0.589
0.604
0.653
0.676
0.714
0.736
0.741
0.855
t
statistic
-18.51
-17.66
-16.05
-10.52
-6.49
-5.53
-4.76
-2.53
-1.27
0.79
2.21
2.47
12.02
2008 Mathematics Instructional Materials Review Draft Report
Degrees
of
freedom
466
427
427
427
427
427
394
427
427
425
427
427
427
p-value
4.55E-58
4.26E-53
5.55E-46
1.81E-23
1.16E-10
2.76E-08
1.35E-06
0.006
0.103
0.784
0.986
0.993
1
# tests
remaining
13
12
11
10
9
8
7
6
5
4
3
2
1
Signific
ance
cutoff
0.004
0.004
0.005
0.005
0.006
0.006
0.007
0.008
0.010
0.013
0.017
0.025
0.050
Page 94
4.7 Calculation of Program Means and Standard Errors
For the comparison of programs, we considered the weighted averages of scores across
all scales and their standard errors. The six scales are weighted as shown in Table 30.
The average score for each program was calculated as the weighted sum of the average
scores in the six scales.
Table 30. Scale weights for overall averages.
Scale
Assessment
Content/Standards Alignment
Equity and Access
Instructional Planning and Professional
Support
Program Organization and Design
Balance of Student Experience
Weight
0.05
0.7
0.04
0.045
0.9
0.75
To calculate the standard error of the average score for each program, we first took the
variance of the average score for each scale. The variance for the program is then the
sum of the square of the weight for the scale from Table 30 times the variance of the
scale. The standard error is then the square root of this value. See Section 4.9 Standard
Error Calculations for more information.
4.8 Program Comparison
Since the goal was to identify no more than three program recommendations, we needed
to test for any statistical ties for third place. To do this, we compared the scores of the
lower-ranked programs to the third-ranked (as determined by the weighted average score
across scales). We performed the comparisons using t-tests, adjusting for multiple
comparisons using the Holm-Bonferroni method. To do so, we compared the ordered pvalues to the nominal significance level (0.05) divided by the number of tests remaining.
As soon as one test was deemed insignificant, the rest were as well. In Table 31 we see
that all lower-ranked elementary programs except Math Expressions are significantly
different from the third-ranked program, Investigations.
The Welch-Sattherwaite equation gives us an approximation to the degrees of freedom
for a t-test comparing weighted averages.
Take
and
to be the standard errors of the two programs to be compared.
The degrees of freedom are then given by
where
2008 Mathematics Instructional Materials Review Draft Report
Page 95
The index ranges over the six response scales.
is the category weight, is the
number of scores in that category and is the standard deviation of observations in that
category.
Table 31. t tests comparing elementary programs to the third-ranked program, with remaining
comparisons and significance cutoff calculated by the Holm-Bonferroni correction for multiple
comparisons.
Program
Math Connects (Elem)
Bridges in Mathematics
Investigations
Singapore Math Standards
Math Out of the Box
Math Trailblazers
Progress in Mathematics
Envision
Growing with Mathematics
Saxon Math (Elem)
Everyday Mathematics
Math Expressions
Mean
score
0.724
0.687
0.635
0.365
0.521
0.521
0.547
0.568
0.575
0.581
0.593
0.621
t statistic
Degrees of
freedom
-20.70
-8.55
-8.51
-6.23
-4.78
-4.60
-3.99
-3.15
-1.06
628
681
647
659
705
643
636
687
709
p-value
2.79E-73
4.03E-17
5.99E-17
4.06E-10
1.06E-06
2.57E-06
3.69E-05
8.55E-04
0.146
# tests
remaining
Significance
cutoff
9
8
7
6
5
4
3
2
1
0.006
0.006
0.007
0.008
0.010
0.013
0.017
0.025
0.050
Table 32 gives the t-test results for the middle school programs. Again, all lower-ranked
elementary programs except Math Thematics are significantly different from the thirdranked program, Prentice Hall Mathematics.
Table 32. t tests comparing middle school programs to the third-ranked program, with remaining
comparisons and significance cutoff calculated by the Holm-Bonferroni correction for multiple
comparisons.
Program
Holt Mathematics
Math Connects (Middle)
Prentice Hall Mathematics
Everyday Math/Transition
ALEKS, Online Mathematics Solution
Mathscape
CPM Middle Grades Program
Saxon Math Intermediate (Middle)
Cognitive Tutor/Carnegie Algebra
Impact
Connected Mathematics 2
McDougal Littell Math Course
Mean
score
0.837
0.723
0.707
0.419
0.392
0.416
0.511
0.562
0.592
0.605
0.625
0.658
t
statistic
Degrees
of
freedom
-16.96
-17.04
-16.06
-11.25
-7.72
-6.17
-5.44
-4.56
-2.77
305
279
270
316
291
322
308
270
322
2008 Mathematics Instructional Materials Review Draft Report
p-value
3.55E-46
2.10E-45
1.67E-41
3.19E-25
9.31E-14
1.04E-09
5.53E-08
3.88E-06
0.003
# tests
remaining
Significance
cutoff
10
9
8
7
6
5
4
3
2
0.005
0.006
0.006
0.007
0.008
0.010
0.013
0.017
0.025
Page 96
Mean
score
0.690
Program
Math Thematics
t
statistic
-1.01
Degrees
of
freedom
330
p-value
0.155
# tests
remaining
1
Significance
cutoff
0.050
In addition to testing for the ties with the third-ranked program, we needed to compare
the second-ranked program to subsequent ones.
Table 33 and Table 34 below give the results of comparing lower-scoring programs to the
second-ranked program for elementary and middle school, respectively. We again use
the Holm-Bonferroni adjustment for multiple comparisons. There is a clear separation
between the top two elementary programs and the rest; there are no statistical ties with
Bridges in Mathematics, the second-ranked program. In middle school, however, there
are two programs, Prentice Hall Mathematics and Math Thematics, which are not
significantly different from the second-ranked program.
Table 33. t-test results comparing lower-scoring texts to the second-highest scoring elementary school
text.
Program Name
Math Connects (Elem)
Bridges in Mathematics
Singapore Math Standards
Math Out of the Box
Math Trailblazers
Progress in Mathematics
Growing with Mathematics
enVision
Saxon Math (Elem)
Everyday Mathematics
Math Expressions
Investigations
Mean
score
0.724
0.687
0.365
0.521
0.521
0.547
0.575
0.568
0.581
0.593
0.621
0.635
t statistic
Degrees of
freedom
-26.28
-13.20
-13.12
-10.43
-9.05
-8.93
-8.28
-7.40
-5.17
-4.09
513
563
534
553
528
593
526
570
590
571
p-value
# tests
remaining
2.39E-97
3.70E-35
1.34E-34
1.10E-23
1.35E-18
2.72E-18
5.09E-16
2.43E-13
1.64E-07
2.43E-05
Significance
cutoff
10
9
8
7
6
5
4
3
2
1
0.005
0.006
0.006
0.007
0.008
0.010
0.013
0.017
0.025
0.050
Table 34. t-test results comparing lower-scoring texts to the second-highest scoring middle school text.
Program Name
Holt Mathematics
Math Connects (Middle)
Everyday Math/Transition
ALEKS, Online Mathematics Solution
Mathscape
CPM Middle Grades Program
Saxon Math Intermediate (Middle)
Cognitive Tutor/Carnegie Algebra
Mean
score
0.837
0.723
0.419
0.392
0.416
0.511
0.562
0.592
t
statistic
Degrees
of
freedom
-17.88
-17.88
-16.93
-12.16
-8.57
-7.02
320
289
279
331
301
333
2008 Mathematics Instructional Materials Review Draft Report
p-value
2.24E-50
6.30E-49
5.26E-45
1.20E-28
2.76E-16
6.16E-12
# tests
remaining
11
10
9
8
7
6
Significance
cutoff
0.005
0.005
0.006
0.006
0.007
0.008
Page 97
Program Name
Impact
Connected Mathematics 2
McDougal Littell Math Course
Math Thematics
Prentice Hall Mathematics
Mean
score
0.605
0.625
0.658
0.690
0.707
t
statistic
-6.29
-5.45
-3.67
-1.96
-0.96
Degrees
of
freedom
318
279
336
348
299
p-value
5.21E-10
5.58E-08
1.43E-04
0.026
0.168
# tests
remaining
5
4
3
2
1
Significance
cutoff
0.010
0.013
0.017
0.025
0.050
4.9 Standard Error Calculations
This section describes several methodological variants to calculate standard error. The
recommended approach is the most straightforward. The more complex variants take into
account assumptions about dependence in the data, but ultimately show that the same
outcomes occur. Thus, we elected to use the approach recommended below.
4.9.1 Recommended Approach
4.9.1.1 Methodology
( p)
Let X ijkl be the score for program p on item l for scale i, grade j, by rater k.
Here:
•
p indexes the 31 curricula
•
i = 1,…, 6, indexes the 6 scales assessed (Content/Standards Alignment,
Equity and Access, etc.)
•
j = 1,…,J, indexes the grade levels.
•
J=6 or 3 for elementary or middle school programs, respectively.
•
k = 1,…,Kj. Kj indexes the reviewers, and ranges from 3 to 5 depending on
the text and grade level.
•
l = 1,…,Lij. Lij index the number of items scored, and varies depending upon
the grade level and scale.
The final weighted average score for program p is
X w( p )
6
i 1
wi X i...
where wi is the weight given to scale i, and X i ... is the average rating given on items in
scale i on program p, averaged over grade levels and raters.
More formally,
2008 Mathematics Instructional Materials Review Draft Report
Page 98
X w( p )
6
i 1
wi
J
Kj
L ij
j 1
k 1
l 1
X ijkl /N i
,
where
J
Ni
j 1
K j Lij
is the number of item scores on scale i for program p.
4.9.1.2 Variance and standard error of weighted average for final score
The precision with which the final score for program p can be assessed depends upon the
number of ratings and the variability of the ratings. More ratings correspond to higher
precision (lower variance and standard error). Lower variability of ratings, indicating
greater agreement among ratings, corresponds to higher precision. In addition, the
weights given to the 6 different categories impact the variance and standard error. Note
also that the standard error (SE) is the square root of the variance of the average.
For the current problem, the variance for the weighted average X w( p ) (Final Score for
program p) can be computed as follows.
6
Var(X w( p ) )
2
i 1
wi Var(X i...)
( p)
Three assumptions are inherent in this computation: (1) independence of the ratings X ijkl
(2) independence of scales, and (3) all items within a scale are assessing program p on
category i (in other words, all items are independent and identically distributed measures
of a true scale average for program p).
2
Var(X i...)
The usual estimator for
2
i
i
/N i .
2
is the sample variance si , computed from the N i scores
( p)
X ijkl
Thus the estimated standard error (SE) for X w( p ) , the Final Score for program p is
6
i 1
2
2
w i si /N i
4.9.1.3 Results
Table 35 and Table 36 below give the standard errors and 95% confidence intervals for
the program means for elementary and middle school programs, using the above
methodology. It is worth noting that in both grade levels the confidence interval for the
top-rated program does not overlap with the second-rated program. Similarly, the
2008 Mathematics Instructional Materials Review Draft Report
Page 99
second-rated program in elementary school does not overlap with the third-rated
program.
Table 35. Confidence interval values for all comprehensive elementary programs.
Values
95% Confidence Interval
Final
Program Name
Score
Std Error
Min
Max
Math Connects (Elem)
0.7245
0.0088
0.7073
0.7417
Bridges in Mathematics
0.6872
0.0083
0.6709
0.7036
Investigations
0.6355
0.0095
0.6169
0.6541
Math Expressions
0.6212
0.0097
0.6022
0.6401
Everyday Mathematics
0.5929
0.0096
0.5740
0.6118
Saxon Math (Elem)
0.5814
0.0097
0.5624
0.6004
Growing with Mathematics
0.5746
0.0092
0.5566
0.5927
envision
0.5681
0.0022
0.5638
0.5725
Progress in Mathematics
0.5472
0.0105
0.5266
0.5678
Math Out of the Box
0.5212
0.0094
0.5028
0.5397
Math Trailblazers
0.5211
0.0095
0.5024
0.5397
Singapore Math Standards
0.3647
0.0090
0.3471
0.3824
Grand Total
0.5782
0.0028
0.5728
0.5837
Table 36. Confidence interval values for all comprehensive middle school programs.
Values
95% Confidence Interval
Final
Program Name
Score
Std Error
Min
Max
Holt Mathematics
0.8370
0.0094
0.8185
0.8555
Math Connects (Middle)
0.7234
0.0119
0.7001
0.7467
Prentice Hall Mathematics
0.7073
0.0119
0.6840
0.7305
Math Thematics
0.6899
0.0123
0.6657
0.7141
McDougal Littell Math Course
0.6579
0.0133
0.6318
0.6841
Connected Mathematics 2
0.6250
0.0136
0.5984
0.6517
Impact
0.6054
0.0145
0.5770
0.6338
Cognitive Tutor/Carnegie Algebra
0.5920
0.0144
0.5637
0.6203
Saxon Math Intermediate (Middle)
0.5624
0.0145
0.5339
0.5909
CPM Middle Grades Program
0.5106
0.0129
0.4854
0.5358
Everyday Math/Transition
0.4194
0.0121
0.3956
0.4432
Mathscape
0.4158
0.0137
0.3889
0.4428
2008 Mathematics Instructional Materials Review Draft Report
Page 100
Values
95% Confidence Interval
Final
Program Name
Score
Std Error
Min
Max
ALEKS, Online Mathematics Solution
0.3917
0.0142
0.3637
0.4196
Grand Total
0.5939
0.0039
0.5863
0.6015
Table 37 and Table 38 give the t-test results (methodology described in Section 4.8
Program Comparison) comparing all lower-rated programs to the third-rated program,
again by grade level.
All lower-rated elementary programs are significantly different from the third-ranked text
except Math Expressions, and all lower-rated middle school programs are significantly
different from the third-ranked text except Math Thematics.
Table 37. t-test results comparing lower-scoring texts to the third-highest scoring elementary school
text.
Degrees
Program
Mean
t
of
score
statistic
freedom
p-value
# tests
Significance
remaining
cutoff
Math Connects (Elem)
0.724
Bridges in Mathematics
0.687
Investigations
0.635
Singapore Math Standards
0.365
-20.70
628
2.79E-73
9
0.006
Math Out of the Box
0.521
-8.55
681
4.03E-17
8
0.006
Math Trailblazers
0.521
-8.51
647
5.99E-17
7
0.007
Progress in Mathematics
0.547
-6.23
659
4.06E-10
6
0.008
enVision
0.568
-4.78
705
1.06E-06
5
0.010
Growing with Mathematics
0.575
-4.60
643
2.57E-06
4
0.013
Saxon Math (Elem)
0.581
-3.99
636
3.69E-05
3
0.017
0.5929
-3.15
687
8.55E-04
2
0.025
0.6212
-1.06
709
0.146
1
0.050
Everyday Mathematics
Math Expressions
Table 38 t-test results comparing lower-scoring texts to the third-highest scoring middle school text.
Degrees
Program
Mean
t
of
score
statistic
freedom
Holt Mathematics
0.837
Math Connects (Middle)
0.723
Prentice Hall Mathematics
0.707
Everyday Math/Transition
0.419
-16.96
2008 Mathematics Instructional Materials Review Draft Report
305
p-value
3.55E-46
# tests
Significance
remaining
cutoff
10
0.005
Page 101
Degrees
Program
Mean
t
of
score
statistic
freedom
p-value
# tests
Significance
remaining
cutoff
ALEKS, Online Mathematics Solution
0.392
-17.04
279
2.10E-45
9
0.006
Mathscape
0.416
-16.06
270
1.67E-41
8
0.006
CPM Middle Grades Program
0.511
-11.25
316
3.19E-25
7
0.007
Saxon Math Intermediate (Middle)
0.562
-7.72
291
9.31E-14
6
0.008
Cognitive Tutor/Carnegie Algebra
0.592
-6.17
322
1.04E-09
5
0.010
Impact
0.605
-5.44
308
5.53E-08
4
0.013
Connected Mathematics 2
0.625
-4.56
270
3.88E-06
3
0.017
McDougal Littell Math Course
0.658
-2.77
322
0.003
2
0.025
Math Thematics
0.690
-1.01
330
0.155
1
0.050
4.9.2 Without assuming independence of scales
4.9.2.1 Motivation
We might expect that a program that scores well on one scale would also score well on
another scale, simply because it is a high-quality program. This would indicate that
program scores on the six scales are not independent. In Table 39 we see the correlations
between the six scales. With correlations ranging from 0.41 to 0.84, it is unlikely that the
scales are independent.
Table 39. Scale correlations.
Assessment
Content
Equity and
Planning and
Program
Student
Access
Support
Organization
Experience
Assessment
1.00
Content
0.41
1.00
Equity
0.50
0.47
1.00
Planning
0.56
0.59
0.50
1.00
Program
0.55
0.66
0.53
0.79
1.00
Student
0.60
0.68
0.53
0.78
0.84
4.9.2.2 Methodology
The assumption of independence of the scales is what allows us to say that
Var(X w( p ) )
6
i 1
2
wi Var(X i...)
Without that assumption, we should adjust the variance for the covariances of the scales
by taking:
2008 Mathematics Instructional Materials Review Draft Report
Page 102
1.00
6
6
Var ( X i... )
wi wmCov(X i... , X m... )
i 1 m 1
Note that
Cov( X i... , X i... ) Var ( X i... )
4.9.2.3 Results
Table 40 through Table 43 give the confidence interval and t-test results using this
modified standard error calculation. We see that the results remain the same as above,
except that the confidence intervals for the top two elementary programs now overlap
slightly.
Table 40. Confidence interval values for all comprehensive elementary programs.
Values
95% Confidence Interval
Final
Program Name
Score
Std Error
Min
Max
Math Connects (Elem)
0.7245
0.0108
0.7034
0.7456
Bridges in Mathematics
0.6872
0.0107
0.6662
0.7082
Investigations
0.6355
0.0117
0.6126
0.6583
Math Expressions
0.6212
0.0115
0.5987
0.6436
Everyday Mathematics
0.5929
0.0118
0.5699
0.6159
Saxon Math (Elem)
0.5814
0.0121
0.5576
0.6052
Growing with Mathematics
0.5746
0.0115
0.5520
0.5973
enVision
0.5681
0.0126
0.5435
0.5927
Progress in Mathematics
0.5472
0.0130
0.5217
0.5727
Math Out of the Box
0.5212
0.0116
0.4985
0.5440
Math Trailblazers
0.5211
0.0118
0.4979
0.5442
Singapore Math Standards
0.3647
0.0114
0.3423
0.3871
Grand Total
0.5782
0.0035
0.5714
0.5850
Table 41. Confidence interval values for all comprehensive middle school programs.
Values
95% Confidence Interval
Final
Program Name
Score
Std Error
Min
Max
Holt Mathematics
0.8370
0.0120
0.8135
0.8605
Math Connects (Middle)
0.7234
0.0147
0.6947
0.7521
Prentice Hall Mathematics
0.7073
0.0150
0.6779
0.7366
2008 Mathematics Instructional Materials Review Draft Report
Page 103
Values
95% Confidence Interval
Final
Program Name
Score
Std Error
Min
Max
Math Thematics
0.6899
0.0146
0.6613
0.7185
McDougal Littell Math Course
0.6579
0.0162
0.6262
0.6896
Connected Mathematics 2
0.6250
0.0173
0.5911
0.6589
Impact
0.6054
0.0174
0.5712
0.6396
Cognitive Tutor/Carnegie Algebra
0.5920
0.0174
0.5579
0.6262
Saxon Math Intermediate (Middle)
0.5624
0.0180
0.5271
0.5976
CPM Middle Grades Program
0.5106
0.0156
0.4800
0.5412
Everyday Math/Transition
0.4194
0.0153
0.3893
0.4494
Mathscape
0.4158
0.0175
0.3815
0.4501
ALEKS, Online Mathematics Solution
0.3917
0.0177
0.3570
0.4264
Grand Total
0.5939
0.0048
0.5844
0.6034
Table 42. t-test results comparing lower-scoring texts to the third-highest scoring elementary school
text.
Mean
Program Name
score
Degrees of
t statistic
freedom
p-value
# tests
Significance
remaining
cutoff
Math Connects (Elem)
0.724
Bridges in Mathematics
0.687
Investigations
0.635
Singapore Math Standards
0.365
-16.59
628
8.17E-52
9
0.006
Math Out of the Box
0.521
-6.94
681
4.47E-12
8
0.006
Math Trailblazers
0.521
-6.89
647
6.50E-12
7
0.007
Progress in Mathematics
0.547
-5.06
659
2.76E-07
6
0.008
enVision
0.568
-3.93
705
4.67E-05
5
0.010
Growing with Mathematics
0.575
-3.71
643
1.14E-04
4
0.013
Saxon Math (Elem)
0.581
-3.21
636
6.88E-04
3
0.017
Everyday Mathematics
0.593
-2.57
687
0.005
2
0.025
Math Expressions
0.621
-0.88
709
0.191
1
0.050
Table 43. t-test results comparing lower-scoring texts to the third-highest scoring middle school text.
Program Name
Mean
t
Degrees of
score
statistic
freedom
Holt Mathematics
0.837
Math Connects (Middle)
0.723
Prentice Hall Mathematics
0.707
2008 Mathematics Instructional Materials Review Draft Report
p-value
# tests
Significance
remaining
cutoff
Page 104
Program Name
Mean
t
Degrees of
score
statistic
freedom
p-value
# tests
Significance
remaining
cutoff
Everyday Math/Transition
0.419
-13.43
305
6.40E-33
10
0.005
ALEKS, Online Mathematics Solution
0.392
-13.61
279
5.32E-33
9
0.006
Mathscape
0.416
-12.65
270
1.97E-29
8
0.006
CPM Middle Grades Program
0.511
-9.09
316
5.64E-18
7
0.007
Saxon Math Intermediate (Middle)
0.562
-6.19
291
1.00E-09
6
0.008
Cognitive Tutor/Carnegie Algebra
0.592
-5.02
322
4.37E-07
5
0.010
Impact
0.605
-4.43
308
6.57E-06
4
0.013
Connected Mathematics 2
0.625
-3.59
270
1.94E-04
3
0.017
McDougal Littell Math Course
0.658
-2.24
322
0.013
2
0.025
Math Thematics
0.690
-0.83
330
0.203
1
0.050
4.9.3 Without assuming that each item is an identically distributed
measure of the true scale mean for a program
4.9.3.1 Motivation
Since each item is a measure of a different aspect of alignment with a particular scale (i.e.
different math standards in Content/Standards Alignment), it would be reasonable to
assume that each item has a different mean value that contributes to the overall mean,
rather than considering them all to be independent draws from one distribution.
4.9.3.2 Methodology
In this situation, rather than consider only the variance of the mean within scale, we begin
with the variance of the scores themselves.
Var ( X w( p ) )
6
i 1
wi
J
Kj
Lij
j 1
k 1
l 1
Var ( X ijkl / Ni )
We estimate
Var ( X ijkl / Ni )
2
il
/ Ni2
by
sil 2 / Ni2 ,
where
is the sample variance of all scores on item l of category i (across programs).
4.9.3.3 Results
Table 44 through Table 47 give results based on this standard error calculation. We see
that the results are identical to the simplest standard error calculation given in Section
4.9.1.
2008 Mathematics Instructional Materials Review Draft Report
Page 105
Table 44. Confidence interval values for all comprehensive elementary programs.
Values
95% Confidence Interval
Final
Program Name
Score
Std Error
Min
Max
Math Connects (Elem)
0.7245
0.0093
0.7063
0.7426
Bridges in Mathematics
0.6872
0.0091
0.6695
0.7050
Investigations
0.6355
0.0091
0.6177
0.6532
Math Expressions
0.6212
0.0093
0.6030
0.6393
Everyday Mathematics
0.5929
0.0091
0.5751
0.6107
Saxon Math (Elem)
0.5814
0.0091
0.5636
0.5992
Growing with Mathematics
0.5746
0.0090
0.5569
0.5924
enVision
0.5681
0.0091
0.5504
0.5859
Progress in Mathematics
0.5472
0.0091
0.5294
0.5650
Math Out of the Box
0.5212
0.0089
0.5039
0.5386
Math Trailblazers
0.5211
0.0093
0.5029
0.5392
Singapore Math Standards
0.3647
0.0091
0.3468
0.3826
Grand Total
0.5782
0.0026
0.5731
0.5834
Table 45.Confidence interval values for all comprehensive middle school programs.
95%
Confidence
Values
Interval
Final
Program Name
Score
Std Error
Min
Max
Holt Mathematics
0.8370
0.0133
0.8110
0.8630
Math Connects (Middle)
0.7234
0.0133
0.6974
0.7494
Prentice Hall Mathematics
0.7073
0.0133
0.6812
0.7333
Math Thematics
0.6899
0.0134
0.6637
0.7161
McDougal Littell Math Course
0.6579
0.0133
0.6319
0.6839
Connected Mathematics 2
0.6250
0.0133
0.5990
0.6510
Impact
0.6054
0.0138
0.5784
0.6325
Cognitive Tutor/Carnegie Algebra
0.5920
0.0133
0.5660
0.6180
Saxon Math Intermediate (Middle)
0.5624
0.0133
0.5364
0.5884
CPM Middle Grades Program
0.5106
0.0133
0.4845
0.5366
Everyday Math/Transition
0.4194
0.0127
0.3944
0.4443
Mathscape
0.4158
0.0133
0.3898
0.4419
ALEKS, Online Mathematics Solution
0.3917
0.0133
0.3656
0.4177
Grand Total
0.5939
0.0037
0.5867
0.6011
2008 Mathematics Instructional Materials Review Draft Report
Page 106
Table 46. t-test results comparing lower-scoring texts to the third-highest scoring elementary school
text.
Mean
Program Name
score
Degrees of
t statistic
freedom
p-value
# tests
Significance
remaining
cutoff
Math Connects (Elem)
0.724
Bridges in Mathematics
0.687
Investigations
0.635
Singapore Math Standards
0.365
-21.06
188
1.22E-51
9
0.006
Math Out of the Box
0.521
-9.01
192
1.09E-16
8
0.006
Math Trailblazers
0.521
-8.83
184
4.08E-16
7
0.007
Progress in Mathematics
0.547
-6.88
188
4.31E-11
6
0.008
enVision
0.568
-5.25
188
2.00E-07
5
0.010
Growing with Mathematics
0.575
-4.75
188
2.00E-06
4
0.013
Saxon Math (Elem)
0.581
-4.22
188
1.90E-05
3
0.017
Everyday Mathematics
0.593
-3.32
188
0.001
2
0.025
Math Expressions
0.621
-1.10
184
0.135
1
0.050
Table 47. t-test results comparing lower-scoring texts to the third-highest scoring middle school text.
Program Name
Mean
t
Degrees of
score
statistic
freedom
p-value
# tests
Significance
remaining
cutoff
Holt Mathematics
0.837
Math Connects (Middle)
0.723
Prentice Hall Mathematics
0.707
ALEKS, Online Mathematics Solution
0.392
-16.81
94
2.22E-30
10
0.005
Everyday Math/Transition
0.419
-15.66
98
1.34E-28
9
0.006
Mathscape
0.416
-15.52
94
5.59E-28
8
0.006
CPM Middle Grades Program
0.511
-10.48
94
8.98E-18
7
0.007
Saxon Math Intermediate (Middle)
0.562
-7.72
94
6.24E-12
6
0.008
Cognitive Tutor/Carnegie Algebra
0.592
-6.14
94
9.89E-09
5
0.010
Impact
0.605
-5.32
90
3.84E-07
4
0.013
Connected Mathematics 2
0.625
-4.38
94
1.54E-05
3
0.017
McDougal Littell Math Course
0.658
-2.63
94
5.02E-03
2
0.025
Math Thematics
0.690
-0.92
95
0.180
1
0.050
4.9.4 Without assuming scale independence or identical distributions
4.9.4.1 Motivation
We might expect that both of the previously discussed assumptions are violated and that
the combined adjustment could change the results.
2008 Mathematics Instructional Materials Review Draft Report
Page 107
4.9.4.2 Methodology
The assumption of independence of the scales is what allows us to say that
6
Var(X w( p ) )
i 1
2
wi Var(X i...)
Without that assumption, we should adjust the variance for the covariances of the scales
by taking:
6
6
Var ( X i... )
wi wmCov(X i... , X m... )
i 1 m 1
Note that
Cov( X i... , X i... ) Var ( X i... ) .
In this situation, rather than consider only the variance of the mean within scale, we
begin with the variance of the scores themselves, to obtain:
J
Kj
Lij
j 1
k 1
l 1
Var ( X ijkl / Ni )
il
Var ( X i... )
Var ( X ijkl / Ni )
We estimate
2
/ Ni2
by
sil 2 / Ni2
where sil2 is the sample variance of all scores on item l of category i (across programs).
We can use
Var ( X i... )
to calculate the covariance, because
Cov( X i... , X i... )
where
Var ( X i... )Var ( X j ... )
is the correlation between scales i and j.
4.9.4.3 Results
Table 48 through Table 51 below give the confidence intervals and t-test results. The
conclusions are identical to those in Section 4.9.1; most importantly the t-test results
remain unchanged from the simplest case.
2008 Mathematics Instructional Materials Review Draft Report
Page 108
Table 48. Confidence interval values for all comprehensive elementary programs.
Values
95% Confidence Interval
Final
Program Name
Score
Std Error
Min
Max
Math Connects (Elem)
0.7245
0.0119
0.7012
0.7478
Bridges in Mathematics
0.6872
0.0116
0.6644
0.7100
Investigations
0.6355
0.0116
0.6127
0.6583
Math Expressions
0.6212
0.0119
0.5978
0.6445
Everyday Mathematics
0.5929
0.0116
0.5701
0.6157
Saxon Math (Elem)
0.5814
0.0116
0.5586
0.6042
Growing with Mathematics
0.5746
0.0116
0.5518
0.5974
envision
0.5681
0.0116
0.5453
0.5909
Progress in Mathematics
0.5472
0.0117
0.5243
0.5700
Math Out of the Box
0.5212
0.0114
0.4989
0.5436
Math Trailblazers
0.5211
0.0119
0.4977
0.5444
Singapore Math Standards
0.3647
0.0117
0.3418
0.3876
Table 49. Confidence interval values for all comprehensive middle school programs.
Values
95% Confidence Interval
Final
Program Name
Score
Std Error
Min
Max
Holt Mathematics
0.8370
0.0169
0.8039
0.8701
Math Connects (Middle)
0.7234
0.0169
0.6903
0.7565
Prentice Hall Mathematics
0.7073
0.0169
0.6742
0.7403
Math Thematics
0.6899
0.0170
0.6567
0.7231
McDougal Littell Math Course
0.6579
0.0169
0.6248
0.6910
Connected Mathematics 2
0.6250
0.0169
0.5919
0.6581
Impact
0.6054
0.0176
0.5710
0.6398
Cognitive Tutor/Carnegie Algebra
0.5920
0.0169
0.5589
0.6251
Saxon Math Intermediate (Middle)
0.5624
0.0169
0.5293
0.5955
CPM Middle Grades Program
0.5106
0.0169
0.4775
0.5437
Everyday Math/Transition
0.4194
0.0162
0.3876
0.4511
Mathscape
0.4158
0.0169
0.3827
0.4489
ALEKS, Online Mathematics Solution
0.3917
0.0169
0.3586
0.4247
2008 Mathematics Instructional Materials Review Draft Report
Page 109
Table 50. t-test results comparing lower-scoring texts to the third-highest scoring elementary school
text.
Degrees
Program Name
Mean
t
of
score
statistic
freedom
p-value
# tests
Significance
remaining
cutoff
Math Connects (Elem)
0.7245
Bridges in Mathematics
0.6872
Investigations
0.6355
Singapore Math Standards
0.3647
-16.41
188
1.98E-38
9
0.006
Math Out of the Box
0.5212
-7.01
192
1.98E-11
8
0.006
Math Trailblazers
0.5211
-6.87
184
4.68E-11
7
0.007
Progress in Mathematics
0.5472
-5.36
188
1.22E-07
6
0.008
envision
0.5681
-4.09
188
3.19E-05
5
0.010
Growing with Mathematics
0.5746
-3.70
188
1.43E-04
4
0.013
Saxon Math (Elem)
0.5814
-3.29
188
6.08E-04
3
0.017
Everyday Mathematics
0.5929
-2.59
188
0.005
2
0.025
Math Expressions
0.6212
-0.86
184
0.195
1
0.050
Table 51. t-test results comparing lower-scoring texts to the third-highest scoring middle school text.
Degrees
Program Name
Mean
t
of
score
statistic
freedom
p-value
# tests
Significance
remaining
cutoff
Holt Mathematics
0.8370
Math Connects (Middle)
0.7234
Prentice Hall Mathematics
0.7073
ALEKS, Online Mathematics Solution
0.3917
-13.22
94
1.82E-23
10
0.005
Everyday Math/Transition
0.4194
-12.31
98
7.87E-22
9
0.006
Mathscape
0.4158
-12.21
94
2.13E-21
8
0.006
CPM Middle Grades Program
0.5106
-8.24
94
5.02E-13
7
0.007
Saxon Math Intermediate (Middle)
0.5624
-6.07
94
1.35E-08
6
0.008
Cognitive Tutor/Carnegie Algebra
0.5920
-4.83
94
2.67E-06
5
0.010
Impact
0.6054
-4.18
90
3.40E-05
4
0.013
Connected Mathematics 2
0.6250
-3.45
94
4.27E-04
3
0.017
McDougal Littell Math Course
0.6579
-2.07
94
0.021
2
0.025
Math Thematics
0.6899
-0.73
95
0.235
1
0.050
2008 Mathematics Instructional Materials Review Draft Report
Page 110
Appendix A.
Programs Reviewed
Table 52. List of core/comprehensive materials submitted for review, including publisher information.
Copyright
Date
Recommended
Grade Levels
ALEKS Corporation
The Math Learning Center
© 2008
© 2007
Grades 6-8
Grades K-5
egedrep@aol.com
toms@mathlearningcenter.org
866.205.1642
503.370.8130
Carnegie Learning, Inc.
© 2008
Grades 5-8
Connected Mathematics 2
Pearson (Prentice Hall)
Grades 6-8
360.260.0435
360.730.1186
253.906.1059
CPM Middle Grades Program
CPM Educational Program
Pearson (Scott Foresman Addison Wesley)
Wright Group/McGraw-Hill
Wright Group/McGraw-Hill
Wright Group/McGraw-Hill
Holt, Rinehart and Winston
Holt, Rinehart and Winston
© 2009
© 2002,
2006
swallace@carnegielearning.com
dorothy.kulwin@pearson.com
kyle.bender@pearson.com
Grades 6-8
hoey@cpm.org
916.391.3301
© 2009
© 2007
© 2007
© 2008
© 2007
© 2007
Grades K-6
Grades 6-8
Grades K-5
Grades PreK-5
Grades 6-8
Grade 7-8
Glencoe McGraw-Hill
Pearson (Scott Foresman Addison Wesley)
Macmillan McGraw-Hill
© 2009
Grade 6-8
sharon.lipton@pearson.com
jenny_arlt@mcgraw-hill.com
jenny_arlt@mcgraw-hill.com
jenny_arlt@mcgraw-hill.com
FrankAtkinson@hmco.com
FrankAtkinson@hmco.com
Susan_arnold@mcgraw-hill.com
jolie_vigen@mcgraw-hill.com
360.896.0929
425.377.1346
425.377.1346
425.377.1346
425.747.7099
425.747.7099
360.281.2500
206.243.1067
© 2008
© 2009
Grades K-5
Grades K-5
Glencoe McGraw-Hill
Houghton Mifflin Harcourt
School Publishers
© 2009
Grade 6-8
© 2009
Grades K-5
sharon.lipton@pearson.com
lisa-spicer@mcgraw-hill.com
Susan_arnold@mcgraw-hill.com
jolie_vigen@mcgraw-hill.com
michael_kinzler@hmco.com
sari.dahl@harcourt.com
360.896.0929
425.485.2039
360.281.2500
206.243.1067
253.241.8050
253.549.4341
Program Name
ALKES, Online Mathematics
Solution
Bridges in Mathematics
Cognitive Tutor/Carnegie
Algebra I
enVision
Everyday Math/Transition5
Everyday Mathematics
Growing with Mathematics
Holt Mathematics
Holt Pre-/Algebra 16
Impact
Investigations
Math Connects (Elem)
Math Connects (Middle)7
Math Expressions
Publisher Name
Email Address
5
Phone Number
Everyday Mathematics has a K-6 curriculum. The publisher combined Everyday Mathematics grade 6 with UCSMP Transition series to form a comprehensive
middle school curriculum, which is a typical progression for many schools.
6
This series covers grades 7&8. It is not included in the comprehensive list of instructional materials considered for the three recommended texts because it does
not cover grades 6-8. However, information is provided about the series.
7
The publisher submitted Mathscape Modular as a bundled supplement to Math Connects (Middle).
2008 Mathematics Instructional Materials Review Draft Report
Page 111
Copyright
Date
Recommended
Grade Levels
© 2009
© 2008
Grades K-5
Grades 6-8
Math Trailblazers, 2/e
Carolina Curriculum Publishing
McDougal Littell
Kendall/Hunt Publishing
Company
© 2008
Grades K-5
Mathscape8
McDougal Littell Math Course
McDougal Littell Pre-/Algebra 16
Glencoe McGraw-Hill
McDougal Littell
McDougal Littell
© 2005
© 2007
© 2007
Grade 6-8
Grades 6-8
Grades 7-8
Prentice Hall Mathematics
Progress in Mathematics
Pearson (Prentice Hall)
William H. Sadlier, Inc.
Saxon, and Imprint of HMH
Supplemental Publishers Inc.
Saxon, and Imprint of HMH
Supplemental Publishers Inc.
Marshall Cavendish International
(Singapore) Pvt. Ltd. (Exclusive
Distributor: SingaporeMath.com
Inc.)
© 2008
© 2009
Program Name
Math Out of the Box
Math Thematics
Saxon Math (Elem)9
Saxon Math Intermediate
(Middle)
Singapore Math Standards
Publisher Name
Email Address
Grades 6-8
Grades K-8
Susan_arnold@mcgraw-hill.com
jolie_vigen@mcgraw-hill.com
Frank_Atkinson@hmco.com
Frank_Atkinson@hmco.com
dorothy.kulwin@pearson.com
kyle.bender@pearson.com
kod@sadlier.com
Phone Number
800.227.1500
Ext.3503
425.747.7099
206.527.5306
206.473.2566
360.281.2500
206.243.1067
425.747.7099
425.747.7099
360.730.1186
253.906.1059
212.312.6132
© 2008
Grade K-5
CherylJohnson@harcour.com
800.220.4313
© 2007/2008
Grade 6-8
CherylJohnson@harcour.com
800.220.4313
© 2008
Grades K-5
jthomas@singaporemath.com
503.680.9724
pam.chiodo@carolina.com
Frank_Atkinson@hmco.com
8
Glencoe McGraw-Hill submitted the Mathscape Unitized series for review as a comprehensive program. The publisher submitted Mathscape Modular as a
supplement to Impact. All references to the comprehensive Mathscape series in this document refer to the Mathscape Unitized version.
9
Saxon Math (Elem) refers to the combination of Saxon Math Kit (grades K-3) and Saxon Math Intermediate (grades 4-5).
2008 Mathematics Instructional Materials Review Draft Report
Page 112
Appendix B.
Review Instruments
This section shows the content of each of the K-8 review instruments: Part 1:
Content/standards Alignment and Part 2: Other Factors.
2008 Mathematics Instructional Materials Review Draft Report
Page 113
Date:
Kindergarten
Program:
Reviewer #:
(Rate each item on the scale 0-not met, 1-partial met, 2-fully met)
K.1. Whole Numbers (Numbers, Operations)
0 1 2
K.1.A Rote count by ones forward from 1 to 100 and backward from any number in the range of 10 to 1.
  
K.1.B Read aloud numerals from 0 to 31.
  
K.1.C Fluently compose and decompose numbers to 5.
  
K.1.D Order numerals from 1 to 10.
  
Evidence
K.1.E Count objects in a set of up to 20, and count out a specific number of up to 20 objects from a larger   
set.
Compare two sets of up to 10 objects each and say whether the number of objects in one set is
K.1.F equal to, greater than, or less than the number of objects in the other set.
  
K.1.G Locate numbers from 1 to 31 on the number line.
  
K.1.H Describe a number from 1 to 9 using 5 as a benchmark number.
  
K.2. Patterns and operations (Operations, Algebra)
0
1
2
K.2.A Copy, extend, describe, and create simple repetitive patterns.
  
K.2.B Translate a pattern among sounds, symbols, movements, and physical objects.
  
Model addition by joining sets of objects that have 10 or fewer total objects when joined and
K.2.C model subtraction by separating a set of 10 or fewer objects.
Describe a situation that involves the actions of joining (addition) or separating (subtraction) using
K.2.D
words, pictures, objects, or numbers.
  
2008 Mathematics Instructional Materials Review Draft Report
Page 114
  
Evidence
K.3. Objects and their locations (Geometry/Measurement)
Identify, name, and describe circles, triangles, rectangles, squares (as special rectangles), cubes,
K.3.A and spheres.
1
2
Describe the location of one object relative to another object using words such as in, out, over,
under, above, below, between, next to, behind, and in front of.
K.4. Additional key content (Geometry/Measurement)
  
  
0
K.4.A Make direct comparisons using measurable attributes such as length, weight, and capacity.
K.5. Reasoning, problem solving, and communication
1
2
0
1
2
  
K.5.B Identify the given information that can be used to solve a problem.
  
K.5.C Recognize when additional information is required to solve a problem.
  
Select from a variety of problem-solving strategies and use one or more strategies to solve a
K.5.D problem.
  
K.5.E Answer the question(s) asked in a problem.
  
K.5.F Describe how a problem was solved.
  
K.5.G Determine whether a solution to a problem is reasonable.
  
Page 115
Evidence
  
K.5.A Identify the question(s) asked in a problem.
2008 Mathematics Instructional Materials Review Draft Report
Evidence
  
K.3.B Sort shapes using a sorting rule and explain the sorting rule.
K.3.C
0
Evidence
Date:
Grade 1
Program:
Reviewer #:
(Rate each item on the scale 0-not met, 1-partial met, 2-fully met)
1.1. Whole number relationships (Numbers, Operations)
0 1 2
Count by ones forward and backward from 1 to 120, starting at any number, and count by twos,
1.1.A fives, and tens to 100.
  
1.1.B Name the number that is one less or one more than any number given verbally up to 120.
  
1.1.C Read aloud numerals from 0 to 1,000.
  
1.1.D Order objects or events using ordinal numbers.
  
1.1.E Write, compare, and order numbers to 120.
  
1.1.F Fluently compose and decompose numbers to 10.
  
1.1.G Group numbers into tens and ones in more than one way.
  
1.1.H Group and count objects by tens, fives, an twos.
  
1.1.I Classify a number as odd or even and demonstrate that it is odd or even.
  
2008 Mathematics Instructional Materials Review Draft Report
Page 116
Evidence
1.2. Addition and subtraction (Operations, Algebra)
0
1
2
1.2.A Connect physical and pictorial representations to addition and subtraction equations.
  
1.2.B Use the equal sign (=) and the word equals to indicate that two expressions are equivalent.
  
1.2.C Represent addition and subtraction on the number line.
  
Demonstrate the inverse relationship between addition and subtraction by undoing an addition
1.2.D problem with subtraction and vice versa.
Add three or more one-digit numbers using the commutative and associative properties of
1.2.E
addition.
  
Evidence
  
1.2.F Apply and explain strategies to compute addition facts and related subtraction facts for sums to 18.   
1.2.G Quickly recall addition facts and related subtraction facts for sums equal to 10.
  
1.2.H Solve and create word problems that match addition or subtraction equations.
  
1.2.I Recognize, extend, and create number patterns.
  
1.3. Geometric attributes (Geometry/Measurement)
Compare and sort a variety of two- and three-dimensional figures according to their geometric
1.3.A
attributes.
1.3.B Identify and name two-dimensional figures, including those in real-world contexts, regardless of
size or orientation.
0
1
2
  
  
1.3.C Combine known shapes to create shapes and divide known shapes into other shapes.
  
1.4. Concepts of measurement (Geometry/Measurement)
1.4.A Recognize that objects used to measure an attribute (length, weight, capacity) must be consistent
in size.
0
1
2
  
1.4.B Use a variety of non-standard units to measure length.
  
1.4.C Compare lengths using the transitive property.
  
1.4.D Use non-standard units to compare objects according to their capacities or weights.
  
Describe the connection between the size of the measurement unit and the number of units
1.4.E needed to measure something.
Name the days of the week and the months of the year, and use a calendar to determine a day or
1.4.F
month.
  
2008 Mathematics Instructional Materials Review Draft Report
Page 117
Evidence
  
Evidence
1.5. Additional key content (Data/Statistics/Probability)
0
1
2
1.5.A Represent data using tallies, tables, picture graphs, and bar-type graphs.
  
1.5.B Ask and answer comparison questions about data.
  
1.6. Reasoning, problem solving, and communication
0
1
2
1.6.A Identify the question(s) asked in a problem.
  
1.6.B Identify the given information that can be used to solve a problem.
  
1.6.C Recognize when additional information is required to solve a problem.
  
Select from a variety of problem-solving strategies and use one or more strategies to solve a
1.6.D problem.
  
1.6.E Answer the question(s) asked in a problem.
  
1.6.F Identify the answer(s) to the question(s) in a problem.
  
1.6.G Describe how a problem was solved.
  
1.6.H Determine whether a solution to a problem is reasonable.
  
2008 Mathematics Instructional Materials Review Draft Report
Page 118
Evidence
Evidence
Date:
Grade 2
Program:
Reviewer #:
(Rate each item on the scale 0-not met, 1-partial met, 2-fully met)
2.1. Place value and the base ten system (Numbers)
0 1 2
2.1.A Count by tens or hundreds forward and backward from 1 to 1,000, starting at any number.
  
2.1.B Connect place value models with their numerical equivalents to 1,000.
  
2.1.C Identify the ones, tens, and hundreds place in a number and the digits occupying them.
  
2.1.D Write three-digit numbers in expanded form.
  
2.1.E Group three-digit numbers into hundreds, tens, and ones in more than one way.
  
2.1.F Compare and order numbers from 0 to 1,000.
  
2.2. Addition and subtraction (Operations, Geometry/Measurement, Algebra)
0
1
2
2.2.A Quickly recall basic addition facts and related subtraction facts for sums through 20.
  
Solve addition and subtraction word problems that involve joining, separating, and comparing and
2.2.B verify the solution.
Add and subtract two-digit numbers efficiently and accurately using a procedure that works with
2.2.C
all two-digit numbers and explain why the procedure works.
  
2.2.D Add and subtract two-digit numbers mentally and explain the strategies used.
  
2.2.E Estimate sums and differences.
  
2.2.F Create and state a rule for patterns that can be generated by addition and extend the pattern.
  
2.2.G Solve equations in which the unknown number appears in a variety of positions.
  
Name each standard U.S. coin, write its value using the $ sign and the ¢ sign, and name
2.2.H combinations of other coins with the same total value.
  
2.2.I Determine the value of a collection of coins totaling less than $1.00.
  
2008 Mathematics Instructional Materials Review Draft Report
Page 119
  
Evidence
Evidence
2.3. Measurement (Geometry/Measurement)
0
1
2
2.3.A Identify objects that represent or approximate standard units and use them to measure length.
  
2.3.B Estimate length using metric and U.S. customary units.
  
2.3.C Measure length to the nearest whole unit in both metric and U.S. customary units.
  
2.3.D Describe the relative size among minutes, hours, days, weeks, months, and years.
  
2.3.E Use both analog and digital clocks to tell time to the minute.
  
2.4. Additional key content (Numbers, Operations, Geometry/Measurement, Data/Statistics/Prob.)
0
1
2
2.4.A Solve problems involving properties of two- and three-dimensional figures.
  
2.4.B Collect, organize, represent, and interpret data in bar graphs and picture graphs.
  
2.4.C Model and describe multiplication situations in which sets of equal size are joined.
  
2.4.D Model and describe division situations in which sets are separated into equal parts.
  
2.4.E Interpret a fraction as a number of equal parts of a whole or a set.
  
2.5. Reasoning, problem solving, and communication
Identify the question(s) asked in a problem and any other questions that need to be answered in
2.5.A order to solve the problem.
0
1
2
  
2.5.B Identify the given information that can be used to solve a problem.
  
2.5.C Recognize when additional information is required to solve a problem.
  
Select from a variety of problem-solving strategies and use one or more strategies to solve a
2.5.D problem.
  
2.5.E Identify the answer(s) to the question(s) in a problem.
  
2.5.F Describe how a problem was solved.
  
2.5.G Determine whether a solution to a problem is reasonable.
  
2008 Mathematics Instructional Materials Review Draft Report
Page 120
Evidence
Evidence
Evidence
Date:
Grade 3
Program:
Reviewer #:
(Rate each item on the scale 0-not met, 1-partial met, 2-fully met)
3.1. Addition, subtraction, and place value (Numbers, Operations)
0 1 2
Read, write, compare, order, and represent numbers to 10,000 using numbers, words, and
3.1.A
  
symbols.
Evidence
  
3.1.B Round whole numbers through 10,000 to the nearest ten, hundred, and thousand.
3.1.C Fluently and accurately add and subtract whole numbers using the standard regrouping algorithms.   
Estimate sums and differences to approximate solutions to problems and determine
3.1.D reasonableness of answers.
Solve single- and multi-step word problems involving addition and subtraction of whole numbers
3.1.E and verify the solutions.
3.2.A
3.2.B
3.2.C
3.2.D
3.2.E
3.2. Concepts of multiplication and division (Operations, Algebra)
Represent multiplication as repeated addition, arrays, counting by multiples, and equal jumps on
the number line, and connect each representation to the related equation.
Represent division as equal sharing, repeated subtraction, equal jumps on the number line, and
formation of equal groups of objects, and connect each representation to the related equation.
Determine products, quotients, and missing factors using the inverse relationship between
multiplication and division.
Apply and explain strategies to compute multiplication facts to 10 X 10 and the related division
facts.
Quickly recall those multiplication facts for which one factor is 1, 2, 5, or 10 and the related division
facts.
  
  
0
1
2
  
  
  
  
  
3.2.F Solve and create word problems that match multiplication or division equations.
  
Multiply any number from 11 through 19 by a single-digit number using the distributive property
3.2.G and place value concepts.
Solve single- and multi-step word problems involving multiplication and division and verify the
3.2.H solutions.
  
2008 Mathematics Instructional Materials Review Draft Report
Page 121
  
Evidence
3.3. Fraction concepts (Numbers, Algebra)
Represent fractions that have denominators of 2, 3, 4, 5, 6, 8, 9, 10, and 12 as parts of a whole,
3.3.A
parts of a set, and points on the number line.
  
3.3.B Compare and order fractions that have denominators of 2, 3, 4, 5, 6, 8, 9, 10, and 12.
  
3.3.C Represent and identify equivalent fractions with denominators of 2, 3, 4, 5, 6, 8, 9, 10, and 12.
  
Solve single- and multi-step word problems involving comparison of fractions and verify the
3.3.D solutions.
  
3.4. Geometry (Geometry/Measurement)
0
0
1
1
2
2
3.4.A Identify and sketch parallel, intersecting, and perpendicular lines and line segments.
  
3.4.B Identify and sketch right angles.
  
3.4.C Identify and describe special types of quadrilaterals.
  
3.4.D Measure and calculate perimeters of quadrilaterals.
  
Solve single- and multi-step word problems involving perimeters of quadrilaterals and verify the
3.4.E solutions.
  
3.5. Additional key content (Algebra, Geometry/Measurement, Data/Statistics/Probability)
0
1
2
3.5.A Determine whether two expressions are equal and use “=” to denote equality.
  
3.5.B Measure temperature in degrees Fahrenheit and degrees Celsius using a thermometer.
  
Estimate, measure, and compare weight and mass using appropriate-sized U.S. customary and
3.5.C
metric units.
  
3.5.D Estimate, measure, and compare capacity using appropriate-sized U.S. customary and metric units.   
  
3.5.E Construct and analyze pictographs, frequency tables, line plots, and bar graphs.
2008 Mathematics Instructional Materials Review Draft Report
Page 122
Evidence
Evidence
Evidence
3.6. Reasoning, problem solving, and communication
0
1
2
  
3.6.A Determine the question(s) to be answered given a problem situation.
3.6.B Identify information that is given in a problem and decide whether it is necessary or unnecessary to   
the solution of the problem.
  
3.6.C Identify missing information that is needed to solve a problem.
3.6.D
Determine whether a problem to be solved is similar to previously solved problems, and identify
possible strategies for solving the problem.
  
3.6.E Select and use one or more appropriate strategies to solve a problem.
  
3.6.F Represent a problem situation using words, numbers, pictures, physical objects, or symbols.
  
3.6.G Explain why a specific problem-solving strategy or procedure was used to determine a solution.
  
3.6.H
Analyze and evaluate whether a solution is reasonable, is mathematically correct, and answers the
question.
  
3.6.I Summarize mathematical information, draw conclusions, and explain reasoning.
  
Make and test conjectures based on data (or information) collected from explorations and
3.6.J experiments.
  
2008 Mathematics Instructional Materials Review Draft Report
Page 123
Evidence
Date:
Grade 4
Program:
Reviewer #:
(Rate each item on the scale 0-not met, 1-partial met, 2-fully met)
4.1. Multi-digit multiplication (Numbers, Operations, Algebra)
0 1 2
4.1.A Quickly recall multiplication facts through 10 X 10 and the related division facts.
  
4.1.B Identify factors and multiples of a number.
  
4.1.C Represent multiplication of a two-digit number by a two-digit number with place value models.
  
4.1.D Multiply by 10, 100, and 1,000.
  
4.1.E Compare the values represented by digits in whole numbers using place value.
  
4.1.F
Fluently and accurately multiply up to a three-digit number by one- and two-digit numbers using
the standard multiplication algorithm.
  
4.1.G Mentally multiply two-digit numbers by numbers through 10 and by multiples of 10.
  
Estimate products to approximate solutions to problems and determine reasonableness of
4.1.H answers.
Solve single- and multi-step word problems involving multi-digit multiplication and verify the
4.1.I solutions.
  
4.1.J Solve single- and multi-step word problems involving division and verify the solutions.
  
2008 Mathematics Instructional Materials Review Draft Report
Page 124
  
Evidence
4.2. Fractions, decimals, and mixed numbers (Numbers, Algebra)
Represent decimals through hundredths with place value models, fraction equivalents, and the
4.2.A number line.
  
4.2.B Read, write, compare, and order decimals through hundredths.
  
4.2.C Convert a mixed number to a fraction and vice versa, and visually represent the number.
  
4.2.D Convert a decimal to a fraction and vice versa, and visually represent the number.
  
Compare and order decimals and fractions (including mixed numbers) on the number line, lists,
4.2.E and the symbols <, >, or =.
  
4.2.F Write a fraction equivalent to a given fraction.
  
4.2.G Simplify fractions using common factors.
  
4.2.H Round fractions and decimals to the nearest whole number.
  
Solve single- and multi-step word problems involving comparison of decimals and fractions
4.2.I (including mixed numbers), and verify the solutions.
  
4.3. Concept of area (Geometry/Measurement, Algebra)
0
0
1
1
2
2
4.3.A Determine congruence of two-dimensional figures.
  
4.3.B Determine the approximate area of a figure using square units.
Determine the perimeter and area of a rectangle using formulas, and explain why the formulas
4.3.C work.
4.3.D Determine the areas of figures that can be broken down into rectangles.
4.3.E Demonstrate that rectangles with the same area can have different perimeters, and that rectangles
with the same perimeter can have different areas.
4.3.F Solve single- and multi-step word problems involving perimeters and areas of rectangles and verify
the solutions.
  
2008 Mathematics Instructional Materials Review Draft Report
Page 125
  
  
  
  
Evidence
Evidence
4.4. Additional key content (Geometry/Measurement, Algebra, Data/Statistics/Probability)
Represent an unknown quantity in simple expressions, equations, and inequalities using letters,
4.4.A
boxes, and other symbols.
Solve single- and multi-step problems involving familiar unit conversions, including time, within
4.4.B
either the U.S. customary or metric system.
4.4.C Estimate and determine elapsed time using a calendar, a digital clock, and an analog clock.
0
1
2
Evidence
  
  
  
4.4.D Graph and identify points in the first quadrant of the coordinate plane using ordered pairs.
  
Determine the median, mode, and range of a set of data and describe what each measure indicates
4.4.E
  
about the data.
4.4.F Describe and compare the likelihood of events.
  
4.4.G Determine a simple probability from a context that includes a picture.
  
4.4.H Display the results of probability experiments and interpret the results.
  
4.5. Reasoning, problem solving, and communication
0
1
2
4.5.A Determine the question(s) to be answered given a problem situation.
4.5.B Identify information that is given in a problem and decide whether it is essential or extraneous to
the solution of the problem.
4.5.C Identify missing information that is needed to solve a problem.
4.5.D Determine whether a problem to be solved is similar to previously solved problems, and identify
possible strategies for solving the problem.
4.5.E Select and use one or more appropriate strategies to solve a problem and explain why that
strategy was chosen.
4.5.F Represent a problem situation using words, numbers, pictures, physical objects, or symbols.
  
4.5.G Explain why a specific problem-solving strategy or procedure was used to determine a solution.
Analyze and evaluate whether a solution is reasonable, is mathematically correct, and answers the
4.5.H
question.
4.5.I Summarize mathematical information, draw conclusions, and explain reasoning.
4.5.J Make and test conjectures based on data (or information) collected from explorations and
experiments.
  
2008 Mathematics Instructional Materials Review Draft Report
Page 126
  
  
  
  
  
  
  
  
Evidence
Date:
Grade 5
Program:
5.1.A
5.1.B
5.1.C
5.1.D
Reviewer #:
(Rate each item on the scale 0-not met, 1-partial met, 2-fully met)
5.1. Multi-digit division (Operations, Algebra)
0 1 2
Represent multi-digit division using place value models and connect the representation to the
  
related equation.
Determine quotients for multiples of 10 and 100 by applying knowledge of place value and
  
properties of operations.
Fluently and accurately divide up to a four-digit number by one- or two-digit divisors using the
  
standard long-division algorithm.
Estimate quotients to approximate solutions and determine reasonableness of answers in
  
problems involving up to two-digit divisors.
5.1.E Mentally divide two-digit numbers by one-digit divisors and explain the strategies used.
  
5.1.F Solve single- and multi-step word problems involving multi-digit division and verify the solutions.
  
5.2. Addition and subtraction of fractions and decimals (Numbers, Operations, Algebra)
Represent
addition and subtraction of fractions and mixed numbers using visual and numerical
5.2.A
models, and connect the representation to the related equation.
Represent addition and subtraction of decimals using place value models and connect the
5.2.B representation to the related equation.
0
1
2
  
  
5.2.C Given two fractions with unlike denominators, rewrite the fractions with a common denominator.
  
5.2.D Determine the greatest common factor and the least common multiple of two or more whole
numbers.
  
5.2.E Fluently and accurately add and subtract fractions, including mixed numbers.
  
5.2.F Fluently and accurately add and subtract decimals.
  
5.2.G Estimate sums and differences of fractions, mixed numbers, and decimals to approximate solutions   
to problems and determine reasonableness of answers.
Solve single- and multi-step word problems involving addition and subtraction of whole numbers,
5.2.H fractions (including mixed numbers), and decimals, and verify the solutions.
  
2008 Mathematics Instructional Materials Review Draft Report
Page 127
Evidence
Evidence
5.3. Triangles and quadrilaterals (Geometry/Measurement, Algebra)
Classify quadrilaterals.
Identify, sketch, and measure acute, right, and obtuse angles.
Identify, describe, and classify triangles by angle measure and number of congruent sides.
Determine the formula for the area of a parallelogram by relating it to the area of a rectangle.
Determine the formula for the area of a triangle by relating it to the area of a parallelogram.
Determine the perimeters and areas of triangles and parallelograms.
Draw quadrilaterals and triangles from given information about sides and angles.
Determine the number and location of lines of symmetry in triangles and quadrilaterals.
Solve single- and multi-step word problems about the perimeters and areas of quadrilaterals and
5.3.I
triangles and verify the solutions.
5.3.A
5.3.B
5.3.C
5.3.D
5.3.E
5.3.F
5.3.G
5.3.H
5.4.A
5.4.B
5.4.C
5.4.D
5.4. Representations of algebraic relationships (Operations, Algebra)
Describe and create a rule for numerical and geometric patterns and extend the patterns.
Write a rule to describe the relationship between two sets of data that are linearly related.
Write algebraic expressions that represent simple situations and evaluate the expressions, using
substitution when variables are involved.
Graph ordered pairs in the coordinate plane for two sets of data related by a linear rule and draw
the line they determine.
5.5. Additional key content (Numbers, Data/Statistics/Probability)
5.5.A Classify numbers as prime or composite.
5.5.B Determine and interpret the mean of a small data set of whole numbers.
5.5.C Construct and interpret line graphs.
2008 Mathematics Instructional Materials Review Draft Report
0
1
2
























  
0
1
2
Evidence
  
  
  
  
0
1
2
  
  
  
Page 128
Evidence
Evidence
5.6. Reasoning, problem solving, and communication
5.6.A Determine the question(s) to be answered given a problem situation.
Identify information that is given in a problem and decide whether it is essential or extraneous to
5.6.B the solution of the problem.
5.6.C Determine whether additional information is needed to solve the problem.
Determine whether a problem to be solved is similar to previously solved problems, and identify
5.6.D
possible strategies for solving the problem.
Select and use one or more appropriate strategies to solve a problem, and explain the choice of
5.6.E
strategy.
5.6.F Represent a problem situation using words, numbers, pictures, physical objects, or symbols.
5.6.G Explain why a specific problem-solving strategy or procedure was used to determine a solution.
Analyze and evaluate whether a solution is reasonable, is mathematically correct, and answers the
5.6.H
question.
5.6.I Summarize mathematical information, draw conclusions, and explain reasoning.
Make and test conjectures based on data (or information) collected from explorations and
5.6.J experiments.
2008 Mathematics Instructional Materials Review Draft Report
Page 129
0
1
2
  
  
  
  
  
  
  
  
  
  
Evidence
Date:
Grade 6
Program:
6.1.A
6.1.B
6.1.C
6.1.D
6.1.E
Reviewer #:
(Rate each item on the scale 0-not met, 1-partial met, 2-fully met)
6.1. Multiplication & division of fractions and decimals (Numbers, Operations, Algebra)
0 1 2
Compare and order non-negative fractions, decimals, and integers using the number line, lists, and
  
the symbols <, >, or =.
Represent multiplication and division of non-negative fractions and decimals using area models
  
and the number line, and connect each representation to the related equation.
Estimate products and quotients of fractions and decimals.
  
Fluently and accurately multiply and divide non-negative fractions and explain the inverse
  
relationship between multiplication and division with fractions.
Multiply and divide whole numbers and decimals by 1000, 100, 10, 1, 0.1, 0.01, and 0.001.
  
6.1.F Fluently and accurately multiply and divide non-negative decimals.
the effect of multiplying or dividing a number by one, by zero, by a number between zero
6.1.G Describe
and one, and by a number greater than one.
single- and multi-step word problems involving operations with fractions and decimals and
6.1.H Solve
verify the solutions.
6.2.A
6.2.B
6.2.C
6.2.D
6.2.E
6.2. Mathematical expressions and equations (Operations, Algebra)
Write a mathematical expression or equation with variables to represent information in a table or
given situation.
Draw a first-quadrant graph in the coordinate plane to represent information in a table or given
situation.
Evaluate mathematical expressions when the value for each variable is given.
Apply the commutative, associative, and distributive properties, and use the order of operations to
evaluate mathematical expressions.
Solve one-step equations and verify solutions.
6.2.F Solve word problems using mathematical expressions and equations and verify solutions.
2008 Mathematics Instructional Materials Review Draft Report
Page 130
Evidence
  
  
  
0
1
2
  
  
  
  
  
  
Evidence
6.3. Ratios, rates, and percents (Nums., Opers., Geom./Measurement, Algebra, Data/Statistics/Prob.)
6.3.A Identify and write ratios as comparisons of part-to-part and part-to-whole relationships.
6.3.B Write ratios to represent a variety of rates.
percents visually and numerically, and convert between the fractional, decimal, and
6.3.C Represent
percent representations of a number.
single- and multi-step word problems involving ratios, rates, and percents, and verify the
6.3.D Solve
solutions.
the ratio of the circumference to the diameter of a circle as the constant π, and recognize
6.3.E Identify
22/7 and 3.14 as common approximations of π.
6.3.F Determine the experimental probability of a simple event using data collected in an experiment.
the theoretical probability of an event and its complement and represent the
6.3.G Determine
probability as a fraction or decimal from 0 to 1 or as a percent from 0 to 100.
6.4. Two- and three-dimensional figures (Geometry/Measurement, Algebra)
0
1
2
  
  
  
  
  
0
1
2
6.4.A Determine the circumference and area of circles.
  
the perimeter and area of a composite figure that can be divided into triangles,
6.4.B Determine
rectangles, and parts of circles.
single- and multi-step word problems involving the relationships among radius, diameter,
6.4.C Solve
circumference, and area of circles, and verify the solutions.
  
6.4.D Recognize and draw two-dimensional representations of three-dimensional figures.
  
the surface area and volume of rectangular prisms using appropriate formulas and
6.4.E Determine
explain why the formulas work.
  
6.4.F Determine the surface area of a pyramid.
  
and sort polyhedra by their attributes: parallel faces, types of faces, number of faces,
6.4.G Describe
edges, and vertices.
  
6.5. Additional key content (Numbers, Operations)
1
2
6.5.A Use strategies for mental computations with non-negative whole numbers, fractions, and decimals.   
2008 Mathematics Instructional Materials Review Draft Report
Page 131
Evidence
  
0
positive and negative integers on the number line and use integers to represent quantities
6.5.B Locate
in various contexts.
and order positive and negative integers using the number line, lists, and the symbols <,
6.5.C Compare
>, or =.
Evidence
  
  
  
  
Evidence
6.6. Reasoning, problem solving, and communication
0
1
2
6.6.A Analyze a problem situation to determine the question(s) to be answered.
  
6.6.B Identify relevant, missing, and extraneous information related to the solution to a problem.
  
and compare mathematical strategies for solving problems, and select and use one or
6.6.C Analyze
more strategies to solve a problem.
a problem situation, describe the process used to solve the problem, and verify the
6.6.D Represent
reasonableness of the solution.
the answer(s) to the question(s) in a problem using appropriate representations,
6.6.E Communicate
including symbols and informal and formal mathematical language.
  
6.6.F Apply a previously used problem-solving strategy in a new context.
  
and organize mathematical information from symbols, diagrams, and graphs to make
6.6.G Extract
inferences, draw conclusions, and justify reasoning.
and test conjectures based on data (or information) collected from explorations and
6.6.H Make
experiments.
  
2008 Mathematics Instructional Materials Review Draft Report
Page 132
  
  
  
Evidence
Date:
Grade 7
Program:
Reviewer #:
(Rate each item on the scale 0-not met, 1-partial met, 2-fully met)
7.1. Rational numbers and linear equations (Numbers, Operations, Algebra)
0 1 2
7.1.A Compare and order rational numbers using the number line, lists, and the symbols <, >, or =.
7.1.B
Represent addition, subtraction, multiplication, and division of positive and negative integers
visually and numerically.
  
  
7.1.C Fluently and accurately add, subtract, multiply, and divide rational numbers.
  
7.1.D Define and determine the absolute value of a number.
  
7.1.E Solve two-step linear equations.
  
Write an equation that corresponds to a given problem situation, and describe a problem situation
7.1.F that corresponds to a given equation.
  
7.1.G Solve single- and multi-step word problems involving rational numbers and verify the solutions.
2008 Mathematics Instructional Materials Review Draft Report
Page 133
  
Evidence
7.2. Proportionality and similarity (Operations, Geometry/Measurement, Algebra)
0
1
2
7.2.A Mentally add, subtract, multiply, and divide simple fractions, decimals, and percents.
  
7.2.B Solve single- and multi-step problems involving proportional relationships and verify the solutions.
  
7.2.C Describe proportional relationships in similar figures and solve problems involving similar figures.
  
7.2.D Make scale drawings and solve problems related to scale.
  
7.2.E Represent proportional relationships using graphs, tables, and equations, and make connections
among the representations.
7.2.F Determine the slope of a line corresponding to the graph of a proportional relationship and relate
slope to similar triangles.
Determine the unit rate in a proportional relationship and relate it to the slope of the associated
7.2.G line.
  
7.2.H Determine whether or not a relationship is proportional and explain your reasoning.
  
7.2.I
Solve single- and multi-step problems involving conversions within or between measurement
systems and verify the solutions.
7.3. Surface area and volume (Algebra, Geometry/Measurement)
Determine the surface area and volume of cylinders using the appropriate formulas and explain
7.3.A why the formulas work.
Describe the effect that a change in scale factor on one attribute of a two- or three-dimensional
7.3.C figure has on other attributes of the figure, such as the side or edge length, perimeter, area,
surface area, or volume of a geometric figure.
Solve single- and multi-step word problems involving surface area or volume and verify the
7.3.D
solutions.
2008 Mathematics Instructional Materials Review Draft Report
  
  
  
0
1
2
  
  
7.3.B Determine the volume of pyramids and cones using formulas.
Page 134
Evidence
  
  
Evidence
7.4. Probability and data (Data/Statistics/Probability)
Represent the sample space of probability experiments in multiple ways, including tree diagrams
7.4.A and organized lists.
7.4.B Determine the theoretical probability of a particular event and use theoretical probability to
predict experimental outcomes.
Describe a data set using measures of center (median, mean, and mode) and variability (maximum,
7.4.C minimum, and range) and evaluate the suitability and limitations of using each measure for
different situations.
0
1
2
  
  
  
7.4.D Construct and interpret histograms, stem-and-leaf plots, and circle graphs.
  
7.4.E Evaluate different displays of the same data for effectiveness and bias, and explain reasoning.
  
7.5. Additional key content (Numbers, Algebra)
Graph ordered pairs of rational numbers and determine the coordinates of a given point in the
7.5.A coordinate plane.
Write the prime factorization of whole numbers greater than 1, using exponents when
7.5.B
appropriate.
0
7.6. Reasoning, problem solving, and communication
1
2
  
1
2
7.6.A Analyze a problem situation to determine the question(s) to be answered.
  
7.6.B Identify relevant, missing, and extraneous information related to the solution to a problem.
  
Analyze and compare mathematical strategies for solving problems, and select and use one or
more strategies to solve a problem.
7.6.D Represent a problem situation, describe the process used to solve the problem, and verify the
reasonableness of the solution.
Communicate the answer(s) to the question(s) in a problem using appropriate representations,
7.6.E including symbols and informal and formal mathematical language.
  
7.6.F Apply a previously used problem-solving strategy in a new context.
  
Extract and organize mathematical information from symbols, diagrams, and graphs to make
inferences, draw conclusions, and justify reasoning.
7.6.H Make and test conjectures based on data (or information) collected from explorations and
experiments.
  
7.6.G
2008 Mathematics Instructional Materials Review Draft Report
Page 135
Evidence
  
0
7.6.C
Evidence
  
  
  
Evidence
Date:
Grade 8
Program:
Reviewer #:
(Rate each item on the scale 0-not met, 1-partial met, 2-fully met)
8.1. Linear functions and equations (Algebra)
0 1 2
8.1.A Solve one-variable linear equations.
  
8.1.B Solve one- and two-step linear inequalities and graph the solutions on the number line.
  
Represent a linear function with a verbal description, table, graph, or symbolic expression, and
make connections among these representations.
8.1.D Determine the slope and y-intercept of a linear function described by a symbolic expression, table,
or graph.
8.1.E Interpret the slope and y-intercept of the graph of a linear function representing a contextual
situation.
  
8.1.F Solve single- and multi-step word problems involving linear functions and verify the solutions.
  
Determine and justify whether a given verbal description, table, graph, or symbolic expression
8.1.G represents a linear relationship.
  
8.1.C
8.2.A
8.2.B
8.2.C
8.2.D
8.2.E
8.2. Properties of geometric figures (Numbers, Geometry/Measurement)
Identify pairs of angles as complementary, supplementary, adjacent, or vertical, and use these
relationships to determine missing angle measures.
Determine missing angle measures using the relationships among the angles formed by parallel
lines and transversals.
Demonstrate that the sum of the angle measures in a triangle is 180 degrees, and apply this fact to
determine the sum of the angle measures of polygons and to determine unknown angle measures.
Represent and explain the effect of one or more translations, rotations, reflections, or dilations
(centered at the origin) of a geometric figure on the coordinate plane.
Quickly recall the square roots of the perfect squares from 1 through 225 and estimate the square
roots of other positive numbers.
  
  
0
1
2
  
  
  
  
  
8.2.F Demonstrate the Pythagorean Theorem and its converse and apply them to solve problems.
  
8.2.G Apply the Pythagorean Theorem to determine the distance between two points on the coordinate
plane.
  
2008 Mathematics Instructional Materials Review Draft Report
Page 136
Evidence
Evidence
8.3. Summary and analysis of data sets (Algebra, Data/Statistics/Probability)
0
1
2
Evidence
  
8.3.A Summarize and compare data sets in terms of variability and measures of center.
Select, construct, and analyze data displays, including box-and-whisker plots, to compare two sets
8.3.B of data.
  
Create a scatterplot for a two-variable data set, and, when appropriate, sketch and use a trend line
8.3.C
  
to make predictions.
Describe different methods of selecting statistical samples and analyze the strengths and
8.3.D
  
weaknesses of each method.
8.3.E Determine whether conclusions of statistical studies reported in the media are reasonable.
  
8.3.F Determine probabilities for mutually exclusive, dependent, and independent events for small
sample spaces.
Solve single- and multi-step problems using counting techniques and Venn diagrams and verify the
8.3.G solutions.
  
8.4. Additional key content (Numbers, Operations)
8.4.A Represent numbers in scientific notation, and translate numbers written in scientific notation into
standard form.
0
  
1
2
  
8.4.B Solve problems involving operations with numbers in scientific notation and verify solutions.
  
Evaluate numerical expressions involving non-negative integer exponents using the laws of
8.4.C exponents and the order of operations.
  
8.4.D Identify rational and irrational numbers.
  
2008 Mathematics Instructional Materials Review Draft Report
Page 137
Evidence
8.5. Reasoning, problem solving, and communication
0
1
2
8.5.A Analyze a problem situation to determine the question(s) to be answered.
  
8.5.B Identify relevant, missing, and extraneous information related to the solution to a problem.
  
Analyze and compare mathematical strategies for solving problems, and select and use one or
more strategies to solve a problem.
Represent a problem situation, describe the process used to solve the problem, and verify the
8.5.D
reasonableness of the solution.
8.5.E Communicate the answer(s) to the question(s) in a problem using appropriate representations,
including symbols and informal and formal mathematical language.
  
8.5.F Apply a previously used problem-solving strategy in a new context.
  
Extract and organize mathematical information from symbols, diagrams, and graphs to make
8.5.G inferences, draw conclusions, and justify reasoning.
Make and test conjectures based on data (or information) collected from explorations and
8.5.H experiments.
  
8.5.C
2008 Mathematics Instructional Materials Review Draft Report
Page 138
  
  
  
Evidence
Math Instructional Materials Evaluation – Other Factors
Date:
Program:
Reviewer #:
Strongly
disagree
disagree
Grade:
Program Organization and Design
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
agree
Strongly
agree
(Rate each item on the scale of 1-Strongly disagree, 2.-Disagree, 3-Agree, 4-Strongly agree)
The content has a coherent and well-developed sequence (organized to promote student learning, links facts and concepts in a way that supports retrieval,
builds from & extends concepts previously developed, strongly connects concepts to overarching framework)
   
Program includes a balance of skill-building, conceptual understanding, and application
Tasks are varied: some have one correct and verifiable answer; some are of an open nature with multiple solutions
The materials help promote classroom discourse
The program is organized into units, modules or other structure so that students have sufficient time to develop in-depth major mathematical ideas
st
The instructional materials provide for the use of technology which reflects 21 century ideals for a future-ready student
Instructional materials include mathematically accurate and complete indexes and tables of contents to locate specific topics or lessons
The materials have pictures that match the text in close proximity, with few unrelated images
Materials are concise and balance contextual learning with brevity
Content is developed for conceptual understanding: (limited number of key concepts, in-depth development at appropriate age level)




































Balance of Student Experience
1 2 3 4
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.










Tasks lead to conceptual development of core content, procedural fluency, and core processes abilities including solving non-routine problems
Tasks build upon prior knowledge
Tasks lead to problem solving for abstract, real-world and non-routine problems
Tasks encourage students to think about their own thinking
The program provides opportunities to develop students’ computational fluency using brain power without use of calculators
Tasks occasionally use technology to deal with messier numbers or help the students see the math with graphical displays
The program promotes understanding and fluency in number sense and operations
The program leads students to mastery of rigorous multiple-step word problems
The materials develop students’ use of standard mathematics terminology/vocabulary
Objectives are written for students
1.
2008 Mathematics Instructional Materials Review Draft Report
Page 139






























1.
2.
3.
4.
5.
6.
7.
8.
The instructional materials provide suggestions to teachers on how to help students access prior learning as a foundation for further math learning
The instructional materials provide suggestions to teachers on how to help students learn to conjecture, reason, generalize and solve problems
The instructional materials provide suggestions to teachers on how to help students connect mathematics ideas and applications to other math topics, other
disciplines and real world context
Background mathematics information is included so that the concept is explicit in the teacher guide
Instructional materials help teachers anticipate and surface common student misconceptions in the moment
The materials support a balanced methodology
Math concepts are addressed in a context-rich setting (giving examples in context, for instance)
Teacher’s guides are clear and concise with easy to understand instructions
Assessment
1.
2.
3.
4.
5.
6.
7.
8.
6.
agree
Strongly
agree
   
   
   




















1 2 3 4
The program provides regular assessments to guide student learning
There are opportunities for student self-assessment of learning
Assessments reflect content, procedural, and process goals and objectives
The program includes assessments with multiple purposes (formative, summative and diagnostic)
Assessments include multiple choice, short answer and extended response formats.
Recommended rubrics or scoring guidelines accurately reflect learning objectives
Recommended rubrics or scoring guidelines identify possible student responses both correct & incorrect
Accurate answer keys are provided
Equity and Access
1.
2.
3.
4.
5.
disagree
Strongly
disagree
Instructional Planning and Professional Support
































1 2 3 4
The program provides methods and materials for differentiating instruction (students with disabilities, gifted/talented, ELL, disadvantaged)
Materials support intervention strategies
Materials, including assessments are unbiased and relevant to diverse cultures
Materials are available in a variety of languages
The program includes easily accessible materials which help families to become active participants in their students’ math education (e.g. “How You Can Help
at Home” letters with explanations, key ideas & vocabulary for each unit, free or inexpensive activities which can be done at home, ideas for community
involvement)




The program includes guidance and examples to allow students with little home support to be self-sufficient and successful
   
2008 Mathematics Instructional Materials Review Draft Report
Page 140












   
Appendix C.
Acknowledgements
Hundreds of people contributed toward the success of the project. Many are listed below.
We wish to acknowledge countless others who provided input into the process – parents,
teachers, district administrators, business and technical leaders, mathematicians, and
other concerned individuals who shared their ideas and feedback on the process and
results.
OSPI staff Jessica Vavrus led the project. Lexie Domaradzki provided crucial executive
oversight. Michelle Mullins, Judy Decker, Megan Simmons and several others provided
key logistical and operations support. Greta Bornemann, Karrin Lewis and Boo Drury
provided mathematics content support.
Relevant Strategies staff Nicole Carnegie provided the bulk of the statistical analysis.
Eugene Ryser coordinated the data collection process. Dr. June Morita provided expert
analysis on the statistical methods. Porsche Everson was the lead author and contributed
to the statistical analysis.
IMR Advisory Group
Name
Organization
Amy MacDonald
Bellevue School District
Anne Kennedy
ESD 112
Carol Egan
Bellingham School District
Carolyn Lint
Othello/Renton School District
Christine Avery
Edmonds School District
David Tudor
OSPI
Fran Mester
Monroe School District
Heidi Rhode
Evergreen School District
Jane Wilson
Evergreen School District
Janey Andrews
Bellevue School District
Karrin Lewis
OSPI
Kristen Pickering
Bellevue School District
Layne Curtis
Vancouver School District
Lexie Domaradzki
OSPI
Linda Thornberry
Bellevue School District
Matt Manobianco
Lake Washington School District
Nicole Carnegie
Relevant Strategies
Porsche Everson
Relevant Strategies
Sheila Fox
S.B.E.
Terrie Geaudreau
ESD 105
2008 Mathematics Instructional Materials Review Preliminary Draft Report
Page 141
IMR Advisory Group
Name
Organization
Terry Rose
Everett School District
Tony Byrd
Edmonds School District
State Board of Education Math Panel
Name
Organization
State Board of Education
Steve Floyd
Math Panel Chair
Brad Beal
Whitworth University
Bob Brandt
Parent
Jane Broom
Microsoft
Dr. Helen Burn
Highline Community College
Dr. Christopher Carlson
Fred Hutchinson
Timothy Christensen
Agilent Technologies
Bob Dean
Evergreen 114 School District
Danaher Dempsey, Jr
Seattle School District
Tracye Ferguson
Tacoma School District
Dr. Elham Kazemi
University of Washington
Yakima Valley Community College & Parent
Paulette Lopez
Advocate
Bob McIntosh
North Thurston School District
Linh-Co Nguyen
Seattle School District & Parent
Dr. Larry Nyland
Marysville School District
Amanda Shearer-Hannah
Bellingham School District
Dr. Kimberly Vincent
Washington State University
Edie Harding
State Board of Education
Kathe Taylor
State Board of Education
K-8 Review Team
Name
Organization
Valerie Adams
Pasco SD
Karen Albers
Richland (Parent)
Dana Anderson
Stanwood Camano SD
Amy Barber
Peninsula SD
Peggy Bergstrom
Richland SD
Debbie Blodgett-Goins
Yakima SD
Robert Brandt
Lake Washington SD
Don Bullis
Othello SD
Richard Burke
Measurement Technology Northwest, Inc.
Heidi Busk
Eastmont SD
2008 Mathematics Instructional Materials Review Preliminary Draft Report
Page 142
K-8 Review Team
Name
Organization
Bruce Camblin
Change Systems for Educators
Cristina Charney
Olympia SD
Ida Lynn Cowart
Kent (Educational Consultant)
Vickie Crane
Lynden SD
Linda Digiorgi
Franklin-Pierce SD
Michelle Fox
Puyallup SD
Sandy Gady
Highline SD
Tamarah Grigg
Vancouver SD
Janet Johnson
Everett SD
Paulette Johnson
Rainier SD
Heather Jordan
West Valley SD
Debra Knox
Moses Lake SD
Mark Laurel
Auburn (Parent)
Linda LoBue
San Juan Island SD
Maria Lourdes Flores
Clover Park SD
Sherry Marlin
Washington State PTA
Monica McDaniel
Stanwood Camano SD
Heather McMullen
Puyallup SD
Monica Nelson
Walla Walla SD
Remy Poon
Seattle Public Schools
Kim Prothero
Federal Way SD
Keith Quentin
Centralia SD
Jeremy Rogers
Ocean Beach SD
Lisa Rollins
Puyallup SD
Toni Sadlowski
Yelm SD
Amy Schall
Eastmont SD
Barbara Shinn
Mukilteo SD
Julianne Sparks
Sumner SD
Paula Strozyk
Sumner SD
Kate Venneri
Walla Walla SD
Teresa Wendland
South Kitsap SD
Jamie Zerb
Pasco SD
National Experts and External Leaders
Name
Charlene Tate-Nicols
Jonathan Weins, Drew Hinds
James Milgram
State
Connecticut
Oregon
California
2008 Mathematics Instructional Materials Review Preliminary Draft Report
Page 143
National Experts and External Leaders
Name
State
Jane Cooney
Indiana
Charlotte Hughes
North Carolina
Skip Fennell
George Bright
Maryland
Washington
2008 Mathematics Instructional Materials Review Preliminary Draft Report
Page 144