Next-Generation MCAS Communications Workgroup
Massachusetts Department of Elementary and Secondary Education
75 Pleasant St., Malden, MA 02148 -- Room 202
Tuesday, April 12, 2016, 11:00 am to 1:00 pm
WORKGROUP MEMBERS AND MEETING PARTICIPANTS
Beth Herman-Davis
Board Member
Beverly Miyares
Professional Development Specialist
Bill Lupini
Superintendent
Bob Gilmore
President
Dan Murphy
Director of Education Policy
Dan Wiener
Administrator of Inclusive Assessment
Dave Buchanan
Assistant Director of Literacy and
Humanities
Erica Brown
Chief of Policy and Practice
Gorman Lee
President
Jake Foster
Director of STEM
Jake Oliveira
President
Jass Stewart
Special Assistant to Chief of Staff
Jeff Wulfson
Deputy Commissioner
Martin Geoghegan
Principal
Maureen LaCroix
Special Assistant to the Deputy
Commissioner
Michol Stapel
Associate Commissioner
Rob O’Donnell
Sr. Policy Analyst
Stephanie Gray
President
Steven Rattendi
Past President
Sue Lane
Senior Advisor to the Commissioner for
P-16 Access and Alignment
Theresa Craig
Board Member
1
New England Association of Teachers of English
(NEATE)
Massachusetts Teachers Association (MTA)
Massachusetts Association of School
Superintendents (MASS)
Massachusetts Association of Science Teachers
(MAST)
American Federation of Teachers–Massachusetts
(AFT-MA)
Massachusetts Department of Elementary and
Secondary Education (ESE)
Massachusetts Department of Elementary and
Secondary Education (ESE)
Massachusetts Charter Public Schools Association
(MCPSA)
Massachusetts Council for the Social Studies
(MCSS)
Massachusetts Department of Elementary and
Secondary Education (ESE)
Massachusetts Association of School Committees
(MASC)
Massachusetts Department of Elementary and
Secondary Education (ESE)
Massachusetts Department of Elementary and
Secondary Education (ESE)
Massachusetts Secondary School Administrators’
Association (MSSAA)
Massachusetts Department of Elementary and
Secondary Education (ESE)
Massachusetts Department of Elementary and
Secondary Education (ESE)
Massachusetts Department of Elementary and
Secondary Education (ESE)
Massachusetts Parent Teacher Association
(Massachusetts PTA)
Association of Teachers of Mathematics in
Massachusetts (ATMIM)
Massachusetts Department of Higher Education
(DHE)
Massachusetts Organization of Educational
Collaboratives (MOEC)
MEETING NOTES
Review and approval process of workgroup recommendations
Wulfson: ESE is in the process of compiling and synthesizing all the workgroup recommendations into one document. The
Department will include considerations and concerns for each recommendation – as articulated by the workgroups, with
additions for ESE. ESE staff will then provide revised recommendations in response to the considerations and concerns.
This synthesis document, once approved by the Commissioner, will be shared with educators and the public through a
yet-to-be-determined public comment process. After reviewing the public comments, ESE will prepare a final document for
review by the Commissioner and BESE.
Accountability Workgroup update
Wulfson: Because of changes in our testing system and the new ESSA law, ESE is using this opportunity to review our
accountability system. The accountability team has met twice with BESE’s Accountability and Assistance Advisory Council
(AAAC) to discuss the current design of our system and hear their opinions about potential changes. The accountability
team is currently getting feedback from the field – likes, dislikes, and improvements. From April to June, the accountability
team is listening. Over the summer, the team will assess different versions and put together a working draft of proposed
changes, which will go to BESE at end of the fall. We are aiming to have proposed modifications finalized by the end of
the year to allow time for regulatory changes and additional communication with stakeholders prior to the start of 2017-18.
Q&A
Some experts insist that student growth percentiles (SGP) is not a useful tool. Who are the AAAC members? There is a
huge risk to weighing so much to something that’s so opaque.
There’s nothing more terrifying to a parent than getting a report that your school didn’t meet requirements so you are now
allowed to choose a different school. We have no frame of reference.
As part of this group, our job is to communicate what SGP is to a very important stakeholder – parents.
Wulfson: The 15-member AAAC is made up of educators, school administrations, and education policy experts from
across the state. We also stay connected with experts in other states, national organizations, etc. We also have experts
on staff. We’ll devote a major part of our next meeting to accountability.
Test Administration Workgroup update
Stapel: This group met twice. Members included a variety of roles by areas of expertise, grade level, and whether they
had given MCAS or PARCC tests. We also put together a survey to get the feedback of an additional 50 educators to the
workgroup’s recommendations. The group discussed testing windows, phasing in CBT testing, time and untimed tests,
etc. Workgroup members were more interested in having one, longer testing window, which is the PARCC model. One
window minimizes disruptions during the school year. 3rd-5th grade teachers wanted shorter sessions, upper grades
wanted longer sessions. We haven’t finalized anything yet. Concerning time vs. untimed, there was a 50/50 split. For
untimed tests, students tend to go back over the material unnecessarily. It’s recommended that student with disabilities
take untimed tests. Schools are looking for professional development and training to transition to CBT. For the test
content review panels – where educators review actual test items – we will rely heavily on our current MCAS ADC
structure.
Q&A
What are the recommended testing windows?
Stapel: The proposed testing windows have been placed online and we will forward this information to the group.
A number of superintendents are concerned by the CBT timeline, which is later in 2019. Superintendents sold technology
plans based on testing online in the immediate future.
Stapel: The next-gen MCAS will have CBT and PBT options for next year.
2
Is ESE considering AI scoring of test items? There is going to be a lot of skepticism from the field, and we need to come
up with answers that are not dismissive. Perhaps the Department should consider grading some tests with people and
others with AI to cross validate.
Stapel: We would have to be very comfortable with AI before we move forward with it. But it is something we are
considering. Recent studies show that computers can be more consistent than people. The GRE and other programs
have used automated scoring. With paper, we will always have human scoring.
Accessibility Workgroup update
Wiener: When ESE joined the PARCC consortium, we learned that MA is doing a lot more around accommodations than
other states. We projected our philosophy, and we influenced other states. Many MCAS accommodations made it in, but
some didn’t. We now have the opportunity to learn from other PARCC states and apply those lessons to MA students with
the next-gen MCAS. We brought 30 people together. The group considered issues such as timed vs. untimed testing. We
also addressed concerns with online testing. We now have accessibilities that are available for all students. And we know
that principals over time have asked for more autonomy with administering the test. With the next-gen MCAS, some test
administration decisions will be at the discretion of the principal. Accommodations have decreased because they are now
available to all students. In the age of universal design, it’s better to have fewer accommodations – student should be able
to pick and choose. The workgroup came to agreement on all recommendations – though they may not have been
unanimous.
Q&A
Great that universal design is a part of the next-gen MCAS. As a collaborative, will 100 percent PBT be an option since a
supermajority of students need accommodations?
Wulfson: We urge collaboratives to make sure that kids can learn to use computers for their own academic and career
success.
Will accessibility take time to develop? Will they look/operate the same now as they will in the coming years?
Wiener: We expect that all accommodations will be ready for the 2017 test because the vendor will have an already
established test platform. We are not anticipating an extended development period for the accessibility features.
Will there be language accessibility features beyond Spanish?
Wiener: This is a discussion that the legislature must have about instruction in English. We aren’t sure we can continue in
Spanish. MA, AZ, and CA are the only English only states.
What’s the exposure plan for CBT accommodations?
Wiener: Part of the CBT transition plan will include video tutorials and practice tests.
High School Testing Workgroup update
Foster: This work group looked at the high school testing program for ELA, math, science, and HSS – with special focus
on the CD and additional discussions concerning CCR. The CCR test, which is currently recommended as an optional test
in grade 11, would be used by MA public higher ed as their placement examine into credit bearing courses. The
workgroup included key ESE and DHE staff, as well as 35 educators – teachers, principals, department heads,
superintendents, and college professors – from across the state. ESE held two planning meetings and four workgroup
sessions. They landed on three options. Option 1 keeps the current MCAS components and adds tests. Option 2 keeps
grades 9-10 components and simplifies 11-12. And Option 3 proposes a dynamic-adaptive ELA and math testing system
and simplifies grades 11-12. In all cases, the science tests remain the same, and HSS is added. The workgroup also laid
out a very extensive transition plan for educators and students.
Q&A
During PARCC high school testing, kids knew the test didn’t count so they didn’t take it seriously. We need to, as a
communications group, explain to parents why the CCR test is worth “opting in” vs. opting out.
3
Foster: The workgroup has recommended ways to incentivize students to take the CCR test. The idea is it would be used
for placement for college; but eventually ESE could partner with the business community to use the test for placement in a
career. We’re also thinking of tying the Adams/Koplik awards to the CCR test.
Lane: The CCR test is meant to help high achieving students speed up and help students who are off track catch up.
There is general concern that higher ed, because it’s a money maker, is more interested in pushing students into remedial
courses. Colleges are not aligned to the K-12 system.
Lane: DHE has been working with K-12 educators since the inception of PARCC – alignment is one of the key goals of
the assessment system. We are continuing this work with MCAS 2.0. And, in addition to the interest of public colleges and
university in the CCR test, private schools are also showing an interest.
The workgroup chart indicates that some of the options offer fewer tests. That’s true from the state’s perspective of having
to develop fewer tests. But in reality, all the options mean more testing for students.
Foster: Correct.
Wulfson: We will send the group ESE’s recent study on the amount and uses of testing in districts.
ELA/Math Standards Review Panel update
Foster: Our second meeting is tomorrow. We’ve gotten a number of public survey responses – about 1,800 total; 400
completed surveys. We will use the survey responses to guide where the panel might recommend refinements. From the
survey, we’ll look at a couple of big areas that need attention. For example, how is disciplinary literacy playing out and
how do we make that clear in mathematics. We have received a number of discrete suggestions, which we’ll deal with
separately. The June process will lead into the public process, which will include the board.
HSS Standards Review Panel and Assessment Workgroup update
Buchanan: In the last couple of months, we’ve been outreaching to educators to review the HSS framework which dates
back to 2003. We are reviewing 150 applications. We anticipate having the panel in place by the end of the school year.
We will be establishing a survey similar to the ELA/math webform that will be released once the panel is formed.
Responses will be used to guide the work of the review panel. Simultaneously, we will be developing an assessment that
will parallel the framework.
Wulfson: We are interested in exploring different assessment models that use PBAs, etc., including models in those states
that have federal money to develop innovative designs. The challenge is finding models that can be scaled up and
administered fairly.
Procurement Workgroup update
O’Donnell: The RFR was posted in March. We’re now responding to and posting vendor questions on a rolling basis. We
are also forming a review team made up of ESE staff, K-12 educators, and higher ed faculty. Responses are due on May
6. The review process will also include vendors demoing their online products. The goal is to have a recommended
vendor by mid June, then to negotiate and select a vendor by mid July to begin test development by late fall.
What is the contingency plan?
Stapel: While the schedule is aggressive, we have enough latitude to ensure we have a test in 2017. Also, much of the
new test – at least in the first year – will be composed of current MCAS and PARCC items. We are not starting from
scratch.
Will the ballot measure affect testing this spring?
Wulfson: No, not for the spring. It’s a prospective measure.
4
Next-gen MCAS presentations at/by professional associations
Stewart: ESE is reaching out to 25 groups, requesting presentations to provide MCAS 2.0 updates to educators, parents,
policymakers, and other stakeholder. All Communications Workgroup members have been contacted, and a number of
presentations have been confirmed. Workgroup members stressed that ESE needs to develop “really simple”
presentations for parents and the general public.
5