Kim Gilson
Sr Consultant Data and Accountability
Region 10 ESC
kim.gilson@region10.org
972-348-1480
February 5, 2015 Webinar
It is the policy of Region 10 Education Service Center not to discriminate on the basis of race, color, national origin, gender or handicap
in its vocational programs, services or activities as required by Title VI of the Civil Rights Act of 1964, as amended; Title IX of the
Educational Amendments of 1972; and Section 503 and 504 of the Rehabilitation Act of 1973, as amended. Region 10 Education Service
Center will take steps to ensure that lack of English language skills will not be a barrier to admission and participation in all educational
programs and services.
Sound Check
Please
let me know if you can
hear me!
Sound Check Complete!

If you cannot here me, please try adjusting the volume on
your speakers. If this does not work, you can call the
number in your registration email and listen through your
phone.
Agenda
 Federal
 State
 2015
Legislative Update
Accountability Preview
 Index
 Misc.
Legislative Update
1-4 Things to Think About
Federal Accountability

Reminder:

Our current Accountability System was accepted by the USDE as part of a
conditional waiver from AYP requirements

Part of the waiver centered on developing a new teacher appraisal system

Press Release from January 23 tells us the USDE is not satisfied with the
teacher appraisal system we are proposing


http://tea.texas.gov/About_TEA/News_and_Multimedia/Press_Releas
es/2015/Commissioner_Williams_reaffirms_state_s_interests_as_waiv
er_discussions_continue/
Our Federal Safeguards are set to rise to 83% this year, and reach 100% by
2019-2020.
ESEA “NCLB” Reauthorization

There are efforts to reauthorize No Child Left Behind, also
referred to as the Elementary and Secondary Education
Act

The House and Senate both have bills

Stay tuned……
The 84th Legislature Is In Session!
https://www.youtube.com/watch?v=FFroMQlKiag
2015 Accountability Development:
tea.texas.gov
2015 Accountability

ATAC met Dec 8

APAC met Jan 21

ATAC meets Feb 5

ELL Working Committee meets this week

Materials Posted Online


http://ritter.tea.state.tx.us/perfreport/account/2015/materials.html
Timeline
APAC Documents
A-F Ratings for Districts

From HB 5

Implemented in 2016-2017

http://ritter.tea.state.tx.us/perfreport/account/2015/20150121mtg/af_options.pdf
ATAC Documents
Bridge Study for New Math TEKS

Remember “TAKS to STAAR” in 2012?
Bridge Study Process
Levels of the Bridge Study
Index 1, 3, and 4!!!!
“State Assessments Evaluated in 2015
Accountability”

The STAAR mathematics bridge studies will compare performance on the new
STAAR mathematics assessments to performance on the 2012–2014 STAAR
mathematics assessments in order to ensure that the performance standards
applied for 2015 accountability purposes are equivalent to the original STAAR
mathematics performance standards. So, for 2015 accountability, districts will
be held to the old performance standards. Information about student
performance in relation to the new performance standards will be available in
fall 2015, but will not be used for accountability purposes in 2015.
Commissioner Williams Recent
Statements

See Dallas Morning News Article

http://www.dallasnews.com/news/local-news/20150201-texas-may-help-schooldistricts-deal-with-tough-new-math-standards.ece
From the education commissioner to members of the State Board of Education to
staffers at the Texas Education Agency, there was talk of additonal teacher training
and chopping the stakes of state-required math tests.
As Commissioner Michael Williams told an auditorium packed with superintendents
and educators from around the state: “We’ll try to make the best decision we
possibly can. But I hear you! I hear you, OK?”
Dallas Morning News Article (cont’d)

Alief Superintendent H.D. Chambers is suggesting that the commissioner
create a “hold harmless” standard for the math STAAR tests for a couple of
years so the scores could only be used to raise accountability ratings.

Texas Education Commissioner Michael Williams, who has considerable control
over the design of the state accountability standards, mentioned a suggestion
that there be “achievement bands” — broad ranges of scores rather than
specific passing points — which would create greater flexibility.
ELL Progress Measure and New Math
TEKS

From “State Assessments Evaluated in 2015 Accountability”

ELL Progress Measures. For students tested in grades 3–8 mathematics, the ELL
student expectations used to determine the ELL Progress Measure results will be
aligned to the STAAR Mathematics Bridge Study equivalent performance standards.
ELL Progress Measures will be available for use in accountability for students tested
in grades 3–8 mathematics; Over 120,000 ELL students in Years 2 – 4 will continue
to be included in 2015 accountability for all subjects.

This means: The calculation will exist for ELL’s in Math to receive an Index 2
score. If those scores are included, then that group will comprise the grades 3-8
Math portion of Index 2 (in addition to accelerated students in Algebra 1)
Score Codes from STAAR Alternate 2 and
Participation/Performance from “State Assessments Evaluated
in 2015 Accountability”
ATAC Documents

Which types of tests work in which Indexes?


http://ritter.tea.state.tx.us/perfreport/account/2015/20141208mtg/assessments_
evaluated_p1.pdf
Which grades and subjects work in which Indexes?

http://ritter.tea.state.tx.us/perfreport/account/2015/20141208mtg/assessments_
evaluated_p2.pdf
Options for Each Index

http://ritter.tea.state.tx.us/perfreport/account/2015/20141208mtg/targetsri.pdf
Post-Secondary Indicators

http://ritter.tea.state.tx.us/perfreport/account/2015/20141208mtg/postsec_cr_options.pdf
New (Old) Vocabulary

“Required Improvement”

Not to be confused with Improvement Required!


You don’t meet the standard, but you grow enough (defined by a formula)
towards the standard that you are not penalized
“Hold Harmless”

If the only reason you miss the target in an Index is because of a particular reason
(STAAR A or STAAR Alternate 2), you are not penalized

Only included if it helps you
Index 1
Things to Think About…
Things to Think About…..

Index 1

Target went up last year

Passing standard is Level I (for the last time!)

Shannon Houssen said in a fall TETN that the target is likely to rise

Federal Requirements say we must be accountable for what we test


New tests “work” easier in Index 1
Index 1 scores can remain stable even if performance drops, as long as
performance doesn’t drop below passing standard

Just because 2014 Index 1 rates remain stable doesn’t mean they’ll survive a rising target
if student performance is declining

Example: 2013 Every student makes a perfect score on their test, Index 1 score is 100

2014 Same students drop by 30%, but are still above the passing standard, Index 1 score is
still 100

2015 Same students drop by an additional 30%, and this takes them below the passing
standard for their grade/subject: Missed Index 1!
Index 1 Can Be Misleading

Region 10 has 2 schools with Index 1 scores in the 90’s who are
Improvement Required because of Index 2

Kids scored above the passing standard

But did not grow

Likely showed decline, but not enough to drop below passing
standard
Index 2
Things to Think About…
Things to Think About…

Index 2

The Commissioner says we will not have Index 2 for Grades 3-8 Math because no
growth measure can be calculated for new Math TEKS

HOWEVER, the ELL Progress Measure is a fixed target and does not require 2 years
of vertical scale scores

ELL’s can still have an Index 2 score
 WILL
IT BE USED IN ACCOUNTABIILTY?????
 Watch
 Keep
for news from the ELL Meeting this week….
watching ATAC/APAC
Things to Think About….

Index 2

Elementaries

Reading is only subject left…



3rd Grade ELL’s CAN be included via ELL Progress Measure!!!
Middle Schools


Except for ELL’s in Math
Reading and Writing…and ELL’s in Math and Algebra 1!!!!

Watch ELL committee this week for recommendations

What do middle school Algebra 1 Index 2 scores look like???

Writing could be 1/3 of Index 2 (or ½ if no Math and your campus has no Algebra 1)!!!!!
High Schools

English II for All (very little 2014 data; STAAR Alt students)

English I for ELL’s (watch ELL committee for recommendations)

Algebra 1 (Look at 2014 data!)
STAAR A

If a student took Regular STAAR in 2014, and they take STAAR A in 2015, a
Progress Measure CAN be calculated, but the document recommends not
USING it
Changing Languages

Reading


Spanish one year, English the next

The test is NOT a direct translation. Vertical scale scores CANNOT be subtracted.
No progress measure

UNLESS student qualifies for ELL Progress measure
English one year, Spanish the next


Writing


The test is NOT a direct translation. Vertical scale scores CANNOT be subtracted.
No progress measure
4th Grade Spanish, 7th Grade English

The test is NOT a direct translation. Vertical scale scores CANNOT be subtracted.
No progress measure

UNLESS student qualifies for ELL Progress measure

Kids have likely “aged out” of ELL Progress measure
Math

In a “normal” year, changing versions still enables the student to receive a Progress
Measure because the test is a direct translation
Changing Languages…The Exception

A small number of students may have taken the STAAR reading Spanish version
in 2013, and transition in 2014 to the STAAR reading English version, but do
not have a STAAR progress measure or ELL progress measure. For example, a
student takes the grade 5 STAAR reading Spanish version during the spring
2013 administration. The following year, the student is tested on the grade 6
STAAR reading English test version. If the student is not eligible for, or
exceeds the time frame of their ELL progress measure plan, the ELL progress
measure will not be reported. In addition, a STAAR progress measure cannot
be calculated because the language versions have changed. Specifically,
STAAR progress measures for reading are calculated only for students who test
in the same language in the prior year and the current year.
Changing Languages….The Exception
Part 2

To address these unique cases in which students have taken the STAAR reading
Spanish version in 2013, and transition in 2014 to the STAAR reading English
version, but do not have a STAAR progress measure or ELL progress measure,
Index 2 is calculated as follows: o Phase-in 1 Level ll (English version): One
point for each percentage of tests meeting the STAAR Phase-in 1 Level II or
above; and o Final Level ll (English-version): One additional point for each
percentage of tests meeting the Final Level II standard.

From Appendix I of 2014 Accountability Manual

Region 10 does NOT advocate making language test version decisions based on
Accountability. Do what’s best for the students!
Things to Think About….


Minimum Size

Watch your student group sizes!

Look at October 2014 Snapshot Information for a Guide

Once a group meets size of 25 tests, they contribute points in the same year!

25 is an easy number to grow above, or decline below

Impact depends on how they peform
High Schools

Look at Quartile Ranking in Distinction Designation Documents for Alg 1 Comparison

How do you compare to your Quartile?

How does your Quartile compare to others?
Things to Think About…

Index 2 could be a leading indicator for Index 1

Going back to Index 1 Example…


If student performance declines consistently in Index 2, it could eventually drop below the
standard and target in Index 1
Index 2 Predicting Index 2

Student group performances that don’t meet minimum size are not reported on
Accountability reports, but they may be on the TAPR (if not masked)


If masked, ask for numbers from your unmasked TAPR!
Example:

2014 Student group had 22 students and did not contribute points.

No data on Accountability Report

But, none of the 22 students met or exceeded growth

2015, student group has 26 students

The group now contributes points

Are they growing????
Index 3
Things to Think About
Things to Think About…

Index 3

Groups are based on which ones made minimum size LAST YEAR (and maintain minimum
size this year)

Example:

2013 Group A has 19 kids


2014 Group A has 27 kids


Not an eligible group for 2014 Index 3
Not an eligible group for 2014 because it didn’t meet minimum size in
2013
2015 Group has 28 kids

This is now an eligible group because it met minimum size in 2014 AND
2015

It could replace a group that was already representing you, or add an
additional group to Index 3
Things to Think About….

Index 3

How do ALL Students perform at Advanced Level III?

Not on accountability report!

IS on TAPR!!!!

If Eco Dis, or one of the Index 3 reported groups, is close to 100% of your
population, then the Index 3 report is indicative of what is happening to your
All Student Group

BUT, if your groups do NOT represent most of your student population, then
you need to find All Students on TAPR

Don’t assume the problem is limited to the students reported in Index 3

Is it really a problem of not meeting SOME students’ needs, or is it a
problem that ALL students’ needs are not being met?

Continue to focus on unique needs of student groups reported in Index 3,
but you may have a widespread curriculum issue
Index 4
Things to Think About
Things to Think About….


Index 4: STAAR Component

Student groups contribute if there are 25 tests for the group

Like Index 3, a group who meets minimum size for the first time could surprise you
ELL’s

Not reported as a group, but their scores CAN be EXCLUDED, and CAN impact the
number of tests for a race/ethnic group

ELL’s in 1st year excluded

ELL’s in year 2-4


Test ANY test in Spanish….INCLUDED

If two or more subject-area Spanish test versions are taken, the student must meet the Final Level
II standard on at least two subjects;

If only one subject-area Spanish test version is taken, the student must meet the Final Level II
standard on the single subject-area test.

All Tests in English…EXCLUDED
ELL’s in year 5+

Follow the same rules as Non-ELL’s
Things to Think About…

A student group may be included in Index 2 and 3 because they meet
minimum size, but are NOT included in Index 4 because their ELL Status
excludes them

If they are being less successful than other students, your Index 4 may look
better than it would if they were included

Example: Hispanic group had 25+ tests in Index 2 last year and contributed points.
If they are not being successful, Index 2 is low and you know it .

Index 3: Group met minimum size for the first time in 2014 and wasn’t included
then, but will be for 2015. If the group is not being successful in Index 3, it hasn’t
shown up yet….but will this year!

Index 4: Group does not meet minimum size this year because ELL exclusions bring
it below the minimum size of 25. Index 4 looks strong in 2015. However, group
meets minimum size in 2016. If the group is not achieving Final Level II, you have
an Index 4 concern….and don’t know it yet.
Instructional Difference Between Index
2, 3, and 4

Index 2 is about growing, regardless of passing status

Index 3 and 4 are about achieving at higher levels than “just passing”

A “remediation” mentality may improve Index 2, BUT…..

If students are not seeing questions of similar rigor to the hardest 5-10
questions on the test in class on a regular basis, you haven’t addressed root
cause of Index 3 and 4.

When students see the hardest types of questions on the day of the test, is
that the first time they’ve been asked to think at that level?

If 1 out of every 10 questions in a class is formatted to that complexity, a student
can make an A all year and NEVER ATTEMPT THAT QUESTION
Final Level II Data

Accountability Reports do not report by Subject

TAPR does, and might include student groups not reported on the
Accountability Report (if they’re not masked)

LOOK TO THE TAPR IN ALL AREAS FOR DATA THAT MIGHT NOT BE REPORTED IN
ACCOUNTABILITY REPORT
Index 4 and Retesters

Administration Cycle Includes July, December, and May

Retests ONLY count if first attempt and retest is in that cycle, meaning first
attempt was in July or December

So, retests COULD count, but would be the exception

See TEA document for Index 4 High School Scenarios

http://ritter.tea.state.tx.us/perfreport/account/2014/eoc_scenarios.pdf

Includes mobility issues and “Which high school gets which administration in Index
4?”
Misc.
1 Possible Way to Earn a High School
Math Distinction

Q1 Attendance

Q1 SAT/ACT Participation

Q1 AP/IB Participation
 EARNS

A DISTINCTION, even though…
Q4 Greater than Expected Growth in Math (Index 2 second point)

0 students grew in Index 2 for Algebra 1

Q4 SAT Performance

Q4 Act Performance

Q4 AP/IB Performance
So if Students Aren’t Growing (at an
alarming rate) in High School….

The campus with the gold star for math on their
Accountability Report could be in danger for Index
2 this year
2+ Races

In CAF file, you may see that 2 or more different races are chosen, but for the
accountability field, “2+ races” isn’t the assigned value

If a student is Hispanic and any additional races, he/she is reported for
accountability purposes as Hispanic. Period.


A Hispanic student cannot be part of the 2+ races group
2+ races can only be non-Hispanic students

Example:

African American and White

Native American Indian and African American
Recording Asylee/Refugee Information
 Information about qualifying unschooled ELL asylees and
refugees will be collected as part of the data collection
for TELPAS.
 This information is necessary to exclude eligible
students’ STAAR results from state accountability ratings
and will NOT be gathered during STAAR data collection.
TEA Student Assessment Division
50
Testing Q&A

Can we put Post-It Notes on the portion of the 5/8 Math/Reading answer
document that we’re not using?


Per TEA, “No.”

Post-it Notes might damage answer documents

Active monitoring is best approach
Student is Absent for Day 1 of a 2-day writing test. They arrive day 2. What
do we do?

They take Day 2 with the other students and take Day 1 on the makeup