2015 TELPAS HOLISTIC
TRAINING FOR CTCS
Assessment, Research & Evaluation Department
January 27, 2015
DISCLAIMER
This training does NOT
take the place of
reading the appropriate
manuals.
2
GENERAL
INFORMATION
3
2014 TELPAS Changes
New standards were set for TELPAS reading in 2014
Domain weights shifted in composite score
calculations.
– Reading is 50%
– Writing is 30%
– Listening and Speaking are each weighted 10%
4
Years in US Schools Data Collection
Years in US Schools Data Collection Changes (DCCM pg. T-30)
– Beginning with calculations made and reported in 2014, students
must be enrolled for 60 consecutive school days in a school year
for that year to count as a year in US schools.
Enrolled for 60 consecutive school days = 1 year in the US
Count restarts if student
– WD’s prior to reaching 60 days and
– Does not enroll in another US school for 10+ consecutive
school days
If enrolled within the final 60 school days of a school year, then next
year will still be 1 st year in US.
5
Years in US Schools Data Collection
 Campuses cannot change the value for
Important
Reminders
years in U.S. schools submitted in a
previous school year.
 Campuses will use the data previously
submitted to generate data that will be
submitted this spring.
 The number of years in U.S. schools on
record for a student cannot decrease.
The value will either remain
the same or increase by 1.
TELPAS Assessment Components
Grades K-1
Grades 2-12
Holistically rated
observational
assessments
– Listening
– Speaking
– Reading
– Writing
Multiple choice online tests in
grades 2-12
– Reading
Holistically rated
– Writing collections
Holistically rated
observational assessments
– Listening
– Speaking
7
Eligibility Requirements
All ELL’s in grades Kinder – 12th are
required to participate.
– including PD’s
In rare circumstances, it may be necessary
for the ARD & LPAC committee to determine
a student should not be assessed in any
given domain.
– Participation must be considered on a
domain-by-domain basis and reasons must
be supported and documented in the IEP
Send the DTC a
TELPAS Move
Form beginning
February 18,
2015.
Handout #1
8
Eligibility Requirements
Handout #1
Date:______
Date:______
An ELL from another school district, state, or
country who enrolls on or after the first day
of the TELPAS testing window will not be
assed by the receiving district in the
HOLISTICALLY rated domains
All newly enrolled ELL’s in grades 2-12 who
enroll on or after the first day of the testing
window, are ONLY required to take the
TELPAS reading test; however, TELPAS
coordinators must verify with DTC that the
student hasn’t already completed testing
elsewhere
9
TEST SECURITY
AND
CONFIDENTIALITY
REQUIREMENTS
2015 TELPAS Rater Manual – pg. 3
10
Test Security
Maintaining the confidentiality of the TELPAS program involves
protecting the contents of all online assessments and student
performance documentation. This requires compliance with, but is
not limited to, the following guidelines
– Personnel meet the requirements, have been trained and
have signed the appropriate oath to have access to the
assessment materials and information
– Campuses implement the controls necessary to ensure
proper storage of secure materials throughout the test
administration
– Campuses place confidential documentation (rating rosters)
in a limited-access locked storage
11
Test Security
Campuses must
account for all
materials
before
during
after
each test
administration.
Campuses ensure that:
 Only personnel who meet the
requirements participate
 They have been trained
 They have signed an oath
 Materials are properly stored
(locked storage)
 Rating rosters are secured when
not in use
Confidentiality Requirements
Testing personnel undergo training and sign
the oath affirming they understand their
obligations concerning the security and
confidentiality of the TELPAS program
Testing personnel administer the tests in
strict accordance with the instructions
Testing personnel do not view, reveal or
discuss the contents of an assessment
before, during, or after a test administration
13
Penalties for Prohibited Conduct
Any person who violates, assists in the violation of,
or solicits another to violate or assist in the
violation of test security or confidentiality, as well
as any person who fails to report such violation is
subject to the following penalties
• Placement of restrictions on renewal or holding of
educator certificate
• Issuance of an inscribed or non-inscribed reprimand
• Suspension of a Texas educator certificate for a set
term
• Revocation or cancelation of an educator certificate
without opportunity for reapplication for a set term or
permanently
14
Penalties for Prohibited Conduct
(continued)
Release or disclosure of confidential test content could
result in criminal prosecution under TEC §39.0303,
Section 52.352 of the Texas Government Code, and
Section 37.10 of the Texas Penal Code. Further, 19 TAC
§249.15 stipulates that the State Board for Educator
Certification may take any of the above actions based
on satisfactory evidence that an educator has failed to
cooperate with TEA in an investigation.
Additionally, irregularities resulting in a breach of test
security or confidentiality may result in the invalidation
of students’ assessments.
15
Security Oaths
and
Confidentiality
Statements
• All personnel must meet the
eligibility requirements and sign a
security oath
• Any person who has more than
one testing role must receive
appropriate training and sign a
security oath for EACH role
• Each oath must be read and
completed AFTER the training
and BEFORE handling or viewing
any secure materials or
confidential information
• Oaths must be kept on file for at
least five years
16
Testing Irregularities
Serious
These constitute severe
violations of test security
and/or confidentiality
and can result in the
individual(s) responsible
being referred to the TEA
Educator Standards and
Certification Legal
Division for consideration
of disciplinary action.
Examples
– Falsifying TELPAS ratings or
writing samples
– Assisting students directly
or indirectly
– Tampering with student
responses
– Directing a student to
rework or change answers
17
Testing Irregularities
NOTE!!!!
It is a violation of state assessment procedures and a
serious testing irregularity to record, discuss, or share
answers to the rating practice and calibration activities.
– NEW: After completion of calibration
activities, raters must destroy all notes taken
about specific student profiles.
Testing Irregularities
Procedural
Less severe and
more common
irregularities that
are typically the
result of minor
deviations in
testing procedures.
Examples:
– Eligibility error
– IEP implementation
issues
– Improper accounting of
secure materials
– Monitoring error
– Procedural error
19
Testing Irregularities (cont.)
Procedural Irregularities (examples)
Eligibility error
– Student not rated in one
or more domains
– Student not administered
the reading test
IEP Implementation error
– SPED student was/was
nor provided an
accommodation
Testing Irregularities (cont.)
Procedural Irregularities (examples)
Improper Account of Secure
Materials error
– Teacher misplaced a writing
collection
Monitoring error
– TA left room unattended when
online tests were open
– Student was not prevented
from using cell phone
Procedural error
– TA who wasn’t trained
rated students
– A writing collection was
not submitted according
to the assembly criteria
– TA failed to provide
student the correct
authorization for online
testing
Reporting Test Irregularities
 Each person is responsible for reporting ANY violation or suspected
violation
 Campus staff should notify their CTC or DTC if they witness an
irregularity or suspect one has occurred
 DTC will contact TEA to report ANY incidents as soon as DTC is made
aware of the situation
 Contact DTC if unsure about whether an irregularity has occurred
22
Campus
Training
Requirements
23
Campus Training Requirements
You are expected to:
Responsibilities
of TELPAS
Coordinators
Train staff on holistic rating training
requirements.
Ensure raters understand the
importance of being properly trained on
the holistic rating process
Ensure new and returning raters
understand which training to take and
for which grade clusters
24
Campus Training Requirements
You are expected to:
Responsibilities
of TELPAS
Coordinators
Train staff on how to access the online
training and calibration components.
Monitor that all staff complete their
training requirements and receive
supplemental training support if needed
Train all staff (as applicable) on
assessment procedures such as how to
assemble writing collections, how to
record students’ proficiency ratings, etc.
25
Coordinators ensure that:
– All ELL’s are identified and
assessed
Prepare
For
Training
– TELPAS Raters, SSP & Verifiers are
identified and properly trained
– Procedures are established to
collect & verify the years in US
for all ELL’s
– There are protocols at the
campus to ensure the validity
and reliability of the rating
process
– Student records are verified
– Materials are collected and filed
as appropriate
TELPAS Roles
Holistic Rater
27
Rater Credentials
RATER:
Teacher designated to be the official rater
of an ELL’s language proficiency
Each teacher selected to rate must:
– have student in class at the time of the spring assessment
– be knowledgeable about the student’s language ability
(instructional and informal)
– have a valid Texas teaching certificate
– be appropriately trained
– rate the student in all eligible domains
Districts must implement validity and reliability
checks during the testing window (2nd rater).
28
New or Returning Rater?
TEA requires that raters who have not rated
during the last three years complete the
BASIC training.
29
Holistic Rating Process
Spring TELPAS Administration Procedures Training
As a key part of this training, information from the TELPAS Rater Manual is reviewed
to prepare raters to proceed with online holistic rating training
New Raters
Returning Raters
Online Basic Training Course
Online Calibration
(Sets 1 and 2*)
Online Calibration
(Sets 1 and 2*)
If not calibrated: Supplemental Holistic Rating Training
*Set 2
required only
if not
successful on
Set 1
Final Online Calibration (Set 3)
30
Online Training Components
Online BASIC TRAINING Course
Required for new raters and raters who have not completed rater training
within the last three school years.
Gives raters practice rating students in each language domains.
Separated in two courses:
(approximately 4-5 hours)
– Basic Training for Grades Kinder-1st – covers the four language
domains
– Basic Training for Grades 2nd – 12th – covers listening, speaking,
and writing
Raters who have completed K–1 training but
not 2–12 training will be new raters if they
switch to 2–12 training, and vice versa.
31
Online Training Components
Online CALIBRATION Course
Required for ALL 2015 raters.
Each calibration set contains 10 students to rate and all holistically
assessed language domains are represented in each set.
(approximately 1 hour )
– Raters taking the grades Kinder-1st course – will rate listening,
speaking, reading and writing
– Raters taking the Grades 2nd – 12th course – will rate in listening,
speaking and writing
A rater needs to rate at least 70 percent of
the students correctly to demonstrate
sufficient calibration.
32
Training & Calibration Activities
When and where does it take place?
Depending on campus
arrangements, raters
may either complete
training and calibration
 during school hours
 after school or
 on weekends
– Raters may access the
training from any
computer that meets the
minimum system
requirements.
– Campuses must inform
staff of the expectations
during the Holistic
campus training.
33
Training & Calibration
Grade
Clusters
Grades K-1
Grade 2
Grades 3-5
Grades 6-8
Grades 9-12
 Raters must know their assigned
grade cluster to select the
appropriate online training.
 Raters should consult with their
campus testing coordinator if they
are unsure of their assigned cluster.
34
Grade Cluster Organization
Online Training Process
Basic Training Course
Kinder - 1
Calibration Clusters
Kinder - 1
Basic Training Course
2 – 12
Calibration Clusters
Grade 2
Grades 3 – 5
Grades 6 – 8
Grades 9 – 12
35
Raters with Students in Multiple
Grade Clusters
Including
students in
Grades 2-12
 Raters are required to train and calibrate in the cluster
in which they have the most ELLs.
 As a best practice, these raters should also review the
online basic training course practice activities in the
additional cluster(s).
Example: A grade 2–5 ESL teacher has most
of her ELLs in grade 3. She must complete
training and calibration in grades 3–5. As a
best practice, she should review the online
practice activities for grade 2 to check her
readiness to apply the rubrics appropriately.
36
Raters with Students in Multiple
Grade Clusters
Including
students in
Grades K-1
 These raters must complete training and calibration for K–
1 and at least one other cluster in 2–12.
 Raters with more than one additional cluster should train in
the cluster in which they have the most ELLs.
 As a best practice, these raters should also review online
basic training course practice activities in the additional
cluster(s).
Example: A grade 1–3 ESL teacher has most of
his ELLs in grade 2. He must complete training
and calibration in grades K–1 and 2. As a best
practice, he should review the online practice
activities for grades 3–5 to check his readiness
to apply the rubrics appropriately for his 3rd
grade students.
37
Preparing for Calibration Sets
Handout #2
New raters must complete
the online basic training
course BEFORE beginning
calibration.
All raters have the option to
review the online basic
training course (which
includes practice rating
activities) BEFORE beginning
calibration.
Basic Training must be
completed by
February 15, 2015!!!
38
Online Calibration Activities
• Raters complete only as many sets as it takes to calibrate
• Many raters will be able to calibrate on the first set of
activities and many others will be able to calibrate by the
end of the second set.
• With supplemental support, very few raters should have
difficulty calibrating by the end of the third and final set
IMPORTANT NOTE
Individuals are not authorized by TEA to serve as raters
unless they complete the state-required training and
calibration activities
39
Resources during calibration
Can raters use them?
Yes, raters should use their rating
rubrics (PLD’s) found on page 33-37
of the Rater Manual and refer to, as
needed, any other information
from the:
• online basic training course handouts
• holistic rating PowerPoints by TEA
• TELPAS Rater Manual
• Educator Guide to TELPAS
40
Other
Things to
Know
About
Calibration
 Raters affirm online that they will keep
the contents of the calibration sets
secure and confidential.
 Calibration activities are taken from a
bank and randomized. Trainees will rate
different sets of students.
 Raters can work at their own pace, go
back and review students, and change
ratings as they work.
41
Other
Things to
Know
About
Calibration
(cont.)
 Raters can exit and return later to
finish. They click a “submit” button
when they are finished with a set.
 After completing a calibration set,
raters immediately see results. Results
show both the rating assigned by the
rater as well as the correct rating.
 Raters see annotations explaining the
correct ratings. Raters should use the
annotations to go back and review any
incorrectly rated students.
42
Tools for TELPAS Coordinators
TELPAS Coordinators can:
 modify the last name of a user
 send a user password reset email
 modify the email address of a user
 retrieve an email address associated with a user’s
TrainingCenter account
 retrieve a username associated with a user’s
TrainingCenter account
 see account status (active/deactivated) for all users.
43
TELPAS
Additional
Roles
Supplemental Support Providers
Assembly & Verifiers
44
SSP Criteria
A Supplemental Support Provider must:



have been a 2013-2014 fully trained rater
in the applicable K-1 or 2-12 span
(last school year)
complete the SSP training in the Texas
Training Center (current school year) and
calibrate successfully on either set 1 or
set 2 in 2015 (current school year)
45
SSP
Training
Reviewing the
number of raters
unable to
calibrate by end
of calibration set
2 last year may
help anticipate
this year’s needs
The SSP training is delivered through a
Web-based presentation accessible from
the SSP resources section of the Texas
TrainingCenter
– Separate trainings for K–1, 2–5, and 6–12
– Training will be available beginning January 26
– Both new and returning supplemental support
providers need to complete this training
– Campuses must train at least one
supplemental support provider
46
Providing Supplemental Support
Handout #2
– CTC’s will notify the DTC who the SSP is by the designated
district deadline (February 22, 2015)
– Raters not successful after sets 1 and 2 must receive
supplemental training
– The rater will meet with a campus-appointed supplemental
support provider (SSP) (no later than February 26, 2015)
– After the rater has received supplemental training, raters will
be given an access code to proceed with the third and final
calibration set (no later than March 6)
47
Models for Providing Support
Individual or group sessions
– Raters attend a one-on-one or group session with a
supplemental support provider before completing their
third and final calibration set.
Brief meeting followed by additional consultation at rater’s request
– Rater meets briefly with the supplemental support
provider for instructions on resources to review.
– Rater schedules time with the support provider to get
clarification, if needed, before completing the third and
final calibration set.
Note: The models are described in detail in the recorded
Web-based training.
48
Assemble and Verify
Grades K-12 Student Writing Collections
– An online training course on writing collection assembly and
verification is available on the Texas TrainingCenter website
– Writing assigned on or after February 16, 2014 may be
considered for the writing collection (writing assigned before
February 16 is not eligible for inclusion)
– Writing samples may continue to be gathered until March 26th,
the date raters are required to turn in students’ rating to the
campus coordinator
– All writing should reflect a student’s current proficiency level
Assemble and Verify
Grades K-12 Writing Collections
• Type 1: Basic descriptive writing on a personal/familiar topic
• Type 2: Writing about a familiar process
• Type 3: Narrative writing about a past event
• Type 4: Personal narratives and reflective pieces
• Type 5: Expository and other extended writing on a topic from
language arts
What Not to Include in a Collection (p. 22)
Verifying the Writing Collection Components (p. 23)
50
Monitoring
Course
Completions
and
Performance
on Calibration
Activities
Training Certificates
Training certificates are generated for
raters who complete training courses
and calibration activities.
– A certificate of completion is issued when a rater
completes the online Basic Training or Assembly &
Verifying course.
– A certificate of successful calibration is issued when a
rater demonstrates sufficient calibration (70 percent or
better) on a calibration set.
52
Available Reports
https://texas.pearson.desire2learn.com/
For online training and calibration
• TELPAS Returning Rater Planning Roster
– Uses records starting from the 2008–2009 school years to generate a
list of previously trained raters (Basic Training)
• TELPAS Confidential Course Completion Roster
– Lists online basic training course completion and in-progress status
and performance by user for the current year
• TELPAS At-a-Glance Training & Calibration Report
– Provides a comprehensive list showing basic training course and
calibration completions for all registered users for the current year
(includes time spent in each calibration set)
• TELPAS Confidential Calibration Summary Report
– Provides calibration summary information by grade cluster
(K–1, 2, 3–5, 6–8, 9–12) and for grade clusters combined
53
TELPAS Returning Rater Planning Roster
54
TELPAS Confidential Course Completion Roster
55
TELPAS At-a-Glance Training & Calibration Report
56
TELPAS Confidential Calibration Summary Report
57
Important Notes About Reports
In the Texas TrainingCenter…
• TELPAS Coordinators are able to access reports to
monitor online training and calibration.
• The reports are updated nightly.
• In order for users to appear in the correct reports, their
location (region, district, campus) must be up to date in
the My Info section.
• Users who have registered in the Training Center and
have started but not completed a calibration set are not
included in the summary reports. Only completed sets
are presented in the reports.
• However, users who have started a training course will
appear in the course roster as being “in progress.”
58
What’s next?
For TELPAS Coordinators
59
Holistic Process
Handout #3
Where do I start?
 Plan your trainings
 With your principal, determine how
your campus will conduct basic
training and calibration activities
 Verify trainings have been
completed (obtain certificates from personnel)
 Download rating rosters,
coversheets, etc from ARE website
 Be sure to collect and have the
correct Years in US schools of all
students (ELLs)
TELPAS Student Rating Roster
61
Schedule
TELPAS
Student
Tutorials
As in the past, tutorials are
separate from tests
•
•
•
Recommended for all students
taking TELPAS
Several different test item formats
Since no sample items with test,
practice with item formats and
online interface is useful
Tutorials available at
http://www.TexasAssessment.com/TELPAS-tutorials
62
Add Dates to Your Calendar
63
Next
Training
TELPAS– Online Testing Training
Friday, February 20, 2015
1:00 – 4:00
CAB Cafeteria
*Bring YEARS in US!!!
e-Schools extraction of
TELPAS registration file
64
Questions and
Answers…