Florida Interim Assessment
Item Bank and Test Platform
(FL IBTP)
Local Assessment Part 2
February 17 and 18, 2014 – Lake Mary, FL
February 20 and 21, 2014 – Tallahassee, FL
Learning Goals
At the end of this workshop, you will understand how to:
• Part One
• Create assessment blueprints
• Part Two
• Build test maps
• Part Three
• Create and schedule assessment forms
• Administer forms
• Enter scores for open response items
• View assessment data and reports
Training Overview
Today, participants will work in groups to create portions of
a sample blueprint, test map, and assessment.
Participants will learn the necessary skills and processes to
create blueprints, test maps, and assessments.
They will leave with the knowledge, skills, and resources to
train district personnel to create assessments.
Agenda
•
•
•
•
Introduction to materials
Training overview
Demonstration of the assessment development process
Guided practice
– Creating a blueprint
– Creating a test map and searching for items
– Creating, scheduling, administering, and scoring an
assessment
List of Materials
• Participants’ Workbook
– Tab 1: Blueprints
– Tab 2: Test maps and searching for items
– Tab 3: Assessment form creation
– Tab 4: Blueprint template, sample blueprint, test map
template, and sample test map
• Job Aids
Materials available for download at:
http://flassessments.com/html-redirects/IBTP/index.htm
Step 1: Create a Blueprint
Grade K math fall interim assessment example:
Step 2: Create a Test Map
The following is an example of the first columns of the
grade K mathematics test map:
Create a Test Map
The following is an example of the last columns of the
grade K mathematics test map:
Step 3: Create an Assessment
Using the test map, create the assessment in the system.
Then schedule, administer, and score the assessment.
Take an Assessment
Now take a three-question test to see how students will see
the test in the system.
Break into Groups
• Groups of five or six
PART ONE:
CREATING BLUEPRINTS
Assessment Blueprints
Use a blueprint to see the content and structure of an
assessment. The blueprint is based on the standards or
benchmarks listed in the course description.
Blueprints include the following:
• Total number of items
• Number of items by standard or benchmark
• Percentage of items at each cognitive complexity
level (Depth of Knowledge, or DOK level)
• Percentage of items at each difficulty level (if item
difficulty statistics are available in the FL IBTP)
Blueprint Considerations
When building an assessment blueprint, consider the
following:
•
•
•
•
•
•
Purpose of the assessment
Overall assessment design
Overall time allocated to assessment administration
Time considerations for item types
DOK levels of individual standards and benchmarks
DOK levels of items
Step 1: Category Breakdown
Before drafting the blueprint, determine the purpose and
type of assessment. In this example, create a blueprint for
a grade K mathematics interim assessment that will be
administered in the fall to gauge how students are
progressing towards mastery of the course standards.
Based on the instructional time spent on each strand of
standards or benchmarks (reporting category), determine
the percentage of items by reporting category.
Grade K math fall interim assessment example:
• Counting and Cardinality = 70–80%
• Geometry = 20–30%
Step 2: Define the Item Range
Next, determine the number and types of items to be
included in the assessment. Consider the typical time
allocation for each item type when designing the
assessment.
Item range
• Approximate the number of items
• Use increments of 5 when determining an item range
(i.e., 5–10 items)
Approximate Time Allocations
Item Type
Minutes Allocated
Point Value
Selected Response (SR)
1–3 minutes
1
Gridded Response (GR)
1–3 minutes
1
Short Response (SHR)
3 minutes
1
Constructed Response (CR) 5 minutes
2
Extended Response (ER)
10–15 minutes
4
Essay Response (ESR)
20–30 minutes
6
Performance Task (PT)
Varies
1–10 (depending on
complexity and
content area)
Step 3: Percentage of Items
by DOK Level
Determine the percentage of items by DOK level.
• DOK level: 1, 2, 3, 4
• Should total 100%
• Give minimum range of 5–10 percentage points (i.e.,
20–30% DOK level 1)
Grade K math fall interim assessment example:
• DOK level 1 = 50–60%
• DOK level 2 = 40–50%
• DOK level 3-4 = 10–20%
Step 4: Number of Items
by DOK Level
Multiply the cognitive complexity level percentage by the
total number of items to get the item range for each DOK
level. Start with the low complexity items.
Grade K math fall interim assessment example:
50–60% DOK level 1
x 10 items =
5–6 DOK level 1 items
Number of Items by DOK Level
Multiply the cognitive complexity level percentage by the
total number of items to get the item range for the DOK
level 2 items.
Grade K math fall interim assessment example:
40–50% DOK level 2
x 10 items =
4–5 DOK level 2 items
Number of Items by DOK Level
Multiply the cognitive complexity level percentage by the
total number of items to get the item range for the DOK
level 3 or 4 items.
Grade K math fall interim assessment example:
10–20% DOK level 3–4
x 10 items =
1–2 DOK level 3–4 items
Number of Items
by Reporting Category
Multiply the reporting category percentage by the total
number of items to get the item range for each reporting
category.
Grade K math fall interim assessment example:
70–80% Counting and Cardinality
x 10 items =
7–8 Counting and Cardinality items
20–30% Geometry
x 10 items =
2–3 Geometry items
Completed Example Blueprint
Grade K math fall interim assessment example:
Step 5: Guided Practice
Complete the assessment blueprint using a blueprint
template such as the one shown here:
District Presentation
Hands-On Practice:
Create a Blueprint
Using the course information from www.cpalms.org, work in
groups to create an assessment blueprint.
Create blueprints for one of the following courses:
• Grade 8 American History (Social Studies)
• Grade 7 Life Science (Science)
Create at least four lines of the blueprint.
Hands-On Practice:
Create a Blueprint
1. Review the course description on www.cpalms.org.
2. Add the standard or benchmark identifiers to the blueprint
template.
3. Based on the instructional time spent on each strand of
standards or benchmarks (reporting category), determine the
percentage of items by reporting category (use district
resources, such as a curriculum map, pacing guide, or scope
and sequence).
4. Determine the number and types of items to be included in the
assessment.
5. Determine the percentage of items by DOK level.
6. Multiply the cognitive complexity level percentage by the total
number of items to get the item range for each DOK level.
Share Your Learning
• Share what you’ve learned about creating blueprints.
• Record notes about the process on your T-chart.
PART TWO:
BUILDING TEST MAPS
What is a test map?
A test map provides item-level details so that the
assessment developer is able to search for and select
specific items to build an assessment.
While the test blueprint is a high-level overview of the
contents of the assessment, the test map is a list of specific
items in the sequence in which they should appear on the
assessment.
Test Planning
• Discuss naming conventions for assessments.
For example:
Subject
Season
Year
Grade Form
Math Spring 2014 Gr K Form 1
Math Spring 2014 Gr K Form 2
• Discuss workflow.
Assign responsibility for blueprint and test map
creation, test creation, scheduling the test, assigning
the test, and monitoring the collection of test results.
32
Test Map Template
The following is an example of a test map template:
Step 1: Complete the First
Columns
Using the information from the blueprint, complete the first
columns on the test map. These columns are completed without
going into the system.
•
•
•
•
•
•
•
•
•
Test Name
Subject
Grade Level or Grade Band
Course Code
Course Title
Test ID (to be completed later)
Standards Source Document
Standard or Benchmark Code
Sequence
Example
The following is an example of the first columns of the
grade K mathematics test map.
Step 2: Search for Items
Now go into the system to search for items. In the Florida Item
Bank and Test Platform, search for the items to be included in
the assessment by subject, grade, or standard. Items will then
be listed on the test map.
Complete the Next Columns
Complete these columns by going into the system:
•
•
•
•
UIN (Name)
Test ID
Manipulatives (math tools)
Item Type
Step 3: Locate the Item UIN
Locate the Unique Identification Number (UIN) of the item
under Properties on the right side of the Item Central page.
Step 4: Record the UIN
Record the item UIN in the appropriate column on the test
map.
Step 5: Test ID Column
The Test ID column cannot be completed until the system
generates the assessment. The system then assigns a test ID
number and displays it on the Test Detail page. Be sure to fill
in the Test ID column on the test map after the assessment is
created.
Math Fall 2014 Gr 6 Form 1 [2251]
Step 6: Manipulatives
Indicate whether or not tools will be used to answer this
item.
Step 7: Item Type
Each item lists the item type. Record the abbreviation for
the item type on the test map.
Step 8: Complete the Last Columns
Complete the last six columns of the test map.
• Key (correct response)
• DOK Level (Webb)
• Item Difficulty
• Maximum Points
• Associated Passage UIN
• Passage Type
Key
For Gridded Response and Selected Response items,
record the correct response in the Key column on the test
map.
For open-ended items, enter “NA” for not applicable.
DOK Level
The DOK level, or context complexity level, is associated with
the standard or benchmark listed in the course description for
the CCSS in mathematics and English language arts, and for
the NGSSS benchmarks in science. Find the content
complexity/depth of knowledge rating in CPALMS.
Determine the DOK level of the item and record it in the DOK
Level column.
Item Difficulty
Item difficulty is a statistical measure of the proportion of
students who have correctly answered an item.
Item difficulty statistics may not be available initially in the
FL IBTP system.
Maximum Points
Determine the maximum points for the item and record that
number in the Max Points column.
Associated Passage UIN
Look for the Passage ID on the right side of each passage
and record it on the test map, as appropriate.
Passage Type
Look for the passage type in the same list and record it on
the test map.
Repeat the Process
Repeat the process for all the items on the assessment
until the test map is complete.
Guided Practice: Search for Items
Use this URL: https://flademo.schoolnet.com
To start an item search, roll the cursor over Assessment
Admin and select Find an Item, Passage or Rubric.
District Presentation
Hands-On Practice:
Create a Test Map
Using the blueprint you created earlier, work in groups to
create a test map.
Create test maps for one of the following courses:
• Grade 8 American History (Social Studies)
• Grade 7 Life Science (Science)
Create at least four lines of the test map.
Hands-On Practice:
Create a Test Map
1.
2.
3.
4.
5.
6.
7.
8.
9.
Sign in to the training site.
Roll the cursor over Assessment Admin and click Find an Item,
Passage or Rubric.
Use the subject, grade, and standards filters to refine the search.
Use Advanced Search to search for multiple items by standard
group.
When the search is narrowed sufficiently, click View Results to view
the items that match the selected criteria.
Review the items that resulted from the search.
Record the appropriate information found under Properties on the
right side of the screen.
To change the search criteria, click Open Filters and change the
selections.
To start a new search, roll the cursor over Assessment Admin and
select Find an Item, Passage or Rubric.
Share Your Learning
• Share what you’ve learned about creating test maps.
• Record notes about the process on your T-chart.
PART THREE:
ASSESSMENTS – CREATING,
ADMINISTERING, SCORING,
AND REPORTING
PART THREE (A):
CREATING AND SCHEDULING A
FORM/ASSESSMENT
Benchmark Assessments
In the Florida IBTP system:
• In Florida, the term “benchmark” refers to the NGSSS
•
•
learning targets, but in the FL IBTP system,
“benchmark assessments” is a generic term that
describes all interim assessments not loaded as
standardized tests.
Tests are created at the district or school level for the
purpose of institution-wide data collection.
Results appear in the School & District Data and
Classrooms modules.
Using Assessment Admin
• Use Assessment Admin to build tests and monitor the
collection of results.
• View data and report options in the Classrooms and
School & District Data modules.
Ways to Create a Test
Express test
Manual test
Create a Test Manually
Use the test map to search for and select test items.
Building a Test
Math Fall 2014 Gr 6 Form 1 [2251]
Hands-On Practice:
Create an Assessment
Create a test using the items from the sample test maps
using grade 8 American History and grade 7 Life Science.
Use the website: https://flademo.schoolnet.com
Test Stages
Private
Draft
Public Draft
Ready for
Scheduling
Scheduled
In Progress
Complete
Scheduling the Test
Math Fall 2014 Gr 6 Form 1 [2251]
Test Schedule Dates
Math Fall 2014 Gr 6 Form 1 [2251]
Online Test Options
Assigning a Test
Hands-On Practice:
Schedule an Assessment
Preview the online test, then schedule the test.
PART THREE (B):
ADMINISTERING A
FORM/ASSESSMENT
Test Administration
The assessments created in the FL IBTP system will be
administered to students online.
Online assessments are administered in two ways:
• Secure Tester
• Browser-based
Proctor Dashboard: Monitor an
Online Test
Math Fall 2014 Gr 6 Form 1 [2251]
• Graphic representation of the
progress of the class
• Links to individual student
scores, answer sheets, and
PDF test booklets
The Student Online Test Experience
• Make sure computers are ready for testing.
• Students sign in and enter online passcodes.
The Student Online Test Experience
PART THREE (C):
ENTERING SCORES FOR OPEN
RESPONSE ITEMS
Entering Scores
Open-ended items must be scored manually. After scoring
each open response item, click Save at the bottom of the
Score page.
PART THREE (D):
REPORTING
Finding the Results
View assessment results using the data and reports found in
the School & District Data and the Classrooms modules.
Benchmark Tests Dashboard
Additional Reports
Pre-formatted Reports
Save and Access Your Reports
So data is always current, save the
report parameters rather than the
results.
Hands-On Practice:
View Data and Report Options
View available assessment data and reports in the School
& District Data and the Classrooms modules.
Using IBTP in Your District
In district groups, discuss what resources are needed to
plan and deliver professional development in your
district.
Evaluation
Your feedback is valuable. We invite you to respond to a
short set of questions.
Thank you very much for your time.
Thank You
NOTE: These materials were developed under a grant from the U.S. Department of Education.
However, the contents do not necessarily represent the policy of the U.S. Department of Education,
and you should not assume endorsement by the Federal Government.
Copyright © 2014 Florida Department of Education. All rights reserved.