Dynamic Learning Map Update
Lansing, Michigan
April 17, 2013
afternoon session
Daryl Mellard
Sue Bechard
Michelle Shipman
Judith Gross
The present publication was developed under grant 84.373X100001 from the
U.S. Department of Education, Office of Special Education Programs. The views
expressed herein are solely those of the author(s), and no official endorsement
by the U.S. Department should be inferred.
DLM 2012/2013 update
Depth of Knowledge Taxonomy
• Extensions
Learning Maps
• Claims, conceptual areas
• Foundation nodes
First Contact Survey
Instructionally Relevant Testlets
• Testlet development
• Example items
• Technology Enhanced Templates
DLM Future Activities
DEPTH OF KNOWLEDGE
DLM Depth of Knowledge Taxonomy
Process
Pre-intentional
Attend
Respond
Replicate
Remember
Understand
Apply
Analyze
Evaluate
Create
Definitions
Behavior reflects a general state but does not reflect intentional behavior. Intent is
inferred by others (e.g., teachers, parents) through facial expressions, movements, or
sounds.
Orients to objects, people or activity. Indicates selective attention to stimuli in the
academic learning environment.
Intentional response using any mode of expression. Indicates joint attention to materials
and activities in the academic learning environment.
Perform rote task in familiar or practiced context.
Retrieve relevant knowledge from long-term memory in a novel context.
Construct meaning from instructional messages, including oral, written, and graphic
communication.
Carry out or use a procedure in a given situation.
Break material into its constituent parts and determine how the parts relate to one
another and to an overall structure or purpose.
Make judgments based on criteria and standards.
Put elements together to form a coherent or functional whole; reorganize elements into a
new pattern or structure.
What’s new?
Process
Definitions
Pre-intentional
Behavior reflects a general state but does not reflect intentional
behavior. Intent is inferred by others (e.g., teachers, parents)
through facial expressions, movements, or sounds.
Attend
Orients to objects, people or activity. Indicates selective attention
to stimuli in the academic learning environment.
Respond
Intentional response using any mode of expression. Indicates
joint attention to materials and activities in the academic learning
environment.
Replicate
Perform rote task in familiar or practiced context.
Remember
Retrieve relevant knowledge from long-term memory in a novel
context.
LEARNING MAPS
Making Connections
Claims
Conceptual
Areas
EE
Map
What are claims?
• Communicate the goals we have for
student learning
• Provide a framework for organizing
teaching and learning both within and
across grades
• Represent the knowledge, skills, and
abilities that support understanding
and proficiency
Making Connections
Claims
Conceptual
Areas
EE
Map
What are conceptual areas?
• Areas that are organized around
common cognitive processes.
• EEs (tied to nodes) are embedded
within conceptual areas.
• May be organized/structured
differently in mathematics and ELA.
Claims and Conceptual Areas: ELA
Claim
Conceptual
Area
EE Count
Claim 1
Students can comprehend text in
increasingly complex ways
1. Determining
Critical
Elements of
Text
66
2. Constructing
Understandings of
Text
77
3. Integrating
Ideas and
Information
from Text
91
Claim 3
Students can communicate
for a range of purposes
and audiences
1. Using
Language to
Communicate
with Others
79
2. Clarifying and
Contributing in
Discussion
72
Claims and Conceptual Areas: ELA
Claim
Conceptual
Area
EE Count
Claim 2
Students can produce writing
for a range of purposes and
audiences
1. Using Writing to 2. Integrating
Communicate
Ideas and
Information in
Writing
47
35
Claim 4
Students can engage in
research/inquiry to investigate
topics and present information
1. Using Sources
and Information
18
2. Collaborating
and Presenting
Ideas
22
Claims and Conceptual Areas:
Mathematics
Claim
Conceptual
Area
EE Count
Claim 1
Number Sense: Students demonstrate
increasingly complex understanding
of number sense.
1. Understand 2. Compare,
number
compose, and
structures
decompose
(counting,
numbers and
place value,
sets
fraction)
21
19
Claim 2
Algebraic and functional
reasoning: Students solve
increasingly complex
mathematical problems,
making productive use of
algebra and functions.
3. Calculate
1. Use operations
accurately and and models to
efficiently using solve problems
simple
arithmetic
19
25
2. Understand
patterns and
functional thinking
12
Claims and Conceptual Areas:
Mathematics
Claim
Conceptual
Area
EE Count
Claim 3
Geometry: Students demonstrate
increasingly complex spatial
reasoning and understanding of
geometric principles.
Claim 4
Measurement Data and Analysis:
Students demonstrate Increasingly
complex understanding of
measurement, data, and analytic
procedures.
1. Understand
and use
geometric
properties of
two- and threedimensional
shapes
1. Understand
2. Represent and
and use
interpret data
measurement
displays
principles and
units of measure
23
2. Solve
problems
involving area,
perimeter, and
volume
15
23
16
Foundation Nodes on the Dynamic
Learning Map
• Currently have 150 foundation nodes
on the map.
• They cover:
– Pre-intentional
– Attend
– Respond
• They are not linked to one academic
area but are foundational to all
FIRST CONTACT SURVEY
STUDENT CHARACTERISTICS
First Contact Survey
•
•
•
•
•
•
•
•
•
Teacher demographics
Student demographics
Classroom setting
Sensory capabilities (accessibility needs)
Motor capabilities
Computer access
Communication
Academic skills
Attention
First Contact Sample: Disability Type
45.0
41.0
40.0
35.0
30.0
26.9
Percent
24.5
25.0
20.0
15.0
10.0
5.0
.3
1.0
.3
.3
0.0
Disability Type
1.0
2.8
1.4
.3
Language Use
Pre – symbolic
Language Users
Emerging
Symbolic
Users
Symbolic Language
Users
Kearns, J., Kleinert, H., Harrison, B., Sheppard-Jones, K., Hall, M., Jones, M.
(2010). What does ‘ college and career ready’ mean for students with
significant cognitive disabilities? Lexington: University of Kentucky.
First Contact Sample: Reading
0
5
10
15
20
Does not read any words when presented
in print or Braille (not including
environmental signs or logos)
35
17
Elem
13
Middle
High
Above first grade level to second grade
level
18
Above second grade level to third grade
level
18
Above third grade level
30
19
Reads only a few words or up to pre-primer
level
Primer to first grade level
25
15
10% of 1%
Judith Gross & Michelle Shipman
INSTRUCTIONALLY RELEVANT
TESTLETS
Testlet Development
1
Essential
Element
2
Dynamic
Learning
Map
3
Essential
Element
Concept Map
4
Testlets
at 5 levels
5
Iterative
Review
1
Essential
Element
Step 1: Aligning the
Essential Element
Common Core
State Standard
Essential
Element
2
Dynamic
Learning
Map
Step 2: Mapping the
Essential Element
Node
Essential
Element
Analyze
connections
Match
Node
Node
Node
Fractions
M.EE.3.NF.1-3 Differentiate a fractional part from a whole
Feelings of Characters
ELA.EE.RL.3.3 Identify the feelings of the characters in a story
Character Responds to a Challenge
EE.RL.6.3 Can identify how a character responds to a challenge
in the story
3
Essential
Element
Concept Map
•
•
•
•
•
•
Step 3: Completing the
Essential Element Concept Map
Essential Questions
Vocabulary
Nodes
Questions to Ask
Misconceptions/Errors
Accessibility/Barriers
Essential Element Concept Map
4
Testlets
at 5 levels
Step 4: Creating Instructionally
Relevant Testlets
4
Testlets
at 5 levels
Step 4: Creating an Instructionally
Relevant Testlet Type
Connect the map…
…to the items developed.
Initial
Precursor
Behavior
Testlet
a
Distal
Precursor
Behavior
Testlet
b
Proximal
Precursor
Behavior
Testlet
c
Target
Behavior
Testlet
d
Behavior
Testlet
e
Successors
5
Iterative
Review
Step 5: Iterative
Review Process
Internal:
• Item development
teams
• Editing content and
graphics
• Sensitivity and bias
• Accessibility
External:
• State visits
Fractions
From Foundation to 4th grade
Essential Element Concept Map
• M.EE.3.NF.1-3 – Differentiate a
fractional part from a whole
–
–
–
–
–
Math
Essential Element
Grade 3
Numbers and Fractions
Standards 1-3
• Covers 5 levels from Foundational Skills
(Initial precursors) through 4th Grade
(Successor skills)
M.EE.3.NF.1-3
Differentiate a fractional part
from a whole
Initial Precursors
F-62 Attend
F-13 Recognize some
Distal Precursors
F-38 Explain set
F-69 Recognize wholeness
F-84 Recognize separateness
Proximal Precursors
F-59 Explain sub-set
F-72 Partition sets
F-2392 Partition length
M-248 Partition shapes
Targets
M-561 Explain part-whole relationship
M-2411 Explain fraction
Successors
(M.EE.4.NF.3)
M-2537 (area), M-2538 (length), M-2539 (sets) - Identify whole
M-2350 (area), M-2349 (length), M-2348 (sets) - Identify half
Another related fraction EE
M.EE.4.NF.1-2
Identify ½ and ¼
Initial Precursor Testlet
Henry and Mudge are Happy
– ELA.EE.RL.3.3 Identify the feelings of
characters in a story.
– 3rd grade
– Initial precursor level testlet
– Task 1
• General task and blind/low vision task
• F-18 Seeks attention of others
• Precursor of ELA-766
– Task 2
• General task
• F-766 Can identify feeling states in self
TEACHER DIRECTIONS
You are going to read the familiar story, Henry and
Mudge, with the student.
You will record the student’s response when you:
• show the student a favorite object OR
• show the toy you have used to represent Mudge
Put the object out of the student’s sight or reach, but
within your reach.
After you finish reading, you will ask the student a
question and record the response.
Henry and Mudge are Happy
DLM
Henry was a boy.
Henry wanted a dog.
TEACHER DIRECTIONS (general)
Stop reading.
Present the object just out of reach but within sight of the student.
Move the item in an effort to attract the students attention, but do
not speak or otherwise prompt.
Record the student’s response (check only one).
 Communicates desire to have the object by:
Reaching for object
Gazing/looking at object
Vocalizing
Using other gesture or indicator
 Smiles, laughs or shows excitement
 Whines, cries, or appears frustrated
 No response
TEACHER DIRECTIONS (blind/low vision)
Stop reading.
Allow the student to touch, smell, or otherwise interact with
the object for a moment.
Remove the item and do not speak or otherwise prompt.
Record the student’s response (check only one).
 Communicates desire to have the object by:
Reaching for objects or searching with hands
Turning to you and reaching
Vocalizing
Using other gesture or indicator
 Smiles, laughs or shows excitement
 Whines, cries, or appears frustrated
 No response
Mom and dad wanted to make Henry happy.
They got him a dog. Henry named him Mudge.
Henry loved Mudge.
Henry was a happy boy.
Mudge was a happy dog.
TEACHER DIRECTIONS
Ask the student, “Henry and Mudge are happy. Are you
happy?”
Record the student’s response (check only one).
 Clear yes or no response by:
Vocalization
Head nod/shake
Other gesture
 Indicated happiness (laughter, smile)
 Indicated displeasure (grimace, frown, grunt)
 No response
Initial Precursor Testlet
Aunt Polly
– ELA.EE.RL.6.3 Can identify how a character
responds
to a challenge in the story
– 6th grade
– Initial precursor level testlet
– Familiar text
• Common Core State Standards English Language
Arts Text Exemplars
• The Adventures of Tom Sawyer
– Task 1 and Task 2
• F-11 Can demonstrate understanding of action
words.
TEACHER DIRECTIONS
You are going to read the familiar story, Aunt Polly, with the student.
You will record whether the student indicates recognition or understanding of the repeated line
during and after the story.
Aunt Polly makes Tom do different things. Each of them is a common action. As you read each
page that begins, “Aunt Polly makes Tom…”:
• model the gesture,
• point to the symbol, or
• model using the voice output device that matches the action.
As you read each page that begins, “Tom hates…” pause at least 10 seconds to give the student
a chance to:
• initiate the gesture,
• point to the symbol, or
• access the single message voice output device.
If the student does not initiate without support after 10 seconds, model the use of the gesture,
symbol or switch.
If the student does not respond after the model, use a prompt.
Aunt Polly
DLM
This is Tom's Aunt Polly.
Aunt Polly makes Tom wash.
Tom hates to wash.
TEACHER DIRECTIONS
Describe how the student responded after you read the
word, “wash” (check only one).
 Independently initiated use of:
Gesture
Symbol
Voice output device
 After presented with a model of the desired behavior, the student
used:
Gesture
Symbol
Voice output device
 After prompting, the student used:
Gesture
Symbol
Voice output device
 Smiled, laughed, vocalized or otherwise indicated
enjoyment
 Protested, cried, or otherwise indicated displeasure
 No Response
Aunt Polly makes Tom comb his hair.
Tom hates to comb his hair.
Aunt Polly makes Tom eat at the table.
Tom hates to eat at the table.
Aunt Polly makes Tom go to sleep.
Tom hates to go to sleep.
TEACHER DIRECTIONS
Describe how the student responded after you read the
word, “sleep” (check only one).

Independently initiated use of:
Gesture
Symbol
Voice output device

After presented with a model of the desired behavior, the student
used:
Gesture
Symbol
Voice output device
 After prompting, the student used:
Gesture
Symbol
Voice output device
 Smiled, laughed, vocalized or otherwise indicated enjoyment
 Protested, cried, or otherwise indicated displeasure

No Response
Tom hates to do lots of things.
Aunt Polly makes Tom do them anyway.
ITEM TEMPLATES FOR
TECHNOLOGY ENHANCED ITEMS
Boardmaker Studio Templates
• Accessibility features
– One and two-switch scanning
– Click to place
• Extensively researched
– Informs design of DLM technology
enhanced item templates
DLM Editor Templates
• Accessibility features
– One and two-switch scanning
– Click to place
• Delivery
– HTML packages will upload into Content
Builder
• Crosswalk
– Studio items will transition into Editor
templates
Sorting
Sorting
Hot Text
Ordering
Labeling
Matching
DLM FUTURE ACTIVITIES
DLM Test Development Research
Activities: 2013/2014
Study / Test
Development
Phase
Cognitive Labs/
Sample Items
Internal Data
Test #1
Internal Data
Test #2
Dates
Purposes / Research Questions
May-June To evaluate:
2013
 Technology enhanced item template
design Student-technology interface
 Item administration processes
 Response processes
Mid-May
2013
Sept
2013
(25 specific research questions have been
drafted.)
Simulation – integration of data systems in
KITE
Full test of KITE functions for the pilot in
Oct
Next Steps
Identify items
Identify and recruit
sample
Finalize data collection
tools
Amend IRB
Logistics
Activities: 2013/2014
Study / Test
Development
Pilot
Test/fixed
form
Sample of
Testlets
Field Test
/adaptive
Rolling Field
Test
Dates
Purposes / Research Questions
October · To evaluate student-technology
2013
interface and testlet
(moved
administration processes
from
· To obtain initial psychometric
July
information on testlet
2013)
functioning
· To evaluate point of entry
identification strategies
· Post administration survey
Winter
2013May
2014
· Statistical analysis to facilitate
adaptive function
· Increase and evaluate additional
accommodations and access
strategies
· Include some students not
eligible for AAS and compare
(cont.)
Next Steps
(same logistics as presented
previously)
Recruit 1800 students in 3 grade
levels:
· 600 students in 3-4,
· 600 students in 7-8
· 600 students in 10-11
Create items in 2 conceptual areas:
· 1 ELA, 1 math:
· 3 testlets each in ELA and Math
for each grade band = 18 testlets
Develop specifics on field test
logistics
THANK YOU!
For more information, please contact:
dlm@ku.edu
or
Go to: www.dynamiclearningmaps.org
The present publication was developed under grant 84.373X100001 from the
U.S. Department of Education, Office of Special Education Programs. The views
expressed herein are solely those of the author(s), and no official endorsement
by the U.S. Department should be inferred.