AIM (Analyzing Instructional Materials)
Evaluation Process and Tools for Selecting
and Implementing Instructional Materials
January 10 and 11, 2007
Synopsys, Partnership for Student
Success in Science
AIM is a collaboration between the BSCS Center for Professional Development and WestEd, building on the
original work of the K-12 Alliance of WestEd.
Goals
• Understand and apply AIM (Analyzing
Instructional Materials) as a process to gather and
analyze evidence to inform decisions regarding
selection and/or implementation of instructional
materials.
• Understand and use AIM as a professional
development strategy focused on instructional
materials and student learning.
• Explore how science concepts are developed and
supported in standards-based instructional
materials.
AIM includes . . .
Pre-Screen
Paper Screen
Implementation
Identify Criteria
Identify Criteria
Gather Evidence
Gather Evidence
Analyze Evidence
& Apply Rubric
Analyze Evidence
& Apply Rubric
Score Components
Score Components
Summarize Results
Summarize Results
Select
Scaling-Up for Full Implementation
The AIM Process
Paper Screen
Implementation
Identify Criteria
Identify Criteria
Science
Content
Work
Students Do
Assessment
Work
Teachers Do
Student
Understanding
Teacher
Implementation
Gather
Evidence
Gather
Evidence
Gather
Evidence
Gather
Evidence
Gather
Evidence
Gather
Evidence
Analyze Evidence
and Apply Rubric
Analyze Evidence
and Apply Rubric
Analyze Evidence
and Apply Rubric
Analyze Evidence
and Apply Rubric
Analyze Evidence
and Apply Rubric
Analyze Evidence
and Apply Rubric
Score
Components
Score
Components
Score
Components
Score
Components
Score
Components
Score
Components
Summarize
Results
Summarize
Results
Merge Paper Screen and Implementation Results
Select Instructional Materials
The AIM Process:
Combining the Paper Screen and Implementation Scores
Score Sheet for AIM Process: Paper Screen
Criteria/Component
Score
Weight
Weighted
Total
Score Sheet for AIM Process: Implementation
Percent
Criteria/Component
CONTENT
STUDENT UNDERSTANDING
Standards Alignment
Pre-Post Assessment of Unit Concept(s)
Accuracy
Investigation
Concept Development (HPL 2)
Activity
Sequencing (HPL 2)
Reading
Context (HPL 2)
Assessment
TOTAL Content Criterion
X 0.40
=
Total Student Understanding
WORK STUDENTS DO
TEACHER IMPLEMENTATION
Engaging Prior Knowledge (HPL 1)
Content Background
Metacognition (HPL 3)
Teaching Strategies
Abilities Necessary To Do Scientific Inquiry
Teaching Strategies for Inquiry
Understandings About Scientific Inquiry
Assessment Strategies
Score
Weight
Weighted
Total
X 0.60
=
X 0.40
=
Percent
Accessibility
TOTAL Work Students Do Criterion
X 0.20
=
TOTAL Teacher Implementation
ASSESSMENT
GRAND TOTAL (Pilot)
Quality
T=
T/23 X
100 =
Multiple Measures
Use of Assessments
Accessibility
TOTAL Assessment Criterion
X 0.20
=
THE WORK TEACHERS DO
(0.6) (pilot score) + (0.4) (paper screen score) = Total Score
Instructional Model
Effective Teaching Strategies
Teaching Strategies for Inquiry
Support for the Work Teachers Do
TOTAL Work Teachers Do Criterion
GRAND TOTAL (Paper Screen)
X 0.20
=
T=
T/23 X
100 =
Score Sheet for AIM Process: Paper Screen
Criteria / Component
Score
Weight
Weighted
Total
Percent
Content
Standards Alignment
Accuracy
Concept Development (HPL 2)
Sequencing (HPL 2)
Context (HPL 2)
TOTAL Content Criterion
X
.40
=
X
.20
=
X
.20
=
X
.20
=
Work Students Do
Engaging Prior Knowledge (HPL1)
Metacognition (HPL 3)
Abilities to Do Scientific Inquiry
Understandings about Scientific Inquiry
Accessibility
TOTAL Work Students Do Criterion
Assessment
Quality
Multiple Measures
Use of Assessments
Accessibility
TOTAL Assessment Criterion
The Work Teachers Do
Instructional Model
Effective Teaching Strategies
Teaching Strategies for Inquiry
Support for the Work Teachers Do
TOTAL Work Teachers Do Criterion
Grand Total
T=
T / 23 x 100 =
AIM Summary Sheet: Strengths and Limitation
Criteria and Components
Content
Standards Alignment
Accuracy
Concept Development (HPL 2)
Sequencing (HPL 2)
Context (HPL 2)
Work
Students Do
Engaging Prior Knowledge (HPL 1)
Metacognition (HPL 3)
Abilities Necessary To Do Scientific Inquiry
Understandings About Scientific Inquiry
The Work
Teachers Do
Assessment
Accessibility
Quality
Multiple Measures
Use of Assessments
Accessibility
Instructional Model
Effective Teaching Strategies
Teaching Strategies for Inquiry
Support for the Work Teachers Do
Summary of Strengths
Summary of Limitations
In How People Learn (National Research Council, 2000), the authors summarize three key ideas about learning based on an
exhaustive study of the research (pp.14-19). These three findings about student learning have parallel implications for classroom
instruction (pp. 19-21), which then suggest a translation of those implications into curriculum materials. As the authors’ state,
these three findings imply the following for students and teachers:
Key
Findings
Key Findings for Students
Key Findings for
Teachers
First
Students come to the classroom with
preconceptions about how the world works. If
their initial knowledge is not engaged, they
may fail to grasp the new concepts and
information that are taught, or they may learn
them for purposes of a test but revert to their
preconceptions outside the classroom.
Recognize
preconceptions and
adjust instruction
Second
Third
To develop competence in an area of a
science discipline, students must (a) have a
deep foundation of usable knowledge, (b)
understand facts and ideas in the context of
a conceptual framework, and (c) be able to
organize that knowledge in ways that
facilitate retrieval and application.
Students must be taught explicitly to take
control of their own learning by defining
goals and monitoring their progress in
achieving them.
Understand the
content and
conceptual
framework for a
discipline
Provide examples
for context
Provide class time
for goal setting and
analysis
Teach
metacognitive skills
How can instructional materials be designed in
response to these findings?
Elements in Instructional Materials
That Address the Key Findings from
How People Learn
Key
Findings
Instructional Materials
 Include structured strategies to elicit and
challenge student preconceptions
First
 Incorporate background for the teacher about
common preconceptions
 Be organized around a conceptual framework
Second
 Connect factual information to the framework
 Provide relevant examples to illustrate key ideas
 Make learning goals explicit
Third
 Integrate metacognitive skill development into
content
Gathering Evidence
Careers in
Science
= Major Concepts
Icons = Work Students Do
= Assessments
= Real World Applications
= Weak Connections
= Strong Connections
Science Content: Conceptual Flow Graphic
An example of a graphic organizer from the AIM Process
showing how teachers might illustrate the connections among major concepts
in a unit from BSCS Biology: A Human Approach
Strengths and Limitations: Science Content
Criteria and
Components
Standards Alignment
Content
Accuracy
Concept Development
(HPL 2)
Sequencing (HPL 2)
Context (HPL 2)
Summary of Strengths
Summary of Limitations
Work Students Do: Evidence Chart
Type of Activity
Student Product
How does this activity build student
understanding of the concept?
Strengths and Limitations: Work Students Do
Criteria and
Components
Engaging Prior
Knowledge (HPL 1)
Work Students Do
Metacognition (HPL 3)
Abilities Necessary To Do
Scientific Inquiry
Understandings About
Scientific Inquiry
Accessibility
Summary of Strengths
Summary of Limitations
Assessment: Evidence Chart
Record the type of assessment in column one. In column two, list the page number of the assessment. In column three,
describe how the assessment helps measure student understanding and inform instruction.
Type of
Assessment
Page
Comments
How does the assessment measure student understanding?
Inform instruction?
Strengths and Limitations: Assessment
Criteria and
Components
Quality
Assessment
Multiple Measures
Use of Assessments
Accessibility
Summary of Strengths
Summary of Limitations
Work Teachers Do: Evidence Chart
Strategies
(Instructional Model, Teaching Strategies
[including inquiry] Assessment
Strategies)
Evidence of Support for implementing the strategies
Pertinent content background information, explanations of specific teaching strategies to improve student
understanding, resources to assist and enhance instruction (e.g. transparencies, test bank, videos, CDs, software),
list of material and equipment needs including information about maintenance and safe use, technical support for the
use of equipment, multi-media, and technology resources.
Strengths and Limitations: Work Teachers Do
Criteria and Components
The Work Teachers Do
Instructional Model
Effective Teaching Strategies
Teaching Strategies for Inquiry
Support for the Work Teachers Do
Summary of Strengths
Summary of Limitations
The BSCS Center for Professional Development offers a variety of professional
development services.
Contact the:
BSCS Center for Professional Development
5415 Mark Dabling Boulevard
Colorado Springs, Colorado
719.531.5550
fax: 719.531.9104
www.bscs.org
Nancy Landes
Director, Center for Professional Development
pd@bscs.org