Dick & Carey Instructional
Design Module Cheat Tool
By Jesse Gentile
USE TOOL
TEST LEARNING
Evaluate your understanding
of the Dick and Carey Model.
Revise
Instruction
Conduct
Instructional
Analysis.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Analyze
Learners &
Contexts
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
Click on any part of the Dick and Carey model to
test your knowledge of that step.
RETURN TO
HOME PAGE
INCORRECT
CORRECT
Identify Instructional Goals
SAMPLE QUESTION
- Answer 1
- Answer 2
- Answer 3
- Answer 4
Instructional Analysis
SAMPLE QUESTION
- Answer 1
- Answer 2
- Answer 3
- Answer 4
Learner/Context Analysis
•
•
•
•
•
SAMPLE QUESTION
Answer 1
Answer 2
Answer 3
Answer 4
Instructional Materials
SAMPLE QUESTION
- Answer 1
- Answer 2
- Answer 3
- Answer 4
Formative Evaluation
SAMPLE QUESTION
- Answer 1
- Answer 2
- Answer 3
- Answer 4
Summative Evaluation
SAMPLE QUESTION
- Answer 1
- Answer 2
- Answer 3
- Answer 4
Assess
Needs To
Identify Goal
1.1
Revise
Instruction
Conduct
Instructional
Analysis.
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Analyze
Learners &
Contexts
MORE
Design &
Conduct
Summative
Evaluation
Step #1 is to figure out if you actually need instruction to solve the problem or
limitations of your client. If instruction is needed, you need state what you want
your learners to be able to do when they have completed your instruction. This
will serve as your instructional goal.
Instructional Goals can come from:
WEB
SITES
• a list of goals
• from a performance analysis
• from a needs assessment
TIPS
• from practical experience with learning difficulties of students
• from the analysis of people who are doing a job
BOOKS
• from some other requirement for new instruction
Non analysis based sources: SME, content outlines, trainers or admin
Dick, Carey & Carey; 2006; pg 6
Assess
Needs To
Identify Goal
1.2
Revise
Instruction
Conduct
Instructional
Analysis.
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Analyze
Learners &
Contexts
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
Front End Analysis
MORE
WEB
“it has long been accepted that careful analysis work is absolutely critical prior to
initiating the design of instruction.” (2006, p17)
Front end analysis is usually made up of:
(1) Performance Analysis (2) Needs Assessment (3) Job analysis
SITES
Identify Goal 1
for initiating
the Design of
Instruction
Is there a
TIPS
Conduct
Performance
Analysis
performance
problem that can
be solved by
Conduct Needs
Assessment as
Required
Conduct Job
Analysis (in
Some training
Settings)
Identify Goal 2
for initiating
the Design of
Instruction
instruction?
BOOKS
Choose
Noninstructional
Solutions
Identify Goal 3
for initiating
the Design of
Instruction
Assess
Needs To
Identify Goal
1.3
Revise
Instruction
Conduct
Instructional
Analysis.
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Analyze
Learners &
Contexts
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
PERFORMANCE ANALYSIS
MORE
WEB
SITES
TIPS
BOOKS
Performance analysis is done basically to discover a clients true problems as well as identify
solutions. If done correctly you should be able to “clearly” describe the existing problem in terms
of failure to achieve the results that an organization wanted. It also involves discovering how
employees are performing versus how they are supposed to be performing. Causes of the problem
should be turned up as well as some possible solutions.
Performance Analysis
Application of the Robinson and Robinson (1995) Performance
Relationship Map (Dick & Carey, 2006 p 21)
Performance Analysis Question
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
What is the problem that was originally
voiced?
Is the voiced problem related to a core
organizational outcome?
Are there established operational goals for
this outcome?
Is the operational goal being met?
Is there an operational need?
Have job performance standards been set for
achieving the operational goal?
Are job performance standards being met?
Is there a job performance need?
Are there external factors outside the control
of local management that are contributing to
operational and job performance needs (e.g.,
government regulations, corporate hiring
freeze, labor contract, corporation’s national
contract with telephone service provider, and
so forth)?
Are there internal factors within the control of
local management that are contributing to job
performance needs?
Are there solutions for the performance
needs?
Performance Analysis Answer
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Analyze
Learners &
Contexts
MORE
SITES
Needs Assessment Concept Map
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
NEEDS ASSESSMENT
WEB
1.3
Revise
Instruction
Conduct
Instructional
Analysis.
Simple Diagram
TIPS
BOOKS
http://chd.gmu.edu/immersion/gap.html
http://www.personal.psu.edu/sjm256/portfolio/kbase/IDD/analysis.html
Assess
Needs To
Identify Goal
1 Websites
Revise
Instruction
Conduct
Instructional
Analysis.
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Analyze
Learners &
Contexts
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
(1) http://www.personal.psu.edu/sjm256/portfolio/kbase/IDD/analysis.html
MORE
Needs Analysis/Assessment tools on Don Clarks awesome ISD website
WEB
SITES
TIPS
(2) http://www.nwlink.com/~donclark/hrd/sat2.html
(5) The Ultimate Page
on Ins. Design Models
(3) DON CLARK’S
WEBSITE
BOOKS
(4) http://www.courses.psu.edu/trdev/trdev518_bow100/N_Apresent/
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Develop
Assessment
Instruments
Analyze
Learners &
Contexts
MORE
WEB
SITES
TIPS
BOOKS
1
Books
Revise
Instruction
Conduct
Instructional
Analysis.
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
How to Identify Your Organization's Training Needs: A Practical Guide to Needs
Analysis by McConnell, John H.
http://www.netlibrary.com.proxy.lib.fsu.edu/
Available online from FSU’s netlibrary.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Analyze
Learners &
Contexts
MORE
WEB
2.1
Revise
Instruction
Conduct
Instructional
Analysis.
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
Step #2: Once you have clearly figured out what the goal of your instruction is going
to be, you need to break down that goal into very clear, concrete steps that the
learner must perform if they were to accomplish the goal.
Analysis refers to analyzing your main goal, breaking it down. What does a person
actually DO, step by step, when performing this goal?
SITES
TIPS
BOOKS
TWO BASIC STEPS OF INSTRUCTIONAL ANALYSIS
1.) Classify your goal into one of four domains of learning. In other words, what KIND
of information are you teaching with your instruction.
A) Verbal information
B) Psychomotor Skills
C) Intellectual Skills
D) Attitudes
2.) Make an organized flow chart type map of the exact steps or info that must be
done by learn when accomplishing the overall (terminal) goal.
Assess
Needs To
Identify Goal
2.2
Revise
Instruction
Conduct
Instructional
Analysis.
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Analyze
Learners &
Contexts
MORE
WEB
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
“Goal analysis is the visual display of the specific steps the learner would do when
performing the instructional goal. Each step is stated in a box as shown in the flow
diagram below:” (p 44)
Step
Step
Step
Step
Step
1
2
3
4
5
SITES
“the statement [written in each box] of each step must include a verb that describes
and observable behavior. Each step should have an observable outcome.” p45
TIPS
BOOKS
When it comes to charting out psychomotor skills and intellectual skills your chart
shows the steps the learner will perform. If the goal is to learn verbal information,
your chart is basically going to list out the verbal information in organized clusters.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Analyze
Learners &
Contexts
MORE
2.3
Revise
Instruction
Conduct
Instructional
Analysis.
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
• In the full page example to the right, look at how the
top of the diagram has broken down the main goal
into six main steps.
• Step two, has 3 sub-steps must be done to
accomplish it: 2.1, 2.2, and 2.3.
WEB
SITES
TIPS
BOOKS
• Several steps have subordinate skills. These are
simpler skills the learner must have if they are going
to be able to accomplish a step. (i.e. you must be
able to add, before you can solve equations)
• The triangle represents some general verbal fact or
knowledge the student must be able to state to
accomplish sub-skill 2.2.2
• Lastly, there are “entry-behaviors,” behaviors the
learner must be able to do before starting the lesson
which won’t be taught.
CLICK TO SEE ALL
THE MAIN PARTS OF A
GOAL ANALYSIS IN
ONE SCREEN
Assess
Needs To
Identify Goal
2
Tips
Revise
Instruction
Conduct
Instructional
Analysis.
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Analyze
Learners &
Contexts
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
“The SME approach [to instructional] tends to stress knowing, whereas the
instructional design approach stresses doing.” p40
MORE
WEB
The “Conduct Instructional Analysis” step which we are now describing is depicted
as a parallel activity to the “Analyze Learners and Contexts” step… the elements of
that step can be completed simultaneously with goal analysis…” p45
SITES
TIPS
BOOKS
Have a subject matter expert perform the goal for you, so you can watch them and
document each step they take in completing the goal. If you are the SME, have
someone else watch you. Have two SME’s explain how they would accomplish the
terminal goal step by step.
Assess
Needs To
Identify Goal
3.1
Revise
Instruction
Conduct
Instructional
Analysis.
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Analyze
Learners &
Contexts
MORE
WEB
SITES
TIPS
BOOKS
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
#3 At the same time you are figuring out exactly what steps learners will take to
perform your instructional goal, you must spend time figuring out the
characteristics of your learners and the places they will be learning or working in.
YOU MUST ANALYZE THREE THINGS:
(1) Who are your learners (known as target population or target audience) and what are they like?
- age, grade, job, experience, job position,
- whether they can do entry behaviors, attitudes toward what you are trying to teach them,
- their motivation levels, learning style preferences
- learners attitudes towards management or administrators
(2) What environment will the learners perform their new skills in?
- Will managers support the new skills?
- Physical aspects (tools, timing, equipment) of the work environment.
- Social aspects. Are learners working alone or in teams? Field or desk work? Pioneers?
(3) The place where learners will be learning your instruction.
- Does it meet your teaching needs?
- Can you make it mirror the workplaces so new skills transfer over easily?
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Develop
Assessment
Instruments
Analyze
Learners &
Contexts
MORE
Recommended by Dick and Carey.
Available online at Library.net
SITES
TIPS
BOOKS
Develop
Instructional
Strategy
Books
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
Task Analysis Methods For
Instructional Design.
WEB
2
Revise
Instruction
Conduct
Instructional
Analysis.
1999 copyright.
Assess
Needs To
Identify Goal
2
Revise
Instruction
Conduct
Instructional
Analysis.
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Web
Develop &
Select
Instructional
Materials
Analyze
Learners &
Contexts
(1) Software to help do task analysis. http://www.taskarchitect.com
MORE
(2) Task Analysis examples at George Mason University.
http://classweb.gmu.edu/ndabbagh/Resources/Resources2/Task%20Analysis.htm
WEB
SITES
TIPS
BOOKS
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
Assess
Needs To
Identify Goal
3.1
Revise
Instruction
Conduct
Instructional
Analysis.
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Analyze
Learners &
Contexts
MORE
WEB
SITES
TIPS
BOOKS
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
#3 At the same time you are figuring out exactly what steps learners will take to
perform your instructional goal, you must spend time figuring out the
characteristics of your learners and the places they will be learning or working in.
YOU MUST ANALYZE THREE THINGS:
(1) Who are your learners (known as target population or target audience) and what are they like?
- age, grade, job, experience, job position,
- whether they can do entry behaviors, attitudes toward what you are trying to teach them,
- their motivation levels, learning style preferences
- learners attitudes towards management or administrators
(2) What environment will the learners perform their new skills in?
- Will managers support the new skills?
- Physical aspects (tools, timing, equipment) of the work environment.
- Social aspects. Are learners working alone or in teams? Field or desk work? Pioneers?
- What will prevent training from being used? Do they lack necessary items to perform?
(3) The place where learners will be learning your instruction.
- Does it meet your teaching needs?
- Can you make it mirror the workplaces so new skills transfer over easily?
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Develop
Assessment
Instruments
Analyze
Learners &
Contexts
MORE
WEB
SITES
3.1
Revise
Instruction
Conduct
Instructional
Analysis.
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
How do you collect all this data about learners? How do you “analyze them?”
Make “site visits” to where they work.
(a) interview learners
(b) interview management
(c) observe learners while they work
(d) give learners surveys or questionnaires
(e) have the learners take a pre-test to see if they have entry behaviors or sub skills
necessary to accomplish your instructional goal.
Write up these results in a learner analysis report.
TIPS
TABLES 5.1-5.6 in Dick and Carey’s 2006 BOOK ARE VERY INFORMATIVE HERE. SEE PAGES 105117 IF POSSIBLE.
BOOKS
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Analyze
Learners &
Contexts
MORE
WEB
SITES
TIPS
BOOKS
3
Revise
Instruction
Conduct
Instructional
Analysis.
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Tips
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
If you really want your learners to carry their new knowledge into the workplace,
then your learning context (where students learn) must mirror your performance
context (the work place) as closely as possible.
Revise
Instruction
Conduct
Instructional
Analysis.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Analyze
Learners &
Contexts
MORE
WEB
SITES
TIPS
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
#4 At this point you are finished with the analysis phases of instructional design.
The next step in the Dick and Carey instructional design model is to write a list of
objectives for your goal based on all this information you have gathered.
What is an objective? According to Dick and Carey, a performance objective is a
detailed description of what students will be able to do when they complete a unit of
instruction. It is also referred to as a behavioral objective or an instructional
objective.
Essentially at this point, you are taking each of the goal step boxes from your flow
chart and making them into a certain type of verbal format (click more).
These clearly written objectives will help you in your next steps to do things like
select or develop the right instructional materials, as well as help you evaluate
whether or not your instruction has been successful.
BOOKS
Write one or more objectives for each skill listed in your instructional analysis,
including your entry behaviors.
Revise
Instruction
Conduct
Instructional
Analysis.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Analyze
Learners &
Contexts
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
According to Mager (1997), there are three main parts of a well written performance
objective:
MORE
(1) Performance
(2) Conditions
WEB
SITES
TIPS
BOOKS
(3) Criterion
Revise
Instruction
Conduct
Instructional
Analysis.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Develop
Assessment
Instruments
Analyze
Learners &
Contexts
MORE
WEB
SITES
TIPS
BOOKS
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
#5 Once you know what actual behaviors look like for each step of your instructional
goal, you must take the time to come up with some way to test learners on each of
those steps to find out if they understood or did them correctly.
Robert Mager states in his book Making Instruction Work, " If it’s worth teaching, it’s
worth finding out whether the instruction was successful. (pg. 83). If you think of
objectives as describing where you are going, the assessment items are the means
by which you find out whether you got there. (FROM: .itma.vt.edu/modules/spring03/instrdes/lesson7.htm)
Why are test items are created now, and not after you develop your instruction?
Your assessment items must come right from your objectives. The performance
you test in each assessment item should match the performance in the objective.
They should not be based on what you think are fun test questions or what you did in
your activity.
If you‘ve written worthwhile objectives, you already know what content to test
for. Then it’s just a matter of creating good test items that measure the acquisition of
the skills, knowledge, or attitudes you are looking for.
(FROM: .itma.vt.edu/modules/spring03/instrdes/lesson7.htm)
Revise
Instruction
Conduct
Instructional
Analysis.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Analyze
Learners &
Contexts
Design &
Conduct
Summative
Evaluation
These kinds of tests are known as CRITERION REFERENCED or OBEJECTIVE
REFERENCED test. Your test measures learners based on criteria or objectives.
MORE
WEB
SITES
TIPS
FOUR TYPES OF TESTS YOU SHOULD DESIGN AND USE
(1) ENTRY BEHAVIOR or PRE-REQUISIT TESTS – does this learner have the basic
entry skills to be able to handle my instructional materials?
(2) PRETESTS – How much of this material does the learner already know? Do they
even need this instruction? Should they skip ahead?
(3) PRACTICE TESTS – Gives learners a chance to rehears new knowledge and skills
during learning so they can see what they are really picking up and what they need to
go back and review.
(4) POST TEST – This is the final test of what learners learned. It can be used to give
course credit. They can also be used to alert the instructor about areas of instruction
that are not working for some reason. (All the learners miss the same test items)
BOOKS
INSTRUCTION IS GOING ON HERE
Give Entry/Pre-tests here.
Give practice test during instruction.
Give post-test when done.
Revise
Instruction
Conduct
Instructional
Analysis.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Analyze
Learners &
Contexts
Designer’s Decision
Objectives Usually Tested
Entry behaviors test
•Are target learners ready to enter
instruction?
•Do learners possess the required
prerequisite skills?
•Prerequisite skills or those skills below
the dotted line in the instructional
analysis
pretest
•Have learners previously mastered
the enabling skills?
•Which skills have they previously
mastered?
•How can I most effectively develop
this instruction>
•Terminal objectives
•Main steps from the goal analysis
Practice tests
•Are students “getting it”?
•What errors and misconnects do they
have?
•Is the instruction pace correct?
•Knowledge and skills for a subset of
objectives within the goal.
•Scope typically at the lesson rather
than at the unit level.
Posttests
• Achieve the terminal objective?
•Have learners mastered the intended
info,skills and attitudes?
•Where should instruct. be revised?
•The terminal objective
•Main steps and their subordinate
skills.
WEB
SITES
BOOKS
Design &
Conduct
Summative
Evaluation
Test Type
MORE
TIPS
Design &
Conduct
Formative
Evaluation
Dick and Carey, 2006, p148-149
Revise
Instruction
Conduct
Instructional
Analysis.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Analyze
Learners &
Contexts
MORE
WEB
SITES
TIPS
BOOKS
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
So, what types of test items can a person create?
- COMPLETION
- SHORT ANSWER
- MATCHING
- MULTIPLE CHOICE
- ESSAY
- PRODUCT DEVELOPMENT
- LIVE PERFORMANCE
KEY: You must note carefully the behavior described
by the verb in each objective. If the verb is to match, to
list, to select, or describe, then you must provide a test
item that allows a student to match, list select or
describe.
CLICK FOR A CHART SHORING
WHICH TEMS ITME TYPES GO BEST
WITH WHICH OBJECTIVES TYPES.
Revise
Instruction
Conduct
Instructional
Analysis.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Analyze
Learners &
Contexts
Design &
Conduct
Summative
Evaluation
Writing Test Items
MORE
WEB
SITES
TIPS
BOOKS
You should write an assessment item for each objective whose accomplishment you want to
measure. Mager provides these steps to follow when writing a criterion assessment item:
(1) Read the objective and determine what it wants someone to be able to do
(i.e., identify the performance).
(2) Draft a test item that asks students to exhibit that performance.
(3) Read the objective again and note the conditions under which the performing should
occur (i.e., tools and equipment provided, people present, key environmental conditions).
(4) Write those conditions into your item.
(5) For conditions you cannot provide, describe approximations that are as close to the
objective
as you can imagine.
(6) If you feel you must have more than one item to test an objective, it should be because
(a) the range of possible conditions is so great that one performance won’t tell you that the
student can perform under the entire range of conditions, or (b) the performance could be
correct by chance. Be sure that each item calls for the performance stated in the objective,
under the conditions called for.
If you follow these steps and still find yourself having trouble drafting an assessment item, it is
almost always be because the objective isn’t clear enough to provide the necessary
guidance.
FROM: http://www.itma.vt.edu/modules/spring03/instrdes/lesson7.htm
Revise
Instruction
Conduct
Instructional
Analysis.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Analyze
Learners &
Contexts
Design &
Conduct
Summative
Evaluation
Criteria for Writing Test Items
MORE
Dick and Carey list several criteria that you should consider when writing test
items:
- Goal-Centered Criteria: Test questions should be built directly from performance
objectives. One per objective.
WEB
SITES
TIPS
- Learner-Centered Criteria: Test Items must be learner appropriate. The vocabulary
should be right for the students age. Items should be culturally sensitive.
- Context-Centered Criteria: test items should match the real world environment and
scenario and equipment as closely as possible.
- Assessment-Centered Criteria: Test items should be printed, formatted, and visually
clear.
BOOKS
FROM: http://www.itma.vt.edu/modules/spring03/instrdes/lesson7.htm
Types of Test Items
TYPE OF BEHAVIOR
STATED IN OBJECTIVE
COMPLETION
SHORT
ANSWER
MATCHING
MULTIPLECHOICE
State/name
X
X
Define
X
X
X
X
Identify
X
X
X
X
Discriminate
X
X
X
Select
X
X
X
Locate
X
X
X
Evaluate/Judge
X
X
X
Solve
X
X
X
ESSAY
X
PRODUCT
DEVELOP
X
LIVE
PERFORMANCE
X
Discuss
X
X
Develop
X
X
X
Construct
X
X
X
Generate
X
X
X
Operate/Perform
X
Choose (attitude)
X
Dick & Carey, 2006, p155
Revise
Instruction
Conduct
Instructional
Analysis.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Analyze
Learners &
Contexts
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
Build your activities from, and base them on your assessment items and objectives.
Do not base assessment items on activities.
MORE
WEB
SITES
TIPS
BOOKS
Don’t just pick the KINDS of test questions you ask off the top of your head. Make
sure the KIND of test question appropriate matched the VERB in the objective it is
testing.
Revise
Instruction
Conduct
Instructional
Analysis.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Analyze
Learners &
Contexts
http://www.itma.vt.edu/modules/spring03/instrdes/lesson7.htm
MORE
WEB
SITES
TIPS
BOOKS
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
Revise
Instruction
Conduct
Instructional
Analysis.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Analyze
Learners &
Contexts
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
Revise Your Instruction Throughout the Design Process
MORE
Notice the dotted lines along the top of the chart starts at the far right item
labeled “Design and Conduct Formative Evaluation.”
WEB
SITES
TIPS
BOOKS
The simple idea here is that the evaluation done before finally publishing your
instructional material will provide you with lots of data to use in going back to
each or particular steps of the whole process and modifying your instructional
product.
For example, if you discover that after giving the lessons to several sample
students, that the instructional materials you chose were hard to see according to
many students, you would go back select some different materials that were
easier to see.
Revise
Instruction
Conduct
Instructional
Analysis.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Analyze
Learners &
Contexts
MORE
#6 Now that you have a goal that has been analyzed, broken down into steps, each
step described by a performance objective and each objective give a test question it
is time to think about creating a type of lesson plan to actually teach your
objectives!
So far we have looked at WHAT to teach. Now we will decide HOW to teach it.
WEB
SITES
TIPS
BOOKS
Design &
Conduct
Summative
Evaluation
BASIC STEPS IN CREATING AN INSTRUCTIOANL STRATEGY
(1) Decide what order you want to teach the content
(2) in and how you want to cluster certain topics during your instruction.
Revise
Instruction
Conduct
Instructional
Analysis.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Analyze
Learners &
Contexts
MORE
WEB
SITES
TIPS
BOOKS
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
UNDER CONSTRUCTION
Revise
Instruction
Conduct
Instructional
Analysis.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Analyze
Learners &
Contexts
MORE
WEB
SITES
TIPS
BOOKS
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
#8 Now that you have your instructional materials developed, it makes good sense
to test run them with some learners so that you can discover any changes or
weaknesses that need to be revised before you officially start using the materials.
Formative Evaluation: trying out your materials with an intention to revise, reFORM,
fix, update, improve those materials based on what your users share with you. The
goal is to use what you find out to go back and revise materials. This is why there is
a dotted line from the tan box back to other boxes. You may need to go back to any
of these other steps.
FORMATIVE EVAL CAN HELP YOU DISCOVER: attitudes toward the content,
interests in the content, and learners’ experience with the medium in design
phase, [what will] reduce expensive mistakes and improve user friendliness in
production phase, [how to] restructure the products for different settings in
implementation phase (Tessmer, 1993).
Revise
Instruction
Conduct
Instructional
Analysis.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Analyze
Learners &
Contexts
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
Dick and Carey focus on three phases of formative evaluation. However, Tessmer
identifies four distinct phases:
MORE
WEB
SITES
TIPS
Expert Review - Experts review a rough version of the instruction with or without the
evaluator present to determine its strengths and weaknesses. They can be
content experts, technical experts, designers, or instructors.
One-to-One - One learner at a time reviews the instruction with the evaluator. The
evaluator observes the learner using the instruction, notes the learner�s
comments, and questions the learner during and after the instruction.
Small Group - The evaluator tries out the unfinished instruction with a group of
learners and records their performance and comments.
Field Test - The evaluator observes the instruction being tried out in a realistic
environment with a group of learners.
BOOKS
Revise
Instruction
Conduct
Instructional
Analysis.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Analyze
Learners &
Contexts
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
Martin Tessmer, 1993, suggested this order for formative evaluation, moving from
left to right in the graphic and making revisions along the way.
MORE
WEB
SITES
TIPS
BOOKS
Revise
Instruction
Conduct
Instructional
Analysis.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Analyze
Learners &
Contexts
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
Having the evaluation focus on your instructional strategy will help it keep in focus.
MORE
WEB
SITES
TIPS
BOOKS
Ask these five questions about your instructional strategy in your evaluation:
(1) Are the materials appropriate for the type of learning outcome?
(2) Do the materials include adequate instruction on the subordinate skills, and
are
these skills sequenced and clustered logically?
(3) Are the materials clear and readily understood by members of the target
group?
(4) What is the motivational value of my materials? Do learners find the materials
relevant to their needs and interests?
(5) Can the materials be managed efficiently in the manner they are mediated?
Revise
Instruction
Conduct
Instructional
Analysis.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Analyze
Learners &
Contexts
"Planning and conducting formative evaluation" by Martin Tessmer (1993)
MORE
WEB
SITES
TIPS
BOOKS
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
Revise
Instruction
Conduct
Instructional
Analysis.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Analyze
Learners &
Contexts
MORE
(2) FORMATIVE EVAL: WHAT, WHY, WHEN, HOW?
http://www.geocities.com/zulkardi/books.html
(3) DESIGNING AND CONSTRUCTING FORMATIVE EVALUATION BY BARRY WILLIAMS
http://www.courses.psu.edu/trdev/trdev518_bow100/D_C10present/
WEB
SITES
TIPS
BOOKS
Design &
Conduct
Summative
Evaluation
Revise
Instruction
Conduct
Instructional
Analysis.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Analyze
Learners &
Contexts
MORE
WEB
SITES
BOOKS
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation
“Other studies have demonstrated that simply trying out materials with a single
learner and revising the materials on the basis of that data can make a significant
difference in the effectiveness of materials.”p278
Many believe that formative evaluation is the key to instructional success. It doesn’t
matter how good you are as a designer, you don’t know if it works until you try it
out.
Sadly, in many cases this step is skipped to save money.
TIPS
Develop &
Select
Instructional
Materials
Revise
Instruction
Conduct
Instructional
Analysis.
Assess
Needs To
Identify Goal
Write
Performance
Objectives
Analyze
Learners &
Contexts
MORE
WEB
SITES
TIPS
BOOKS
Develop
Assessment
Instruments
Develop
Instructional
Strategy
Develop &
Select
Instructional
Materials
Design &
Conduct
Formative
Evaluation
Design &
Conduct
Summative
Evaluation