Instructional Program Checklist

advertisement

Instructional Materials (Print-based): Criteria to Consider

Many of you are will be working on studies that call for assessment of print-based materials (content analyses, perhaps). Following are elements that evaluators commonly consider when tasked with such an assignment. Two exceptionally good resources you’ll want to consult include:

 Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A taxonomy for learning,

teaching, and assessing: A revision of Bloom’s taxonomy of educational

objectives. New York: Longman.

Marzano, R. J. (2001). Designing a new taxonomy of educational objectives.

Thousand Oaks, CA: Corwin Press, Inc.

1.

Objectives

Does the program have them?

Are they well-written?

Are they worthwhile?

Comments:

2.

Content

Does the content relate directly to the objectives?

If not, how does the extra/ancillary material add to or detract from the program?

Comments:

3.

Presentational style (flow)

4.

Are learners given the information they need to attain each objective?

How is the information written (e.g., is it clear and to the point)?

Are examples provided?

Do the examples provide a range of possibilities? Do they reflect a spectrum of instances?

Comments:

Practice

Is practice or review provided for each objective?

What form does it take (e.g., written or oral)?

© MJBober, 1998 (rev. 2000, 2004, 2008) 1

7.

Does it seem to be sufficient (quality, scope, etc.)?

Is practice/review matched to the objective(s)?

Are the practice items and directions to learners/teachers well written?

Is constructive feedback (relative to skill, understanding, or progress) provided

(or made available)?

Comments:

5.

Assessment

How is learner performance assessed?

Are the objectives, the content, and the assessment aligned?

What is the quality of the writing (of the test items)?

How are the objectives weighted?

Are the directions for administering assessment clear?

How is the assessment scored? How difficult a task is scoring?

Is scoring subjective (norm-referenced) or objective (criterion-referenced)?

Comments:

6.

Appeal

Is there variety in the types of activities?

How are learners motivated to complete the tasks before them? What hooks them?

In what ways, if any, is the product appealing (e.g., representative of good design principles)?

Is the format easy to follow? Is it consistent?

Do graphics/icons/symbols enhance or detract from comprehensibility?

Does the format seem to fit the content?

Comments:

Efficiency

© MJBober, 1998 (rev. 2000, 2004, 2008) 2

Do the objectives justify the length (amount of time) of the program

(unit/lesson)?

Are there opportunities for enrichment or remediation?

Comments:

8.

Ease of use

Instructor/facilitator

[If not self-paced] can instructors/facilitators learn to “use” the program easily?

[If the program (unit/lesson) is not self-contained] are the set-up notes clearly delineated?

Learner

[Whether or not self-paced] are directions/instructions for how to proceed clear and unambiguous?

Are outcomes/performance criteria (for activities) explained?

*Are instructor and learner materials well-aligned? Do they complement one another?

9.

Comments:

What are three to five positives about the product?

10.

What are three to five negatives about the product?

© MJBober, 1998 (rev. 2000, 2004, 2008) 3

Other issues you might want to consider:

 Appropriate for multiple ways learners might be grouped (individually, teams, whole class, etc.)?

Mix of instructional strategies?

Promoting higher-ordering thinking, complex skills, problem-solving?

Underlying instructional philosophy?

Adaptable for different learning styles and/or ability levels?

Scalable for learners with differing prerequisite skills/competencies?

© MJBober, 1998 (rev. 2000, 2004, 2008) 4

Instructional Materials (Print-based): Criteria to Consider

By now you’ve realized that the criteria list above has limited utility. First, its qualitative nature could translate into hours of laborious analysis. As important, its structure doesn’t well attend to potential rater bias/subjectivity.

Following, then, are some ways to transform it into a scaled tool easily used (after brief training) by multiple raters.

1.

Objectives

Evident?

Well-written?

Worthwhile?

Yes

Yes

Yes

No

No

No

Overall rating

Deficient

Adequate

Good

Excellent

Comments:

Deficient

Adequate

Good

Excellent

2. Content

Extent to which content relates directly to the objectives?

Comments:

© MJBober, 1998 (rev. 2000, 2004, 2008) 5

3. Presentational style/flow

Learners given the information they need to attain each objective?

Information clear and to the point?

Quality of the examples provided?

Examples provide a range of possibilities OR reflect a spectrum of instances?

Comments:

Deficient

Deficient

Deficient

Deficient

4. Practice

Practice or review provided for each objective?

Form of the practice

Yes

Oral

Adequate

Adequate

Adequate

Adequate

No

Written

Good

Good

Good

Good

Excellent

Excellent

Excellent

Excellent

© MJBober, 1998 (rev. 2000, 2004, 2008) 6

Sufficiency (quality, scope)?

Extent to which practice/review matches objectives?

Quality of the directions?

Deficient

Deficient

Deficient

Comments:

Etc ...

Adequate

Adequate

Adequate

Good

Good

Good

Excellent

Excellent

Excellent

© MJBober, 1998 (rev. 2000, 2004, 2008) 7

Download