 The Assessment Toolkit
 Using Authentic Assessment to Assess SLOs
 Selection of Direct and Indirect Methods of Assessment
 Selection Of Test Question Types To Measure To Specific SLOs.
 Test Item Analysis
Outcomes: At the completion of Module Three faculty should be able to:
1.
Explain why a variety of assessments are necessary.
2.
Distinguish between direct and indirect methods of assessment.
3.
Discuss the concept of authentic assessment and design an authentic assessment for your content area.
4.
Select appropriate methods of assessment to match SLO to be measured.
5.
Explore the use of CATs to assess student knowledge of content, attitudes and self assessment and reactions toward teaching and teaching materials.
6.
Identify questions you want answered about your teaching and learning and select an appropriate CAT to answer the question.
7.
Use a rubric to assess Student Learning Outcomes (SLO).
8.
Utilize Bloom’s Taxonomy to design test questions to align with student learning outcomes being measured.
9.
Evaluate tests items to validate that they indeed measure the SLO.
10. Identify the strengths and weakness of different types of test questions.
11. Select appropriate type of test questions to measure specific SLOs.
12. Create an assessment plan that outlines the specific methods that will be used to assess the specific expected student learning outcomes for a course.
13. Create a matrix that shows the appropriate activity and assessment for specific SLOs.
14. Write effective test items by applying helpful hints.
15. Use a Test Blueprint to evaluate tests items to validate that they indeed measure the
SLO and provide challenges for students at higher levels of learning.
Updating Curriculum to Support Learning
DCCC Perkins Grant
III - 2

Stage Three: Assess the Outcomes
What is Assessment and Why Do IT?
Direct and Indirect Assessment
Authentic Assessment
The Assessment Toolbox
Toolbox of Authentic Assessment
Designing Authentic Assessment Worksheet
CATs – (Classroom Assessment Techniques)
Using CATS to Answer Questions About Teaching and Learning
Using Rubrics for Direct Assessment of Student Work
Rubric Worksheet
Outcomes/ Assessment Alignment Worksheet
Using Bloom’s Taxonomy to Write Test Questions
Direct Verbs Provide Clues
Pros and Cons of Types of Tests
How to Write Effective Questions in Various Formats
Test Blue Print
Teaching Approach Assessment Matrix
My Plan
4
5
15
16
17
18-19
20
21
22
9-10
11
12
13
14
6
7
8
Updating Curriculum to Support Learning
DCCC Perkins Grant
III - 3

Stage Three:
How do I assess student learning outcomes?
What is Assessment and Why Do It?
Assessment is an ongoing process that investigates student learning. The continuous improvement aspect of the assessment cycle models the commitment to the quality of learning within a program and a college.
The Assessment Process
Source: Fulks, Janet. (2004) “ The SLO Toolkit.” Bakersfield College.
Updating Curriculum to Support Learning
DCCC Perkins Grant
III - 4

Direct assessment methods are measures of student learning that require students to display their actual knowledge and skills rather than report what they think their knowledge and skills are.
(Summative)
 Capstone project
 Case studies
 Demonstrations
 Exams, quizzes, and tests
 Internship and/or clinical evaluations
 Licensure exams
 Performance assessments
Indirect assessment methods ask students to reflect on their learning rather than to actually demonstrate it. (e.g., student opinions about what they learned and or student ratings of instruction. Also, other self report data like employer surveys, advisory group.)
(Formative)
 Classroom Assessment Techniques (C.A.T.s)
 Focus groups
 Graduation and retention rates
 Interviews
 Job/graduate school placement statistics
 Reflective journals
 Surveys (employer satisfaction)
 Portfolio evaluation
 Pre and post-tests
 Simulations
 Advisory Group suggestions
What is it?
“A form of assessment in which students are asked to perform real-world tasks that demonstrate meaningful application of essential knowledge and skills., says Jon Mueller.
Traditional Assessment Authentic Assessment
Students select an answer or recall information to complete the assessment such as in test like multiple-choice, true/false, matching or fill-in-the blanks
Focuses on recall/recognition
Student perform a task which is simulated to a real word situation
Requires higher order thinking
Norm-referenced that can compare or rank students
Criterion-referenced that can identify strengths and weaknesses
Updating Curriculum to Support Learning
DCCC Perkins Grant
III - 5

The Assessment Toolkit
Tools
Capstone Projects: Assessment of the integrated the skills, knowledge, and abilities of students at the completion of a program of study.
CATs: Classroom assessment activites used to gather information about student levels of knowledge about the content, about how they perceive they are learning and what they perceived about the instructor’s ability and the instructional materials.
Expected Learning Outcomes Statements Survey: Assesses directly how well students feel that they have mastered the expected learning outcomes for the course. Students complete a brief survey that asks them to rate how well they have learned each outcome. The simplest way to do this is to list each stated expected learning outcome and have students indicate on a scale how well they feel they have mastered each outcome. Leave space for students to provide evidence that they successfully mastered the outcome.
Performance Assessment: Students demonstrate skills and abilities in a real-time situation. Performance assessments, like portfolios, require well-designed instruments, criteria, rubrics, and norming between reviewers.
Portfolios: A display of students’ abilities through a collection of artifacts. Portfolios are good ways to assess change over time. Instructions to the students must be explicit, based upon the purpose and uses of the portfolio.
Rubrics should be available to show students what type of evidence needs to be collected and what criteria they must achieve.
Non-Graded Quizzes: Activities that provide great feedback on student learning. Non-graded quizzes can be given in a variety of formats (paper & pencil, questions in a PowerPoint presentation, etc.). Be sure to provide students with the correct answers to the quiz items, so they can learn from their mistakes! This choice of assessment is a great way to provide immediate feedback to the student.
Reaction Papers: Require students to write down their reactions (thoughts, feelings, attitudes, etc.) to course material or topics. Although reaction papers can be formal graded assignments, they can also be used solely for the purposes of assessing student learning. In this case, faculty members ask their students to take a few minutes of class time and write down their reaction to a fact, opinion, skill, procedure, attitude, critique, etc. that was addressed in the course. The CAT called the “Minute Paper” is a sample of a reaction paper.
Rubric: A scoring tool that lays out the specific expectations for an assignment. Rubrics divide an assignment into its component parts and provide a detailed description of what constitutes acceptable or unacceptable levels of performance for each of those parts.
Student-Generated Exam Questions A process where instructors ask students (either alone, in pairs, or in groups) to generate potential exam questions. Having students create potential exam questions helps the instructor assess (1) If students can identify key concepts; (2) What students see as fair or reasonable exam questions; and (3) How well the students can answer these potential exam questions.
Using this approach will motivate students to read the assignments and prepare better for tests.
Updating Curriculum to Support Learning
DCCC Perkins Grant
III - 6

Assignments /
Assessment
Descriptions
Application
Cards
On an index card have student choose from a list of principles or concepts you just discussed and indicate how they will apply it.
Concept Map These are visual images of important concepts. Uses a variety of ways of learning, visual, spatial, linguistic, logical, mathematical and kinesthetic. Engages both sides of the brain. Students remain involved throughout the session.
There are cluster Maps… flow chart maps, Burger Map, Free Flow map, and star-burst.
Graphic
Organizers
Use such things as a categorization grid allows you to organizes concepts and characteristics of those concepts; a Fishbone diagram to discover “cause and effect”; ; Ranking Ladder to sort a list and then rank items by placing them in a particular order; T-Graph is useful during brainstorming. You see issues and concerns related to both sides of a topic a time line to show historical events; 5
W’s chart is an introductory experience to help you gather information and summarize; Venn-diagram to “compare and contrast” concepts,
In the News Have students bring to the class articles, new items, editorials or cartoons related to the topic. Divide the class into small groups and ask members of each subgroup to share their item with one another and to choose the 2 or 3 most interesting items. Reconvene and ask members from sub-groups to share.
Media Diaries Each student tracks a current issue on TV, radio, and in print sources and compares and contrast the like and different ways each focuses on the issue.
Discipline Specific Examples of Authentic Learning
Geometry
Economics
English
Health
You want to install new carpet and a baseboard in you room, but you need to know how much carpet and wood to purchase. Write a set of directions to follow, measure and calculate the square footage of each item needed, and determine the cost of carpet and pad (use the Internet to research).
While studying the stock market and its effects on the economy, students will
“purchase” a stock or stocks with $5000 and plot their earnings over a three month period.
Research a career that interests you and report on the educational and job requirements, career outlook, job description, etc.
Record and analyze your food consumption over a given amount of time.
Updating Curriculum to Support Learning
DCCC Perkins Grant
III - 7

“…best identified through careful consideration of the nature of the task, the cognitive level of the objective, and the most feasible context for the assessment.”
Source: Garavalia, L, Marken, P and Sommi, R. (2003) “Selecting Appropriate Assessment Methods: Asking the
Right Questions” American Journal of Pharmaceutical Education Vol. 66, Summer 2003 p. 111
 _____________________________________________
 _____________________________________________
 _____________________________________________
 _____________________________________________
Student Learning Outcomes Authentic Assessment Idea
Updating Curriculum to Support Learning
DCCC Perkins Grant
III - 8

CAT Description
Assessing Course Related Knowledge And Skills
Application Cards
Background knowledge probe
Categorizing Grid
After teaching about an important theory, principle, or procedure, ask students to write down at least one real-world application for what they have just learned to determine how well they can transfer their learning.
Used to determine what students already know to determine a starting point for new instruction
Give students a grid containing two or three main categories plus a scrambled list of terms and have them sort the terms into the categories.
Concept Maps
Direct Paraphrasing
Students draw or diagram the mental connections between a major lecture concept and other concepts they know. This helps students to see connections.
Ask students to write a layman’s “translation” of something they have just learned to assess their ability to comprehend and transfer concepts.
Documented Problem
Solutions
Focused Listing
Muddiest Point
One Minute Paper
One Sentence
Summary
Pro and Con Grid
Students are asked to document the steps they take in solving a problem.
Students will gain more awareness of and control of their problem solving routines.
Give students a main term or concept and have them generate all the related terms they can think of. Practicing this technique can help students learn to focus attention ,improve recall and become familiar with terminology.
Have students write what they find least clear or most confusing about a particular lesson or topic.
 “What point is confusing to you?”
During the last few minutes of the class period, ask students to answer on a half-sheet of paper:
 “What is the most important point you learned today?”
Students summarize knowledge of a topic by constructing a single sentence that answers the questions ,”Who does what to whom, when, where, how, and why?” The purpose is to require students to select only the defining features of an idea.
Have students create a grid of the pros and cons for a given decision. This forces students to search for two sides of an issue or question.
Updating Curriculum to Support Learning
DCCC Perkins Grant
III - 9

CAT Description
Assessing Learner Attitudes, Values, and Self-Awareness
Classroom Opinion
Polls
Students respond to a survey or by hand-raising and quick counts to items presented by the instructor.
Diagnostic Learning
Logs
Students keep records about assignments they have done and the kind of errors or successes they experienced. At regular intervals they go back over these logs and reflect on their learning strategies.
Double-entry Journals In taking notes on class or readings, students keep a simultaneous running commentary of questions, ideas and connections in a second column.
Interest/Skill/
Knowledge
Checklist
The instructor prepares a list of course-specific skills, interests and knowledge and students assess themselves with regard to their interest or level of skill.
Punctuated Lectures Periodically during the lecture, the lecturer pauses to allow students to reflect and write on their own attention and behavior during the class in order to identify how actively they are listening or participating.
A ssessing Learner Reactions to Instruction
Do it Yourself Quiz
Film Feedback
Handouts Feedback
Students are asked to write out the five (5) questions about the key points in the reading and then, answer each. This assignment allows the teacher to see who has done the reading and what comprehension they have of the reading.
After viewing a film, the instructor asks students to answer the following two questions:
1. Explain (in as much detail as you can) the main idea illustrated by the film.
2. What can you suggest that I do to make using films a more useful learning experience for a student like you?
I need your help to decide which handouts are most useful to you.
1. The handout that was most useful to me was . . .
2. The reason this handout was so useful was . . .
3. The handout that was least useful to me was . . .
4. The reason this handout was least useful was . . .
Teacher Feedback Have students answer the following questions anonymously.
1. One thing that you (the instructor) do that helps my learning is . . .
2. One thing that you do that interferes with my learning is . . .
3. A practical suggestion I have that will improve the class for me is . . .
Updating Curriculum to Support Learning
DCCC Perkins Grant
III - 10

?
________________________________________
?
________________________________________
?
________________________________________
Course:
___________________________________________________________________________
Name of
CAT
Purpose
CAT seeks to get information or to answer the following:
Results Ways to Improve
Instruction
Updating Curriculum to Support Learning
DCCC Perkins Grant
III - 11

Using Rubrics for Direct Assessment of Student Work
What are the parts of a rubric?
Task Description: ______________________
Scale
Level I
Scale
Level 2
Dimension 1
Scale
Level 3
Dimension 2
Dimension 3
Descriptions of
Dimensions
1.
A task description (the assignment)
2.
A scale of some sort (levels of achievement, possibly in the form of grades). Scales typically range from 3 to 5 levels.
3.
The dimensions of the assignment (a breakdown of the skills/knowledge involved in the assignment)
4.
Descriptions of what constitutes each level of performance (specific feedback)
Four Formats for
Rubrics
1. Checklist
2. Rating Scale
3. Holistic Rating Scale
4. Descriptive Rubric
Updating Curriculum to Support Learning
DCCC Perkins Grant
III - 12

__________________________________________ Rubric
Task Description: _________________________________________
Scales
Dimensions
Updating Curriculum to Support Learning
DCCC Perkins Grant
III - 13

Select an SLO from your course syllabus and describe how you would teach to that outcome and how you would assess it.
Ways I can teach the outcome
Ways I can assess or measure the outcome.
Updating Curriculum to Support Learning
DCCC Perkins Grant
III - 14

Student will experience more success in your classes if they know up front what is expected of them.
Levels of Learning
Bloom’s Revised Taxonomy
Action Verbs
Create
Putting together ideas to develop an original idea.
Action Verbs: assemble, compose, create, construct, design, develop, formulate, imagine, invent, plan, write
Example:
1.
Plan appropriate science activities for young children.
2.
Construct a basic web page using HTML.
Evaluate
Judging the value of ideas, materials and methods by developing and applying standards and criteria
Action Verbs: appraise, argue, assess, critique, defend, evaluate judge, measure, predict, prioritize, rate, select, support, validate, value
Examples:
1.
Select the project that reflects use of quality animation and sound.
2.
Defend the use of use of the Lock-out/Tag-Out system.
Analyze
Breaking information down into its parts
Action Verbs: analyze, compare, contrast, criticize, debate, detect, differentiate, discriminate, distinguish, examine, experiment, inquire, inspect, investigate, question, separate, test, troubleshoot
Examples:
1.
Troubleshoot a piece of equipment by using logical deduction.
2.
Inspect equipment’s safety features before start up.
Apply
Using strategies, concepts, principles and theories in new situations
Understand
Explaining the meaning of information
Remember
Recalling or recognizing specific information
Action Verbs: apply, calculate, choose, compute, demonstrate, determine, develops, dramatize, draw, employ, illustrate, interpret, manipulate, practice, operate, schedule, show, sketch, solve, use.
Examples:
1.
Choose the appropriate color for a clients hair texture.
2.
Operate tools associated with the HVAC trade.
Action Verbs: classify, describe, discuss, explain, identify, locate, recognize, report, restate, translate, paraphrase
Examples:
1.
Describe the parts of a computer.
2.
Translate passages from your Spanish text into English.
Action Verbs: define, duplicate, label, list, match, memorize, outline, quote, recall, recite, repeat, reproduce, show, state,
Examples:
1.
Outline the steps in giving CPR.
2.
Label the diagram of the respiratory system.
Source: Krathwohl, D. R., Bloom, B. S., & Masia, B. B. (1973). Taxonomy of Educational Objectives, the Classification of
Educational Goals. Handbook II: Affective Domain . New York: David McKay Co., Inc.
Updating Curriculum to Support Learning
DCCC Perkins Grant
III - 15

 _____________________________________________
 _____________________________________________
 _____________________________________________
 _____________________________________________
Updating Curriculum to Support Learning
DCCC Perkins Grant
III - 16

 More answer options reduce the change of guessing
 Measures knowledge, application and analysis goals
 Can cover a wide sampling of content or outcomes
 Student have to spend more time reading more answers so reduces the number of questions to be presented
 Difficult to write 4 or 5 reasonable choices
 Takes more time to write them.
 Can present many items
 Easy to score
 Used to assess misconceptions, cause-effect reactions
 Most difficult question to write objectively
 Ambiguous terms can confuse
 Increases chance of guessing
 Efficient
 Used to assess student understanding of associations, relationships and definitions

 Encourages more appropriate

Measures higher order outcomes study habits
Easier to construct
 Measures higher order outcomes
 Focus on ability of students to apply skills or knowledge in real life situations.
 Allows for student creativity
 Focuses on student application of knowledge
 Difficult to assess higher-order outcomes.
 Takes more grading time
 Not efficient to test large amounts of content
 If you give the choice of 3 or more essay options, you can find out what they know, but not what they don’t know.



Labor and time-intensive
Tests student individually and are not for testing groups.
Low reliability
 Time consuming to write
 Answers may vary
 Low reliability
Updating Curriculum to Support Learning
DCCC Perkins Grant
III - 17

Check the hint(s) you need to adopt.
Check
Eliminate excessive wording or irrelevant information from the stem.
Include in the stem any word(s) that might have otherwise been repeated in each alternative.
State the stem as a direct question rather than as an incomplete statement
Make all alternatives plausible and attractive to the less knowledgeable or skillful student.
Make sure there was only one correct or best response per item.
Make alternatives approximately equal in length.
Present alternatives in some logical order (e.g., chronologically, most to least).
Randomly distribute the correct response among the alternative positions throughout the test having approximately the same proportion of alternatives a, b, c, d, and e as the correct response.
Use at least four alternatives for each item. Make the alternatives grammatically parallel with each other, and consistent with the stem
Use negatively stated stems sparingly. When used, underlined and/or capitalized the negative word(s).
Use the alternatives "none of the above" and "all of the above" sparingly. When used, such alternatives were occasionally the correct response.
Source:
How to Write Better Tests http://www.indiana.edu/~best/write_better_tests.shtml
Updating Curriculum to Support Learning
DCCC Perkins Grant
III - 18

Check
Avoid grammatical or other clues to the correct response
Explain whether or not a response could be used more than once and indicate where to write the answer.
Include directions which clearly states the basis for matching the stimuli with the response.
Include more responses than stimuli.
Keep the lists of items short and put the brief responses on the right
When possible, arrange the list of responses in some systematic order (e.g., chronologically, alphabetically).
When possible, reduced the amount of reading time by including only short phrases or single words in the response list.
Ask questions that will elicit responses on which experts could agree that one answer is better than another.
Avoid giving the student a choice among optional items as this greatly reduces the reliability of the test.
Indicate for each item a point value or weight and an estimated time limit for answering.
It is generally recommended for classroom examinations to administer several shortanswer items rather than only one or two extended-response items.
Prepare essay items that elicit the type of behavior you want to measure.
Phrase each item so that the student's task is clearly indicated.
Use the verbs from Bloom’s Taxonomy to write essay questions to test different levels of thinking.
Source:
How to Write Better Tests http://www.indiana.edu/~best/write_better_tests.shtml
Updating Curriculum to Support Learning
DCCC Perkins Grant
III - 19

3.
4.
1.
2.
Date:
____________
# Items:
__________
Outcomes
Levels of Thinking
Remember Understand Apply Analyze Evaluate Create Total/
%
Total/%
Instructions:
1 st : Analyze the test you have written by examining each question (item) and determine at which level of critical thinking this is written. 2 nd Determine which Student Learning outcome each question measures. Write the number of the question in the box that coincides with the correct SLO tested and the level of Critical Thinking. Once all items have been checked add the total items for each outcome on the vertical total line, as well as for each level of critical thinking on the horizontal total line. The percentages under the vertical column will indicate how much of the test is written with higher level thinking in mind. You can also determine which SLOs need more questions to adequately measure them.
Updating Curriculum to Support Learning
DCCC Perkins Grant
III - 20

4.
5.
2.
3.
Write the SLOs in the first column on the left. In the center column indicate what you do to teach to that SLO and in the last column on the right indicate how you assess the outcome.
SLO Teaching Approach Assessment
1.
Updating Curriculum to Support Learning
DCCC Perkins Grant
III - 21

1.
Course in which I wish to make changes:
______________________________________________
2.
Effective assessment principles I will adopt:
______________________________________________
______________________________________________
______________________________________________
______________________________________________
3.
Strategies I plan to use:
______________________________________________
______________________________________________
______________________________________________
______________________________________________
Updating Curriculum to Support Learning
DCCC Perkins Grant
“I never teach my pupils; I only attempt to provide the conditions in which they can learn.”
Albert Einstein
III - 22

 One thing I will take from the workshop that I can apply is …
___________________________________________________________
___________________________________________________________
___________________________________________________________
 I would have learned better if you would have…
___________________________________________________________
___________________________________________________________
___________________________________________________________
 The handout materials were…
___________________________________________________________
___________________________________________________________
___________________________________________________________
 Using the scale of 1 to 5, please place a check by the number where would you rate how you felt your expectations were met for this session.
1
Did not meet my expectations
2
It was ok
3
I learned some things
4
It met my expectations
5
It met and exceeded my expectations
Updating Curriculum to Support Learning
DCCC Perkins Grant
III - 23