Module
5
Assessment of Student Learning
Module 5
ASSESSING DEEP UNDERSTANDING AND REASONING SKILLS
Objectives

Gain understanding of the different frameworks to organize and label
reasoning skills

Differentiate the different methods of assessing deep understanding and
reasoning

Explain how objective tests can be used to assess reasoning skills

Discuss the specific guidelines in constructing interpretative exercises and
essay items.

Construct sample items for interpretative and essay items
Introduction
This lesson will examine the assessment of deep understanding and reasoninghow students use their knowledge for more complex thinking. There are different
ways to conceptualize skills such as thinking and reasoning – the interpretative
exercises and the essay questions. In this lesson, you can explore on the utility of
Page 1 of 22
Module
5
Assessment of Student Learning
objective test like multiple choice, not only as way to assess knowledge and
simple understanding, but of deep understanding and reasoning as well.
Reasoning skills
Reasoning involves some kind of mental manipulation. The task is to
employ knowledge to interpret and draw inferences, solve a problem, make
judgment or decision, or engage in creative or critical thinking. Reasoning has
three components: the mental skill needed to perform the task, the declarative or
procedural knowledge or understanding needed and the task itself. As shown in
Figure 1, mental skills, knowledge and simple understanding and the task could
be in variety of forms.
Page 2 of 22
Module
5
Assessment of Student Learning
Figure 1. Major components of reasoning
Assessing reasoning skills is challenging because the target is difficult to
define. It is one thing to note the importance of teaching and testing higher-order
thinking skills or reasoning skills, but operationalizing these general ideas into
specific learning targets is far from straightforward. There are many frameworks
that have been developed to represent different ways to organize, label and define
thinking skills, and these are summarized in Table 1.
Page 3 of 22
Module
5
Assessment of Student Learning
Table 1. Reasoning Skill Frameworks
Framework
Bloom’s
Taxonomy
Definition
Higher-order thinking skills
Components
Application (apply
situations)
Analysis
consistency)
in
novel
(distinguishing
Evaluation (judgment of quality or
worth of something)
Ennis’
Critical
Thinking
Synthesis (combining elements)
Decision making or judgment Dispositions (clarify problems,
about the merits or worth of a gather information,
make
belief or action
inferences, conduct advanced
clarification, make judgments
Quellmalz
Cognitive skills
and Hoskyn’s
Framework of
Reasoning
Strategies
Skills
(
detecting
bias,
inconsistencies,
illogical
arguments)
Analysis (identify components)
Comparison ( contrasts, relates
similarities and differences)
Inference
and
interpretation
(deductive, inductive thinking)
Marzano’s
Dimensions
of learning
Complex
thinking
reasoning strategies
Evaluation (judgment)
or Extending and refining knowledge
(comparing,
classifying,
inducting, deducting, analyzing
error, abstracting)
Using knowledge meaningfully (
decision making, investigation,
problem solving, experimental
inquiry, invention)
Page 4 of 22
Module
5
Assessment of Student Learning
ASSESSING REASONING AND DEEP UNDERSTANDING
It was clear in the last lesson that each of the assessment methods can be
used to measure any learning target. Reasoning can be measured by selectedresponse items, and recall knowledge can be evaluated in student essays.
However, some methods are better than other in assessing particular types of
targets. In this lesson, it presents how selected-response and short answer items
discussed in the previous lesson work for assessing deep understanding and
reasoning.
Then it focuses on the construction of two paper-and-pencil test
method that are better suited for assessing reasoning- namely, interpretative
exercises and essays.
Short –answer and Selected-response Items
Short-answer items can assess thinking skills when students are required
to supply a brief response to a question or situation that can be understood only by
the use of the targeted learning skills. Reasoning tasks, like decision making and
critical thinking, however are not assessed very well with short-answer items.
Examples
(Comparing)
How does a pine tree differ from an oak tree?
Name one difference between vertebrate and invertebrate animal
Page 5 of 22
Module
5
Assessment of Student Learning
(Deductive reasoning)
Coach Mike substitutes his basketball players by height, so that the first
substitute is the tallest player on the bench, the next substitute is the next tallest,
and so forth. Reginald is taller than Sam, and Juan is taller than Reginald. Which
of these players should Coach Mike play first?
(Analysis/prediction)
The principal needs to decide if the new block schedule allows teachers to
go into topics in greater detail. He can ask a parent, teacher, or a principal from
another school. Who should he ask to get the most objective answer?
(Investigating)
Several paper towel companies claim that their products absorb more
liquid that the other brands. Design an experiment to test absorbency of each
brand of paper towel
(Analysis)
List the anatomical structure s of the kidney, explain the function of each
part, and describe how they all work together
Binary-choice
This item can be used to assess reasoning skills in several different ways.
1. Students can be asked to indicate whether a statement is a fact or
opinion.
Example: If the statement is fact, circle F; if it is an opinion, circle O.
F
O
Literature is ancient Rome’s most important legacy
F
O
Earth is a very beautiful planet.
Page 6 of 22
Module
5
Assessment of Student Learning
F
F
The best way to wash a car is with a sponge.
2. Logic can also be assessed by asking if one statement follows logically
from another using binary-choice item
Example: If the second part of the sentence explains why the first sentence is
true, circle T for true; if it does not explain why the first part is true, circle F for
false.
T
T
T
F
F
F
Food is essential because it tastes good.
Plants are essential because they provide oxygen.
Reggie is taller because he has blue eyes.
Multiple Choice
Simple multiple choice items can also be used for assessing reasoning in
two ways. One is to focus on a particular skill and the other is to assess the extent
to which the students can use their knowledge and skills in performing a problem
solving or other reasoning tasks.
Examples: ( Focusing on a particular task)
(Distinguishing fact from opinion)
Which of the following statement about our solar system is a fact rather
than an opinion?
a. The moon is made of attractive white soil.
b. Stars can be grouped into important clusters.
c. A star is formed from a white dwarf.
d. Optical telescopes provide the best way to study the stars
(Identifying assumptions)
Page 7 of 22
Module
5
Assessment of Student Learning
When Patrick Henry said “give me liberty or give me death,” his
assumption was that:
a. Everyone would agree with him
b. Thomas Jefferson would be impressed by the speech
c. If he couldn’t have freedom he might die as well
d. His words would be taught to students for years
(Comparison)
One way in which insects are different from centipedes is that:
a. They are different colors
b. One is an arthropod
c. Centipedes have more legs
d. Insects have two body parts
(Analysis)
Roy decided to go sailing with a friend. He took supplies with him so
he could eat, repair anything that might be broken, and find where on the
lake he could sail. Which of the following supplies would best meet his
needs?
a. Bread, hammer, map
b. Milk, bread, screwdriver
c. Map, hammer, pliers, screwdrivers
d. Screwdriver, hammer, pliers
(Synthesis)
What is the main idea in the following paragraph?
Ann picked a pretty blue boat for her first sail. It took her about an
hour to understand all the parts of the boat and another hour to get the sail
on. Her first sail was on a beautiful summer day. She tried to go fast but
couldn’t. After several lessons she was able to make her boat go fast.
a. Sailing is fun
b. Ann’s first sail
c. Sailing is difficult
d. Going fast on a sailboat
Examples: ( Ability to perform a reasoning task)
Page 8 of 22
Module
5
Assessment of Student Learning
(Hypothesizing)
If there were a significant increase in the number of hawks in given area,
a. The number of plants would increase
b. The number of mice would increase
c. There would be fewer hawk nests
d. The number of mice would decrease
(Problem-solving)
Farmers want to be able to make money for the crops they grow, but
too many farmers are growing too may crops. What can the farmers do to
make more money?
a. Try to convince the public to pay higher prices
b. Agree to produce fewer crops
c. Reduce the number of farmers
d. Work on legislation to turn farmlands into parks
(Critical thinking)
Pablo is deciding which car to buy. He is impressed with the sales
representative for the Toyota, and he likes the color of the Mitsubishi.
The Toyota is smaller and gets more kilometers to the gallon. The
Mitsubishi takes larger tires and has a smaller trunk. More people can ride
in the Toyota. Which car should Pablo purchase if he wants to do
everything he can to ensure that his favorite lake does not become
polluted?
a. Toyota
b. Mitsubishi
c. Either car
d. Can’t decide from the information provided
(Predicting)
Suppose that Central Luzon, which grows most of the country’s rice,
suffered a drought for several years and produced much less rice than
usual. What could happen to the price of the rice?
a. The price would rise
b. The price would fall
c. The price would stay the same
d. People would eat less rice.
Page 9 of 22
Module
5
Assessment of Student Learning
Interpretative Exercises
The best type of short-answer or selected-response item for assessing reasoning
skills is usually the interpretative exercises. This type of test consists of some
information or data, followed by several questions, which are based on the information or
data, which can take the form of maps, paragraphs, charts, a story, a table, or pictures.
Strengths:
1. It is possible to measure more reasoning skills in greater depth because there
are many questions about the same information.
2. It is possible to separate the assessment of the reasoning skills from content
knowledge of the subject.
3. It is relatively easy to use materials that students will encounter in everyday
living, such as maps, newspaper articles, and graphs.
4. The results are more reliable because it provides a standard structure for all
students and are scored objectively
Limitations:
1.
It is time consuming and difficult to write.
2. Unable to assess how students organize their thoughts and ideas
3. Most items rely heavily on reading comprehension
Page 10 of 22
Module
5
Assessment of Student Learning
Guidelines in constructing interpretative exercise
1.
Identify the reasoning skills to be assessed . The sequence you use is
important because you want the exercise fit your learning targets, not have
learning targets determined by the interpretative exercise.
2. Keep introductory material as brief as possible. This will minimize the
influence of general reading ability, and students can complete the reasoning.
3. Select similar but new introductory material. If you use the same material in
the class, you will measure rote memory rather than reasoning. The material
should vary slightly in form or content, but it should not be completely new.
4. Construct several test items for each exercise. The test items can be shortanswer, multiple choice, or binary-choice. This will obtain a better sample of
the proficiency of students’ reasoning skills.
5. Construct items so that the answers are not found in the question. You do not
want to use questions that can be answered without even reading the
introductory material.
Example1: Interpretative exercise (recognizing the relevance of the information)
Joy lost her pencil on her way to school. It was red and given to her by
her grandmother. She wanted the teacher to ask the class if anyone found the
pencil.
Key:
Circle Yes if the information in the sentence will help the class find the pencil.
Circle No if the information in the sentence will not help the class find the pencil.
Page 11 of 22
Module
5
Assessment of Student Learning
Yes
No
1. The pencil was new.
Yes
No
2. Sally rides the bus to school.
Yes
No
3. The pencil is red.
Yes
No
4. The pencil was a present from Joy’s grandmother.
Yes
No
5. The pencil had a new eraser.
Example 2. Interpretative exercise (analysis, inference, error analysis)
Figure 1. Number of Elementary, High School and College Students Graduating from
Region x
Based on Figure 1. Circle T if the statement is true and F if the statement is false.
T
T
F
F
1. In 1990, there are more college graduates than high school graduates.
2. From 1992 to 1993, the number of elementary graduates decreased.
T
F
3. Overall, there were more elementary graduates than high school
graduates.
Answer each of the following questions:
In what year the difference between high school and college graduates was lesser?
___________
Page 12 of 22
Module
5
Assessment of Student Learning
Example 3. Interpretative exercise (inference, prediction)
Table 1. Students’ score in Addition and Subtraction
Addition
Quiz 1
18
10
9
16
Carlo
Kate
Jane
Fely
Subtraction
Quiz 2
16
10
8
15
Quiz 1
19
18
14
15
Quiz 2
20
19
15
16
Study the table and answer the following questions:
1. What inference can you make about the average scores of the students in Quiz 2?
2. If the other students’ score have the same pattern just like the other in the table,
predict the reliability of addition and subtraction scores
Essay items
The essay question is especially useful for measuring ability to organize,
integrate, and express ideas. It provides freedom of response. It also requires the
students to interpret information, give arguments and explanations, evaluate the merit of
the idea, and conduct other types of reasoning, thus it is an excellent way to measure deep
understanding and mastery of complex information.
Strengths:
1.
The highest level of understanding, complex thinking and reasoning skills can
be assessed.
2. Preparation is less than for selection-type of test.
3. The integration and application of ideas is emphasized.
Page 13 of 22
Module
5
Assessment of Student Learning
4. It motivates better study habits and provides students flexibility in how to
respond.
5. It discourages rote learning and guessing.
Limitations:
1.
Reading and scoring is very time-consuming, highly subjective and
notoriously unreliable.
2. There is inadequate sampling of achievement due to time needed for
answering the questions.
3. It is difficult to relate to intended learning outcomes because of freedom to
select, organize and express ideas.
4. Scores are raised by writing skills and bluffing, and lowered by poor
handwriting, misspelling and grammatical errors
Types of Essay Questions
1. Restricted-Response Questions. It places strict limits on the answer to be given;
the boundaries of the subject matter to be considered are usually narrowly defined
by the problem.
Examples:
Why are tomatoes better for your health than potato chips?
What is the effect of inflation of raising the prime interest rate?
Page 14 of 22
Module
5
Assessment of Student Learning
Describe the relative merits of selection-type test items and essay questions for
measuring learning outcomes at the comprehension level. Confine your answer to
one page.
2. Extended-Response Questions. This type gives the students almost unlimited
freedom to determine the form and scope of their responses. The students must
be given sufficient freedom to demonstrate skills of synthesis and evaluation, and
just enough control to assure that the intended intellectual skills will be called
forth by the question.
Examples:
1. Explain how the fertilizers farmers use to grow crops may pollute our river
and streams.
2. Describe the major events that led to People Power Revolution in 1986.
3. Give an example, new to me and not one from class, of how the law of supply
and demand would make prices of some products increase.
4. Write a critical evaluation of this test using the rules and standards for test
constructions described in the textbook. Include a detailed analysis of the
test’s strengths and weaknesses and an overall evaluation of its overall quality.
5. In teaching a particular lesson, prepare a complete plan for evaluating student
achievement. Be sure to include the procedures you would follow, the
instruments you would use, and the reason for your choices.
Page 15 of 22
Module
5
Assessment of Student Learning
Guidelines in constructing essay questions
1.
Construct the item to elicit skills identified in the learning target. A good way to
begin writing the item to match the target is to start with a standard stem. Then
modify it as needed for the subject and level of student ability. Examples are
shown in the table below.
Skills
Comparing
Relating Cause
and Effect
Justifying
Summarizing
Generalizing
Inferring
Classifying
Creating
Applying
Analyzing
Synthesizing
Evaluating
2.
Stem
Describe the similarities and differences between…..
Compare the following two methods of ….
What are the major causes of …?
What would be most likely the effects of …?
Which of the following alternatives do you favor and why?
Explain why you agree or disagree with the following statement
State the main points included in…..
Briefly summarize the contents of…
Formulate several valid generalizations from the following data.
State a set of principle that can explain the following events.
In light of the facts presented, what is most likely to happen when..?
How would Senator X be likely to react to the following issues?
Group the following items according to…
What do the following items have in common?
List as many ways as you can think of for….
Make up a story describing what would happen if…
Using the principle of …. as a guide, describe how to solve the
problem
Describe a situation that illustrates the principle of…
Describe the reasoning errors in the following paragraph.
List and describe the main characteristics of…
Describe a plan for providing that…
Write a well-organized report that shows….
Describe the strengths and weaknesses of …
Using the given criteria, write an evaluation of….
Write the item so that the students clearly understand the specific task. If the
students will need to interpret what is asked, many answers will be off target.
Page 16 of 22
Module
5
Assessment of Student Learning
When students misinterpreted the task, you don’t know if they have the targeted
skills or not, leading to invalid conclusions.
3.
Indicate the criteria for scoring their responses. This can be labeled as scoring
plan, scoring criteria, or attributes to be scored.
Examples:
(For Scoring Writing skills)




Organization
Clarity
Appropriateness to audience
Mechanics
(For Scoring an Argument)





Distinguishing between fact and opinion
Judging credibility of a source
Identifying relevant material
Recognizing inconsistencies
Using logic
(For Scoring Decision Making)




Identifying goals or purpose
Identify obstacles
Identifying and evaluating alternatives
Justifying the choice of one alternative
4. Indicate approximately how much time students should spend on each essay-item.
You can get idea by writing draft answers, and as you gain more experience the
responses of previous students to similar questions will be helpful. Make sure
Page 17 of 22
Module
5
Assessment of Student Learning
that even the slowest writers can complete their answers satisfactorily in the time
available.
5.
Avoid giving students options as to which essay questions they will answer.
When doing this, each student may be taking a different test. Differences in the
difficulty of each question are unknown, thus making scoring problematic.
Guidelines for scoring responses in an essay item
1. Outline what constitutes a good or acceptable answer as a scoring key. It is better
to have the points specified before reading student answers so that you are not
unduly influenced by initial papers you already read.
2. Select an appropriate scoring method. Scoring could either be holistic – overall
judgment about the answer, giving it a single grade or score or analytic – giving
each of the identified criteria separate points. Analytic scoring is preferred for
restricted response questions; however, it can be time-consuming.
3. Clarify the role of writing mechanics.
Decide in advance whether spelling,
grammar and other criteria will be included as factors in evaluating responses.
This certainly will influence your overall impression of an answer.
4. Evaluate all of the students’ answer to ne question before proceeding to the next
question. Scoring essays by questions rather by students can maintain a more
uniform standard judging the answer and helps reduce the halo effect –the
Page 18 of 22
Module
5
Assessment of Student Learning
grader’s impression of the paper as a whole is apt to influence the grades assigned
to the individual answers.
5. Score the answers anonymously. This will reduce if not eliminate the bias during
scoring. This can be done by having the students write their names on the back of
the paper or by using code numbers.
6. Whenever possible, have two or more persons grade each answer.
Obtain two
independent judgments, especially where the results are to be used for important
and irreversible decisions.
Page 19 of 22
Module
5
Assessment of Student Learning
Activity 1
A. 1. Identify the thinking or reasoning skill illustrated by each of the following
examples using the following choices:
a. Analysis
b. Synthesis
c. Critical thinking
d. Decision making
e. Problem solving
f. Inference
g. Evaluation
__1. Suppose you were President Arroyo and had decided not to give Former
President Estrada the pardon. Why would you not give him pardon? What would
you do instead?
__2. State your reasons for agreeing or disagreeing with the following
statements: Religious people are more likely to help others.
__3. Given what you know about sailing, what would most likely occur if a
novice sailor tried to sail directly into the wind?
__4. Examine three different human cultures. What is common in all three
cultures , and what principle about being human does it suggest?
__5. Examine four recent presidential speeches. Is any part of the speeches the
same?
__6. How can the Philippines reduce the rate of child labor?
__7. Suppose you had to choose between increasing taxes to reduce the budget
deficit or decrease government spending to reduce the deficit. Which would you
choose? Why? How would your choice affect retired persons?
__8. Examine the data on birth rates. What is likely to happen to the birth rate by
the year 2010? Why?
__9. Which type of test is best for measuring deep understanding and reasoning?
Why?
__10. What will most likely to happen if the peso continues to go weak against
the dollar?
Page 20 of 22
Module
5
Assessment of Student Learning
A.2. Indicate whether each of the following would be best measured by an
objective item (O), an interpretative exercise (I), or an essay question (E).
__1. Discerning the meaning of a series of pictures
__2. Asking students about the validity of an argument used in a debate
tournament
__3. Analyzing a passage to identify irrelevant information and opinions
__4. Being able to construct a logical argument
__5. Kowing the sequence of steps involved in problem solving
__6. Giving examples of the relationship “commensalism”
__7. Being able to distinguish critical thinking from decision making
__8. Determining whether Michelangelo would be regarded as a great artist if he
lived today and, if so, why
__9. Identifying several valid generalizations from the data presented
__10. Make conclusions on the relationship of two variables in a given chart.
B. Answer the question in at most two paragraphs.
How does essay and interpretive exercise differ with objective tests in as
far as measuring deep understanding and reasoning is concerned?
Page 21 of 22
Module
5
Assessment of Student Learning
Activity 2.
1.
Using the contents of this lesson, devise some reasoning learning targets .
Then construct three objective test items (1 short-answer,1 binary-choice,1
multiple choice) and two essay items ( 1 restricted-restricted, 1 extendedresponse). Include test directions and key to correction. For the essay,
include scoring guidelines.
2.
Look for any map, graph, table, chart , a short story or any stimulus. Form
this, construct 5 interpretive exercise items (1 recognizing relevance of
information , 1 error analysis, 1 analysis,1 inference, and 1 prediction item).
Label the items properly as to what reasoning skills they are assessing.
Include key to correction
Activity 3
Share insights gained on the following:
a.
Reasoning skills as a quantifiable attribute
b. Ease in preparing but difficulty in scoring essay items
Page 22 of 22