Critical evaluation of
research information
Laura Jeffrey
Researcher Training
Course outline
• Importance of evaluation
• Forms of value
– Subjective
• Cognitive bias
– Objective
– Intersubjective
• Relationship between the forms of value
The need to evaluate information
• Much training is about directing you to the
right information = searching and retrieval
• One of the distinctive elements of
postgraduate research is that you have to be
critical and reflect on what you find
• What defines your evaluative criteria?
Ecology of resources
• Term developed by Rosemary Luckin
• Resources are interconnected and they evolve
e.g. natural resources
• Information resources are transformed into
knowledge
• Knowledge becomes a resource
• Therefore prior knowledge shapes
what we go on to create
Role of the researcher
• In theory we can select almost any
information to complete a task
• In practice we filter it by selecting resources
we think most appropriate
• Motivation - affected by the learning we have
already done
Other factors
• But, filtering is done for us BEFORE we get the
chance to make a judgement
• People
• Technologies
• Cost
• Skills
• Copyright, IP
Value
• Filtering process = value judgement
– By researcher
– Made on their behalf
• What forms of value are there and how do
they work together to create information
literate researcher?
Subjective form of value
• Decisions we make
– Is this what I want, do I need this, is it relevant?
• Privileges you as the researcher in the
decision making process
• If we omit it we get groupthink (Janis, 1972)
or battery cognition (Blaug, 2007)
• Importance therefore of asserting individual
criticality
Cognitive bias
• SO… subjectivity is vulnerable to bias and
hunches
• Concept of cognitive bias was developed in
1970s by Tversky and Kahneman
• Four main groups
– Social
– Memory
– Probability/belief
– Decision making
Social biases
• Ascribe positive or negative traits to self,
individuals or groups
• Loading values or anticipating action based on
prior experience or a bias against self,
individuals or groups
• Academic impact: need to verify information
and not rely on own views;
important to remember when
analysing human subjects
Memory biases
• How you perceive past events
• False memory, positive memory, imbalanced
memory
• E.g. A Photo, a Suggestion, a False Memory
• Academic impact: importance of
accurate record keeping and
note taking
Probability and belief
• To disregard or to pay too much attention to
probability
• Academic impact: need to treat each research
finding as distinct and to judge it in its own
right
Decision making biases
• Influences on your decisions by own biases or
those of a group
• Academic impact: need to be objective and
consider all possible routes of enquiry and
treat all research findings as valid until proved
otherwise e.g. Semmelweis reflex
Cognitive biases
• On your table, group the forty cards into four
piles of ten
– Social
– Memory
– Probability
– Decision
Objective form of value
• Scientific measures of validity or reliability
• Exists so that subjective values don’t unduly
influence work
• Omit this scheme of value and we risk
information (and knowledge formed from it)
becoming counterknowledge (Thompson,
2008).
Intersubjective form of value
• Based on the shared values of a community
e.g. morals, ethics, laws, economics
• Allows for discussion of scientific method as it
can’t explain everything
• Acceptance in a community
• If we omit this then values are relativist
Three forms of value
Subjective
Objective
Intersubjective
Summary of relationship of values
• Awareness of benefits of all three values
• Awareness of drawbacks of each individual
value
• Experience variation in your engagement with
information
• Understand when different forms of value
should be prioritised or used in conjunction
• Application to the research process