IEN Inaugural Annual Conference
September 24th 2004
Evaluation: A tool for
meaning-making in a Higher
Education Quality Assurance
culture
Content
1.
2.
3.
4.
5.
Conceptions about how mind works
The OTC ‘culture’ as a context for the
Quality Assurance framework
The OTC Quality Assurance framework
Evaluation as a tool for ‘meaning making’
within the OTC culture
How ‘meanings’ are communicated as
catalysts for both individual and collective
development and progression
Conceptions about how mind works
Bruner (1995)

Computational View


Information processing
Culturalism


Meaning-making
’ Situated’ learning and thinking
‘Education is not simply a technical
business of well managed information
processing…..It is a complex pursuit of
fitting a culture to the needs of its
members and of fitting its members and
their ways of knowing to the needs of
the culture’
Bruner 1986:43
Guiding Principles
Vision and Mission
O.T.C. QA Framework
1. Quality Control
2. Quality Assurance
3. Continuous Quality
Improvement
Ethos
Pedagogical Practice
Guiding principles






Inclusiveness
Openness
Relevance
Student centeredness
Accountability
Accessibility
College Vision and Mission
‘The Open Training College will be
recognized for the major role it plays in
the advancement of the inclusion of
people with disabilities in Irish Life; the
excellence of our training; our
commitment to our students; the
quality of our relationships with
agencies;…………………………….
Mission/vision statement cont.
our advancement of adult lifelonglearning opportunities and our
contribution to change and best
practice at the level of the service-user,
the staff member and the agency in the
disability sector’.
College ethos and values






Enhancing the status of people with a
disability
Fundamental principles
Valuing students as colleagues
Close working relationships
Cost effectiveness and value for money
Strategic planning
Pedagogical practice



Shared understanding of the adult
learning process
Courses designed to provide and
stimulate diverse learning experiences,
building of the skills and knowledge
that our learners bring to their learning
Clearly articulated learning outcomes
and related assessment procedures
The Open Training College Quality
Assurance framework
Please see handout
Evaluation process model for
course evaluation
Please see handout
How do we evaluate?
‘Responsive Evaluation’


To covey a message rather than specific
data
Stake (1975)
‘Evaluation is an OBSERVED VALUE
compared to some STANDARD’
Scriven (1967)
OTC standards
Represented in the:
 Guiding principles
 Vision and Mission statement
 Ethos
 Pedagogical Practice
Interpreting the evaluation summaries

The basic task of an evaluator is made barely
tolerable by the fact that he/she does not
have to solve this equation in some numerical
way nor to obtain a descriptive summary
grade but needs merely to make a
comprehensive statement of what the
programme is observed to be, with useful
references to the satisfaction and
dissatisfaction that appropriately selected
people feel toward it’
Stake (1975)
Making meanings

Meaning making involves situating encounters
with the world in their appropriate cultural
contexts in order to know ‘what they are about’.
Although meanings are ‘in the mind’ they have
their origins and their significance in the culture
in which they are created. It is this cultural
situatedness of meanings that assures their
negotiability and ultimately, their
communicability’.
(1986:3)
Bruner
How meanings are
communicated
1.
2.
3.
4.
Evaluation summaries (QA coordinator)
Meaning-making within and as a result of
the shared micro- culture of the OTC
(tutors)
Feedback to Programme Boards through
Annual Course Review Reports (tutors)
Recommendations made to Academic
Council (Programme Boards)
5.
6.
7.
Ratification of improvements and
developments (Academic Council)
Implementation of the same (OTC
team)
Dissemination to all stakeholders (QA
coordinator)
Summary


We make meanings of evaluation data
by filtering our interpretations through
our minds in order to measure findings
against specified standards
Macro and micro cultures shape mind
and provide us with a toolkit by which
we construct and communicate
meanings

If meanings from evaluations
procedures are to be catalysts for
development and progression;
evaluation procedures must be
embedded within a clearly defined
shared organizational culture to which
its members are committed.