The European Quality Observatory
How to find the Most Suitable Quality Strategy?
Dr. Ulf-Daniel Ehlers
University of Duisburg-Essen/
European Foundation for Quality in E-Learning
Abstract
The role of education for todays societies is seen as crucial in order to
participate in the global concert of economies, provide prosperity and
well being to citizens, improve conditions of life and reduce poverty as
well as build stable societal structures. The quest for quality, i.e. the very
nature of this education, is thus the central question in all debates in all
educational sectors today – especially for countries on the rise, trying to
provide higher education to strengthen the path into a better future
development. In Europe this quality debate is more and more on stake,
defining quality as one of the main policy objectives for education with
the aim of developing societies in such a way that individuals can
advance according to their abilities, and economies can prosper.
1. Quality as a Educational Leitmotiv
There is no doubt that quality is the most decisive factor determining the
future of any higher education system. This is the reason for the great
variety of concepts, suggestions and debates which now encompass a
large section of society and effect many social sub segments. It shows
that the question of quality touches the heart of the educational debate.
We can regard quality more and more as a subjectively individual and
collectively influential category. How should learning opportunities look
like and learning environments be structured, now and in the future?
How do we meet the demand for building high quality learning capacities
in higher education which are an important corner stone to transform our
societies into learning societies?
The concept of quality in the public perception and debate today has
gained the significance of a leitmotiv for the educational field in all
European countries, having the same importance like “equality” or
“scientific orientation” in the educational debates of the 1970s in some
European countries. It becomes clear that the debate on quality is a
debate about how learning and education should look like in the future.
It is a debate about values and cultures and it takes place on basis of
diverse experiences and convictions.
Such concepts do not appear as empirical accurately defined and
operationalised notions but are rather constituted by a dense bundle of a
broad range of arguments, objectives, convictions and procedures
1
(Terhart, 2000, p. 809). Quality in e-learning in this sense has become a
leitmotiv for educational policies, a slogan for practitioners and a huge
demand for learners. Achieving high quality is a hotly debated and
much-sought-after goal in all segments of society and education. It is
less characterised by its precise definition but rather by its positive
connotation.
What is so difficult with quality, that everybody wants to achieve and
nobody can really define it? The very nature of quality is that it is a
multi-dimensional concept and it is not possible to generally define a set
of quality standards applicable to all countries and all educational
sectors.
Quality embraces all the main functions and activities of higher
education: teaching, research, staffing, students, infrastructure, and the
academic environment. It is the relation between the expectations and
expected outcomes and the observed results. Continuous and
permanent assessment and improvement are necessary to reach this
objective. Quality – as much as education – is rooted in cultural values
and traditions. Therefore quality strategies and definitions always have
to be specifically taking into account the very context of their application.
To find a suitable model for quality development is of crucial importance
for quality development in higher education. Accreditation sets a frame
for quality development which needs to be filled with more elaborated
macro and micro strategies. Due to the enormous variety of divers
strategies in the field of quality development it is difficult to tell which of
the available concepts fit the specific needs in the given context. After
having chosen a suitable model for quality development it is important
that this is not implemented in a mechanistic manner into an
organization but that it rather stimulates processes of pedagogical
professionalisation. The utmost goal of quality development has to be to
foster a professionalisation process of educational practitioners. It
becomes clear in recent debates that achieving quality is not only about
finding a strategy but rather about filing this strategy with life. Living the
quality ideal is thus much more important than a criteria-oriented
checklist like mechanistic quality understanding. It is about integrating
professionalisation processes of the educational actors, like teacher,
trainers and other stakeholders into strategies and reference models
which are existing already.
The task to develop or to provide a high quality educational experience
is, however, an extremely difficult challenge. The paper suggests to
bring together the two key points of quality development in higher
education: Finding a strategy for quality development and implementing
it as an ongoing professionalisation process. For this purpose three
2
developments are described: The Quality Development Cycle which
describes the quality development process from the needs analysis
stage to the stage where the new values and processes are
incorporated into the everyday work of all stakeholders. Secondly the
concept of Quality Literacy which is necessary for a continuous quality
improvement in an organisation. Thirdly a Decision Support Concept
(an internet based database) is described which can help educational
actors to find a quality strategy which fits their specific purposes.
2. The Quality Development Cycle: Competences and Processes
Modern quality development moves from input oriented approaches to a
process oriented philosophy of permanent improvement. It involves the
student not as a passive receiver any longer but as an active producer
of his/ her own learning process. To view quality development as such –
an active process of participation and negotiation – means to challenge
believes and existing values of all actors involved. The nature of quality
development is then a constant adaptation process of the offered
educational services to the target groups which are to be educated.
Newer approaches highlight this aspect already, elaborating negotiation
as important for successful quality development (c.f. Ehlers/ Fehrenbach
2004). Quality development understood in this way goes the whole way
of structuring educational activities and processes AND at the same
time aims at having an impact on the learning process. Only if this goal
is achieved quality development can be seen as education oriented
quality development – as opposed to the often implemented model of
(only) organisation oriented quality development.
3
Figure 1: Quality Development Cycle (adapted from Ehlers/
Pawlowski 2004)
This relates especially to the open nature of quality which in itself is not
a normative definition but a relation between the perceived and the
offered provision. Within this open concept of quality development, we
can identify four steps educational actors have to engage into, to
develop quality. For each of these steps which can be conceptualised as
a cycle of quality development certain competences are necessary to
perform the intended processes of analysis, selection, adaptation and so
on. We suggest therefore in this paper to bring together a process
model with the concept of quality literacy (see chapter 3) to describe the
necessary components for successful quality development. In the
context of the Quality Development Cycle, the dimensions of quality
literacy applies to the different steps of quality development (Fig. 1), as
described below.
According to the presented model (Fig.1), quality development takes
place as a sequence of four steps which involve (a) a needs analysis,
(b) a decision process, (c) a realisation phase and (d) an incorporation
phase.1
The cycle thus takes on an organisations’ perspective. This is important to note
because it is especially developed to answer the question how an educational offer
1
4
Needs Analysis: In this phase the needs for quality, the situation and
the context of the educational scenario are subject of examination. The
needs analysis phase includes in itself an iterative cycle which consists
of an analysis phase of the current situation, a negotiation processes
between the involved stakeholders (e.g. learners, teachers,
administration), and a definition phase where the needs are finally
defined.
Stakeholders who are involved in these processes need the ability to
evaluate and define the needs of all stakeholders which are involved in
the educational scenario and negotiate between them to achieve a high
quality of the offered learning environment (Quality Analysis).
Additionally Knowledge about the possibilities of quality development
and about quality strategies or good practice examples could be of help
in the needs analysis phase.
Decision Phase: In the decision phase the previously defined needs for
quality development are matched with available approaches (Quality
Knowledge is needed). If those approaches sufficiently meet the
requirements, they have to be chosen as model for the quality
development project, and the next phase can be entered. If there is no
strategy which meets the needs, a new, own quality strategy has to be
developed. For this phase two competences are especially important:
Quality Knowledge and Quality Analysis skills. When it comes to
developing an own strategy the ability of Quality Innovation, i.e.
creatively and innovatively developing a fitting quality strategy, gains
importance.
Realisation Phase: In the realisation phase the quality strategy is
implemented into the organisation and thereby adapted to the specific
organisations’ needs. The new set of rules and processes have to be
“transformed” into the organisations’ “language” and be refined for the
organisations’ specific context. This process to a large extent involves
experiences, adaptation processes, evaluation and analysis
competencies.
The usage of models and instruments for quality development like
checklists, process descriptions and/or evaluation questionnaires,
requires a high amount of Quality Experiences. The adaptation of these
instruments and models demands for the ability of innovation and
can be provided through an organisation, e.g. a university, to be of high quality. It is not
primarily concerned with helping learners, who have to choose a course or a program,
helping them to find an offer of high quality. For each phase in the quality development
cycle certain competencies are required for the actors performing the quality
development process.
5
modification and is conceptualised in the dimension of Quality
Innovation. Critical analysis and assessment form an integral part of this
phase. Quality Analysis thus becomes important.
Incorporation Phase: The incorporation phase relates to the
modification of activities and actions which have to be performed by the
individual actor of an organisation as a result of the quality development
process. Quality development – in the final consequence – is always
directed at modifying the behaviour of individual actors of an
organisation – be it the tutors or teachers or the authors of courses, the
system administrators or the organisational representatives.
In the incorporation phase it is therefore examined whether the changed
processes and new values which are suggested in a new quality
strategy are incorporated into the activity patterns of the stakeholders. A
great deal of critical analysis skills and evaluation experiences is
necessary for this phase. Quality Analysis therefore becomes important
in this phase.
3. Quality Literacy
As we have seen above, for each step of quality development certain
necessary competences can be identified. These competencies can be
referred to as quality literacy. They involve
- Knowledge about quality development for general orientation and
selection,
- Experience with the usage of instruments for quality
development,
- the ability of innovation and modification to adapt instruments and
concepts to the own situation or develop new and
- analysis abilities for assessing own needs and evaluate existing
tools and concepts.
To provide a quality enhanced environment in the above described
sense, quality literate actors are necessary. The concept of quality
literacy (fig. 2) aims at describing skills which enable individuals in the
situation of quality development to act competently. Sometimes these
situations are very complex, e.g. when it comes to restructuring whole
organisational processes. Sometimes, though, there is only little
complexity when only one instrument is applied to perform quality
assurance, e.g. a questionnaire at the end of a program or course.
It has to be noted that quality literacy applies to all forms educational
environments, like traditional and/or blended learning. The concept of
quality literacy is derived from the concept of media literacy according to
Baacke (1996). He explains media literacy as a concept which describes
6
the abilities which individuals need, to act competent in a world which is
mediated through media (for further elaboration see Ehlers 2005). In
accordance with that quality literacy then describes the ability of
educational professionals to act competent in quality development
processes.
Figure 2: Dimensions of Quality Literacy
Quality literacy is a concept which can not exclusively be learned by
means of books or trainings but requires experience and practice. It
rather has to take into account an ongoing learning-, reflection- and
negotiation process between the stakeholders involved – including the
students. Quality literacy (fig. 2) can be seen as a set of competences
which contribute to carrying out quality development.
1. Dimension: Knowledge About Quality
This dimension addresses the “pure” knowledge about the possibilities
of today’s quality development and up-to-date quality strategies in elearning. The term quality strategies refers to all guidelines, structures,
rules, tools, checklists or other measures which have the goal of
enhancing the quality of an educational e-learning-scenario.
2. Dimension: Quality Experience
This dimension describes the ability of using quality strategies. It is
based on the experiences actors have with activities in quality
development and applying quality measures and strategies to e-learning
scenarios.
3. Dimension: Quality Innovation & Adaptation
This dimension relates to the ability which goes beyond the simple use
of existing instruments and strategies. It refers to the modification,
creation and development of quality strategies and/or instruments for
7
ones own purpose. An innovative and a creative aspect are important
for this dimension: Innovation in the sense of further development and
adaptation processes of quality strategies within the given system, and
creativity in the sense of thinking and developing new strategies for
quality development.
4. Dimension: Quality Analysis
Quality Analysis relates to the ability to analyse the processes of quality
development critically in the light of ones own experiences and the own
situation and context. It is important to evaluate different objectives of
quality development and negotiate between different perspectives of
stakeholders. To “analyse critically“ means the ability of differentiation
and reflection of existing knowledge and experiences with education and
quality development.
For Learners this would mean to be aware of the responsibility which
they have for quality in education as a co-producer of learning success.
For providers this means to enable flexible negotiation processes in the
educational offers in which individual objectives and preferences but
also societal contexts and organisational structures are integrated into
the definition of quality objectives for education.
4. The European Quality Observatory - Decision Support for Quality
Development
Today there are a variety of quality strategies and approaches
developed and in use already. Therefore it becomes obvious that quality
development often does not have the problem that there are no suitable
strategies available but rather that the responsible actors in education
do not really know what their demands are and which of the existing
models and strategies fit their needs. The European Quality Observatory
(EQO), a European consortium of expert organisations in the field of
quality in E-Learning, led by the University of Duisburg-Essen, took
these aspects as a starting point for a research project which started in
March 2003, to develop and implement a framework to collect, analyse
and compare quality approaches for education. The result is an internet
based repository which contains over 100 quality strategies from all
European countries which can be searched, compared and analysed by
all registered users.
The core achievements of the European Quality Observatory can be
summarised in four areas:
8
1. to provide a conceptual framework for analysis, description and
comparison of quality approaches in education (spec. E-Learning)
on a European level: the EQO Metadata Model (see below);
2. to establish an internet-based repository for quality approaches
(quality development, quality management, quality assessment) in
the field of E-Learning for users of these approaches (e. g. teachers,
tutors, developers, CEOs, etc.);
3. to provide recommendations for the use and experiences of other
users of quality approaches on a European level;
4. to provide services to support the implementation of quality
approaches & support the community of users.
The project has created a European quality community by collecting,
analysing and synthesising the different approaches currently in use, as
there are quality management, quality assurance and quality
assessment approaches. It is facilitating cross-cultural understanding of
quality systems by providing the quality approaches and concepts
through the internet based repository which is free for use for
everybody.
The theoretical and scientific core and the basis for the decision support
is a metadata model for the analysis, description and thus comparison of
quality approaches – the EQO Metadata Model. As the field of quality in
education is highly complex, the EQO model covers a theoretical and a
practical analysis of quality approaches and is therefore divided into two
main parts: In the first part the quality approaches are analysed on a
conceptual, document level, using the official documentation and
publications about the different approaches. A second level allows the
possibility of analysing experiences which users made when using the
particular strategies in their concrete everyday educational work. The
EQO Model for the first time provides a framework to handle the great
variety of approaches in the field of educational Quality. The analysis of
strategies and approaches covers the following three main categories:
1. General Description: In this category general information on the
quality approach are analysed.
2. Context: This category analyses the intended area of usage and
the educational context the quality approach is applied to.
3. Method: This category summarises information about the scope of
the quality approach.
The general description about quality approaches deals with
information such as title (name of the approach), textual description of
the actual version of this approach, the language in which this approach
9
is expressed, the location where it can be accessed as well as copyright
and other restrictions for using this approach (such as costs and
charging schemes). The context category summarises information
about the educational context, such as the educational level (e. g.
“university”) and the industry sector or educational institution the
approach is related to (e. g. “manufacturing industry”). Also the target
group this quality approach aims to is retrieved (e. g. “author” of learning
materials) as well as the cultural or regional coverage of the approach
(e. g. “not restricted to a country”). Furthermore the EQO model
expresses if the quality approach was developed for a specific topic
within a classification scheme and for which educational processes it
can be applied. In a last subcategory the quality goals that the approach
addresses respectively the sense in which quality is defined in that
approach are expressed. The user is given the possibility to rate the
importance of certain criterions here. The method category defines if
the quality approach focuses on the results of a process or the process
itself (e. g. “product-oriented”) and the methods the quality approach
uses (like “benchmarking”, “evaluation”, “standards”, etc.).
The analysis of practical experiences with using quality approaches is
covered through the category experiences. This category is used for
collecting and analysing experiences users made by applying a specific
quality approach. While the first three main categories of the EQO
Model describe the generic quality approach, the experience category
describes the instantiation of the approach for a specific implementation.
Although model and instantiation might be quite different and therefore a
mapping of experiences to a certain model could be difficult in some
cases, the EQO project expects the experiences part to provide valuable
information to quality practitioners. The experiences section is a fairly
innovative analysis category, which will become more important in the
future because it aims at analysing the impact of quality concepts. The
idea is to gather data on how educational processes are affected by
quality instruments.
The EQO Model is implemented into an internetbased repository
(http://www.eqo.info) which provides decision support functionality to
actors in the field of education. It contains today around 100 strategies
and has build up a community of users which contribute to the repository
and comment and peer review the strategies. The Approach supports
especially the first three steps of the Quality Development Cycle (see
chapter 2) which were described above by providing an information
basis for each of the necessary steps quality development involves.
1. Analysis Phase: This analysis is done on the basis of the EQO
Metadata model which categories function here as an assessment
10
framework. An example for this could be a manager of an
organisation in communications industry who is searching for a
quality approach especially dealing with improving the motivation of
learners in non school vocational education.
2. Decision Phase: On the basis of the needs analysis, existing
quality approaches are analysed and then linked to the users’
needs. As a result educational actors are presented with individually
fitting quality approaches for their specific situation. The manager
mentioned above will receive a list of quality approaches dealing
with his problem e. g. especially designed for appliance in his
country, using different methods like certification and policies, and
dealing with the improvement of technical or pedagogical quality
goals, etc. Users are supported in their decision for a quality
approach by information of a database of other users’ experiences
and assessments.
3. Realisation Phase: In this step the chosen approach has to be
adapted to the organisations’ specific needs. Individual quality
profiles can be created as a result, that help to implement the
approach. The analysis categories of the EQO Model are designed
to facilitate the individual adaptation process.
The European Quality Observatory Approach and the repository
supports the decision-making process by providing a structured method
as it is described above. Quality approaches are compared using
metadata and are analysed in order to support decisions. Decisionmakers and users can therefore study quality management and quality
assurance approaches and compare them using the above-mentioned
metadata. In this way, decisions for quality approaches can be
considerably accelerated and simplified.
In order to allow the user to easily find a suitable approach for a specific
educational purpose four different options to search the database are
provided in the repository:2 a quick search, a browse function, an
advanced search and a recommendation mechanism, which
recommends quality approaches for specified quality needs. The quick
search, which is the only function for unregistered users, allows an easy
full-text search in the database. After entering a keyword the user is
provided with matching search results. The browse option allows the
user to scan for quality approaches fitting a specific category and
criterion to choose from. For this purpose a set of default categories
(language, educational level, target group, country, process, goal, focus
2
Following description taken from Ehlers,
U.-D., Hildebrandt, B., Pawlowski,
J.M., Teschler, S. (2004
11
and method) is presented which can be further specified by choosing
from given pull-down menus. For example, figure 4 shows a browse
inquiry for the “country” “Norway“. For this chosen browse option the
search results will give an overview of all approaches in the database
that are (intended to be) applied in Norway.
Figure 3: Contribute a quality approach
The advanced search will form a combination of the other functions
described above. It allows to search for different categories and
criterions at the same time. Thus the user of the repository is able to
search more specific and gets modified results in the search list
differentiated according to the previously specified quality needs.
Additionally a recommendation mechanism will be provided. A user can
specify a profile of specific quality needs, can apply different weights to
the used factors and variables and will then be provided with a list of
fitting quality approaches for his/her situation on basis of other users’
and experts’ estimations.
Apart form the repository the portal provides a variety of other functions
to the user. Apart from highly useful and well structured information on
developments and discussions in the area of quality in E-Learning,
research results and downloadable publications, the user will get the
12
opportunity to get into contact with other users and experts in the field of
quality of all over Europe.
Figure 4: Browse through quality approaches
5. Summary and Conclusion
The article suggests the integration of process- and formalised quality
models with competency, resp. professionalisation oriented approaches.
It emphasizes the importance of viewing quality development not as a
static add-on to education, e.g. as an isolated evaluation approach at
the end of a course. Quality development rather is viewed as a key
aspect, occurring in every single development and delivery process of elearning courses and programs.
Quality development is a process in which the interests and
requirements of the e-learning stakeholders have to be considered as a
whole and combined to a comprehensive concept. Quality in this respect
is seen as a relation between the demands and needs of a stakeholder
group and the actual delivery of e-learning. In order to shape this
relation in the best possible way a negotiation process is necessary
which involves all stakeholders and integrates their preferences and
situations against the background of the given economical and
organisation situation. These negotiation processes occur in different
positions of the learning environment.
13
The question how to find and apply the best quality approach for a
specific context can be envisioned as a decision cycle with four steps.
However, to decide which quality approach is suitable, to choose from a
set of possible strategies, and to adapt those strategies to the specific
situational context certain competencies are necessary. For these
competencies we suggest the concept of quality literacy. It covers
competencies like knowledge of quality development, experiences in
using particular instruments, modification skills and the ability of
thoroughly analysing ones own situation and needs.
6. References
1. Baacke, D. (1996): Gesamtkonzept Medienkompetenz. [The
Concept of Media Literacy] In: agenda. Zeitschrift für Medien,
Bildung, Kultur, [Agenda. Journal for Media, Education, Culture]
März/ April 1996, S. 12-14.
2. Ehlers, U.-D., Fehrenbach, T. (2004): PQM - Partizipative
Qualitätsentwicklung im E-Learning. Bildungsprozesse als Basis
für eine neue Lernerorientierung. [PQM-Participative Quality
Development in E-Learning. Educational Processes as a Basis
for a new Learner Orientation] In: Deutsches Institut für Normung:
Fachbericht zur DIN PAS 10321. Berlin
3. Ehlers, U.-D., Pawlowski, J.M. (2004): E-Learning-Quality: A
Decision Support Model for European Quality Approaches, In:
Fietz, G., Godio, C., Mason, R. (2004): eLearning for international
Markets. Development and Use of eLearning in Europa. Bielefeld
4. Ehlers, U.-D., Hildebrandt, B., Pawlowski, J.M., Teschler, S.
(2004): The European Quality Observatory. Enhancing
Quality for Tomorrow’s Learners. In: Supporting the Learner in
Distance Education and E-Learning, Proceedings of the Third
EDEN Research Workshop, Oldenburg, Germany, S. 138-145
5. Terhart, E.: Qualität und Qualitätssicherung im Schulsystem.
Hintergründe - Konzepte - Probleme. In: Zeitschrift für Pädagogik,
Heft 6/2000 (46. Jg.), S. 809 – 830
Autobiography
DR. ULF-DANIEL EHLERS studied English, Social Sciences and
Educational Sciences at the University of Bielefeld where he finished his
Ph.D. on “Quality in E-Learning” in 2003. He is now the coordinator of
the “European Quality Observatory” (http://www.eqo.info) for the
Department for Information Systems for Production and Operations
Management at the University Duisburg-Essen. Before he has been
14
responsible for the evaluation of the project “L³: Lifelong Learning”, in
which a comprehensive infrastructure for E-Learning was developed by
a consortium of 20 partners and which was led by SAP AG. In close
collaboration with SAP he led the development of the L³-QualityStrategy.
Dr. Ehlers is an internationally recognized researcher and innovator in
the area of E-Learning focussing mainly on quality management. He has
extensive experience in helping individuals achieving superior learning
performances and has run several projects and evaluations in the field
of e-learning and knowledge management as well as e-business
including new-technology-consulting for small and medium sized
enterprises. Dr. Ehlers developed the Learners’ Quality Model for elearning, which is a basis for learner centred quality development in elearning. He is the author/ publisher of several books and more than 50
articles and book chapters, has been a featured speaker at numerous
international conferences throughout Europe, and is member of several
national associations for e-learning and education in Germany.
15