INF5220 Qualitative Research Methods
UNIVERSITY OF OSLO
Faculty of Mathematics and Natural
Sciences
INF5220 – Qualitative Research
Methods
Spring 2006
Initial draft for PhD upgrade document.
Working title: “Enquiries into the
ideology of quality management and
process improvement”
Written by:
Petter Øgland
Version 0.6
June 7th, 2006
Spring 2006
Page 1 of 15
INF5220 Qualitative Research Methods
Table of Contents
1
2
Introduction ................................................................................................................. 3
Theory ........................................................................Error! Bookmark not defined.
2.1
Knowledge management ..................................................................................... 6
2.2
Process improvement .......................................................................................... 6
2.3
Information infrastructures ................................................................................. 6
2.4
Problem area ....................................................................................................... 8
2.5
Research questions .............................................................................................. 8
2.6
Structure of the upgrade document ..................................................................... 9
3 Method .......................................................................Error! Bookmark not defined.
3.1
Action research and quality improvement ........................................................ 11
4 Results ........................................................................Error! Bookmark not defined.
4.1
The COBOL programmers at NTAX ............................................................... 13
4.2
The documentation process as NTAX .............................................................. 13
4.3
The NTAX management information system ................................................... 13
4.4
Quality management at DNV............................................................................ 13
5 Discussion ..................................................................Error! Bookmark not defined.
5.1
Two ways of understanding ...............................Error! Bookmark not defined.
5.2
Knowledge about the current system .................Error! Bookmark not defined.
5.3
Knowledge beyond the current system ..............Error! Bookmark not defined.
6 Conclusion .................................................................Error! Bookmark not defined.
References ......................................................................................................................... 14
Spring 2006
Page 2 of 15
INF5220 Qualitative Research Methods
1 Introduction
This document has been written as an assignment for the course INF5220 – “Qualitative
Research Methods”. It was suggested by the lecturer that the assignment should be
written as a first draft for the “PhD upgrade document” that all PhD students are required
to write and present before starting the second year of PhD studies (UiO, 2006). By
taking the course in research methods early on in the studies, this should prove useful for
planning case studies and thus preventing problems that might arise due to insufficient
understanding of research methodology before starting empirical research.
The general idea behind the PhD study is that the traditional way of installing quality
management systems (QMS) often fails. Usually, quality management systems are
implemented according to some kind of waterfall model, going through stages of
specification, design, implementation and a final stage of continuous improvement.
Some people (e.g. Seddon, 1997; Power, 1997) have argued that the introduction of
quality management systems in this manner may not only have no impact on quality at
all, but may even make reduce quality of performance.
In the 1990’s there emerged a body of literature on complexity theory (e.g. Complex
Adaptive Systems, CAS), providing a framework for understanding economies,
organizations, engineering practices and cognitive processes from a perspective of
biological processes. Dooley (1995; 2000) is one researcher who has tried to link CAS
with total quality management (TQM), and thus implying that quality management could
in some cases benefit from viewing the QMS as a CAS rather than a traditional system.
A QMS consists of processes, people and technology. When it comes to understanding
and applying the principles of CAS for evolving and maintaining management
information systems (MIS), there is an evolving theory of Information Infrastructures
(Ciborra et al, 2000). Information Infrastructures (II) consists partly of seeing the
development and maintenance of the Internet as a prime example of how to evolve and
maintain an MIS in complex and unpredictable environments, and partly of applying a
postmodern epistemology (e.g. Actor-Network Theory, ANT) for understanding the
interaction between man and machine without becoming biased towards any of the two.
The purpose of the research is to contribute to the body of knowledge for Information
Infrastructures and Management Information Systems by summarizing aspects of what I
have been part of as a Quality Manager at the Norwegian Directorate of Taxes (NTAX)
2000-2005, and further investigations at NTAX and elsewhere in the period 2006-2008.
Systems thinking can perhaps be seen as a standard way of understanding quality
management systems (Deming, 1993; xxx). In the next chapter I will try to describe how
I see Information Infrastructure as a suitable framework for understanding quality
management systems in certain situations.
Action Research (AR) was developed in the late forties as a method for researching
organizations and stimulating improvement and change (Lewin, 1946). Checkland
Spring 2006
Page 3 of 15
INF5220 Qualitative Research Methods
(1981) argues that AR is the natural way of doing science in a systems thinking context.
Goldstein (1992) argues that organizational change through the application of CAS
should follow the principles of Lewin, although certain changes in perception need to be
done, a path that paves the foundations for the roadmap suggested by Dooley et al (2000),
and thus the research method I’ve chosen to describe in chapter three.
As the thesis is based on a series of papers, the intent of chapter four is to present the
insights gained through each of these papers. So far there is only one paper to present, a
study on the process improvement of making COBOL programmers work in a more
homogenous manner in order to reduce the risk of software becoming too personalized.
In chapter five there is a short discussion of the results, showing how they seem to add to
the theory of Information Infrastructures and Management Information Systems. The
preliminary conclusions of the thesis so far, and some ideas on where I want to travel
with my next few papers, is presented in the final chapter.
1.1 Problem setting and motivation
1.2 Research setting
1.3 Research questions
1.4 Expected contributions
Spring 2006
Page 4 of 15
INF5220 Qualitative Research Methods
2 Existing work in this domain
As a part of the PhD curriculum, I plan to take courses in qualitative methods (INF5220),
knowledge management (INF5190), process and product improvement (INF5180) and
information infrastructures (INF5210). So far I’ve almost completed the course on
knowledge management, and this draft version of the upgrade document is written to
fulfill the requirements of the qualitative research course.
This means that I do not yet have all the theoretical understanding for describing related
research. In fact, one of the major problems since I got introduced to the IS community
in September 2005 and participated on the 9th PhD Days Workshop in
September/October has been to define the problems I want to research in a manner that
will make both to me personally (wanting to use the PhD to lead to new jobs and
possibilities in quality management), the requirements of the Norwegian Tax
Administration in addressing practical problems relevant for their current status, and to fit
with the philosophy of the IS group at the Institute of Informatics at the University of
Oslo.
A possible way of structuring the theory section is by summarizing relevant aspects of the
courses I’ve taken or plan to take, thus putting Knowledge Management (KM), Process
and product improvement (SPI) and information infrastructures (II) in context with the
problem of designing Management Information Systems.
Spring 2006
Page 5 of 15
INF5220 Qualitative Research Methods
3 Theoretical positioning and research framework
3.1 Knowledge management
In the reports I made for the INF5190 course, I tried to understand the KM concepts from
the perspective of TQM, and thus focusing on explicit knowledge rather than on the tacit
knowledge which was what the course mainly focused on. When addressing the theory
in the context of the upgrade document, perhaps it would be better to discuss the ideas
with a more clear relevance to the problem of designing Management Information
Systems.
3.2 Process improvement
Although I haven’t taken this course yet, I’ve gone through some of the lectures and read
some of the course literature. Much of it relates to quality management. It uses CMMI as
a framework for discussing processes improvement, and it seems to place CMMI in the
context of scientific management and total quality management. In other words, this is
the course that corresponds more or less directly to my current way of thinking, and it
uses the same models and methods that I would use for improving processes.
It could be interesting to try to contribute something to the theory being discussed here,
but the way of designing waterfall projects for installing process improvement systems
etc is not the way I prefer to work and neither is it a way I believe is a particularly good
way of working with these issues. Current experience from how DNV is trying to aid
NTAX in getting certified to the ISO 9001:2000 standard, and my experience with the
Wipro CMM assessment, tells me that the traditional top-down way of installing a quality
management system through the use of some great project is not the best way of doing it.
It may perhaps be a good way of getting the infrastructure for running the quality
management system, but it doesn’t take into account that it is difficult to know how to
design quality indicators and know what to measure.
My impression is that a much better way of installing quality management systems would
be by growing them through the practice of an evolutionary or what Hanseth describes as
Information Infrastructures (II).
In this section, I might also say something on systems thinking (Checkland, 1981).
3.3 Information infrastructures
This is another course I haven’t taken yet, but as I’ve read some of the literature that is
used, I feel I may be able to present my current impression of what the theory is all about.
In fact, some of the most insightful comments I’ve read on II are the review comments
Hanseth and Lyttainen received when trying to have his latest II article published in ISR.
Spring 2006
Page 6 of 15
INF5220 Qualitative Research Methods
A recent external investigation of the IT function of NTAX concluded that NTAX could
be interpreted as a Machine Bureaucracy developing in the direction of an Administrative
Adhocracy (Mintzberg, 1983; Statskonsult, 2002). When standardized and repetitive
routines of the tax administration is taken over by computers, the computer department
automatically grows in power, regardless of how power may have been defined according
to the organizational charts.
From a quality management point of view, this is interesting in two aspects. Firstly, the
power relationships defined on the organizational charts may not be the best way to
understand the operational QMS. Secondly, the IT strategy and the methods of further
computerizing the Machine Bureaucracy could have a major strategic importance for how
the QMS is maintained and improved.
As quality management normally works within the paradigm of viewing the organization
as a cybernetical system to be optimized (Morgan, 1997: chapter 3), the Administrative
Adhocracy produces a challenge in terms of a paradigm of complexity, flow and growth
(ibid: chapter 8). Some writers (e.g. Kelly, 1994: 196) have pointed out that the Japanese
quality management practices (for the IT function) follows this new paradigm (see also:
Liker, 2004; Womack & Jones, 2003).
One of the core ideas in complexity theory as applied to management problems, is to let
go of central planning and control, but rather distribute the system into weakly tied units
and let this loosely connected system steer itself bottom-up (e.g. Kelly, 1994: 468-469).
However, in order to let loose on the overall control, more control much be put into each
separate unit. Kelly (ibid: 469) describes this as the “Fourth Law of God”, stating that
“the only way to make a complex system work is to begin with a simple system that
works; […an ecology] is created from assembling it incrementally from simple modules
that can operate independently.”
If we regard the QMS as a Complex Adaptive System (CAS) (Battram, 1996), the Fourth
Law could be interpreted as controlling and improving the complete QMS by making
sure each weakly tied unit follows a strictly deterministic logic like, say, a computer
program. According to the CAS theory, this would constitute a robust strategy for
making the organization “automatically” grow, long-term, in the direction of
“organizational excellence” (Oakland, 2001).
Without explicitly referring to the theory of CAS, it is my impression that similar ideas
are prevalent in the information systems research environment at UiO. It has, for
instance, been suggested (Hanseth, 2005) that insights from current research on
broadband deployment and development of communication standards (research that
reveal CAS-like structures and strategies), should be used for implementing information
systems in health-care organizations.
At the moment there seems to be much interest in applying complexity theory into the
area of organization and management research (Stacey et al, 2000; Urry, 2003), including
Spring 2006
Page 7 of 15
INF5220 Qualitative Research Methods
the problems of quality management (Dooley et al, 1995). The idea of seeing the quality
manager as an actor (component) within a CAS seems to be implicit in some of this
research (e.g. Dooley, 2002), but investigations of the role of the quality manager as a
leverage node of the system (and what the programmable logic of this node might be)
appears to be unexplored territory.
3.4 Problem area
In most guidelines we’ve found on installing quality management systems, there is a
strong focus on top management commitment. Often when TQM and BPR projects fail,
it is common to complain about lack of top management commitment. However, there
are many issues top management have to focus on, and in many cases it may not be
expected that they are able to be fully committed to the implementation and continuous
improvement of the organizations quality management system.
Information systems development (ISD) involves risk, and failures remain common
despite advances in development tools and technologies. While the Norwegian Tax
Authorities (NTAX) have generally received positive publicity with respect to their
efforts on improving public services, there is a high level of complexity in running
software development projects and maintaining software solutions. There is a high level
of interaction between departments within NTAX and a low tolerance for error. The
organization is constantly being monitored by the Department of Finance (FIN) and the
Auditors General (AG).
As strategy for preventing problems, NTAX use various quality assurance methods, such
as developing and improving technical schemes and management methods, certifying
personnel to international standards, and participating in networks for sharing best
practices.
The current quality assurance approach is developed as a set of policies, strategies and
standards, defined as a part of the information security management system (NTAX,
1996), and to be used as a part of the general NTAX management system. The Quality
Management System (QMS) was operationally implemented five years later by
appointing a Quality Manager for coordinating the Quality Function (Quality
Management Organization) (NTAX, 2001).
There are many challenges in managing the Quality Function, and from the point of view
of NTAX, it is important to understand how to organize and utilize quality management
in a manner that both generates efficient quality improvement and fits with corporate
culture (NTAX, 1997; DNV, 2005).
3.5 Research questions
The aim of the research is to gain insights on the following question:
How does one handle the lifecycle of quality management systems given the
scenario that there is little or no top management commitment?
Spring 2006
Page 8 of 15
INF5220 Qualitative Research Methods
We want to research this main question by breaking it down into several subquestions
related to the Information Systems framework we use for analysis.
3.6 Structure of the upgrade document
This is what the PhD Manual (UiO, 2006) says about the upgrade document:
By the end of the first year, the student shall develop and present a ‘1st year report’ that will make their
research visible to the group and assure that they are, or alternatively provide apt support to make
them, on course to successfully complete their PhD. The content of the 1 st year report should include
the following (8000-12.000 words in total):









Thesis title
Introduction – Problem Area Description, and Research questions
The significance of the issue to be discussed by the proposed research, and why further
research on the issue is required.
How the research issue related to/draws upon existing research and is discussed in the
relevant literature – which should be critically reviewed.
The theoretical approach/framework being followed and how this compares with possible
alternative approaches.
Any preliminary fieldwork that has been carried out or previous work done by the student
with is being drawn upon.
Research design and methodology to be employed, as well as a justification for this approach.
Intended plan for fieldwork, if relevant, including arrangements for access.
Anticipated outcomes of the proposed research.
As I got my PhD proposal accepted on the 9th of March 2006, it feels a bit early
presenting an upgrade document, as I have not yet a full overview of related research and
I do not even have a full understanding of the research question I’m trying to answer.
When I wrote the initial PhD proposal, I had only a superficial understanding of the
culture and ideas of the information systems group at the Institute of Informatics at the
University of Oslo. My ambition for doing a PhD in quality management has been that
of having a background in industrial mathematics from 1991 and working in the field of
quality control and industrial engineering for 15 years.
When working with quality control and process improvement, the mode of working is
similar to that of doing action research. However, unlike the tradition of information
systems (IS), my approach has not been qualitative. It has been a mixture of qualitative
and quantitative methods, with a focus on applying quantitative methods due to the belief
of “what gets measured gets done”.
However, as I gradually get to know the IS community, I get the impression that the kind
of problems that I need to address in order to be a part of this community are the sociotechnical problems that may have implications for design of information systems, or in
my case Management Information Systems (MIS).
Consequently, in this first attempt to write an upgrade document, I will try to refocus my
research problem from problems related to the role of the quality manager of large public
organizations to the problems of designing and continually redesigning Management
Spring 2006
Page 9 of 15
INF5220 Qualitative Research Methods
Information Systems in compliance with the requirements and guideless of the ISO 9000
series and additional international, regional and de factor standards such as EFQM,
CMMI, TickIT, ITIL and CobiT.
Designing quality management systems is a non-trivial task. My own experience with
the ISO 9001:2000 has showed me that things are not as simple as they seem. There is
also international research showing how and why TQM, Six Sigma and other types of
quality management programs tend to fail. What we are looking for is robust design. It
doesn’t matter if it should take a bit of time to implement the quality management
system, as long as it gets gradually implemented in compliance with the philosophy of
Deming and general “best practice” philosophy of doing quality management.
Spring 2006
Page 10 of 15
INF5220 Qualitative Research Methods
4 Research approach
4.1 Methodology
As the aim of the thesis is to contribute to the understanding of how to improve the
practice of quality control and process improvement, and consequently it may be
reasonable to try to understand the problems from the perspective of the practitioners
perspective. In the case of classical quality control (Shewhart, 1938), the mode of
thinking, when achieving process control and improvement, corresponds to that of
physicians identifying physical parameters. The ontology and epistemology suggested
useful for quality management, from
Shewhart describes this perspective as American pragmatism
Although This perspective is, however, not a strictly positivist point of view,
imprinvestigating this perspective corresponds to
In terms of methods, I should try to frame what I’m doing as action research. Perhaps I
could even write a paper on this…
As this document is also a part of the research methodology, it should be possible to
discuss how to apply action research (AR) within the framework of updating this
document. In fact, although the update document is primarily meant as a document for
controlling the first year of output from the research, the upgrade document should be
updated and presented also for the second year, and it should evolve into the “kappa” for
the thesis of the third year.
4.2 Action research and quality improvement
The basis for creating quality improvement by applying the system of ISO 9000…
The method will focus on building a general QMS model based on the CAS point of
view. The model should be sufficiently abstract to fit various organizations, but also
sufficiently detailed to be used for predicting performance (on a simple level) of a
concrete Quality Function, such as the NTAX Quality Function, and also other
organizations if possible.
For building such a model, literature on organizations, management and complexity need
to be done, emphasizing the dynamics of an organization from a Machine Bureaucracy to
an Administrative Adhocracy.
Data collection will focus on interviewing and assessing the quality manager and key
personnel within his circle of influence. The assessments will be structured according to
various quality management standards, and will be based on available documentation.
Interviews will be done partly to validate the assessments and partly as interpretive
investigations in order to understand how the world-view of the quality manager is
Spring 2006
Page 11 of 15
INF5220 Qualitative Research Methods
compliant with the conditions of the CAS model. If there seems to be a reasonable level
of consistence, then the method will focus on how to use CAS as a language for
discussing management issues.
A typical way of doing assessments of the quality manager could be to ask the questions
in ISO 9004 (ISO, 2000c: Appendix A) and categorize the results according to the ISO
9004 maturity scale from 1 to 5. The parts of the CMM / ISO 15504 and CobiT
assessments standards that relate to quality management could be used in a consistent
manner.
In general the research will follow the conventions of Management Research (EasterbySmith et al, 2002; Collis & Hussey, 2003), including aspects of Action Research when
needed, although further details have to be discussed with each of the quality managers
that are to be used as research subjects in the investigation.
Spring 2006
Page 12 of 15
INF5220 Qualitative Research Methods
5 Current status
The thesis is currently expected to be comprised of 5-6 papers, addressing various issues
related to quality management and process improvement from the perspective of
information systems theory.
5.1 The COBOL programmers at NTAX
I’ve written on IRIS article on the COBOL programmers, but it’s a problematic paper
because, even though the empirical data may be interesting, the paper is not clear on what
research questions it is trying to solve. As I write this upgrade document, I feel that I
should try to redesign the COBOL paper, as I still have time, in order to focus more on
the question of how to design Management Information Systems.
5.2 The documentation process as NTAX
The documentation process is another part of the Management Information System.
Although I plan to focus on technicians, the methods I apply could be used for the
documentation as a whole, as I did when I worked as quality manager. Perhaps one of
the design problems could relate to the fact that the people got irritated by having to
follow the standards they had made themselves.
5.3 The NTAX management information system
After these two single cases, it could be interesting to talk to Barbakken in order to try to
understand the NTAX management information system from the perspective of top
management.
5.4 Quality management at DNV
In order to get supplementary insights, it could be interesting to interview the people at
DNV in order to understand how they designed their QMS and how they are continuously
redesigning the system.
Spring 2006
Page 13 of 15
INF5220 Qualitative Research Methods
References
Coghlan, D. & Brannick, T. (2001): “Doing Action Research in Your Own
Organization”, SAGE: London.
Easterby-Smith, M., Thorpe, R. and Lowe, A. (2002): Management Research: An
Introduction. 2nd Edition. Sage Publications: London.
Battram, A. (1996): Navigating Complexity: The Essential Guide to Complexity Theory
in Business and Management, The Industrial Society: London.
Checkland, P. (1999): Systems Thinking, Systems Practice. Wiley: Chichester.
Ciborra, C. U. (2000): From Control To Drift: The Dynamics of Corporate Information
Infrastructures, Oxford University Press: Oxford.
Collis, J. and Hussey, R. (2003): Business Research: A practical guide for
undergraduate and postgraduate students, 2nd Edition, Palgrave Macmillan: New York.
DNV (2005): Kvalitetsarbeid i IT-funksjonen, Vurdering av systemet for kvalitetsstyring
i IT-funksjonen i Skattedirektoratet, Rapport unntatt offentlighet, 03.10.2005, Det Norske
Veritas: Høvik.
Dooley, K. (2002): “Organizatonal Complexity,” International Encyclopedia of Business
and Management, M. Warner (ed.), Thompson Learning: London, p. 5013-5022.
Dooley, K., T. Johnson, and D. Bush (1995): “TQM, chaos, and complexity”, Human
Systems Management, 14(4): 1-16.
Hanseth, O. (2005): “Globally Scaleable Information Structures” presentation at PhD
Days Workshop in Information systems, 30 Sept – 1 Oct 2005, Department of
Informatics, University of Oslo: Oslo.
ISO (2000a): Quality management systems – Fundamentals and vocabulary (ISO
9000:2000), International Standards Organization: Geneve.
ISO (2000b): Quality management systems – Requirements (ISO 9001:2000),
International Standards Organization: Geneve.
ISO (2000c): Quality management systems – Guidelines for performance improvements
(ISO 9004:2000), International Standards Organization: Geneve.
Kelly, K. (1994): Out Of Control: The New Biology of Machines, Social Systems and the
Economics World, Addison-Wesley: New York.
Liker, J. (2004): The Toyota Way, Addison-Wesley: New York.
Spring 2006
Page 14 of 15
INF5220 Qualitative Research Methods
Morgan, G. (1997): Images of Organization, SAGE Publishing: London.
Mintzberg, H. (1983): Structure in Fives: Designing Effective Organizations, PrenticeHall: Upper Saddle River, New Jersey.
NTAX (1997): Strategi, politikk og standarder for IT-sikkerhet i Skatteetaten, Version
2.1, SKD no. 6/96, Directorate of Taxes: Oslo.
NTAX (1998): Strategisk plan for bruk av IT i Skatteetaten, Version 1.3, SKD no. 62/96,
Directorate of Taxes: Oslo.
NTAX (2001): Stillingsbeskrivelse – Kvalitetssjef (underdirektør), Notat 21.03.01,
Directorate of Taxes, IT department: Oslo.
Oakland, J.S. (2001): Total Organizational Excellence: Achieving World-Class
Performance, Butterworth-Heinemann: Oxford.
Power, M. (1997): The Audit Society: Rituals of Verification. Oxford University Press:
Oxford.
Seddon, J. (1997): In pursuit of quality: The case against ISO 9000. Oak Tree
PressBooks: London.
Statskonsult (2002): Organisering av IT-funksjonen i Skatteetaten, Statskonsult: Oslo.
Urry, J. (2003) Global Complexity, Polity: Cambridge.
UiO (2006): Phd Manual, Information Systems Group, Department of Informatics,
University of Oslo, http://www.ifi.uio.no/forskning/grupper/is/docs/PhDManual.pdf.
Womack, J.P. and Jones, D.T. (2003), Lean Thinking: Banish Waste and Create Wealth
in Your Corporation, 2nd Edition, Free Press Business.
Spring 2006
Page 15 of 15