A. Procedures of the analytic hierarchy process
A decision is the choice for a decision alternative in order to attain an objective. Saaty’s
analytic hierarchy process (AHP) is a technique for multi-criteria decision analysis. This
technique aims to support the analysis of complex decisions. It facilitates a quantitative
comparison of how well decision alternatives perform in fulfilling multiple criteria relevant to
the objective.
AHP structure
The AHP structures a decision into a hierarchy of factors. In essence, the hierarchical levels
comprehend the objective of the decision, the (sub) criteria, and the alternatives. The criteria
need to be mutually exclusive, clear, comprehensive, and to be of importance within the
same order of magnitude; the difference in importance should be less than ten times as
important.
Pairwise comparisons
The AHP offers a pairwise comparison approach to estimate the weights of the criteria and
the priorities of the alternatives. The weights and priorities are derived from a matrix of
pairwise comparisons of two criteria or alternatives. Pairs of criteria are compared to judge
the criteria’s relative importance in attaining the objective. Pairs of alternatives are compared
to judge the relative preferences for these alternatives with regard to a specific criterion.
Judgments are given on a scale ranging from the numerical value 1 reflecting equal
importance or preference up to and including 9 reflecting extremely greater importance or
preference. An example of a pairwise comparison of the importance of two criteria is:
With respect to the objective,
1 equally
3 moderately
criterion 1 is a
5 strongly
more important criterion than criterion 2.
7 very strongly
9 extremely
2, 4, 6, 8. Intermediate values
1
The two criteria can be reversed, if the latter criterion is a more important criterion than the
first criterion. Table 1 provides the matrix of pairwise comparisons of n criteria: A = ( a ij).
Acriteria
criterion1
criterion2
…
criterionn
criterion1
a11
a12
…
a1n
criterion2
a21
a22
…
a2n
…
…
…
…
…
criterionn
an1
an2
…
ann
Table 1. Matrix A
Both reflexivity (aij = 1 along the diagonal) and reciprocity (aij = 1/aji) are assumed in this
matrix. Therefore, only values above the diagonal must be entered. This means that for n
criteria, one needs n(n-1)/2 distinct comparison judgements.
On the basis of the same approach, the priorities of the alternatives are computed. These
priorities are derived from the pairwise comparisons of the preferences for the alternatives
with regard to each criterion.
Inconsistency
For each set of pairwise comparisons, the AHP provides a measure of consistency to show
how consistent each pairwise comparison is with regard to the remainder of the
comparisons. This measure is based on the λmax , which is the maximum eigenvalue of the
matrix of pairwise comparisons. This eigenvalue is always greater than or equal to n for
positive, reciprocal matrices, and is equal to n if the matrix of pairwise comparisons is
consistent. Normalising this measure by the size of the matrix, Saaty defines the consistency
index (C.I.) as:
2
C.I. = (λmax – 1)/ (n-1).
For each size of matrix n, 500 random matrices were generated and their mean C.I. value,
called the random index (R.I) is computed. The consistency ratio (C.R.) indicates how far the
pairwise judgements deviate from a purely random matrix of pairwise comparisons, and is
defined as:
C.R. = C.I./R.I.
Saaty’s rule of thumb is that 10 per cent of the inconsistency of the random matrix is allowed.
This implies that a value of the C.R. ≤ 0.1 can be considered reasonable, C.R. ≤ 0.2
tolerable, and C.R. ≥ 0.2 should be revised or discarded.
Weighting factors and priorities
In case of acceptable degrees of inconsistency, weighting factors and performance priorities
can be calculated. The principal right eigenvector approach is recommended by Saaty. In
this approach, the eigenvalue of the matrix of pairwise comparisons is used to estimate the
weights of the criteria:
wi = (∑n j=1 aij * wj) / λmax for all i = 1,2,…,n.
This eigenvector method can be interpreted as being a simple averaging process by which
the final weights are the average of all possible ways of comparing the relative weights of the
criteria, or the relative priorities of the alternatives.
Group average
In case the pairwise comparisons have been judged in a group setting, and a group average
is to reflect the opinion of the group as a whole, the use of the geometric mean of all pairwise
comparisons is recommended. When the AHP supports a group of x (x = 2,3…) decisionmakers, the cells of the matrix are filled with aggregated pairwise comparisons as computed
by the geometric mean of the individuals’ comparisons:
3
aij = (aij(1) * aij(2) * … * aij(x)) (1/x) for all i,j = 1,2,…,n.
Weighting factors (w) for the outcome measures are calculated based on the group scores
for each pairwise comparison.
In case the AHP supports multiple decision makers that individually have judged the pairwise
comparisons, the arithmetic mean can be used to average the weights and priorities of the
individual respondents. For example, the arithmetic mean of a weight assigned by x decision
makers is calculated as follows:
W1 = (w1(1) + w1(2) + … + w1(x)) / x.
Overall priorities of the alternatives
When the weighting factors (w) of all criteria are known, the priorities (p) of the alternative
under the varying criteria are known, the overall performance or priority of the alternatives
can be computed. It is the summation of all products of the priority of an alternative and the
weight of the corresponding criterionn:
∑(pn * wn) for n = 1,2,3.
Conclusion
The AHP is meant to support the decision making process. The decision structure
systematically reflects those factors that are perceived to be relevant to a decision. The
pairwise comparisons support the decision makers to judge the impact of these factors on
the decision. The inconsistency ratios guide the decision makers towards the revision of
judgements that are at odd with the remainder of their judgements. The weighting factors
derived from the pairwise comparisons help them to build the logical foundation of choosing
the best decision solution.
4
B. Questions from the questionnaire (English translation)
1. Which screening technique do you prefer regarding the property: a correct test outcome
for a person with colorectal cancer? And to what extent do you prefer this technique?
Fecal test
Colonoscopy
Fecal test
Sigmoidoscopy
Fecal test
Virtual colonoscopy
Colonoscopy
Sigmoidoscopy
Colonoscopy
Virtual colonoscopy
Sigmoidoscopy
Virtual colonoscopy
2. Which screening technique do you prefer regarding the property: a correct test outcome
for a person with NO colorectal cancer? And to what extent do you prefer this technique?
Fecal test
Colonoscopy
Fecal test
Sigmoidoscopy
Fecal test
Virtual colonoscopy
Colonoscopy
Sigmoidoscopy
Colonoscopy
Virtual colonoscopy
Sigmoidoscopy
Virtual colonoscopy
Value
Meaning
1
Equally preferred
3
Moderately preferred
5
Strongly preferred
7
Very strongly preferred
9
Extremely preferred
2,4,6,8
Intermediate values
5
3. Which screening technique do you prefer regarding the property: safety? And to what
extent do you prefer this technique?
Fecal test
Colonoscopy
Fecal test
Sigmoidoscopy
Fecal test
Virtual colonoscopy
Colonoscopy
Sigmoidoscopy
Colonoscopy
Virtual colonoscopy
Sigmoidoscopy
Virtual colonoscopy
4. Which screening technique do you prefer regarding the property: convenience? And to
what extent do you prefer this technique?
Fecal test
Colonoscopy
Fecal test
Sigmoidoscopy
Fecal test
Virtual colonoscopy
Colonoscopy
Sigmoidoscopy
Colonoscopy
Virtual colonoscopy
Sigmoidoscopy
Virtual colonoscopy
Value
Meaning
1
Equally preferred
3
Moderately preferred
5
Strongly preferred
7
Very strongly preferred
9
Extremely preferred
2,4,6,8
Intermediate values
6
5. Which screening technique do you prefer regarding the property: frequency? And to what
extent do you prefer this technique?
Fecal test
Colonoscopy
Fecal test
Sigmoidoscopy
Fecal test
Virtual colonoscopy
Colonoscopy
Sigmoidoscopy
Colonoscopy
Virtual colonoscopy
Sigmoidoscopy
Virtual colonoscopy
Value
Meaning
1
Equally preferred
3
Moderately preferred
5
Strongly preferred
7
Very strongly preferred
9
Extremely preferred
2,4,6,8
Intermediate values
7
6. What property is more important to you in selecting a screening technique? And to what
extent?
Correct test if
cancer
Correct test if NO
cancer
Correct test if
cancer
Safety
Correct test if
cancer
Load of the test
Correct test if
NO cancer
Safety
Correct test if
NO cancer
Load of the test
Safety
Load of the test
7. What property is more important in determining the load of the screening test? And to
what extent?
Inconvenience
Frequency
Value
Meaning
1
Equally important
3
Moderately more important
5
Strongly more important
7
Very strongly more important
9
Extremely more important
2,4,6,8
Intermediate values
8
8. Suppose you have been invited for a free colorectal cancer screening with the fecal test.
Do you intent to participate?
1 = definitely not attend
2 = probably not attend
3 = perhaps not attend/perhaps attend
4 = probably attend
5 = definitely attend
9. Suppose you have been invited for a free colorectal cancer screening with colonoscopy.
Do you intent to participate?
1 = definitely not attend
2 = probably not attend
3 = perhaps not attend/perhaps attend
4 = probably attend
5 = definitely attend
10. Suppose you have been invited for a free colorectal cancer screening with
sigmoidoscopy. Do you intent to participate?
1 = definitely not attend
2 = probably not attend
3 = perhaps not attend/perhaps attend
4 = probably attend
5 = definitely attend
11. Suppose you have been invited for a free colorectal cancer screening with virtual
colonoscopy. Do you intent to participate?
1 = definitely not attend
2 = probably not attend
3 = perhaps not attend/perhaps attend
4 = probably attend
5 = definitely attend
9