NSW Department of Finance and Services
Tendering Manual Appendix 7A – Evaluation Methodologies
EVALUATION METHODOLOGIES
Evaluation of Tenders
Qualitative Narrative Assessment
This methodology, as its title implies, is a written assessment which discusses the
relative merits of tenderers to assist in making a judgement about the tender
providing best value for money using a qualitative comparison.
By itself, it is inappropriate for more complex, technical evaluations and other
situations where this will not distinguish between the value offered by the
tenders.
Comparative Assessment & Ranking Method
Comparative assessment involves ranking the tenders in relative order of merit
against the requirements of the RFT documents, evaluating each evaluation
criterion and the overall merit of each tender against all requirements.
The Tender Evaluation Plan (TEP) must define the ranking method (as per the
options described in this appendix) to be used during the evaluation to assist in
comparing tenders. In particular, specific guidance should be provided on the
scoring or other system to apply in the ranking of tenders and on any other
matters to be considered in the determination of relative value for money.
The TEP should identify the required justification for all ranking judgements. For
example, Tender X provides a higher quality of (brief details) and level of (% for
X and % for Y) and provides better value for money than tender Y, and is
therefore preferred on this criterion. Any qualitative arguments would be
supported by quantitative analysis.
Least Cost Method
Least Cost or 'first past the post' involves selection of the lowest reasonable
priced tender or cost item that meets all mandatory requirements and is
otherwise acceptable. If an item or tender does not meet all mandatory criteria it
is passed over and no longer considered. Where the procurement involves more
than just meeting base requirements, this may not identify the tender providing
the best value for money.
Normalising
RFT documents should nominate the terms required to enable proper
comparisons of tenders. Where potential service providers have submitted
tenders subject to disparate terms and conditions it is necessary to ‘normalise’
the competing tenders so that there is a common basis for comparison.
Adjustments may be required for the following terms by assessing their effect on
relative tender prices or the relativity of other criteria, for example:









firm versus variable prices; 
different price variation provisions; 
Free into Store (FIS) versus Free on Board (FOB); 
where maintenance/training and the like are included or excluded; 
differing warranty periods; 
PWM-0633, Issue – Edition 3 – Version 1.00 - Tendering Manual – August 2010
1
NSW Department of Finance and Services
Tendering Manual Appendix 7A – Evaluation Methodologies


differing settlement discount arrangement; and 

differing payment arrangements. 
Matrix Selection Method
This method is used where it is necessary to evaluate using a number of variables
or criteria. The method combines a numerical scoring system and criteria
weighting to identify the level of importance of each of the evaluation criteria.
The score is a measure of the level of compliance or merit whilst the weighting
identifies the relative level of importance of each evaluation criteria.
The first step is to identify and assign a weighting to each of the measured
evaluation criteria. As the desirability of the criteria increases the weighting of the
criteria increases. The weighting could be determined by considering the relative
importance of each criterion using a paired comparison matrix.
The individual weightings would be decided prior to the release of any RFT
documents and the percentage weighting for the price component (eg. 70%) and
the percentage weighting for the non-price component (eg. 30%) included in the
RFT documents. The breakdown of the weighting for each non-price criterion
would not normally be provided in the RFT documents. Mandatory “pass or fail”
criteria such as financial capacity would not be weighted or scored against.
There should be a balance between more and less desirable evaluation criteria
and their weightings so that innovations such as new product types, alternatives
and other innovative offers, are given due consideration.
Note that too many mandatory evaluation criteria might exclude reasonable
potential tenderers from responding.
The comparison of tenders is achieved by using a scoring or merit scale to rate
each tenderer’s level of compliance and relative value (as with the numerical
scoring methods described below). These scores are then multiplied by the
weighting of each criterion to provide a weighted score. For example a 0-5
scoring scale is used below. All tenders are assumed to have met any mandatory
(pass or fail) criteria in this example.
Matrix Selection Table
Criteria
A
Tender 1
Tender 2
Tender 3
B
C
D
E
F
G
H
Weighting
Score
Weighted
Score
Score
Weighted
Score
Score
Weighted
Score
1.
10
3
30
3
30
5
50
2.
6
5
30
3
18
2
12
3.
4
4
16
3
12
5
20
Total
20
76
60
PWM-0633, Issue – Edition 3 – Version 1.00 - Tendering Manual – August 2010
82
2
NSW Department of Finance and Services
Tendering Manual Appendix 7A – Evaluation Methodologies
To calculate the weighted score for each tender:





Column B multiplied by column C equals the weighted score for Tender 1; 
Column B multiplied by column E equals the weighted score for Tender 2; 
Column B multiplied by column G equals the weighted score for Tender 3. 
On the face of it, Tender 3 (total weighted score of 82) appears the best,
notwithstanding the tender has performed poorly with criterion two.
The evaluator would need to apply judgement to decide whether this tender is in
fact the best given the variability of scores and sensitivity of the result to
changes.
Sensitivity Analysis
Sensitivity testing allows key factors that determine the outcome to be identified
by showing how sensitive the preferred outcome is to changes in the criteria
weightings and other variables.
If a slight variation in the scoring causes a disproportionate change in the
evaluation results, then the significance of the evaluation scores should be
questioned. Sensitivity analysis could include a variation in criteria weightings of
up to 5%. The extent of change within the range of reasonable criteria weightings
identifies the order of certainty and significance of the total weighted scores.
If altering weightings and scores within a reasonable range with the chosen
methodology does not change the overall result, then the overall evaluation score
outcome is likely to be reliable.
Scoring Tenders
Detailed evaluation requires scoring or otherwise comparing/evaluating tenders
against each of the evaluation criteria (other than cost, which is scored using the
relative prices and costs assessed). To score, you must choose an appropriate
scale and define an objective method for assessing each tender’s score.
Tailor the scale range used in any particular tender scoring to reflect the nature
of the tender process and proposed contract. Sometimes a scale of 1 to 5 is
appropriate, where “1” represents non or poor compliance (conformance) and “5”
exceeds requirements, such as:
5 points — Meets or exceeds the requirement in all respects;
4 points — Meets the requirement but may be marginal in minor aspects;
3 points — Meets the requirement except for minor aspects;
2 points — Does not meet the requirement, but may be adaptable or made
acceptable;
1 point — Does not meet the requirement to a major degree.
In other cases, a broader range of scoring is appropriate, reflecting finer degrees
of differentiation between tenders and their merit.
A scale from 1 to 10 is generally considered convenient, where the relationship
between the score and its relative measure of compliance could be interpreted as
follows:
PWM-0633, Issue – Edition 3 – Version 1.00 - Tendering Manual – August 2010
3
NSW Department of Finance and Services
Tendering Manual Appendix 7A – Evaluation Methodologies
0
Non-compliance or poor response. Well short of requirements and includes
unsubstantiated claims about capability.
1-2
Unsatisfactory response. Does not meet minimum requirements or is
inadequately substantiated.
3-4
Marginal compliance. Contains certain material deficiencies that prevent
full compliance with requirements.
5-6
Satisfactory compliance. Contains minor deficiencies preventing full
compliance.
7
Full compliance. Adequate response to requirements and appropriately
substantiated.
8
Exceeds compliance. Very satisfactory response; more than adequate
response to requirements and well substantiated.
9
Significantly exceeds requirements. Excellent response; surpasses all
requirements and is fully substantiated.
10
Superior or outstanding compliance. Well exceeds the specified
requirements, for example, a new breakthrough technology.
Another method of scoring tenders (see the Consultant Management System User
Guide, Aspects of a Tender Evaluation Plan) uses the following:
100
Meets all requirements of an ideal proposal.
90
Meets most requirements of ideal proposal.
80
Meets many of the requirements of ideal proposal.
70
Meets a number of the requirements of ideal proposal.
60
Meets requirements, but only just satisfactory for this criteria.
<60
Fails to meet the requirements. May reject this proposal.
When scoring tenders, consider each tender against one criterion before moving
on to the next. This focuses attention on the individual criterion and helps ensure
that a score is not confused by other aspects of the tender.
To assess a tender score, refer to the list of questions or sub-criteria for each of
the criteria. For example, one of the criteria may have six elements to look for in
the tender. If (using the scale 1 to 10) there is little or no evidence of their
coverage, then assign a score of “0”. If some elements are covered and others
not, then a score of “3 to 6” may apply. If all six are well covered, then a score of
“7 or 8” is warranted. If the tenderer clearly offered more than required, then
score it “9 or 10”.
The scores on each of the criteria are then weighted, and a total weighted score
calculated for each tender. Together with the assessed price or estimated total
procurement cost converted to a weighted score, these results are used to
determine which two or three tenders to shortlist or single preferred tender to
identify for final assessment.
The final assessment of tenders should focus on:



verification of the tenderers’ capability and credentials through independent
checks, interviews of referees, and presentations to client representatives (as
needed); 
confirmation of their preparedness and ability to enter into a satisfactory
contract; and 
PWM-0633, Issue – Edition 3 – Version 1.00 - Tendering Manual – August 2010
4
NSW Department of Finance and Services
Tendering Manual Appendix 7A – Evaluation Methodologies

confirmation of the details of their offers, particularly costs and deliverables
as needed. 
As a result of this final assessment, the scores awarded during the detailed
evaluation may be modified to reflect new or revised information.
The evaluation effort may be reduced by culling tenders based on a mandatory or
minimum performance standard set for some or all of the evaluation criteria. The
RFT documents must clearly specify what mandatory levels of performance apply
with any criteria.
Paired Analysis
Paired analysis is a contemporary, useful way to both evaluate tenders and
establish evaluation criteria weighting’s. The basic approach is to compare all
tenders (or items to be weighted) two at a time, and determine which of each
pair is the better or more important.
An example is shown in Table 1 below.
Table 1
Paired Analysis for One Criterion
Tender
A
B
C
D
E
Red Company
A
Yellow Company
B
B
Blue Company
C
A
B
Green Company
D
A
B
C
Gold Company
E
E
E
E
E
2
3
1
0
4
20
30
10
0
40
Capability Criterion Total
%
In Table 1, five tenders (with ten paired combinations) are evaluated. The table
shows the results for one criterion (in this case, Capability), and each criterion
has its own table. The Total shows the number of times that each tender was the
better of a pair. The % is the normalised score for Capability that can then be
weighted.
PWM-0633, Issue – Edition 3 – Version 1.00 - Tendering Manual – August 2010
5
NSW Department of Finance and Services
Tendering Manual Appendix 7A – Evaluation Methodologies
A refinement of this technique, called Qualitative Differentiation, includes an
indicator of how superior the better of a pair is. Table 2 illustrates the method
used to weight criteria. It shows both the more important criteria of each pair
(the letter) and the extent to which it is more important by adding a number as
follows:
1 = Marginally more important
2 = Moderately more important
3 = Significantly more important
If the method is used to rank tenders for a specific criterion, then the definitions
would be:
1 = Marginally better
2 = Moderately better
3 = Significantly better
The resulting values are summed to give totals and normalised by converting to a
percentage.
When there are few criteria, it is common for one to end up with a zero %. Since
this is a weighting, it must be changed to a positive number and the other
percentages then adjusted accordingly.
Different scoring methods give different results. Table 4 shows the results of
three different approaches to paired analysis using the example data from Table
2.
The approaches are:





Standard method used in Table 1. 
Qualitative differentiation used in Table 2, where relative importance is
assigned. 
Balanced scoring. This approach also uses relative importance, but applies the
balanced scoring in Table 3 for the relative importance. 
PWM-0633, Issue – Edition 3 – Version 1.00 - Tendering Manual – August 2010
6
NSW Department of Finance and Services
Tendering Manual Appendix 7A – Evaluation Methodologies
Table 2
Analysis Table With Qualitative Differentiation
Criteria
A
B
C
D
E
F
G
H
I
Functionality
A
Support
B
A2
Capability
C
C1
C1
Vision
D
A3
B2
C3
Technology
E
A1
E1
E1
E2
Viability
F
F1
B1
F1
F2
E1
Price
G
A1
B2
C1
D1
G2
F1
Contractual
H
A2
B2
C1
H2
H1
H1
G2
Risk
I
I3
B1
I2
I3
I2
I1
G1
I2
9
8
7
1
5
5
5
4
13
15.8
14.0
12.3
1.8
8.8
8.8
8.8
7.0
22.8
Total
%
Table 3
Scores Used in Balanced Scoring
Qualitative Difference
Better of Pair
Worse of Pair
1
5
3
2
6
2
3
7
1
Equal
4
4
PWM-0633, Issue – Edition 3 – Version 1.00 - Tendering Manual – August 2010
7
NSW Department of Finance and Services
Tendering Manual Appendix 7A – Evaluation Methodologies
Table 4 shows how the different methods of paired analysis result in different
weightings of the criteria.
Also notice the range of weightings. The standard paired analysis tends to
converge, qualitative differentiation accentuates differences, and balanced scoring
is roughly in between. Each approach has its strengths and, of course, an
average of them could be used.
The balanced scoring method has particular merit where there are few (four or
less) criteria or tenders, because each gets a score. All the methods are useful for
exploring the sensitivity of evaluation results.
Table 4
Comparison of Results from Different Methods
Standard
Total
A
B
C
D
E
F
G
H
I
5
5
5
1
4
4
3
3
6
13.9
13.9
2.8
11.1
11.1
8.3
8.3
16.7
Paired Analysis
% Wt 13.9
Qualitative
Rank
2
2
2
9
5
5
7
7
1
Total
9
8
7
1
5
5
5
4
13
14.0
12.3
1.8
8.8
8.8
8.8
7.0
22.8
Differentiation
% Wt 15.8
Balanced
Rank
2
3
4
9
5
5
5
8
1
Total
36
36
31
16
31
34
31
27
43
12.6
10.9
5.6
10.9
11.9
10.9
9.5
15.1
2
5
9
5
4
5
8
1
Scores
% Wt 12.6
Rank
2
Total Procurement Cost
It is crucial to tender evaluation that tender prices and costs are assessed fully,
accurately, and on a comparable basis for all tenders, particularly those in
contention. The concept of total procurement cost (otherwise known as whole of
life cost) offers an effective method for ensuring that all costs and benefits are
taken into account, and that all tenders are costed uniformly where operating,
maintenance and disposal cost criteria are involved.
Even in apparently straightforward situations, an item or procurement product
may have a cheaper initial price, but require more expensive maintenance or
more frequent replacement. To identify the best value for money with total
procurement cost, all of these factors must be considered.
PWM-0633, Issue – Edition 3 – Version 1.00 - Tendering Manual – August 2010
8
NSW Department of Finance and Services
Tendering Manual Appendix 7A – Evaluation Methodologies
Where the tenders are comparable in terms of their total post construction or
supply cost and product characteristics, a detailed total procurement cost
calculation is not required, the assessed contract price alone is enough to assess
relative value for money.
Determining the total procurement cost requires the assessing of all relevant
factors over the expected life cycle of the particular procurement products or
assets. This includes:





the tender price with assessed adjustments and loadings; 
other costs not covered by the tender price, such as operating, maintenance,
disposal and changeover costs; and 
costs savings. 
For capital acquisitions or assets, the total net cost to the client organisation of
the asset is estimated for each month or year over the life of the asset so a
discounted cash flow and net present value can be calculated.
The following are guidelines for use when assessing the total procurement cost:




















the total procurement cost is the net cost of the procurement after accounting
for all contract payments, other costs, and all benefits attributed to the
procurement, including estimates of the cost of the following for the estimated
product/asset life cycle: 

installation; 

operating; 

maintenance and support; 

spare parts; 

licence fees; 

rise and fall or cost adjustment with inflation; 

currency variations; 

contract management; 

quality control and other management; 

replacements; and 

disposals. 
the length of the life-cycle can be the same as the contract term, or a notional
asset life that should be assessed considering the following: 

the design lives nominated; 

experience of the client organisation, advice of relevant industry experts,
or other organisations regarding asset life and use; 

allowances for technological change and obsolescence in asset elements;
and 

the period that the tenderer will guarantee availability of spares,
maintenance and service for asset elements. 
where tenderers propose solutions that have substantial differences in the
asset life periods, consideration will need to be given to the replacement cost
for solutions offering shorter life periods; 
PWM-0633, Issue – Edition 3 – Version 1.00 - Tendering Manual – August 2010
9
NSW Department of Finance and Services
Tendering Manual Appendix 7A – Evaluation Methodologies

the likelihood of increased maintenance and operating costs, reduced
reliability, and reduced performance levels over extended life periods will also
need to be considered;

operating costs should include all personnel, supervision and client support,
including associated expenses, overheads and statutory costs; 


























maintenance costs should allow for any down time while maintenance is
carried out, including lost business and substitute facilities; 
the capital cost of any spare parts should allow for their estimated availability
over the asset life period; 
confirm with referees or advisers the level of operating and maintenance
costs, and the need for spare parts advised by tenderers; 
where a cost or benefit is exactly the same regardless of which tender is
selected, its inclusion in the procurement cost calculation will not make any
difference to the evaluation outcome, but its inclusion may be necessary or
desirable, however, to indicate the client organisation’s total cash flow and
funding requirements; 
identify everything to be covered by the tender price, by other suppliers, and
by client resources, ensure the RFT documents are clear on what costs the
tender is expected to cover, and confirm with the preferred tenderer its
tender covers all the required costs; 
where costs are not covered by the tender price, and the RFT documents are
unclear about their inclusion, allow an assessed cost in tender comparisons
and agree (confirmed in writing) a price for their inclusion with the preferred
tenderer; 
where a tenderer omits something by mistake and this is not documented and
clear in the tender, the tenderer should be obliged to stand by the tendered
price, and either confirm this in writing or, depending on the conditions of
tendering, withdraw the tender; 
compute all cost estimates from the same base date, for example, the
expected date of awarding the contract; 
estimate the effect of rise and fall or cost escalation for inflation, allowing for
any alternative formulas proposed by tenderers; 
estimate the effect of exchange rate variations, particularly where the foreign
content varies between tenders, seek advice on how to allow for these
variations, check the changes and differences with customs duty, and allow
for this in the estimates; 
include estimates for any options or changes that may be sought by the client
organisation at a later time, and where this affects tender relativity make an
allowance based on optional prices or the rates tendered for such variations; 
any discounted cash flow calculation should be based on the rate advised by
the NSW Treasury for this purpose; and 
confirm relevant uncertain details in the total procurement cost calculation
with the tenderers in contention, to make sure there is no misunderstanding
about the basis of the calculation. 
Value for Money Analysis
With all procurement, the objective is to obtain the best value for money,
sometimes within a fixed budget limitation. Deciding which tender gives the best
value for money requires the considering of a number of factors.
PWM-0633, Issue – Edition 3 – Version 1.00 - Tendering Manual – August 2010
10
NSW Department of Finance and Services
Tendering Manual Appendix 7A – Evaluation Methodologies
Other methods are outlined above and in the Consultant Management System
and Contracts User Guide, which involve weighting and scoring for price and nonprice criteria and adding the weighted scores to give totals that help identify the
best value for money tender.
The diagram below shows the total procurement cost and tender evaluation
scores for tenders A to G, and illustrates the effects of different factors on the
selection process.
When the tender evaluation results are shown in the way plotted in the diagram
below, a number of facts are more readily apparent:



tender A can clearly be passed over, as it has the highest cost and the lowest
score; and 
tender G is clearly the winner, as it has the lowest cost and the highest score. 
Suppose, however, that there are no tenders A and G:





if there is a minimum acceptable score, then tender E can be passed over; 
if there is a strict budget limit and/or the costs exceed the estimated
reasonable cost, then tenders B and D can be passed over; and 
either way tender B can be passed over as it is clearly beaten by D, which has
both a lower cost and a higher score, and tenders F and C with lower cost and
higher or equal score. 
After this process of elimination, assume the choice is between tenders C, F and
D. The question, then, is whether their higher evaluation scores justify the extra
costs of tenders D and F.
One way of doing this is to divide their scores by their costs, and select the
tender with the highest score per dollar of cost. This is called a value for
money or cost effectiveness ratio.
TOTAL
PROCUREMENT
A*
COST
B*
D*
BUDGET
F*
C*
E*
0
G*
MINIMUM
EVALUATION
ACCEPTABLE
SCORE
SCORE
PWM-0633, Issue – Edition 3 – Version 1.00 - Tendering Manual – August 2010
11
NSW Department of Finance and Services
Tendering Manual Appendix 7A – Evaluation Methodologies
A refinement of this approach is to examine the marginal cost of the extra scores.
A more effective method is based on relating the tender scores to that achieved
by the lowest scoring acceptable tender and the maximum possible score. This
method assumes that there is a diminishing return from additional technical value
and imposes limits on the extent to which higher technical and other non-price
scores can influence the final assessment of value for money.
To do this, one method is to calculate a Score Index (SI) for each tender. The SI
measures how much a tender is better than the “basic” tender within the bounds
of the maximum score possible. It is calculated as follows:
SI =
Tender Evaluation Score + Max. Possible Score
----------------------------------------------------Lowest Score + Max. Possible Score
The total assessed cost of each tender is then divided by its SI to obtain a
measure of its cost relative to its evaluation score, that is, its Relative Value Cost
(RVC). That is:
RVC =
Tender Cost
---------------Score Index
The tender with the lowest RVC is selected as offering the best value for money.
The following is an example of the RVC method.
Suppose that the costs and scores of tenders C, F and D in the previous diagram
are as follows, and that the maximum possible score is 100.
C:
Cost
$490,000
Score
65
F:
Cost
$500,000
Score
75
D:
Cost
$535,000
Score
85
PWM-0633, Issue – Edition 3 – Version 1.00 - Tendering Manual – August 2010
12
NSW Department of Finance and Services
Tendering Manual Appendix 7A – Evaluation Methodologies
The SI and RVC for each tender would be as follows:
C:
F:
D:
SI =
65 + 100
-------------65 + 100
= 1.00
RVC =
$490,000
-------------1.00
= $490,000
SI =
75 + 100
-------------65 + 100
= 1.061
RVC =
$500,000
-------------1.061
= $471,429
SI =
85 + 100
-------------65 + 100
= 1.121
RVC =
$535,000
-------------1.121
= $477,162
As tender C is the lowest scoring acceptable tender, its RVC is the same as its
cost, and the calculation is not really necessary.
On the basis of these calculations, the additional scores of tenders F and D
indicate better value for money than the lower priced tender C. Tender F offers
the best value for money. As long as the cost of tender F with contingencies is
affordable by the client (allowing for the extra cost risks with lower value
tenders), it should be accepted over tenders C and D.
Sensitivity Analysis
Before deciding which tender to accept, perform a sensitivity analysis on the
tender scores and the assessed prices or estimated total procurement costs.
For some tender processes, there is a degree of uncertainty surrounding the right
criteria weightings, scores, and assessed prices or estimating total procurement
costs. Sensitivity analysis tests the robustness of the tender evaluation, allowing
for this uncertainty. Its immediate objective is to indicate circumstances under
which the selected tender may not be chosen, how sensitive to changes with
uncertain elements the selection may be, and so help identify the best tender.
PWM-0633, Issue – Edition 3 – Version 1.00 - Tendering Manual – August 2010
13
NSW Department of Finance and Services
Tendering Manual Appendix 7A – Evaluation Methodologies
Sensitivity analysis can show how the tender evaluation outcomes are affected by
variations in each component. As the components are varied, the evaluation is
repeated and any change in the relative ranking of tenders is noted. This analysis
highlights:







those criteria which have the most critical impact on the ranking of tenders; 
whether small changes in the weighting of criteria or the scores awarded
make any difference to the ranking of tenders; 
the tender price components and cost assumptions that contribute most to
the estimates of total procurement cost; and 
whether small changes in the assessed tender prices or assumed costs makes
any difference to the ranking of tenders. 
If the outcome of the tender evaluation is sensitive to small changes to the
weighting of the criteria or the scores awarded, or to small changes in assessed
tender pricing or cost assumptions, then examine carefully the justification for
those weightings, scores and assumptions and the tender rankings proposed.
Where the overall scores of tenders are sensitive to changes and/or close, they
may be considered equivalent, and then the one with the lower cost may
determine the best value tender.
Tender evaluation recommendations must be based on justifiable and welldocumented grounds, to ensure the outcome is beyond doubt, supported by the
available evidence, and able to withstand independent scrutiny.
Remember that tender scoring and sensitivity analyses do not automatically
indicate which outcome is the correct one or even the one most likely to be
correct. The scoring is an indication of relative merit, subject to the variability of
the factors and component scores involved. Sensitivity analysis simply presents a
variety of possible outcomes, so the evaluation team can see the impact of a
range of possibilities and examine more closely factors that are most critical to
their recommendation. The analysis assists the evaluation team to make the
judgement about what tender gives the best value for money.
PWM-0633, Issue – Edition 3 – Version 1.00 - Tendering Manual – August 2010
14