BPI
Best Procurement Implementation
Best Practice Toolkit
Evaluation matrix user guide
Contents
•Engage the Commodity Team
•Components of the evaluation matrix
•Weights
•Sub-Weights
•Evaluating the ability to supply a national contract
•Evaluating the total cost of supply
•Scoring and benchmarking the suppliers
•The evaluation matrix as a worksheet in excel
2
Engage the Commodity Team
EVALUATION CRITERIA - DRAFT
SPECIFICATION
EXAMPLE
EXAMPLES OF
MEASUREMENT
PRICE
?Total cost of Acquisition
?The delivered price quoted by a
supplier in relation to the lowest
delivered price by any supplier
?Cost structure break-down through open
book costing
?Benchmarking
DELIVERY
?order to receipt lead time
?Flexibility (changing delivery
schedules and methods)
?Packaging ( bar-coding/delivery
notes/ invoicing)
?Benchmark
meets NHSS requirements
?Benchmark on Turn-around time to meet
varying Health Board delivery requirements
?Capability to meet NHSS requirements
QUALITY
?Fitness for purpose
?Conformance
?Returns
?Customer Complaints
?Accreditation
?Environmental
?Sustainability
?Past performance
?Investment and absorption of nonrecurring costs (e.g. implementation
costs)
?Customer service
?Account management
?Open-book costing
?Benchmarking
?Past performance records
?References from other customers
?NHSS proposal - Business plan and
organisation
structure
(including
implementation
and
management
information)
Fit/comfort
Sensitivity
Grip/dexterity
Ease of donning
3
-1
-1
1
-1
-1
1 1
1
-1
-1 -1
1 -1
-1 1
1
2
4
2
4
6
%
-1 1 1 3
1 -1 1 1
1 1 -1 -1
1 1 1
-1
1 -1
-3
-1 -1
Weight
Ease of donning
Sensitivity
Grip/dexterity
Rank
Durability
Total
Strength/Protection
Fit/comfort
AFTER-SALES
?Certificate
Durability
•Introduce the decision
matrix concept
Strength/Protection
•The evaluation criteria was
agreed at a meeting with all
Commodity Team
8 27%
6 20%
4 13%
6 20%
4 13%
2
7%
30 100%
The components in the evaluation Matrix
After-Sales
Quality
Costing
Structure
Delivery
Commercial
7 KEY components in the evaluation matrix
4
Components of the evaluation matrix defined
Quality
P
E
R
F
O
R
M
A
N
C
E
After-sales
Total Quality Management System. This includes past performance data on achieving an ave
of 85% one day replacement turnaround/ complaints/ replacement and rejected goods process,
certification and customer quality awards.
The complete service offering. This includes dedicated staff/customer satisfaction/perf mgt
and uptake reporting/process improvements and cost reductions /Core & non-core list.
Costing
Structure
Ability to assess the suppliers total cost model. This includes open book costing and price
fixing.
Delivery
On time delivery and flexibility. This includes turnaround times/ the suppliers ability to meet
unscheduled requests/past performance data on achieving 85% or more on-time deliveries and
delivery point improvements.
Commercial
Commercial viability. This includes the effectiveness of the implementation plans/board level
support network/ financial stability and the level of the organisation’s influence over the supplier
(For example, negotiating power).
Ability to
supply a
national
contract
The ability of the supplier to supply to all delivery points in the preferred delivery option.
Price
The total price tendered including all discounts and additional costs.
5
How the evaluation matrix uses weighting
The next stage is to find the
weight for each component
in the evaluation matrix
Input Data
ABILITY TO SUPPLY A
NATIONAL CONTRACT
PERFORMANCE
After-Sales Quality
Weight
Supplier A
Costing
Structure Delivery
Commercial
Ability to Supply to all
Delivery Points
PRICE
TOTAL SCORE
Pricing with flexible Total Value of
delivery periods
Supply
10.00
5.00
10.00
5.00
10.00
20.00
40.00
5.63
0.80
2.00
0.38
1.27
13.33
38.79
100.00
62.19
Supplier B
Supplier C
The sub-weights will be
included in the score for each
performance criteria. Sub-weights
will be applied to each tender
question and summed into a total
%
All calculations in the evaluation matrix have been automated.
6
Each Team contributes to finding the weight
In the decision
matrix:
Commodity Team
asked to answer
‘Yes’ or ‘No’ on
what criteria is
more important
than another
Is quality more important than the ability to supply to all delivery
points?
Is quality more important than price?
Is the costing structure more important than delivery?
Is costing structure more important than commercial aspects?
After-sales
Quality
Costing Structure
Delivery
Is costing structure more important than the ability to supply to all
delivery points?
Commercial
Ability to supply to delivery points
Is the costing structure more important than price?
Is delivery more important than the commercial aspects?
Price
1
1
1
-1
1
1
-1 -1
1
-1
-1 1
-1 1
1 1
1 1
-1 1
1 1
-1 -1
1
-1
-1 -1
-1 1
%
Weight
Weight
-1 -4 6 2 5%
-1 2 3 8 19%
-1 -4 6 2 5%
1 3 2 9 21%
-1 -1 5 5 12%
-1 1 4 7 16%
4 1 10 23%
1
43 100%
-1
-1
-1
1
1
Specific questions have been designed to derive the opinion of each Team member
concerning the level of importance for each criteria
7
Rank
Total
Price
Ability to supply to
delivery points
Delivery
Commercial
Is the critieria BELOW more important
than the criteria on the right?
Quality
Is after-sales service more important than the ability to supply to all
delivery points?
Is after-sales service more important than price?
Is quality more important than costing structure?
Is quality more important than delivery?
Is quality more important than commercial aspects?
Costing Structure
Is after-sales service more important than quality?
Is after-sales service more important than the costing structure?
Is after-sales service more important than delivery?
Is after-sales service more important than commercial aspects?
After-sales
‘Yes’ replaced by 1,
‘No’ replaced by -1
-1
After-sales
Quality
Costing Structure
Delivery
Commercial
Ability to supply to delivery points
Price
STEP 2
Total for
each
criteria
1
1
1
-1
1
1
-1
-1
-1
1
1
-1
1
1
1
1
1
-1
1
-1
-1
-1
-1
1
1
-1
1
-1
1
-1
-1
-1
1
1
1
-1
-1
-1
1
-1
-1
-4
2
-4
3
-1
1
4
6
3
6
2
5
4
1
2
8
2
9
5
7
10
43
STEP 3
-
Sum of all the
negative
numbers in the
total column
=
Weight for each criteria
Enter the Team responses as a 1 or -1 into the decision matrix which will automatically
calculate the total/Rank/Weight/Weight %
8
Weight %
Weight
Rank
Total
Price
Ability to supply to
delivery points
Commercial
Quality
Is the critieria BELOW more important
than the criteria on the right?
After-sales
EVALUATION CRITERIA - DRAFT
Delivery
STEP 1
Costing Structure
Calculate the weight from the Commodity Team responses
5%
19%
5%
21%
12%
16%
23%
100%
Each Commodity Team member contributes to finding the
sub-weights Each Team member rates
each topic between 1 – 3,
STEP 1
(3 is critically important, 2 is
important and 1 is nice to have)
STEP 2
fx = (AVERAGE)
Organisation
A
Sales and technical
support through the life
cycle of the contract
Organisation
B
Organisation
C
●Find the average of all Team member
responses
Aver
age
STEP 3
3
3
3
3.00
Agreement to price
fixing
3
3
3
3.00
Dedicated account
manager
3
3
2
2.67
Level of the
organisation's
Negotiating power with
supplier
3
3
3
3.00
=
9
Sub-weight for each tender
question (the importance of each
tender question)
Each sub-weight will place a level of importance against
each tender question
Tender Question
Evaluating The Suppliers Response
Tenderers are asked to provide details of the sales and technical support
that will be available to the organisation during the life cycle of the
contract.
Step 1
SubWeig
ht
yes
1
3.00
3.00
yes
1
2.33
2.33
yes
1
1.33
1.33
Calculation
Has the supplier mentioned technical support, customer service support,
implementation support, the reporting and planning for regular
reviews both internally and externally?
Will there be dedicated staff?
Is the number of dedicated staff substantial enough for the contract?
Yes = 1
No =
0
Yes/No
At least 1 staff should be dedicated
Step 3
Step 2
Yes = 1
No = 0
x
10
Sub –
Weight
Score for
=
Each
tender que
Calculate the total score for each criteria
Evaluating The Suppliers Response
Yes/No
Has the supplier mentioned technical support, customer
service support, implementation support, the reporting and
planning for regular reviews both internally and externally?
At least 2 staff should be dedicated
Yes = 1
No = 0
SubWeight
Calculation
yes
1
1.83
1.83
no
0
1.00
0.00
1
2.56
yes
Total Score for After-sales
7.68
STEP 1
2.56
4.39
57.15%
Weight
Supplier A
Quality
Costing
Structure
Delivery
Commercial
10.00
5.00
10.00
5.00
10.00
5.71
3.00
1.13
7.30
3.21
All steps are automatic.
11
Calculate as a
percentage
STEP 3
PERFORMANCE
After-Sales
}
STEP 2
Sum the total score
Enter the criteria
specific percentage
into the evaluation
matrix
Calculation of weights and sub-weights have been
automated
Weight %
NO YES
NO YES
Western Isles
SAS
State hospital
Tayside
Shetland
Orkney
&
Is after-sales service more important than quality?
Is after-sales service more important than the costing structure?
Is after-sales service more important than delivery?
Is after-sales service more important than commercial aspects?
Is after-sales service more important than the ability to supply to all
delivery points?
Is after-sales service more important than price?
Is quality more important than costing structure?
Is quality more important than delivery?
Is quality more important than commercial aspects?
Is quality more important than the ability to supply to all delivery
points?
Enter Data
Lothian
28%
14%
0%
16%
12%
12%
19%
100%
Lanarkshire
12
6
0
7
5
5
8
43
Highland
1
4
7
3
5
5
2
Weight
Rank
Price
Total
6
0
-6
1
-1
-1
2
Glasgow
1
1
-1
-1
1
-1
-1
Grampian
1
-1
-1
1
1
Forth Valley
-1
1
Ability to supply to
delivery points
Commercial
Delivery
-1
-1
-1
1
1
-1
1
Fife
1
1
1
1
1
1
-1
Borders
Dumfries and
Galloway
1
1
Ayrshire and Arran
Blood Transfusion
Service
-1
-1
-1
1
1
Argyll And Clyde
1
-1
-1
-1
-1
-1
-1
Costing Structure
Quality
After-sales
WEIGHT
NO NO NO NO NO NO NO NO NO NO NO NO NO
Borders
Dumfries and Galloway
Fife
Forth Valley
Grampian
Glasgow
Highland
Lanarkshire
Lothian
Orkney
Shetland
Tayside
State hospital
SAS
Western Isles
2
1
3
3
2
1
1
1
1
3
2
2
2
2
2
1
3
Average
Blood Transfusion Service
AFTER-SALES
Sales and technical support through the life cycle of the contract
Is there dedicated staff
More than 2 dedicated staff
Dedicated account manager
Single point of contact
Professional level of the single point of contact
Ayrshire and Arran
Argyll And Clyde
SUB-WEIGHT
2
##
##
##
##
##
Just type ‘Yes’ or
No’ respectively for
each participating
organisation in the
yellow shaded
boxes
Just type the exact
rating that each
Commodity Team
Member has advised
in the yellow shaded
boxes
3 spreadsheets will automatically calculate the weight and sub-weight data
12
Measuring the ability to supply a national contract
Organisation
E
Organisation
D
Organisation
C
Organisation
B
Organisation
A
National Delivery Points
Score for all
commodities
Total for all
Geographies
Maximum
Score
Score for all
Geographies
All delivery points
yes
yes
yes
no
NO
4
18
22.22
All delivery options
YES
YES
YES
YES
YES
13
18
72.22
Stationary
yes
NO
NO
NO
NO
1
18
5.56
Computer Consumables
yes
NO
YES
YES
NO
14
18
77.78
Small Electronic Office Equipment
YES
YES
YES
YES
YES
18
18
100.00
Total
5
3
4
3
2
50
90
55.56%
Step 1
Suppliers answer
‘Yes’ or ‘No’ on
their ability to
deliver to each
delivery point
Step 2
Yes =1
50
Step 3
The spreadsheet
will calculate the
total score:
=
No =0
Total Score X 100
All calculations in the evaluation matrix have been automated.
Max Score
Sliding
\
Rating
The sliding rating system awards each supplier a score relative to their ability to meet the
delivery requirements.
13
Measuring the total cost of supply
ENTER DATA
24 Hours
(Normal
Delivery
Period)
Lowest
Supplier
Supplier
Supplier
Cost
A
B
C
48 Hours
£10,000.00
£11,000.00
£11,000.00
£12,000.00
£12,000.00
£12,000.00
£13,000.00
£25,000.00
Step 2
Step 1
Enter the total cost
(including all
discounts/charges)
for the each
turnaround time
Enter the
total amount
of the lowest
cost supplier
72 Hours
24 Hr
Scoring
£6,000.00
£6,000.00
£6,000.00
£9,000.00
48 Hr
72 Hr
Scoring Scoring
91
91
83
Step 3
The spreadsheet
will calculate the
% scoring:
Lowest Cost
X 100
Supplier Cost
100
92
48
Total Score
100
100
67
Percentage
Score
291
283
198
97.0%
94.4%
66.0%
Step 4
Sum and
find % total
score for
each
supplier
=
Sliding
\
Rating
The sliding rating system awards each supplier a score relative to the lowest cost. The
lower the cost, the higher the score.
14
Score and benchmark each supplier
The total score is the sum of all
the 7 evaluation components
Input Data
ABILITY TO SUPPLY A
NATIONAL CONTRACT
PERFORMANCE
After-Sales Quality
Weight
Supplier A
Costing
Structure Delivery
Commercial
Ability to Supply to all
Delivery Points
PRICE
TOTAL SCORE
Pricing with flexible Total Value of
delivery periods
Supply
10.00
5.00
10.00
5.00
10.00
20.00
40.00
5.71
3.00
1.13
7.30
3.21
13.33
38.79
100.00
72.48
Benchmark the tender score
by ranking each supplier
All calculations in the evaluation matrix have been automated.
15
RANKING
1