Case-based reasoning for contractor prequalification
S. Thomas Ng
Department of Building and Real Estate, Hong Kong Polytechnic University, Hung Hom,
Kowloon, Hong Kong.
Nigel J. Smith
Professor of Construction Project Management, Department of Civil Engineering, University of
Leeds, Leeds, LS2 9JT, United Kingdom.
R. Martin Skitmore
Head, School of Construction Management, Queensland University of Technology, Gardens
Point Campus, 2 George Street, GPO Box 2434, Brisbane, Q 4001, Australia.
Abstract: The contractor prequalification process is characterised by the lack of an universally accepted system and
appropriate quantitative and qualitative information. This has led to the development of a number of proprietary
prequalification systems together with an over-reliance on human judgement for assessment in practice. To improve the
reliability and objectiveness of decisions being made, prequalification needs to be carried out on a more rational basis.
An emerging technology in artificial intelligence, namely case-based reasoning (CBR), appears to have high potential to
satisfy the specific characteristics of the prequalification domain. The aim of this paper is to demonstrate, through the
development of a prototype system, the practicality and suitability of CBR approach for prequalification.
Keywords: Case-based reasoning, contractor prequalification, experiential knowledge.
1. INTRODUCTION
Contractor prequalification is a widely used process intended to ensure a pool of competitive,
competent and capable contractors from which tenders may be sought. Despite their strategic
significance, existing prequalification processes have their limitations. Not all are fully
structured and the formulation of decision criteria depends on human judgement (Ng, 1997a).
The procedure for narrowing down the number of contractors to a short-list is usually carried
out on an informal basis (Merna and Smith, 1990). The information concerning contractors'
attributes consists of both quantitative and qualitative types (Ng, 1996), while the assessment
methods used for appraising qualitative information require a predictive judgement of the
experts (Nguyen, 1985). As a result, the current practice of prequalification does not guarantee
the selection of able and willing tenderers. This can have a significant effect on the
successfulness of construction projects.
Attention to date has been focused on making prequalification more rational in order to
improve the reliability and fairness of the decisions. Latham (1994) recommends the use of a
centralised computer-based system to standardise the prequalification tasks among the public
clients. A Data-Base Management System (DBMS) has been developed for prequalifying
contractors in practice (Department of Environment, 1992). Paradoxically however, the use of
DBMS not only fails to fully address the existing problems of prequalification, but also
increases subjectivity by restricting the range of decision criteria to that which might not fully
reflect the specific
requirements of the client and project.
Research efforts have been diverted to the development of Knowledge-Based Systems (KBS)
(Russell and Skibniewski, 1990; Ng and Skitmore, 1995) which, although designed to mimic
the problem solving process of experts, are weak in modelling ill-defined domains (Riesbeck
and Schank, 1989). This is a particular problem in prequalification since the construction
project requirements and client objectives are dynamic in nature and it is therefore difficult to
specify different sets of rules to cover every situation. Case-Based Reasoning (CBR) offers a
potential solution to this as it is particularly suited to domains that are not completely
understood or when the concept is open-ended (Kolodner, 1993).
To demonstrate the practicality and suitability of the CBR approach for prequalification, a
prototype system - EQUAL (denotes Expert QUALifier) - has been developed and evaluated.
In this paper, the architecture and key features of EQUAL are described.
2. CASE-BASED REASONING
CBR is an emerging technology in artificial intelligence that "solves new problems by adapting
solutions that were used to solve old problems" (Riesbeck and Schank, 1989: 25). This
approach offers a paradigm that is close to the way people solve problems (Barletta, 1991)
since, in practice, "human experts are not systems of rules, they are libraries of experiences"
(Riesbeck and Schank, 1989:15). When dealing with problems in the real world, people are
often reminded of a previous similar problem (Schank et al, 1994). Reasoning by reusing or
modifying experiences is, therefore, a powerful and frequently applied paradigm for human
problem solving (Aamodt, 1990).
In CBR, a 'cases' are defined as "contextualised pieces of knowledge representing an
experience" (Kolodner, 1993:13). Cases represent specific knowledge for specific situations.
They make explicit how a task is carried out and how a piece of knowledge is applied or what
particular strategies for accomplishing a goal were used. In this sense, a case represents the
experiential knowledge of the experts in itself. This allows the inference of the CBR systems to
be based directly on previous cases (Kolodner, 1987).
The mechanism of CBR is shown in Figure 1. The previous experience of decision makers is
extracted and stored as cases in the case-base. Given the details of a new case, the CBR system
searches its case-base for an existing case that exactly matches the input specification. The
solution of the retrieved case is used to solve the problem without further modification if the
new and retrieved cases are the same. If, however, there are no identical cases, the system
retrieves the cases that best match the input situation. Since the best matching cases may not
properly reflect the new problem, the solutions of the retrieved cases may need to be adapted
before they become useful. The new solution is used to solve the current problem, and it is then
stored in the case-base to improve the system.
Figure 1 Mechanism of case-based reasoning
Riesbeck and Schank (1989) have used graduate admissions as an example to illustrate that
some real world problems have no right answers, and no one will know if they have made the
right choice. This illustration parallels the construction prequalification domain. They suggest
that, although the decision makers can use rules of thumb (such as "always take the student
with the
highest GCSE score"), the actual decision-making process usually depends on a 'gut-feeling' of
the situation as a whole. Thus, decision-makers can cite arguments on both sides of the issue
and then make a choice that seems best at the time. Such reasoning often depends upon
memory.
In contractor prequalification, decision makers equally rely on experiential knowledge to
determine the decision criteria and assessing the capability of contractors. Empirical studies by
Ng (1996) indicate that clients select the decision criteria by referring to similar
prequalification systems designed for the previous projects (see Figure 2). If the recalled
systems were developed according to the same characteristics as the present client and project,
the same decision criteria could be employed. If not, the decision maker could adapt the criteria
to suit the particular circumstances of the present client and project.
Figure 2 Decision cycle for criteria formulation
3. ARCHITECTURE OF EQUAL
The CBR approach has been adopted for developing a computer-based prequalification system
- EQUAL. EQUAL is a prototype CBR system designed to perform the prequalification tasks.
It consists of five interrelated modules: criteria formulation, screening and reviewing, overall
suitability and final scoring, finance, and performance. In addition, system input and output are
provided to the users for interrogating any of these modules. Each of the five modules is
designated for storing and evaluating various prequalification data. They are also designed to
interact with one another.
Figure 3 shows the architecture of EQUAL and the interactions between its modules. The five
modules are represented as grey-shaded boxes. The white boxes and cylinders within each
module denote the processes or data stores respectively, while the boxes as shown in the system
input and output symbolise the information required and generated by the system.
Figure 3 Architecture of EQUAL
The system originates from the criteria formulation module where a set of appropriate criteria
for the subsequent assessment modules is determined. The user is asked to define the client,
project and prequalification objectives through the system input. Based on this information, the
module searches its case-base and retrieves a case (prequalification system) that can reflect the
stated objectives. If the retrieved case was based on different objectives, adaptation may be
needed. The prequalification system is adapted, and this solution is stored in the case-base for
future use. The recommended criteria are reported to the user as an output.
The user is required to enter the necessary contractors' information into the screening and
reviewing module when a contractor applies for inclusion. In addition, the year of the latest
annual accounts of the contractor has to be specified by the user. This is used for retrieving
financial information, which will have already been entered into the finance module by the staff
in finance department. The financial information is transfered to the screening and reviewing
module. All information relevant to the screening or reviewing process (depending on which
prequalification stage it is being used for) is assessed according to the decision rules. An
intermediate result is generated and passed to the overall suitability and final scoring module.
This result determines whether or not a contractor should be included for further assessment.
The reasons for rejecting or excluding a contractor are reported to the user through the system
output.
Finally, the contractors are assessed by the overall suitability and final scoring module. This
module co-ordinates with and obtains information from the screening and reviewing, finance,
and performance module. Any further details about the contractors are entered by the user
through the system input. When all the necessary information is gathered by this module, the
assessment can be carried out. For the overall suitability assessment, contractors who have
similar technical, managerial and financial capabilities are retrieved by EQUAL. Their
performance is reported to the user. Based on the performance of the applicant and the retrieved
contractors, the system recommends an appropriate category and size of work for the new
applicant. The decisions regarding the approved category and size of work are reported to the
user. At the same time, the information of the new contractor is stored for future reference.
When this module is used for the tender listing process, a hypothetical case is generated to
represent the ideal contractor for the specified project type and amount. The hypothetical case
is then used to identify how good a contractor is comparing with the ideal contractor (the
hypothetical case) for the project. Contractors on the appropriate approved list will receive a
similarity score. Contractors with highest scores are reported to the user via the system output.
The reasons for adopting a modular structure were to satisfy the actual functional requirements
of the users. Contractors' information is usually maintained by different divisions across the
client organisation. For instance, the finance department is responsible for the financial details;
the project team is required to provide the project information; whereas the rest of the
information is maintained by the contract team. Therefore, it would be more efficient if
separate modules were developed for data maintenance. The use of modular structure can also
improve the data security and integrity. Shimazu et al (1993) have asserted that a CBR system
cannot be used for highly confidential information in the absence of security control. By
controlling the access of users to a particular module, such as restricting the project team to
interrogate the finance module, sensitive or confidential information can be protected against
unauthorised retrieval.
While it is necessary to divide the case into modules, it is important to be able to reconstruct
the case. To enable the structure of the case to be preserved, links must be set up between the
relevant modules. The principle of linking the modules (case-bases) of EQUAL is similar to
that in a relational database management system. In EQUAL, the cases in each module were
labelled and stored according to the name of contractors. A link was then created by setting up
a pointer to another module. Since the name of cases for both modules were common,
information could be transfered from one module to another by simply referring to the name of
contractors.
The architecture of EQUAL was represented and developed using a PC-based CBR shell
ReMind(tm) version 1.0. The detailed design of each module in ReMind(tm) is described in the
following sections.
4. CRITERIA FORMULATION MODULE
This module aims to determine a set of decision criteria that best reflects the organisational,
project and prequalification objectives of the client. Appropriate criteria are formulated by
reusing and adapting (if required) the prequalification systems of other similar contracting
organisations. When developing this module, three issues were considered. First, the module
must be flexible enough to cover a range of unusual situations even if similar cases do not exist
in the case-base. Second, the
module should be able to retrieve the relevant cases from the case-base. Third, the module
should be capable of generating a new solution when the retrieved cases cannot fully reflect the
specific objectives of the client.
Case Representation
Aamodt and Plaza (1994) have pointed out that the problem of case representation is to decide
what to store in a case and to find an appropriate structure for describing case contents. For this
module, the case features included the category and size of work for which an existing
prequalification system was used; the type of organisation for which it was developed; and the
client, project and prequalification objectives the prequalification system was based upon
Russell and Skibniewski (1988). These features constituted the problem part of the case while
the solution part of the case was represented by the criteria and sub-criteria adopted in the
prequalification system.
Since a number of sub-criteria could be included in each prequalification system, representing
each of these sub-criteria as a unique feature would increase the size of the case-base. This can
affect the efficiency of the system as more memory may be needed for case storage, and a
longer time may be required for case input, retrieval and adaptation. To avoid this, the decision
criteria and sub-criteria were represented as a hierarchy of symbols.
As shown in Figure 4, the hierarchy was first split according to the two main prequalification
stages, that is the "standing list" and "tender list". Branching off from these two parent
categories were the relevant prequalification processes, including screening, overall suitability
assessment, reviewing, and final scoring. Decision criteria pertinent to each of these processes
were then defined. The lower level of the hierarchy was the decision sub-criteria.
Besides reducing the size of case memory, the use of symbolic hierarchy for representing
decision criteria and sub-criteria has two further purposes. First, additional categories can be
appended to the tree if necessary. Such flexibility is essential to the users as new criteria and
sub-criteria may emerge when cases are added to the case-base. Second, criteria or sub-criteria
of one parent category can be related to another through the multiple parentage facility of
ReMind(tm) if they are common to both. In Figure 4, criteria pertinent to the overall suitability
assessment were linked to the final scoring process. The linkage at an intermediate level, such
as financial capability or technical capability, implies that all corresponding sub-criteria below
the hierarchy become applicable to the final scoring as well. This not only improves the clarity
of the hierarchical tree, but also reduces repetition during the case representation stage.
Figure 4 Symbolic representation for decision criteria
As mentioned at the beginning of this section, a major consideration for this module was
flexibility. That means the module should not be restricted to certain common types of
organisation only. This is particularly important during the early stage of implementation while
the number of stored cases is relatively small. To resolve this problem, a symbolic hierarchy
for the types of organisation was created (see Figure 5). The types of organisation were
classified into three generic categories namely private, public and quasi-governmental. Each
category was split into a number of sub-categories or even further sub-categories. In this way, a
general-to-specific hierarchy was created. Although the hierarchy illustrated in Figure 5 may
not reflect all the possible categories of organisation, further categories may be added later.
Figure 5 Symbolic representation for types of organisations
The purpose of creating such hierarchy is to guide the system during similarity searching.
Consider the district council and the city council, since they were both classified as local
government organisations, they would be considered by the system as similar. If a new case
was district council, while there were no previous cases of a district council in the case-base,
county council may be retrieved as a close matching case. The multiple parentage facility in
ReMind(tm) also allows unusual generic categories to be compared with the existing cases. As
illustrated in Figure 5, the quasi-governmental client was defined as the second generic parent
of non-commercial, central government and local government organisations. When no quasigovernmental cases are available in the case-base, the system will regard its "children" as the
next most similar categories for similarity matching.
Indexing and Retrieval
The criteria are generated in stages. The module first determines the suitable criteria for the
screening and reviewing process. The criteria for the overall suitability and final scoring
process are then compiled. After the initial formation stages, a set of criteria is composed.
These criteria are then subject to final refinement. During this stage, historical prequalification
systems are retrieved according to the organisational, project and prequalification objectives.
The basic criteria are modified to reflect the requirements of the client and project.
To ensure suitable cases are retrieved at the relevant stages, different "views" were created in
ReMind(tm). The purpose of developing a different "view" is to facilitate multiple retrievals
using the same case-base. When a new "view" is created, the weighting for similarity matching
will be set to zero and the adaptation settings and formulae will be ignored. The use of various
"views" enabled a new similarity matching scheme to be created for each retrieval stage. More
importantly, only those cases that are relevant to a particular stage would be declared as stored
cases for case matching and retrieval. This eliminates the possibility of irrelevant cases being
retrieved.
Kolodner (1993) has postulated that the ability to retrieve relevant cases from the case-base
depends on the appropriate selection of retrieval mechanism. Nearest neighbour retrieval was
adopted in this module as the number of prequalification systems (cases) available did not
justify the use of an inductive approach. With nearest neighbour retrieval, the features should
be indexed for similarity matching. Since there was no exact basis for statistical analysis, the
indices were derived from the domain expert by critiquing the intermediate prototype. Miller
(1984) has described critiquing as discussing the pros and cons of the proposed approach as
compared to alternatives that might be reasonable or preferred. A prototype, with indices
assigned by the author based on engineering judgement, was presented to the domain expert. A
domain expert was then asked to critique and propose alternate indices for the next prototype.
The indices were adjusted according to the expert's recommendations.
Adaptation
The biggest challenge within CBR is to apply the solution of an existing case to a new situation
that is a close, but not an exact match (Kass, 1989). In this module, three types of adaptation
strategy are provided for the users. They are the null adaptation, critic-based adaptation and
generalised adaptation. Null adaptation could be used when the retrieved case is exactly the
same as the new one. The user can simply copy the solutions of the matching case to the new
case without any changes. When the retrieved case is not exactly the same as the new case,
critic-based adaptation could be used (Brown and Lewis, 1993). During the adaptation, the
system will prompt the user to accept the solutions of the retrieved case. If the users wish to
make changes to any solutions, they can simply jump to the next adapting field without
accepting the recommended solution. The user will then be allowed to enter an appropriate
solution according to the different circumstances between the new case and the retrieved case.
Finally, a generalised adaptation strategy is provided in this module. As discussed in the
previous section, some of the clients may share the characteristics of both public and private
clients. Examples of these are the quasi-governmental clients. For these clients, adaptation
from either the private client or public client, no matter how closely matched they are, may not
properly meet their distinct characteristics and requirements. For this reason, adaptation from
two or more cases may be required. Generalised adaptation is to combine the outcomes of the
cases. Currently, the user is allowed to select up to two cases for generalisation. Adaptation
formulae were developed in ReMind(tm) to perform the generalisation function. Such formulae
allow the module to work out the combined solutions for the new case.
5. SCREENING AND REVIEWING MODULE
The purpose of this module is to eliminate the inherently unsuitable contractors with the
reasons for rejection or exclusion being provided for the decision makers. The desire to
incorporate the explanation facilities justifies the use of decision rules. The decision rules were
developed in the formula editor of ReMind(tm).
Representation of Rules
The formula editor allows decision rules to be created diagrammatically by combining the data
fields and various formula functions provided. A typical decision rule namely "R2-FS:
Turnover" is presented in Figure 6. The rule was to determine if the turnover of the contractor
is greater than or equal to three times the applied price range. If it was not, a statement reads as
"insufficient turnover ..." would be returned as the answer for this rule. The upper part of each
rectangular box in Figure 6 shows the result during the assessment. For instance, "False" in the
">=" box represented that, after the evaluation, the turnover of the company was not found to
be greater than three times the applied price range. This approach was used to develop the
decision rules for this module.
Figure 6 A typical rule in the screening and reviewing module
Although the graphical formula editor is easy to understand and use, representing rules
diagrammatically with this editor becomes unmanageable when they get more complex. More
importantly, since some of the rules for the screening process could be applicable to the
reviewing process, representing rules in different levels of detail was considered as a more
efficient strategy. In this module, the rules were represented in four levels as shown in Figure
7. Here, financial stability is used as an example to illustrate the structure of rules in this
module.
Figure 7 Decision rules represented in four levels of abstraction
The top level (Level 0) included rules for identifying the global results for the screening and
reviewing processes. By considering the results of individual criteria at the next level, the rules
would recommend if a contractor is eligible for further assessment, or if any constraints should
be imposed. For example, if the contractor did not satisfy the requirements of financial
analysis, a recommendation for rejection would be suggested.
Rules at Level 1 were criteria-specific. The rules in this level were derived from the outcomes
of the rules generated in Level 2. The decision structures of the rules were based on the
decision trees. The results of this level could be used to provide the detailed justifications for
failing a contractor.
To determine the financial stability of a contractor, a number of detailed financial tests, such as
the turnover analysis and financial ratios analysis, need to be performed (see Figure 7). Level 2
contained rules for carrying out more specific tests. Rules at this level were normally specific
enough to produce results for the upper levels. However, there were some circumstances where
a further level of rules (Level 3) might be inevitable. Ratio analysis is a good example of this.
The reason was due to the fact that a number of specific tests, such as profit margin, current
ratio, and asset turnover, were involved in the analysis. The results generated at the third level
could be useful for other rules in the upper level, such as trend analysis in this case.
In ReMind(tm), the rules created at a lower level can be connected to and manipulated by the
rules at the higher level. The principle of this can be illustrated by Figure 6, where the "FS:
Turnover (-0)" was a formula field (as distinguished by a small rectangle box at the bottom
right hand corner of the tile) to collect information from another case-base. The details of this
formula are hidden unless the developer opens it for viewing. This approach enables a complex
rule to be developed in a more manageable way.
6. OVERALL SUITABILITY AND FINAL SCORING MODULE
The role of this module is to determine the suitability of contractors for inclusion into the
applied standing list and to propose the tender list for a project. The approach used for
determining the overall suitability of contractors was based on the proposal of Kolodner (1987).
Kolodner has suggested that when a failed case is recalled by the decision maker during the
assessment, the previous solution should be adapted to avoid similar errors being repeated. Put
into the prequalification context, bad solutions regarding the approved category and price range
can be reflected by the poor performance of the approved contractors. In regard to which
contractors should be included in the tender list, an assessment was based on the degree of
matching between the approved contractors and the hypothetical ideal contractor for the
project.
Case Representation
A major concern for this module was how to manage a large amount of features systematically.
Apart from the information essential to the overall suitability assessment, further contractors'
information must be imported to this module for the final scoring process. The problem was
compounded by the existence of causal relationships among these features. To allow the causal
relationships to be represented effectively, a qualitative model as illustrated in Figure 8 was
created through the Q-model editor of ReMind(tm).
Figure 8 Causal relationships of case features and decision criteria
In this model, the case features were reassembled to the thirty-five predetermined decision
criteria. The causal relationship was defined diagrammatically in the Q-model editor. The
example in upper part of Figure 8 shows that the five case features have a positive causal
relationship to an intermediate outcome namely "financial stability (overall suitability)", and
this together with the "financial stability (ave)" (that is the average score for financial
performance), in turn, has a positive relationship with the decision criterion financial stability.
Besides positive relationships, an inversely proportional relationship can also be represented in
the Q-model. The second half of Figure 8 shows that the number of accidents has a negative
relationship (denoted by a negative sign) to the health and safety factor of the contractor. Some
features might have a more significant relationship to the decision criteria than the others. To
represent the significance of the features, weightings were assigned to the links. These
weightings were derived through expert's critique.
Indexing and Retrieval
The initial retrieval for both the overall suitability process and the final scoring process was
carried out by template retrieval. Different templates were set up according to the category and
size of work. Contractors who were not approved for a particular category or size of work
would be eliminated during this retrieval process.
Contractors who had been retrieved by template retrieval were then assessed according to the
nearest neighbour mechanism. The indices used for nearest neighbour mechanism were based
on the information collected from an earlier study (Ng, 1996). However, the user is allowed to
modify the indices as necessary. Maher and Balachandran (1994) have claimed that the use of
flexible indices can increase the robustness of case retrieval in real world domains. The
selected indices would then be used for computing the similarity scores.
Adaptation
Adaptation might be required during the overall suitability process. Parameterised adaptation
was provided in this module. Adaptation rules as proposed by Bailey and Smith (1994) were
developed to guide the adaptation process. The adaptation was based on the performance of the
new contractors and the similar contractors. The rules were that if the overall performance of
the new contractor was below an average of 0.5, the contractor would be rejected. For a
contractor who has an overall performance of over 0.5, the performance of the similar
contractors would be examined. If the performance of the similar contractors were above 0.5,
the new contractor might be included on a probationary basis.
7. OTHER MODULES
Although the three modules as described above can be regarded as the main components of
EQUAL, the significance of the performance module, finance module and system input and
output should not be overlooked.
7.1 Performance Module
The performance module was developed to store the performance-related information of the
contractors. Most information relevant to this module was fuzzy or symbolic in nature (Ng,
1997b). This kind of information was represented symbolically in ReMind(tm) through the
symbol editor. Symbolic values, such as "good", "satisfactory" and "poor", were first defined in
the editor. To enable the system to calculate a similarity score during similarity retrieval, the
symbolic values must be ordered. This could be done by assigning the priorities to the symbolic
values in the symbol editor. In this example, "good" ranks higher than "satisfactory" which in
turn has a higher rank than "poor".
Ideally, the ordered symbols should be modelled in Q-model within the performance module.
However, since the value for the overall performance for each project must be transfered to the
overall suitability and final scoring module for further assessment, two serious problems
appeared. First, ordered symbols could not be transfered from one case-base to another.
Secondly, when a qualitative model was developed, one or more virtual Q-nodes (can be
thought of as a new intermediate field for holding the averaged total values of several fields)
would be created. Also, the virtual Q-nodes created in the performance module could not be
transfered to another module. These problems were due to the limitations of ReMind(tm).
To enable the performance score to be transfered to the overall suitability and final scoring
module, the symbolic values in the performance module had to be left unordered. Rank
ordering was assigned in the overall suitability and final scoring module by creating the same
symbolic values as used in the performance module. Unordered symbols were transfered from
the performance module to the overall suitability and final scoring module. These symbols
were transformed into text through the text functions provided by the formula editor. The text
value allows the system to find an existing ordered symbol that matches it and converts it back
to a symbolic value for assessment. The ordered symbolic values transfered from the
performance module were then modelled by the Q-model in the overall suitability and final
scoring module (see Figure 9).
Figure 9 Modelling contractors' performance in the Q-model of ReMind(tm)
The rectangular boxes at the left-hand side of the Figure were the sub-criteria used for
depicting contractors' financial performance. Information from three recent project reports
would be used for this assessment. The financial performance for each project was summarised
(see the second boxes from the left of the Figure). They were then averaged to determine the
average score for the financial performance of the contractors (see the third box from the left of
the Figure). This score together with the averaged scores from other performance-related
criteria would become the score for the overall performance of the contractors (the right box in
the Figure).
7.2 Finance Module
The finance module is responsible for storing the financial data of the contractors and
converting the raw data into useful financial ratios for use in other modules. This module is
perhaps the least complicated one in terms of case representation. Almost all data fields, except
the name of the
contractor, the year of account and a confirmation for audited account, were numerical in
nature. In ReMind(tm), a similarity score is automatically generated in each numerical field,
such as contractors' turnover, according to the mean and standard deviation of the stored values.
Therefore, no further models had to be developed to manipulate the information as would be
required for symbolic fields.
Despite this, the similarity scores for most of the raw data, such as total asset or total current
liabilities, are meaningless to the assessment. This data must be transformed into relevant
financial ratios before a similarity score is computed. In this module, formulae of the required
financial ratios were represented in the formula editor. The numerical values derived by these
formulae would, in turn, be used to generate similarity scores.
7.3 System Input and Output
The system input and output provide the users with an interface for interrogating the system.
System input included various on-screen forms for the entry of new cases and the maintenance
of the existing cases. These forms were developed using the form editor of ReMind(tm). The
system input forms were arranged into sections and the case features were grouped under the
appropriate sections. To improve the legibility and user-friendliness, the features were
converted to a question-answer format. Prompts were added to the system input forms to assist
the users in answering the appropriate questions.
The output of the system is essential for decision support. Although the current prototype of
EQUAL cannot produce structured reports, outputs significant to decision making are
available. In the criteria formulation module, a report regarding the recommended decision
criteria for the specified type of client and project is provided to the user. This report enables
the client to prepare the prequalification questionnaire based on the recommended criteria. In
the screening and reviewing module, the recommendation as to whether a contractor should be
included in the next assessment process and the reasons for rejection or non-inclusion are
reported. This information is useful to the client and contractors as it can be used to justify the
decisions. The output of the overall suitability and final scoring module is the recommendation
for the approved category and size of work a contractor should be included on a standing list
after the overall suitability assessment. On the other hand, the names and scores of the best
matching contractors generated by this module can be used for compiling the required list of
tenderers. The output generated by EQUAL should assist the decision makers in performing
the prequalification tasks more effectively and efficiently.
8. VALIDATION OF EQUAL
Validation determines if the right system was built. The validation of EQUAL took two forms:
a Turing test and a face validation. A Turing test validates a system by asking the independent
domain experts to evaluate the results provided by experts and results from the designed system
without knowing the performers' identity (O'Keefe et al, 1987; Green and Keyes, 1987). This
can eliminate any prejudice, for or against, using a computer system. A group of semi-experts
were selected to take part in this test. Each of them was presented with two scenarios: one a
civil engineering job and the other a maintenance job. The same scenarios were entered to the
criteria formulation module of EQUAL. The outputs of EQUAL were manually transfered to
the
answering sheet as used by the semi-experts. Two independent experts experienced in
contractor prequalification were invited to assess the outputs. The results indicated that the
solutions generated by the current prototype of EQUAL were generally more accurate than the
semi-experts.
A face validation (O'Keefe et al, 1987) was carried out to identify the performance of EQUAL.
During the face validation, a series of short demonstrations was undertaken by the author, and
the domain experts were asked to use their knowledge and intuition to subjectively compare the
system's performance against that of human experts (Anick, 1993). The participants of face
validation included the experts who were or were not involved in the knowledge acquisition
process. The experts in the face validation indicated that CBR was a suitable approach for
modelling the prequalification tasks. The performance of EQUAL was highly satisfactory to
the experts involved in the face validation.
9. CONCLUSIONS
A novel conceptual framework was proposed by the authors for the development of a
specification for computer-based system for contractor prequalification. The framework
consisted of system input and output and five interrelated modules: the criteria formulation
module, screening and reviewing module, overall suitability and final assessment module,
finance module, and performance module.
A new form of computer-based DSS for contractor prequalification was developed using the
CBR approach. The prototype system - EQUAL - was developed, according to the established
conceptual framework, using a PC-based CBR tool called ReMind(tm). The system begins by
prompting the users to enter information regarding the client and project. It then searches its
case-base for a similar case; that is a prequalification system of another client and/or category
of work. Cases which best match the input information are retrieved, and if necessary, adapted
to reflect the present situation. New solution, in terms of decision criteria for each assessment
process, is generated by EQUAL for decision support.
EQUAL demonstrates that a practical solution can be produced even when the knowledge
about a particular prequalification system is weak. The solutions obtained from previous cases
can be modified to meet the current situation through the adaptation functions provided in the
system. Both subjective and objective information can be manipulated in EQUAL to provide
decision support. The prototype decision system tests indicate that the requirements of the
prequalification domain can be addressed through the development of a CBR system.
ACKNOWLEDGMENT
This research is funded by the Engineering and Physical Science Research Council (EPSRC)
under the grant number GR/J16121. The authors would like to thank UMIST, University of
Leeds, Queensland University of Technology, Hong Kong Polytechnic University and all
industrial collaborators for providing the requisite supports and information for this research.
REFERENCES
Aamodt, A., Knowledge-Intensive Case-Based Reasoning and Sustained Learning,
Proceedings: ECAI-90, Ninth European Conference on Artificial Intelligence, August 6-10,
Stockholm, Sweden, L.C. Aiello (ed.), Pitman Publishing, 1990, pp. 1-6.
Aamodt, A., Plaza, E., Case-Based Reasoning: Foundational Issues, Methodological
Variations, and System Approaches, AI Communications, Vol. 7, No. 1, 1994, pp. 39-59.
Anick, P.G., Integrating Natural Language Processing and Information Retrieval in a
Troubleshooting Help Desk, IEEE Expert: Intelligent Systems and Their Applications, Vol. 8,
No. 6, December, 1993, pp. 9-17.
Bailey, S.F., Smith, F.C., Case-Based Preliminary Building Design, Journal of Computing in
Civil
Engineering, ASCE, Vol. 8, No. 4, October, 1994, pp. 454-467.
Barletta, R., An Introduction to Case-based Reasoning, AI Expert, August, 1991, pp. 43-49.
Brown, S.J., Lewis, L.M., A Case-Based Reasoning Solution to the Problem of Redundant
Resolutions of Nonconformances in Large-Scale Manufacturing, In Innovation Applications of
Artificial Intelligence 3, R. Smith & C. Scott (eds.), AAAI Press, 1993, pp. 121-133.
Department of Environment, CMIS, Contractor Management Information System, The
Department of Environment, 1992.
Green, C.J.R., Keyes, M.M., Verification and Validation of Expert Systems, Proceedings:
WESTEX87, Western Conference on Expert Systems, Institute of Electrical and Electronics
Engineers, 1987, pp. 38-43.
Kass, A., Adaptation-Based Explanation: Explanations as Cases, Proceedings: Sixth National
Workshop on Machine Learning, June 26-27, Cornell University, Ithace, New York, Morgan
Kaufmann Publisher Inc., 1989, pp. 49-51.
Kolodner, J., Extending Problem Solver Capabilities Through Case-Based Inference,
Proceedings: Fourth International Workshop on Machine Learning, University of California,
Irvine, June 22-25, P. Langley (ed.), Morgan Kaufmann Publishers Inc., 1987, pp. 167-178..
Kolodner, J., Case-Based Reasoning, Morgan Kaufmann Publishers Inc., 1993.
Latham, M., Constructing the Team: Final Report of the Government/Industry Review of the
Procurement and Contractual Arrangements in the UK Construction Industry, HMSO, July,
1994.
Maher, M.L., Balachandran, M.B., Zhang, D.M., Case-Based Reasoning in Design, Lawrence
Erlbaum Associates, Inc., 1995.
Merna, A., Smith, N.J., Project Procured by Privately Financed Concession Contract, UMIST,
1994.
Miller, P.L., A Critiquing Approach to Expert Computer Advice: ATTENDING, Research
Notes in Artificial Intelligence 1, Pitman Advanced Publishing Program, 1984.
Ng, S.T., Skitmore, R.M., CPDSS: Decision Support System for Contractor Prequalification,
Civil
Engineering System, Vol. 12, 1995, pp 133-159.
Ng, S.T., Smith, N.J., Skitmore, R.M., Case-Based Reasoning for Contractor Prequalification A Feasibility Study, Developments in Artificial Intelligence for Civil and Structural
Engineering (ed. B.H.V. Topping), Civil-Comp Press, 1995, pp. 61-66.
Ng, S.T., Case-Based Reasoning Decision Support for Contractor Prequalification, A Thesis
Submitted to the University of Manchester Institute of Science and Technology for the Degree
of Doctor of Philosophy, April, 1996.
Ng, S.T., Skitmore, R.M., Smith, N.J., The Viability of Rationalising the Decision Criteria for
Contractor Prequalification, Construction Management and Economics, 1997a, In Press.
Ng, S.T., Smith, N.J., The Subjectivity of Contractor's Information and Assessment Methods
for
Contractor Prequalification, Construction Management and Economics, 1997b, In Press.
Nguyen, V.U., Tender Evaluation by Fuzzy Sets, Journal of Construction Engineering and
Management, ASCE, Vol. 111, No. 3, 1985, pp. 231-243.
O'Keefe, R.M., Balci, O., Smith, E.P., Validating Expert System Performance, IEEE Expert,
Winter, 1987, pp. 81-87.
Riesbeck, C.K., Schank, R.C., Inside Case-Based Reasoning, Lawrence Erlbaum Associates
Publishers, 1989.
Russell, J.S., Skibniewski, M.J., Decision Criteria in Contractor Prequalification, Journal of
Management in Engineering, ASCE, Vol. 4, No. 2, 1988, pp. 148-164.
Russell, J.S., Skibniewski, M.J., Qualifier-2: Knowledge-based System for Contractor
Prequalification, Journal of Construction Engineering and Management, ASCE, Vol. 116, No.
1, March, 1990, pp. 157-171.
Schank, R.C., Kass, A., Riesbeck, C.K., Inside Case-Based Explanation, Lawrence Erlbaum
Associates, 1994.
Shimazu, H., Kitano, H., Shibata, A., Retrieving Cases from Relational Data-Bases: Another
Stride Towards Corporate-Wide Case-Base Systems, Proceedings: Thirteenth International
Joint Conference on Artificial Intelligent, August 28 - September 3, Chambery, France, Morgan
Kaufmann Publishers Inc., 1993, pp. 909-914.