Knowledge Acquisition and
Modelling
Knowledge Acquisition and Elicitation
Ref: Knowledge Acquisition in Practice: A step by step guide, Milton, Springer-Verlag
Knowledge Engineering

Transfer View





Human knowledge transferred to knowledge base
=>knowledge exists and is accessible
Typically interviews and task execution and observation used
for KA
End result set of rules that exercise knowledge made explicit
Modelling View




Need to build models
Incremental, evolutionary process
Model is an approximation of reality
Models are subjective
unstructured interview
interviews
KA Typology
semi-structured
interview
structured interview
natural techniques
observation techniques
group meetings
questionnaires
card sorting
three card trick
rep grid technique
KA techniques
limited time
contrived techniques
constrained tasks
limited information
20-questions
commentating
teach back
laddering
process mapping
modelling techniques
concept mapping
state diagram mapping
Natural Techniques
Interview Techniques



Knowledge engineer asks questions of the expert or end user.
Essential method for acquiring explicit conceptualisations and
knowledge, but poor for tacit knowledge.
Variations:

Unstructured interview


Semi-structured interview



Free flowing, used in early stages of elicitation, can produce basics of
knowledge domain, basically broad chat
Main technique for elicitation
Pre-defined questions sent to expert prior to interview, supplementary
questions asked at interview. Can be used as part of validation.
Structured interview

Pre-defined set of questions, can simply be filling in a questionnaire at the
interview.
Interview Techniques

Dependent on




The questions asked
Ability of the expert to articulate the knowledge
Model built on knowledge elicited during interview
Model reviewed by the expert
Modelling Techniques
Modelling Techniques





Use of knowledge models with experts
Used as validation and refinement
Can show a basic model to an expert and prompt them
to modify.
Can show a complete model of knowledge provided by
one expert to a second expert to cross-validate.
Can create one from scratch with an expert – start with
a blank page
Model Based Knowledge Acquisition



Each model emphasizes certain aspects of the system to
be built and abstracts from others.
Each model is indicative of one view of the world
Models provide a decomposition of knowledgeengineering tasks:

while building one model, the knowledge engineer can
temporarily neglect certain other aspects.
Knowledge Modelling Process
Knowledge Modelling


Use skeletal models
Or generic tasks


Generic tasks are templates of problem-solving activities that
can be configured together to describe any intelligent activity.
Modelling Frameworks
Knowledge Modelling

At least five different types of knowledge are distinguished:

Tasks-goals


Problem-solving methods


describe the primitive reasoning steps in the problem solving process.
Ontologies


ways to achieve the goals described in tasks. In some knowledge
modelling frameworks, problem-solving methods define subtasks to which
other problem solving methods can be applied. We will call such a
decomposition a task instance.
Inferences


correspond to the goals that must be achieved during problem solving.
describe the structure and vocabulary of the static domain knowledge.
Domain knowledge

refers to a collection of statements about the domain.
Principles






Divide and conquer.
Configuration of an adequate model set for a specific
application.
Models evolve through well defined states.
The model set supports project management.
Model development is driven by project objectives and
risk.
Models can be developed in parallel.
Recommended Reading
Knowledge Engineering: Principles and Methods
Rudi Studer, V. Richard Benjamins and Dieter Fense
Data & Knowledge Engineering (1998)
Volume: 25, Issue: 1-2, Publisher: Elsevier
 http://www.hubscher.org/roland/courses/hf760/readings/st
uder98knowledge.pdf
Contrived Techniques
Knowledge Capture – Specialised
Techniques




Contrived Techniques
Primarily for deep, tacit knowledge
Involve the expert performing tasks they would not
normally do as part of their job.
Most of these techniques come from psychology
Knowledge Capture – Specialised
Techniques

Important types:








Concept (card) Sorting
Three Card Trick (Triadic)
Repertory Grid Technique
Constrained Tasks
20-questions
Commentary
Teach Back
Usually involve expert doing two types of task:

Tasks they normally perform


Commentary is useful here
Tasks designed to probe the expert

Concept sorting or Triadic
Concept (Card) Sorting

Way of finding out how an expert compares and orders
concepts



Works best in small groups
Simplest form is card sorting







Can reveal knowledge about classes, properties and relations
Collection of concepts (or other knowledge objects) are written on
separate cards
Cards sorted into piles by an expert in to piles - each card in a pile
must have something in common
Each time the cards are sorted it will be based on an attribute and
each pile will represent a value
Enables significant elicitation of properties and dimensions
Used to capture concept knowledge and tacit knowledge
Use in conjunction with triadic method
Can also sort objects or pictures instead of cards
Concept Sorting – How To ?







Decide what classes of concepts you want to explore (in
particular their properties – attributes and values)
Write the name of each concept on a separate card
At the session explain to the expert what is going to
happen
Ask the expert to name the piles
Write down (or record) the results of the sort
Collect the cards and ask the expert to sort again
Repeat until the expert can’t sort anymore
Triadic Elicitation Method (3 card trick)





Used to capture the way in which an expert views the concepts in a
domain.
Present three random concepts and ask in what way two of them are
similar but different from the other one.
Answer will give an attribute.
A good way of acquiring tacit knowledge.
How does it work ?

Select 3 cards at random

Identify which 2 cards are the most similar?




– Why?
– What makes them different from the third card?
Helps to determine the characteristics of our classes
Picking 3 cards forces us into identifying differences between them
 There will always be two that are “closer” together

Although which two cards that is may differ depending on your perspective
Triadic Elicitation – How To?









Explain to the expert that you are trying a technique to draw out
deeper knowledge
Collect all cards previously used
Shuffle cards and randomly select 3
Place them on the table, two close together one further away
Ask how the two close together are similar and the other different
Write down (or record) what the expert says using an attribute
Use the results to find another card sort to find the values of all
concepts for this attribute
If the expert can’t identify an attribute, just pick another 3 cards
Repeat until the expert can think of no more differences
20-Questions




Expert asks questions of the engineer
The Knowledge Engineer thinks of an object/concept in the domain
Expert asks yes/no questions to the knowledge engineer in order to
deduce an answer.
Knowledge Engineer





notes the questions and the order in which they are asked
need not know much about the domain, or have an answer in mind, just
answer “yes” or “no” randomly
The questions asked provide a good way of quickly acquiring
attributes in a prioritised order.
Can provide an insight into the key aspects, properties or categories
and their relative priorities.
Note that the main purpose of this exercise is not really to try and
find out what the Engineer is thinking of, but to determine the
important properties!
20-Questions – How To?










Decide on set of concepts you need to explore in more detail
Explain to the expert what is going on
Ask the expert to imagine that you the engineer have the same level of knowledge they
do about the set of concepts
Instruct the expert that they should ask the least number of questions to deduce the
answer
Engineer can only answer yes and no
Explain that the best way is to ask questions which split the concepts in half so that each
question eliminates half the possible solutions
Start
As each question is asked write it down (or record it)
When a number of questions have been asked take the expert back to an earlier
question and change the answer you gave to prompt the expert to ask further questions
After the session extract the attributes and values (or new concepts) from the questions
asked and these will be added to the knowledge base
Laddering





Involves the construction, modification and validation of trees.
Accessing personal construct system
Take a group of things and ask what they have in common
 Then what other ‘siblings’ (brothers/sisters) there might be
A valuable method for acquiring concept knowledge and, to a lesser
extent, process knowledge.
Can make use of various trees:
 concept tree
 composition tree
 attribute tree
 process tree
 decision tree
 cause tree
Example
Source: Bourne and Jenkins , Eliciting Managers' Personal Values: An Adaptation of the Laddering Interview Method,
Organizational Research Methods, SAGE 2005
Concept Tree



Hierarchical diagram of concepts showing classes and members
Activities to create
 Move nodes (concepts) around the tree
 Add new node
 Deleting nodes
 Renaming nodes
Difficulty is avoiding the problems which requires knowing:
 All links on the tree represent an ‘is-a’ relationship
 Terminology to describe the tree
 What classes to use in the tree
 Naming conventions to use
 How to deal with complex cases (e.g. multiple parents, synonyms)
Concept Tree – ‘is-a’ relationship


Is-a = is a type of
Different to ERDs
lorry
car
traffic
steam ship
vehicle
ship
sailing ship
shipping lanes
pollution
traffic issues
congestion
Road safety
What are
the
mistakes
in this
tree?
Concept Tree - Terminology






Root node
Leaf node
Branch
Parent
Children
Descendants
Concept Tree – What classes to use?



Class is a concept which has children on a tree
Other concepts are related to it by an is-a relationship
To develop classes use either a top-down or bottom-up
approach


Top-down start with a set of general classes and refine
Bottom-up start to develop classes by grouping those concepts
that are similar
Repertory Grid technique







Used to elicit attributes for a set of concepts
Used to rate concepts against attributes using a numerical
scale
Uses statistical analysis to arrange and group similar
concepts and attributes
Allows the expert to provide a rating of each concept for
an attribute in concept sorting
A useful way of capturing concept knowledge and tacit
knowledge
When many ratings are provided using many attributes
statistics can be applied to find clusters and correlations
Requires special software
Repertory Grid – How To?

1st stage


Concepts are selected (between 6 & 15)
Set of approx. same no. of attributes is also required



2nd stage



Expert provides a rating for each concept against each attribute
Numerical scale is used
3rd stage



Should be such that values can be rated on a continuous scale (e.g. small to large)
Chosen from knowledge previously elicited
Ratings are applied to cluster analysis to create a visual representation of the
ratings called a focus grid
Concepts with similar scores will be grouped together, attributes with similar
scores will be grouped
4th stage


Engineer walks expert through the results to gain feedback and prompt for
further knowledge about the groupings
If needed more concepts and attributes are rated and included in the grid
Repertory Grid Example



Domain elements are certain types of crime:
petty theft, burglary, drug-dealing, murder,
mugging and rape.
This is one expert’s view on the issue.
Consider carefully whether any pair of attributes
are very similar, by comparing horizontal lines in
this grid.


Beware, when making this comparison, that the
expert may have inadvertently ‘inverted’ the scale
for just one of two similar constructs.




The closest is probably the personal/impersonal one
and the major/petty one.
For example, in the example the major/petty
construct has a value of 5 for ‘major’. If the expert
had chosen 1 instead, and 5 for ‘petty’, then this
construct and the personal/impersonal one would
look very different.
Further analysis may lead you to omit one pairing
of constructs.
Following that you would draw up a table
showing how similar or dissimilar each domain
element is from the others.
For example, when the absolute-value metric is
used, the (numeric) difference.
Constrained Tasks


Expert performs a task they would normally do, but with
constraints.
Variations:



limited time
limited data
Useful for focusing the expert on essential knowledge and
priorities
Commentary and protocol generation



Expert provides a running commentary of their own or
another’s task performance.
A valuable method for acquiring process knowledge and
tacit knowledge.
Variations:





self-reporting
imaginary self-reporting
self-retrospective
shadowing
retrospective shadowing
Knowledge Analysis and Modelling
Knowledge Analysis


Identifying the elements needed to build the knowledge
base
Concepts



Attributes


Qualities or features belonging to a class of concepts
Values


Things that constitute a domain
Main elements of the k-base
Specific qualities or features of a concept that differentiate it from
other concepts
Relations

Way in which concepts are associated with one another
Concepts

Physical concepts






Experts, roles of experts
Organisations and groups

Producers, suppliers, consumers,
departments
Areas of knowledge



Marketing, physics, chemistry
Problems, solutions, advantages,
disadvantages
Physical phenomena


Activities performed by experts
Issues


Purpose of components or roles
Tasks

Documents, databases, websites
People and roles
Functions

Plans, goals, requirements
Sources of information


Products, components, machines
Pieces of information


Mechanisms and forces
Other issues

Constraints, behaviours, states
Attributes

Of physical objects


Of information


Source, format, importance
Of people


Shape, age
Gender, age, personality
Of organisation

Size, turnover, product range
Values


Come in different varieties
Dependent on type


Categorical


For values that are numbers
Text


For values that are adjectives
Numerical


Adjective, number, sentence, paragraphs, hyperlinks, images,
pictures
For values that are one or two sentences
Hypertext

For values that are chunks of hypertext
Relations








Has part
Performs
Followed by
Requires
Causes
Produces
Can have an inverse relation
Short exercise

Think of something that illustrates each one of these
Knowledge modelling




K-model = way of viewing the knowledge in the k-base
Each model provides a different perspective on the
knowledge
Helps clarify the ‘mess’ that is the knowledge
Can be used in elicitation
Trees







Diagram showing hierarchical arrangement of nodes
Node = concept
Link = relationship
Concept tree
Composition tree
Cause tree
Mixed tree
Concept tree



Each link is an is-a relation
Taxonomy
Read from right to left
Taken from www.pcpack.co.uk
Other types of tree

Composition tree



Process tree



Special form of composition tree
All nodes are tasks
Attribute tree


All links are has-part
Used to show components and sub-components of a concept
Shows attributes and values to describe a concept
Mixed tree

Contains more than one type of relation
Matrices

Attribute matrix



Presents set of properties of
a concept (attributes and
values)
Concepts on vertical axis
Attributes and values on
horizontal axis
Relationship matrix


Shows two sets of
concepts related to one
another using a specified
relationship
Cells show which pairs of
concepts have the
relationship
Maps





Shows an arrangement of nodes linked by arrows
Each node represents concept
Link represents relationship
Concept maps
Process maps
Concept map

Many different types
Knowledge Analysis – How to?



How do you identify concepts from interview transcripts
and documents?
Need some codification
Highlighters – different colours for different things
Typical project



47 steps proposed by Milton
Knowledge Acquisition in Practice: A step by step guide,
Milton, Springer-Verlag
Phase I


Phase II


Initial capture and modelling
Phase III


Start, scope and plan the project
Detailed capture and modelling
Phase IV

Share and store knowledge
Phase I – Start, Scope and Plan

Identify a project










How it can benefit, what it involves
Gather opinions from relevant people
Document ideas as project proposal
Seek agreement on proposal from key people
Start knowledge capture
With domain experts break the domain into different topics
and rank against key criteria
Identify a proposed scope and finalise
Identify sources of knowledge
Define and understand the type of project to be able to create
a schedule
Collate the proposal, scope and schedule into a project plan
and disseminate with other materials to team
Phase II – Initial Capture and Modelling




Learn the basics of the domain from documents and informal
conversation with experts
Prepare for semi-structured interviews then execute and
transcribe
Analyse results to identify concepts, create a concept tree to
develop a taxonomy and validate with experts
Create a k-page for each concept





K-page = 2 column table showing all knowledge associated with a
concept
Create a glossary
Build a meta-model showing the relationships between
concepts and relationships
Build appropriate k-models
Continue with validation models
Phase III – Detailed Capture and Modelling






Use further interviews and modelling to capture more
detailed knowledge
Finalise k-model
Prepare prototype end product used to carry out
assessment exercise with sample of end-users
Produce a completion plan defining what needs to be
done to complete the project
Use specialised techniques to do detailed knowledge
capture
If needed cross-validate between experts and resolve
conflicts
Phase IV – Share and store knowledge






Define and create format of end-product
Create provisional end-product using k-base
Give to experts for full validation
Create final end-product and release for use
After use for some time assess impact on organisation
and document it
Conduct complete product review to learn lessons and
make suggestions to change methodology
Ensure end-product is useful, usable and
used

End-users must find



Find product useful
Find product easy to use
Actually use it