Advanced Knowledge
Technologies






University of Aberdeen
University of Edinburgh
University of Sheffield
Open University
University of Southampton
http://www.aktors.org
MIAKT





Oxford University
King’s College, London
University of Sheffield
Open University
University of Southampton
CoAKTinG
 University of Edinburgh
 Open University
 University of Southampton
MIAKT: Medical Informatics and
Knowledge Technologies
Supporting triple-assessment (TA)
(collaborative decision-making) for the
diagnosis and treatment
of breast cancer
Oxford University
Kings College
Open University
University of Sheffield
University of Southampton
To support collaboration for the e-Scientist
Intelligent meeting spaces:
Decision rationale, group memory capture
Planning, coordination support
Instant messaging/presence
Open University
University of Edinburgh
University of Southampton
Advanced Knowledge
Technologies
 Representation and Reasoning
 Ontologies: domain and process models
 Interoperability
 Integration with databases for scalability
(Semantic Web)
 Reasoning services – local and distributed
 Natural language processing
 Application domain – academic CS
People involved
MIAKT
Oxford University
Mike Brady,
Jon Whitely
King’s College London
David Hawkes,
Christine Tanner,
Yalin Zheng
The Open University
Enrico Motta,
John Domingue,
Liliana Cabral
University of Sheffield
Yorick Wilks,
Fabio Ciravegna,
Kalina Bontcheva
University of Southampton
Nigel Shadbolt,
Srinandan Dasmahapatra,
Paul Lewis,
Bo Hu,
Hugh Lewis
CoAKTinG
University of Edinburgh
Austin Tate
Stephen Potter
Jessica Chen-burger
Jeff Dalton
Open University
Marc Eisenstad
Simon Buckingham Shum
Jiri Komzak
Michelle Bachler
University of Southampton
David De Roure
Nigel Shadbolt
Danius Michaelides
Richard Beales
Kevin Page
Ben Juby
Breast Cancer –
Statistics & Screening
 EU: 24% of cancer cases
19% of cancer deaths
1 in 8 of women will develop breast cancer
during the course of their lives
1 in 28 will die of the disease.
 5 year survival rate for localized breast cancer
is 97% for early detection
is 77% if the cancer has spread at diagnosis
is 22% if distant metastases are found
 Screening for ages 50+
 M Brady: MIAS (features), e-Diamond (priors)
 Knowledge technology support
MIAKT:
Patient Management -Triple Assessment
Imaging:
Mammography/Ultrasound/MR
Mammography (X-ray)
Position breast on small flat plate, with X-ray plate
under it.
Flat plate above your breast.
When machine is switched on, breast pressed down
between plates by machine to get clearest picture.
Two pictures are taken: from above and from the
side.
Ultrasound
usually used for women
under 35 (breasts too
dense or solid to give a
clear picture with
mammography) It is
also used to see if a
breast lump is solid
contains fluid (a cyst)
Histopathology
 Fine needle aspiration cytology
 Core biopsy
 With imaging guidance
MIAKT:
Triple assessment
Triple Assessment
 Clinical and radiological opinions are
used independently to decide upon
further intervention
 The most suspicious opinion prevails
(normal/definitely benign, probably
benign, indeterminate, probably
malignant)
 Needle biopsy is mandatory for all
abnormalities classified as
indeterminate or more suspicious
 Needle biopsy results are discussed
in the context of imaging and clinical
findings at multidisciplinary
meetings
Radiology and Histopathology
images: Different scales
Same
Patient:
multiple
descriptors
Ontology, Annotation,
Language Generation
Annotated images for retrieval in context
Memory aids for specialists
Report generation from annotations
Ontologies for descriptive grounding
Correlative reasoning across specialisation
(across scales orders of magnitude apart)
 Distributed reasoning (grid?)





Knowledge Engineering
 Lexical and ontological issues
 Decision support
 Records – image and text
Diagnostic Mammography:
different views
Cranio-caudal (CC) &
Mediolateral oblique (MLO)
views
lateromedial
mediolateral
To pinpoint exact size & location of breast abnormality
and to image surrounding tissue and lymph nodes.
Diagnostic Mammography:
BI-RADS Ontology - Masses
 1. SHAPE
1.a. Round
1.b. Oval
1.c. Lobular
1.d. Irregular
 2. MARGINS
2.a. Circumscribed
2.b. Microlobulated
2.c. Obscured
2.d. Indistinct
2.e. Spiculated
 3. DENSITY:
3.a. High density
3.b. Equal density
3.c. Low density
3.d. Fat containing – radiolucent –
oil cyst, lipoma, or galactocele
as well as mixed lesions such as
hamartoma or fibroadenolipoma.
[and/or histologic terms]
 MASS: space occupying lesion seen in two different
projections.
 If potential mass seen in single projection, called DENSITY
until 3-D confirmation.
BI-RADS Ontology – Masses
(details)
MASS: space occupying lesion seen in two different projections.
If potential mass seen in single projection, called DENSITY until 3-D confirmation.
1. SHAPE
a. Round: spherical, ball-shaped, circular or globular
b. Oval: elliptical or egg-shaped.
c. Lobular: has contours with undulations.
d. Irregular: none of the above.
2. MARGINS [modify the shape of the mass]
a. Circumscribed Margins: abrupt transition between the lesion and the surrounding tissue. Without
additional modifiers there is nothing to suggest infiltration.
b. Microlobulated Margins: undulate with short cycles producing small undulations.
c. Obscured Margins: hidden by superimposed or adjacent normal tissue; cannot be assessed any further.
d. Indistinct Margins: poor definition of margins raises concern of infiltration by the lesion; not likely due to
superimposed normal breast tissue.
e. Spiculated Margins: lines radiating from margins of mass
3. DENSITY: x-ray attenuation of lesion relative to the expected attenuation of an equal volume of
fibroglandular breast tissue; most cancers are of equal or higher density; never fat containing but may
trap fat.
a. High density
b. Equal density
c. Low density
d. Fat containing – radiolucent - oil cyst, lipoma, or galactocele as well as mixed lesions such as hamartoma
or fibroadenolipoma. [When appropriate, histologic terms may be included]
Microcalcification
 Microcalcifications:
(<.5mm) specks of
calcium in milk ducts.
 About half of the
cancers detected by
mammography
appear as a cluster of
microcalcifications.
 Microcalcifications are the most
common mammographic sign of
ductal carcinoma in situ
 P(micro-Ca | DCIS)=0.9
BI-RADS: Calcifications
 Benign calcifications usually larger than
malignant ones -- coarser, often round with
smooth margins, more visible.
 Malignant calcifications very small, require
magnifying glass.
 When specific aetiology not possible,
description of calcifications should include
their distribution and morphology.
 Benign calcifications only reported if judged to
be susceptible to misinterpretation.
Multiple views on domain




Structure, anatomy
Function, physiology
Pathology
Patient history
BI-RADS Ontology: Calcification




Typically benign
Intermediate concern
Higher probability
Distribution modifiers
BI-RADS Ontology: Calcification
 1. TYPICALLY BENIGN
1.a. Skin Calcifications
1.b. Vascular Calcifications
1.c. Coarse Calcifications
1.d. Large Rod-Like
Calcifications
1.e. Round Calcifications
1.f. Lucent-Centered
Calcifications
1.g. Eggshell or Rim
Calcifications
1.h. Milk of Calcium
Calcifications
1.i. Suture Calcifications
1.j. Dystrophic
Calcifications
1.k. Punctate Calcifications
 2. INTERMEDIATE
CONCERN
2.a. Amorphous or
Indistinct
Calcifications
 3. HIGHER
PROBABILITY OF
MALIGNANCY
3.a. Pleomorphic or
Heterogeneous
Calcifications
3.b. Fine, Linear or
Branching Calcifications
 4. DISTRIBUTION
MODIFIERS
4.a. Grouped or
Clustered
4.b. Linear
4.c. Segmental
4.d. Regional:
4.e.Diffuse/Scattered
BI-RADS Ontology:
Calcification (details)
TYPES AND DISTRIBUTION OF CALCIFICATION:
1. TYPICALLY BENIGN a. Skin Calcifications: typical lucent centered deposits that are pathognomonic. Atypical forms confirmed by
tangential views to be in the skin.
b. Vascular Calcifications: Parallel tracks, or linear tubular calcifications clearly associated with blood vessels.
c. Coarse Calcifications: Classic calcifications produced by an involuting fibroadenoma.
d. Large Rod-Like Calcifications: Continuous rods, occasionally branching, diameter > 1mm usually, may
have lucent centers, if calcium surrounds rather than fills an ectactic duct. Found in secretory disease,
"plasma cell mastitis", and duct ectasia.
e. Round Calcifications: When multiple, of variable size. Considered benign and when small [under 1 mm],
frequently formed in acini of lobules. Under 0.5 mm are termed punctate.
f. Lucent-Centered Calcifications: Less that 1 mm to greater than 10 mm, smooth surfaces, round or oval,
have lucent center. “Wall" thicker than the "rim or eggshell" type. Included are areas of fat necrosis,
calcified debris in ducts, and occasional fibroadenomas.
g. Eggshell or Rim Calcifications: Very thin, under 1mm thickness, appear as calcium deposited on the
surface of a sphere. Although fat necrosis can produce these thin deposits, calcifications in the wall of
cysts are the most common "rim" calcifications.
h. Milk of Calcium Calcifications: Consistent with sedimented calcifications in cysts. Often less evident in
craniocaudal image -- appear as fuzzy, round, amorphous deposits; sharply defined on 90° lateral -semilunar, crescent shaped, curvilinear (concave up), or linear defining dependent portion of cysts.
i. Suture Calcifications: Ca deposited on suture material, relatively common in post-irradiated breast,
typically linear or tubular in appearance and knots are frequently visible.
j. Dystrophic Calcifications: Usually form in irradiated breast or following trauma. Irregular in shape, usually
> 0.5 mm, often have lucent centers.
k. Punctate Calcifications: Round/oval, < 0.5 mm with well-defined margins.
Example Mammogram
 2 cm mass (tumour)
 microcalcifications
Microcalcifications:
Clinical Procedures (guideline)
Microcalcifications
Discharge!
Further Ultrasound and
Mammography
Clustered
heterogeneous
Clinical exam
+
Needle core biopsy
With
Specimen radiography
Normal or
Definitely benign
Treatment
Significant
abnormality
Equivocal
result
Triple
assessment
Histopathology
•Fine needle aspiration cytology
•Core biopsy
•With imaging guidance
Descriptors when drawing sample:
mm
Sampling pattern for stereotactic FNAC
F
FNAC
Histopathology slides
 A histological slide has an immense amount of
data, the closer you look, the more there is
 Histological images are complex with
challenges at both the segmentation and
feature classification level
 Think in terms of two scales – low-power, highpower
Histopathology
Histopathology slides:
low power/high power
Histopathology slides:
diagnostic criteria
Reporting guidelines
MIAKT: Technology Palette
 MIAS –
 Medical image registration (X-ray, MR)
 Segmentation and feature extraction
 Image Classification
 AKT –
 Ontology development
 (Distributed) reasoning services
 Image annotation against ontologies
 Natural language generation
 Decision support – belief nets?
MIAKT
 Abstract away from the details of TA meeting
 Collaborative problem solving/decision making
 Possibly distributed, virtual presence
 Well-defined goals, well-defined contributory
skill sets
 Structured protocol
 Require recall of contents of events of past
meeting
 Report generation (audit trail)
Enhance Technologies
 Ontologically annotated audio/video streams
 Issue handling, tasking, planning and
coordination
 Collective sense-making and group memory
capture
 Enhanced presence management and
visualisation
 Adaptive information systems
Technology
Integration
Aim: To support e-Science collaboration by integrating
and demonstrating the utility of:
 intelligent task-orientated messaging, collaborative
planning, issue, activity and constraint management
(I-X Process Panels/<I-N-C-A>: Edinburgh)
 peripheral awareness of the online presence,
availability, attributes and location of colleagues,
documents, and devices (BuddySpace: OU)
 real time conversational mapping of meetings,
providing shared visual focus and group memory
capture (Compendium: OU)
 multimedia meeting mark-up, replay and navigation
(HyStream: Southampton)
Jabber
 Jabber is a set of XML-based protocols for realtime messaging and presence notification
 Communicates with other instant messaging
services through gateways
 Many clients available – see
http://www.jabbercentral.org/
Jabber
Compendium
 Provides a methodological framework, plus an evolving
suite of tools, for collective sense-making and group
memory.
 Intersection of collaborative modelling, organisational
memory, computer-supported argumentation and
meeting facilitation.
 Centres on face-to-face meetings, potentially the most
pervasive knowledge-based activity in working life, but
also one of the hardest to do well.
Compendium
BuddySpace
 ‘Enhanced Presence Management for Collaborative
Working, Messaging, Gaming and Beyond’
 The concept of presence is a rich combination of
attributes that characterise an individual's…
– physical and/or spatial location
– work trajectory
– time frame of reference
– mental mood
– goals and intentions
http://kmi.open.ac.uk/projects/buddyspace/
BuddySpace
Compendium
Process Panels
 Based on notion of the representation of a product as a
set of nodes making up the components of the product
model, along with constraints on the relationship
between those nodes and a set of outstanding issues
 Investigates the use of shared models for task directed
communication between human and computer agents
who are jointly exploring a range of alternative options
for activity.
Process Panels
HyStream
Smart spaces
 Devices in the room enables us to capture
continuous (real time, multi-way, multi-cast,
ontologically informed, …) metadata
 Other devices provide ‘presence’ information
 Consider an experimental laboratory instead of
a meeting room:
 Instruments
 Electronic log books
 Visualisation