Selection of Clinical Trials:
Knowledge Representation and Acquisition
Savvas Nikiforou
Committee:
Eugene Fink
Lawrence O. Hall
Dmitry B. Goldgof
Part of the project:
Automated Matching of Patients
to Clinical Trials
Faculty:
Lawrence O. Hall
Dmitry B. Goldgof
Eugene Fink
Students:
Lynn Fletcher
Princeton Kokku
Savvas Nikiforou
Bhavesh Goswami
Tim Ivanovskiy
Rebecca Smith
Expert System
The system analyzes a patient’s data and
determines whether the patient is eligible
for Moffitt clinical trials.
Expert System
• Guides a clinician through related questions
• Identifies appropriate medical tests
• Selects matching clinical trials
• Minimizes pain and cost of selection process
Outline
• Previous work
• Eligibility decisions
• Knowledge base
• Knowledge entry
• Experiments
Previous Work
• Medical expert systems
• Knowledge acquisition
• Medical systems at USF
Medical Expert Systems
• If-then rules:
– Mycin (1972), Puff (1977), Centaur (1977)
• Qualitative reasoning:
– Oncocin (1981), Eon (1995), OncoDoc (1998)
• Bayesian networks:
– Hepar (1990), AIDS2 (1990)
Knowledge Acquisition
• Teiresias (1974): Knowledge for Mycin
• Salt (1985): Elevator-design rules
• Opal (1987): Knowledge for Oncocin
• Protégé (1987, 2000):
General-purpose tools for developing
knowledge acquisition interfaces
Medical Systems at USF
Selection of clinical trials for cancer patients
• Bayesian networks (Theocharous)
• Qualitative reasoning (Fletcher and Hall)
No knowledge acquisition tools
Outline
• Previous work
• Eligibility decisions
• Knowledge base
• Knowledge entry
• Experiments
Example: Eligibility Criteria
• Female, older than 30
• No prior surgery
• Breast cancer, stage II or III
Example: Questions
Sex:
Female
Male
Age:
25
Example: Conclusion
Patient is not eligible
Example: Questions
Sex:
Female
Male
Age:
35
Example: Questions
Cancer stage:
Prior surgery? Yes
I
II
III
IV
No
Unknown
Example: Conclusion
Patient is eligible
Full Functionality
• Orders and groups the questions
• Considers multiple clinical trials
Old System
• A programmer has to code the questions
New System
• A programmer has to code the questions
• A nurse enters the questions
through a friendly interface
• Problem: Build the interface
Outline
• Previous work
• Eligibility decisions
• Knowledge base
• Knowledge entry
• Experiments
Main Objects
• Questions
• Medical tests
• Eligibility criteria
Types of Questions
• Yes / No / Unknown
• Multiple choice
• Numeric
Examples of Questions
Prior surgery? Yes
Cancer stage:
Age:
I
II
III
IV
No
Unknown
Tests
A medical test answers several questions.
It involves certain pain and cost.
Example Test: Name and Cost
Test name:
Mammogram
Cost:
50.00
Pain:
1
Example Test: Questions
• Yes / No
Question:
Breast cancer?
Example Test: Questions
• Multiple choice
Question:
Cancer stage
Options:
I
II
III
IV
Example Test: Questions
• Numeric
Question:
Tumor size
Min
Max
Prec
0
25
0
Eligibility Criteria
• A logical expression that determines
eligibility for a specific clinical trial
Example: Criteria
AND
Age > 30
Prior-surgery = NO
OR
Cancer-stage = II
Cancer-stage = III
Outline
• Previous work
• Eligibility decisions
• Knowledge base
• Knowledge entry
• Experiments
Tests and Questions
Adding tests
Modifying a test
Adding yes/no Adding multiple
questions
choice questions
Adding numeric
questions
Deleting
questions
Adding
Adding Tests
Test name:
Yes/No
Modifying
M-Choice
Mammography test
Cost:
45.50
Pain:
1
Numeric
Deleting
Modifying a Test
Test name:
Adding
Yes/No
Modifying
M-Choice
Mammography
Mammogram test
Cost:
50.00
45.50
Pain:
1
Numeric
Deleting
Adding Yes/No
Questions
• Text
Breast cancer?
Adding
Yes/No
Modifying
M-Choice
Numeric
Deleting
Adding Multiple
Choice Questions
• Text
Cancer stage
Adding
Yes/No
Modifying
M-Choice
Numeric
Options
I
II
III
IV
Deleting
Adding Numeric
Questions
• Text
Tumor size
Adding
Yes/No
Modifying
M-Choice
Numeric
Deleting
Min
Max
Prec
0
25
0
Deleting Questions
Breast cancer?
Cancer stage
Tumor size
Patient’s age
Adding
Yes/No
Modifying
M-Choice
Numeric
Deleting
Deleting Questions
Cancer stage
Tumor size
Adding
Yes/No
Modifying
M-Choice
Numeric
Delete
Demo
Eligibility Criteria
Adding eligibility
criteria
Selecting
questions
Selecting
tests
Defining an
expression
Deleting
expressions
Editing
questions
Example: Eligibility Criteria
• Female, older than 30
• Breast cancer, stage II
• Post-menopausal or surgically sterilized
Adding Eligibility
Criteria
Selecting
questions
Adding
criteria
Selecting
tests
Defining an
expression
Trial number
Trial name
001
Clinical trial A
Deleting
Editing
expressions questions
Adding
criteria
Selecting
tests
Selecting Tests
Selecting
questions
General questions
Blood test
Mammogram
Biopsy
Urine test
Defining an
expression
Deleting
Editing
expressions questions
Selecting
Questions
Age:
Adding
criteria
Selecting
questions
Selecting
tests
Defining an
expression
0
From: 30
Deleting
Editing
expressions questions
To: 150
I II III IV
Cancer stage:
Prior surgery?
Yes
No
Unknown
Post-menopausal?
Yes
No
Unknown
Surgically sterilized?
Yes
No
Unknown
Defining an
Expression
Adding
criteria
Selecting
questions
Selecting
tests
Defining an
expression
Deleting
Editing
expressions questions
Age > 30
Cancer-stage = II
Post-menopausal = YES
Surgically-sterilized = YES
Defining an
Expression
Adding
criteria
Selecting
questions
Selecting
tests
Defining an
expression
Deleting
Editing
expressions questions
AND
Age > 30
Cancer-stage = II
Post-menopausal = YES
Surgically-sterilized = YES
Defining an
Expression
Adding
criteria
Selecting
questions
Selecting
tests
Defining an
expression
Deleting
Editing
expressions questions
AND
Age > 30
Cancer-stage = II
Post-menopausal = YES
Surgically-sterilized = YES
Defining an
Expression
Adding
criteria
Selecting
questions
Selecting
tests
Defining an
expression
Deleting
Editing
expressions questions
AND
Age > 30
Cancer-stage = II
OR
Post-menopausal = YES
Surgically-sterilized = YES
Defining an
Expression
Adding
criteria
Selecting
questions
Selecting
tests
Defining an
expression
Deleting
Editing
expressions questions
AND
Age > 30
Cancer-stage = II
OR
Post-menopausal = YES
Surgically-sterilized = YES
Demo
Outline
• Previous work
• Eligibility decisions
• Knowledge base
• Knowledge entry
• Experiments
Experiments
Performance of seven novice users
• Entering tests and questions
• Entering eligibility criteria
time per question (sec)
Entering Tests and Questions
Learning curve
100
80
60
40
20
0
0
1
2
3
number of a test set
4
time per question (sec)
Entering Eligibility Criteria
Learning curve
100
80
60
40
20
0
0
1
2
3
4
5
6
7
8
number of a clinical trial
9
10 11
entry time (sec)
Entering Eligibility Criteria
1600
1400
1200
1000
800
600
400
200
0
0
5
10
15
20
25
number of questions
30
35
Summary
• Learning time: 1 hour
• Adding a test: 2 to 10 minutes
• Adding eligibility criteria: 30 to 60 minutes
• Building a knowledge base for Moffitt
breast-cancer trials: 8 to 10 hours
Main Results
• Formal model of selection criteria
• Representation of related knowledge
• Friendly interface for knowledge entry
Future Work
• Probabilities of different answers
• Logical connections among questions
• Detection of identical and related questions