Selection of Clinical Trials: Knowledge Representation and Acquisition Savvas Nikiforou Committee: Eugene Fink Lawrence O. Hall Dmitry B. Goldgof Part of the project: Automated Matching of Patients to Clinical Trials Faculty: Lawrence O. Hall Dmitry B. Goldgof Eugene Fink Students: Lynn Fletcher Princeton Kokku Savvas Nikiforou Bhavesh Goswami Tim Ivanovskiy Rebecca Smith Expert System The system analyzes a patient’s data and determines whether the patient is eligible for Moffitt clinical trials. Expert System • Guides a clinician through related questions • Identifies appropriate medical tests • Selects matching clinical trials • Minimizes pain and cost of selection process Outline • Previous work • Eligibility decisions • Knowledge base • Knowledge entry • Experiments Previous Work • Medical expert systems • Knowledge acquisition • Medical systems at USF Medical Expert Systems • If-then rules: – Mycin (1972), Puff (1977), Centaur (1977) • Qualitative reasoning: – Oncocin (1981), Eon (1995), OncoDoc (1998) • Bayesian networks: – Hepar (1990), AIDS2 (1990) Knowledge Acquisition • Teiresias (1974): Knowledge for Mycin • Salt (1985): Elevator-design rules • Opal (1987): Knowledge for Oncocin • Protégé (1987, 2000): General-purpose tools for developing knowledge acquisition interfaces Medical Systems at USF Selection of clinical trials for cancer patients • Bayesian networks (Theocharous) • Qualitative reasoning (Fletcher and Hall) No knowledge acquisition tools Outline • Previous work • Eligibility decisions • Knowledge base • Knowledge entry • Experiments Example: Eligibility Criteria • Female, older than 30 • No prior surgery • Breast cancer, stage II or III Example: Questions Sex: Female Male Age: 25 Example: Conclusion Patient is not eligible Example: Questions Sex: Female Male Age: 35 Example: Questions Cancer stage: Prior surgery? Yes I II III IV No Unknown Example: Conclusion Patient is eligible Full Functionality • Orders and groups the questions • Considers multiple clinical trials Old System • A programmer has to code the questions New System • A programmer has to code the questions • A nurse enters the questions through a friendly interface • Problem: Build the interface Outline • Previous work • Eligibility decisions • Knowledge base • Knowledge entry • Experiments Main Objects • Questions • Medical tests • Eligibility criteria Types of Questions • Yes / No / Unknown • Multiple choice • Numeric Examples of Questions Prior surgery? Yes Cancer stage: Age: I II III IV No Unknown Tests A medical test answers several questions. It involves certain pain and cost. Example Test: Name and Cost Test name: Mammogram Cost: 50.00 Pain: 1 Example Test: Questions • Yes / No Question: Breast cancer? Example Test: Questions • Multiple choice Question: Cancer stage Options: I II III IV Example Test: Questions • Numeric Question: Tumor size Min Max Prec 0 25 0 Eligibility Criteria • A logical expression that determines eligibility for a specific clinical trial Example: Criteria AND Age > 30 Prior-surgery = NO OR Cancer-stage = II Cancer-stage = III Outline • Previous work • Eligibility decisions • Knowledge base • Knowledge entry • Experiments Tests and Questions Adding tests Modifying a test Adding yes/no Adding multiple questions choice questions Adding numeric questions Deleting questions Adding Adding Tests Test name: Yes/No Modifying M-Choice Mammography test Cost: 45.50 Pain: 1 Numeric Deleting Modifying a Test Test name: Adding Yes/No Modifying M-Choice Mammography Mammogram test Cost: 50.00 45.50 Pain: 1 Numeric Deleting Adding Yes/No Questions • Text Breast cancer? Adding Yes/No Modifying M-Choice Numeric Deleting Adding Multiple Choice Questions • Text Cancer stage Adding Yes/No Modifying M-Choice Numeric Options I II III IV Deleting Adding Numeric Questions • Text Tumor size Adding Yes/No Modifying M-Choice Numeric Deleting Min Max Prec 0 25 0 Deleting Questions Breast cancer? Cancer stage Tumor size Patient’s age Adding Yes/No Modifying M-Choice Numeric Deleting Deleting Questions Cancer stage Tumor size Adding Yes/No Modifying M-Choice Numeric Delete Demo Eligibility Criteria Adding eligibility criteria Selecting questions Selecting tests Defining an expression Deleting expressions Editing questions Example: Eligibility Criteria • Female, older than 30 • Breast cancer, stage II • Post-menopausal or surgically sterilized Adding Eligibility Criteria Selecting questions Adding criteria Selecting tests Defining an expression Trial number Trial name 001 Clinical trial A Deleting Editing expressions questions Adding criteria Selecting tests Selecting Tests Selecting questions General questions Blood test Mammogram Biopsy Urine test Defining an expression Deleting Editing expressions questions Selecting Questions Age: Adding criteria Selecting questions Selecting tests Defining an expression 0 From: 30 Deleting Editing expressions questions To: 150 I II III IV Cancer stage: Prior surgery? Yes No Unknown Post-menopausal? Yes No Unknown Surgically sterilized? Yes No Unknown Defining an Expression Adding criteria Selecting questions Selecting tests Defining an expression Deleting Editing expressions questions Age > 30 Cancer-stage = II Post-menopausal = YES Surgically-sterilized = YES Defining an Expression Adding criteria Selecting questions Selecting tests Defining an expression Deleting Editing expressions questions AND Age > 30 Cancer-stage = II Post-menopausal = YES Surgically-sterilized = YES Defining an Expression Adding criteria Selecting questions Selecting tests Defining an expression Deleting Editing expressions questions AND Age > 30 Cancer-stage = II Post-menopausal = YES Surgically-sterilized = YES Defining an Expression Adding criteria Selecting questions Selecting tests Defining an expression Deleting Editing expressions questions AND Age > 30 Cancer-stage = II OR Post-menopausal = YES Surgically-sterilized = YES Defining an Expression Adding criteria Selecting questions Selecting tests Defining an expression Deleting Editing expressions questions AND Age > 30 Cancer-stage = II OR Post-menopausal = YES Surgically-sterilized = YES Demo Outline • Previous work • Eligibility decisions • Knowledge base • Knowledge entry • Experiments Experiments Performance of seven novice users • Entering tests and questions • Entering eligibility criteria time per question (sec) Entering Tests and Questions Learning curve 100 80 60 40 20 0 0 1 2 3 number of a test set 4 time per question (sec) Entering Eligibility Criteria Learning curve 100 80 60 40 20 0 0 1 2 3 4 5 6 7 8 number of a clinical trial 9 10 11 entry time (sec) Entering Eligibility Criteria 1600 1400 1200 1000 800 600 400 200 0 0 5 10 15 20 25 number of questions 30 35 Summary • Learning time: 1 hour • Adding a test: 2 to 10 minutes • Adding eligibility criteria: 30 to 60 minutes • Building a knowledge base for Moffitt breast-cancer trials: 8 to 10 hours Main Results • Formal model of selection criteria • Representation of related knowledge • Friendly interface for knowledge entry Future Work • Probabilities of different answers • Logical connections among questions • Detection of identical and related questions