From: AAAI Technical Report SS-92-02. Compilation copyright © 1992, AAAI (www.aaai.org). All rights reserved.
Automated Interpretation
of Diagrams for Specification
of Medical Protocols
JohnW.Egar, AngelR. Puerta, MarkA. Musen.
KnowledgeSystems Laboratory
Stanford University, Stanford, CA94305-5479USA
Weare buildinga visual programming
environment
that enables medicalresearchers to organizeand describe complex
researchprotocols(specified treatmentplans) so that this informationcan be incorporated
into decision-supportsystems.Wehavedesignedour systemwith a five-layer architectureanda collection
of compilersfor various visual languages.This designpermitsthe interchangeof modulesandthe coordination of different visual languageswithinthe samesystem.
1. Objective
Althoughdiagramming
environments
are often excellent tools for modelingcomplex
relationships, they are also difficult to implement.
Ourresearchteamis buildingknowledge-based
tools that will help researchersto designandexecute
medicaltreatmentplans[1, 2]. Oneof our tasks in this project is to create severalvisual programming
languagesthat clinical researchersand computerscientists can use to modelthe problem-solving
methodsand the medicalbackground
informationfor manyprotocolsin a particular field, andthe structure of individualprotocols.Thispaperdescribeshowwe
haveimplemented
these visual languages.
2. DesignConsiderations
Previousexperiencewith domain-specific
knowledge-acquisition
tools [2, 3] has convincedus that flowchartsare a
particularlyuseful wayfor expertphysiciansto describethe management
of clinical trials for experimental
therapies. We
are nowincludingclassification trees, state charts, entity-relationshipdiagrams,influencediagrams,anddata-flowdiagramsto describe the necessarydomaininformationandproblem-solving
methods[1].
Environments
for multiplevisual programming
languagesdo exist [4, 5, 6, 7, 8, 9]. Withfewexceptions[4, 10, 11,
12], these environments
haverelied to somedegreeon syntax-directedediting. Theseeditors guidethe user so that she
constructsonlysyntacticallycorrect diagrams.This strategyrequiresa custom-tailored
editor for eachvisual language.In
its extremeform,syntax-directedediting forces the user to modelin a top-down
fashion,introducingsuccessivelyless abstract placeholdersaccordingto rigid replacement
rules. Because
of these limitations, wehaveelectedinsteadto treat diagramsas syntax-neutraldocuments
that are compiledafter they are saved.
Certaininterface dynamics
are also necessaryfor such a diagramming
tool to be successful. Theease andspeedof
use of the tool is of particular concern.Theability to nest diagramswithinnodesandedges,andtherebyto avoidmuch
visualclutter, is anothercriterion. Hypertext
links to text andformsare desirable,as is the ability to interchange
notational
conventions
to suit the expertrather thanthe system.Otherdesigngoalsincludereliability, n~ainminability,
portability, and
modularity.Finally, a simplegraphicaluser-interfaceconvention,suchas drag-and-drop,
is a valuablefeature.
3. SystemDescription
Theimplementation
of such a battery of visual languagespresentsa formidableprogramming
task. After buildingtwo
custom-tailoreddiagrameditors, wedevelopedthe followingfive-layeredapproach:
186
ChosenPalette
(Figure la)
~ram
=
Library of
Plleltes
File (Figurela)
~G~ F’de
(Figure
lb)
(a)
[
’
’
~
rL~
4
/*
~o~.al~tZ tt tl= ,/
O.tO0000
O. 4eo~e
o.3ooooo
e.’~o~o
P.......
(c)
.r.j
tructure (Figure lc}
~" ~ (VlxuaI-L.nguage
compiler.)
~. ~
n
(d)
j
Figure 1. Sampledata files passed betweeneach layer. (a) Palette
and diagram.(b) GRLfile. (c) Networkstructure. (d) Target
| [ IM3 ]
~
r~] (Inferential
~ Mnchin*~)
Figure 2. Data-flow diagramof system design.
1. Theuser interface for all diagramming
is a general, syntax-independentdiagrameditor (Figure
la).
2. A syntax-independent,but editor-specific, diagram
reader generates a high-level, textual description of a saveddiagramand its nested subdiagrams. The textual language we use is the Graphical Representation Language(GRL),
described by Manke[13] (Figure lb).
3. Asingle parser converts the GRLtextual description into dynamicobjects that maintaininformationon the position,
appearance, labeling, connections, and other attributes of each node, link, and diagramin a nested group of diagrams(Figure lc).
4. Frommultiple visual-language compilers, the systemchooses one to produceappropriate target code for the individual diagram(Figure ld).
5. Oneor more inferential programs,which operate on the target code producedin layer 4, generate parameters and
proceduresfor the knowledge-acquisitiontool or for the decision-support system.
The visual appearanceof the nodesin a diagramis dictated by the objects in the palette used to construct the diagram.
Wecan create newpalettes easily by dragginggraphical imagesinto a blank palette. Thus, the domainexpert can use those
notational conventionswith whichshe is most familiar, and no alterations need to be madeto other layers of the system.
Whenthe user has finished constructing a set of diagrams, she selects a compiler button, and the systemproduces
either the appropriate target code or explicit error messages.Currently, the modelingsystempredetermineswhichvisuallanguagecompiler is used. However,if diagramsare drawnor nested within a diagramof a different paradigm,or if the
particular paradigmused is not knownin advance, the system should be able to match the appropriate visual-language
compiler to each diagramor subdiagramaccording to attribute clues in the GRLdescription.
4. Status Report
Wehavefully implemented
the first three of the five layers of our architecture. At the first level, we nowuse a commercial editor [14], whichhas a drag-and-dropinterface. The three other editors that we tested (two built by our group,
and one built by other researchers [15]) were slower and more awkwardto use. Wehave designed and used more than
dozen palettes and paradigms relating to medical protocols. Wehave tested our amendedversion of GRL,and have found
it sufficiently general to describe all relevant aspects of these diagramming
paradigms.Wehave built a GRL
parser, and
have tested it with several visual languages. Furthermore,we have determinedthat the networkof objects that the GRL
parser creates is adequate for the three visual-language compilers that we have implemented.
187
Developing
the visual-languagecompilers(in layer 4) is moreproblematic.Formalmodelsof textual languagesare
wellunderstood,andtools exist that facilitate compilergenerationfor these languages.Generalmodelsfor visual languages, however,are fundamentally
morecomplex:Symbols
in visual languagesmustbe parsedaccordingto their position,
connections,size, appearance,andorientation with respect to other symbols,whereassymbolsin textual languagesare
related only by concatenation.Researchershaveonly recently developedformalgrammars
andtools that aid in the generation of visual-language
parsers [10, 11, 12, 16]. After weprogrammed
compilersfor belief networks,state-transition diagrams,and proceduralflowcharts, several common
graph-manipulation
motifs emerged.For example,parsers of
dkected-graph
languagestend to identify originatingandterminatingnodes,andthen to simplifyintermediatepaths accordingto a fewsyntacticalrules for the nodetypes. Werealizedthat these syntacticalroles wereeasiest to describedeclaratively, andthat wemightbe able to generatevisual-language
parserswith suchdescriptions.Consequently,
our system
is convergingon a graph-grammar
productionsystem,wheredeclarative, subgraph-replacernent
grammars
supportthe automaticgenerationof visual-languagecompilers.
5. Discussion
Whileimplementing
a visual programming
environment
for the designof clinical protocols, wehavelearned that the
responsiveness
andthe expressivenessof sucha modeling
tool mustbe emphasized,
evenat the expenseof user guidance.
This shift in emphasishas led us awayfromsyntax-directed,interpretive editors, andtowardour presentsyntax-neutral,
compiledenvironment.
Byemploying
just onesyntax-neutraleditor, wemaintaina single interface behavior,despite the
multiplicityof languages.Byseparatingout the compilationof diagramsfromthe processof editing them,weallowfaster
editing, and, hence,offer a moreresponsivetool. Sincethe notationalconventions
for eachvisual languageare sp~ified
in replaceablepalettes, wecan adaptour visual languagesto the user’s preferences.Bytranslating diagramsandpalettes
fromall the different languagesinto a common
representation(GRL)
wecaninsulate a user’s choicesof editor andpalettes
fromthe systemdesigner’schoicesof visual-language
compilersandinferential machinery.Wecan easily incorporateinto
these layers softwareproducedoutsideour laboratory.Althoughwecan expressall visual languagesthat wehaveso far
neededin modeling
clinical protocolsby usinga node-and-link
editor [14], wecouldintroducea different type of editor
shouldthe needarise. Thus,in additionto enhancingresponsiveness,this layereddesignmakesour systemeasy to maintain.
Ourcompiledapproachdoes,however,introducethe needfor visual-language
parsing. Further, to providethe expressiveness neededto describe complexprotocols, wemusthavemanyvisual languages.Visual-language
parsers are sometimeschallengingto implement,but the onesthat wehavebuilt havecommon
features that makea formalgraph-grammar
productionsystema promising
strategy for buildingother parsers. Webelievethat the specificationandgenerationof visual-languageparsers throughattributed graphgrammars
will yield higher-qualitycompilerswith far less programming
effort, as compared
to ad hoeprogramming.
Sucha specificationcouldalso lead to formaltesting of the modelsconstructed throughdiagrams.Givena clear mapping
frommedicalideas to the kindsof representationsusedin our decision-support system,wemightbe able to discernpatterns andto detect errors in our computational
modelsof clinical protocols.
6. FuturePlans
Weplanto developa large library of visual-language
compilersfromdeclarativespecifications.Thevisual languages
will be used by clinical researchersto modelmedicalinformation,data structures, andproblem-solving
methods.Webelieve that the heterogeneous
nature of medicalreasoningwill require this variety of languages.Whether
suchan arsenal
of tools will be sufficient, andwhichdiagramming
paradigms
will be mostsuccessfulfor modeling
clinical-trial management,remainto be tested.
Weshall continuelinking diagrammatic
languagesto existing programs,andto inferential programs
associatedwith
knowledge-acquisition
tools andtheir targeted decision-supportsystems.Thecomplex
constraints that wehaveidentified
in thoseclinical protocolsthat wehavealreadyinvestigatedsuggestthat an iterative cycleof modeling,
implicationviewing, anddebugging
maybe necessaryto achievea consistentandreasonableprotocoldesign. Hence,our plansincludethe
implementation
of visual languagesfor the acquisitionandpresentationof constraints.
Weaimto implement
at least a sufficient set of programming
languagesfor the designof clinical trial protocols.Presumably,the sametools can be used to test howmuchtextbookinformation,medical-recordinformation,andmedical
problem-solving
informationcan be expressedclearly throughdiagrams.
188
Acknowledgments
Weconducted this work with the support of the National Library of Medicineunder grants LM-05157,LM-07033,
and LM-05208,of the Agencyfor Health Care Policy and Research under grant HS06330,and of the Digital Equipment
Corporation.
Weare most grateful to Alan Chungand JonathanSchwartzfor helping us with the Diagram[editor, to Josef Schreiner
for guiding our visual programming
research, to DavidCombsfor building the foundations for this research, and to Lyn
Dupr~for providingeditorial assistance.
References
[ 1] Egar J W,Puerta A R, Musen MA. Diagrammatic
Acquisition of MedicalInformationfor Clinical Trials. KSL-Report
91-59, KnowledgeSystems Laboratory, Stanford University, Stanford, CA, October 1991, submitted to MEDINFO
92: Seventh WorldCongress on Medical Informatics.
[2] MusenMA. An Editor for the ConceptualModelsof Interactive Knowledge-Acquisition
Tools. International Journal
of Man-MachineStudies 1989, 31:673--698.
[3] MusenMA, Fagan L M, CombsD Mand Shortliffe E H. Use of a DomainModelto Drive an Interactive KnowledgeEditing Tool. International Journal of Man-Machine
Studies. 1987, 26:105-121.
[4] ChangS-K. A Visual LanguageCompilerfor Information Retrieval by Visual Reasoning. IEEETransactions on Software Engineering 1990, 16:1136-1149.
[5] Lakin F. Spatial Parsing for Visual Languages.In ChangS-K, IchikawaT, LigomenidesP A (eds). Visual Languages.
NewYork: PlenumPress, 1986:35-85.
[6] BacklundB, I-Iagsand O and Pehrson B. Generation of Graphic language-Oriented Design Environments. SICSResearch Report R-89-8911,SwedishInstitute of ComputerScience, Stockholm,Sweden,1989.
[7] Arefi F, HughesC E and Workman
D A. Automatically Generating Visual Syntax-Directed Editors. Communications
of the ACM1990, 33:349-360.
[8] Borges J A, Johnson R E. MultiparadigmVisual ProgrammingLanguage. In Proceedings of the 1990 IEEEWorkshop
on Visual Languages. Los Alamitos, CA: IEEE ComputerSociety Press, 1990:233-240.
[9] Casner, S. Building CustomizedDiagrammingLanguages. In ChangS-K (ed). Visual languages and visual programming. NewYork: PlenumPress, 1990: 71-95.
[10] HelmR, Marriott K and OderskyM. Building Visual LanguageParsers. In RobertsonS P, OlsonG M, Olson J S (eds).
HumanFactors in Computing Systems: CHI ’91 Conference Proceedings. Amsterdam:Addison-Wesley1991:105112.
[11] Nagl M. A Software DevelopmentEnvironment Based on Graph Technology. In Ehrig H, Rozenberg G, Rosenfeld
A (eds). Graph-Grammars
and their Application to ComputerScience: Third International Workshop.NewYork:
Springer-Verlag, 1987:458-477.
[12] Golin E J, Reiss S P. Parsing in a Visual Environment.Technical Report No. CS-89-06,Dept. of ComputerScience,
BrownUniversity, Providence, RI, February, 1989.
[13] MankeS and Paulisch F N. GraphRepresentation Language: Reference Manual. On-line documentation, Department
of Informatics, University of Karlsruhe, Germany,December,1990.
[14] Steele K, AbedT, ChungA, KedoinR, Rosner R, RyanR, Schwartz J and Skinner B. Diagram!V. 1.1. Lighthouse
Design, Ltd., ChevyChase, MD,1991.
[15] Paulisch F N and Tichy WE EDGE:An Extensible GraphEditor. Software Practice and Experience, 1990, 20:63-88.
[16] Costagliola G and ChangS-K. DRParsers: A Generalization of LRParsers. In Proceedingsof the 1990 IEEEWorkshop on Visual Languages. Los Alamitos, CA: IEEEComputerSociety Press, 1990:174-180.
189