From: AAAI Technical Report SS-98-04. Compilation copyright © 1998, AAAI (www.aaai.org). All rights reserved.
Multi Modal Methods of Information Transmission
Jim Brander
Interactive Engineering Pty Ltd
HarrisPark
Sydney
Australia
intcng@ozemail.com.au
Abstract
substrate, or for glue that will bind the methodsthat have
been implemented.The search for a "glue" is reminiscent
of the early descriptions of Expert Systems.Analgorithm
for combiningrules wouldbe described, followed by a
description of the properties of the Help systemto be
implementedlater that wouldexplain the ES results. The
flexibility and reasoning powerof the Help system would
far surpass the limitations of the Expert Systemalgorithm,
to the point whereif the Help system ever became
available, one could throw awaythe Expert System.
Different methodsof transmissionof
informationare describedwithin a network
formalizationof the analyticoperators.
Examples
are given of whereswitching
occurs between
the methods.Control
of
switchingis takingplaceat the lowest
possiblelevel, the operatorsthemselves.
It is
suggestedthat, for complex
highlevel tasks,
no simple meansof problemdecomposition
and assemblyof separate dedicated
subsystems
is possible,nor is one needed.
Introduction
The desire for different modesof reasoning to attack
different problemsseems reasonable whenthe limitations
of various current methodsare assayed. The obvious
problemwith different modesis that someonehad to
understand the problemcompletely to choose the modenot very likely for reasonably hard problems.Success
dependsvery muchon the level of reasoning being
attempted - are weattemptinga rapid
response to a
complex
scene,
or theslowandpainful
accretion
ofnew
concepts
overa period
ofyears.
Evenatthese
extreme
ends
ofthespectrum,
someunderlying
flexibility
offormand
capability
forself-modification
isrequired
ifwearetouse
the term "reasoning". There have been manyattempts to
decomposeproblemsinto disparate elements to suit
preconceivednotions of reasoning methodsor efficient
algorithms. Except in narrow areas, the decompositionand
reassemblyprocess is rarely successful without destroying
the essenceof the problem,or requires a large effort on the
part of the humanusers to hold all the pieces together in
phase.
It might be worthwhileto analyze what is meantby multimodalreasoning. Are there different formsof reasoning,
exemplified by Expert Systems and Constraint Handling
Systems,
oraretheymerely
narrowinterpretations
ofmore
general reasoning. Arc we searching for a common
Similarly, if wecould find a glue or "super-algorithm"
which understood enough about the operation of each of
the different modesof reasoningto be able to assign tasks,
it coulddo all of the reasoning, perhapsnot as efficiently
but certainly morecompletely. Each of these modesof
reasoning has madecertain assumptions to allow operation
in its specific domain.Whilethese assumptionsare at least
partially understoodby the users, enthusiasmfor a
particular modeoften outweighscareful analysis of the
potential variability of the overall problemstructure.
Wemight visualize the different modesof reasoning as
different forms of information transmission amongnodes,
as
Constraint
~easoning
Figure1
106
SoMng
In the Expert Systemcase, we might use goal directed
searching or dataflow. In the Constraint Handlingcase, we
allow alternatives at the boundarynodes, and turn the
constraints around. In the case of Consistent Reasoning,we
use the pathwayswithin the statements, and any of the
other forms of reasoningis a subset, with operations on the
information and its direction of propagationbeing
determinedat the operators.
Control
Perhaps we should look moreclosely at the analytic
operators weuse. It turns out that in our current
implementations, we can hardly combinetwo analytic
operators before we are forced to start stripping away
inferences. Witheach inference lost, it becomesmore
important that a humanuser understand the intended use of
the implementation. Our methodsof mathematical and
logical analysis are incomplete,and haveinconsistencies
needingto be papered over by a humanuser, particularly at
the interface betweenlogic and numbers,or where
existence of objects is relevant, or wheretopological
changecan occur during analysis.
Figure 2
The structure implements
IF A<B THENC <D
but contains far morethan wouldbe initially assumedfrom
the textual form. The nodes and links maycon~in logical
states, singular numericvalues, alternative numericvalues,
lists of objects or lists of alternative lists. Thearrowson the
connectionsindicate the potential directions of information
flow, in that informationcan flow in either direction, and,
if consistency is maintained,can flow consecutively in
either direction. IfA is actually less than B, then a TRUE
flows out of the less than operator to the implication
operator. If its control pin is asserted, a TRUE
flows out of
the consequentpin to the other less than operator.
However,ifC had been greater than D, then a FALSE
wouldhave flowed the other way. If control had not been
asserted, a FALSE
mayhave flowedout of the control pin.
Thestructure is clearly a logical model,and its application
to constraint reasoning and rule based reasoning should
also be clear. Anessential point is that it is not using
Booleanlogic. To do so woulddestroy too manypotential
inferences. Onlyif the problemis perfectly static can webe
sure the inferences discarded will never be needed. This is
an exampleof where the methodsof analysis we choose
can serve to defeat us. Logic, both in our beadsand in a
complexproblem, is better described by a squirming mass
of connectionsthan it is by numbers,whichsit as a thin
layer on top of logic.
A networkimplementationof analytic operators will be
used to highlight somedifficulties with a modeintegration
approach.The networkis intended for use in high level
planningand design, wherethere can be no artificial
distinction amongrules, relations and constraints. Planning
requires that a potential structure be assembled,evaluated
perhaps using consistent reasoning, and then the planning
systemmoveto a newstate, inconsistent with the last, and
continue. As different forms of reasoning are needed,
different modesof information mmsmissionthrough a
common
structure are utilized. This paper will attempt to
showthrough exampleshowthe Integration of different
methods of information transmission through a common
realized sU’uctureseemsessential if we are to find a wayof
combiningdifferent modesof reasoning.
Reasoning
Structure
The elements of the networkstructure are the analytic
operators, ranging over objects, numbersand logic, and
being close analogues of the common
operators. The main
distinction with previous attempts at networkformalismis
that an attempt has been madeto fully modelthe basic
properties of each operator so that no potential inference is
lost. Anotherdifference is the dynamicnature of the
structure. A simple logical constraint mayserve as an
introductory example.
If we wish to combinemethodsof reasoning where the
implementationshave destroyed manyinferences, either the
system combiningthem mayneed to recreate the
inferences, or we should not have destroyed themin the
first place.
The conversion betweentext form and networkstructure is
not alwaysdirect, as inferences mayexist for us in our
understandingof the text form that wouldnot survive direct
conversion to networkform with its operator independence.
107
Onesuch case is Figure 3.
Information Transmission and Structural
Change
Transmissionof information through the structure varies
among
Searching
Dataflowwith killing
Consistent dataflow
Dataflowwith structure growth
Structure assemblyand activation
dependingon the purpose to whichthe structure is
currently being put.
Figure3
IFA <B THENX--- 5
ELSEX = 10
Wecan easily conclude from this statement that X may
have the values 5 or 10 (the range 5,10). Whena statement
such as this is convertedto networkform, additional
structure is addedto allow the inference (and structure to
control the inferenceis also built - the statementitself may
not be unconditionallytrue).
Oneother element needs explanation for the following
discussion. A cluster of operators forminga statement itself
becomesan element attached to a logical spine, over which
control maybe exercised, the spine beginningat a logical
variable. If the spine is not activated, an Unknowable
state
is usedto control the invalid statements,as a False state
wouldimply inversion.
The realized networkform allows extensibility, by a user
and by the operators of the structure acting directly uponit
to changethe topology or accrete newstructure. This
extensibility is in clear distinction to other methodswhich
rely on a stack, up which they maybecomestuck.
The structure which performsthe reasoning is both capable
of changeto its topology, and recovery from that change,
unlike other methodswhichimposedirectional barriers as a
result of either a misunderstandingof the meaningof
analytic operators or as an artifact of their implementation.
Searchingmovesthrough a structure initially emptyof
information,attemptingto return to the search starting
point with useful information. Searchingis recalled if
dataflow overrides the need. Searchingmayinstigate
structural change.
Dataflowwith killing causes information to be pushed
downapplicable pathways.If newinformation is to replace
old, the old informationis killed (the relevant part of the
structure is emptiedof information) before newinformation
is transmitted.
Consistent dataflow allows newinformation to overlay old
without killing, wherethe newinformation is consistent
with the old, as a newrange of 3..7 is consistent with an
existing range of 2.. 10. Constraint dataflowis not limited to
numbers,
objects also being providedas alternatives.
Dataflowwith structure growthimplies that arrival of
information at a node causes growthof new structure,
through whichinformation is again transmitted, either from
the originating node, or from the other structure to which
the newly created structure becomesattached.
Structure assemblyand activation describes whereexisting
structures are sought and selected basedon particular
properties, they are connectedand then logically activated
by driving logical states to their spines.
Backtrackcan occur on structure creation and change, all of
the logical structure being madeout of the samestuff.
All of these methodscan be used together in any parallel
combinationor sequence. A search through a structure may
identify an area suitable for consistent reasoning, whichon
completionor failure causes killing of informationor
creation of newstructure. Constraint reasoning on some
area of the structme mayallow transmission of information
whichthen permits constraint reasoning in another area.
Some Examples
Theindexmayitself be a list, as
ASD[{BFG[GHJ[X,{FGH[3]}],4],PQR}]
Bridging Methods Of Reasoning
¯ . . . ¯ . . . . .............
.
"¯’.
The[ ] indicates the indexvalue, the { } indicate a list
structure.
. . ¯ . . . ,
Searching
’’-..
¯ - -
Herethere is a mixtureof single values, alternatives and
lists. Theindex operator need understandnothing of the
structureof the list, insteadrestricting itself to the
operationof taking the elementrelevant to it fromthe list,
then creating a newindex operator and passing the
remainingindex to the newstructure. If there are ranges in
the index value, these ranges cause newoperators to form,
with initial connectionto all the alternative objects. Later
pruningeither on the index or the output list reducesthe
numberof alternatives, destroying the operator supporting
the alternatives if the alternatives reduce to a singular
value.
The sametransmission mechanism
that transmits lists in
goal directed searching and dataflowalso transmits
alternatives in Consistent Reasoning,so it doesn’t matter
what mixture the index is of different "modesof
reasoning". There maybe areas of the modelwherethere
are too manyalternatives to handle, so this part of the
modelwill wait for other modesof reasoning to reduce the
variability to the point whereconsistent reasoningcan
begin - another exampleof the essential intertwining of
different modesof reasoning.
."°
Io,,o1
...........
.........
//
¯.
’...
Consistent
Reasonlng
¯ . . . . . .........
:
...
¯ ...."
Figure 4
Someconstraint handling tools (CHARME)
provide
Demon
operator whichwill allow constraint solving to fall
back to a programmaticmeanswhenappropriate - that is,
whena changein the range occurs, a procedureis run. The
analog for a network where programmaticmeansdoes not
exist is a DEMON
operator whichcan be built into the
networkstructure, as shownin the diagram. Here the
structure is identical in formon either side of the operator,
the only difference being the methodof transmission of
information Control can be exercised so that bridging only
occurs whenthe range has fallen to a low level, or whena
unique value is presented. A fall back to a programmatic
meansis an admissionof failure of the reasoning paradigm,
implyingthe initial meansis insufficiently extensible for
the problem.
It mightbe claimedthat this formof list indexingis no
morethan a crude form of parsing, except that it should be
notedthat the result of all this generationaland
transmissionactivity is to finally connecttwoor more
objects, without any knowledgeabout direction of
informationflow, or if informationflow is currently
possible, and to have the methodof connectionreversible,
both by killing and backtracking.
List Indexing
Start Hint
Withinthe network, lists maybe indexed, as ASD[X],
whereXis singular, or has alternatives. Figure5 illustrates
the structure obtaining wherethere are two alternative
values for the index, and the objects indexed are numbers.
’Z
Figure 6
Figure 5
109
Resourceusageis an important aspect of project planning.
A resource usage operator in a planning networkhas
consistent informationon most of its connections, but also
needs to handle informationwhichis inconsistent with that
already sent. AStart Hint is typical - there is some
calculation that says whereresource bookingshould
preferentially occur, and the calculation changesits output
value dependingon the range and position of the available
bookings.This newvalue, inconsistent with the last, must
be transmitted through a networkwhich everywhereelse is
using consistent dataflow. Figure6 illustrates the different
types of transmission - links markedwith a 1 are using
bidirectional dataflow (that is, newvalues flow back and
forth, overlayingthe old), links markedwith 2 have a value
that is transmitted once, and the link markedwith 3 has
informationthat is repeatedly killed and re-transmitted. The
changein methodof data transmission is occurring at the
lowestpossible level, at each operator. In this case, the
receiving operator mustunderstandits role. If killing
escapedthis single connection,all other informationin the
network wouldbe destroyed. The examplein shownin
Figure 5 maynot be defining, but it is a goodexampleof
howsterile a single methodof "reasoning" wouldbe.
stabilizing. If welooka little moreclosely at eachiteration,
wecan see a multiplicity of methodsin action.
Actl~"~Constraln
Search////
I
~onstrain 2
Figure8
The planning system needs to find and assemble the
elements of the plan, and then handle wavesof constraint
action. It can’t start to constrainthe use of resourcesuntil it
is knownhowmuchresource is to be used. Wedon’t know
whethera range evenexists until other areas in the
reasoning modelhave stabilised - 0 does not describe
existence. A complexplan has manylevels and local areas
at whichiteration maybe occurring at different phases, not
the simple sequential phasing shownhere..
A planning systemthat could not automateall of the
planning cycle wouldbe of limited use, requiting the
human
users to fill in the gapsin its analytic capabilities,
and risk loss of phase amongthe components.The method
of information transmission needs to smoothlychange from
Searching to Consistent Datafiowto Dataflowwith Killing,
so that a newstate maybe found that is inconsistent with
the previous one. Askilling proceeds, someof the structure
maybe taken apart and destroyed, and newstructure built
in its place.
Application Area - High Level Planning
Businessor project planningis typified by the construction
of a modelwhich simulates the expected behavior of the
real activity. The morecompleteand extensive the
simulation, the morethat people can cometo understand
the issues involvedand the consequencesof their actions.
Therewill usually be constraints on the plan, there maybe
options whichserve as cases, and there will usually be
rules that needto be built into the model.
As already mentioned,Booleanlogic is inappropriate in a
high level area becauseof its destruction of knowledge.It
should also be evident that a destructive methodof
reasoning, reductio ad absurdum,is inappropriate where a
suitable structure has still to emergeas part of the planning
process.
Consistent - Inconsistent - Consistent
As an exampleof rules, there maybe a clause in the project
contract describing incentive paymentsthat apply to
completion.
Figure 7
The values used in Constraint Reasoningremain consistent
as a wayof ensuringthat no constraint is violated. This
workswell until a situation develops wherea changeis
necessary- there just isn’t enoughresource to do the job on
time. Mostplans go through manyiterations before
110
reasoning, either in isolation or in macrocombination,
seem far too narrow to describe the methodsof reasoning
required in the fields of high level design and planning.
References
Cras, J-Y. 1993. A Reviewof Industrial Constraint Solving
Tools. A.I. Intelligence, Oxford,U.K.
Brander, J. and Dawe,M. 1998. Use of Constraint
Reasoningto Integrate Risk Analysis with Project
Planning. The International Journal of Project and
Business Risk Management.Forthcoming
Figure 9
In simple form, a rule linking project duration and project
cost might be
IF ProjectDtaation < 200 THENIncentive = 400
ELSEIncentive = 0
Thisrule (or is it a constraint) is built into the network
model, allowing an undirected connection betweenduration
and cost. Initially, the range of projectDurationspans 200.
Additional constraints maybe applied on either cost or
duration, with resultant switchingoccurring at an operator
within the slructure of the rule to constrain projectDtaation
or Incentive. SomeConstraint Handling systems have
attempted a kludge to connect betweenBooleanlogic
conditions and constraints - each ldudge pushes us further
awayfrom a seamless handling of complexproblems. The
rule shownhere is simple, but mayinstead be of arbitrary
complexity, and use any mixture of the techniques in its
evaluation.
Conclusion
This brief overviewof different methodsof information
transmission through a networkstructure mayhave
illustrated howmulti modalreasoning should have a
commonsubstrate.
Hopesthat we can somehowimplementlarge pieces of a
problemusing different methodsof reasoning, then
combinethemat a high level are likely to provefalse.
Examplesin high level planning illustrate that the planning
problemwouldbe destroyed if it were cut into elements
seeminglysuitable for different modesof reasoning. Put
another way, the only level at whichdifferent forms of
reasoningwill be successfully combinedis at the level of
the analytic operators, because only they are close enough
to the problemto decide howparticular areas need to be
handled.
The Expert System and Constraint Handling modesof
Iii