From: AAAI Technical Report FS-92-02. Copyright © 1992, AAAI (www.aaai.org). All rights reserved. Becoming Knowingly Interactive Actuators with Sensors and John Budenske and Maria Gini * ComputerScience Dept. University of Minnesota 4-192 EE/CSci Building 200 Union Street SE Minneapolis, MN55455 Abstract In order for a mobile robot to accomplish a non-trivial task, the task must be described in terms of primitive actions of the robot’s actuators. Weclaim that the transformation from the high level task description to the primitive actions should occur primarily at execution time, when information about the environment can be obtained through sensors. Our claim is based on the premise that proper application of knowledge in the integration and utilization of sensors and actuators increases the robustness of execution. This paper addresses the issue of what knowledge needs to be knownabout sensors, actuators and processes in order to be able to use them. The proposed method of execution works for any sensor/actuator existing on the robot when given such knowledge. Knowledge About Utilizing Sensors/Actuators A very large part of our civilized world is unstructured and dynamic. This fact has been one of the leading obstacles to mass-quantity introduction of robots into everyday life. Thoughthere are factories and buildings in which highly structured environments have been introduced for easy integration of robots, the ratio of unstructured to structured environments will always be high. Thus, society’s utilization of robotics will be dependent upon the ability of robots to move safely in these unstructured and dynamic environments. This means that robots must achieve their given tasks despite unexpected changes in their environment or failures of their sensors. The variability of the world makes it impractical to develop very detailed plans of actions before execution (the world might change before execution begins and thus invalidate the plan). It makes muchmore sense to *Thisworkwasfunded inpartby theNSFundergrants NSF/CCR-8715220 mad NSF/CDA-9022509, and by the AT&TFoundation. 23 develop beforeexecution an abstract, detail-less plan of whatgenerally shouldbe accornplishcd, and then makethenecessary details explicit during execution. Of course, unstructured anddynamic environments require therobotto remainreactive to environment changes, andso "directed control froma plan"mustgive-way to "directive reaction" within therobot. Thus,we arefacedwitha problem: how cana robot execute an abstract planto accomplish goalsandstill remainreactive to its environment? Our approach to thisproblem is to determine, at execution time,the sensors, actuators, andprocesses necessary foraccomplishing an abstract plan,as wellas thecommands necessaryto control theseentities. Depending on informationsensedfromtheenvironment, theflowof datafrom sensors intoactuators iscontinuously reconfigured. Our approachis basedon thepremisethatproperuse of knowledge increases the robustness of planexecution without reducing theability to reactto theenvironment [Bro86,Fir87,Ark90,Fir92].The centraldilemma is defining how to transform thegivenabstract-goal intoan explicit representation of whatis necessary to achieve thegoal.We callthistransformation process Explication andhavediscussed itsimplementation in previous papers[Bud91, Bud92]. In developing theconceptof Explication, a numberof sub-problems quickly becameapparent: ¯ Whatinformation is relevant to accomplishing the goalandthusneedsto be sensed? ¯ Whatactuation details needto be utilized to accomplishthegoal? ¯ Howis sensedinformation usedto aidthegeneration of actuatorcommands? ¯ How does the systemknow when an errorhas occurred,and whenthegoalhas beenaccomplished? Whatknowledgeneedsto be knownaboutsensors, actuators, andprocesses in orderto be ableto integratetheirutilizations intoa complete system, and thencontrol thesystem forsuccess? From: AAAI Technical Report FS-92-02. Copyright © 1992, AAAI (www.aaai.org). All rights reserved. Manyof these questions have been addressed in the aforementioned papers. In this paper we concentrate on the last of the above questions. To utilize any sensor, actuator, or process, knowledgeabout its usage must be available. This knowledgemust cover the entity’s functionality, control characteristics, and limitations. We refer to this knowledgeas a Sufficient Description of an entity and define it to be all of the knowledge about the entity which is required by the Explication process to utilize the entity in accomplishinggoals. First, we briefly discuss the concept of Explication, and then present the contents of a Sufficient Description and our current results for an indoor mobile robot. Explication Weview a plan as a collection of partially ordered abstract-goals (such as "Move Through Doorway") much like what a classical planning system would produce. Executing an abstract-goal is difficult because the abstract-goal implicitly represents a large collection of information and details. To be executed, a plan must explicitly specify the details of howto utilize the sensors and actuators. For the same abstract-goal, different environment situations may require different sensors, different programmingand control of the sensors, and different strategies. Explication is also difficult because of the need to adapt to the environment as the commandis being executed. Twoconcepts are crucial for Explication. First is the notion of Relevant Information Need (RIN). Explication must deal with the question of "what information (both sensed perception or internal state) is relevant and thus needed in order to execute this abstract-goal?" To put it another way, the abstract goal: "how to achieve X" is replaced by two things: a) explicit knowledge on "how to achieve" using the sensors and actuators and b) an explicit representation of the needed information. This representation uses abstraction to reduce dependency on the source of the information. Abstract informational needs can then be met by abstract information providers. Wehave utilized the concept of "Logical Sensor" [Hen84, Hen85] to be such an abstract information provider, and we have expanded it to allow other logical entities. Equally crucial is the concept of Utilization-Detail. Explication must deal with the question of "what details of using sensors, actuators, and processes are necessary in order to execute this abstract-goal?" Again, abstraction is used to represent control of sensors and actuators. The control primitives are commandsand their parameters for both discrete and continuous actions. Wepropose a framework for Explication, called the Logical Sensors/Actuators (LSA). The framework is being implemented in an object oriented programmingenvironment resulting in a flexible robotic sensor/control system which we call the Logical Sensor Actuator Testbed (LSAT). Primarily, the framework is a col- 24 lection of reconfigurable components and flexible component structures which can be combined to achieve abstract-goals through execution. The framework consists of the knowledge, mechanisms,and structures used by the Explication process. The concept of Sufficient Description is included as one type of knowledge. Implementation of LSAT The LSATframework is being implemented on a SUN SPARC4/330 computer which interacts with a TRC Labmate mobile robot over two separate serial lines. The system is being written in C, Lucid CommonLisp with the CommonLoops Object System (CLOS) and LispView windowing system. An object oriented approach has been used in the implementation, where an object corresponds to a Logical Sensor/Actuator (LSA) entity. Within the LSATframework, we have developed five classes of LSAs: 1. SENSOR:Takes raw/partially processed data as input and outputs data which are further processed. 2. DRIVER:Accepts as input multiple commands for individual hardware/drivers, acts as an interface to the hardware, performs command scheduling, resolves minor commandconflicts, and routes major conflicts to its controlling LSA. Accepts sensor data as input and 3. GENERATOR: outputs a commandmeant for a DRIVER.This class can be viewedas a low level, feedback control, looping mechanism between a sensor and the actuator. It is much like the SENSOR,only it 4. MATCHER: also takes as input a description of a goal or error situation. The processing consists of matching the goal/error to the input sensor data. 5. CONTROLLER:Accepts processed data from any other class, as well as commands and parametervalues from other CONTROLLERs higher-up in the hierarchy, and outputs the control commands and parameter-values (referred to earlier as UtilizationDetail) to its sub-LSAs. CONTROLLERs are the main entities used to implement Explication. They control the other LSAs, and manipulate them through the process of Explication. One of the purposes of the framework is to differentiate the domain dependent and independent mechanisms. This will allow us to implement a platformindependent testbed, and to perform empirical studies on the use of knowledgeduring execution and the identification of relevant information, utilization-details, and the contents of a Sufficient Description. Wehave derived a "template" for each LSAwhose "slots" must be filled in with knowledge. This procedure constitutes deriving the Sufficient Description of a given LSA, and it is our belief that only through empirical research can such knowledge be developed. From: AAAI Technical Report FS-92-02. Copyright © 1992, AAAI (www.aaai.org). All rights reserved. Contents of a Sufficient Description The contents of a Sufficient Description for a LSAhas been developed empirically as the LSAThas been implemented. From our results we have developed a template where the knowledge is grouped into six categories, as shownin Table 1. Selection Name: Polar-Direction Expected-Data- Type: angle Uptodateness-Level: immediate Access-Type: default Source-Selection-Function: n/a Source- Verification-Function: n/a Mapping-to-Input: input-dir-source Characteristics: Primary-Effects: Utilization Characteristics (for each RP): Number-RPs: 3 Name: Speed Type: velocity Units: mm/sec Value-Ranges: slow, moderate, fast Value-Increments: faster, slower Default- Value: slow Result-Effects-Mapping: (Table (slow. 25) (moderate. 40) (fast. 60) (faster. +5) (slower. Name: Aim Type: direction Units: degrees*100 Value-Ranges:right, left, straight Default- Value: straight Result-Effects-Mapping: (Table (right. -t-50) (left. -50) (straight 0)) Name: ForeBack Type: direction Units: n/a Value-Ranges: forward, backward Default- Value: forward Result-Effects-Mapping: (Function setforeback forebackparmlist) Execution Methods Characteristics: LM-MOVEMENT Side-Effects: SLOW-MOVEMENT,PROPELLING Appropriate- Usage- Context: move-through-doorway, move-under-table, close-quarters-move, push-object, traverse-hallway Inappropriate- Usage-Context: avoid-fast-obstacles, cross-open-area Selection- Utility-Function: none Results Characteristics: Output-Type: ppgo-commands Output-Format: L-drv-LM-format Output-Resolution: 5mm Output-Accuracy: 75 Output-A verage-min: 25mm Output-Average-max: 2000ram Error-Rate: 10 Output-StDev: (-10mm. +10mm) Output-Repeatability: RIV-set: init, reset, ready, shutdown,goal, null-input, waiting, propelling, aiming, idle? Integration Characteristics: Interactions-to-Monitor-For: obstacle-avoid Errors-to-Monitor-For: bump, front-align Time-to-Execute-One-Cycle: 0.01see Min-Cycle-Interval: 1.0see Hardware-Resources-Required: none Limitations: none Needed Information Characteristics KIN): Number-RINs: 2 Name: Path-Blocked Expected-Data- Type: binary Uptodateness-Level: immediate Access-Type: default Source-Selection-Function: n/a Source- Verification-Function: n/a Mapping-to-Input: input-path-source (for Methodof Initialization Method of Resetting Method of Fault Checking Method of Shutdown Method of Returning RIV Method of Accessing Result Data Method of Primary Behavior each Table 1 - Contents of a Sufficient Description for the LSA "L-gen-propel" The first category, Selection Characteristics, contains the knowledge needed by a CONTROLLER to decide whether this LSAwould suffice (and later be best) a source to satisfy a RIN. This knowledge is very much 25 From: AAAI Technical Report FS-92-02. Copyright © 1992, AAAI (www.aaai.org). All rights reserved. like that used in automated planning to select operators to expand a plan network. The next category is the Results Characteristics which includes knowledge about the data which the LSAoutputs. Some of this information is also used by CONTROLLERs in the selection of LSAs. The Integration Characteristics is the knowledgeused to ensure that the instantiation of the LSAwill proceed correctly and that potential interactions are checked and monitored for. Limitations refers to miscellaneous information on what interactions may reduce the capabilities of the LSA. The Needed Information Characteristics is the knowledge required for each of the "data inputs" of the LSA. Input data for a LSAis viewedas an informational need. The CONTROLLER will use this knowledge during its determination of Relevant Informational Needs on its level (ie. a CONTROLLER’s RINs will include the tLINs corresponding to sub-LSAdata inputs). More importantly, the LSAitself uses this knowledgeduring its execution as a guide in collecting its input data from its indicated sources, as well as determining when its RINs are unsatisfied (ie. no source named, or source LSAhas become faulty). For example, the Source Verification Function is used to interact with a selected source LSA to determine if the output data from that LSAis valid for use. The Source Selection Function will be used by the CONTROLLER to determine which of two acceptable LSAswould "best" satisfy this RIN. Currently this research is not investigating howto select the best sensor (including sensor positioning), but only acknowledges the need for such knowledgewithin the Sufficient Description. The reader is referred to [Hag89, Abr92] for more thorough research in that area. The Utilization Characteristics pertains to knowledge about the input parameters of the LSAused to control its behavior. Utilization-Detail knowledgecovers the acceptable ranges of input values and attempts to describe the correspondence between numerical parameter values used by the actual algorithms within the LSAand the symbolic parameter values used by the CONTROLLER to manipulate the LSA. For example, across multiple LSAs for generating movement commands, it is difficult for a CONTROLLER to contain knowledge about what velocities are acceptable for that individual LSAand what effects each individual desired velocity has. Thus, it is better for the CONTROLLER to communicate symbolically to the sub-LSA. Instead of passing downa specific velocity (ie. 50 mm/sec), relative values are passed such as "slow .... medium"or "fast". The LSAthen makes the decision on the actual numerical value. If during execution, the CONTROLLER desires faster movement, it just passes another relative symbolic value, "faster" onto the active LSA, and the LSAwill adjust its velocity calculation appropriately. Thus, knowledge about specific calculations and adjustments to velocity is held within the sub-LSA, and the CONTROLLER only re- 26 quires knowledge about which abstract, symbolic parameter values to use for a specified type of UtilizationDetail. The last category is the Execution Methods Characteristics, which is procedural knowledge about what needs to be done within a LSAin order to achieve a specific internal state. These are the algorithms for initialization, resetting, shutting down, and performing the primary behavior of the LSA. These methods conceptually constitutes the working "guts" of the LSA. The contents of a Sufficient Description mayvary or emphasize different aspects from one LSAtype to another. For example, the DRIVERLSA Output Descriptions is not as important as the SENSOR LSA’s since DRIVERsdo not "output" to other LSAs, they only communicate to the hardware. It is important to note that Sufficient Descriptions of CONTROLLER LSAs must also be gathered. Many CONTROLLERs will utilize other CONTROLLERs as sub-LSAs and thus will need knowledge about the sensors, actuators and processes that the sub-CONTROLLER utilizes. Since muchof this information is dependent on the selected sub-sub-LSAs of the sub-CONTROLLER, CONTROLLERs must have mechanisms to progressively collect and update their RIN and UtilizationDetail knowledge as they determine sub-LSAs. These mechanismsare still in development. A Short Example The example given in Table 1 is for a LSAof the class L-Generator, subclass L-gen-propel. L-gen-propel takes in information about directions to move, and about obstacles in the path and generates simple actuator commands for moving the robot forward or backward. The Sufficient Description of L-gen-propel’s is used by other LSAsto determine how to interact with L-gen-propel. To further illustrate this example, assume that one of the abstract-goals the system is attempting to accomplish is one to movethe robot directly through a narrow doorway, and the robot is already aligned to the passageway. The particular abstract-goal corresponds to a L-Controller LSAof the subclass L-ctrl-close-quartersmove. This LSA attempts to move the robot slowly through the narrow passage, and monitor the movement for mis-alignments, obstacles, and proper progression. In applying its Explication knowledge to the problem, L-ctrl-close-quarters-move determines that it requires the robot to moveforward (this is interpreted as a need for a LSAwhich will produce velocity/steering actuator commands).This need can be satisfied by selecting a LSAbased on features mapping to knowledge within the Sufficient Descriptions of all LSAsin the system. The L-ctrl-close-quarters-move queries the library of LSAsfor one which has a behavior, or Primary-Effect, of LM-MOVEMENT. Since there are a number of LSAs which deal with moving the robot (LM-MOVEMENT), other features (such as Side-Effects, AppropriateUsage-Context, Inappropriate- Usage-Context) are used From: AAAI Technical Report FS-92-02. Copyright © 1992, AAAI (www.aaai.org). All rights reserved. to further down-select to a single choice. L-gen-pounce Within L-gen-propel, any type of mapping function is designed to move the robot slowly through an area or tables could be used, including ones based on other of potential obstacles, but where no" contacts are exparameters. Thus, the Sufficient Description can abpected. Thus, SLOW-MOVEMENT is a desirable bestract the numerical values needed in the execution havior. Also, both abstract-goals of close-quarter-move of actuator commandsfrom the types of interactions and move-through-doorway (a higherlevelgoal)fall needed between L-ctrl-close-quarters-move and L-genwithintheAppropriateUsage-Context. propel. OnceL-gen-propel has beenselected, L-ctrl-closequarters-move willextract information aboutthe LSA. Related Research Forexample, L-gen-propel requires twotypesof inputThere has been a number of approaches to combindata:Path-Blockedand Polar-Direction. Both of ing planning and reacting for a mobile robot [Fir87, thesebecomeadditional RelevantInformation Needs Geo87, Ros89, Pay90, Sim90, Gat91]. None of these of L-ctrl-close-quarter-move, andsources forthatinapproaches have emphasized the determination of what put dataare determined and selectedthroughtheir sensor/actuator/process knowledge is necessary in orSufficient Descriptions (in thesamemannerthatLder to utilize them. Other work has been done in degen-propel was selected). Uponselection, dataflow veloping schema based computational processes for senpathsareestablished betweenthecorresponding LSA’s sory based robotics [Lyo89, Ark90]. There has also been outputandL-gen-propel. L-ctrl-close-quarters-move a great deal of research in sensor selection. Hager and "hooks"the two LSAsintoL-gen-propel throughthe Mintz [Hag89] use decision theory for deriving a set of slot Mapping-to-Input (ie. the nameof the LSAselected sensor observations which will yield desired sensor data. for Path-Blockedis placed in L-gen-propel’s slot: inputManyothers [Hut89, Abr92] are investigating sensor path-source, and Polar-Direction in input-dir-source). planning techniques for determining locations and moWhenL-gen-propel requires its input data, it will actions, required for multiple vision sensors in order to cess it directly from the LSAswhich are deriving it. view (monitor) the features of interest during task exL-ctrl-close-quarters~move utilizes L-gen-propel’s Suffiecution. This research is similar to the selection of a cient Description to determine what other information source of information (ie. an LSA). Thoughsuch selecis relevant and where it is needed. In the same mantion mechanisms are important, they are not emphaner, the slots Errors-to-Monitor-For and Interactionssized in the LSATresearch as much as how knowledge to-Monitor-For also provide indications as to other about (and required by) selection mechanismsrelate LSAswhich may he needed to aid in the monitoring of the other componentsof a Sufficient Description. the goal accomplishment. Thus, the Sufficient DescripHenderson [Hen84, Hen85] combines a declarative detion of L-gen-propel aids in determinating information scription of the sensors with procedural definitions of relevant to accomplish the goal. sensor control. This research, termed "Logical Sensors" There are two last ways in which the Sufficient Deconsists of logical/abstracted views of sensors and senscription is used to by other LSAsto interact with Lsor processing, muchlike howlogical I/O is used to ingen-propel. Both deal with the control of the LSAdursulate the user from the differences of I/O devices and ing its execution. The first is in utilizing the Execution operating systems. Workhas been done to extend the Methods(Initialization, Resetting, etc.). As mentioned concept to include actuator control [All87]. In both, the before, these are procedural knowledge mechanisms development of a Logical Sensor includes determining which all LSAsmust have and they are used extensively knowledge about how to use sensors, much the same as during LSAinteractions. The last use of the Sufficient the knowledgecontent of Sufficient Descriptions. Description is in interacting with the Result-Parameters of the LSA. Simply, L-ctrl-close-quarter-move’s knowlConclusion edge about moving through the door passageway inCurrently, we are implementing the process of Explicacludes the fact that most LM-MOVEMENT LSAs have tion within the LSATframework (see [Bud92] for deResult-Parameters dealing with Speed, Aim, and Foretails on results). As we have been building the library Back as well as knowledge about potential values for of LSAsused by Explication, we have been empirically them. The Value-Ranges and Value-Increments slots developing our definition of Sufficient Description. This are accessed to identify allowable values for these paresearch is truly work-in-progress as we have not comrameters. As the LSAis executing, and propelling the piled a great deal of results on howthe use of the knowlrobot through the passageway, L-ctrl-close-quartersedge within a Sufficient Description is enhancing the movemonitors the progression, and adjusts the allowsystems operation. Wehave only compiled "templates" 1able parameter values which it has knowledge about. of what knowledgeis currently required for Explication to utilize a LSA, and attempted to formalize it into 1Thoughthe possibility that none of the allowable paan understandable explanation. We have determined rameter values will be understood by the controlling LSA that the development of the LSATframework, includhas not been address, a process of trial and error could be applyed. BUT,that leads to automatedlearning, and that ing the derivation of knowledge about Informational can be someoneelse’s thesis. Needs, Utilization Details, Sufficient Descriptions, and From: AAAI Technical Report FS-92-02. Copyright © 1992, AAAI (www.aaai.org). All rights reserved. Explication has provided us greater insight to the use of sensors for accomplishing intelligent behavior of mobile robots. Wehope, by identifying how abstract plans should be executed, that what sensor, actuator, and processing knowledgenecessary to achieve them is identified as well. [Hen84] [Hen85] References [Abr92] [All87] [Ark90] [Bro86] [Budgl] [Bud92] Steven Abrams and Peter K. Allen, "Sensor Planning in an Active Robotic Work Cell," Proc. of the DARPAImage Understanding Workshop, January 1992, 941-950. Peter K. Allen, "A Framework for Implementing Multi-Sensor Robotic Tasks," Proceedings of the DARPAImage Understanding Workshop, February 1987, pp 392-398. Ronald C. Arkin, "The Impact of Cybernetics on the Design of a Mobile Robot System: a Case Study," IEEE Trans on Systems, Man, and Cybernetics, 20(6):12451257, 1990. R. A. Brooks, "A Robust Layered Control System for a Mobile Robot," IEEE Journal of Robotics and Automation, RA-2(1):14-23, 1986. John Budenske and Maria Gini, "Determining Mobile Robot Actions which Require Sensor Interaction," Working Notes of AAAI Fall Symposium on Sensory Aspects of Robotic Intelligence, Monterey, 1991, 117124. John Budenske and Maria Gini, "Achieving Goals Through Interaction with Sensors and Actuators," IEEE/RSJ International Conference on Intelligent Robotics and Systems, Raleigh, 1992, 903-908. [Fir87] R. J. Firby, "An Investigation into Reactive Planning in Complex Domains," Proc Sixth National Conference on Artificial Intelligence, Seattle, July 1987, pp 202-206. [Fir92] R. J. Firby, "Building Symbolic Primitives with Continuous Control Routines," Proc. Artificial Intelligence Plannign Systems, College Park, Maryyland, June 1992, pp 62-69. [Gatgl] Erann Gat, "Robust, task-Directed, reactive Control of AutonomousMobile Robots," Ph.D. Dissertation, Virginia Polytechnic Institute and State University, 1991. [Geo87] M. Georgeff and A. Lansky, "Reactive Reasoning and Planning," Proc Sixth National Conferenceon Artificial Intelligence, Seattle, July 1987, pp 677-682. Greg Hager and Max Mintz, "Computational methods for task-directed sensor data fusion [Hag91] 28 [Hutg0] [Lyo89] [Payg0] [Ros89] [Sim89] and sensor planning," International Journal of Robotics Research, 10:285-313, 1991. T. Henderson and E. Shilcrat, "Logical Sensor Systems," Journal of Robotics, 1(2):169193, 1984. T. Henderson, C. Hansen, B. Bhanu, "The Specification of Distributed Sensing and Control," Journal of Robotic Systems, 2(4):387-396, 1985. S. Hutchinson and A. Kak, "Planning sensing strategies in a robot work cell with multisensor capabilities," IEEE Trans on Robotics and Automation, RA-5(6):765-783, 1989. D. M. Lyons and M. A. Arbib "A Formal Model of Computation for Sensory-Based Robotics," IEEE Trans on Robotics and Automation, RA-5(3):280-293, 1989. D. W. Payton et al. "Plan Guided Reaction," IEEE Trans on Systems, Man, and Cybernetics, 20(6):1370-1382, 1990. S. Rosenschein and L. Pack Kaelbling, "Integrating Planning and Reactive Control," Proc. NASA Conference on Space Telerobotics, Vol 2, G. Rodriguez and H. Seraji (eds), JPL Publ. 89-7, Jet Propulsion Laboratory, Pasadena, Ca, 359-366. Reid Simmons, "An Architecture for Coordinating Planing, Sensing, and Action," in Innovative Approaches to Planning, Scheduling and Control: Proc. 1990 DARPAWorkshop, pages 292-297, San Mateo, Ca, 1990. M. Kaufmann.