From: AAAI Technical Report SS-02-04. Compilation copyright © 2002, AAAI (www.aaai.org). All rights reserved. Middlewarefor Robots? Christopher D. Gill and William D. Smart Departmentof ComputerScience Washington University in St. Louis One Brookings Drive St. Louis, MO63130 United States {cdgiii,wds}@cs.wusti. edu Abstract Roboticsystems,particularlythoseinvolvingmultiple robotsandsensors,are increasinglydistributedandbeterogeneous.Weask the questionwhethermiddleware canbe usedto build these systems,to alleviate unnecessarycomplexities of usinglow-leveltools whilestill preservingnecessarylevels of systemperformance. We motivatethe answerto that questionwithpreliminary performance studiesof a particularopen-source middlewareimplementation, and pose additional openquestions to furtherexplorethe suitability of middleware in buildingdistributedroboticssystems. exactly we meanby ’~aiddleware", and discuss its use in the context of distributed robotics and embedded sensor systems. Wethen present somepreliminary experimental results that attempt to quantify boundson the costs of using such middleware,comparedto a simpler informationdistribution mechanism.Finally, we conclude by posing someopen questions on the use of middlewarefor disu’ibutedrobotics applications. Middleware What is Middleware? for Robots Bakken(2001) describes middleware Introduction "... a class of software technologies designedto help Roboticssystems inherently require significant interaction managethe complexityand heterogeneity inherent in and coordination of diverse hardwareand software elements. distributed systems. It is defined as a layer of softSpecialized hardware and software maybe appropriate at ware abovethe operating systembut belowthe applilowlevels, such as within local sensor busses, or actuator cation programthat provides a common programming systems. However,the aggregation of such systems, even abstraction across a distributed system... In doingso, within a single robot, requires careful considerationof arit provides a higher-level building block for programchitectural issues to promoteflexibility, performance and remers than Application Programming Interfaces (APIs) use of both hardwareand software. Theseissues are espesuch as sockets that are providedby the operatingsyscially importantwhenwe consideruse that spans families of tem. This significantly reduces the burdenon applirobots and embedded systems, types of applications, models cation programmers by relieving themof this kind of of programming,and different kinds of operating environtedious and error-prone programming.Middlewareis ments. also informally called "plumbing"becauseit connects parts of a distributed systemwith data pipes and then One of the most important problems that must be addressed for a distributed robotics or embedded systemsappasses data betweenthem." plication is theuserprogramming model. Howmuchofthe Middlewareresides above the operating system and netdistributed nature ofthesystem should theuserhavetodeal layers and provides a consistent distributed programwithexplicitly, andhowmuchshould behidden byabstrac- work mingmodelto the application developer. Examplesof welltions? Theanswer to thisquestion islikely to varyfrom system to system, depending ontheuserandtheapplica- knownmiddlewarearchitectures include: tion. Some users will notcare that their system isdistributed,RPCSun’s original client-server procedural programming while others willwantfine-grained control overcertain asmodel. pects ofthedistribution mechanism. A major challenge in DCOM Microsoft’s distributed componentmodel for Windesigning anarchitecture fordistributed applications ofthis dowsplatforms. sortis toprovide a programming modelthatallows both JavaRMITheJavaCommunity’s RemoteMethodInvocaviews toworktogether seamlessly. tionmodel fortheJavalanguage. Inthispaper, wediscuss theuseofmiddleware fordistributed robot control systems. Webegin bydescribing what CORBAThe ObjectManagement Group’s(OMGs)standardforlanguage andplatform independent distributed Copyright (~)2002, American Association forArtificial Intelliobject computing. gence (www.aani.org). Allrights reserved. Why use Middleware? Middlewareis useful for a numberof reasons whendesigning large, distributed system. Portability Middleware offers a commonprogramming modelacross language and/or platform boundaries, as well as across distributed end systems. Intel-based computers running Microsoft Windows,programmedin Java can easily communicatewith Motorola-based machines running Linux, programmedin C++. Most importantly, this ease of communication happensseamlessly, without the application programmer ever havingto worryabout it. Reliability If wehavea tried and tested set of middleware, it can be reused and optimizedwith confidenceover many applications. It allows the application programmerto never have to worryabout any of the low-level aspects of distributed systems development.The same can also be said for havinga well-tested socket library that is reused in manyapplications, althoughsuch a library hides fewer of the low-level details of the communication from the programmer. Managingcumplexity Low-level programming abstractions, such as sockets and threads, can be mademore accessible through suitable (possibly object-oriented) libraries. However, programming combinations of these abstractions can be excessively tedious and error prone. Programming within the context of patternaware (Schmidtet al. 2000) middlewarecan drastically reduce both chanceof introducing errors into the code, and the amountof pain that the programmer must endure whenimplementingthe system. Why use CORBA? In this paper, we argue for using CORBA middlewarefor distributed robotics and embedded sensor applications. This choice is motivatedby two mainfactors, open standards and platform/language independence. The CORBAstandard (Object Management Group 2001b)itself is bothopento extensionanduse, and closed to modification without full review under the OMG standards process. This is goodbecauseit meansthat the standard is relatively stable (betweenmajorrevisions), should remain general-purpose (since the OMG has memberswith diverse application interests), but is still subject to changeas the technologies involved mature. CORBA is also platform and language independent. We feel that this is especiallyimportantfor applicationsin distributed robotics and embeddedsystems. Manyresearchers use C or C++to programtheir robots navigation systems, but feel morecomfortableimplementing their high-level reasoning systems in Lisp. Wewouldlike to be able to develop code on our desktops, then deploy in on an embeddedprocessor with the minimum of alterations. The platform and language independenceof CORBA makesthis mucheasier. Robot-Specific Middleware Nowthat we have identified what we actually meanby middleware, and motivatedour specific interest in CORBA mid- dleware,wego on to lookat someof the issues that are relevant to using middlewarefor robotics and embedded sensor applications. Data Propagation and Fusion Sensor data are responsible for the vast majorityof communications traffic in a distributed robot or embeddedsensor system. Thesedata can range from fine-grained readings from a single sensor elementto the results of a complexreasoning process. These data can also vary in size from a few bytes in size up to manymillions of bytes, and in granularity of transmission fromsingle events to continuousstreams. Thesevariations naturally inducedifferent levels of overhead.Welook at this issue in somemoredetail, and give somepreliminaryexperimentalresults below. Concurrency, Coordination and Control Manyforms of concurrency,coordination and control are useful and often necessary in a distributed system. For example, consider a teleoperation task, wherethe teleoperated robot is particularly simple, and does not carry manycomputational resources. Muchof the sensor interpretation and reasoning is done in a remotemachine, or by a humanoperator. The controlling system might need to (1) monitor the remote system’s bumpersin order to makelocal navigation decisions, (2) activate a certain high-cost sensor occasionally, and retrieve its readings and (3) record a stream profiling data summarizingthe current state of the remote robot. In the first task, sensitive responseto environmental changes wouldlikely need to be handled in an asynchronous manner, which maysuggest a fine-grained asynchronous event (O’Ryan&Schmidt 1999) push model, from the remoterobot to the local system.In the secondcase, an asynchronous messaging (Obj 1998) model would support multiple concurrentqueries fromthe local end-systemto the remoterobot. Finally, the third case mightbest be handled by a standard distributed object computing(DOC)method invocationfromthe robot to the local system,to ensurethe profile wasdelivered intact and in sequence.__ An Or mese approaches must ee supponeo searnlessly and with the minimum cost to the programmerin maintenance and feature complexity. Webelieve middlewarecan meet this challenge, offering a consistent and reasonable programming environmentfor distributed robotic systems. Quality of Service The Quality of Service (QoS)requirementsof a robotic systemwill vary according to the current task, the operating environmentand the systemloads. For example,timeliness constraints for a slow-moving robot mappinga static office environmentmight be quite loose, with few severe consequencesif they are missed. However, the constraints for a fast-movingautonomous vehicle, traveling overroughterrain are likely to be verytight, withpotentially disastrous results if deadlinesare missed.Weconsider someof the QoSimplications of middlewarein this paper. Furthermore,previous research on real-time performance(O’Ryanet al. 2001)and scheduling(Gill, Levine, Schmidt2001, Loyall et al. 2001; Gill, Cytron, &Schmidt 2002)demonstratesthe suitability of middleware for enforcing distributed QoSproperties. Scalable Programming Models System developers require programming modelsthat are scalable. Anexampleof such scalability lies in the complexityof the programming abstractions that are used. A programming modelthat covers manylow-level details maybecometedious and errorproneto programunless those details are appropriately encapsulatedand can be configuredat higher levels of abstraction. For example, a robot might be able to provide raw sensor readings, calibration settings and timing information fromits laser range-finder,and allowthe various settings to be manipulatedby the programmer.However,it should also be able to return a "distance to object" reading, using sensible defaults, for the user whois not interested in the exact settings of the device. This means(at least) twoviews the samedevice, as a generic distance sensor, and as an actual laser range-finder. A goodprogramming modelshould accommodateboth views and allow them to inter-operate seamlessly. It should also be able to scale across sensors. For example,the high-level user should be able to ask for the distances returnedby all sensors on the robot, regardless Iof type, and get reasonableresults. Scalability also deals withhoweasyit is, froma software point of view, to add another robot or embedded sensor to an existing system.Ideally, there shouldbe little cost in terms of programmingtime and integration work. These considerations motivate our interest in CORBA middleware.The CORBA programmingmodel allows programmersto extend existing interfaces polymorphically,replace instances of objects, and add newobjects to an existing system. Furthermore, CORBA supports implementations that address the performance and footprint issues of scalability, whichis the subject of the next section. What About Speed and Bloat? Oneof the most-cited reasons for not adoptingmiddlewarebased architectures on robotics platformsis that the additional layers of abstraction will slowthings downand cause codebloat. In this section, we addressthese two issues and provide the results of someempirical experimentsto support our claim that middleware offers a net benefit to developers of distributed robotic applications. Speed Results Toassess the relative time cost of using middleware for data transmission, comparedto simple socket data transfer, we conducted two experiments to benchmarkthe performance of each approach. These benchmarksare intended not as a definitive answerto the question of howmucha given system maypay for middleware’sflexibility, but rather to establish reasonable upper and lower boundson those expectations. Figure1 showsthe averageresults of ten trials of sending one million pairs of double precision floating point numbers using TAO(Center for Distributed Object Computing WashingtonUniversity), a high-performanceand real-time open-source CORBA-compliant ORBimplemented in C++. IWhether or not this is inherentlya badidea to beginwithis outsideof the scopeof this paper. J ~Nm __ ~ mwlm(.Nmm~,w #(~OAL4L) ~ m~mm.NNm ~mAl -~mi~mm-w,w~m (ommjk) Figure 1: CORBA Data Transmission BenchmarkResults Weran each set of trials locally on a server machine,locally on the robot, fromthe server to the robot, andfromthe robot to the server. The robot computeris an Intel PentiumIII 800Mzbased system, with 128MBof RAM,running a 2.2.19 linux kernel The server is a dual processor PentiumIII IGI-Iz system, with IGBof RAM, nmninga 2.4.8 linux kernel. Themachinesare on different subnets, and the communicationbottleneck betweenthem is a standard 802.11 wireless Ethernetlink, runningat 1 IMbits/second. In each trial we varied the granularity of the transmission by factors of 100, first sendinga single buffer, then 100equal subdivisionsof the data, then 10,000subdivisions, and finally 1,000,000. Wenote that in the first tworuns, of I and 100buffer subdivisions, little overheadwas added by the CORBA per-call setup itself. Thecost of the transmission includes parametermarshaling, so these represent a baseline for assessing a lower boundon total achievable throughput(several opfimizations such as removingendian marshalingwherepossible could improveon this result for specific cases). Wealso note that as the per-call overhead increases, the conuibutionof networkoverheadrelative to the total overheaddiminishes,resulting in a convergenceof the local and distributed transmissioncurves. Wecontrast these results with an implementationusing sockets, that helps us identify an upper boundon the expected throughputin each case. O’Ryan,et aL, have made careful studies of the contribution of each infrastructure componentin TAOto the overall middlewaretransmission overhead(O’Ryanet al. 2000). Togetherwith our socket results, these profiles helpcalibrate expectationfor the achievable performanceof middlewarefor distributed robotics. The timings for the socket experiments experiments are shownin Figure 2. It should be noted that no marshaling or UNMarshallingof the data is taking place in the socket implementation.This will cause the amountof computation being done to be inherently less than in the CORBA implementation,since the Intel architecture does not natively use networkbyte order (meaningthat all data are converted upontransmissionand reception). It does, however,give us a lower boundon the amountof time needed to ship data ¯ CORBA/NEST (Subramonian, Gill, & Sharp 2001; Subramonian&Gill 2002) - this approachaddresses reduction of the middleware infrastructure itself, froma perspective of composingonly the neededfeatures into a customizedand inter-operable version of the full-featured middleware.Webelieve this approachis the most suitable to systemssuch as distributed robotic and sensor systems in whichdifferent QoSproperties such as timeliness and footprint mustbe interchangedwith features required by each application. Byprovisioningexactly the policies and mechanisms needed, and moreimportantly supporting integration at the point of composition,we believe sophisticated systemsdesign trade-offs can be achievedwithout an undueburdenof complexityon the systemdevelopers. ~m~mmm~ Conclusions: Applicability of Middleware Although we strongly believe that CORBA-based middlewareoffers great advantagesfor robotics and embedded sensor applications,wedo not claimthat it is a goodsolutionfor between machines. all such systems.Theapplicability of a middleware solution dependsheavily on the hardwareof the systemon whichit must be deployed,and the applications that it will be runBloat ning. Oneargumentagainst middlewareis that it adds unnecesManyrobotics research platforms are based on common sary bloat to a system. Whilefull-featured middlewareimoff-the-shelf (COTS) hardware, with computationaland netplementations such as CORBA-compliant ORBsmay have workingabilities comparableto a desktop computerworksignificant memory footprints, subsets ofsuchmiddleware station, and run a "standard"operatingsystemthat controls canoffer manyofthebenefits without theexcessive overmostof their operation. Webelieve that it is this class of head. Current research is exploring various waysto ensure systemthat will mostbenefit fromthe application of middleonly the necessary features of middlewareare provisioned, ware. Argumentscan be madethat certain QoSneeds canand thus reducethe overall footprint on a case-by-casebasis. not be met by such a COTSsolution, and that specialized The key insight is that the footprint of middlewareis hardware, operating systems and communicationsinfrasonly appropriate or inappropriate with respect to a particutructure mustbe used. However,there is someevidencethat lar allocation of resources. Resourcestend to be aggregated a relatively cheap COTS systemcan meet relatively strininto hierarchies in distributed systems, with resource-rich gent real-time QoSdemands.Earlier experiments(Levine et backbonesconnecting islands of adequate but diminished aL 1999) on several general-purposeand real-time operatresources, which mayin turn connect resource-attenuated ing systemsindicated that undercertain conditions generalleaves of the overall sensor/robot network. Three mainarpurposeoperatingsystemscan performin someareas as well eas of research are addressingthese issues at the leaves of as, or in somecases better, than someRTOSs.For example, provisioninghierarchy: the results of these experimentsshowedthat thread context switching overheadhad lower latency and jitter in a COTS ¯ RMI/JINI(Sun Microsystems1999) - this approach alopen-sourceLinuxkernel than in all but one RTOS,as long lows a reduced-feature protocol bridge to inter-operate as the numberof threads per CPUwas small. with a full-featured Java middlewareinfrasu’ucture. Specifically, Jini allowsdeviceswith resourceconstraints Open Questions too stringent to support RMI,to interact with endsystems Rather than drawingconclusions from the preliminary exthat can support RMI.Thus, combiningRMIwith Jini periments presented in this paper, we will instead end by allows devices at very small resource scales to be used asking somequestions. Weconsider these questions to be within a middlewarecontext. importantones that should be addressedin any further re¯ CORBA/Smart Transducers (Object ManagementGroup search into the use of middleware on distributed robotic and 2001a) - this approach takes a similar approach to embeddedsensor systems. RMI/Jini, but removeslanguage specificity. In partic¯ Whatare the real costs of deployingCORBA-based soluular, CORBA supports a variety of language and plattions on standard robotic platforms, such as the RWItype formchoices. TheSmartTransducersspecification allows of robot? Furthermore,are there configurationsof system individual devices to be addressed as though they were features that increase or decreasethese costs? on endsystems supporting a complete CORBA ORB. ¯ Whatare the relevant round-trip latency and data throughThus, the CORBA/Smart Transducers approach offers adput transmissionprofiles for realistic data transfer patditional transparency to programmersof these kinds of systems. terns in common robotics use-cases? Figure 2: Socket Data TransmissionBenchmarkResults 4 Canuser studies identify howmiddlewarehelps or hinders ease of programming? Furthermore, can these studies measurethe perceived performanceimpact of middlewareon robot usability, fromthe user’s perspective? Whatare the abstractions that are useful to peoplebuilding and using robots? Whichabstractions hold over different communitiesof use (for exampleMLresearchers, path-planningresearchers, military, etc.)? Whichabstractions extend or specialize the moregeneral abstractions for particular domains? Can CORBA-based approaches be combinedwith simpler strategies (such as sockets), so that the strengths of one can makeup for the weaknessesof another? Howdoes the minimumtransmission (MTU)size a network affect the tradeoffs between sockets and CORBA? Betweensmall, frequent packets and larger, less frequent ones? References Bakken,D. 2001, Middleware.In Urban, J., and Dasgupta, P., eds., Encyclopediaof Distributed Computing.Kluwer. to appear. Center for Distributed Object Computing. Washington University. TAO: A High-performance, Real-time Object Request Broker (ORB). www.cs.wustl.edu/,,~schmidt/TAO.html. Gill, C. D.; Cytron, R.; and Schmidt,D. C. 2002. Middleware SchedulingOptimizationTechniquesfor Distributed Real-Timeand EmbeddedSystems. In Proceedings of the 7th Workshopon Object-oriented Real-time Dependable Systems. San Diego, CA: IEEE. Gill, C. D.; Levine, D. L.; and Schmidt,D. C. 2001. The Design and Performanceof a Real-TimeCORBA Scheduling Service. Real-TuneSystems, The International Journal of Time-CriticalComputing Systems, special issue on RealTimeMiddleware20(2). Levine, D. L.; Flores-Gaitan,S.; Gill, C. D., and Schmidt, D.C. 1999. Measuring OSSupport for Real-time CORBA ORBs.In Proceedings of the 4th Workshopon Objectoriented Real-time DependableSystems. Santa Barbara, CA: IEEE. Loyall, J.; Gossett,J.; Gill, C.; Schantz,R.; Zinky,J.; Pal, P.; Shapiro, R.; Rodrigues, C.; Atighetchi, M.; and Karr, D. 2001. Comparingand Contrasting Adaptive Middleware Support in Wide-Areaand EmbeddedDistributed Object Applications.In Proceedings of the 2 I st International Conference on Distributed ComputingSystems (ICDCS21), 625--634. IEEE. Object Management Group. 1998. CORBAMessaging Specification, OMG Documentorbos/98-05-05 edition. Object ManagementGroup. 2001a. Snu~rt Transducers Interface. Object ManagementGroup, OMGDocument orbos/2001-06-03edition. Object ManagementGroup. 200lb. The CommonObject Request Broker: Architecture and Specification, 2.5 edition. O’Ryan,C., and Schmidt, D. C. 1999. Applyinga Realtime CORBA Event Service to Large-scale Distributed Interactive Simulation. In 5th International Workshopon Object-oriented Real-Time Dependable Systems. Monterey, CA:TF.~E. O’Ryan,C.; Kuhns, F.; Schmidt, D. C.; and Parsons, J. 2000. ApplyingPatterns to Developa Pluggable Protocols Framework for ORBMiddleware.In Rising, L., ed., Design Patterns in Communications.CambridgeUniversity Press. O’Ryan,C.; Schmidt, D. C.; Kuhns,F.; Spivak, M.; Parsons, J.; Pyarali, I.; and Levine, D.L. 2001. Evaluating Policies and Mechanisms to Support Distributed RealTime Applications with CORBA. Concurrency and Computing: Practice and Experience13(2):507-541. Schmidt,D. C.; Stal, M.; Rohnert, H.; and Buschmann, F. 2000. Pattern-OrientedSoftwareArchitecture: Patterns for Concurrentand NetworkedObjects, Volume2. NewYork: Wiley & Sons. Subramonian, V., and Gill, C. 2002. OMG WorkshopOn Embedded&Real-Time Distributed Object Systems. In Experiences with Middlewarefor a NetworkedEmbedded Software TechnologyOpenExperimentalPlatform. Object ManagementGroup. Subramonian, V.; Gill, C.; and Sharp, D. 2001. OOPSLA2001 WorkshopTowardsPatterns and Pattern Languages for OODistributed Real-time and EmbeddedSystems. In Towardsa Pattern Languagefor NetworkedEmbedded Software Technology Middleware. ACM. Sun Microsystems. 1999. Jini Connection Technology. www.sun.com/jini/index.html.