Statement May, 2014 TUCKER BALCH, ASSOCIATE PROFESSOR SCHOOL OF INTERACTIVE COMPUTING, COLLEGE OF COMPUTING GEORGIA INSTITUTE OF TECHNOLOGY Research on robot teams Beginning with Tucker’s Ph.D. research at Georgia Tech with Ron Arkin much of his work has focused on behavior-­‐based robotics, especially for robot teams. Prof. Balch’s work has addressed some of the most important questions facing the field of cooperative robotics. We enumerate some of the most important work in terms of those questions below: Q: How much communication is enough? Tucker addressed this question with Ron Arkin in the paper “Communication in reactive multiagent robotic systems.” They showed that just a single bit of communication to indicate a primitive signal such as “I see food” conveys substantial value. More complex messages such as “I see food at location XY” provide only incrementally additional benefit. This paper was also one of the first to show how robots can communicate through the environment by changing it. This paper with Arkin appeared in the first issue of Autonomous Robots and has been cited 570 times. Q: How can robots move together effectively? Cooperative movement in specific formations is important in applications such as tactical aircraft activities and Army ground formations. Accordingly, it is also important for robot teams. The challenge is how to provide control that enables robots to maintain formation while also attending to other important considerations such as obstacle avoidance. Ron Arkin and Tucker developed a behavior-­‐based approach to this problem that was demonstrated on DARPA vehicles (figure at right). This work was among the first to address this problem. It was reported in IEEE Transactions on Robotics and Automation. It has been cited 2055 times. Q: How can lots of robots move together effectively? The behavior-­‐based approach to formation control Prof. Balch developed with Arkin works well for a team, namely 2 to 4 robots, but it was not clear how it might scale to large numbers of robots. The solution, developed with Maria Hybinette, was to borrow from chemistry: Treat formation control as something like crystal formation: Robots are attracted to specific “bonding sites” defined by the locations of their neighbors. This work was presented at ICRA and has been cited 293 times. Q: How should robots move cooperatively to gather information as a team? The methods described above for cooperative robot movement provided effective formation control, but to some extent they lacked a purpose. Why should robots be moving in formation? In work with his first Ph.D. student, Ashley Stroupe, Tucker began to investigate purposeful movement for teams in the context of information gain. If robots should Statement May, 2014 be moving to observe something, how could they do that most effectively? Ashley and Dr. developed an approach to this problem whereby each robot would consider where it’s partners would move to maximize information, then it would move in the best direction to add information for the team. The result is coordinated, purposeful motion of the team. An example of the trajectories a team of robots might follow using this approach is illustrated on the right. Tucker’s papers with Ashley on this topic have been cited more than 300 times in total. Q: How can pervasive networks assist robots in team tasks? In 2005 Daniel Walker, Keith O’Hara and Balch invented the “Gnat”: A small (about 1.5” diameter) communication and computation node that could be manufactured at scale. They manufactured 100s of Gnats, and investigated how they could be used to solve team tasks. The idea is that Gnats would be distributed into the environment, perhaps from the air, and that they would prepare an enhanced environment for mobile robot teams to interact. The Gnats are able to self organize into communication networks that reflect the topology of their environment. Balch’s group demonstrated how robot teams could solve navigational, coverage and foraging tasks more effectively. All together, this work has been cited about 100 times. From social robots to social animals More recently Dr. Balch has been aiming at a different core research question: Q: How can we describe the behavior of social animals? This may seem like a significant departure from his earlier work, but as we outline this research you will see that it is not. The central idea is to assume that the animals we observe are acting according to a robot program. Once we make that assumption we can apply a wealth of formal tools the task of inferring what that program is (e.g., HMMs, POMDPs, etc.). The assumption that animals act according to a behavior-­‐based program is not too radical if one considers that one of the leading approaches to robot programming, behavior-­‐based control, originates in models of behavior developed by biologists. Why the answer to this question important Biologists, psychologists and others seek models of behavior to explain their subjects. Most models created in this context are only partly functional or partly descriptive. For instance, a researcher might infer a differential equation that predicts an aspect of behavior such as the number of food items collected over time. On the other hand, the models Tucker is interested in are complete explanations of behavior, because they describe the full behavioral “loop” from sensing to acting. Other types of models are not complete in this way. Statement May, 2014 It is in the context of many, many agents that this work is distinguished from others. The models his systems learn are for individual agents, but the models account for behavior that arises through interaction. Thus these executable models (e.g., robot programs) are falsifiable hypotheses of system behavior, because they can be tested in simulation with thousands of agents. Executable models are also transferrable. Suppose, for instance, that we were able to build an algorithmic description (robot program) of individual animal behavior, that when applied across a many agents enables an entire insect colony to solve an important problem. We could adapt that result for robots, or human decision-­‐making tasks. This work has been carried out in 3 main stages: 1. Track the animals with sensors. 2. Build executable models of their behavior. 3. Assess accuracy of the models in simulation. Each of these steps is a research topic on its own. We outline them individually below. Multi-­‐target tracking Tracking animals in video is a hard problem. For Tucker’s work, the tracking problem is made more challenging because we must track many animals at once. Two key challenges arise here: 1) Identification (who is who) and 2) Maintaining separate, correct tracks for animals through interaction and occlusion events. Dr. Balch’s work in this area began when he was a postdoc at CMU. James Bruce, Manuela Veloso and Tucker co-­‐authored a paper on CMVision, a fast color-­‐based image segmentation algorithm. The paper contributed two novel ideas: First a reformulation of color classification as a table look up, and second, a very fast “blob” finder based on run length encoding. This work became the de facto standard library for color-­‐based segmentation in several communities including RoboCup. It is also part of the ROS release (a well known open source robot operating system). The image at right illustrates how an original image (left) is segmented into colored regions (right) by CMVision. This paper has been cited 646 times Color segmentation can only serve as a tracking algorithm if each target is individually marked. This obviously limits its application. So, after Tucker moved to Georgia Tech, he worked with Prof. Frank Dellaert and student Zia Khan to create true multi-­‐target trackers. They devised probabilistic descriptions of the problem, and then translated these into particle filter-­‐based trackers. The image at left shows one of their results in ant tracking. The work with Frank and Zia resulted in 6 papers at top robotics and computer vision conferences (ECCV, IROS), and two separate papers in the journal PAMI (highest impact factor of journals in computer science). All together these papers have been cited more than 1200 times. Statement May, 2014 These trackers focused on maintaining correct association through occlusion, but they did not solve the identification problem. Tucker’s Ph.D. student Adam Feldman worked on a solution to the combined ID and track association problem in his dissertation work. They created a system that forms “tracklets” or short track segments of individual agents with high confidence. They then combine that information with data from low spatial resolution RFID tags to combine the tracklets into full tracks with correct IDs. More recently, in work funded by the National Institutes of Health with Kim Wallen at Emory, Dr. Balch has developed a novel tracking system for captive animals based on active radio frequency ID tags (see figure below left). It is the first system we know of that can track dozens of animals in real time over several weeks in all weather conditions (above center). This work is recent, but already the team has been able to use the tracking data to infer social relationships between individual monkeys and represent that data as social tie strength graphs (above right). The monkey tracking work is in collaboration with Kim Wallen at Emory, Dan Walker, Aaron Bobick, Jim Rehg, and Irfan Essa at GT, and students Brian Hrolenok, Hanuma Malladi, Rahul Sawhney, Pipei Huang, and Michael Novitzky. Next steps in tracking: Prof. Balch is collaborating with Jim Rehg to develop his algorithms for tracking animals in outdoor environments into a scalable multi-­‐target tracker. This work is in support of an NSF award focused on understanding social animal behavior. Executable models of behavior The objective for Tucker’s work in this area is to create descriptions of behavior that can be executed in simulation or on a robot. The best example of this work is a project with student Richard Roberts (right). They used instance-­‐based learning (KNN) to train a robot to solve navigation problems such as obstacle avoidance, path following, and even running a slalom course (pictured). The training takes place as a human drives the robot through example courses. The robot “observes” how the human responds to the navigational challenges. The robot in essence learns a model of the human’s driving behavior that can be executed later. This same approach can be used to enable robots to learn the behavior of social animals. Tucker and his team have explored this by learning models of simulated agents, then validating the system behavior in simulation. The image at left illustrates the performance of simulated agents performing a foraging task that was learned from observation. Statement May, 2014 Q: Can we learn switching hybrid controllers from observation? In recent work with his student Brian Hrolenok, Tucker has been able to automatically learn executable models of schooling fish behavior and foraging ant behavior. An important contribution of this work is that the models include controllers for individual control modes, and the rules for switching between controllers. The models are acquired using tracks of animal movement (with tracking methods described above), and learned using a variation of the Baum Welch algorithm. Broader Impact: Personal robots for CS education In addition to his research in robotics and social animal systems, Tucker has led several initiatives in education. Starting in 2008, Tucker led a team of Georgia Tech researchers in collaboration with educators at Bryn Mawr College in the creation of a curriculum, software, textbook and robot hardware to teach introductory computer science. The work was initially funded by Microsoft and then the NSF. Collaborators included Prof. Mark Guzdial, Dr. Jay Summet and Daniel Walker at Georgia Tech and Prof. Doug Blank and Prof. Deepak Kumar at Bryn Mawr. Dr. Balch’s primary contribution to this effort centered on the development of the robot and software for controlling it, as well as developing a program for teaching the course at Georgia Tech. The robot and curricula were initially tested in Georgia Tech’s CS 1301: Intro to Computing, and are now a permanent part of that course (above, right) CS 1301, using these robots, is taught to more than 350 students each semester at Georgia Tech. Overall, about 2,000 Georgia Tech students have been introduced to Computer Science with these robots. 5,000 robots have been distributed to students at 30 institutions. Broader Impact: Massive Online Open Course (MOOC) on Machine Learning in Finance Tucker has recently initiated teaching and research in the application of Machine Learning to Finance. As part of that work, he teaches a course, CS 7646: Machine Learning for Trading on campus at Georgia Tech. In 2012, Tucker converted the first portion of that course into a MOOC that was offered via Coursera. This course is arguably the most successful of Georgia Tech’s MOOCs, with more than 100,000 students enrolling, and 5,000 completing the course. Prof. Balch has taken advantage of his experience in this new domain to gather, analyze, and report on data and techniques to help future MOOC instructors and teachers. He have published this information as articles on a public blog followed by thousands of viewers (http://augmentedtrader.wordpress.com).