Persistent UAV Service: An Improved Scheduling Formulation and Prototypes of System Components Byung Duk Song, Jonghoe Kim, Jeongwoon Kim, Hyorin Park, James R. Morrison* and David Hyunchul Shim Department of Industrial and Systems Engineering Department of Aerospace Engineering KAIST, South Korea Friday, May 31, 2013 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 Presentation Overview • Motivation • UAV service system concept • Comparison with existing research • UAV service system: Components and prototype – Central planning: Deterministic customer paths – UAV guidance system – Automatic replenishment station • System demonstration • Concluding remarks ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 2 Presentation Overview • Motivation • UAV service system concept • Comparison with existing research • UAV service system: Components and prototype – Central planning: Deterministic customer paths – UAV guidance system – Automatic replenishment station • System demonstration • Concluding remarks ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 3 Motivation • Large expensive UAVs – Usually military purpose – Operate for many hours – Travel long distances • Small inexpensive UAVs – A lot of application area such as tracking, communication relay, environmental / fire / national boundary monitoring, cartography, disaster relief and so on. – Limited duration of mission – Limited distance • Methods to ensure persistent operation can increase effectiveness of small UAVs – Collection of UAVs, refueling stations, automatic guidance – Algorithms to orchestrate the system operations ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 4 Presentation Overview • Motivation • UAV service system concept • Comparison with existing research • UAV service system: Components and prototype – Central planning: Deterministic customer paths – UAV guidance system – Automatic replenishment station • System demonstration • Concluding remarks ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 5 UAV Service System Concept Random arrival of customer information Heterogeneous UAVs Random path and duration UAV service system Persistent UAV service Vision technology Service station 2 UAV 1 UAV operation system Moving objective trajectory Object 2 UAV 3 Service station 3 Object 3 Object 1 UAV 4 UAV 2 Central planning Automatic replenishment station Service station 1 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 6 UAV 5 UAV Service System Concept Random path and duration Heterogeneous UAVs UAV service system Persistent UAV service Vision technology Service station 2 UAV 1 UAV operation system Moving objective trajectory Object 2 UAV 3 Service station 3 Object 3 Object 1 UAV 4 UAV 2 Central planning Automatic replenishment station Service station 1 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 7 UAV 5 UAV Service System Concept Heterogeneous UAVs UAV service system Persistent UAV service Vision technology Service station 2 UAV 1 UAV operation system Moving objective trajectory Object 2 UAV 3 Service station 3 Object 3 Object 1 UAV 4 UAV 2 Central planning Automatic replenishment station Service station 1 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 8 UAV 5 Presentation Overview • Motivation • UAV service system concept • Comparison with existing research • UAV service system: Components and prototype – Central planning: Deterministic customer paths – UAV guidance system – Automatic replenishment station • System demonstration • Concluding remarks ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 9 Comparison with Existing Research <Decentralized task assignment algorithm> <Automatic landing & recharge> 1. Persistent path following with multiple shared service stations distributed across the field of operations 2. Prototype components for a system seeking to provide a persistent UAV security escort service <Automated 1.5 Hour persistent surveillance mission with three autonomous vehicles> [1] M. Alighanbari and J. P. How, “Decentralized task assignment for unmanned aerial vehicle”, Proceedings of the 44 th IEEE Conference on Decision and Control, and the European Control Conference, December 2005 [2] M. Valenti, D. Dale, J. P. How and D. P. de Farias, “Mission health management for 24/7 persistent surveillance operations”, AIAA Guidance, Navigation10 and Control Conference and Exhibit, August 2007 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – Presentation Overview • Motivation • UAV service system concept • Comparison with existing research • UAV service system: Components and prototype – Central planning: Deterministic customer paths – UAV guidance system – Automatic replenishment station • System demonstration • Concluding remarks ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 11 UAV Service System: Components and Prototype Customer information UAV schedule Central planning by MILP Web or smart phone UAV guidance system Automatic control feedback Tracking Customer UAV Automatic replenishment Replenishment station ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 12 Central Planning: Deterministic Customer Paths Customer information UAV schedule Central planning by MILP Web or smart phone UAV guidance system Automatic control feedback Tracking Customer UAV Automatic replenishment Replenishment station ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 13 Persistent UAV Service ■ Persistent UAV service system with heterogeneous UAVs and multiple service stations - A system of UAVs that is supported by automated replacement systems can support long term or even indefinite duration missions in a near autonomous mode with multiple service stations - The UAVs can return to any service station, replenish their resources and resume their duties Service station 2 UAV 1 Moving objective trajectory Object 2 UAV 3 Service station 3 Object 3 Object 1 UAV 4 UAV 2 Service station 1 UAV 5 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 14 Customer Paths ■ To follow a time-space trajectory, the trajectory is divided into pieces (split jobs) Service station 2 UAV 2 Start 5 1 UAV 2 8 9 6 4 2 7 10 UAV 1 End UAV 1 3 Split job Start point End point Start time End time 1 50,250 150,250 13:10 13:11 2 150,250 250,250 13:11 13:12 3 250,250 350,250 13:12 13:13 4 350,250 450,250 13:13 13:14 5 450,250 550,250 13:14 13:15 6 550,250 650,250 13:15 13:16 - From point (50,250) to (950,350) 7 650,250 750,250 13:16 13:17 - From 13:10 to 13:20 8 750,250 850,250 13:17 13:18 9 850,250 950,250 13:18 13:19 10 950,250 950,350 13:19 13:20 Service station 1 dij ( xei xsj )2 ( yei ysj )2 d ji ( xej xsi )2 ( yej ysi )2 ▪ Objective moves ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 15 UAV 1 UAV 2 Assumptions ■ Assumptions 1. Moving target’s path and location at specific times are known. 2. UAVs start its travel from a recharge station 3. Recharge time for a UAV is constant 4. Initially all UAV batteries or fuel tanks are empty 5. UAV travel speed is constant ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 16 Initial Mathematical Formulation ■ Notation i, j : Indices for jobs s : Index for stations k : Index for UAVs r : Index of a UAV’s rth flight NJ : Number of split jobs NUAV : Number of UAVs in the system NSTA : Number of recharge stations NR : Maximum number of flight per UAV during the time horizon M : Large positive number (xjs, yjs) : Start point of split job j (xje, yje) : End point of split job j Dij : Distance from split job ith finish point to split job jth start point, Dij ≠ Dji Ei : Start time of split job i Pi : Processing time or split job i qk : Maximum traveling time of UAV k Sok : Initial location(station) of UAV k TSk : Travel speed of UAV k ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 17 Initial Mathematical Formulation ■ Notation ΩJ : = {1, …, NJ}, Set of split jobs ΩJD : = {1, …, NJ+1}, Set of split jobs and dummy jobs ΩSS : = {NJ+2, NJ+4, …, NJ+2∙ NSTA}, set of UAV flight start station ΩSE : = {NJ+3, NJ+5, …, NJ +2∙ NSTA+1}, set of UAV flight end station ΩA : = (ΩJD U ΩSS U ΩSE) = {1,…, NJ+2∙NSTA+1}, set of all jobs and recharge stations ■ Decision Variables ▪ Xijkr = 1 if UAV k processes split job j or recharges at station j after processing split job i or recharging at station i during the rth flight; 0, otherwise ▪ Yikr = 1 if UAV k processes split job i during its rth flight; 0, otherwise. ▪ Cikr is job i’s start time by UAV k during its rth flight or UAV k’s recharge start time at station i; otherwise its value is 0. ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 18 Initial Mathematical Formulation ■ Mathematical formulation Minimize D kK rR i A j A ij X ijkr Subject to Initial recharge station constraints j JD X sok , jk 1 1 (k K ) (i JD SS , j JD SE , k K , r R) X sjkr 1 (k K , r R) sSS jJD Recharge station constraints Cikr Pi Dij / TSk C jkr M (1 X ijkr ) X sSE i JD X i JD iskr iskr 1 (k K , r R) X i JD s 1,ikr 1 (k K , r 1... N R 1, s SE ) X kK rR i A X j A ijkr ijkr 1 ( j J ) X j A X ijkr Yikr (i J , k K , r R) jikr 0 (i JD , k K , r R) C kK rR ikr i A j A ij Ei (i J ) / TSk X ijkr PX i JD j A i ijkr qk (k K , r R) X sdkr X d ,s 1,kr (k K , r R, s SS ) X dikr X idkr 0 (k K , r R, i J ) X iskr 0 (k K , r R, s SS ) Fuel constraints Dummy job constraints Cikr 0 (k K , r R, i A ) X ijkr {0,1} (k K , r R, i A , j A ) iJD Start time constraints M Yikr Cikr (i J , k K , r R) D Cskr Cs 1,kr 1 (k K , r 1... N R 1, s SE ) Split job assignment constraints j JD SE Yikr {0,1} (k K , r R, i A ) ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 19 Decision variables Reduce Variables and Constraints ■ Mathematical formulation Minimize D kK rR i A j A ij X ijkr Subject to Initial recharge station constraints j JD X sSE i JD X i JD (i JD SS , j JD SE , k K , r R) X sjkr 1 (k K , r R) sSS jJD Recharge station constraints Cikr Pi Dij / TSk C jkr M (1 X ijkr ) X sok , jk 1 1 (k K ) iskr iskr 1 (k K , r R) X i JD s 1,ikr 1 (k K , r 1... N R 1, s SE ) kK rR i A Split job assignment constraints X j A ijkr ijkr 1 ( j J ) X j A X ijkr Yikr (i J , k K , r R) jikr 0 (i JD , k K , r R) C kK rR ikr i A j A ij Ei (i J ) / TSk X ijkr PX i JD j A i ijkr qk (k K , r R) X sdkr X d ,s 1,kr (k K , r R, s SS ) X dikr X idkr 0 (k K , r R, i J ) X iskr 0 (k K , r R, s SS ) Fuel constraints Dummy job constraints Cikr 0 (k K , r R, i A ) X ijkr {0,1} (k K , r R, i A , j A ) iJD Start time constraints M Yikr Cikr (i J , k K , r R) D Cskr Cs 1,kr 1 (k K , r 1... N R 1, s SE ) X j JD SE Yikr {0,1} (k K , r R, i A ) ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 20 Decision variables Improved Formulation ■ Mathematical formulation Minimize D kK rR i A j A ij X ijkr Subject to Initial recharge station constraints j J SE Cikr Pi Dij / TSk C jkr M (1 X ijkr ) X sok , jk1 1 (k K ) (i J SS , j J SE , k K , r R) sSS jJ SE X sjkr 1 (k K , r R) M j J SE Recharge station constraints X iskr 1 (k K , r R) sSE iJ SS C kK rR iJ SS X iskr iJ SS X s1,ikr 1 (k K , r 1... N R 1, s SE ) Cskr Cs 1,kr 1 (k K , r 1... N R 1, s SE ) X kK rR i A Split job assignment constraints j A X ijkr i J SS ijkr j A ij / TSk X ijkr PX iJ j A i ijkr qk (k K , r R) Cikr 0 (k K , r R, i A ) 1 ( j J ) X jikr 0 (i J , k K , r R) Start time constraints Ei (i J ) ikr D i A j A X ijkr Cikr (i J SS , k K , r R) X ijkr {0,1} (k K , r R, i A , j A ) X iskr 0 (k K , r R, s SS ) ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 21 Fuel constraints Decision variables Improved Formulation ■ Complexity : Number of decision variables and constraints Kim et al. (2012) Improved formulation Difference Total # of binary decision variable NUAV∙NR∙{(NJ+2∙NSTA+1)2 +(NJ+2∙NSTA+1)} NUAV∙NR∙(NJ+2∙NSTA)2 (NJ+2∙NSTA+1)2 +(NJ+2∙NSTA+1) -(NJ+2∙NSTA)2 Total # of continu ous decision variable NUAV∙NR∙ (NJ+2∙NSTA+1) NUAV∙NR∙(NJ+2∙NSTA) NUAV∙NR Total # of decision variable NUAV∙NR∙{(NJ+2∙NSTA+1)2 +2∙(NJ+2∙NSTA+1)} NUAV∙NR∙{(NJ+2∙NSTA)2 +NJ+2∙NSTA} (NJ+2∙NSTA+1)2 +NJ+2∙NSTA+2 -(NJ+2∙NSTA)2 Total # of constraints NUAV{2(NR-1)∙NSTA+1} + NJ(3NUAV∙NR+2) 2 + NUAV∙NR{( NJ+NSTA+1) +2NJ+4NSTA+5} NUAV{2(NR-1)∙NSTA+ NR∙ NSTA +1} + NJ(NUAV∙NR+2) 2 + NUAV∙NR{( NJ+NSTA) +2NJ+3NSTA+3} NUAV∙NR{( NJ+NSTA+1)2 -( NJ+NSTA)2+NSTA+2} +2∙NJ∙ (NUAV∙NR) -NUAV∙ (NR∙ NSTA ) ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 22 Computational Results ■ Comparison of computational result NJ NSTA NUAV # of D.V # of const CPU Time Obj. Value # of D.V # of const CPU Time Obj. Value CPU Time Redu ction 8 2 2 780 722 3.00 2048 624 566 1.84 2048 1.6x 14 3 6 5796 5002 15.84 1846 5040 4222 4.36 1846 3.6x 15 3 6 6336 5508 220.3 724 5544 4680 35.78 724 6.2x 20 4 8 14384 12048 N/A N/A 12992 10592 348.97 2894 - Kim et al. (2012) Improved formulation ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 23 UAV Guidance System Customer information UAV schedule Central planning by MILP Web or smart phone UAV guidance system Automatic control feedback Tracking Customer UAV Automatic replenishment Replenishment station ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 24 UAV Guidance System ■ Roles of UAV guidance system 1. Receive and implement the schedule from the MILP. 2. Convert the video from the UAV cameras into usable information for directing the motion of the UAVs 3. Enable a human overseer to monitor the UAV progress via video and adjust feedback control gain values for various situations 4. Allows for a human overseer to initiate emergency actions such as immediate landing. ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 25 UAV Guidance System ■ System components < AR drone 2.0 > 1280 720 pixel front camera 320 240 pixel belly camera < WIFI > < Ipad 3> ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 26 UAV Guidance System 1. The color video from the camera is acquired via TCP port and processed using OpenCV framework. Descriptions ① : Front(Bottom) Camera Video ② : Start Procedure Interface ③ : Color Filtered Video ④ : Control Gain Adjustment Sliders ⑤ : Emergency Landing Button 2. The image is separated into three RGB channels. These three images are used to determine the color of the targeted image. 3. Control inputs including the longitudinal-lateral tilt angles, height and yaw angular velocity are calculated from the number and mean coordinate of target pixels in the processed image. ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 27 UAV Guidance System ■ P-D gain controller block diagram ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 28 Automatic Replenishment Station Customer information UAV schedule Central planning by MILP Web or smart phone UAV guidance system Automatic control feedback Tracking Customer UAV Automatic replenishment Replenishment station ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 29 Automatic Replenishment Station ▪ Each AR Drone 2.0 uses a three cell lithium polymer battery ▪ four copper leads (three for each terminal and one for the ground terminal) were threaded from the battery inside the UAV to the four feet of the drone ▪ The service station consists of four pads, one for each foot of the drone. ▪ Each such pad connects to the UAV battery via the leads on the drone feet ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 30 Presentation Overview • Motivation • UAV service system concept • Comparison with existing research • UAV service system: Components and prototype – Central planning: Deterministic customer paths – UAV guidance system – Automatic replenishment station • System demonstration • Concluding remarks ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 31 System Demonstration: Layout ■ Demonstration description < Demonstration layout > Split job 8 Station 3 < Schedule by MILP > Split job 7 UAV 2 ∙∙∙ Split job 2 Split job 1 UAV Start station Assigned job End station Service start time Service end time 1 1 1,2,3,4 2 2 10 2 2 5,6,7,8 3 10 18 Hand-off 5m UAV 1 Station 2 Station 1 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 32 System Demonstration: Video ■ Demonstration video ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 33 Presentation Overview • Motivation • UAV service system concept • Comparison with existing research • UAV service system: Components and prototype – Central planning: Deterministic customer paths – UAV guidance system – Automatic replenishment station • System demonstration • Concluding remarks ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 34 Concluding Remarks • Towards a persistent UAV service • Components of such a system – System orchestration MILP (deterministic customer paths) • Improved formulation • Reduced computational time – UAV guidance system • Vision for UAV localization relative to customer, location flags and platforms • Feedback control for UAV via iPad controller – Automatic replenishment stations (battery recharge) • Demonstration of proposed UAV service system • Future directions – – – – Real time customer requests Random customer behavior during service Implementation outdoors Improved UAV localization algorithms ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 35 Back up materials ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 Improvement : Efficient formulation • To enhance the cplex computational power, efficient mathematical formulation was developed • Delete unnecessary decision variable and dummy job concepts - Delete Yjkr decision variable because Cjkr decision variable can replace it. ▪ Yikr = 1 if UAV k processes split job i during its rth flight; 0, otherwise. ▪ Cikr is job i’s start time by UAV k during its rth flight or UAV k’s recharge start time at station i; otherwise its value is 0. Cjkr Yjkr =0 =0 >0 =1 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 37 Improvement : Efficient formulation • Delete the concept of dummy job which is used for idle UAVs by allowing direct flight from start(end)station to end(start) station sSS jJD X sjkr 1 (k K , r R) X sSE i JD iskr 1 (k K , r R) Dummy job d ij 0, i ss , j dummy job stations sSS jJ SE X sjkr 1 (k K , r R) sSE iJ SS X iskr 1 (k K , r R) ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 38 Literature Review • Scheduling methods without a distance or time restriction – – • Scheduling methods for limited flight duration – – – – • A. L. Weinstein and C. Schumacher, “UAV scheduling via the vehicle routing problem with time windows,” In Proc. AIAA Infotech@Aerospace 2007 Conference and Exhibit, Rohnert Park, California, 2007 T. Shima, S. Rasmussen and D. Gross, “Assigning micro UAVs to task tours in an urban terrain,” IEEE Transactions on Control Systems Technology, Vol. 15, No. 4, 2007, pp. 601 – 612 Y.S. Kim, D.W. Gu and I. Postlethwaite, “Real-time optimal mission scheduling and flight path selection, IEEE Transactions on Automatic Control, Vol. 52, No. 6, 2007, pp. 1119-1123. B. Alidaee, H. Wang, and F. Landram, “A note on integer programming formulations of the real-time optimal scheduling and flight selection of UAVS,” IEEE Transactions of Control Systems Technology, Vol. 17, No. 4, 2009, pp.839-843 Scheduling method for persistent UAV operation – • T. Shima and C. Schumacher, “Assignment of cooperating UAVs to simultaneous tasks using genetic algorithm,” In Proc. AIAA Guidance, Navigation, and Control Conference and Exhibit, San Francisco, 2005 J. Zeng, X. Yang L. Yang and G. Shen, “Modeling for UAV resource scheduling under mission synchronization,” Journal of Systems Engineering and Electronics, Vol. 21, No. 5, 2010, pp. 821-826 M. Alighanbari and J. P. How, “Decentralized task assignment for unmanned aerial vehicle”, Proceedings of the 44th IEEE Conference on Decision and Control, and the European Control Conference 2005 seville, spain, december 12-15, 2005 Battery recharge/exchange methods – – – – – J. How, thesis papers at MIT, 2005, 2007 A.S. Kurt, B.H. Clarence, R.R. Johnhenri, D.W. Richardson, Z.H. White, Q. Elizabeth and G. Anouck, “Autonomous Battery Swapping System for Small-scale Helicopters”, 2010 IEEE International Conference on Robotics and Automation R. Godzdanker, M. J. Rutherford and K. P. Valavanis, “ISLANDS: A self-leveling platform for autonomous miniature UAVs”, 2011 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pp 170-175 A.O.S. Koji, K.F. Paulo and James R. Morrison, “Automatic battery replacement system for UAVs: Analysis and design” Journal of Intelligent and Robotic Systems, Special Issue on Unmanned Aerial Vehicles (Springer), a Special Volume on Selected Papers from ICUAS’11, Vol. 65, No. 1, pp. 563-586, January 2012. First published online September 9, 2011 M. Valenti, D. Dale, J. P. How and D. P. de Farias, “Mission health management for 24/7 persistent surveillance operations”, AIAA Guidance, Navigation and Control Conference and Exhibit, 20-23 August 2007, Hilton Head, South Carolina ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 39