Update Presentation 1
Weeks 1-4
Optic Flow QuadCopter Control
Oscar Merry
Contents
•
•
•
•
•
•
•
•
•
Introduction
OpenCV
Honeybee Vision
Research Paper 1 – AR Drone LabVIEW Toolkit
Research Paper 2 – Optical Flow Quadrotor controller
Other Research Papers
General Problems
Aims of Project
Discussion
Introduction
• Optical Flow:
– The estimation of the motion field created by a moving
camera with respect to a rigid scene.
• Last 4 weeks:
–
–
–
–
Literature Analysis
Research into Optic Flow
Research into honeybee navigation
Research into OpenCV
OpenCV
• Open-source C / C++ library for advanced computer
vision.
• Built in functions for many optical flow processing techniques:
– Canny edge detector
– Feature tracking
– Lucas–Kanade
– Horn-Schunck
– Shi-Tomasi
Honeybee Vision
• Each compound eye in the honeybee contains ~ 5000 ommatidia
• Each ommatidium has 9 photoreceptor cells grouped into 3 classes
– Ultraviolet sensitive
– Blue sensitive
– Green sensitive
• Bees control their flight speed by keeping optical flow constant. (The same
process is used for landing) This is done via the Green photoreceptors.
• Bees recognise objects both through strong contrasts in luminance or
colour and through optical flow.
Research Paper 1 - AR Drone LabVIEW
Toolkit
• Michael Mogenson Masters Thesis:
– “A software framework for the control of low-cost
quadrotor aerial robots.”
• Created a LabVIEW toolkit for the control of Parrot
AR Drone.
AR Drone LabVIEW Toolkit
• Toolkit has multiple virtual instruments. (VIs)
– Regular Comms VIs:
• Main VI for control commands and comms management
• Video VI for video reading and decoding
• Nav Data VI for reading navigation data
– ‘Thinking’ Vis:
• State VI to estimate position in X,Y,Z Cartesian coordinates
• Various image processing Vis (Fast blob detection, dense optical
flow, image space conversion, ROI VI)
• Toolkit has wrapper for OpenCV Library.
AR Drone LabVIEW Toolkit – State VI
• Populates a rotation matrix between the drones coordinate
system and the ground using Euler orientation angles from
navigation data.
• Applies rotation matrix to the velocities measured from
optical flow from the bottom camera.
• Rotated velocities integrated into a position with the
timestamp data.
• Suffers from problem of drift.
AR Drone LabVIEW Toolkit –
Achievements
• Benefit of LabVIEW – modifications without recompiling
• Demonstrations:
– Face tracking
– Indoor hallway navigation (Via vanishing point)
– Fly through hoop
• Problems:
• Indoor hallway navigation fails if 90 degree turn or if facing wall
• State Estimation VI suffers from drift
Research Paper 1 - Optical Flow
Quadrotor controller
• “Optical Flow-Based Controller for Reactive and Relative
Navigation dedicated to a Four Rotor Rotorcraft”
• Eduardo Rondon, Isabelle Fantoni-Coichot, Anand Sanchez,
Guillaume Sanahuja
• Produced a controller for a Quadrotor based on the optical
flow from 2 cameras. (1 for velocity regulation, 1 for obstacle
avoidance)
• Implemented obstacle avoidance for indoor navigation.
Optical Flow Quadrotor controller
Optical Flow Quadrotor controller Achievements
• Velocity regulation via optical flow controller.
• Sets the inverse time-to-contact to a threshold to
stop the vehicle if obstacle detected.
• Has lateral and altitude avoidance.
Other Research Papers
• “Optic flow based slope estimation for autonomous landing” de
Croon et al.
– Achieved slope following and autonomous landing.
• “Combined Optic-Flow and Stereo-Based Navigation of Urban
Canyons for a UAV” Hrabar et al.
– Used optical flow to balance UAV in canyon.
– Used stero vision to navigate T and L junctions.
• “An adaptive vision-based autopilot for mini flying machines
guidance, navigation and control.”
– used optic flow and IMU data for guidance navigation and control,
specifically automatic hovering, landing, and target tracking.
– Use feature tracking to reduce optic flow computation.
General Problems
• Camera Calibration – In order for the depth of a scene to be
known the camera must be calibrated on a known object. (Or
height above ground must be known)
• Video Delay – Video encoding, decoding, and transmission
must be fast enough.
Project Aims
• Hovering Stabilisation
• Position and velocity control
• Smooth Landing Execution
• Obstacle recognition and avoidance (with a focus on
methods that have similarities to honeybees e.g.
flower recognition)
Discussion
• Communication:
– Hardware
– State commands (roll angle, pitch angle, yaw rate,
climb rate, ??)
– Video frames
• Formalize state commands.