Service Advisory Operation of 2014 Vireo, 2015 Phoenix A1, and Indago without using Scout Planner Revision: June 5, 2015 1 Overview Flight plans for Sentera’s UAV platform’s that use Virtual Cockpit ground control station software can be generated and executed completely within Virtual Cockpit without the need to use Farm Intelligence’s Scout Planner software. This service advisory document outlines the recommended steps to use Virtual Cockpit to plan a flight and capture imagery data for agricultural and aerial mapping flights. This applies to the following UAVs supplied and supported by Sentera: 2014 Vireo 2015 Phoenix A1 2014 and 2015 Indago Ag Flight operations of the UAVs, including pre-flight procedures, normal flight procedures, and emergency procedures are not addressed in this document. These instructions can be found in the operating manual provided with the systems and information provided during operator training. This document is a supplement to those original instructions. Topics covered in this advisory are: Loading background maps into Virtual Cockpit using Google Earth Creating a flight plan at the field prior to flight Adjusting area waypoint parameters to achieve data collection goals Access and download the images and geolocation data off the cameras after a flight Optional: How to create a Pix4D Mapper project and load flight imagery and data Optional: How to create an Agisoft PhotoScan project and load flight imagery and data Page 1 of 15 2 Loading background maps into Virtual Cockpit using Google Earth Sentera recommends to always use a background map when flying UAVs using Virtual Cockpit. The easiest way to import images is to use map maker application inside Virtual Cockpit to capture maps from Google Earth. 1. Connect the laptop to the Internet. 2. Launch Google Earth. If Google Earth is not installed, download and install Google Earth, then launch it. 3. Navigate in Google Earth until the map area of interest is displayed on the screen. Press the ‘R’ key on the keyboard to reset the camera view to aim the camera straight down and define the top of the screen as North. 4. Launch Virtual Cockpit. 5. Launch the Map Maker Application in Virtual Cockpit: a. Settings Map Settings (alternately press F7 key and Map Settings) b. Press the [Launch Map Maker Application] button near the bottom of the screen 6. The Map Maker application window will open: a. Press FileGoogle Earth Import in the upper left of the screen (at this point the Map Maker application will grab the map image from Google Earth and paste a preview in the Map Maker screen). b. Press the [Make Map] button in the lower right of the screen. Page 2 of 15 c. Browse to the folder where you wish to save the map image and associated .wf world file. Typically the folder should be c:\Program Files (x86)\Procerus Technologies\Virtual Cockpit 2.5H\Maps\... d. Two files are saved: an image file and an associated .wf file that contains information about the geo-location of the image. 7. Close the Map Maker application The remaining map loading steps should already be familiar from your Vireo, Phoenix A1, or Indago Ag operating manuals. They are summarized here for review: 8. In the Map Settings screen (F7), press the [Add] button. 9. Select Color Image from the Map Type drop-down menu 10. Press the [Load] button and browse to the folder location where you saved the map image from above. Select the image you want to load and press [Open]. 11. Press the [Ok] button in the bottom left of the Map Setup window. 12. The new map should now be shown in the list of maps in the F7 Map Settings screen. a. Ensure that the check box next to the map(s) you wish to use is checked. b. Optional: you may uncheck other maps that you do not wish to use. c. Optional: Use the arrow keys to change the order in which the maps are listed and displayed. d. Optional: Use the [Center] button to center the map view on a particular map in the list. e. Optional: Double-click on a map in the list and check the “Default Startup” checkbox to save it as the default starting location. Page 3 of 15 3 Create a flight plan at the field with Virtual Cockpit Creating a flight plan in Virtual Cockpit is very easy. 1. Right-click a spot on the map where you want the UAV to land. 2. From the right-click menu select “Move Takeoff/Land Here” Left: “Move Takeoff/Land Here” for Vireo/Phoenix; Right: “Move Takeoff/Land Here for Indago” 3. Click and drag the T (Takeoff), R (Rally), and L (Land) waypoints to the desired locations based on wind direction and landing location. Tip: Vireo/Phoenix A1 recommended landing approach distance is 1000 ft. Page 4 of 15 4. OPTIONAL: if you want the UAV to automatically fly to the rest of the flight plan waypoints after climb-out, double click on the Takeoff waypoint and check the box “Auto Trans to NAV Mode”. If you do not check this box then the UAV will remain in the Takeoff waypoint until the operator presses the “Nav” button in Virtual Cockpit. 5. Now add additional waypoints to your flight plan. Right-click a spot on the map (e.g. the center of a field), select “Insert Command” and then select the waypoint type. Typically, “Area Waypoint” will be used for mapping flights. 6. The default settings for the Area Waypoint will be populated and should be adjusted based on data collection goals. See the next section of this document for further details. Page 5 of 15 4 Adjust area waypoint parameters to achieve data collection goals The default settings for Area Waypoints must be adjusted to your specific data collection goals. Here are some steps and recommendations to do that. 1. Double click on the center point in the Area Waypoint. 2. Click and drag the corners of the Area Waypoint to resize, or enter values into the waypoint edit window. 3. Area waypoints may be rotated using the small yellow circle at the top side of the area waypoint or by typing in a “Box Angle” in the waypoint edit window. 4. Adjust the altitude using the Alt HAL entry. The area waypoint automatically re-calculates how many back and forth passes are needed at different altitudes. Tip: The default altitude is currently 164 ft. (50 m) height above launch. Sentera does not recommend ever flying a fixed wing aircraft such as Vireo/Phoenix A1 any lower than this. Indago may be flown lower if desired. Flying at 400 ft. or 500 ft. HAL is best for covering large areas. WARNING: Vireo/Phoenix A1 and Indago control their altitude to “height above launch” which means it does not adjust altitude to flow terrain such as hills and valleys. It is up to the operator to ensure they do not fly into the side of a hill or tall obstacles such as trees. 5. Adjust the overlap in the “Min Overlap 5%” entry. The system automatically re-calculates the number of back and forth passes needed at different overlaps. This overlap value is the same for both long-lap and side-lap. Page 6 of 15 Tip: The default overlap is 30% which is a good value to ensure a minimal level of overlap for all adjacent pictures. Many image mosaic stitching programs require higher overlap to work reliably. Sometimes up to 80% overlap is required to successfully stitch data sets captured in certain conditions. Other times 30% overlap works. 60% overlap is a good choice if you are uncertain about the overlap value needed. 6. The Standoff Time (sec) can be adjusted for Vireo and Phoenix. The default is 15 seconds which is much longer than necessary. Change this value to “7” for best turning results under most conditions. “9” or “11” might work better in windy conditions. Tip: set Standoff Time (sec) to 7 for Vireo and Phoenix for best turn performance in area waypoints. 7. “Approach Spd”, “Speed”, and “Dwell Time” parameters in the Area Waypoint can be adjusted for Indago. These speeds Dwell Time cannot be adjusted for Vireo/Phoenix A1 in the Area Waypoint window. Page 7 of 15 Approach Spd: Ground speed that Indago flies from its current location to the beginning of the area waypoint. 15mph is recommended for normal operations. Speed: Ground speed that Indago flies while inside the area waypoint. 15mph is recommended for normal operations. Dwell Time: the time that Indago pauses to hover at each photo point. Setting this to ‘0’ will cause Indago to maintain constant ground speed through the photo point. 8. Multiple area waypoints may be used to more efficiently cover non-rectangular fields. There is a limit of 14 waypoints in the “commercial” version of Virtual Cockpit. Page 8 of 15 5 Access and download the images and geolocation data off the cameras The Dual Band Sensor has 2 SD cards; one for each imager. Files, images, and folders can be accessed on the SD cards over the payload’s Wi-Fi network (if Wi-Fi module is installed) or by removing the SD Cards and using an SD Card reader. To access the folders on the SD Cards over the Wi-Fi network do the following: 1. Verify your computer is connected to the Dual Sensor’s Wi-Fi network. The Wi-Fi network is typically named: “Fourthwing_XXXX” or “Sentera_XXXX”, where the 4-digit XXXX number sometimes is programmed to match the serial number of the autopilot installed in the UAV. The Wi-Fi network password is the 4 digit number twice, e.g. Network: “FourthWing_3301” would have a password: “33013301” 2. Browse to the following two network folder locations by typing the folder addresses below into the Window’s “RUN” command or typing into the address bar of Windows file explorer: \\192.168.143.141\SDcard\Snapshots\ This folder contains near infrared (NIR) images and metadata used to generate NDVI \\192.168.143.142\SDcard\Snapshots\ This folder contains color (RGB) images and metadata 3. Copy the folders of interest from the SD Cards to your computer. Delete unwanted folders from the SD cards. Page 9 of 15 6 Optional: How to create a Pix4D Mapper project and load flight imagery and data from Sentera Dual Sensor payloads: 1. Download the folder of imagery / logfiles from the payload SD Cards. This folder should contain: o o o o o o o o o o o 2. .. All images.. autoexposure.csv firmware-version.txt fs-version.txt image-metadata.txt Imager.conf imu.csv jpegs.csv metadata.csv procerus-metadata.csv System.conf For best results, process the imagery to remove lens distortion and electronic rolling shutter distortion. Three common approaches to do this are: 1. Use the Sentera provided distortion correction software to batch process the images. When using Pix4D disable internal camera parameter optimization in the processing software. This has already been accounted for in the lens distortion removal in the processing done by the Sentera software. 2. Use Pix4D’s built-in tools to correct for lens distortion. To do this you must input the lens distortion coefficients into Pix4D. (click Project > Image Properties Editor, and follow the instructions from Pix4D) Select “Perspective Lens” and input image width 3808 pixels by image height 2754 pixels. There are two sets of lens distortion coefficients in the table below. Look at the "Lens" parameter in your Imager.conf logfile to determine which set to use. If the first character of the lens number is a 5 (e.g. "58-203"), use the first set of coefficients. If the first character of the lens number is a 6 (e.g. "69-255"), use the second set of coefficients. Again, if you used the Sentera provided batch processing software you should not do this step in Pix4D. Focal Lens length 5* 4662.25 6* 4609.67 Principle Principle Point x Point y R1 R2 R3 T1 T2 1962.35 1288.80 -0.2845735 0.3349859 -0.7456187 -0.00011867467 -0.00048929617 2007.21 1258.77 -0.3643960 0.1110922 0.2828553 -0.00027052967 -0.00117184600 3. Or, use another software tool of your choice to remove the lens distortion. Camera parameters if required by other software: 1.67um x 1.67um pixel size 8mm focal length Image resolution for 10 MP sensors: width 3808 pixels by height 2754 pixels 3. Open Pix4D and create a new project. Page 10 of 15 4. Add images to the Pix4D project. Page 11 of 15 5. After clicking "Next", wait for Pix4D to parse EXIF data. This may take a few minutes. Pix4D will eventually warn you about missing camera models. This prompt may appear many times. Choose OK for each of the prompts. TIP: Hold the “Enter” key to quickly progress through the prompts 6. Add geolocation and orientation data to the project. Choose "From file…". Verify the file format is "Pix4D Latitude - Longitude". Browse to the image-metadata.txt file. Pix4D automatically recognizes this metadata format, but other software tools may require the user to define the how the data is organized in the file. The image-metadata.txt file contains columns for filename, latitude (degrees), longitude (degrees), MSL altitude (m), yaw (degrees), pitch (degrees), and roll (degrees), in that order. 7. Click Next and Finish to create the project. Page 12 of 15 7 Optional: How to create an Agisoft PhotoScan project and load flight imagery and data from Sentera Dual Sensor payloads: 1. Download the folder of imagery / logfiles from the payload SD Cards. This folder should contain: o o o o o o o o o o o ..All images.. autoexposure.csv firmware-version.txt fs-version.txt image-metadata.txt Imager.conf imu.csv jpegs.csv metadata.csv procerus-metadata.csv System.conf 2. For best results, process the imagery to remove lens distortion and electronic rolling shutter distortion. Three common approaches to do this are: 1. Use the Sentera provided distortion correction software to batch process the images. 2. Use the built in lens distortion parameters in Agisoft Photoscan. There are two sets of lens distortion coefficients in the table below. Look at the "Lens" parameter in your Imager.conf logfile to determine which set to use. If the first character of the lens number is a 5 (e.g. "58-203"), use the first set of coefficients. If the first character of the lens number is a 6 (e.g. "69-255"), use the second set of coefficients. Again, if you used the Sentera provided batch processing software you should not do this step. Lens fx 5* 6* 4662.25 4662.25 1962.35 1288.80 -0.2845735 0.3349859 -0.7456187 -0.00011867467 -0.00048929617 4609.67 4609.67 2007.21 1258.77 -0.3643960 0.1110922 0.2828553 -0.00027052967 -0.00117184600 fy 3. cx cy k1 k2 k3 p1 p2 Or, use another software tool of your choice to remove the lens distortion. Camera parameters if required by other software: 1.67um x 1.67um pixel size 8mm focal length Image resolution for 10 MP sensors: width 3808 pixels by height 2754 pixels 3. Open PhotoScan and create a new project. Page 13 of 15 4. Select Workflow -> Add Photos. Choose all of the desired photos. 5. Add geolocation and orientation data to the project. On the left side of the screen, under "Ground Control", choose the "Import" button. Browse to the image-metadata.txt file associated with the loaded imagery and select "Open". Page 14 of 15 6. Configure the options as follows. Click "OK". o Delimiter: Comma o Start import at row: 1 o Columns (check "Load orientation") Label: 1 X: 3 Y: 2 Z: 4 Yaw: 5 Pitch: 6 Roll: 7 7. On the left side of the screen, under "Ground Control", click the "Settings" button. Change the "Coordinate System" from "Local Coordinates" to "WGS 84 (EPSG::4326)". Page 15 of 15