1
Project BLISS
Boundary Layer In-Situ Sensing System
Customer
Dr. Suzanna Diener
Northrop Grumman
Faculty Advisor
Dr. Donna Gerren
Team
Kyle Corkey
Devan Corona
Grant Davis
Nathaniel Keyek-Franssen
Robert Lacy
John Schenderlein
Rowan Sloss
Dalton Smith
2
Outline
• Project Purpose
• Project Description
• Test Overview and Results
▫ Delivery System
▫ Measurement System
▫ Cloud Observation System
• Systems Engineering
• Project Management
3
Motivation
• Northrop Grumman
Atmospheric Boundary
Layer Model
Verification
▫ Inertial wind velocity
▫ Cloud base altitude
Applications:
Airborne pollution monitoring
Prediction of forest fire advances
Images from: followgreen.com inhabitat.com
4
Project Deliverables
• 3-Dimensional U-, V-, Winertial wind vector data inside
the measurement cylinder
• Cloud base altitude and cloud
footprint images above the
measurement cylinder
Airspace Defined By
Measurement Cylinder
5
Levels of Success
Delivery System
Level 3:
Execute flight plan
following points spaced
no more than 30 meters
apart spanning the
defined airspace in the
15 minute time limit with
Measurement System
onboard and collecting
data
Measurement System Cloud Observation System
Level 3:
Deliver U-, V-, W- inertial
wind velocity vector field
with temporal and spatial
location for each
measurement accurate
to 1 m/s with a
resolution of 0.1 m/s.
Level 3:
Deliver cloud cloud base
altitude measurements and
cloud footprint images at
1/4 Hz during the 15 minute
test period.
6
Concept of Operations
Legend
Airspace Test
Volume Subject
To Modeling
Within
Project
Scope
In-Situ Relative Wind
Velocity Data
Collection and Cloud
Imaging
NG model
wind vector
100 m
200 m
200 m
200 m
200 m
Northrop
Grumman
Wind Model
Results
100 m
100 m
100 m
200 m
100 m
Inertial Wind
from In-Situ
Data and Cloud
Base Altitude
Physical
Wind
Vector
Wind Vector and
Cloud Data Used to
Verify Northrop
Grumman Model
Wind Vector
of in-situ
data
7
Delivery System Final Design
Delivery System
Goal: Move the measurement
system through the cylinder
while meeting spatial (30 m)
and temporal (15 min)
requirements.
0.92 m
Implementation:
• Skywalker X-8 flying wing
• Pixhawk Autopilot
• Ground Control Station
2.08 m
8
Measurement System Final Design
Measurement System
Goal: Deliver a 3-Dimensional
U-, V-, W- Inertial Wind Data within
accuracy of 1 meter per second.
Implementation:
 Rapid Prototyped 5-Hole Probe
 5 differential pressure
transducers
 1 absolute transducer
 Inertial Navigation System
 Post-Processing Algorithm
9
Functional Block Diagram
Northrop
Grumman Wind
Model
GPS
Power
Module
Inertial
U-,V-,WWind Vector
Field
GPS
Coordinates
14.8V
5V
Post Processing
Algorithm
Speed
Controller
Serial
Command
PWM
PWM
Pixhawk Flight
Controller
14.8V
Aircraft
State &
Wind
Pressure
Motor
Antenna
Delivery System
9V
Arduino Due
Electrical Power
System
Analog
Voltage
Inertial
Navigation
System
Serial
Thermistor
Analog
Pressure
Transducers
Air
Pressure
5-Hole Probe
Elevon
Servos
Manual
Commands
Electrical Power
System
SD Card
Measurement System
Aircraft State
& Wind
Pressure
Relative
Wind
The Measurement
System is packaged
in the Delivery System
10
Cloud Observation System Final Design
• Stereovision imaging system pointed
vertically in measurement cylinder
• Two Canon Powershot A3400
cameras with CHDK firmware hack
▫ Precise time-lapsing (Documented drift
of 2ms/day)
Cloud
▫ Turn off autofocus
Height
▫ Automatically take picture every 4
seconds
• Algorithm to detect common features,
compute disparity, and base altitude
85m
15m 15m
85m
11
Cloud Observation System Processing
Algorithm
Left Image
Dist =
f *b
disparity
f=focal length
b=camera separation
Right Image
12
Functional Block
Diagram Continued
X
Cloud Base
Camera Field
of View
Camera Field
of View
Northrop Grumman
Wind Model
Battery
Cloud Base Altitude
& Footprint
Computer with Post
Processing Algorithm
Power
Power
Vertical
Camera
.RAW Image
Left and Right
Images
Internal SD
Card
Vertical
Camera
.RAW Image
Internal SD
Card
Cloud Observation System
Battery
13
Major Changes Since TRR
• Pawnee National Grasslands was cancelled as final data
collection site
▫ Testing closer to CU allowed for more test and data collection
flights
 More data using less of our pilots time
▫ Customer prefers this approach as 115 m height will satisfy her
models
• Due to inadequate documentation, initial calibration
algorithm was abandoned
▫ New calibration method is unable to accurately predict angles
greater than 30° in α or β (flow separation)
▫ This corresponds to a maximum perpendicular wind of 10 m/s
Project
Purpose
Design
Description
Testing
Overview
Testing
Results
Systems
Engineering
Project
Management
14
Critical Project Elements
CPE
Requirement
Motivation
Obtaining a COA
4.1.1
UAV cannot legally fly without a COA
Flight Path
1.1.1.1, 3.1
To meet required spatial and temporal
measurement resolution
Rapid Prototyping 5-hole probe
1.2
Used to measure wind
Calibrated 5-hole probe
1.2.3
Need to geometrically calibrate the probe to
accurately measure wind
Aircraft State Knowledge
1.2.2
Needed to convert relative wind to inertial wind
Wind Post Processing Algorithm
1.2.1
Needed to take raw data and create inertial wind
vector field
Cloud Observation Algorithm
2.2.2
Deliver cloud data within required error bounds
Project
Purpose
Design
Description
Testing
Overview
Testing
Results
Systems
Engineering
Project
Management
15
Delivery System Testing Overview
• Flight Plan Test
▫ Verify that autopilot can follow the flight path in a real
world test
▫ Test conducted at Table Mountain
▫ Verify that 30 meter spatial resolution (REQ. 1.1.1) and
15 temporal requirement (REQ 3.1) are met
 This requirements would allow Delivery System to
achieve Level 3 success
Project
Purpose
Design
Description
Testing
Overview
Testing
Results
Systems
Engineering
Project
Management
16
Delivery System-Flight Test
• Flight Path Test
Compares Simulated
Flight Path To Flight
Test Data.
• 94% Coverage of Data
Collection Locations
Achieved in Testing.
Project
Purpose
Design
Description
Testing
Overview
Testing
Results
Systems
Engineering
Project
Management
17
Delivery System-Flight Test
Simulated Flight Path
Flight Path From Testing
18
Measurement System Testing Overview
• Calibration and Verification of the 5-hole probe
▫ Allow the probe to convert 5 pressures to 3-dimensional
relative wind
▫ Verify Requirement 1.2.3
• INS Verification
▫ INS uncertainties over time during flight testing
▫ Verify Requirement 1.2.2
Project
Purpose
Design
Description
Testing
Overview
Testing
Results
Systems
Engineering
Project
Management
19
Measurement System- Calibration Results
• NASA 3rd order multiple linear regression model
used
• Predicts α, β, and total pressure
• Model accurately matched calibration data
▫ R2 = 0.999
Project
Purpose
Design
Description
Testing
Overview
Testing
Results
Systems
Engineering
Project
Management
20
Measurement System- Calibration Results
Project
Purpose
Design
Description
Testing
Overview
Testing
Results
Systems
Engineering
Project
Management
21
Measurement System - Verification of Calibration
• Calibration data was verified using
ITLL wind tunnel
▫ The probe was set to a random alpha,
beta, and velocity and data was
collected
▫ Calibration data was then used to
predict angles and compared to the
known values
▫ Required accuracies of α within 3.44°
and β within 2.97° verified until flow
separation
▫ R2 values:
 α – 0.928
 β - 0.908
Project
Purpose
Design
Description
Testing
Overview
Angularity uncertainty verification
Airspeed
αpredicted
αtrue
α error
21 (m/s)
9.90°
12.57°
2.67°
22 (m/s)
-0.94°
-0.63°
0.31°
24 (m/s)
-7.12°
-8.64°
1.52°
25 (m/s)
-13.43°
-12.85°
0.58°
25 (m/s)
-0.29°
1.01°
1.30°
Airspeed
βpredicted
βtrue
β error
21 (m/s)
7.44°
10.34°
2.90°
22 (m/s)
-1.64°
-2.75°
1.11°
24 (m/s)
9.63°
10.94°
1.31°
25 (m/s)
-17.50°
-16.95°
0.55°
25 (m/s)
8.91°
7.19°
1.72°
Testing
Results
Systems
Engineering
Project
Management
22
Measurement System-INS test
• INS accuracy outputs during data collection are within
necessary requirements defined by error budget
▫ INS was tested against a potentiometer to verify these outputs
Maximum
Error
Allowed
Error
Aircraft Velocity 0.17 m/s
0.50 m/s
Roll
0.09o
0.50o
Pitch
0.09o
0.50o
Yaw
0.21o
0.50o
Project
Purpose
Design
Description
Testing
Overview
Testing
Results
Systems
Engineering
Project
Management
23
Measurement SystemINS test
GPS velocity
Flight begins
uncertainties
• All
uncertainties
below
maximum
allowable
error
Project
Purpose
GPS signal
acquired
Euler Angle
Uncertainties
Flight begins
GPS signal
acquired
Design
Description
Testing
Overview
Testing
Results
Systems
Engineering
Project
Management
24
Cloud Observation System (COS) Testing Overview
• Full Scale Test
▫ Reason: Verify COS base altitude measurement meets
error requirement, 10% for clouds at 2km (REQ. 2.2.3)
Cloud Height
Model Error Prediction
2 km
210 m
3 km
470 m
4 km
810 m
▫ Method: Compare COS measurements to CU Skywatch
Ceilometer data
Project
Purpose
Design
Description
Testing
Overview
Testing
Results
Systems
Engineering
Project
Management
25
Cloud Observation Full Scale Test
• Error grows with cloud
height, as model predicts
• ≈58% of COS
measurements fall within
10% of Ceilometer
• Each image set has 40+
matched features
Project
Purpose
Design
Description
COS Measurement & Standard Error of the Mean
Testing
Overview
Testing
Results
Systems
Engineering
Project
Management
26
Verification from Testing
▫ Wind sensing unmanned aerial system able to measure
inertial winds accurate to 1 m/s with 94% of data collection
meeting the 30 meter spatial requirement
 Winds aloft and inexpensive autopilot led to less than Level 3
success of Delivery System (100% required for level 3)
 1 m/s accuracy of inertial winds meant the Measurement
System achieved Level 3 success
▫ Confidence in Cloud Observation System improves when
measuring lower altitude clouds with distinct features
 Best results when 40+ matching features found in image sets
Project
Purpose
Design
Description
Testing
Overview
Testing
Results
Systems
Engineering
Project
Management
27
Validation from Testing
Customer Objectives & Needs
Results
“Algorithm to determine near real-time
wind vector fields (u-, v-, w- directions)”
Can create u-, v-, w- vector field within
error requirements shortly after
offloading data from flight.
“Exploit high spatial and temporal
resolution in horizontal grid space over
500 meter radius area, 200 meters above
ground with 10 meter resolution”
De-scoped the project for a 100 meter
radius, 115 meter above ground with 30
meter resolution. Resolution could be
accomplished on low wind days.
“Observations of cloud cover conditions
including base altitude coincident with
wind data”
Cloud base altitude with near 10% error
for clouds below 3km. Data is collected
during flight of UAS.
Project
Purpose
Design
Description
Testing
Overview
Testing
Results
Systems
Engineering
Project
Management
28
Systems Engineering ‘V’
Customer Needs
• In-situ as well as
remotely observed
wind observations
• 1 m/s accuracy with .1
m/s precision and 1
m/s variance
• 500m radius cylinder
with a height of
200m
• Instantaneous
collection of points
spaced 10m
horizontally and
vertically
Operation
And Upgrades
Project Planning
System
Validation
CONOPS
System
Requirements
System
Verification
High-Level
Design
Subsystem
Integration
Detailed
Design
Unit testing
Software Development
Hardware Fabrication
Project Timeline
Project
Purpose
Design
Description
Testing
Overview
Testing
Results
Systems
Engineering
Project
Management
29
Systems Engineering ‘V’
Customer Needs
Operation
And Upgrades
Project Planning
De-scope
CONOPS
System
Requirements
• Reduced the radius of the
measurement cylinder (100m).
• Reduced spatial (30m) and
temporal (15min) resolution
High-Level
Design
System
Validation
System
Verification
Subsystem
Integration
Detailed
Design
Unit testing
Software development
Hardware Fabrication
Project
Purpose
Design
Description
Testing
Overview
Testing
Results
Systems
Engineering
Project
Management
30
Systems Engineering ‘V’
Customer Needs
Operation
And Upgrades
Project Planning
System
Validation
CONOPS
Lessons Learned
• Requirements and
Levels of Success
development needed to
be more quantifiable
and performance based.
System
Requirements
Lessons Learn
Testable
High-Level
Design
System
Verification
Subsystem
Integration
Detailed
Design
Unit testing
Software development
Hardware Fabrication
Project
Purpose
Design
Description
Testing
Overview
Testing
Results
Systems
Engineering
Project
Management
31
Systems Engineering ‘V’
Customer Needs
Lessons Learned
• Allocate plenty of time for
post processing
CONOPS
System
Requirements
Lessons Learned
• Communication
is imperative
• Develop offramps
High-Level
Design
Lessons Learned
• Ensure designs are finalized
before machining and
implementation
Project
Purpose
Operation
And Upgrades
Project Planning
Design
Description
Detailed
Design
System
Validation
System
Verification
Subsystem
Integration
Unit testing
Software development
Hardware Fabrication
Testing
Overview
Testing
Results
Systems
Engineering
Lessons Learned
• Test early and often
• Flight Testing is a
slow process
• Schedule buffer is
crucial
Project
Management
32
Systems Engineering ‘V’
Customer Needs
Operation
And Upgrades
Subsystem Integration
• Successfully integrated
measurement system
with the delivery system
Project Planning
CONOPS
System
Requirements
System
Validation
System
Verification
High-Level
Design
Subsystem
Integration
Detailed
Design
Unit testing
Software development
Hardware Fabrication
Project
Purpose
Design
Description
Testing
Overview
Testing
Results
Systems
Engineering
Failure Analysis
• Crash footage
analyzed to
prevent further
incidents
Project
Management
33
Systems Engineering ‘V’
Customer Needs
Operation
And Upgrades
Project Planning
System
Validation
CONOPS
Delivery System
• 94% of data collection points
achieved within 15 minute
temporal req. with exceeding
10 m/s
• 100% data collection
achievable on low wind
days
High-Level
Design
Did we build the product
right?
Detailed
Design
Cloud Obs. System
System
Verification
Subsystem
Integration
Unit testing
Software Development
Hardware Fabrication
• 58% of points in test were
within required 10% error
• Less error with lower altitude
clouds
Project
Purpose
Verification
System
Requirements
Design
Description
Testing
Overview
Testing
Results
Systems
Engineering
Measurement System
• Wind angularity during
calibration known to
𝛼=3.44°, 𝛽=2.97°
• INS Euler angular
accuracy less than 0.5°
Project
Management
34
Systems Engineering ‘V’
Customer Needs
Operation
And Upgrades
Project Planning
Validation
CONOPS
PDD Objectives
• Near real time
UVW wind
vector fields
• High spatial and
temporal
resolution
• Observations of
cloud cover
conditions
Project
Purpose
System
Requirements
Did we build the right product?
• Post processing in progress
High-Level
Design
System
Validation
System
Verification
Subsystem
Integration
Detailed
Design
Unit testing
Software Development
Hardware Fabrication
Design
Description
Testing
Overview
Testing
Results
Systems
Engineering
From Testing
Deliver Inertial U, V-, W- wind
vector field
Flight path hit
94% of points in a
windy test
Cloud base
altitude
measurements/fo
otprint
Project
Management
35
Project Management Approach
• Listened to everyone’s ideas and suggestions
▫ Group effort, certain people had areas of expertise and were better suited to specific
tasks
• Tasks assigned through 15 minute meetings and over email
▫ Formal meetings were challenging to arrange, these were implemented through our faculty
advisor meetings
• Dynamic schedule
▫ The project was ever evolving, the final schedule was significantly different than our
planned CDR schedule
• Being tough on all team members but recognizing efforts of each individual
• Took on technical tasks and was willing to help members with any task to assure tasks were
done correctly and on time
Project
Purpose
Design
Description
Testing
Overview
Testing
Results
Systems
Engineering
Project
Management
36
PM Successes and Lessons
• Successes:
• Lessons Learned:
▫ Flexible Schedule
▫ Things never work the first time
 Finished project on time
▫ Flight testing, electronics,
calibration
▫ Team morale always high
▫ Important to test early and often
 Great team chemistry
▫ Everyone has different personalities
▫ Happy customer
and is motivated differently
 Success of project a large
▫ Important to understand the best
factor
way to work with each member
Project
Purpose
Design
Description
Testing
Overview
Testing
Results
Systems
Engineering
Project
Management
37
Final Budget
•
•
•
•
•
Estimated Expenses at time of CDR: $4708.29
Total Expenditures thus far: ~ $4650
Remaining Margin: ~ $350
Notable savings from shipping budget allocation
Additional Expenses
▫
▫
▫
▫
▫
Machining Hardware
Mounting Hardware
3 Motors
An additional battery
Paint and glue
• Only remaining expense will be
printing the final report
Team Members
8
Average Hours Per Week
20
Weeks
31
$65,000 Salary
$31.25/hr
Subtotal
$155,000
200% Overhead
$310,000
Total
$465,000
Budgeted
Design
Description
Under(Over)
Delivery System
$
1,265.00
$
1,437.97
$
-172.97
Measurement System
$
2,562.47
$
2,412.90
$
149.57
Cloud System
$
355.97
$
241.90
$
114.07
Shipping
$
500.00
$
80.10
$
419.90
$
454.15
$
-454.15
Miscellaneous Expenses -
Margin
Project
Purpose
Actual
Testing
Overview
Testing
Results
$
292
Systems
Engineering
$
348.13 $
Project
Management
56.13
38
Acknowledgements
• We would like to thank all of the PAB, our
advisor Dr. Gerren, our customer Dr. Diener
from Northrop Grumman, Trudy Schwartz,
Bobby Hodgkinson, Matt Rhode, and especially
James Mack for being our pilot and helping us to
finish our project and be successful.
39
Questions?
40
Levels of Success
Delivery System
Level 1:
Certified to operate in an
airspace defined as a
cylinder with a 100 meter
radius and 200 meter
height above ground level.
Level 2:
Executes flight plan following
points spaced no more than
30 meters apart spanning the
defined airspace.
Level 3:
Execute level 2 flight
plan with Measurement
System onboard and
collecting data
Motivation: The measurement system needs to be
transported through the measurement cylinder to meet
special and temporal requirements.
41
Levels of Success
Measurement System
Level 1:
Wind measurement system
collects relative wind data
with resolution of 0.1
meter/second.
Level 2:
Post-process the relative
wind data from a ground test
to compute the U, V, W
inertial wind velocity vector
components.
Level 3:
Deliver U-, V-, W- inertial
wind velocity vector field
with temporal and spatial
location for each
measurement.
Motivation: Provide Northrop Grumman with data
precise enough to verify a boundary layer wind model.
42
Levels of Success
Cloud Observation System
Level 1:
Image the cloud footprint
above a 100 meter radius
cylinder at 1/4 Hz for a 15
minute period.
Level 2:
Level 3:
System is tested in full scale
Deliver cloud footprint
to take distance
images and cloud base
measurement with less than altitude measurements at
10% error up to 2km
1/4 Hz during the 15 minute
test period.
Motivation: Provide Northrop Grumman with cloud observation
data to correlate with wind vector field measurements.
43
Experimental
Setup
Legend
BLISS Measurement
and Delivery System
Data points –
Spaced at most
30m radially in
3D space
Cloud observations
constrained to the
measurement
cylinder’s vertical
projection
100 m
Physical Wind
Velocity Vector Field
(u-,v-,w-)
Cloud Observation
System stereovision
cameras
Atmospheric clouds
located high above
test volume
200 m
≤ 30 m
In-Situ relative wind
velocity data collection
44
Delivery System Flight Test- Backup
45
Delivery System Flight Test-Backup
46
Flight Path Test-Backup
• Mean Loiter Altitude
Error 5.7 Meters.
• 0.086 Meter Accuracy
in Altitude
Measurement.
47
Flight Path Test-Backup
• 28% Radial Position Error during Data
Collection in High Winds.
• Most Radial Position Error can be
Attributed to Navigational Errors Due
to Wind.
• Tests were conducted in high wind
environments due to Customer Desire.
• 12% Radial Position Error during
Autonomous Flight in Low Winds.
• Winds Aloft Increased Loiter Duration
and Downwind Loiter Radius.
48
Airspeed and Groundspeed-Backup
49
Power Consumption Model-Backup
• 3464.2 mAh Predicted
• 2058.4 mAh Used
• 41% error between model
and test data.
• Likely due to under
estimates of motor
efficiency, estimated at 50%.
• At 80% motor efficiency,
error between model and
test data is 5%.
50
Battery Voltage-Backup
• 12.1 V Needed To Produce
Minimum Required
Thrust
• Minimum Battery Voltage
of 4S LiPo 12 V.
• Battery Voltage
Monitored During Flight
Testing with 14 V Cutoff
During Autonomous
Flight.
51
GPS vs Barometric Altitude-BACKUP
52
Calibration
• Due to inadequate
documentation, initial
calibration code procedure had
to be abandoned
• New calibration method is
unable to accurately predict
angles greater than 30° in α or
β (flow separation)
• This corresponds to a
maximum perpendicular wind
of 10 m/s
Aircraft cruise
velocity magnitude
17 m/s
Resultant air velocity
magnitude
20 m/s
30°
10 m/s
Wind gust
magnitude
53
Calibration – Beta Comparison
54
●
Post Processing Software
REQ 1.2.1: Relative wind data shall be post processed in order to determine the U-,
V-, W- inertial wind vectors at each point.
55
INS test
• GPS position uncertainty during data collection
Flight begins
GPS signal
acquired
56
INS test
57
Probe Calibration geometry
Probe tip
β
V∞
α
w
u
v
58
Cloud Observation System
59
Budget
$3,000.00
$2,562.47
$2,412.90
$2,500.00
$2,000.00
$1,500.00
$1,265.00
$1,437.97
CDR Expense Estimation
Expenses at SFR
$1,000.00
$355.97
$500.00
$241.90
$500
$80.10
$-
Delivery System
Measurement
System
Cloud System
Shipping
$479.00
$0
Miscellaneous
Expenses