Smarter Travel Marketing
2023
AI RLI N E DI GITAL
OPTIMIZ ATION YEARBOOK
Airline industry insights for a higher conversion rate and
better user experience
Sponsored by:
2023 | © Diggintravel
ABOUT THE AUTHOR
ABOUT DIGGINTRAVEL
Iztok Franko is passionate about digital
marketing and e-commerce. He has more than
10 years of experience as a CMO and CIO in
airline, travel and multinational companies.
He currently works as a strategic digital
marketing and ecommerce consultant for
global airline and travel brands.
Diggintravel is a content platform, designed for
airline ecommerce and digital marketing
professionals.
Through practicing true data-driven digital
marketing, testing several ecommerce
processes and experimenting with different
team management approaches, he has
established a conversion optimization
framework that works.
Iztok also regularly writes and speaks
about travel and airline marketing, ecommerce,
conversion optimization and
ancillary topics.
He is the founder of diggintravel.com,
a content platform for smarter travel marketing.
Diggintravel was built to help digital marketers
in the travel industry, especially airline
professionals who run and manage online
sales.
We provide in-depth insights and research on
airline ecommerce and digital marketing.
Diggintravel specialzies in airline conversion
rate optimization and ancillary revenue
disciplines and by combining both helps airlines
become true digital retailers.
If you are an airline looking to grow your online
sales; we can help you with tailor-made
workshop for airlines, advanced analytics
designed for airline booking funnel and
conversion optimization consulting services.
2
ABOUT THE SPONSOR
Re-architect your digital channels
around your customers
At Branchspace, we are a passionate team with deep cross-functional experience in airline &
travel tech, travel retail and end-to-end customer experience.
We want to be the most forward-thinking & trusted technology partner for airlines and other
travel companies. We break barriers of legacy technology & thinking. Jointly with our
customers, we create better ways to plan, book and experience travel.
We offer transform consulting services and Triplake, our best-in-class digital commerce
platform. With Triplake, we deploy the latest technology & retail thinking and make it easy for
you to drive personalised end-to-end experiences for your customers and meet your revenue
goals.
We’re an IATA Strategic partner for digital innovation to help shape the future of travel. We
have been a long-term trusted partner for LH Group, IAG, Aegean, TAP, Air Malta & more,
since 2013.
3
THE FOLLOWING EXPERTS PROVIDED
INSIGHTS FOR THIS YEARBOOK:
During our research, we conducted several in-depth interviews with various digital leaders on our
Diggintravel Podcast. Key interview highlights and experts’ insights are provided in our detailed survey
results chapters in the special “Ask the Expert” sections. Full audio podcast interviews are available on
the Diggintravel Blog and Diggintravel Podcast websites.
ANDRES BUCCHI
ISMAEL MONZON
Chief Data Officer at LATAM
Airlines
Digital Growth Team
Manager at Air Europa
Andres is a seasoned executive with a solid
technical and entrepreneurial background,
experienced in the tech and retail industries. He is
the Chief Data O cer at LATAM Airlines and held a
prior role as Vice President – Data and Analytics
at Sodimac as well as working in data science at
Uber.
BEN LABAY
Ismael has 15 years of professional experience
working on several international B2C and B2B
digital products which have a few million monthly
users. He is a longtime supporter of Diggintravel
research and currently leads Air Europa’s growth,
digital analytics and CRO initiatives.
URSULA SILLING
CEO at Branchspace
Experimentation Expert,
CEO at Speero by CXL
Ben is the CEO of Speero by CXL, a speaker on all
things research and experimentation, an artist, and
a conservation science consultant. He’s done 10+
years in academia and conservation science, and
5+ years working in the marketing and digital
industries.
ffi
ffi
2023 Airline Digital Optimization Yearbook
Ursula is the CEO of Branchspace, where she is
leading a team that is transforming digital
commerce for aviation and travel. Ursula has vast
airline experience: she was prior Chief Commercial
O cer at Kenya Airways, Commercial Director at
Air Malta, EVP – Commercial at Brussels Airlines,
and worked for or with many other airlines.
4
MATT RAVLICH
Former Digital Analytics
and Optimization Manager
at WestJet Airlines
Matt is currently a Managing Director for Alberta
Consulting Group (ACG), where he specializes in
optimizing their clients’ digital experiences using
the current tools they have and helping them
prepare for the future. Prior to that, Matt was
Manager of Digital Analytics and Optimization at
the second-largest Canadian airline, WestJet
Airlines, where his main focus was growing
maturity levels with WestJet analytics and
optimization programs.
MARIANA FONSECA
MEDINA
Senior Vice President –
Loyalty Product at
Mastercard, prior Managing
Director – Digital Customer
Experience at American
Airlines
Mariana is a results-oriented executive with a
passion for product management. She upscaled
digital teams in a large airline organization
environment at American Airlines, and she built
one almost from scratch in a startup setting at
Virgin Voyages.
JAKE MALONEY
Director Ecommerce and
Digital Products at Frontier
Airlines
Jake is an accomplished leader with 19 years of
experience across multiple industries. He is highly
versed in digital solutioning, development, and
analytics. Jake is an expert at creating customercentric solutions and capitalizing on cost-effective
approaches to exceed company goals.
RICH PAGE
Conversion Rate
Optimization Expert
Rich is a CRO expert and consultant with 15 years
of experience. He helps ecommerce businesses
increase their conversion rates and revenue by
discovering their users’ issues, doubts, and needs.
2023 Airline Digital Optimization Yearbook
MARK HURSH
Senior Director of Digital
Customer Experience at
Southwest Airlines
Mark is an experienced leader who is passionate
about digital product management, experience
design, business strategy development, largescale
cross-functional program management, and
project execution. Mark has a background in the
Travel industry with an emphasis on the airline,
hotel/lodging, car rental, and meta search
verticals.
LORENZO CARRERI
CRO, User Research and
Experimentation Consultant
Lorenzo has 12 years of experience in digital and
tech, focusing on media buy, growth and
experimentation. He consults businesses on CRO,
analytics, user research and experimentation.
5
WHERE’S WHAT
PART I.
INTRO AND HIGH LEVEL RESULTS
INTRODUCTION
8
Author’s note
Taking experimentation and scienti c
11
decision-making beyond the digital space
ABOUT THE SURVEY
About the 2022 Airline Digital Optimization
13
survey
The Diggintravel Airline Optimization
15
Optimization maturity model
KEY FINDINGS
Findings summary – State of Airline Digital
18
Optimization
Laggards
20
Challengers
22
Visionaries
23
Leaders
24
Airline Digital Optimization Teams Growth
27
PART II.
SURVEY DETAILS PER SECTION
SECTION I - People
SECTION II - Skills and Knowledge
SECTION III - User and UX Research
30
41
50
59
SECTION V - Test Quantity
66
SECTION VI - Tools
75
SECTION VII - Internet Booking Engine
85
SECTION VIII - Organization Support
95
fi
SECTION IV - Digital Analytics
Smarter Travel Marketing
2019
TITLE OF DIGITAL
SURVEY
INTRODUCTION
Airline industry insights for a higher conversion
Month 2019 | © Diggintravel
AUTHOR’S NOTE
When I started with the rst airline digital
optimization survey back in 2017, I never could
have imagined that we would get to its fth
anniversary and that it would grow so big. In our
rst survey we benchmarked 28 airlines, mostly
out of curiosity to see how airlines were building
and optimizing their websites and other digital
touchpoints. Back then, conversion rate
optimization (CRO) and experimentation were not
widely known or adopted among airline digital
experts, so for a lot of them, our rst published
report was an introduction to this exciting digital
marketing discipline. Five research projects later,
we’ve almost doubled the number of surveyed
airlines – from 28 in 2017 to 55 in 2023 – and
many airlines are using our frameworks to grow
the maturity of their digital teams.
I don’t think it’s an overstatement to say that the
2023 Airline Digital Optimization Yearbook is the
result of the most comprehensive annual airline
digital research and benchmarks in existence.
Speaking of benchmarks, let me share a story
about airline digital benchmarking. During our
2023 research I talked to a CIO of one of the
biggest airlines in the world. He ran across our
past research and wanted to know how our
benchmarking process works. This airline
executive understood that our survey is an
assessment of the process (e.g., how airlines do
digital analytics, UX research, experimentation,
etc.), but he was interested in benchmarking the
output. We agreed that our survey measures the
“input,” whereas what the customer experiences at
the end (airline digital products) is the “output.” I
understand the appeal of global ranking of airline
websites or apps, but I think it would be very
di cult to do it globally and agree on joint success
metrics. What I offered to this airline leader is an
explanation that while our survey doesn’t measure
the “output,” we see a strong correlation between
the “input” and the “output.” What do I mean by
that?
During the last six years of doing benchmarks of
the airline digital optimization process, as well as
doing several workshops and consulting projects
where we educated airlines on CRO and helped
them plan the path to improve their maturity, I saw
a clear connection between a structured and
systematic digital optimization process and a
good airline digital user experience. Simply put,
airlines that commit to a data-driven, scienti c
approach to optimizing their digital user
experience build better digital products in the long
run. On the other hand, I still see many airlines
buying expensive digital platforms as a shortcut to
a great product (output), without committing to
the process (input).
I know my answer was not completely satisfactory
for the airline executive, who wanted to see how
his airline’s digital user experience compares to
their main competitors at that exact moment. But I
can assure him or any other airline leader that
investing in a systematic and structured digital
optimization process is the best long-term
guarantee for a good output.
Enjoy this yearbook!
Iztok
8
fi
fi
fi
fi
2023 Airline Digital Optimization Yearbook
ffi
fi
5TH AIRLINE DIGITAL OPTIMIZATION SURVEY
AND RESEARCH
In our last Yearbook we had a section titled “Will
2022 be the year for digital recovery and smarter
digital marketing?” A year later, we can con dently
say that after the industry took a big step back
during the COVID-19 pandemic, we nally saw a
rebound when it comes to airline digital
optimization and digital optimization teams. Many
airlines started to rebuild their digital teams; 60%
of digital leaders in our survey said they increased
the size of their digital teams over the past 12
months.
What's more, some airlines, like Air Europa, fully
committed to digital and direct distribution
because of the disruption and changed consumer
behavior that happened during the pandemic. It’s
not only the digital optimization teams that
improved. We see a positive change in overall
airline digital optimization maturity. While there
are still many airlines at the beginning of their
digital optimization journey, our research shows
that there are more airlines that are systematically
embedding CRO and experimentation into their
digital product teams and process. This trend is
not exclusive to the airline industry;
experimentation leader and our Yearbook
contributor Ben Labay sees a continuous trend of
CRO evolving into business operations in other
industries as well. The “CRO way of thinking,” that
failure is a feature and not a bug, is being
recognized and accepted more and more.
This change can be seen in our survey analysis, as
we saw a shift from Level 3 to Level 4 in almost all
of our eight digital optimization areas. Some
smaller airlines, like the aforementioned Air
Europa or Scoot, made a transition from a oneman-band CRO enthusiast to building a full-scale,
centralized digital optimization team.
On the other hand, there are bigger airlines that are
embedding CRO and experimentation principles in
their distributed digital product teams. During our
research, we ran across several roles from large
airlines like easyJet, Eurowings Digital, British
Airways, and Qatar Airways that show CRO and
experimentation are being widely adopted across
their digital ecosystems. Airlines are combining
experimentation with agile thinking and processes
to build better digital products based on data and
user feedback. Some airlines, like easyJet and
Eurowings Digital, have experimentation and a
scienti c approach to decision-making present in
their company mission statements. Airlines, like
LATAM, went even a step further and are
implementing an experimentation-based scienti c
approach to decision-making in non-digital units
and processes. You’ll nd several examples of
roles and organization models along with experts’
interviews throughout this Yearbook. We hope
they can help your airline progress on this journey
and take the next step on your path to digital
optimization maturity.
Finally, we’re glad to announce we have partnered
with Branchspace as a partner and a sponsor for
this Yearbook. They are a relatively new player in
the airline digital world, but one that promotes
experimentation and CRO. Our survey showed
airline internet booking engines (IBEs) are still one
of the biggest hurdles for CRO and
experimentation, so new IBE providers that provide
such functionalities on their platforms are a breath
of fresh air. On the next page, you can read about
how Branchspace sees the evolution of digital
optimization in the airline industry over the last
ve years and how CRO can unlock your airline
personalization initiatives.
fi
9
fi
fi
fi
2023 Airline Digital Optimization Yearbook
fi
fi
CRO AND EXPERIMENTATION ARE BECOMING A
BIG PART OF AIRLINE DIGITAL PRODUCT
OPERATIONS
EVOLUTION OF DIGITAL OPTIMIZATION IN
THE AIRLINE INDUSTRY
Radoslaw Dutkowski
Director Customer Success
"The customer-focused culture not only improves customer satisfaction and resolves
problems in digital interfaces, but also increases conversion and generates extra revenue.”
Optimisation techniques have become game changers for innovative airlines that focus on customer
experience and continuously search for new revenue opportunities. The airline industry has had to
quickly adapt to the changing customer experience — especially due to the pandemic — with
optimisation techniques and experimentation becoming a part of the customer-focused culture to
improve overall satisfaction, resolve digital interface problems, increase conversion and generate
extra revenue. Airlines have developed their capabilities in digital techniques and personalisation
mode in optimisation tools has sped up the process of bringing converting variants to the market.
Looking back to 2019, with the start of the pandemic, airlines significantly developed their capabilities
in optimisation techniques and became bolder in introducing experimentation into their customer
experience improvement process. In many cases, it is now a permanent step in the process and
often no changes are done to the customer-facing digital environment without any prior testing and
proving that the alternative variant improves measured variables.
In a nutshell, captured opportunities are a great motivator for airlines as they almost instantly can
monetise optimisation gains when the experiment brings positive results, which is not always the
case. Therefore, airline decision-makers are more inclined to invest in optimisation teams and
develop the team’s skills and tools to analyse the data and select promising test candidates more
accurately.
Another interesting trend is that airlines using personalisation mode in optimisation tools speed up
the process of bringing converting variants to the market. Implementing changes to the digital
environment often takes a long time. However, the winning variant can be used for personalization
immediately after the experiment ends, until the vendor updates the production environment.
At Branchspace, our technology delivers optimisation tools out of the box. We help airlines benefit
from a range of features such as personalised interfaces, improved conversion rates, and increased
revenue. We make the process of experimentation and implementation faster and easier so that
airlines can see the results of the optimisation process more quickly.
2023 Airline Digital Optimization Yearbook
10
ASK THE EXPERT:
TAKING EXPERIMENTATION AND SCIENTIFIC
DECISION-MAKING BEYOND THE DIGITAL SPACE
Andres Bucchi
Chief Data Officer at LATAM Airlines
This Yearbook is all about airline digital
optimization and experimentation in the digital
space. But you're working on concepts of
experimentation on another level.
“You already know a lot about that in the digital
world, as it’s your area of expertise. But that can
translate fairly well into the physical world. When I
joined the Pricing team at Uber, I realized that this
(experimentation) was driving everything. You
would see people go out of one experiment and
start thinking immediately about the next
experimentation time slot which could come a
month from then. Everybody threw ideas, then they
would do some back-of-the-envelope calculations
and everything revolved around it. The
organization was set so that this could be fast.
You would see some teams competing (with
different approaches to the same problem), it was
crazy. Like you thought you were working for
NASA or something.”
But how can you run experiments in the physical
world where A/B testing is not possible? What are
the challenges of running experiments in a
physical environment?
“I’m going to talk about pricing rst, but we can
talk about how this translates to home
improvement. We did a lot of experimentation
fi
fi
2023 Airline Digital Optimization Yearbook
there and some cases that might relate to aviation
as well. So, the main problem with changing prices
is that you affect a speci c set of (Uber) Drivers or
Riders if you want to do A/B testing, like in
traditional randomized control trials.
The problem with that is that if you affect let’s say
a bunch of Drivers with a more effective matching
algorithm, they would be exposed to this more
effective algorithm, and in theory what would
happen is they are able to take more trips. That
means they can satisfy more demand with the
same supply. The problem is that on the other end,
there are some drivers that have not been treated
with this new algorithm. What you’ll see in the
experiment is a big bias, because those drivers are
not only affected by the effect that the algorithm
itself would yield, but also by the fact that the
treated group is being more effective and kind of
taking demand from them.”
So, it’s not like in the digital world, A versus B? It’s
more like comparing two physical things?
“Yes, it’s a physical thing, so they compete for the
same resource. And you cannot really separate
everything where everything is super entangled
and it doesn’t matter if it’s one, two or three
alternatives. You’re never going to run out of
rendering your webpage or right buttons to click.
But in this case it’s the physical world, and you
actually run out of stock. And so this was a big
problem, and it was solved with experimental
setups.
Instead of using traditional A/B testing with
random control trials, you would do time sampling.
You have these windows where you will turn (the
algorithm) on, then off, etc.
11
ASK THE EXPERT:
Then you would get your pseudo-randomized
sample, which was good enough for running these
experiments. There were other alternatives, where
you sample in clusters that do not interact with
each other – this is something Facebook does a lot
– and what you end up having is groups that are
comparable in time. There is a technique that’s
called difference in differences which is very widely
used for this.”
If we translate this to the airline world, could the
same logic be applied to airline prices? So you
could say different groups on the same route, or
even on the same plane?
“Yeah, totally. You got it. This is actually the same
problem that the airlines have and the problem with
rolling out new pricing algorithms, or new revenue
strategies. It’s that you share the same routes. It’s
not like you can “treat” a plane and then “un-treat” it
for some customers, right? You would get into big
trouble, probably. Some people would end up
hating you, others loving you. But probably what
would happen is that the ones that love you would
get all the tickets rst. So then you would bias
everything.
fi
fi
2023 Airline Digital Optimization Yearbook
One way to go about this is using what traditional
(big tech) companies use when they compete
against a limited stock of physical goods. Amazon
has been doing this for a while now. I think eBay
started doing this way back when they realized
that their network had a limited capacity, so
treatment and controls would compete for a
limited resource. And they started doing either
switchbacks or comparing routes that do not
interact with each other and setting up these
difference-in-differences A/B testing type of
scenarios.”
NOTE:
If you want to learn in detail about running
experiments in a physical airline environment, you
can nd an article and the full interview with Andres
on the Diggintravel Podcast website.
12
Smarter Travel Marketing
2019
TITLE OF DIGITAL
SURVEY
ABOUT THE
SURVEY
Airline industry insights for a higher conversion
Month 2019 | © Diggintravel
small
22%
large
33%
Diggintravel’s 2023 Airline Digital Optimization
Survey is the fth survey and benchmark of
digital optimization in the airline industry.
Going into the fth year of our evaluation, the
survey will provide you with an overview of the
evolution of this exciting ecommerce discipline.
This survey certainly won’t be the last, as
Diggintravel will continue following the digital
optimization developments and airline
ecommerce trends. Every year, the survey has
evolved as we continue to develop our maturity
model and our digital optimization
questionnaire. The 2023 survey was signi cant,
as it marked the fth anniversary of
Diggintravel’s airline digital research. This is
why, apart from the yearly benchmarks, we
wanted to see how airline digital teams and
optimization practices have evolved since our
rst survey back in 2017.
medium
45%
LCC
33%
FSC
67%
The survey investigates the maturity of the
digital optimization processes and key
challenges airline professionals face in their
digital and conversion optimization efforts.
During the last quarter of 2022, our survey
questionnaire was sent to more than 110 airline
senior ecommerce, digital optimization and
digital marketing executives.
The survey represents the views and insights of
55 carriers (a 6% increase from last year’s
turnout, where 52 airlines participated). The
survey will provide you an intriguing insight into
the state and developments of digital
optimization for the airline industry.
Africa
13%
Americas
13%
Middle East
18%
Asia-Paci
fi
fi
fi
14
fi
fi
ABOUT THE 2023 AIRLINE
DIGITAL OPTIMIZATION
SURVEY
AIRLINE DIGITAL OPTIMIZATION MATURITY
MODEL
WHAT WERE WE ASKING?
The goal of the survey was to evaluate digital
optimization maturity within airline organizations.
For evaluation of maturity, we evaluated several
digital optimization maturity models and revised
last year’s Diggintravel Airline Digital Optimization
Maturity Model.
The new and revised Diggintravel Airline Digital
Optimization Maturity Model consists of 8 areas:
• 7 general digital optimization areas: people,
skills, analytics, user and UX research, test
quantity, tools, organizational support
• 1 airline-speci c area: internet booking engine
(IBE)
You can see the exact questions and results for
each area in the special section at the end of this
report.
Each area was ranked from Level 1 (basic) to Level
5 (most advanced).
We also added new questions to evaluate the
maturity of the digital optimization processes in
addition to the 8 questions by area. These were
the new questions added to the questionnaire:
Does your team follow a process for conversion
optimization?
Does CRO and digital optimization have its own
budget?
How are experiment results and learnings
implemented and shared across the organization?
Based on the answers to the additional maturity
questions, the nal maturity score was adjusted.
fi
fi
2023 Airline Digital Optimization Yearbook
Since personalization is a hot topic and we believe
it is tightly connected to digital optimization and
CRO (conversion optimization), we also evaluated
whether airlines combine conversion rate
optimization and personalization activities and
what kind of personalization engines they use.
Additionally, we asked each participant about the
key challenge that prevents them from taking
digital optimization to the next level and about
their 2023 investment priorities.
It seems that in 2023, digital optimization and CRO have
nally arrived on airline agendas at full scale. Last year
there was a breakthrough in terms of airline CRO roles
(you’ll nd many examples in our “Examples” sections
in the second part of this Yearbook), but more than that,
we’ve seen several airlines establishing CRO programs
and initiatives to embed CRO in their digital product
processes. During our research we’ve been in touch with
two large traditional airlines that were working on
setting up centralized CRO functions to provide
experimentation guidelines to several distributed digital
product teams. Hopefully our maturity model can
provide insights to other airlines that want to go on this
journey as well. The goal of the Diggintravel Airline
Digital Optimization Maturity Model is to help airlines
evaluate the level of their digital optimization process in
each of the key areas.
2023 Airline Digital Optimization Yearbook
fi
fi
THE DIGGINTRAVEL AIRLINE DIGITAL
OPTIMIZATION MATURITY MODEL
The maturity model also helps airlines strategically
plan the actions needed to advance their digital
optimization process in each area. Progress in most
cases is an evolution of step-by-step advancement
and rarely a revolution (skipping steps in the model).
Finally, the maturity model is also a tool for selfassessment for airlines to realistically assess their
real digital optimization and ecommerce competence.
In a lot of cases, there is an organizational belief that
ecommerce and digital optimization competence is at
a higher level than it actually is. Only when we do a
systematic evaluation of each area (especially user
research activities in place and quantity of tests) do
we usually get the realistic picture.
16
DO YOU WANT TO TAKE YOUR DIGITAL
OPTIMIZATION TO THE NEXT LEVEL?
This special airline digital optimization
workshop is the best rst step!
Discover all of the most important
aspects of airline conversion
optimization (CRO) in a hands-on,
1-2 day workshop.
Learn – we’ll provide you with
airline ecommerce and CRO best
practices and benchmarks based
on our research.
Identify - we’ll deep dive into your
booking funnel analytics and help
you identify key optimization
scenarios.
Plan – we’ll evaluate your CRO
maturity and help you plan the next
steps based on our CRO maturity
model.
Execute - we’ll help you prepare and
execute rst optimization scenarios
and A/B tests.
CONTACT iztok.franko@diggintravel.com
FOR A DRAFT WORKSHOP AGENDA
fi
fi
2023 Airline Digital Optimization Yearbook
17
Smarter Travel Marketing
2019
TITLE OF DIGITAL
SURVEY
KEY FINDINGS
Airline industry insights for a higher conversion
Month 2019 | © Diggintravel
FINDINGS SUMMARY – STATE OF AIRLINE
DIGITAL OPTIMIZATION
Based on the revised Diggintravel Airline Digital
Optimization Maturity Model and survey results by each
area, we classi ed each participating airline into one of
the following four categories: Laggards, Challengers,
Visionaries, Leaders
We used additional questions about the maturity of
digital optimization process (Is process documented;
Does it have dedicated budget; How are experiment
results shared) to adjust the nal maturity assessment
for each participating airline.
We grouped our eight maturity model areas into two
major groups:
According to our benchmark, we detected:
•
Completeness of vision—here we grouped areas
that relate to understanding the importance of the
digital optimization process, culture and
organization (People, Skills, and Organizational
Support).
•
•
•
•
15 Laggards
7 Challengers
25 Visionaries
8 Leaders
The graph below shows how these categories correlate
with the size and type of airlines.
• Ability to execute—here we grouped areas that relate
to the ability to execute a digital optimization
process in practice (Analytics, User and UX Research,
Test Quantity, Tools, and Internet Booking Engine).
fi
fi
2023 Airline Digital Optimization Yearbook
19
According to our maturity model, Laggards are the
airlines that benchmarked the lowest (around
Level 2 or lower on average) across all digital
optimization framework maturity model areas. 15
out of 55 airlines are in this group, and 14 out of
the 15 airlines in this group are traditional fullservice carriers.
I. People and company support:
These airlines are either just starting out with their
digital optimization process or have not yet
recognized digital optimization and CRO as a
critical part of their company growth. For twothirds of airlines in this group, digital optimization
is recognized and supported on a department level
(usually the ecommerce department) or even an
individual level (ecommerce or digital optimization
specialist/enthusiast).
13 (out of 15) airlines don’t have a dedicated fulltime person responsible for digital optimization.
Conversion optimization is performed either by an
online marketing specialist with general digital
marketing knowledge (8 airlines), or an ad-hoc or
part-time conversion optimization resource (5
airlines). Only 1 airline out of 15 in this group
claimed to have a full-time conversion
optimization resource, and only 1 other has a
small digital optimization team in place.
II: Skills and knowledge:
Only 4 out of 15 airlines in this group claimed to
have a deeper knowledge and experience with
digital optimization (CRO, UX, analytics, A/B
testing, content & copywriting). All others said they
have general digital marketing knowledge or basic
knowledge about CRO. Based on the answers in
the “People” category, we can infer that airlines in
this group either don’t have a general
understanding of the skills it takes to execute
digital optimization, or they lack the dedicated
resources and team to do so in practice on an
ongoing basis.
fi
2023 Airline Digital Optimization Yearbook
fi
fl
LAGGARDS
As you’ll see in the “Skills” section, digital
optimization requires a complex and large set of
specialized skills. This is a challenge for smaller
airlines, where there are typically fewer employees
with specialized skillsets compared to larger
companies.
III. User & UX research, tools and analytics:
Understanding your users is the core of digital
optimization, and this is one of the areas where
Laggards struggle the most. 12 out of 15
Laggards only do basic web analytics when it
comes to user research. The other 3 Laggards do
basic user & UX research (some session
recordings, simple A/B testing, heat maps and
click maps, ad-hoc customer surveys). None of the
airlines in this group do regular customer
feedback and survey analysis, form analysis, or
occasional unmoderated or moderated user
testing.11 airlines in this group identi ed web
analytics as their only digital optimization tool. 4
others use simple digital optimization tools
(simple A/B testing tools, simple user research
tools), and none of the Laggards claimed to use
advanced customer research tools, customer
experience analytics tools, or a simple
personalization tooling.
In addition, all but one of the Laggards claim to
have internet booking engines (IBEs) that are not
exible enough for A/B testing. Most airlines in
this group (11 out of 15 airlines) do basic web
analytics with some additional settings (goals,
simple funnel report, etc.).
IV. Testing quantity:
None of the 15 airlines in this group claimed to do
regular testing (2-5 tests per month). 8 airlines
said they do none or minimal testing; the other 7
only do A/B testing and other forms of
experimenting on an ad-hoc project basis only. If
testing is performed, it’s usually done for major
website redesigns. Most identi ed lack of
knowledge and resources and technology
constraints as their biggest obstacles to doing A/B
testing on a regular basis.
20
V. Process, budget and knowledge sharing:
Most Laggards haven’t yet started or are in the
very beginning stages of their digital optimization
journey, so the process is not documented and
structured. Only 1 of the 15 airlines claimed to
have a documented digital optimization process; 8
claimed to have a process for digital optimization
but it's not documented. 6 out of 15 airlines
claimed they have no budget for CRO, and the
other 9 claimed that they have a budget, though
it’s not dedicated but rather part of the overall
ecommerce budget.
NEXT STEPS FOR LAGGARDS:
EDUCATE!
Learn about digital optimization and CRO as
much as possible and understand the value of a
structured digital optimization process.
Find and train an internal digital optimization
champion who will put experimentation and CRO
on the map. It’s crucial to evangelize, advertise
and advocate for experimentation and A/B
testing within the ecommerce and digital
departments to foster the testing and data-driven
culture. Once education and intra-department
acceptance is achieved, it will enable airlines
from this group to build a good case for
organizational buy-in of digital optimization and
experimentation. At that point, a small, agile
digital optimization team and process can be
formed.
2023 Airline Digital Optimization Yearbook
21
CHALLENGERS
We classi ed as Challengers the airlines that are
not Laggards (so their average score per area is
above Level 2), but which are ranked higher in
“Ability to Execute” (above Level 3 on average)
than “Completeness of Vision” (below Level 3 on
average). 7 airlines t these parameters, so this is
the smallest of the four groups. Challengers do
more of “execution” but in a less structured and
organized way. Or they have advanced digital
optimization tools and do advanced analytics and
testing, but digital optimization and
experimentation are done on an individual level or
within a small digital optimization team. The value
of experimentation and CRO are not recognized on
a broader organizational level.
I. People and company support:
4 out of 7 airlines in this group have a CRO team in
place, and 2 airlines claimed they don’t have a
dedicated conversion optimization resource. This
means digital optimization is mostly done on an
individual level. Digital marketing specialists
perform digital optimization activities, but CRO is
still not a core ecommerce process. This is evident
from the responses regarding organizational
support: Only 1 of the 7 airlines in this group said
that CRO is recognized and supported by top
management. For 5 out of 7, CRO is recognized on
an individual or ecommerce department level.
II. Skills and knowledge:
4 out of 7 airlines in this group claimed to have a
deep knowledge of conversion optimization (CRO,
UX, analytics, A/B testing, copywriting). However,
this knowledge is mostly gathered on an individual
level, by CRO or digital optimization enthusiasts
NEXT STEPS FOR CHALLENGERS:
EMPOWER!
Challengers understand the value of digital
optimization and CRO but still struggle to get the
companywide exposure and support needed. CRO
experts and their department managers need to
be proactive in explaining the value of digital
fi
fi
2023 Airline Digital Optimization Yearbook
and specialists. The other 3 airlines claimed to
have only basic CRO knowledge and skills.
III. User & UX research, tools and analytics:
Airlines in this group are more advanced when it
comes to user research, analytics and digital
optimization tools. 5 out of 7 do user research
activities on an above-average level (Level 3 or
Level 4). All are well above average in digital
analytics (they do detailed funnel analytics,
attribution modeling and cross-device tracking),
and 5 out of 7 even claimed they do advanced
digital analytics and use advanced tools for
visualization (like Google Data Studio & BigQuery,
Tableau, Qlik, Power BI). The challenge for this
group is that activities are sometimes still done
for silo tests and optimizing micro metrics, rather
than within a long-term, companywide
optimization and digital product plan.
IV. Test quantity:
This group performs A/B tests and experiments at
a slightly above-average rate. 4 out 7 airlines do A/
B testing on a regular basis, but only 1 airline in
this group does at least 5-10 tests every month.
Challengers need to be careful not to use tests as
the main optimization activity, but rather use them
for validation of hypotheses only.
V. Process, budget and knowledge sharing:
Lack of organizational recognition for digital
optimization and CRO is evident for Challengers.
Only 1 of the 7 airlines doesn’t have a budget for
digital optimization activities (for 5 others, it’s a
part of the ecommerce budget), and 1 claimed to
have a dedicated budget for optimization and
experimentation. All but 1 of these airlines claim
to have a digital optimization process in place, yet
only 3 of them have it documented and structured.
optimization to their key stakeholders. Be
transparent about experimenting and make case
studies out of winning tests with clear ROI
calculation. This will help you get the recognition and
resources needed to take the next step – forming
digital optimization and CRO teams and structuring
activities and tests in a strategic optimization plan.
22
Visionaries is the largest group, consisting of 25
airlines. The airlines in this group are not
Laggards, so their average score is above Level 2
across all areas. Visionaries are airlines that
ranked higher in “Completeness of Vision” (above
Level 3 on average) than “Ability to Execute” areas
(below Level 3 on average).
I. People and company support:
Visionaries have a clearer understanding of the
potential and importance of CRO and digital
optimization than Laggards and Challengers. Their
average score is above Level 3 across the People,
Skills and Organizational Support areas. 76% of
airlines in this group claim to have a digital
optimization or CRO team in place. 6 even claimed
to have Level 5 full-scale conversion teams. 40%
of airlines in this group claimed to have top
management support for digital optimization and
CRO (at least Level 4).
II. Skills and knowledge:
19 out of 25 airlines in this group claimed to have
a deep knowledge of conversion optimization
(CRO, UX, analytics, content & copywriting). 9
airlines even stated they have advanced CRO
knowledge (UX excellence, advanced analytics,
conversion centered design) or better.
III. User & UX research, tools and analytics:
Visionaries have the means and vision but don’t
execute at the same level as Leaders or even, in
some areas, Challengers. Therefore, this area,
along with testing quantity, is where they lag
behind. 44% of airlines in this group still don’t do
systematic user research and user feedback
activities, and 48% mostly use simple tools for
such activities (simple user research tools, simple
A/B testing tools). One area that really prevents
Visionaries from executing better is the internet
booking engine. 72% said their IBE makes it
di cult to do A/B testing and experiments, and
none of the airlines in this group said their IBE is
exible enough to do more complex experiments
(Level 4 or Level 5).
IV. Testing quantity:
This is the key area in which Visionaries need to
improve. Experimentation is a crucial part of the
digital optimization and CRO process. Limitations
with tools and especially IBE result in less-thanoptimal execution of A/B testing and
experimenting. Only 1 out of 25 airlines in this
group do at least 5-10 A/B tests and experiments
per month. Further, 52% of airlines in this group
still test only on a project or ad-hoc basis.
V. Process, budget and knowledge sharing:
96% of Visionaries follow a digital optimization
process, 56% claim it is documented. 68% say they
have a budget for conversion optimization, but for
most it's part of Marketing or ecommerce budget.
NEXT STEPS FOR VISIONARIES: ENABLE!
Educate the IT team on the importance of
experimentation to get the right tools in place.
Implement an agile, modular digital platform and
internet booking engine that supports different
ows, instances and more complex A/B testing
(like split path testing).
Consolidate development to support CRO
activities and start building digital optimization
teams (digital product owner, conversion
ffi
2023 Airline Digital Optimization Yearbook
fl
fl
fl
VISIONARIES
specialist, analytics specialist, front-end & backend developers). Dedicate development resources
for testing (as part of your product development
sprints). Start with agile and systematic user
research activities to really understand your users’
friction points. Once the proper tools and team are
in place, increasing the number of experiments is
a critical next step for this group, as it has a direct
in uence on faster learning, growth and ultimately
innovation.
23
8 out of 55 airlines were classi ed as Leaders.
These are the airlines that are the best at doing
digital optimization and experimentation. Airlines
in this group are on average close to Level 4
(some are even higher) in all areas. There are 3
medium-sized airlines in this group; the other 5 are
all large airlines. It’s not a coincidence that bigger
airlines dominate this group. The resources and
expertise needed to build a CRO-centric
organization can be an issue for smaller airlines.
However, digital optimization should not solely be
the domain of the biggest airlines. Even smaller
airlines can advance to a higher level of digital
optimization maturity with a smart and agile
strategy. There are airlines in this group that have
achieved a high level of CRO maturity and high
experimenting volumes with a combination of insourced resources and outsourced help.
I. People and company support:
Digital optimization and experimentation have
become a part of the culture for these airlines.
CRO, testing and experimenting are recognized as
crucial processes on an organizational level. All
Leaders claim CRO has support at least at the VP
level, and 6 out of 8 even said conversion
optimization is recognized as a crucial activity by
their organization (Level 5).
All Leaders have dedicated CRO teams in place
with advanced CRO skills and knowledge. 4 out of
8 claimed to have a full-scale digital optimization
team (a team that consists of UX researcher,
copywriter, front-end developer, back-end
developer, and other experts), or several digital
optimization teams.
Roles like Conversion Optimization Program
Manager, Senior Digital Optimization Specialist,
and CRO Front-End Developer are common in this
group. Most of the digital optimization teams are
hybrid: a combination of internal digital
optimization resources and dedicated outsourced
CRO experts.
fl
2023 Airline Digital Optimization Yearbook
fi
fl
LEADERS
II. Skills and knowledge:
Leaders are pushing their CRO skillset to the next
level. They all have at least a deep knowledge of
CRO, UX, analytics, A/B testing, content and
copywriting. More than half (5 out of 8 airlines) of
this group go even further and claim to have
advanced digital optimization knowledge (Level 4
or Level 5).
III. User & UX research, tools and analytics:
Leaders recognize that understanding your users
is key. They all do regular user research and
feedback activities, extensive UX research, and
usability testing. Half (4 out of 8) of these airlines
claim to do scheduled and planned moderated and
unmoderated user testing, customer struggle
scores and prototype testing.
When it comes to analytics, all but one of the
Leaders do advanced digital analytics and use
special visualization tools. All Leaders claimed to
use testing platforms, and 3 out of 8 have built
their own tools for testing. One other key
characteristic of Leaders – they all have exible
internet booking engines. Half of the airlines in
this group even claimed to have a completely
exible IBE (Level 5) that empowers
experimentation.
IV. Testing quantity:
A true Leader can be recognized by the number of
experiments they run. All airline digital
optimization Leaders test and experiment on a
regular and planned basis (at least 2-5 tests per
month). What’s more, 6 out of 8 airlines in this
group claimed to do testing at a respectable scale
of 5-10 experiments per month.
V. Process, budget and knowledge sharing:
All but one of the Leaders follow a documented
digital optimization process. All Leaders have a
budget for conversion optimization, 50% say it’s a
special dedicated budget just for optimization and
experimentation.
24
NEXT STEPS FOR LEADERS:
SCALE, AUTOMATE, AND FOSTER A CULTURE OF EXPERIMENTATION
Even Leaders can take the next step since digital
optimization and CRO is about constant learning
and growth. While airline CRO Leaders are the best
among the airline industry, a gap still exists here
compared to the travel industry experimentation
elite like Airbnb, Uber, Booking.com, Skyscanner,
Hopper or eDreams.
We’ve seen several airlines doing that lately by
exploring different models to embed
experimentation and CRO in their digital product
teams. This requires additional investment in
digital, and so it’s not a coincidence that half of
our Leaders increased their digital optimization
team signi cantly over the past 12 months.
The real challenge for this group is nding a way
to do experimentation on an even larger scale and
catch up with the travel and digital elite. Leaders
need to “democratize” experimentation and work
on creating a decentralized organization to
support experimentation and innovation on a
bigger scale.
fi
fi
2023 Airline Digital Optimization Yearbook
25
2023 AIRLINE DIGITAL OPTIMIZATION
SURVEY – 10 KEY STATS
Our 2023 survey shows that most airlines are re-investing in their digital initiatives and teams. In fact,
several traditional carriers claimed that the pandemic was an opportunity for them to commit to digital,
ecommerce and direct distribution.
Below you can nd 10 key stats from our 2023 Airline Digital Optimization Survey that highlight the key
developments over the past 12 months:
60%
51%
of airlines have increased their digital
optimization teams over the past 12 months (see
the next section for more detail).
User-centricity remains a problem for airlines: 51%
don’t do regular user research activities. Only 20%
do scheduled moderated and unmoderated user
and prototype testing.
40%
of airlines ranked 2.5 or lower across all 8 digital
optimization categories, an improvement
compared to 58% in our 2022 survey. 40% of
airlines ranked 3.0 or higher on average; this share
was 31% in 2022. 8 airlines were classi ed as
Leaders, and these Leaders averaged 4.1 across
our 8 CRO categories.
58%
of airlines have a digital optimization team in
place.
of airlines have a documented and structured
conversion optimization process (a big
improvement, as this share was only 23% in 2022).
58%
of airlines claim that their internet booking engine
(IBE) is not exible enough to allow
experimentation.
of airlines said investment in the internet booking
engine (upgrade, optimization or simpli cation) is
one of their focus areas in 2023.
75%
15%
of airlines have dedicated budgets for CRO and
experimentation. For 56%, it’s part of the
marketing or ecommerce budget.
fi
fi
2023 Airline Digital Optimization Yearbook
fi
of airlines don’t run experiments at all or only do
so on an ad-hoc, project basis (on the other hand,
16% of airlines run at least 5-10 experiments per
month).
62%
45%
fl
56%
of airlines combine digital optimization and
personalization activities within the same team.
52% say they use a personalization engine or a
tool for personalization.
26
2023 AIRLINE DIGITAL OPTIMIZATION SURVEY –
AIRLINE DIGITAL OPTIMIZATION TEAMS
GROWTH
The COVID pandemic disrupted the airline industry,
and many had to downsize their digital
optimization teams: 37% of airlines in our 2022
survey said they had to decrease the size of their
digital teams.The good news is that for most
airline digital teams, the period of decline is over
and we’ve seen airlines starting to re-invest and refocus on their digital initiatives. 60% of airlines in
our 2023 survey said they increased their digital
teams during the past 12 months, and 22% even
claimed to have grown their digital optimization
teams signi cantly. During 2022, we saw an in ux
of new airline digital and specialized CRO roles on
our Diggintravel Airline Digital Job Board.
Re-hiring some of the lost digital optimization
roles is the rst step, but it’s not enough. The
pandemic caused a lot of uctuation among
digital leaders and other digital optimization roles,
and a lot of internal knowledge and knowhow was
lost. Our survey shows that the majority of airlines
now have digital optimization teams in place, but
there is still a gap when it comes to knowhow and
other key areas of the digital optimization process,
like user and UX research. And while the increase
in the share of airlines that have a documented
and structured conversion optimization process
(from 23% in the previous survey to 45% this year)
is encouraging, more than half of airlines still don’t
have one. Filling the skill gap and establishing a
process that connects digital product and digital
optimization teams should be the next step on
their experimentation and CRO journey.
Did you decrease or increase your digital optimization (product, analytics, UX) team during the
past 12 months?
We increased our team signi cantly
We increased our team slightly
The size of our digital team stayed the same
Untitled 1
We had to do a slight reduction in our team because of COVID
Untitled 2
We had to decrease our team signi cantly because of COVID
Untitled 3
0%
6%
11%
17%
29%
34%
40%
27
fl
fi
fl
fi
fi
fi
2023 Airline Digital Optimization Yearbook
23%
It all started
with an effortless
shopping experience
What if Airline Booking Flows were
transparent and self-explanatory?
And what if consumers would feel
like being taken care of while travelling?
Visit branchspace.com
to learn more
28
Smarter Travel Marketing
2019
TITLE OF DIGITAL
SURVEY
2023 SURVEY
DETAILS
Airline industry insights for a higher conversion
Month 2019 | © Diggintravel
Smarter Travel Marketing
I.
2019
T IT L E O F D I G ITA L
SURVEY
PEOPLE
Airline industry insights for a higher conversion
Month 2019 | © Diggintravel
WHO PERFORMS DIGITAL OPTIMIZATION (CRO) TASKS IN YOUR
ORGANIZATION?
•
•
•
•
Level 1: Online marketing specialist
Level 2: Ad-hoc or part-time conversion optimization resource
Level 3: Full-time conversion optimization resource (CRO specialist)
Level 4: Small digital optimization team (product owner, web analyst, designer, developer, CRO
specialist)
• Level 5: Full-scale digital optimization team (in addition to the small team - UX researcher, copywriter,
front-end developer, back-end developer, other experts), or several digital optimization teams
40%
33%
27%
20%
13%
7%
0%
Level 1
Level 2
Level 3
By Size
70%
40%
56%
30%
42%
20%
28%
10%
14%
Level 1
Small
Level 5
By Type
50%
0%
Level 4
Level 2
Level 3
Medium
2023 Airline Digital Optimization Yearbook
Level 4
Level 5
Large
0%
Level 1 Level 2 Level 3 Level 4 Level 5
LCC
FSC
31
KEY STAT: AIRLINE DIGITAL OPTIMIZATION TEAMS ARE
GROWING
The airline industry has come a long way when it
comes to digital maturity and digital optimization
teams. When we started with our benchmarks in
2017, only a good third of airlines had digital
optimization teams in place; now, 58% of airlines
have a dedicated conversion optimization team.
With airlines re-investing in and focusing on digital
channels, it’s not surprising that in our survey,
more airlines than ever claimed to have a full-scale
digital optimization team or even several digital
teams with CRO as their core focus (18% of
airlines were at Level 5).
When it comes to digital optimization teams, there
is a big difference between LCC airlines (78% of
them having dedicated teams) and FSC airlines
(only 49% have dedicated digital optimization
teams).
While most airlines recognize digital optimization
is not a job for one person, there is still a
signi cant share (31%) of airlines that don’t have a
specialized and dedicated CRO resource in place.
This share jumps to 51% for traditional FSC
airlines, as many of them still don’t understand the
value of a structured digital optimization process.
Having a dedicated CRO person or a program
manager is the rst critical step airlines need to
take if they want to build an optimization program.
The next step is setting up a team that will prepare
and execute the conversion rate optimization
(CRO) roadmap. Most airlines that don’t have a
CRO team in place claim lack of budget, digital
maturity, knowledge, or talent as the key obstacles
to doing so. Identifying rst optimization
scenarios, getting early wins and communicating
the ROI of increasing conversion rates should help
airlines get (budget) support for their optimization
and experimentation programs. Establishing a
small, centralized optimization team (UX
You can read how Air Europa’s digital team
evolved through different stages of our maturity
model, from a single CRO enthusiast to a full-scale
digital team that runs CRO activities, in our “Ask
the Expert” section.
Most digitally mature airlines understand the
importance of digital optimization. All 8 Leaders
from our research have dedicated optimization
teams. Furthermore, 4 out of 8 Leaders claimed to
have a full-scale team (Level 5) or several digital
product and optimization teams.
With the growth of airline digital teams (we see
more and more airlines getting several digital
product teams in place), determining the best way
to structure and organize your team becomes
more important. Many airlines are trying to gure
out the optimal way to embed optimization and
CRO in their digital teams and processes. As you
progress in your CRO maturity (from Level 3 to
Levels 4 and 5), choosing which organizational
model works best for your airline can be
challenging. There is no right answer; some
airlines have CRO as a separate team, others have
CRO embedded in their product teams. What
matters is that you nd the model that empowers
your digital product teams to leverage CRO and
experimentation best practices in their product
lifecycle without too much hassle and
bureaucracy.
32
fi
fi
fi
fi
fi
2023 Airline Digital Optimization Yearbook
researcher / designer, front-end developer) is
crucial for a successful program. Having
dedicated developer resources for
experimentation also has a huge impact on the
number of experiments you’ll be able to run (see
Section V).
You can learn more about different organizational
models, from centralized and hybrid to completely
decentralized models, in our “Ask the Expert”
section with digital expert and former Digital
Analytics and Optimization Manager at WestJet
Airlines Matt Ravlich.
During our interviews and airline CRO work last
year, we saw an increased trend of more digitally
mature airlines working on hybrid or matrix models
that allow different teams to optimize and
experiment on different digital products in parallel
and, more importantly, to deploy new products and
features much faster.
fi
2023 Airline Digital Optimization Yearbook
You can learn how American Airlines transitioned
to a decentralized model that enabled them to
deploy new features within a couple of days in our
“Ask the Expert” section.
You can also nd an example of a CRO role
embedded in the matrix Tribes and Chapters
organizational model that Air New Zealand
implemented in the “Examples” of Section VIII.
33
KEY TRENDS: WHAT
CHANGED COMPARED
TO 2022
Airlines who claimed to have a dedicated
optimization team have them set up in different
ways; 69% of airlines who have a CRO team said
their team is a combination of in-house and
outsourced resources. The rest (31%) claimed
they run digital optimization and testing
(experimentation) completely in-house. None of
the surveyed airlines claimed to outsource their
testing and optimization completely to an external
agency/service provider.
50%
40%
30%
20%
10%
Level 1
Level 2
Level 3
2023
Level 4
Level 5
2022
When it comes to the setup of digital optimization
teams, we didn’t see a big shift compared to last
year’s survey. The share of airlines that have a
digital optimization or CRO team in place (Level 4
or Level 5) stayed the same at 58%, and so did the
share of airlines where CRO is still performed by
an individual (Level 1 to Level 3).
By comparing the 2023 and previous survey
results, you can see that the biggest jump
happened in the Level 5 category: 18% of the
surveyed airlines claimed to have a full-scale
digital optimization team or several digital
optimization teams, a 6% increase compared to
2021-22. In our intro section, you’ve seen that 60%
of airlines increased their digital optimization
teams last year, which implies that airlines who
already have teams in place are either investing in
building several digital product teams or are
expanding existing digital teams with specialist
roles. During the pandemic, many specialist roles
were lost due to downsizing, but we can now see
airlines recruiting for specialized digital
optimization roles (e.g., analytics, digital research,
CRO specialists, and program managers) again.
We run optimization and testing (experimentation) both
in-house and together with an agency/service provider
We run optimization and testing (experimentation) both in-house and together with an agency/service provider
We run optimization and testing (experimentation)
completely in-house
We run optimization and testing (experimentation) completely in-house
0%
2023 Airline Digital Optimization Yearbook
10%
20%
30%
40%
50%
60%
70%
34
fi
fi
0%
Experimentation and testing require a speci c
skillset, so it makes sense for airlines who are
starting to build their CRO program to outsource
some of the roles. But regardless of how you plan
to build your digital optimization team (in-house,
outsourced, or hybrid), you need to have a digital
and ecommerce strategy that recognizes
conversion optimization as a strategic pillar. You
need to map strategic assets (knowhow, process,
experimentation learnings, user research data)
and see what you can outsource. You can
outsource specialized skills (e.g., advanced digital
analytics, front- or back-end development, UX
research, etc.); however, the strategic part
(planning, managing, goals) of the testing and
optimization process needs to be in-house. You
need to de ne clear metrics, focus on things that
have a direct impact on the bottom line, and be
able to explain the value of increasing conversion
rates to your management to get support.
ASK THE EXPERT:
AN AIRLINE’S JOURNEY: FROM ONE-MAN BAND
TO A FULL-SCALE DIGITAL OPTIMIZATION AND
CRO TEAM
Ismael Monzon
Digital Growth Team Manager at Air
Europa
NOTE: All Ask the Experts interviews are excerpts from
interviews with airline digital leaders on the Diggintravel
Podcast. If you want to learn more, you can nd the full
interviews by listening to the Diggintravel Podcast or
visiting the Diggintravel blog.
Ismael’s journey started in 2017 when he joined Air
Europa as a digital analyst, and he’s been working
on their digital optimization maturity ever since.
“I joined the company in 2017. I joined it as Senior
Digital Analyst, as a member of the web team of
Air Europa. It was a small team, but the board
directors at that period wanted to increase the
digital channel sales. I would de ne this period like
one-man band. I was the only person who was
working on the analytics and CRO tasks, and I had
a lot of work. There were multiple funnels to track
and optimize. I remember that I had to improve the
technical implementation; for example, I
remember that the Enhanced Ecommerce was not
con gured."
Once they set up the analytics basics, Ismael
started to work on other parts of the CRO process.
Because they didn’t have in-house resources, they
worked with a specialized agency to get their rst
CRO activities going.
things to do. At the end of 2018, I started to share
our CRO project with Fusion which was a CRO
consultancy company that was specialized in
CRO for airlines.”
At that time, Air Europa was moving from Level 2
to Level 3 in our Airline Digital Optimization
Maturity Model, which means they had a
dedicated person, but not a team, to manage
conversion optimization, digital analytics, and UX
research. After building the foundations with
analytics and measurement, Ismael started to
work on implementation of other CRO tools in
addition to digital analytics with Google Analytics.
They built a small team and started with customer
experience analytics.
“In 2019, we moved to the next step. We created
the digital analytics team. Two people joined it.
We wanted to move faster, doing more things, so
these two people helped me to go faster with all
our tasks. As I told you before, our platform was
complicated. There were too many issues, too
many bugs. We started to work with Tealeaf and
we got results quickly. We started to detect bugs,
we started to review sessions, identify usability
problems."
Then Air Europa went through a turbulent time
during the COVID crisis, during which they
developed a new digital platform. With the
rebound and recovery of the airline industry, Air
Europa started to invest in their digital and CRO
team again.
“The initial plan was very basic. There wasn’t user
research or session replay. A lot of tasks, a lot of
2023 Airline Digital Optimization Yearbook
fi
fi
fi
fi
35
ASK THE EXPERT:
The airline saw a shift from customers using
traditional o ine channels to using their direct
channels and decided to take advantage of this
opportunity.
digital analytics, content. More or less in the last
eight months of 2022, there were 12 new people
in the team.
“In 2021, the company improved its sales through
the direct channel. The directors wanted to keep it
and improve the sales. I think that is our reality. A
lot of customers have moved from physical
purchases to digital purchases, not only in airlines
but for a lot of business. So the directors of the
company wanted to go farther with the digital
sales, and we had a new site and app, but we
wanted farther.
Last month we have been working on de ning the
processes, the methodology all worked together
on how to integrate everything, the new people
with the current processes. We split the team into
two areas. One of them is more related with
product management; it is the digital product
area. The other one is the digital growth. The rst
one, the digital product team focuses on pulling
the platform with the product management,
working with It and providers such as Amadeus
or our IT team. And the digital growth team
focuses on selling."
So we started to plan for the digital channel and
took advantage of this new platform. The
approach was that the team had new members in
all areas – in product management, UX designers,
2023 Airline Digital Optimization Yearbook
fi
fi
ffl
36
ASK THE EXPERT:
AIRLINE DIGITAL OPTIMIZATION MATURITY:
HOW TO TAKE THE NEXT STEP?
Matt Ravlich
Digital Expert and former Digital
Analytics and Optimization Manager at
WestJet Airlines
NOTE: The process of organizing digital teams as your
optimization program grows and matures is a
fascinating topic, and it’s one we’ve explored in detail in
the past; if you want to learn more about various
experimentation organizational models on a broader
company level, listen to our interview with
experimentation leader Stefan Thomke on the
Diggintravel Podcast.
Matt Ravlich developed his own optimization
maturity model, so we were curious to hear his
view on the best way to start the airline digital
optimization maturity journey.
“There are 5 steps in the model. I think it’s
important too that sometimes it’s not really a
linear process as well. It’s like you can move from
different stages back and forth depending on your
needs. First off is enterprising, I think this is more
when you onboard the tools for testing and
personalization. I’m very familiar with Adobe
Target, and that’s what we used at WestJet, and
that’s what I’m using with a lot of our clients today,
which has the testing capabilities but also
personalization. But the rst step in the
optimization model is enterprising, and this is
where you have maybe one individual who’s part of
a digital team or part of a content team who starts
working on testing."
fi
2023 Airline Digital Optimization Yearbook
The role Matt described is something we’ve called
“digital optimization enthusiast” or “digital
optimization champion” in the past in our models.
"That’s a great name for it, actually. Very
enthusiastic about testing, sees the potential. I
feel like I was this person when I was at WestJet.
[laughs] This was my introduction to getting into
data and testing. I started working and doing a bit
of this and building out that program. I saw the
potential and got very excited and bought in really
easily, and then I tried to tell everyone, “Hey, this
works.”. You can start doing a few tests here and
there. My suggestion is always to look for lowhanging fruit with colors, copy, things like that.
Easy things you can easily change."
Most airlines move through different maturity
phases (for example, from a centralized team to
models like LATAM Airlines that have several
digital teams working on optimization). Looking
back through their journey at WestJet, I was
curious about some of the key learnings for Matt
and his team.
I became the team lead for optimization, and then
I really started to dig into why – what I noticed was
the team was doing some great testing and having
some great results and giving people reports, but
nothing was being actioned. Changes weren’t
taking place on the site. It was obvious that there
were clear winners that weren’t being accepted. I
was curious why this was because it had been
proven that certain things generated more revenue
in certain places, or a better experience, but a lot
of times there was testing results that were being
ignored."
37
ASK THE EXPERT:
"What I realized was with the centralized model, it
was really hard for one team to go in to another
team – say a product team that’s very focused on
their roadmap – and try to say, “Hey, you need to
do this” or put tickets on their roadmap and say,
“This is important” without the buy-in. It was like
you threw things over the fence and just hoped
that people would take it.
What I really think is essential is getting the buy-in
of the product teams that you’re working with and
the things that you’re testing to understand the
testing process and understand what goes into it,
and then get them excited about wanting to test
things and wanting to run experiments and make
that really good user experience. I think ultimately,
their goal is to make the best experience for their
guests, and testing is essential to doing that.
So getting into their sprint cycle or their process
and building testing into that process is key, and
that’s something that we really try to do at
WestJet. That’s where you start to move into the
hybrid model and move up the maturity, when you
get testing right into that sprint."
I like being a loyal customer,
but Customer Retention
requires an extra mile.
What if a digital commerce platform would enable
Customer Retention end-to-end, from earn to burn
and integrated into all digital touchpoints?
Visit branchspace.com to learn more
2023 Airline Digital Optimization Yearbook
38
ASK THE EXPERT:
TRANSITION FROM DISTRIBUTED TO DEDICATED
AIRLINE DIGITAL PRODUCT TEAMS
Mariana Fonseca Medina
VP of Digital, Loyalty and Marketing at
Virgin Voyages, prior Managing
Director – Digital Customer Experience
at American Airlines
During her time as a Managing Director of Digital
Customer Experience at American Airlines,
Mariana was involved in a big digital
transformation project, part of which was focused
on how the American Airlines digital teams were
organized.
“I was very fortunate that during my time at
American, we went through a massive digital
transformation, trying to move away from
distributed teams – teams that basically took care
of – call it a platform or a type of technology.
There was someone that did the common
services, and there was a team that did website
stuff and app stuff, more around the platform. We
actually transitioned to product teams that were
more aligned to OKRs."
Mariana emphasized the importance of having
proper metrics if you want “productization” to
work:
"All of these learnings around how to set proper
OKRs and establish teams actually came from that
long and di cult time, but ultimately, in my
opinion, one of the most successful
implementations of productization that I’ve seen.
It’s a good hybrid for a super large company of
having product teams – that is, teams whose only
responsibility is to be the most effective at
Mariana mentioned that having a dedicated
backlog is crucial. If you want to make your digital
product-focused organization work, having a
dedicated backlog for each of the products is key
for building and optimizing several digital products
at the same time.
Mariana shared an example of the American
Airlines seat map digital product, which was their
pilot case for the new way of working and
organization.
"I became the team lead for optimization, and then
It started with pilots that we did maybe four or ve
years ago around very small products just to see
how it would work. We took a team that would
only work on the seat map, which sounds like a lot,
but if you think of companies like Booking.com,
they have full teams just working on one widget
within the home page. So a team working on the
seat map wasn’t so bad.
So you had a full team, you had dedicated product
designers or UX resources, you had dedicated
engineering resources, and then obviously you had
a dedicated product owner, and they had a
backlog. They said, ‘Our number one goal is to
drive attachment, more people selecting seats.’
Then they did the proper amount of research and
understanding what the customer wanted and
where the gaps were."
39
fi
ffi
2023 Airline Digital Optimization Yearbook
shopping and buying, for example – with
technology teams focused on supporting
shopping and buying for them, where they can
have a dedicated backlog and teams and they can
become really fast and nimble in delivering."
ASK THE EXPERT:
Once they enabled a dedicated product team to
decide and work on their own, Mariana and the
team saw a big improvement in the amount of
time needed to deploy new product features.
"That team was able to drive their own features.
They didn’t have to check with anybody else. They
prioritized the highest things that they thought
were going to contribute to value, and then they
would deploy when ready. They actually did it with
this concept of scheduled weekly or monthly
releases, which was great because every time
they learned something new – let’s say we have
to have a different disclosure for families, another
big pain point. Families want to sit together. They
would be able to develop it and deploy it within a
couple of days. Coming from an airline that took a
really long to do anything, establishing that was
quite incredible."
2023 Airline Digital Optimization Yearbook
When you transition to a decentralized model, the
obvious question is: How do you manage to
ensure that all product teams are aligned?
"This gets asked a lot, like people are going to go
rogue and build whatever they want. I’m like, isn’t
that what leadership is here for, ultimately? We
are the checks and balances. If teams do go
rogue, then shame on leadership for not sharing
what the larger corporate objectives are so that
they understand they need to support those – so
they’re not going rogue; they should be building
their OKRs in support of large corporate
objectives.
You do have to create some central structures to
support decentralization of product teams and
encourage a lot more collaboration than before in
a way that’s organic and not forced. But I think all
of that is well worth the effort because at the end
of the day, you end up having a much better way
of working."
40
II.
Smarter Travel Marketing
2019
T IT L E O F D I G ITA L
SURVEY
SKILLS AND
KNOWLEDGE
Airline industry insights for a higher conversion
Month 2019 | © Diggintravel
Q: WHAT IS THE CURRENT LEVEL OF DIGITAL OPTIMIZATION
SKILLS IN YOUR ORGANIZATION?
• Level 1: Basic online and digital marketing knowledge
• Level 2: Basic conversion optimization, UX and analytics knowledge
• Level 3: Deeper knowledge about conversion optimization: CRO, UX, analytics, A/B testing, content &
copywriting
• Level 4: Advanced CRO knowledge (Level 3 + UX excellence, user behavioral knowledge, advanced
analytics including segmentation)
• Level 5: Experts in conversion optimization (Level 4 + conversion-centered design, analytics experts,
testing automation, personalization)
40%
33%
27%
20%
13%
7%
0%
Level 1
Level 2
Level 3
Level 4
By Size
By Type
40%
50%
32%
40%
24%
30%
16%
20%
8%
10%
0%
Level 1
Small
Level 5
Level 2
Level 3
Medium
2023 Airline Digital Optimization Yearbook
Level 4
Level 5
Large
0%
Level 1 Level 2 Level 3 Level 4 Level 5
LCC
FSC
42
KEY STAT: ADVANCED DIGITAL OPTIMIZATION SKILLS
ARE GAINING TRACTION
A signi cant portion (36%) of the airlines in our
survey still don’t have the in-depth knowledge and
skills required to do ongoing and systematic
digital optimization. These airlines said they have
basic online marketing knowledge or basic
conversion optimization knowledge (Level 1 or
Level 2). Most of the small airlines (50%) are in
this group.
But on the other side of the spectrum, the share of
airlines that claim to have advanced CRO and
digital optimization skills within their
organizations (Level 4 or 5) is growing and has
reached 27%. As in most categories, low-cost
airlines are also more advanced when it comes to
advanced digital optimization skills, as 39% have
deeper CRO knowledge and skills (at least Level 4)
within their digital teams. Only 21% of traditional
full-scheduled carriers claimed to have these
skills. We can also see that advanced skills are
mostly the domain of large airlines, where teams
are bigger and there are more specialized roles.
The biggest group when it comes to skills is still in
the middle; 36% of airlines claim to have deeper
knowledge of conversion optimization (Level 3)
but do not have advanced skills within their
teams. This means that most airlines are
recognizing the importance of digital optimization
and CRO skills for their digital teams but don’t
have the size or the resources yet to acquire highly
specialized skillsets. Smaller airline digital teams
can ll skills gaps (e.g., advanced analytics,
specialized tools, UX research, etc.) with external
help via specialized agencies or freelance
specialists. You can nd some recommendations
on that in our “Ask the Expert” section.
increasing conversion in an ongoing and
systematic way (at least Level 3). When we looked
at other categories (e.g., user & UX research
activities, test quantity) and asked airlines about
the reason they don’t do more CRO activities, lack
of knowledge and skills was stated as one of the
main reasons. Acquiring the right skillset and
knowhow is the rst step, and we see more and
more airlines recognizing this.
Conversion optimization requires people with
different roles and skills to work as a team with a
joint goal and to follow a structured process. It is
a continuous process of analyzing data
(analytics), understanding user behavior (user and
UX research), optimization, and testing. You need
to connect all your digital experts (analytics,
product, user research, CRO) and follow a
systematic CRO process. By implementing a
structured conversion optimization process and
agile CRO teams, airlines can connect these
different roles and skillsets, which will result in
higher conversion rates and a better digital
experience. The good news is that airlines are
starting to recognize this, as almost half of the
surveyed airlines (45%) said their digital teams
follow a structured process for digital
optimization; 40% of airlines said they have a CRO
process in place, but it is not documented or
structured. Only 15% of airlines said they don’t
have a CRO process in place.
Over the last two years, we’ve seen several airlines
systematically add CRO elements to various roles
in their digital teams, from digital analytics and
front-end development to customer insights. As
you can see from our survey results, low-cost
airlines are at the forefront of this trend.
It is an encouraging sign that now almost twothirds of the surveyed airlines claim to have the
skills required for digital optimization and
fi
fi
fi
fi
2023 Airline Digital Optimization Yearbook
43
In our “Examples” section, you can see a job
description for the CRO Program Manager role
at easyJet, along with other digital roles with
embedded CRO elements, to learn how easyJet
has built a full-scale CRO program.
Does your team follow a process for digital optimization (CRO)?
Yes, it is documented/structured.
Yes, but it isn't documented/structured.
No.
Untitled 1
0%
7%
14%
2023 Airline Digital Optimization Yearbook
21%
29%
36%
43%
50%
44
KEY TRENDS: WHAT
CHANGED COMPARED
TO 2022
40%
32%
24%
16%
8%
0%
Level 1
Level 2
Level 3
2023
Level 4
Level 5
2022
“Skills and knowledge” is the category where we
saw the biggest drop in our 2021-22 survey.
Because of the COVID crisis, many airlines had
to downsize their digital teams, and we saw a
shift from Level 4 back to Level 2 CRO skills. A
lot of specialized knowledge and skills were lost
during the turbulent pandemic times.
process in this year’s survey – a big improvement
from only 29% in 2021-22.
Another thing we’re seeing lately, especially with
airlines that are focused heavily on ancillary
revenue, is the trend of merging marketing,
ecommerce, ancillary revenue, and revenue
management into a joint digital revenue process.
Most advanced airlines (low-cost airlines are at
the forefront here) are expanding CRO and
experimentation concepts beyond the core digital
product.
You can nd examples of ancillary revenue roles
with CRO and experimentation elements in our
“Examples” section. In the “Ask the Expert”
section, you can learn how one of the most
successful airlines when it comes to generating
ancillary revenue, Frontier Airlines, is leveraging
experimentation with their pricing models – and
why making small, iterative tweaks based on user
feedback can have a really big impact when it
comes to generating ancillary revenue and your
digital products.
Similarly to the “People” category, we’re seeing a
rebound when it comes to skills and knowledge.
Our survey shows that the biggest jumps
happened in advanced skills (Levels 4 and 5), as
27% of airlines claimed to have such skills within
their departments, while this share was only 21%
in 2021-22. An additional signal that airlines are
re-investing in their digital teams is that we’re
seeing more airline job opportunities for
specialized CRO and digital optimization posted
on our Airline Digital Job Board. Advanced skills
are mostly the domain of large airlines.
The other big jump, one that is more signi cant,
happened around airlines having a CRO process
in place: 45% of airlines claimed to follow a
structured digital and conversion optimization
45
fi
fi
2023 Airline Digital Optimization Yearbook
ASK THE EXPERT:
HOW TO FILL THE SKILL GAP
Ismael Monzon
Digital Growth Team Manager at Air
Europa
One question we often get from airline people who
are going down the path of digital growth, growing
the digital team from a small team to a bigger one,
is: How do you decide which skills to get in your
team and what to outsource?
“I always think the same. It depends quite a lot on
the company and the mentality of the company. In
Air Europa, we have a lot of strong [external]
partners, but it’s more related with the tools. For
example, we work with AB Tasty or we work with
Quantum metrics."
Because they have expertise for their tools and the
processes covered by their tools?
"It’s not only the tool. That is right, and they are the
experts and they help us with the con guration or
with some adjustments, but the main thing is that
we work together. They provide us help with the
tool and we know what we want to achieve.
Working together is ne because working with
different perspectives, different minds, different
experiences is a perfect balance.
But it’s also true that, at least in my experience
with some agencies, it’s a bit complicated because
at the end, you need something quite fast. I know
the agencies are quite exible, but it’s true that it
sometimes changes quite a lot every day, and you
need to move faster. You have a quick weekly
meeting and you have some work, but in the
middle of the week the priorities have changed.
fi
fl
fi
fi
2023 Airline Digital Optimization Yearbook
So sometimes I think it’s a bit complicated to work
with external providers."
You need to be very aligned and really
communicate the strategy for this part for it to
work. Then the agency can basically work as fast
as you want, right?
"Sometimes we needed something that we have
experienced in the past. For example, for the SEO
or CRO area. I think, at least in my experience,
something that works quite well is working with a
freelancer. Maybe they can work together with the
team with constant communication."
And bring new knowledge?
"Yeah, because freelancers or people who are very
focused on several parts, I think they can help us
to solve very speci c or very high-level issues [that
require specialized knowledge]. It’s something that
we have worked in the past, and I think it has
worked.
In the past we thought it’s better to work with a
very highly recognized agency. We started with a
lot of ideas with the top consultants, but at the
end, after a year, that member moved to another
team and it’s quite complicated. A lot of priorities,
a lot of clients. You are not the big latest client.
The big agency moves resources to another latest
client. So sometimes it’s complicated, and I prefer
a closer relationship, people who work together
[as a part of your team]. You work always with the
same people. It’s not a project manager that you
don’t know who is working after him or her. That’s
my opinion."
46
ASK THE EXPERT:
AIRLINE ANCILLARY REVENUE LEADERS MERGE
MARKETING, ECOMMERCE, MERCHANDISING,
AND PRICING WITH CRO AND
EXPERIMENTATION
Jake Maloney
Director Ecommerce and Digital
Products at Frontier Airlines
Frontier is an ecommerce powerhouse. In your
most recent Q1 earnings release, the company
announced that it is selling an average of $69.28 in
ancillary products per passenger with the goal of
potentially $70 this calendar quarter. That’s an
enormous amount of optional products. How has
Frontier gotten to the point where you’re so
successful in attaching so much to the shopping
cart beyond just the base fare?
“We are indeed one of the most successful in
selling ancillary and bringing in that revenue.
There’s a lot of moving parts to that, but I think a
big part of that is really thinking through how we
merchandise things and when we merchandise
things, but also pricing variations and really
intelligently thinking through how pricing affects
that attachment and the propensity of the
customer to ultimately convert. So again, there’s a
lot of moving parts to that, but it’s primarily pricing
strategy and merchandising decisions."
Are you able to target customers with certain types
of products? Are you able to get personalized
offers or personalized targeting with the ancillary
products that you sell to customers?
"We aren’t doing that today from a product
perspective. I think that alludes to the magic of the
fl
2023 Airline Digital Optimization Yearbook
customer experience that we have today and the
pricing structure that we utilize. Because we’re
not able to test those things, we are actually
making some pretty big movements just from
minor user experience tweaks and pricing
strategy. We do have that on our roadmap, but
again, I think it really boils down to we do utilize
dynamic pricing and cohort analysis, so a lot of
that is one-to-one and one-to-many based."
Is experimentation a big part of this, trying to
learn about what works and what doesn’t?
“Absolutely. Those decisions are made multiple
times a day, actually. There are pricing models
and variations that are constantly being iterated
through."
Is it fair to say that product ideas come from
your colleagues who have more of a marketing
background or marketing focus, rather than
strictly ecommerce?
“Somewhat. We work really, really closely with
the pricing and revenue management team, who
own the ancillary side of things from a product
and pricing perspective. With that said, you’ve
got to have that constant contact and
collaboration between them when it comes to
how you display, how and when you offer those
things, and at what part of the ow. So there’s
constant collaboration between those two
teams."
47
EXAMPLES OF CRO SKILLS IN ACTION
Here you can see an example of
different digital optimization roles
that show how easyJet is building
their CRO program and
embedding different CRO
elements in various digital roles.
2023 Airline Digital Optimization Yearbook
48
EXAMPLES OF CRO SKILLS IN ACTION
Ryanair, Volotea and airBaltic are examples of
airlines merging ancillary revenue and
merchandising roles with CRO elements (A/B
testing, behavior analytics, targeting, data
science)..
2023 Airline Digital Optimization Yearbook
49
III.
Smarter Travel Marketing
2019
T IT L E O F D I G ITA L
SURVEY
USER AND UX
RESEARCH
Airline industry insights for a higher conversion
Month 2019 | © Diggintravel
Q: WHAT USER AND UX RESEARCH ACTIVITIES DO YOU
CURRENTLY DO?
• Level 1: Basic analytics and conversion reports; sales reports
• Level 2: Level 1 + advanced analytics reports, session recordings, simple A/B testing, heat maps and
click maps, ad-hoc customer surveys
• Level 3: Level 2 + regular customer feedback and survey analysis, form analysis, occasional
unmoderated or moderated user testing
• Level 4: Level 3 + scheduled moderated and unmoderated user testing, customer struggle scores,
prototype testing
• Level 5: Level 4 + user testing at scale, biometric research, anomaly detection leveraging AI/ML
30%
25%
20%
15%
10%
5%
0%
Level 1
Level 2
Level 3
Level 4
By Type
By Size
40%
50%
32%
40%
24%
30%
16%
20%
8%
10%
0%
Level 5
0%
Level 1
Small
Level 2
Level 3
Medium
2023 Airline Digital Optimization Yearbook
Level 4
Level 5
Large
Level 1 Level 2 Level 3 Level 4 Level 5
LCC
FSC
51
KEY STAT: USER AND UX RESEARCH REMAINS ONE OF
THE MOST NEGLECTED AREAS OF DIGITAL
OPTIMIZATION
We’re keeping the same title and key stat we used
in our last yearbook, and it’s intentional. We’ll keep
repeating this message until we manage to push
user and UX research higher on airlines’ agendas.
50% of airlines from our survey still don’t do any
user research activities besides looking at data
(Level 1), or they do simple activities to
understand their customers (Level 2). These
activities provide only quantitative data, and no
real qualitative insights to understand the context
behind the numbers.
Only 20% of airlines claimed they do systematic
moderated and unmoderated user testing
activities (Level 4), which are recognized among
the most valuable UX research activities by many
digital leaders. Test quantity and user and UX
research were where airlines scored lowest on
average across the eight categories from our
digital optimization framework, so these two areas
remain they key areas with room for improvement.
Both are crucial for a successful CRO program.
Traditional FSC airlines really struggle with a usercentric digital approach (this is what user and UX
research is all about). Almost a third (32%) of FSC
airlines said basic analytics and reports (Level 1)
are their only methods of user and UX research,
and 75% of small airlines don’t do any qualitative
user and UX research.
Currently, the majority of the airlines do either
advanced analytics reports, session recordings,
simple A/B testing, heat maps and click maps, and
ad-hoc customer surveys (25% of airlines are at
Level 2), or regular customer feedback and survey
analysis, form analysis, and occasional
unmoderated or moderated user testing (29% are
at Level 3) in addition to that. However, when it
comes to unmoderated and moderated user
2023 Airline Digital Optimization Yearbook
testing, these are still mostly not part of the user
research, or they are only done occasionally
(although there are some positive changes, as you
will see in the next section). Only 20% of airlines
do regular, scheduled moderated and
unmoderated user testing, which are probably the
most important user and UX research activities.
As in other areas, we see bigger airlines in
particular re-investing in digital teams, which
includes user research and specialized UX
research roles and skills.
You can see how British Airways is growing their
research team and how research is collaborating
with analytics and digital optimization in the
“Examples” section.
When we asked airline digital optimization
professionals in which user research activity they
see the most value, we saw some positive
changes compared to our 2021-22 survey.
There isn’t much change at the top: Digital
analytics (76%), A/B testing (56%), and website
and customer polls (51%) are still the most
popular user and UX research activities. However,
the positive change is that it seems more airlines
are starting to recognize the importance of
moderated user testing (a jump from 15% to 29%)
and unmoderated user testing (an increase from
10% to 20%). Recognition of both areas doubled
compared to our last survey.
In general, there is a positive shift across all user
research activities: airline digital optimization
experts are relying less on quantitative (data)
activities as their only insight into customer
behavior, as we see more traction for qualitative
user and UX research methods.
52
We also see increased adoption of digital
experience analytics, which is a mix between
quantitative and qualitative research. Digital
experience analytics combines both analytics and
UX research and can help airlines implement the
metrics that track and measure relevant data
about the customer journey, with the ultimate goal
being to eliminate friction in the booking funnel,
identify customer experience gaps, and increase
conversion rates.
You can nd an example of the Digital Customer
Analyst role at easyJet with CRO elements and
various digital experience analytics tools in the
“Examples” section.
What user research activities help you to understand your users the most? Please select the top 3
that bring you the most valuable insights:
Digital Analytics
A/B testing
Website and customer polls
Untitled 1
NPS surveys
Untitled 2
Face-to-face user interviews
Untitled 3
Click maps/scroll maps/mouse-hover maps
Untitled 4
Moderated user testing
Untitled 9
Unmoderated user testing
Untitled
Customer struggle scores
Untitled 5
Eye tracking
Untitled 6
Form analysis
Untitled 7
0%
13%
26%
39%
2023
fi
2023 Airline Digital Optimization Yearbook
51%
64%
77%
90%
2022
53
KEY TRENDS: WHAT
CHANGED COMPARED
TO 2022
40%
32%
24%
16%
It’s really important for airlines to go beyond data.
When it comes to a culture of testing and
experimentation, the really successful companies
don't rely only on data and A/B testing. They also
always use qualitative methods, and the user
research and user voice have equal say along with
the data. Watching real people use your product
and go through your booking funnel can help you
tell really powerful stories not only with data, but
also by building empathy with the end user, by
showing quotes or even short videos. This is what
we encourage companies to do after user testing:
to take the videos of the users, make a short video
of the most powerful “aha!” moments, and share
them with the key stakeholders to get support for
your initiatives.
8%
0%
Level 1
Level 2
Level 3
2023
Level 4
Level 5
2022
As in the prior two categories, we see a positive
trend and move to the right in our maturity
model in the UX and user research area. In our
2022 survey, we found that airlines had taken a
step back, from Level 3 to Level 2 or Level 1.
This was problematic, as it meant even more
airlines were relying on data and analytics only
to understand their users’ behavior. In our
2022-23 data, we’ve seen a shift from Levels 2
and 3 to Levels 3 and 4. The biggest increase
happened in Level 4, which means more airlines
are doing scheduled moderated and
unmoderated user testing, combined with other
activities like customer struggle scores and
prototype testing. Large airlines have the most
resources to do more UX research activities
(33% of large airlines are at Level 4), but others
can do them using agile methods and tools or by
calling on external resources to ll the
knowledge gaps.
fi
2023 Airline Digital Optimization Yearbook
In the “Ask the Expert” section, you can learn how
airline digital giant Southwest is using user
research methods like moderated and
unmoderated user testing at scale (they ran
almost 300 usability tests on thousands of people
to get their feedback) to build great digital
products.
54
ASK THE EXPERT:
HOW AIRLINE DIGITAL LEADERS ARE DOING
USER RESEARCH AT SCALE
Mark Hursh
Senior Director of Digital Customer
Experience at Southwest Airlines
We talk about how a data-driven, more modern
approach to managing digital product is to
measure, optimize, analyze, and do user research.
You told me before that you’ve run a lot of usability
tests and a lot of A/B tests over the last two years.
Have you experienced at some point that you
tested some of these ideas and they didn’t work?
“I try to maximize and our team tries to maximize
all of our available ways that we can solve
problems. Think of it like we have top-down
initiatives where we as a Southwest family need to
all come together to support the greater good, but
then we reserve capacity and hold it for product
optimization at a feature level. We make sure we
have enough that we can respond to the market
and do ongoing prioritization. That’s not on an
annual basis, but more quick to market so we can
shift and dodge as appropriate.
Then we even have a more detailed level of
optimization that is down at a product level. We
can do targeting and personalization, depending
on the frontend’s capability set, at a very quick
pace. We might be running tests when you’re
going through Southwest.com or the app, and
you’ll see a different journey experience if you
have a certain product on your pro le or not. We
try to adjust as appropriate.
fi
fl
fl
2023 Airline Digital Optimization Yearbook
Down at that lower level where we’re doing actual
product optimization, we have teams that are
coming up with hypotheses, looking at the
clickstream data, looking at customer activity,
looking at customer feedback, and saying,
“Wouldn’t it be good if we go test this?” Even down
to the nuance of the impact of a color. Color has
really big impact, and if you use it in the right
spots, it can have a big deal. We changed a
component in one of our ows and we saw a 0.4%
increase in the ow’s completion rate. Just by
changing a – I mean, it was literally a 5x5 pixel
area on the screen, and changing it from one hue
to another. So those have huge impacts.
We try to do that as much as we can, and then on
the usability side, before we even get to that mode,
we have a great usability practice that a team
member on our team brought to us a few years
ago. We’ve just kept building it up since then. But
we can go out and run a test on a problem area
either at the micro or at the macro level very
quickly and go out to a sample of a really large
billion-person-plus audience and ask them to try
out our new experiences."
You’re talking about unmoderated remote usability
tests, like user testing.
"Yeah, we’ve done both. We can do moderated and
unmoderated user testing in those situations, but
COVID actually forced us into that. We used to
bring people onsite, do galvanic skin response,
hook them up to systems, have a webcam that’s
monitoring to see if they’re smirking."
55
ASK THE EXPERT:
“I had no idea how many different forms faces
could be and all the different things that show if
you’re pensive or concerned or elated. Laura
brought all that to the table.
But since we’ve shifted to that more internetbased approach, accessing our larger audience –
Laura gave me a rundown yesterday; I think last
year we ran almost 300 usability tests on
thousands of people to get their feedback. Which
is so much fun because of our brand. We actually
have a halo effect to the point where we have to
take off our names and change the colors,
because otherwise people see it and they’re like,
“Oh, that’s Southwest! I like Southwest,” and you
go, “Ugh, that just tainted the experiment. Now
I’ve got to go adjust it.” [laughs] But we still love
hearing it. It’s just a funny side effect.
Can you share how large your user testing groups
are? How many people work for you in those
areas?
"We’re scaling up right now. It’s a small but mighty
team of a handful of folks. But the fun thing that
we learned has been we’re a big company. There’s
big company stuff that comes with that. We’ve
learned, though, that there’s a lot of people at the
company that like doing that. We’re working on
evangelizing that even more. And our designers
may not be user researchers from a classically
trained perspective, but we nd that they love
doing that in many cases. So we’ve worked to
empower them. It might be a small handful of
folks that have the actual title, but we have a lot of
folks that play in that space, so we like to work
with them."
We call it “always be testing,” which is a lot of fun
to get into that learning and analytical,
hypothesis-driven approach."
fi
2023 Airline Digital Optimization Yearbook
56
EXAMPLES OF USER AND UX RESEARCH IN
ACTION
Here is an example of how
British Airway is investing in
their internal UX Research
team and how the Digital
Product Researcher role works
hand in hand with the
Analytics, Optimization, and
Customer Experience teams by
providing quantitative and
qualitative insights.
2023 Airline Digital Optimization Yearbook
57
EXAMPLES OF USER AND UX RESEARCH IN
ACTION
Here is an example of the
Digital Customer Analyst role
at easyJet with CRO elements
and various digital experience
analytics tools.
2023 Airline Digital Optimization Yearbook
58
IV.
Smarter Travel Marketing
2019
T IT L E O F D I G ITA L
SURVEY
DIGITAL
A N A LY T I C S
Airline industry insights for a higher conversion
Month 2019 | © Diggintravel
Q: WHAT KIND OF DIGITAL & CRO ANALYTICS DO YOU DO?
• Level 1: Basic web analytics (e.g. standard Google Analytics reports)
• Level 2: Level 1 + advanced web analytics setup (e.g. enhanced ecommerce for GA, custom goals,
events, metrics and simple funnel reports)
• Level 3: Level 2 + detailed funnel analytics, attribution modeling, cross-device tracking
• Level 4: Level 3 + advanced tools for visualization (like Google Data Studio & BigQuery, Tableau, Qlik,
Power BI)
• Level 5: Level 4 + data science (predictive analytics, machine learning with e.g. R or Pyhton), anomaly
detection leveraging AI/ML
40%
33%
27%
20%
13%
7%
0%
Level 1
Level 2
Level 3
Level 4
By Size
By Type
50%
50%
40%
40%
30%
30%
20%
20%
10%
10%
0%
Level 1
Small
Level 5
Level 2
Level 3
Medium
2023 Airline Digital Optimization Yearbook
Level 4
Level 5
Large
0%
Level 1 Level 2 Level 3 Level 4 Level 5
LCC
FSC
60
fi
KEY STAT: DIGITAL ANALYTICS REMAINS ONE OF THE
MORE MATURE DIGITAL OPTIMIZATION AREAS
When it comes to digital analytics, there weren’t
big changes (see more details in the next section),
and this is good news for airlines. Digital analytics
remained among the most developed digital
optimization areas, as 62% of airlines from our
survey claimed they do detailed digital analytics in
their primary web analytics tool (Level 3) or, even
more, that they use advanced visualization tools
and reports in addition to their main web analytics
platforms (Level 4 or Level 5). The largest group
(35%) of airlines claimed to be at Level 4, which
means they are doing advanced digital analytics in
their primary digital analytics platform and they
are using BI or other platforms for aggregation
and exploration. 50% of LCC airlines are at
advanced maturity of digital analytics (Level 4 or
5), compared to only 30% for FSC.
On the other end of the spectrum, there are still
13% of airlines that rely on basic web analytics
reports (Level 1). 25% of airlines have Enhanced
Ecommerce tracking features enabled in their
analytics setup (Level 2), but they still don’t do
detailed tracking of their booking funnel. This
share jumps to 43% for traditional, full-scheduled
airlines. Detailed funnel tracking or attribution
modeling are two key activities to understand
how your digital campaigns and booking funnel
are performing. Exact and detailed measuring
should be the starting point for all of your digital
optimization and CRO activities. It’s also an
activity that doesn’t require a big additional
investment, as it can be done in most of the
existing digital analytics platforms. Google
Analytics is still the most popular digital analytics
platform among the surveyed airlines; 56% of
airlines said they use Google Analytics, and
another 33% claimed to use its enterprise version
– Google Analytics 360. Adobe Analytics is the
other platform used by airlines for digital
analytics (15% of airlines in our survey). Google’s
solution for mobile analytics, Firebase, is the
most popular solution for mobile and app digital
analytics. There were also several airlines in our
survey that claimed they are in the process of
migrating to GA4 (Google Analytics 4). You can
nd two different airline digital analytics roles
based on different digital marketing stacks (one
for Google, one for Adobe) in our “Examples”
section.
What is your primary web analytics platform (e.g. Google Analytics, GA360, Adobe Analytics /
Omniture, Firebase, Monetate ....)?
Google Analytics
GA360
Firebase
Untitled 1
Adobe Analytics
Untitled 2
Other
Untitled 3
0%
9%
17%
26%
2023
2023 Airline Digital Optimization Yearbook
34%
43%
51%
60%
2022
61
KEY TRENDS: WHAT
CHANGED COMPARED
TO 2022
40%
science role with A/B testing elements in our
“Examples” section.
You can learn more about how to leverage data
and marketing tools to build audiences and
provide personalized experiences in the “Ask the
Expert” interview in Section VI – Tools.
32%
24%
16%
8%
0%
Level 1
Level 2
Level 3
2023
Level 4
Level 5
The other challenge when it comes to using data
and leveraging analytics is that sometimes there
can be too much data and metrics. Digital
departments and teams use different tools and
can track and analyze data in different tools. This
siloed approach when it comes to data and
metrics can result in different teams optimizing
different parts of the customer journey, or different
parts of the digital product, and using different
metrics, instead of having uni ed goals.
2022
As we mentioned at the beginning of this section,
we didn’t see signi cant change when it comes to
airline digital analytics maturity. The level of
maturity and the usage of digital analytics
platforms are almost at the same levels as in our
2022 survey.
In the “Ask the Expert” section, you can read about
two traps to avoid when it comes to digital
product metrics, and why it is important to nd a
balance between business and product metrics.
The number of airlines at Level 4 increased slightly
(35% vs. 33% in our last survey), but only one
airline (2%) stated that they’re leveraging data
science (Level 5) when it comes to digital
optimization and CRO. With democratization of
data science, advanced techniques like predicting
conversion probability, anomaly detection, churn
prediction, recommendation engines, and building
various segments and clusters based on your
digital analytics data are becoming more
accessible. There is an untapped opportunity, the
next level for airlines to reach when it comes to
using digital analytics data: going from reactive to
proactive analytics when it comes to digital
optimization of their digital products. Combining
CRO elements (experimentation, user research)
with marketing data science could be the ultimate
step towards smart, data-driven digital marketing.
You can nd an example of a marketing data
62
fi
fi
fi
fi
2023 Airline Digital Optimization Yearbook
ASK THE EXPERT:
TWO TRAPS TO AVOID WITH DIGITAL PRODUCT
METRICS
Mariana Fonseca Medina
VP of Digital, Loyalty and Marketing at
Virgin Voyages, prior Managing
Director – Digital Customer Experience
at American Airlines
What would you say is most important when you’re
setting up a metric system (KPIs) or an OKR
system or any measurement system for digital
teams and digital products?
"I would say you have to measure what you can
in uence. This is where sometimes, digital OKRs
and business level objectives are not 100%
aligned. I think every CEO would like to say, “I have
a revenue goal of doubling bookings, and I need
your OKR to be doubling bookings.” Well, the
people responsible for the checkout page cannot
in uence doubling bookings. They can in uence
doubling the throughput of people paying on that
page or reducing the number of errors on that
page.
So it’s nding that sweet spot and creating
objectives and key results of things that you can
in uence that correlate or tie into the broader
business objectives, but not falling in the trap of
two things.
One is adopting those business objectives as your
digital OKRs, because you’re never going to be
able to measure success. If I commit to doubling
bookings, then all I can do is, I don’t know, change
the forms, improve the UX, remove some error
rates. I’m not going to be able to accomplish my
The other thing that you should avoid is the
opposite: having objectives and key results of
things you can measure, but that actually don’t
contribute at all to the revenue targets, or to
whatever the overall corporate targets are. If I have
a goal of decreasing time on task, but time on task
doesn’t necessarily mean more revenue, better
conversion, or higher attachment, then I am not
contributing to that corporate goal.
That for me is the biggest learning. It’s tough
because purist digital product managers want that
metric that they can tackle, and they may not be
seeing the corporate goals. And then business
leaders who are not digital native only want the
business goal, but they don’t understand that the
digital product teams can only in uence so much.
It’s nding that negotiation and balance between
the two."
t be measurable in the short term (the duration
of an experiment) yet believed to causally drive
long-term strategic objectives […] The hard part
is nding metrics measurable in a short period,
sensitive enough to show differences, and that
are predictive of long-term goals. For example,
‘Pro t’ is not a good OEC, as short-term
theatrics (e.g., raising prices) can increase
short-term pro t, but may hurt it in the long run.
Customer lifetime value is a strategically
powerful OEC.
63
fl
fl
fi
fi
fi
fi
fl
fl
fl
fi
2023 Airline Digital Optimization Yearbook
goal. A goal that cannot be accomplished should
not really be a goal.
EXAMPLES OF DIGITAL ANALYTICS ROLES IN
ACTION
Here you can see two
examples of airline digital
analytics roles on two different
analytics platforms: one from
LOT Airlines on Adobe
Analytics and a digital
marketing tech stack (Adobe
Analytics, Adobe Target, Adobe
Campaign), and a CRO & Data
Analyst role for Corendon
Airlines on Google digital
marketing platforms (Google
Analytics, Google Tag
Manager, Google Optimize).
2023 Airline Digital Optimization Yearbook
64
EXAMPLES OF MARKETING DATA SCIENCE
ROLE
Here you can see an
example of a
marketing data
science role with CRO
elements like
A/B testing.
2023 Airline Digital Optimization Yearbook
65
V.
Smarter Travel Marketing
2019
T IT L E O F D I G ITA L
SURVEY
TEST QUANTITY
Airline industry insights for a higher conversion
Month 2019 | © Diggintravel
Q: HOW MANY TESTS AND EXPERIMENTS DO YOU DO?
•
•
•
•
•
Level 1: None or minimal: 1-2 tests per quarter
Level 2: Ad-hoc testing or project based: 1-2 tests per month
Level 3: Regular and planned testing: 2-5 tests per month
Level 4: Interactive testing: 5-10 tests per month
Level 5: Disciplined testing: 10+ tests per month
40%
33%
27%
20%
13%
7%
0%
Level 1
Level 2
Level 3
Level 4
By Size
By Type
50%
50%
40%
40%
30%
30%
20%
20%
10%
10%
0%
Level 1
Small
Level 5
Level 2
Level 3
Medium
2023 Airline Digital Optimization Yearbook
Level 4
Level 5
Large
0%
Level 1 Level 2 Level 3 Level 4 Level 5
LCC
FSC
67
KEY STAT: TESTING AND EXPERIMENTATION ARE
FINALLY GAINING TRACTION, ESPECIALLY WITH LOWCOST AIRLINES
Testing and experimentation volume still remains
one of the biggest pain points and the lowest
ranked digital optimization area, but we’re nally
seeing some positive progress.
56% of airlines don’t test (20% are at Level 1) or do
it only on an ad-hoc basis (36% are at Level 2).
Testing quantity is especially problematic for
traditional (FSC) airlines, as 27% don’t test or do
minimal testing (Level 1) or rarely run experiments
(32% of FSC airlines are at Level 2).
The positive news is that 44% of the airlines in our
survey are doing regular and planned testing
(Level 3 or above). Low-cost carriers are at the
forefront here; 50% claimed they do
experimentation at a respectable scale of 5-10
tests per month (Level 4).
Testing quantity is really where digital optimization
leaders distinguish themselves from the rest, and
low-cost airlines made big strides in this area
during the last year. Most low-cost airlines
increased their experimentation velocity to 5-10
tests per month. Per one of our Yearbook
contributors, velocity and quality are two of the
biggest levers in experimentation, and you should
be constantly exploring your limits of both. As you
start running more experiments and increase test
quantity, you learn and optimize your testing
process, which enables you to run more complex
tests – and run them more e ciently. Another
Yearbook contributor and past podcast guest
Mark Hush from Southwest Airlines disclosed that
their digital team ran more than 130 A/B tests and
more than 240 usability tests in the span of one
year. In our “Examples” section, you’ll see that
another low-cost airline powerhouse, easyJet, is
The key for airlines is to leverage experimentation
to de-risk ideas and reduce the risk of running into
identifying product issues late in the development
process. As you can see from the Forrester
1:10:100 rule, the cost of identifying problems
increases exponentially over time.
68
fi
fi
ffi
fl
2023 Airline Digital Optimization Yearbook
running up to 10 tests a month within their CRO
program.
Another shift we’re seeing is that many airlines are
starting to move CRO and experimentation beyond
the domain of specialists and into the hands of
product owners. While democratizing
experimentation can increase velocity, airlines are
facing new challenges, as most people are new to
testing. Orchestration, education, and guidelines
for teams from different channels running
experiments is crucial in this process. Over the
past year, during our research we’ve seen
examples of airlines working on an internal CRO
and experimentation centralized support function
to educate digital product owners and provide
guidelines on how to embed experimentation in
their digital product work ow. You can nd an
example of this type of centralized role at Qatar
Airways in our “Examples” section.
Experimentation and testing can be utilized both
as risk mitigation as well as exploration. You can
get ideas for new products or features from past
experiments, and 36% of airlines claim to do so.
As you can see from our data, airlines get the
most ideas from web analytics, but data is not
enough. The key to high-quality, high-impact A/B
tests and experiments is doing conversion
research leveraging both data and user and UX
research.
Where do test and experiment ideas come from at your organization? Please select Top 3!
Web analytics
Customer journey analysis and customer struggle
Ideas from the executive team
Untitled 1
Insights from previous experiments
Untitled 2
Customer data (CRM, personas, user surveys)
Untitled 3
Competitive analysis
Untitled 4
Online resources about other experiments or web design & UX best practices
Untitled 9
Gut feeling
Untitled
Heat maps, click maps, session recordings, eye tracking
Untitled 5
Usability labs
Untitled 6
0%
10%
20%
30%
40%
50%
60%
70%
2023
fi
2023 Airline Digital Optimization Yearbook
fi
fi
See our “Ask the Expert” section for insights from
two CRO experts on how the eld has changed
over the last ve years and how companies are
nally embracing the concept of iterating on a
product quickly by identifying the key issues early
in the process and based on customer feedback.
69
KEY TRENDS: WHAT
CHANGED COMPARED
TO 2022
50%
40%
30%
20%
10%
0%
Level 1
Level 2
Level 3
2023
Level 4
Level 5
2022
If we compare our 2023 results to results
from our 2022 survey, we can see that the
biggest shift happened from Level 2 (1-2
tests per month) to Level 3 (2-5 tests per
month). This is a welcome sight, and it
probably means that most airlines that had to
scale down their experimentation volume
because of the pandemic are now reinvesting in digital products and running
more experiments. The shift from ad-hoc,
project-based testing to regular testing is the
crucial step airlines need to take towards
data-driven digital marketing.
But while we see new airlines making this
progress, more than half still don’t do regular
testing and experimentation. Lack of
resources remained the main reason airlines
gave as to why they’re not running more
experiments.
What prevents you from running more A/B tests and experiments?
Lack of resources & budget
Platforms complexity & limitations
Other priorities & time constraints (a lot of other initiatives)
Untitled 1
Lack of skills & knowledge
Untitled 2
Lack of testing tools
Untitled 3
Organization
Untitled 4
Lack of experimentation culture
2022 Airline Digital Optimization Yearbook
Untitled 6
0%
6%
11%
2023 Airline Digital Optimization Yearbook
17%
23%
29%
34%
40%
70
We’ve seen that as CRO programs mature, airlines
want to increase the number of tests and run more
complex experiments. This is where core digital
platform limitations can impact the quantity and
complexity of your tests. In exible digital
platforms and booking engines that don’t support
experimentation have been a common theme in
our CRO surveys over the last ve years.
requirement list when airlines are implementing
new or upgrading existing digital platforms.
As you will see in our next two sections, you
cannot run a competent CRO program without
proper testing tools.
This year, almost two-thirds of airlines claimed
that they will invest in their core digital platforms
in the next 12 months, yet only a third will invest in
A/B testing capabilities (see Section VIII for more
data). This makes us believe that A/B testing
capabilities are still not at the top of the
It all started
with a simple and
engaging mobile app
What if engaging Mobile Apps came off-the-shelf to
be smoothly integrated with your digital commerce
platform, just requiring easy activation?
Visit branchspace.com to learn more
fi
fl
2023 Airline Digital Optimization Yearbook
71
ASK THE EXPERT:
HOW HAVE CRO AND EXPERIMENTATION
CHANGED OVER THE LAST FIVE YEARS?
NOTE: During our research for our 2023 Airline Digital Optimization Yearbook, we reached out to several thought leaders
in experimentation and CRO and asked them how they see the evolution of this eld. Here are some of their answers.
Rich Page
Conversion Optimization Expert
with more than 15 years of
experience
Lorenzo Carreri
CRO, User Research and
Experimentation Consultant
"The top two changes I’ve seen happening are:
One, collecting and acting on qualitative feedback.
Eric Ries released the book Lean Startup in 2011.
One of the many messages in the book was to
iterate on a product quickly based on customer
feedback. 13 years later there are still tons of
companies that don’t follow the advice of this
book. But things have gotten much better. Five
years ago my jaw would have dropped if a client
shared any customer research work they had
previously done before hiring us. Today it’s not a
rarity anymore. Which is a great advancement for
the industry.
"Over the last ve years, CRO has now become
much more about the quality of A/B tests than the
quantity, and doing conversion research to power
higher impact ideas. Understanding your user's
needs, issues, doubts, and hesitations through
user testing and surveys has become a
particularly important part of this conversion
research, instead of relying on analytics insights or
best guess."
Two, the need for one source of truth. Five years
ago the idea of having pre-purchase and postpurchase data together in one place was pretty
rare (unless you were a very large business). In the
past few years, I’ve seen more mid-size (and small
too) businesses invest in having one source of
truth (data warehouse) for the entire customer
journey. This allows them to understand their
acquisition and retention better, and also make
more con dent decisions when evaluating the
results of an experiment."
72
fi
fi
fi
2023 Airline Digital Optimization Yearbook
EXAMPLE OF CRO AT SCALE
Here is an example of a Front-End Developer role at easyJet, leading the CRO program by launching up to 10
tests per month of different scopes (small, medium, large).
2023 Airline Digital Optimization Yearbook
73
EXAMPLE OF CRO ROLE SUPPORTING DIGITAL
PRODUCT MANAGERS
Here is an example of a Web Optimization Specialist role at Qatar Airways; this role, together with the Testing and
Optimization O cer role, provides guidelines and support for digital product owners to run tests.
ffi
2023 Airline Digital Optimization Yearbook
74
Smarter Travel Marketing
VI.
2019
T IT L E O F D I G ITA L
SURVEY
TOOLS
Airline industry insights for a higher conversion
Month 2019 | © Diggintravel
Q: WHAT TOOLS DO YOU USE FOR YOUR DIGITAL OPTIMIZATION
ACTIVITIES?
• Level 1: Basic web analytics platform, performance monitoring
• Level 2: Level 1 + advanced web analytics; simple A/B testing tools (client-side); simple
user research tools
• Level 3: Level 2 + advanced customer research tools; customer experience analytics
tools; simple personalization tooling
• Level 4: Level 3 + advanced A/B testing tools (server-side) or own testing platform; project
management tool for optimization
• Level 5: Level 4 + own testing platform embedded in the core digital platform; advanced
personalization tools; predictive analytics & optimization tools
30%
25%
20%
15%
10%
5%
0%
Level 1
Level 2
Level 3
Level 4
By Size
By Type
60%
40%
48%
32%
36%
24%
24%
16%
12%
8%
0%
Level 1
Small
Level 5
Level 2
Level 3
Medium
2023 Airline Conversion Optimization Survey
Level 4
Level 5
Large
0%
Level 1 Level 2 Level 3 Level 4 Level 5
LCC
FSC
76
KEY STAT: COMPLEX OPTIMIZATION TOOLS ARE MOSTLY
THE DOMAIN OF LARGE AIRLINES
24% of the airlines in our survey claimed they
don’t use a tool for testing (Level 1) and that they
rely on a basic web analytics platform for their
digital optimization activities. Another 27%
claimed that they use simple A/B testing and user
research tools (Level 2). Most of the small airlines
don’t use digital optimization tools, as more than
half (58%) claim they only use a basic analytics
platform. 54% of traditional full-scheduled airlines
are at Level 1 or Level 2. On the other end of the
spectrum are 18% of airlines that use advanced
server-side A/B testing tools or their own testing
platforms (Level 4). 39% of LCC airlines use
advanced tools (Level 4) compared to 22% of FSC
airlines, another sign that low-cost airlines are
more mature when it comes to digital
optimization. Large airlines have more resources
and more complex technology stacks, and 72% of
large airlines are using advanced digital
optimization tools (Level 3 or Level 4).
Digital analytics and looking at past data is not
enough if you want to do systematic optimization.
Some airlines in our survey said they track monthon-month conversion rate improvements on a
granular level, and if the improvement is
signi cant, they attribute it to the change they’ve
made. However, there are many factors that can
make those assumptions wrong, from seasonality
and audiences to pricing and availability. If you
want to really measure the impact of your digital
product changes, you need to run experiments
and tests. You need to do systematic user
research and testing as well, and implementing at
least simple A/B testing, digital experience
analytics, and simple user research tools should
be the next step for all airlines that don’t use such
tools at the moment.
The following chart is a representation of the
most popular options among airlines when it
comes to A/B testing platforms.
Do you use A/B testing tools (Google Optimize, Adobe Tests, Optimizely, Oracle Maxymiser,
VWO...) or do you have your own testing platform?
Google Optimize
Adobe
We don't use a testing tool
Untitled 1
Own testing platform
Untitled 2
Others
Untitled 3
Optimizely
Untitled 4
Oracle Maxymiser
Untitled 5
2022 Airline Digital Optimization Yearbook
0%
6%
11%
17%
2023
fi
2023 Airline Digital Optimization Yearbook
23%
29%
34%
40%
2022
77
Most airlines are using third-party vendors’ A/B
testing tools. Google Optimize is still the most
popular testing tool among the airlines, followed
by Adobe testing tools. Oracle and Optimizely
were other tools that got several votes in our
survey. Google has announced (you can nd the
link in the “Resources” section) that Google
Optimize and Optimize 360 will no longer be
available after September 30, 2023, so this is
something airlines need to consider. Google
suggested that they are investing in A/B testing in
Google Analytics 4.
The share of airlines that said they have their own
testing platforms (or that they use a combination
of their own and a third-party platform) remained
the same at 16%. Airlines that have built their
own testing platforms can be more agile with
their experimentation initiatives. The share of
airlines that claimed they don’t use an A/B testing
tool dropped a bit to 20%.
When planning on building a digital optimization
and experimentation program, a testing platform
is one of the key elements you need to consider.
Do you use a vendor (third party) or do you build
your own testing capabilities? Do you consolidate
all digital optimization tools (analytics, digital
experience analytics, testing) under one vendor or
do you use many different ones?
Here you can see a grouping of various third-party
tools posted on LinkedIn by our Yearbook
contributor and experimentation expert Ben
Labay. Ben grouped the tools based on their
experience with working with them at Speero and
based on how these tools collect and assign data,
what metrics they can handle, and what types of
analysis they can do. Server-side tools or in-house
testing platforms allow more complex metrics
(like margin, average booking value, and ancillary
revenue) but are more complex to build,
implement, and manage. You can nd an example
of an A/B testing tool and other CRO tools used
by easyJet in our “Examples” section. There is
also a link to a CXL article with a list of
alternative A/B testing tools for Google Optimize
in our “Resources” section.
Source: Ben Labay, Speero
fi
fi
2023 Airline Digital Optimization Yearbook
78
KEY TRENDS: WHAT
CHANGED COMPARED
TO 2022
in large airline organizations, of insourcing these
skills into their digital teams. Some airlines that
started their testing initiatives with simple A/B
testing and digital optimization tools made the
next step by starting to use more advanced A/B
testing tools during the past 12 months.
30%
24%
18%
12%
6%
0%
Level 1
Level 2
Level 3
2023
Level 4
Level 5
2022
As in other areas, we’re seeing a trend when it
comes to digital optimization tools. We see
more advanced airlines moving from Level 3 to Level
4, meaning more airlines are using advanced testing
tools or are working on developing their own testing
platforms. The trend is logical if interpreted in the
context of others: with the industry recovering after
the pandemic and airlines investing back into their
digital teams and skills, there is a need for more
complex digital optimization tools. More complex
tools require more specialized skillsets and
knowledge, and therefore we see a trend, especially
Do you combine digital optimization and
personalization activities (within the same
team)?
The other trend we’re watching in our yearly
surveys is whether airlines use personalization
tools and if they combine CRO and personalization
activities within the same team. We’ve shared a lot
of resources and expert takes in our past
Yearbooks on why it makes sense to combine
personalization, digital optimization, and
experimentation. We see that airlines are
recognizing this as well; 75% of airlines in the
survey claimed to combine digital optimization
(CRO) and personalization activities within the
same team. This is a 6% increase compared to our
2022 survey.
To say that data and personalization have been
popular topics in the airline industry over the last
couple of years is an understatement. Our survey
con rms that personalization is at the top of
airline agendas, as 52% of the airlines in our
survey said they use a personalization engine.
Do you use a personalization engine?
No
No
Yes
Yes
0%
11%
23%
34%
46%
fi
2023 Airline Digital Optimization Yearbook
57%
69%
80%
0%
7%
15%
22%
30%
37%
45%
52%
79
This is a signi cant increase from 39% in 2022. In
addition, 60% of airlines listed personalization
and customer data (CRM, CDP, personalization
engines) as one of the focus areas of their digital
team in the next 12 months (see Section VIII for
more detailed results).
Successfully integrating data, data platforms,
personalization engines, and your marketing and
experimentation initiatives is de nitely
challenging. Running experiments based on
conversion research is the best way to
understand the impact and calculate the ROI of
personalization initiatives.
You can learn more about how to connect the
dots with data, digital marketing platforms,
personalization, and testing in our “Ask the
Expert” section.
fi
fi
2023 Airline Digital Optimization Yearbook
80
ASK THE EXPERT:
HOW HAVE CRO AND EXPERIMENTATION
CHANGED OVER THE LAST FIVE YEARS?
NOTE: During our research for our 2023 Airline Digital
Optimization Yearbook, we reached out to several thought
leaders in experimentation and CRO and asked them how
they see the evolution of this eld. Here are some of their
answers.
Ben Labay
CRO & Growth Expert and CEO at
Speero
"There are a couple primary ways that CRO is
evolving. One is quite tactical, and centers on data
security and privacy and site performance
standards. This area has CRO tackle attribution,
personalization, and data quite differently than
before. We're more prone to build and leverage
CDPs and data warehouses for our work, we're
more prone to focus on session-based behavioral
monitoring and changes, etc.
The second is more strategic and programmatic. It
has CRO evolving into business operations more
and more. It has the CRO way of thinking, that
failure is a feature and not a bug, operationalizing
into orgs more and more. Centers of excellence,
ambassadors, specialists in change management
related to experimentation combined with Agile
thinking and processes. CRO is borrowing from
the dev ops movement here in many ways."
fi
2023 Airline Digital Optimization Yearbook
81
HOW TO CONNECT THE DOTS WITH DATA,
PERSONALIZATION, AND TESTING
Matt Ravlich
Digital expert and former Digital
Analytics and Optimization Manager at
WestJet Airlines
You said optimization is a combination of
experimentation and creating personalized
customer experiences. In your mind, what kind of
tools do you need to run a proper optimization
program?
“I think it’s having a good understanding of the
data you have, and then a big thing is combining
that data with your other sources. In my previous
experience, there’s been siloed data in different
areas. I feel for a lot of the analysts sometimes
because I know that there’s a lot of stitching they
have to do, but then there’s also a lot of data
wrangling that has to go on around, “Where do I
get the data for this?” You have to go to different
teams, gure out where everything is.
When I think about airlines, there’s loyalty data,
there’s the digital data, there’s data coming in from
ights, there’s revenue management information.
One thing that we started doing is our teams
realigned and we started working heavily with our
revenue management data teams. We would
actually send them a daily data feed of our digital
data so they could start combining it into ight
search data to help them start to look at those
patterns and see where they could decipher what
people’s interests were and how they could better
work that into their pricing or offer discounts,
things like that.
One of the things I think really helps is if you do
have a centralized data repository. I don’t like to
say that because I know it’s di cult. It’s not an
easy thing to do.”
By centralized data deposit, you mean like a CDP
platform?
"Yeah, or I like to say data lake, but yeah, CDP is
key. From my experience working with Adobe
products, I’ve seen people start to use the AEP
product to centralize their data. There’s a schema
where APIs are used to get streaming data to be
pumped into the centralized data resource in AEP,
and then they can take advantage of what’s called
the Real-Time CDP in Adobe where basically the
pro les get built. You can actually get a real-time
pro le of your customers, and you start to see
what events are happening on your site, but also
what’s happening in other areas. Like, are they a
loyalty member? Are they not? Can we convert
them to a loyalty member?"
And then you build personalization scenarios on
top of those.
"Exactly. So there’s the promise of 1:1
personalization, which I think is eventually – it’s
tough. I think it’s a tough go, and to be honest, I
haven’t seen 1:1 really working anywhere that I’ve
been at fully. But what I do see working very well is
the segmentations that can happen through that
type of data.Within RT CDP, people start to quality
for certain segments. For example, say
somebody’s an anonymous user that comes to
your site; they do some actions, but then they ll
out a form and add their email.
fi
82
fl
ffi
fi
fi
2023 Airline Digital Optimization Yearbook
fi
fl
ASK THE EXPERT:
That starts to build their pro le. Then maybe that
email is already in your loyalty system. Then you
can stitch that data together and start to see their
larger picture, and maybe they start to qualify for a
loyalty tier, they’re a Bronze member, they’ve own
to a certain place using points – things like that
that they can start to qualify for. Or maybe they’ve
searched for a trip to the Bahamas; now you can
start to show them certain advertising for Bahama
ights. That information can be used onsite and
offsite through this RT CDP using destination,
such as Facebook, Google 360, things like that."
You were talking about data management
platforms, decision management software,
optimization tools. Would you say these are the
blocks (next to digital analytics, of course) – your
core analytics platform that helps you not only run
but also automate the optimization and
personalization process?
“Yeah. I think that’s where we’re going. A future
state is that I think most airlines or most
companies, just knowing that a lot of people are
using Adobe Experience Cloud, are probably going
to make the move to Adobe Experience Platform
at some point if they haven’t already, because they
can align their web data to be available, and then
also connect it through to Adobe Target.
Those pro les that I was talking about earlier tie
into Adobe Target and allow for more
personalization, and more testing as well. You can
do a lot of testing with your segments.
tracking certain things?” Say loyalty member
logins; that’s something I’ve talked a lot about.
Anonymous users, what people search for, the
tra c to your booking site, things like that.
Understanding the key KPIs is going to help you
and your program grow because I think once you
understand that and you’re tracking the right
things, you’re getting the right data, and then your
strategy will align and only help improve
everything.
I’ve been mentioning the Experience Platform a lot,
and people are moving in that direction with CDP.
But also, I think a good thing to note is taking
inventory of what you have today. Take a step back
and say, “Yes, we should probably start to look at
these new tools, but what can we do with what we
have today?”
If you’re using Adobe Analytics, if you’re using
Google Analytics, what data are you getting out of
that? Are you able to create audiences that you
can start to look at and break down and use for
your marketing data? Can you leverage Audience
Manager in a way today that helps you with your
offsite advertising and digital marketing? Can you
also personalize things based on the data you’re
receiving today?
We always look to what the latest and greatest is,
but also, I think we should look at, “What do we
have today and are we using it in the right way?
Can we do better with what we have?”
But I think it all comes down to, is your strategy
aligned with the business goals? You can have a
great marketing stack and tools, the best in the
world, but if you’re not aligned to your business
goals, then I don’t think your program is going to
work out as well. So understanding what you’re
tracking on your website and why, like “Why are we
83
fl
fi
fi
2023 Airline Digital Optimization Yearbook
ffi
fl
ASK THE EXPERT:
EXAMPLES OF CRO TOOLS IN ACTION
Here is an example CRO Manager role
at easyJet with CRO elements and A/B
testing (Google Optimize) and other
user tracking tools.
And here’s a similar example of a CRO
role at Wizz Air with key stakeholders
and main tools.
2023 Airline Digital Optimization Yearbook
84
VII.
Smarter Travel Marketing
2019
T IT L E O F D I G ITA L
SURVEY
INTERNET
BOOKING ENGINE
(IBE)
Airline industry insights for a higher conversion
Month 2019 | © Diggintravel
Q: HOW FLEXIBLE IS YOUR INTERNET BOOKING ENGINE (IBE)
FOR A/B TESTING AND OTHER EXPERIMENTS?
• Level 1: Our IBE doesn't support A/B testing and experiments
• Level 2: Simple A/B testing and experiments are possible but with workarounds and take a long time &
effort to implement
• Level 3: We can do basic and semi-complex experiments and A/B tests on an ongoing basis
• Level 4: We can do split-path A/B testing, have & test several versions of booking ow at the same time
• Level 5: We completely own and manage booking ow and have no limitations with A/B testing and
experiments
40%
33%
27%
20%
13%
7%
0%
Level 1
Level 2
Level 3
Level 4
By Size
By Type
60%
40%
48%
32%
36%
24%
24%
16%
12%
8%
0%
Level 1
Small
Level 5
Level 2
Level 3
Medium
Level 4
Level 5
Large
0%
Level 1 Level 2 Level 3 Level 4 Level 5
LCC
2023 Airline Conversion Optimization Survey
FSC
fl
fl
86
KEY STAT: BOOKING ENGINE FLEXIBILITY REMAINS A BIG
PAIN POINT FOR TRADITIONAL AIRLINES
58% of the airlines in our survey claimed they have
internet booking engine (IBE) solutions that are
not exible for A/B testing (Level 1 or Level 2).
35% from this group said their IBE solutions allow
them to run A/B tests, but it takes a lot of time and
effort to do it (Level 2), and 24% said their IBE
solution doesn’t support A/B testing at all (Level
1). On the other end of the spectrum, only 14% of
airline optimization professionals in our survey are
satis ed with the testing capabilities of their IBE
platforms (Level 4 or Level 5).
Even if their booking engines are not exible,
airlines can still run experiments on other
platforms (e.g., email campaigns, advertising
campaigns, copywriting, landing pages, etc.).
These are good places to start with simple tests
and to get your rst learnings. However, in the
long run, more complex experiments, so-called big
bets, are the ones that provide true value and
signi cant results.
In a recent thought-provoking article from
Reforge, the authors claimed that the tests
marketers are most comfortable with are simple
to run, most likely to “succeed,” but don’t actually
create much informational value. They’re
designed to generate small wins that will move
you up to “local optimum” within the framework of
what you’re already doing, and nothing more (a
small improvement but not a step-change winner).
Big bets (more complex tests in the airline world)
could be changing your customer acquisition
strategy, changing your product bundling and
ancillary strategy, or rolling out a different or
several booking funnels. These changes are not
possible without exible booking products.
As in prior years, internet booking engine exibility
is an area where airlines struggle the most overall
and where the gap between low-cost airlines
(LCC) and traditional, full-scheduled carriers (FSC)
is most notable, especially at the bottom.
65% of FSC airlines in our survey claimed to have
an internet booking engine that makes testing and
experimentation within their booking ow either
impossible or it takes a long time and effort to
implement it. Small airlines are in a tough spot
when it comes to internet booking engines; most
rely on third-party legacy IBE products that don’t
allow much control and exibility (83% of small
airlines are at Level 1 or Level 2).
Do you have a custom built IBE or do you use
an IBE product-solution?
Most popular IBE product solutions
Amadeus
IBE product solution
Navitaire
In-house or custom built solution
Sabre
Untitled 1
Mix (IBE product solution + custom development)
Others or not speci ed
Untitled 1
Untitled 2
0%
9%
17%
26%
34%
43%
60%
0%
7%
14%
21%
29%
36%
43%
50%
87
fl
fl
fl
fl
fi
fl
fi
fi
fi
fl
2023 Airline Digital Optimization Yearbook
51%
Only 14% of airlines in our survey said they can
run more complex tests like split-path A/B
testing, test several versions of booking ow at
the same time (Level 4), or even completely own
and manage booking ow and have no
limitations with A/B testing and experiments
(Level 5). However, all but one of the eight
Leaders in our survey claimed to have booking
engines that allow them to run complex
experiments. To be good at CRO, you need to
have a exible booking engine. Even great A/B
testing or other customer experience tools won’t
help much if your IBE doesn’t allow experiments
on your main digital product.
Because of the complex distribution and IT
landscape (and legacy solutions), FSC airlines’
internet booking engines are much less exible
than the ones LCC airlines are using. 74% of FSC
airlines in our survey disclosed that they use a
third-party IBE solution. The vast majority of
third-party IBE products come from traditional
big vendors like Amadeus and Sabre (see survey
results for most popular IBE product solutions).
These product solutions were often built as an
extension of the legacy PSS platforms, and as
such are not the best t for modern digital
optimization and A/B testing. Most airlines that
use IBE product solutions claim that complex
tests are either not possible or are time- and
cost-consuming due to IBE constraints. On the
other hand, 55% of LCC airlines claimed they
developed their own booking engines (most have
a custom-built IBE that is based on APIs from
the back-end and have full exibility with the
front-end), so they have much more exibility
with customization and running tests. Navitaire
is the most popular IBE platform among LCC
airlines; however, most LCC airlines use it only as
a base and have custom development on top.
engine, or even building a custom booking
engine with A/B testing functionalities, is the
other part. Build or buy is always a di cult
decision, and the answer depends on resources,
organization and priorities. The good news is
that in recent years, new modern airline digital
platform solutions have emerged that have A/B
testing and personalization functionalities
already embedded. Using such solutions, even
small airlines or airlines that don’t want to
develop solutions in-house can run tests and do
personalization within their booking ows.
You can nd an interview on the topic of
building agile airline digital platforms with A/B
testing functionalities in our “Ask the Expert”
section.
In the prior section (Section VI – Tools), you saw
why testing tools are the rst step and a very
important part of your digital optimization tech
setup. Integrating these tools with your booking
fl
fl
fl
ffi
fl
fl
fi
fl
fi
fi
fl
2023 Airline Digital Optimization Yearbook
88
KEY TRENDS: WHAT
CHANGED COMPARED
TO 2023
Almost all airlines (98%) do at least some kind of
CRO activities on their website. However, many
airlines still lack the resources or knowledge to
do digital optimization activities on their mobile
app. Only 46% of airline digital professionals in
our survey answered that they do digital
optimization activities on their airline app. While it
is still rare, there are some innovative airlines that
are expanding CRO and experimentation best
practices to other channels.
40%
32%
24%
You can learn about what Southwest Airlines’
digital team learned about running experiments
on their In ight Portal in our “Ask the Expert”
section.
16%
8%
0%
Level 1
Level 2
Level 3
2023
Level 4
Level 5
2022
Which of the following platforms do you
optimize (user research, optimization, A/B
tests)?
If we compare the 2023 results to our previous
survey, we can see there is a status quo when it
comes to internet booking engine exibility. A
similar share of airlines claims to have a booking
engine that is not exible for A/B testing (Level 1
or Level 2): 58% in this year’s survey, compared to
54% in 2022.
27% of airlines said they can do basic and semicomplex experiments and A/B tests on an
ongoing basis (Level 3), which is an 8% increase
compared to our previous survey. This is good
news, as it means more airlines can do
experimentation on their core digital products.
However, we see a slight regression when it
comes to running complex A/B tests within
booking ows: 14% of airlines are at Level 4 or
Level 5, compared to 27% the year before. As
mentioned in the prior section, large airlines
represent the vast majority of airlines with exible
booking engine platforms.
When it comes to digital touchpoints where
airlines do CRO and digital optimization, main
airline websites are still the predominant channel.
Mobile websites (mobile or responsive airline.com
Mobile apps
Untitled 1
Web check in, kiosks
Untitled 2
Other
Untitled 3
0%
14%
29%
43%
57%
71%
86%
100%
89
fl
fl
fl
fl
fl
2023 Airline Digital Optimization Yearbook
Desktop website (main ailrine.com website)
ASK THE EXPERT:
WHERE AND HOW TO START WHEN IT COMES TO
MODERN AIRLINE DIGITAL PLATFORMS AND
WHY EXPERIMENTATION SHOULD BE PART OF IT
Ursula Silling
CEO at Branchspace, prior Chief
Commercial Officer at Kenya
Airways, Commercial Director at
Air Malta, EVP Commercial at
Brussels Airlines and more
Most airlines are still tied to long RFP cycles when
it comes to implementing new IBE or digital
product solutions. Is there an alternative, more
agile way? Like doing a pilot on a smaller case
where you see if there is a t, or redesigning or
testing a new partner or platform on a smaller
scale before committing to a long project?
“I think getting to know each other rather than just
doing an RFP and then immediately going for the
marriage – it’s a very good point. Airlines often do
the opposite. We’ve got lots of examples where,
for example, we’ve been doing a digital review to
really understand and benchmark performance
and also very clearly help to nd quick wins to
improve in the current environment rather than just
looking for completely new solutions. That’s one
part. Also benchmarking touchpoints and
understanding the performance across
touchpoints, even also physical ones.
We also do UX and UI design as we realize how
important the design is for customer experience.
We’ve been doing this as well for a couple of
airlines where suddenly they get to appreciate our
innovative thinking, our approach, how thorough
we are when going through this and how we
fi
fi
fi
fi
2023 Airline Digital Optimization Yearbook
engage the whole organization and all of the key
stakeholders to really make this work whilst
bringing in our know-how and experience. And this
on an international basis and relevant for the
speci c environment of the airline.
I think such a project can actually be another good
starting point to create value and to get to know
each other. I compare it with the example where
you’ve got a shop and you don’t want to completely
renovate it immediately when you have in mind
some new customer experiences, maybe some new
products and how to best position them, and you
start by really doing a super well-designed shopping
window to get some more interest.
And suddenly you realize, “Hm, this works. Now I’ve
got the con dence to restructure the main part of
the shop, and maybe later on I will add other
elements on top of this. And I have found the right
partner to ensure it will become a success.”
The “shopping window” is a nice analogy. You could
even expand it by opening other shops, other
touchpoints?
"Exactly, open other shops, maybe add additional
possibilities such as how you can reach the shop,
maybe with some additional self-servicing outside
of hours via self servicing machines so you can still
buy your products. You can compare this in the
digital world with all of the e-servicing that you
would do for travel."
90
ASK THE EXPERT:
“Maybe you suddenly want to experiment to sell
completely different products, so you start your
innovation journey, and you get the con dence.
And you don’t only get the con dence in the
partner; you actually get the con dence for your
team to be able to do this, because this team
needs to grow as well. Sometimes maybe your
partner could move faster, but if the team doesn’t
start to get the con dence – and initially there is
often a certain fear with these changes – then
you should do the steps more slowly. If suddenly
they gain the con dence, “We can make this work,
now we can really do more, this is great,” suddenly
it gets the domino and it gets stronger and faster
together all the time."
Con dence for your digital team is a very
important but often neglected part of this.
Experimentation is another way for airline digital
teams to build con dence, as they can learn and
measure the impact of their changes. Why is A/B
testing as part of an airline IBE platform still such
a rare sight?
So suddenly, if I take this example of
experimentation and optimization, or A/B testing –
airlines have been hearing that this is what Expedia
has been doing and it works so well. This is how
they became so successful, so good at customer
experience. But airlines do not know how to get
there. We include this in the experience, in the
technology solution that we offer. It’s part of the
platform. It means it makes it easy to really start
and experiment and make it happen, and then
adjust to where you want to go. This is, I think,
another area where we help.
And we can support with this as well, setting it up
or managing it completely. We have been doing this
for a few airlines for a long time, as the augmented
team. Or we can do this to train, to help, to
introduce how to set this up. And this applies for a
number of different areas so that we can ensure for
us, we set out, we’re a partner. I think that’s maybe
the big difference for the success of tomorrow, not
just to implement a technology and run away, but
really to ensure that the business can succeed."
"One of the problems which I have seen over time
– because there is often not this a nity to
technology and to the latest developments in
digital within the airlines, there is also this – I
sometimes call it “technology spaghetti.” There is
not clarity. Where should I start? What do I really
need? What is this vendor promising, how many
other things do I still need to really make this
happen? How will I ever bring this together with
where I am?
With our solution, we’ve got an out-of-the-box
approach, if you want to. You can get all of the
essential parts. It’s very modular at the same
time. And you can also keep adding other
modules if you want to.
fi
ffi
fi
fi
fi
fi
fi
fi
2023 Airline Digital Optimization Yearbook
91
I can be all
different kinds
of a consumer.
A digital-web
strategy of an
airline should
reflect that.
What if dedicated Website Portals enabled a unified but personalised
experience, irrespective in which role a consumer appears?
Visit branchspace.com
to learn more
2023 Airline Digital Optimization Yearbook
92
ASK THE EXPERT:
EXPANDING CONVERSION RESEARCH AND
EXPERIMENTATION TO OTHER AIRLINE DIGITAL
TOUCHPOINTS
Mark Hursh
Senior Director of Digital Customer
Experience at Southwest Airlines
Most airlines apply this testing and user research
to their main channels (desktop, app, etc.). I
haven’t seen many do it on a kiosk level. I saw what
you do on the in- ight entertainment when you’re
testing. What was the learning experience there,
and the feedback? Does it work the same as on
your desktop?
“I’m fortunate that we have the in- ight portal
team on our digital experience team because
longer term, I think we can do so much in that
space. We have a great entertainment experience
onboard, and we have the ability for you to access
internet and free messaging and all that great
stuff. We made the entertainment fully free a few
years ago and saw great take rates.
But I also see a really big opportunity – and we
just need to spend more time in the space – to
help better prepare you for your connection. We
have connecting ight information, but we don’t
have the push noti cations, as an example. If you
have the app that’s telling you, “Hey, here’s what
you should expect when you land in Houston for
your next section of your journey. Here’s the gate
you’re going to arrive at. Here’s where you’re going
to go. Here’s how much time you have to get there,
and that’s enough time for you to go take your kids
to the bathroom or get folks fed,” it reduces that
level of journey anxiety that comes from traveling.
Travel is anxiety-prone. It’s a stressful experience.
fl
fl
fi
fl
fi
fl
fl
fl
fl
2023 Airline Digital Optimization Yearbook
So how do we take stress out of that for you? The
in- ight moment is a great moment to do that
while you’re sitting there. We actually see most
people, the most amount of time is actually spent
watching the ight map. People will sit up and put
up a movie and put it in picture-in-picture mode.”
Since we’re talking about the in- ight
entertainment portal and you brought up Wi-Fi, how
much control does Southwest have over that
design versus your Wi-Fi partner? Do they let you
do the frontend, or is there any kind of hand-off, if
you will, between the Southwest designed portion
and your partner?
"We work hand in hand with them on the portal
experience. On the portal experience we’ve had
some great success with our current partner. A
few years back, we did some research; we found
learnings that we had from other app experiences,
which was “Let’s put the key things in front of
people.” We had this giant hero shot when you
were on the in- ight portal, and then you had to
scroll down to nd the key things that you wanted
to go do, like watch a movie or what
entertainment. We all had a simple hypothesis,
which is “Let’s get rid of the hero shot and move all
that stuff up to the top.” Lo and behold, 25%
increase in take rates."
That’s an example of testing that we were talking
about.
93
ASK THE EXPERT:
“Yeah. Free messaging was up I think 20% or 30%.
Movies were up in similar fashion. We worked
hand in hand in the research element of that to
improve that experience. And all the providers
have a different type of structure; some of them
have out-of-the-box, some of them have fully
customizable types of things. We have been
fortunate in the area where we had that early
partnership in that space, so we’re highly
in uential in that. And we’re constantly testing in
that space. We just launched Venmo recently and
I was super surprised by the take rates of that
product. It’s doing great.
that space, and he came up with this great idea,
which is changing buttons around based on your
phase of journey. We saw increased take rates from
those as well. Your rst button you might see is free
movies and free live TV at the top, and then
throughout your journey, we’ll pop up a button when
you’re at a certain phase in the ight when drink
service is about to begin so you can have context
about your drink service and it’s handy. And then if
you’ve used other products and services, some of
those pieces go away and we pop something else
up there, just to make it much more contextual."
But there’s lots of little nuance that you may not
even know. Steven is our digital product owner in
fl
fi
fl
2023 Airline Digital Optimization Yearbook
94
VIII.
Smarter Travel Marketing
2019
T IT L E O F D I G ITA L
SURVEY
O R G A N I Z AT I O N A L
SUPPORT
Airline industry insights for a higher conversion
Month 2019 | © Diggintravel
Q: WHO IS A DIGITAL OPTIMIZATION AND EXPERIMENTATION
SPONSOR IN YOUR ORGANIZATION?
• Level 1: None – digital optimization & experimentation is recognized on an individual level
• Level 2: Head of Ecommerce – digital optimization & experimentation is recognized on a departmental
level
• Level 3: Director level – digital optimization & experimentation is recognized by director or higher
management
• Level 4: VP level – digital optimization & experimentation is recognized and supported by VP level
executive (top management support)
• Level 5: Entire organization – digital optimization & experimentation is recognized as a crucial activity
and has company-wide (C-level) support
40%
33%
27%
20%
13%
7%
0%
Level 1
Level 2
Level 3
Level 4
By Size
By Type
50%
50%
40%
40%
30%
30%
20%
20%
10%
10%
0%
Level 1
Small
Level 5
Level 2
Level 3
Medium
2023 Airline Conversion Optimization Survey
Level 4
Level 5
Large
0%
Level 1 Level 2 Level 3 Level 4 Level 5
LCC
FSC
96
KEY STAT: DIGITAL, OPTIMIZATION AND
EXPERIMENTATION ARE MOVING UP ON AIRLINES’
AGENDAS
While optimization and experimentation are
recognized and supported on a department level
for the biggest cluster of airlines in our survey
(35% are at Level 2), there is now a signi cant
share of airlines (38% are at Level 4 or Level 5) for
whom digital optimization and experimentation
have company-wide exposure and top
management support.
As in other categories, LCC airlines are at the
forefront when it comes to recognizing a datadriven approach to building great digital products.
55% of LCC airlines are at Level 4 or Level 5, which
means digital optimization and experimentation
are recognized on the top level. Partially this is
because of background and organizational
structures; direct digital channels are the
predominant (or even only) distribution channels
for LCC airlines, while FSC airlines were built on
indirect distribution and have shifted towards
direct channels in the last decade. LCCs (and
small airlines) also have a atter organization, so
digital leaders have more access to C-Level
executives.
But there is more to it than history; LCC airlines
recognize that digital needs to be embedded in the
company’s DNA if airlines want to compete with
other global travel and big tech giants. Most of the
modern digital roles and examples we’ve shared in
other sections of this Yearbook come from LCC
airlines. This is not a coincidence. Some of the
airlines from our examples have digital, data-driven
and experimentation-based decision-making in
their company vision and mission statements. You
can nd two such cases in our “Examples” section.
and less hierarchical organizational models that
are more suitable for building digital products. In
our “Examples” section, you’ll see that Air New
Zealand is working on a matrix Chapters and
Tribes model, where Data and CRO chapters are
supporting Brand, Marketing, Retail and UX tribes.
During our research and interviews with airline
digital leaders, we ran across other FSC airlines
that are working on establishing experimentation
support functions (Center of Excellence model) to
enable their digital product teams to experiment.
Another sign that CRO and experimentation are
gaining more exposure is a positive shift when it
comes to airline budgets. 50% of airlines in last
year’s survey said they didn’t have any budget for
CRO and experimentation. This share dropped to
only 30% this year. For most of the surveyed
airlines (56%), CRO is part of the marketing and
ecommerce budget, but 15% of airlines have a
dedicated budget for experimentation.
Data-driven decision-making should not be part of
digital optimization teams only. It shouldn’t be
seen as a growth hack or a way to trick the user
into purchasing a product. To achieve long-term
growth, airlines need to embrace optimization
and experimentation as a strategic initiative.
But while LCC airlines are at the forefront, several
traditional carriers are experimenting with modern
fi
fl
fi
2023 Airline Digital Optimization Yearbook
97
How are experiment results and learnings implemented and shared across the organization?
Shared to broader ecommerce & digital teams (or other commercial teams)
Actively spread throughout the organization (entire organization has access to
experiment results and stakeholders are proactively informed about all experiments)
Discussed exclusively within the optimization team
Untitled 1
Not shared across the organization
Untitled 2
0%
10%
20%
30%
40%
50%
60%
70%
51%
60%
Do digital optimization & experimentation have their own budget?
No, digital optimization & experimentation are not mentioned speci cally in the budget.
Yes, it's part of the marketing or e-commerce budget.
Yes, we have a dedicated budget for experimentation.
Untitled 1
0%
9%
17%
26%
34%
2023 Airline Digital Optimization Yearbook
43%
fi
98
KEY TRENDS: WHAT
CHANGED COMPARED
TO 2022
We hope that this Yearbook and our other
Diggintravel content can help airlines in adopting
a modern, data-driven digital mindset and will be
recognized as a valuable resource on their digital
transformation journeys.
40%
32%
24%
16%
8%
0%
Level 1
Level 2
Level 3
2023
Level 4
Level 5
2022
The shift from Level 3 to Level 4 that we’ve seen in
other areas of our maturity model is re ected in
organizational support. There was another notable
improvement: there are fewer airlines where
digital optimization and experimentation are
recognized only on an individual level (Level 1), as
it seems that some of those airlines have moved
to Level 2 of our maturity model. Recognition for
CRO and experimentation typically starts with an
enthusiast (you can nd an example of such a
journey in Section I); however, the key goal of the
internal CRO champion is to start embedding
experimentation in the broader digital teams and,
later on, throughout the organization.
In our previous Yearbook, we noted that some
airlines “used” the pandemic to focus on digital
and building their direct ecommerce channels.
This trend will continue in 2023, as many airlines
are re-investing in their digital teams and digital
platforms. Investment in mobile apps, booking
engines, personalization and data are at the top of
airlines’ agendas for 2023.
fl
fi
2023 Airline Digital Optimization Yearbook
99
What will be the focus area of your digital optimization team in the next 12 months? [Please
select up to 3]
Investment in mobile app
Investment in internet booking engine (upgrade, optimization or simpli cation)
Investment in personalization and customer data (CRM, CDP, personalization engines)
Untitled 1
Investment in data & analytics (e.g. migration to GA4, other analytics implementations)
Untitled 2
Investment in customer servicing (Manage My Booking, Check-in)
Untitled 3
Investment in core website (e.g. frontend, CMS, home page, key landing pages…)
Untitled 4
Investment in retailing and product enhancements
(e.g. retailing UX features, adding new ancillary products in the booking ow)
Untitled 9
Investment in experimentation and A/B testing platform
Untitled
Investment in customer noti cations and other post-booking customer engagement and targeting
Untitled 5
Investment in other digital touch-points (e.g. B2B or B2T portals)
Untitled 6
Others
Untitled 7
0%
10%
20%
30%
40%
50%
60%
70%
2023
fl
100
fi
fi
2023 Airline Digital Optimization Yearbook
EXAMPLES LOW-COST AIRLINES PROMOTING
DATA-DRIVEN AND EXPERIMENTATION
CULTURE
easyJet and Eurowings Digital are two lowcost airlines that have digital, data-driven and
experimentation-based decision-making in
their company vision and mission statements.
2023 Airline Digital Optimization Yearbook
101
EXAMPLES AIR NEW ZEALAND CHAPTERS AND
TRIBES MODEL
Air New Zealand is working on a matrix
Chapters and Tribes model, where Data
and CRO chapters are supporting Brand,
Marketing, Retail and UX tribes.
2023 Airline Digital Optimization Yearbook
102
REFERENCES & SOURCES
REFERENCED DIGGINTRAVEL RESEARCH AND ARTICLES:
Diggintravel 2022 Airline Digital Optimization survey
https://diggintravel.com/2022-airline-digital-optimization-yearbook/
Airline Scienti c Decision-Making [LATAM Airlines case – Part I]
https://diggintravel.com/airline-scienti c-decision-making-latam/
Step-by-step Evolution of Airline Digital Optimization and CRO Team
https://diggintravel.com/airline-digital-optimization-and-cro-team-air-europa/
Airline Digital Talks: Eveline Lee, Scoot
https://diggintravel.com/airline-digital-talks-eveline-lee-scoot/
How to Grow Your Airline Digital Product Team
https://diggintravel.com/airline-digital-product-teams/
How to Build a Modern Airline Digital Experience
https://diggintravel.com/how-to-build-a-modern-airline-digital-experience/
Airline Ancillary Revenue Leaders Talks [Frontier Airlines Case]
https://diggintravel.com/airline-ancillary-revenue-leaders-frontier-airlines-case/
Airline Digital Optimization Maturity: How to Take the Next Step
https://diggintravel.com/airline-digital-optimization-maturity/
How to Build Innovative Airline Digital Products [Southwest Airlines Case]
https://diggintravel.com/innovative-airline-digital-products-southwest-airlines-case/
REFERENCED ONLINE RESOURCES
Google Optimize Sunset
https://support.google.com/optimize/answer/13257663
Google is Shutting Down Optimize in 2023: 22 Alternatives for A/B Testing
https://cxl.com/blog/ab-testing-tools/
Some Testing is a Waste of Time: Making Business Cases for Big Bets
https://www.reforge.com/blog/roi-of-testing
39 Conversion Rate Optimization (CRO) Tools Used by Experts
https://measuremindsgroup.com/cro-tools-used-by-experts
Experimentation Program Maturity Audit
https://speero.com/experimentation-program-maturity-audit
The evolution of optimization: How to make your program grow and mature
https://albertacg.com/the-evolution-of-optimization-how-to-make-your-program-grow-andmature/
103
fi
fi
2023 Airline Digital Optimization Survey
Smarter Travel Marketing
Airline industry insights for a higher conversion
Month 2019 | © Diggintravel