Seven questions to ask before you believe a revenue uplift figure

We reviewed the uplift figures that revenue management and pricing vendors publish, and found none with a published method or an independent audit. We hold our own work to the same test. The size of a gain depends heavily on what is measured, as one of the better published studies shows.

Meyer and colleagues, with a Lufthansa Group co-author, report a live test of a price-sensitivity model at an anonymised airline. On four origin and destination markets it raised revenue by 6.37%. Once the model was rolled out to about 360 markets, an intervention analysis in May 2019 found an average increase of 2.37%. Scope and method differ between the two, so the second number does not disprove the first. It does show why one percentage cannot be carried from one setting to another.

The seven questions

1. What is the revenue base? A gain on seat fees alone is not a gain on total revenue, and it can come partly out of fares.
2. Is it an average, or an "up to"? An "up to" figure describes the best case and says little about the typical one.
3. Was there a control at the same time? Comparing with last year cannot separate software from a change in the market.
4. What share of the network was treated? A result on a few chosen routes may not hold across all of them.
5. What were the competitors doing? An early adopter against rivals with older tools may gain more than one in a market where all have moved.
6. Did load factor and capacity change? Yield and load factor can each rise while revenue falls.
7. Who paid for the study? This does not make a figure wrong, but you should know the answer.

The questions are our own synthesis. No standard list exists.

How airlines test instead

Testing prices on individual passengers rarely works for an airline, because systems set prices by route, seats are shared and passengers come back. Two designs are in the published record.

Iberia ran competing algorithms on a set of flights in adjacent weeks over nine months, while a group of control flights stayed on the existing system (Talluri and colleagues, 2010).

LATAM and ADC Consulting compared switchback designs, in which a route alternates between treatment and control by week or by day, with fixed groups of routes. On a panel of 80 routes, for paid seat selection, switchbacks reduced standard errors by 32.5% to 67.7%. In their simulations of a 3% uplift over eight weeks, statistical power was about 36% with fixed groups, 74% with weekly switching and 84% with daily switching.

A small airline has fewer routes to compare, which makes this harder and makes the discipline more necessary. We agree the design with your finance team before the first change, and we report the result either way.