The quality-value paradox:
What irrational and imaginary users and forecasters can
tell us about the so-called “real” world
H A RO L D B RO O KS
N OA A / N S S L
H A RO L D. B RO O KS @ N OA A .G OV
Disclaimer
Opinions expressed herein are entirely mine (except for quotations.) I do not know if anyone above or below me in my organization currently, in the past, or in the future, agree or
disagree with any of these opinions. I also do not know if anyone that I have ever talked to, directly or indirectly, about the content of this presentation agrees or disagrees with
any opinion expressed here. All misrepresentation, misinterpretation, or confusion is entirely my responsibility. Similarly, anyone I have ever taken a class from, starting with Mrs.
Walther in kindergarten, nor anyone I have ever taught, bears any responsibility for anything you don’t agree with. I have always attempted to shout to people who are swimming
when llamas are present, “Look out! There are llamas!” As with people in my organizational or educational history, no one with whom I have shared an office or co-authored a
paper with, living or dead, should be blamed for stuff that you don’t care for. Judgment calls are not subject to protest and repeated questioning of judgment is unsporting
conduct and is subject to sanction. I freely admit that conversations with other people over the last 30 years have influenced my views, but that they may or may not agree with or
appreciate the final impact their influence had. If they like something and agree with, they are free to express that, but their decision not express an opinion should not be taken
as implicit or explicit support or opposition. It could be framed from a Wittgensteinian viewpoint, “That of which we cannot speak, we must remain silent.” Side effects of listening
to or paying attention to this talk include lethargy, boredom, and confusion. Sudden onset of sleepiness has been reported in the past. Do not attempt to operate vehicles or large
machinery until you know how this talk has affected you. Seek immediate attention from the session chair if this talk lasts more than 20 minutes. Do not listen to this talk if you
are allergic to it or any of its ingredients. Tell your doctor, medical or otherwise, if you have been to regions where certain fungal conditions are common. Seek immediate medical
attention if you become dizzy or lose your balance during this talk. Adverse reactions, including death, have occurred. If you or a loved have become ill or died as a result of
listening to this presentation, contact your nearest personal injury attorney, but only those who advertise heavily on television or that are members of the Westboro Baptist
Church. Beware of women who weigh the same amount as a duck. They are almost certainly witches. Remember, “strange women lyin' in ponds distributin' swords is no basis for
a system of government. Supreme executive power derives from a mandate from the masses, not from some farcical aquatic ceremony.”
Almost finally, “Mønti Pythøn ik den Hølie Gräilen Røtern nik Akten Di Wik Alsø wik Alsø alsø wik Wi nøt trei a høliday in Sweden this yër? See the løveli lakes The wøndërful
telephøne system And mäni interesting furry animals The characters and incidents portrayed and the names used are fictitious and any similarity to the names, characters, or
history of any person is entirely accidental and unintentional. Signed RICHARD M. NIXON Including the majestik møøse A Møøse once bit my sister... No realli! She was Karving her
initials on the møøse with the sharpened end of an interspace tøøthbrush given her by Svenge - her brother-in-law - an Oslo dentist and star of many Norwegian møvies: "The Høt
Hands of an Oslo Dentist", "Fillings of Passion", "The Huge Mølars of Horst Nordfink"... We apologise for the fault in the subtitles. Those responsible have been sacked. Mynd you,
møøse bites Kan be pretti nasti... We apologise again for the fault in the subtitles. Those responsible for sacking the people who have just been sacked have been sacked. Møøse
trained by YUTTE HERMSGERVØRDENBRØTBØRDA Special Møøse Effects OLAF PROT Møøse Costumes SIGGI CHURCHILLMøøse Choreographed by HORST PROT III Miss Taylor's
Møøses by HENGST DOUGLAS-HOME Møøse trained to mix concrete and sign complicated insurance forms by JURGEN WIGG Møøses' noses wiped by BJØRN IRKESTØM-SLATER
WALKER Large møøse on the left hand side of the screen in the third scene from the end, given a thorough grounding in Latin, French and "O" Level Geography by BO BENN
Suggestive poses for the Møøse suggested by VIC ROTTER Antler-care by LIV THATCHER The directors of the firm hired to continue the credits after the other people had been
sacked, wish it to be known that they have just been sacked. The credits have been completed in an entirely different style at great expense and at the last minute. Executive
Producer JOHN GOLDSTONE & "RALPH" The Wonder Llama Producer MARK FORSTATER Assisted By EARL J. LLAMA MIKE Q. LLAMA III SY LLAMA MERLE Z. LLAMA IX Directed By 40
SPECIALLY TRAINED ECUADORIAN MOUNTAIN LLAMAS 6 VENEZUELAN RED LLAMAS 142 MEXICAN WHOOPING LLAMAS 14 NORTH CHILEAN GUANACOS (CLOSELY RELATED TO THE
LLAMA) REG LLAMA OF BRIXTON 76000 BATTERY LLAMAS FROM "LLAMA-FRESH" FARMS LTD. NEAR PARAGUAY and TERRY GILLIAM & TERRY JONES”
Armaments 2:9-21 “And Saint Attila raised the hand grenade up on high, saying, "O Lord, bless this Thy hand grenade that with it Thou mayest blow Thine enemies to tiny bits, in
Thy mercy." And the Lord did grin and the people did feast upon the lambs and sloths and carp and anchovies and orangutans and breakfast cereals, and fruit bats and large chu...
[At this point, the friar is urged by Brother Maynard to "skip a bit, brother"]... And the Lord spake, saying, "First shalt thou take out the Holy Pin, then shalt thou count to three, no
more, no less. Three shall be the number thou shalt count, and the number of the counting shall be three. Four shalt thou not count, neither count thou two, excepting that thou
then proceed to three. Five is right out. Once the number three, being the third number, be reached, then lobbest thou thy Holy Hand Grenade of Antioch towards thy foe, who
being naughty in my sight, shall snuff it."
Text for Today
 Armaments 2:9-21
Users respond to forecasts
 “…didn’t like the bloody wind, which didn’t go down as
the weathermen had predicted”
Harry Butcher, Commander, USNR
Naval Aide to Dwight D. Eisenhower, Supreme Commander, SHAEF
Describing the D-Day forecast
Quality and value
• Quality-how well forecast matches events
• Value-benefits decision-makers get from using forecasts
• Q-V paradox-increasing quality may not increase value
Expected Value Maximization
 Users make decisions to maximize expected value
 Users who do this are called rational
 If probability of adverse event exceeds the
misclassification ratio, take action
𝐶(𝐹𝑎𝑙𝑠𝑒 𝑎𝑙𝑎𝑟𝑚)
𝑝>
𝐶 𝐹𝑎𝑙𝑠𝑒 𝑎𝑙𝑎𝑟𝑚 + 𝐶(𝑀𝑖𝑠𝑠𝑒𝑑 𝑒𝑣𝑒𝑛𝑡)
 Are real people rational?
St Petersburg Paradox
 We’ll flip a coin until it comes up tails. I’ll pay you $2k for
the total number of flips. ($2 for tail on first, $4 for
second, $8 for third, etc.) How much are you willing to
pay to play this game?
 Expected value is $, so an expected value maximizer
would pay infinite
 Daniel Bernouilli, published same year as his work that
teaches about the airspeed velocity of a swallow
Why people aren’t rational actors, but…
 Utility functions
 Risk attitudes
 Still possible to model wide range of decision problems
(ECMWF estimate of value)
What is your favorite color? (Ellsberg)
Suppose you have an urn containing 30 red balls and 60 other balls that are either black
or yellow. You don't know how many black or how many yellow balls there are, but that
the total number of black balls plus the total number of yellow equals 60.
Gamble A: You receive $100 if you draw a red ball.
Gamble B: You receive $100 if you draw a black ball.
Which gamble do you prefer?
What is your favorite color? (Ellsberg)
Suppose you have an urn containing 30 red balls and 60 other balls that are either black
or yellow. You don't know how many black or how many yellow balls there are, but that
the total number of black balls plus the total number of yellow equals 60.
Gamble C: You receive $100 if you draw a red ball or yellow ball.
Gamble D: You receive $100 if you draw a black ball or yellow ball.
Which gamble do you prefer?
Most people prefer A to B and D to C, even though A and C are identical.
Ambiguity aversion
Framing
Imagine that the US is preparing for the outbreak of disease expected to kill 600 people.
Two alternative programs have been proposed. (Tversky and Kahneman 1981)
If Program A is adopted, 200 people will be saved.
If Program B is adopted, there is 1/3 probability that 600 people will be saved, and 2/3
probability that no people will be saved.
Which of the two programs would you favor?
Framing
Imagine that the US is preparing for the outbreak of disease expected to kill 600 people.
Two alternative programs have been proposed. (Tversky and Kahneman 1981)
If Program C is adopted, 400 people will die.
If Program D is adopted, there is 1/3 probability that nobody will die, and 2/3
probability that 600 people will die.
Which of the two programs would you favor?
72% prefer A. 22% prefer C.
Information Deficit Fallacy
 “If we give people more information, they’ll do what we
thing they ought to do.”
 Much evidence this is false
 Paradox of choice

Adding in options adds stress for evaluation
Other issues
 Other things come into the decision
 LaDue & Correia-concern for family affected fleeing decisions
 “Insertion points”
 Opportunities for information that allows decision makes to use it
 The 3X3 rule (Galluppi)
 Look at image from 3 feet for 3 seconds to get information
Before the screen goes black
 1-Improving quality of forecasts may not improve value
 2-Are decision makers paying attention?
 5-
Before the screen goes black
 1-Improving quality of forecasts may not improve value
 2-Are decision makers paying attention?
 3-Probably as hard or harder to make a forecast high
value as it is to make it high quality