Predicting TV Commercial Effectiveness from Story Boards
Prof. dr. Victor A.F. Lamme, Dr. H. Steven Scholte, Neurensics
May 2013
The pre-testing of TV commercials is common practice, given the huge costs of airing, and the need
to allocate marketing budget specifically to the most effective ways of influencing consumers’
decisions. Functional MRI based neuromarketing techniques have proven a particularly useful tool,
given its capability to predict effectiveness in a highly reliable way. For example, it has been shown
that the ability of anti-smoking campaigns to induce a change in smoking behavior is better predicted
by brainscans than by the opinions of smokers or marketing experts (Falk et al 2012). We have shown
that the effectiveness of TV commercials (measured by Effie award winning) can be predicted with
over 80% accuracy using fMRI based evaluation techniques (Lamme & Scholte 2013).
Huge sums of money can be saved in airing costs by this kind of pre-testing. Even so, it remains
problematic having to conclude that a TV commercial is better not aired (or with limited budget)
after the costs of production have been made. Some TV commercials take hundreds of thousands to
make. It is possible to alter a TV commercial after production, by re-editing, using different music,
altering voice overs, or changing the pack-shot. If guided by proper neuromarketing research and
expertise, this approach can sometimes improve the commercial considerably. But even more helpful
would be a tool that allows for the testing of effectiveness prior to production.
We have conducted a study to investigate whether this is a viable option. Already prior to the
computer age, ideas for TV commercials were often pitched in the form of a story board. A sequence
of images was drawn, accompanied by a brief story explaining the narrative and general idea or
concept. Many major studios still use this technique in some form or another to compare ideas and
present them to clients. We tested to what extent these kinds of story boards could predict the
impact on the brain – and hence the effectiveness - of the actual full-production TV commercial.
REVERSE ENGINEERING OF STORY BOARDS FROM TV COMMERCIALS
Twelve actual TV commercials were reverse engineered to story boards in a systematic fashion: key
shots of the TV commercial were transformed into hand made drawings, typically 10 or so per
commercial. A (male) voice over was generated, explaining the action, and reading the final packshot message1. Drawings and voice over were compiled into a slide show, of equal length as the
original commercial. Subjects, while lying in the MRI scanner - first viewed the 12 story boards
versions, followed by the 12 actual TV commercials. Note that subjects did not know these
commercials, as they were foreign (mostly UK and US) commercials not aired in the Netherlands.
1
We also used another method, where the original soundtrack of the TV commercial was combined with the
drawings, but this yielded worse results, probably because of the often irritating or confusing mismatch
between the soundtrack and the still images.
Predicting TV commercial effectiveness from Story Boards
1
To evaluate TV commercials, we use a proprietary method where we record activity from 13 neural
networks, that each represent a typical (largely unconscious) emotion or valuation given by the brain.
These emotions / valuations (called mappers) can be grouped into four clusters: positive emotions,
negative emotions, personal appeal, and general impact. Our previous research has shown that the
balance between positive and negative emotions is most indicative of TV commercial effectiveness.
We correlate the mapper values obtained for story boards with those obtained for the actual TV
commercial, to test to what extent story boards have any predictive value for the neural impact of TV
commercials.
The example shown in figure 1 illustrates the principle of analysis. Both the TV commercial evoke
fairly similar (but not identical) mapper values. Note for example how in both, negative emotions are
generally larger than positive emotions. This similarity can be expressed in the correlation value R
between the two series, which is this case is 0.76. A value of 1.0 would indicate perfect similarity, a
value of -1.0 perfect dissimilarity (so that values are their exact opposite).
Figure 1: Spider plot of 13 neural emotions / valuations (mappers) for an example TV commercial (red) and its Story
Board version (gray). Values are not identical, but fairly similar (note how in both negative emotions are higher that
positive emotions) resulting in a correlation value R of 0.76
Predicting TV commercial effectiveness from Story Boards
2
TV COMMERCIALS ARE PREDICTED FROM THEIR STORY BOARDS
In the sample of 12 TV commercials and story boards we tested we found cases of fairly high
correlation and cases of nearly zero correlation, but never negative correlations (fig 2, left). On
average the correlation value is 0.35. This value is comparable to the correlation value of 0.37 that is
obtained when each individual mapper value of the TV commercial is plotted against the mapper
value obtained for the story board (fig 2 scatter plot on the right). Statistically, the correlation is
highly significant, with a p-value of 0.0004.
Figure 2: Correlation values between the spider plots of 12 TV commercials and their corresponding story boards.
Correlations are always positive (left). Scatter plot of the individual TV commercial mapper values (13 mappers x 12
commercials) against the mapper values obtained for the story boards. Correlation value is highly significant (right)
These results indicate something very important: indeed the relatively simple images and narrative of
story boards have a reliable predictive value for the neural impact that TV commercials will have that
are made according to these story boards
Indeed the relatively simple images
and narrative of story boards reliably
predict the neural impact that TV
commercials will have that are made
according to these story boards
Predicting TV commercial effectiveness from Story Boards
3
AVOIDING MISTAKES: NEGATIVE EMOTIONS AND IMPACT ARE PREDICTED BEST
In evaluating our results, we found that not all mappers are predicted equally well. For some
mappers (such as Attention) there is a very high correlation between what is measured in the story
board and the TV commercial. In others, however, the correlation is low (figure 3).
Figure 3: Scatter plot of the individual TV commercial values of the mapper ‘Attention’ against the mapper values
obtained for the story boards. Correlation is 0.72 (left). Correlation values for each mapper vary considerably in our
sample: negative emotions and impact mappers are predicted best (right)
On the whole, we find that negative emotions (Fear, Anger, Disgust, Danger) and impact valuations
(Attention, Novelty) are predicted best. It can be that this is due to the sample of TV commercials we
tested. We used foreign commercials, and maybe negative emotions come across most clearly in a
foreign language. Or it could be that negative emotions and the impact of a commercial best survive
the transformation into a story board.
Either, way, this has important implications for what story board testing can bring to the industry.
The tool is apparently best suited to avoid mistakes: What can be predicted best from a story board
is to what extent a TV commercial will annoy (anger), or evoke negative emotions like fear, danger or
disgust. These happen to be all emotions of which we know they need to be avoided to create a
neurally effective message, that can convert consumers into buying the product that is advertised for
(Lamme & Scholte 2013). Also the amount of attention that will be drawn can be predicted very well.
Predicting TV commercial effectiveness from Story Boards
4
IMPROVING PREDICTIVE VALUE: MAIN AXES CORRELATIONS ARE HIGHER
For some individual mappers, the predictive value of story boards is high, for others it is lower. Can
this be improved?
An obvious way of improving predictive value is to collapse emotions to their main axes (positive
emotions, negative emotions, personal appeal and general impact). At the level of these main axes,
story board values correlate more strongly, and also very significantly with TV commercial values (R =
0.48, p = 0.002). The same phenomenon is observed as with the individual mappers, that negative
emotions and general impact are predicted best. However, also positive emotions and personal
appeal have significant correlations, and hence predictive value.
Figure 4: Scatter plot of the Main Axes TV commercial values (4 x 12) against the main axes values obtained for the
story boards. Correlation is 0.48, and highly significant (left). Of the four main axes, values are predicted best for
general impact, followed by negative emotions, personal appeal, and positive emotions (right).
This implies that if we restrict our scope to predicting the neural impact at the level of main axes of
emotions, prediction is much more reliable than at the level of individual emotions. All main axes
show positive predictive values, indicating that the story board carries information about all
emotional aspects of the final TV commercial.
This makes story board testing a viable tool for the evaluation of a TV commercial prior to
production.
Predicting TV commercial effectiveness from Story Boards
5
NEARLY PERFECT PREDICTION IN THE DIMENSION THAT MATTERS: POS-NEG BALANCE
Earlier research has shown that the most important parameter for the effectiveness of a TV
commercial is the balance between positive and negative emotions (Lamme & Scholte 2013). Much
more than the absolute values of these emotions, this balance predicts effectiveness: will consumers
buy the product (or ‘buy’ the message) and show behavioral conversion?
Why is that the case? Positive emotions can be high, but if these are offset by equally high negative
emotions, the net result is a message that leaves mixed feelings. Therefore, consumer behavior is
best predicted by the net balance of these two opposing forces (positive main axis value – negative
main axis value).
We find that story boards are very well suited to evaluate the future pos-neg balance of a TV
commercial. Correlation values increase to 0.64 (p = 0.03) when story boards are compared to TV
commercials along this dimension.
Figure 5: Scatter plot of the TV commercial Pos-Neg values (12) against the Pos-Neg values obtained for the story
boards. Correlation is 0.64, and significant at p = 0.03.
This has very important implications. The future effectiveness of a TV commercial (indexed by its posneg balance) turns out to be the value that is best predicted by its story board. This makes story
board testing a most useful tool if TV commercials are targeted at real conversion of consumers. It is
the power to convince, to induce a change in behavior, to move consumers towards shops that is
best evaluated using fMRI based story board testing.
This adds to the growing evidence that the real power of fMRI based neuromarketing lies indeed in
its capability to predict consumer behavior, instead of consumers’ opinions (Falk et al 2012, Berns &
Moore, 2012).
Predicting TV commercial effectiveness from Story Boards
6
AFTERTHOUGHTS: TOWARDS A NEW WAY OF TV COMMERCIAL PRODUCTION
We find that even a very simple story board, consisting of still images and a spoken narrative that
explains the action and reads the pack-shot, can predict the neural impact of a TV commercial.
Prediction at the level of individual emotions is moderate (r = 0.37), higher at the level of main axes
of emotions (r = 0.48) and highest at the level of the balance between positive and negative
emotions (r =0.64). The latter is of most value when it comes to predicting consumer behavior and
conversion.
This means that the tool of fMRI based story board testing is most suited to compare different TV ad
proposals at the level of storyboards when a decision needs to be made about which of these
proposals will most likely result in consumer conversion. This may require a change from common
practice, where story boards are compiled relatively late in the decision process: typically to brief
production after a concept has been selected and fleshed out.
We propose that story boards should be used relatively early. Already in the conceptual stage, when
several ideas are discussed, story boards should be made, and tested for their effectiveness in
convincing consumers.
For the first time a tool is available that, when properly used, can change the industry of TV
commercial production. No longer do we need to wait for a commercial to work. We can already
know it in advance.
For the first time a tool is available
that, when properly used, can change
the industry of TV commercial
production. No longer do we need to
wait for a commercial to work. We
can already know it in advance.
REFERENCES
Falk EB, Berkman ET & Lieberman MD (2012) From Neural Responses to Population Behavior. Neural
Focus Group Predicts Population-Level Media Effects. Psychological Science, April 17, 2012
0956797611434964
Lamme VAF & Scholte HS (2013) TV commercial effectiveness predicted by functional MRI.
Neuromarketing Science and Business Association Magazine,
Berns GS & Moore SE (2012) A neural predictor of cultural popularity, Journal of Consumer
Psychology 22, 154–160
Predicting TV commercial effectiveness from Story Boards
7