Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
[ Current Topics | Complete List of All Active Topics | RSS feed | Search ]
PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 1 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
4/17/10 10:17 PM
Page 2 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
4/17/10 10:17 PM
Page 3 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
4/17/10 10:17 PM
Page 4 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
4/17/10 10:17 PM
Page 5 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
-- Edward Tufte, September 6, 2005
How to make engineers write concisely with sentences? By combining journalism with the
technical report format. In a newspaper article, the paragraphs are ordered by importance, so
that the reader can stop reading the article at whatever point they lose interest, knowing that
the part they have read was more important than the part left unread.
State your message in one sentence. That is your title. Write one paragraph justifying the
message. That is your abstract. Circle each phrase in the abstract that needs clarification or
more context. Write a paragraph or two for each such phrase. That is the body of your
report. Identify each sentence in the body that needs clarification and write a paragraph or
two in the appendix. Include your contact information for readers who require further detail.
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 6 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
-- William A. Wood (email), September 8, 2005
This doesn't exactly fit `rocket science' but I was not sure where else to put it. A better title might be "Power Point does
lifesaving--NOT."
If you want to see low levels of useful data per slide, let alone irrelevance to the task at hand [presumably saving lives
threatened by Katrina], you could hardly beat those referred to below.
"On lunch at work, but still would prefer no identification if referenced. Thought you might like to experience what our elite
fire and hazmat volunteers are going through. This is insane.
"It appears that all people under FEMA for over two weeks must take "awareness and prevention of sexual harassment,"
"equal rights officer orientation," and "valuing diversity" training programs. The programs total 3-4 hours.
"The mandatory training matrix is here: http://training.fema.gov/EMIWeb/downloads/Mandatory05Matrix.doc
"The letter adopting the matrix is here: http://training.fema.gov/EMIWeb/downloads/Mandatory%20TrainingII.doc
"Powerpoints for the trainings:
http://www.training.fema.gov/emiweb/downloads/DF506/DF%20506%20Sexual%20Harassment%20Visuals.PPT
http://www.training.fema.gov/emiweb/downloads/Df434/DF%20434%20Intro%20to%20Equal%20Rights%20Powerpoints.ppt
http://www.training.fema.gov/emiweb/downloads/DF416/DF%20416%20Diversity.ppt
-- John Liljegren (email), September 8, 2005
I wasn't surprised to see my remark about Nature and Science contested but I was surprised
to see who did the contesting, because what I see as the faults of these journals are exactly
the sort of faults that you often criticize in your books. They both publish a lot of work that
is fashionable and apparently exciting, but they don't insist on including the supporting
information that allows readers to know exactly how the experiments were done -- half the
time they wouldn't even allow authors to include this information because they would say it
made the article too long. What happens in practice, therefore, is that high-profile authors
will publish a claim-staking exercise in Nature or Science and then, if you are lucky, follow
it up later in a journal of lower prestige with a "full paper" that includes the essential details
omitted the first time round.
Let me quote (in suitably anonymized form) an e-mail that I received last week from a
distinguished colleague in the US:
Thank you for bringing the (journal) paper by (authors) to my attention. I have
not been keeping up on the literature (relevant comments below) and was not
aware of it but absolutely agree with it. The paper by (other authors) (Nature,
2003) is pure bullshit, and the editors and reviewers responsible for letting it be
published in Nature should hang their heads. (Name) and coworkers have hit the
major problems (a totally incorrect assay procedure, highly suspect
immunoblotting results) on the head in the third paragraph of his discussion....
This is just one isolated example, of course, but one out of many that one could cite.
-- Athel Cornish-Bowden (email), September 11, 2005
Scientific articles as a model for technical reports
Moving from the PP slide-format to the Nature-style concise report would be an enormous
improvement for any applied technical organization. Fretting about the differences between
Nature and the Proceedings of the National Academy of Sciences is not relevant to
improving NASA workaday technical reporting.
Nature and Science publish about 15,000 authors and co-authors a year, which means, given
their high rejection rate, they disappoint perhaps 50,000 aspiring contributors each year.
That is a lot of enemies to make. Nature and Science rejections probably annoy and bruise
more scientists than all other scientific journals combined. Very few scientific publications
have high rejection rates, in large because publication is financed by page charges (not
unlike a vanity press), paid for by the author's research grants or institution.
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 7 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
unlike a vanity press), paid for by the author's research grants or institution.
Just about everyone who has attempted to publish has their own personal collection of
injustices to retail. These horror stories describe biased, incompetent, envious referees and
idiotic editorial decisions--at least at every journal with a rejection rate greater than 0%. The
anger and the whiny sense of entitlement occasionally exhibited by rejected authors can
become rather intense (even experienced on this little board with a contribution acceptance
rate of about 30%). Publication horror stories and associated gossip are rampant in the social
sciences and humanities, where rejection rates for the top journals routinely exceed 90%.
Talking to journal editors, not just the multitude of rejected authors, will fill out this picture.
Over the years I've served on a dozen editorial boards of research journals and have gotten a
pretty good idea of selection processes. This wisest thing I heard was from the editor of
journal with a 1 in 20 acceptance rate: it is easy to identify and reject the 90% of the
submissions below the line; but for the top 10% (only half will be published), it's a lottery
(depending on the quirks of the referees). For NSF proposal reviewing (at least in the social
sciences), there were usually just a few star proposals, which then led reviewers to ask how
deep in the pool of routine dustbowl empiricism do we wish to dip? For journals and for
grants, overall I was impressed with the care and integrity of the selection processes; I think
most of us involved, other than the true believers, were seeking to find something, anything,
that was good, novel, and true. For marginal submissions, those on the edge of accept/reject,
non-meritorious factors tend play a more important role in the decision (as it also does in
faculty hiring in my experience).
The performance of a journal must be measured in aggregate and not merely by the
embittered anecdotes of the rejected; that is why citations per published article and
circulation numbers are relevant. Measured by citations per article, Nature and Science are
close to the top, sometimes at the top. And they are by far the most widely circulated
scientific journals. A measure of overall system performance is whether every minimally
competent article gets published somewhere, if not in the most-cited and widely circulated
journals. That is surely the case, since the median number of citations resulting from a
published scientific article is zero.
-- Edward Tufte, September 11, 2005
Will Microsoft improve PP?
The record for incremental reform in the cognitive style of PowerPoint is not promising. In
the many release versions of PP, the intellectual level has not been raised. New releases have
drifted toward ingrown self-parody, featuring ever more elaborated PP Phluff and presenter
therapy. These changes have made the new version different from the previous version, but
not smarter. There are no incentives for meaningful change in a monopoly product with an
86% gross profit margin, only incentives to make it different, somehow, from the previous
release. PP competes only with itself.
-- Edward Tufte, September 20, 2005
Unfortunately, NASA is not re-evaluating the use of PowerPoint, instead, the "MINIMUM
INTEROPERABILITY SOFTWARE SUITE" requires all federal employees and contractors
employed at NASA centers to have a current version of MS Office installed on their desktop
computer. Back of the envelope cost*: $13,200,000 every time a new version comes out.
Appropriately, the justification for these standards is contained in a PP presentation.
For masochists: http://desktop-standards.nasa.gov/
* 60,000 on-center employees X $220/upgrade = $13,200,000
-- Robert Simmon (email), September 21, 2005
NOAA hurricane reports better than PP
Here's a counterpoint to PowerPoint: the current NOAA National Hurricane Center forecast
discussion for Hurricane Rita.
The forecast is a succinct technical communication that effectively conveys reasoning and
uncertainty.
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 8 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
http://www.nhc.noaa.gov/text/refresh/MIATCDAT3+shtml/ 211447.shtml
HURRICANE RITA DISCUSSION NUMBER 16
NWS TPC/NATIONAL HURRICANE CENTER MIAMI FL
11 AM EDT WED SEP 21 2005
THE RECONNAISSANCE PLANE WILL NOT BE IN THE AREA OF RITA UNTIL LATER
THIS MORNING. HOWEVER...SATELLITE IMAGES INDICATE THAT THE CLOUD
PATTERN IS TYPICAL OF AN INTENSE HURRICANE WITH A CLEAR EYE
SURROUNDED BY VERY DEEP CONVECTION. INITIAL INTENSITY IS ADJUSTED
UPWARD TO 120 KNOTS AT THIS TIME. HOWEVER...OBJECTIVE T-NUMBERS FROM
BOTH TAFB AND THE UNIVERSITY OF WISCONSIN CIMSS ARE PEAKING NEAR
7.0 ON THE DVORAK SCALE...SUGGESTING WINDS OF NEAR 140 KNOTS. I
WILL WAIT FOR THE PLANE TO REACH RITA TO INCREASE THE WINDS
FURTHER...IF NECESSARY. THE ENVIRONMENT IS CONDUCIVE FOR
STRENGTHENING AND RITA...AS KATRINA DID...WILL BE CROSSING THE LOOP
CURRENT OR AN AREA OF HIGH HEAT CONTENT WITHIN THE NEXT 12 HOURS OR
SO. THIS WOULD AID THE INTENSIFICATION PROCESS. THEREAFTER...THE
INTENSITY WILL BE CONTROLLED BY CHANGES IN THE EYEWALL WHICH ARE
DIFFICULT TO PREDICT. THE HEAT CONTENT IN THE WESTERN GULF OF
MEXICO IS NOT AS FAVORABLE AS IN THE AREA OF THE LOOP CURRENT SO
SLIGHT WEAKENING IS ANTICIPATED....BUT RITA IS EXPECTED TO MAKE
LANDFALL AS A MAJOR HURRICANE...AT LEAST CATEGORY THREE.
THERE HAS BEEN NO CHANGE IN THE STEERING PATTERN. RITA IS MOVING
WESTWARD AT 11 KNOTS SOUTH OF A STRONG HIGH. AS THE HIGH MOVES
EASTWARD...RITA WILL GRADUALLY BEGIN TO MOVE TOWARD THE WESTNORTHWEST AND NORTHWEST BASICALLY TOWARD THE TEXAS COAST. THE
OFFICIAL FORECAST IS VERY CLOSE TO THE MODEL CONSENSUS AND HAS NOT
CHANGED FROM THE PREVIOUS FORECAST.
BOTH THE GFS AND THE GFDL SUGGEST THAT THE WIND FIELD WILL EXPAND.
THEREFORE THE FORECAST WIND RADII HAVE BEEN ADJUSTED ACCORDINLY. ON
THIS TRACK AND DUE TO THE LARGE WIND FIELD ASSOCIATED WITH RITA...A
HURRICANE WATCH WILL LIKELY BE ISSUED LATER THIS AFTERNOON OR
TONIGHT.
FORECASTER AVILA
FORECAST POSITIONS AND MAX WINDS
INITIAL
12HR VT
24HR VT
36HR VT
48HR VT
72HR VT
96HR VT
120HR VT
21/1500Z
22/0000Z
22/1200Z
23/0000Z
23/1200Z
24/1200Z
25/1200Z
26/1200Z
24.3N
24.5N
25.0N
25.7N
26.6N
29.0N
32.5N
35.5N
85.9W
87.9W
90.0W
92.0W
94.0W
96.5W
97.5W
97.0W
120
135
130
125
120
100
40
25
KT
KT
KT
KT
KT
KT...INLAND
KT...INLAND
KT...INLAND
-- Robert Simmon (email), September 21, 2005
The Rita forecast is certainly a big improvement on a typical PowerPoint presentation, but
it's not beyond criticism.
1. Why put it all in capital letters? Mixed upper/lower-case text is much easier to read
and understand than all upper-case, as has been realized by the people who design
traffic signs in many countries (though curiously not France, where I live) for at least
thirty years. Computers have been able to cope with mixed upper/lower-case text for at
least the same amount of time: surely the National Hurricane Center isn't still
struggling with 1960s-vintage mainframes?
2. I'm not sure if this is intended for the general public, but assuming that it is, can
people be expected to know what TAFB, CIMSS, GFS and GFDL are?
However, I agree that it conveys reasoning and uncertainty in an honest manner, and that it
is clearly intended for readers who think about what they are reading.
-- Athel Cornish-Bowden (email), September 21, 2005
My guess is the NWS all-caps style 1) is left over from teletype days and 2) remains
because there's quite a bit of overlap between NOAA and the Navy's oceanographic
community and the Navy still uses a lot of all-caps Courier. At this point I think they keep
all-caps out of nostalgia.
As a New Orleans evacuee I have come to exclusively rely on NOAA reports. Interestingly
I'm now following Rita closely because Tulane's medical school was supposed to reform in
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 9 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
I'm now following Rita closely because Tulane's medical school was supposed to reform in
Houston this Saturday. That's now postponed until 1 October. I'm also staying in College
Station, which is expected to get some of the wind and rain.
-- Niels Olson (email), September 21, 2005
The NWS caps style is also re-inforced by the fact almost all military orders (pick any
service) are created in this teletype format. There is a argument that is eliminates a degree of
ambiguity, at the expense of readibility. In other words, a capital letter contains the same
meaning as a lower case letter. There is no implied difference of meaning based on
capitalization.
But historically, it is based on telegraph and teletype styles. The format and ordering of the
paragraphs can contain significance. In military communications there are generally headers
and footers to tell you where you are in the message, what information to expect next, and
that you have reached the end of the section or message. Assuming that you have the rosetta
stone to decode often obtuse headers and footers.
By the way, this subject is very topical to me because we are frequently driven to produce
reports in powerpoint rather that word. I usually object to call a powerpoint presentation a
report; prefering to call it a brief or presentation. It is a very common to produce a
powerpoint brief with notes pages and print the combination as a report. Even that
combination leaves a lot to be desired in presenting complex issues, because all you have
really done is add a text area about the same size as the powerpoint slide to amplify the
slide. This, of course, means that the slide is probably not understandable to someone who
is not already intimate with the subject material. But even in this case the notes pages very
frequently suffer from the same choppiness.
I work in an organization where we are frequently more motivated to produce emerging
findings quickly rather than conducting thorough analysis to produce quality findings. This
means not only is the powerpoint brief terse and choppy, but often the results are misleading
or wrong. Not to mention contentious, because it is nearly impossible to vet the findings
and achieve concensus in a short period (such as less than a week in the Columbia case). I
am almost sure that this situation existed in the Columbia case. And, as in my organization,
you can almost assure that none of the results presented are accredited at the time they were
first presented.
-- Clyde Smithson (email), September 30, 2005
After thinking about this over the weekend the thought struck me that the NWS style
messages are more a function of data packing than anything to do with readability. Since
these messages are transmitted across an electronic network that has a fixed data rate it is
more important to reduce the character set so as to provide greater message bandwidth. It is
probably the case that the engineers have employed a bit packing technique to maximize the
amount of data carried in a word. Typically 8 or 4 bytes, 64 or 32 bits, but 2 or 1 bytes are
encountered as word length of data transmitted. In a bit packing system if a value is binary
(say on or off) only 1 bit is needed to represent the data, and so on based on the number of
states. So why waste more data than needed.
To go back to the telegraph example, Morse code has 39 defined characters (26 letters, 10
numerals, and 3 special characters) so would require 6 bits (2 to the 6th bits = 64) to
represent in a digital message. This means that in a 4 byte word (32 bits) that you could
represent 5 Morse code characters and you waste 2 bits of information, a 6.25% waste.
Standard ASCII, which does contain lower case and more special characters, has 128
different characters (2 to the 7th bits). A 4 byte word could contain 4 Standard ASCII
characters but wastes 4 bits of information, 12.5% waste. Extended ASCII has 256
characters (2 to the 8th bits) so a 4 byte word contains exactly 4 characters. Engineers
sometimes deal with the lost bits by using them to contain other information such as
headers, or sometimes split characters across multiple words. This, of course, requires more
software at either end of the message to encode and decode the information.
But from an economy of scale ASCII takes twice the bandwidth of Morse code and
Extended ASCII takes four times the bandwidth. This does matter even today with high
speed internet because much of the government and military infrastructure runs through
older systems with lower data rates. Additionally, as both institutions become more
network-centric, our desire to put more data through the network approaches or exceeds the
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 10 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
network-centric, our desire to put more data through the network approaches or exceeds the
capacity of the faster networks to carry all this data. This extra data flowing across the
networks is not the equivalent of extra knowledge. The NWS example would provide no
more information to the reader if mixed upper and lower case were used, but would
consume 2 or 4 times the "data" space if ASCII or Extended ASCII were used.
For some history (http://en.wikipedia.org/wiki/ASCII#History)
-- Clyde Smithson (email), October 3, 2005
Return to Flight Task Group biographies
Here are the biographies of those on the Return to Flight Task Group who saw the NASA
engineering by PowerPoint and denounced it in their final report (quoted extensively in the
last 2 pages of my essay).
As I wrote in the essay above, "Both the Columbia Accident Accident Investigation Board
(2003) and the Return to Flight Task Group (2005) were filled with smart experienced
people with spectacular credentials. These review boards examined what is probably the best
evidence available on PP for technical work: hundreds of PP decks from a high-IQ
government agency thorough practiced in PP. Both review boards concluded that (1) PP is
an inappropriate tool for engineering reports, presentations, documentation; and (2) the
technical report is superior to PP. Matched up against alternative tools, PowerPoint loses."
Here are the biographies of the NASA PowerPoint critics:
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 11 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
4/17/10 10:17 PM
Page 12 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
4/17/10 10:17 PM
Page 13 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
-- Edward Tufte, October 18, 2005
For an excellent technical report on a complex engineering matter, see the NASA report on
the foam loss from the external tank during the recent launch of the Discover (STS-114).
The report is written in sentences and paragraphs, not bullet-point grunts and slides.
The report presents many extraordinary images of the tank post-launch. The difficult
analytical issue is the lack of comparisons of the tank conditions from the previous 112
launches.
Here is the link to the report http://www.nasa.gov/mission_pages/shuttle/main/
Then go to the link "full report, 2.1 MbPDF".
-- Edward Tufte, October 23, 2005
What a stunningly beautiful piece of work, thank you.
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 14 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
This note needn't be considered for the thread. Just a couple of clean-up ideas, in addition to
"are be...," it may be good to look at:
-- 5 grafs from the very bottom, "In the final report, 7 of Task Group Members... " -- First
graf below the detailed slide analysis, last line: "This is a lot of insecure format... ." -- The
"Marketing Strategy - Good4U" PP slide seems orphaned.
Thank you so much for the close work of your team. DJ
-- David Johnson (email), November 17, 2005
Many of the contributions to this thread are helpful -- especially those that point out flawed
PowerPoint design. Has anyone found an example of skilled information design in a
PowerPoint presentation?
I feel that we can only learn so much from examples of what NOT to do...
-- Scott L. Mitchell (email), December 9, 2005
PP is a competent Projector Operating System for full screen images and videos, replacing
the little forward-back button in old-fashioned projector systems. PowerPoint is neither the
best nor the worst Projector Operating System. It faces strong competition from the projector
itself with its own forward-back controls. A Projector Operating System, however, should
not impose Microsoft's cognitive style on our presentations.
PP has some low-end design tools helpful in constructing PowerPoint parodies.
PP might also help show a few talking points an informal meetings, but why not instead
print out an agenda on a piece of paper?
PowerPoint may now and then benefit the bottom 10% of all presenters. PP forces the really
inept to have points, some points, any points.
-- Edward Tufte, December 9, 2005
One of my professors, Dr Sandor Vigh, embeds what he calls zoomimage graphics in his PP
presentations. I can't say I've seen anyone else do this and I haven't used Quicktime myself
enough to know the buttonology how-to, but he basically puts a very high resolution image
into a Quicktime frame that occupies the entire PowerPoint slide. So it's a slide projector.
But he can zoom and pan! So he scans in Nettergrams, histology slides, whatever, at, say,
3000x2000 and then explores the details of the image while the projected image is always
above the resolution threshold of the projector. This is vastly superior to 'mere' projection,
despite the LCD projector. And look, it displays on the web!
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 15 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
If there's no image above, you can view the zoomimage here.
Use the mouse to pan and shift/ctrl to zoom. Zoom in on the bones of the intermediate
phalanges of the four fingers. Compare the fine lattice work of trabecular bone in the ends to
the dense hollow tubes of cortical bone in the shafts. Update 2 April 2009 You can make
your own zoomimages here
-- Niels Olson (email), December 9, 2005
To follow up on Niels Olson's comment, the precise technology being used is Quicktime
VR. Quicktime VR is typically used to stitch together 3-D panoramas or a 3-D view of a
solid object - like Niels, I haven't used to to display two-dimensional images in this manner
before. Dr. Vingh's use of it in his slides is an interesting end-run around the resolution
limits of a normal computer presentation.
-- Zach Heaton (email), December 15, 2005
Elizabeth Lane Lawley, a professor visiting Microsoft, comments on "the culture of the
deck":
http://mamamusings.net/archives/2005/11/19/the_culture_of_the_deck.php
Her experience at Microsoft is comparable to that of the NASA Return to Flight Task Group
with regard to the persistent disutility of using PP decks to replace technical reports.
-- Edward Tufte, December 29, 2005
PowerPoint Does New Orleans
All but one of the committee 'final reports' for Mayor Nagin's Bring New Orleans Back
Commission are out. Most are in PowerPoint. The report on levee recommendations is the
lone exception.
NOTE ADDED, JULY 1, 2006 BY NIELS OLSON:
Does plain text or PowerPoint tell the story better? Compare Howard Reich's piece in
today's Chicago Tribune, Crisis of culture in New Orleans, to the official Culture Committee
report of Mayor Nagin's Bring New Orleans Back Commission.
-- Niels Olson (email), January 20, 2006
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 16 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
On the subject of writing, a favorite professor of mine said: "What's in the head goes on the
page." The "final reports" to which Niels Olson links are astonishingly awful, and ought to
be required reading for the those who would defend the general utility of Power Point. That
these are "final reports" -- not merely tools to supplement the oral presentation of
conclusions from actual written reports -- suggests that an antiliterate (I was going to write
subliterate, but the word is insufficiently strong) approach now dominates public policy
discussion in the United States. What a shame.
-- Alexey Merz (email), January 20, 2006
Cognitive Style of PowerPoint 2nd edition now published
Just published is the 2nd edition of The Cognitive Style of PowerPoint. It is now 32 pages
long; the original essay was 24 pages. The new edition contains the material on PowerPoint
and rocket science that opens this thread, a long discussion of the causes of presentations
(sorting out variation among users, content, and presentation methods), and an essay on lists.
This new edition is also a chapter in Beautiful Evidence.
For more information and to order click here.
-- Edward Tufte, April 25, 2006
Thank you thank you thank you! I am an environmental scientist, and my company does
environmental impact assessment 1-10 day training workshops -- WITHOUT Powerpoint!
We have a programmer who prepares an occasional animated presentation in Flash (which is
also helpful because it is only images, so we can offer it in any country, in any language) -so I will have a projector in the room, and you should see everyone's face fall when they
first enter and see the projector, and watch them light up when I tell them it is only there to
scare them! Our trainings use case studies, flip charts, an occasional video or a cartoon on an
overhead during the break, and music...NO Powerpoint or even Apple's lovely and way
easier and more beautiful implementation of a similar system, called Keynote. I try and try
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 17 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
easier and more beautiful implementation of a similar system, called Keynote. I try and try
to explain to prospective clients why we don't use it -- but sometimes we are actually
required to use it by, gasp, their TRAINING departments....anyway, this is a long- winded
way of saying that the PP disease is very hard to cure but there are many of us who simply
refuse to be infected...and thanks for providing some good ammunition to use with our
clients.
-- Leslie Wildesen, Ph.D. (email), June 23, 2006
It's not just reports that suffer.
I've recently attended a "technical" training course in which we were presented with an
enormous number of PowerPoint slides, most of which were merely read aloud. Not only
was this course mind-numbing to teach — I can only pity the instructor — it's also an
active waste of time.
At the end of the course we were given a book containing one slide per page, along with a
small amount of notes below. I've struggled to find the worst of these slides, and I've
recreated one of the candidates here (90k PNG). I'm assuming direct scans would be
forbidden, and I've neglected to include the "content" provider's logo.
I am still unable to find any shred of meaning in this slide, or many of the others.
-- Dan Avis (email), June 29, 2006
DISCOVERY LAUNCH
Here is a link to William Harwood's excellent account of shuttle risks in the upcoming
flight, scheduled for this Saturday. This link provides context for my comments that follow.
http://www.spaceflightnow.com/shuttle/sts121/060629preview/part1.html
____________________________
About 18 months ago in Houston I reviewed the shuttle Probability Risk Assessment (PRA)
material for NASA. PRA works with a list of possible threats, estimates their probablilities
and expected losses, and then seeks to assist decision-making for shuttle risk-reduction.
After the PRA group presented their results, I had two major suggestions:
(1) They should prepare a detailed summary matrix (on, of course, 11" by 17" paper),
ordering the risks and providing, in a comments column, relevant background for each
estimate. Let that intense matrix, backed up by similar more-detailed 11" by 17" arrays of
risk estimates, be the main presentation device and analytical tool for making decisions. This
was designed to replace their chippy and twiddly PP slides, which made a hash of their good
technical work and made it difficult to assess the overall risk context.
(2) The PRA assessments did not take into account a major risk factor in both the
Challenger and Columbia accidents: on-ground intellectual failures in engineering analysis.
In the case of the Challenger, the analytic process on the day before the accident was
seriously deficient, in the sense that--in hindsight to be sure--the Challenger would not have
been launched on that very cold day (which compromised the O-rings and caused the
accident) if smarter engineering analysis and better decision-making had taken place. In the
case of the Columbia, better analysis and decision-making during the flight might have
yielded rescue efforts to try to save the crew, which was endangered by damage to the
Columbia suffered at launch. I suggested to the PRA group that on-ground analytic problems
contributed to something like 1.3 of the 2.0 accidents in the 113 flights. But there was no
risk assessment of such in the PRA; that is, about 65% of the directly observed empirical
risk in the 113 flights was not accounted for by the PRA model. The shuttle itself was
considerably less risky than what was happening on the ground in decision-making about the
shuttle.
At the meeting, I also handed out Richard Feynman's famous discussion of shuttle risks,
which Feynman prepared as a part of the Challenger investigation in 1987.
____________________________
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 18 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
The analysis for the upcoming launch of the Discovery in July 2006, as the link above
indicates, was an intense evaluation of risks and trade-offs.
On the basis of reading some of the public documentation (and no direct knowledge) for the
upcoming flight in the last few weeks, I think that NASA has made a reasonable and wellinformed decision for the upcoming flight. It was also a contested decision. I would vote for
the launch. The on-ground factors that contributed to 1.3 shuttle losses appear to be
mitigated by the thorough analysis for this flight. The current risk number is a cloudy 1 in
100, which is risky but has been acceptable in the past. The cloudy contributions to risk are
the recent changes in the foam, which turns Discovery into something of an experiment.
In the Discovery discussions, a telling distinction was made between "programmatic risks"
and "crew risks." The programmatic risk is very high right now no matter what happens.
Having flown once in 3+ years, the shuttle program might well collapse if unable to fly
soon (within a year or so), or if there is another accident even if the crew escaped unharmed.
This rescue scenario is itself troublesome, since the rescue launch must quickly take into
account what caused the need for the Discovery crew-rescue in the first place.
-- Edward Tufte, June 29, 2006
DISCOVERY LAUNCH
Another detailed and excellent account by William Harwood on the eve of the flight:
http://www.spaceflightnow.com/shuttle/sts121/060630griffin/
There is a Feynman-like clarity to the Discovery analysis done by Michael Griffin, NASA's
director. Now it just has to be confirmed empirically!
ET, Saturday, 6.53am
-- Edward Tufte, June 30, 2006
POWERPOINT FOR DISCOVERY FLIGHT READINESS REVIEW: THE FOAM SLIDES,
OR "MAKE THAT CALL NOW, THAT'S 1-800-. . . . "
Here's the PP deck for "STS-121, Flight Readiness Review, External Tank Project (ET119):"
http://images.spaceref.com/news/2006/2006.06.16.ET_FRR.pdf
This pdf file should be up in a separate window to read in parallel with the comments
below:
These slides summarize the results of the enormous amount of resources (probably >$1
billion, some estimates are much higher) devoted to the external tank foam problem.
The slides do not display a sense of engineering intelligence or discipline. In the main
report, there is a persistent habit of dequantification and a general absence of units of
measurement. The back-up slides are more quantified and at a higher intellectual level.
Several of the slides look like they were produced by a designer lacking in scientific
training.
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 19 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
The key overview slide (page 3) is a very good idea but a presentation mess. The good idea
is to have an intense and fairly detailed summary early in the presentation. But PP's
lightweight resolution and lousy design tools compromise the summary slide. Students of PP
design might, however, appreciate the 5 sets of orange drop-shadows, 4 wavy-purple color
fields, 3 unintentionally 3D blue time-lines, 2 overactive grids, and floating-off-in-space
bullets in the highlight box (with an arbitrary change from dots to dashes midstream in the
box). All this stuff on one over-produced but importantslide.
In real science, every photograph has a scale of measurement built right in to the
photograph. This low-resolution display method makes it impossible to do so. (Even the
shuttle close-out photos, just about the most documentary type of photographs one can
imagine, have no scales of measurement and no rulers in the pictures.)
The bullet lists tend somewhat to be base-touching grunts, which show effects without
causes, actions without actors, verbs without subjects, and nouns without predicates. The
branding with 3 logos on every slide (the title slide has 4 logos) is unprofessional, pitchy,
turfy. Are we doing engineering analysis or marketing here? Some 20% of the space of
every slide (already a a very low resolution display method) is devoted to branding and to
the boxed-in awkward and repetitive slide titles. It is as if each and every slide has to remind
the viewer what the presentation is about. So the top 20% of every slide is something to
skip, perhaps putting some viewers in the mode of skipping and sliding through the rest of
the slide. It is as if the top of every slide announces "nothing important here, you've seen it
all many times before."
In several slides, the visually most active materials are the cross-hatched exploding 3D
arrows linking the external tank to the magnified areas. Why are the arrows pointing
anyway? It's just a simple linking line. The idea here of close contextualized imaging of the
problem areas is a very good one, but the badly-drawn giant blue arrows are silly, and result
in making the dequantified images of the foam problem areas too small.
The typography is poor, with odd hierarchies (underlined bold italic in parentheses at one
point). Is "O2" the proper way for NASA contractors and NASA to write the oxygen
molecule (even wikipedia uses a subscript)? Does the slide designer know how to write a
subscript in PP?
The overlapping statistical graphics on page BU-2 are presented as decoration, not evidence.
The report is 33 slides long; yet about 10 slide-equivalents are essentially content-free
(compulsive repetitive branding, twiddly hierarchical organization, empty space, assorted
title pages, and so on). This PP fluffed-up material here and quite a bit more could easily be
placed in a technical report on 4 pages of an 11" by 17" piece of paper (folded in half), an
exercise left to the student.
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 20 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
The tone and style of the presentation seem alienated from professional engineering. It looks
like the slides were prepared by a PP designer, assisted now and then by an engineer. Or
maybe it is just the PP pitch style diluting the content. At an FRR?!
I hope the actual engineering for the shuttle is a lot better than the evidence for the
engineering shown in this presentation.
How much does a problematic presentational style signal poor engineering? Is it just PP or a
PP designer weakening the quality of evidence? Or are there deeper intellectual failures? The
dequantification, the failure to follow professional engineering conventions, the infomercial
tone are worrisome. There is no sign of engineering discipline here, except in the back-up
slides. Thus the effect of the presentation is to suggest that there just might be some
problems with foam engineering and analytical quality. A danger of problematic
presentational styles, such as NASA PP, is not only that they enable sloppiness but also that
they can place the truth in disrepute.
It is also a shame that all that expensive engineering work winds up being represented in
this manner at a Flight Readiness Review.
-- Edward Tufte, July 1, 2006
Discovery foam problems occur on launch pad
This morning, Monday July 3, news of a small foam crack came out. On the evening of July
2, after 2 tankings and drainings for launch attempts scrubbed because of the weather,
inspectors found a 5" crack in the foam on the Discovery external tank on the launch pad.
Here's an excerpt from the KSC Ice/ Debris Team:
"The inboard strut for the L02 Feedline Bracket assembly at XT-1129 was found to be
cracked. The damage is approx 5-6 inches long and appears to originate near the where the
strut connects to the feedline and extends toward the ET. The TPS crack is approx 1/4 inch
wide with an offset of approx 1/4 inch. An IPR was initiated for this item. Inboard views of
the remaining visible brackets did not reveal any similar damage. Outboard views of the
feedline brackets revealed areas of TPS debris in the gap between the feedline and the
bracket - this condition was noticed at XT-1129, 1377, and 1623. No obvious indications of
crushed foam or debris was detected at the XT-1871 and 1978 brackets."
There's a picture of the foam crack and the full inspection report here.
Note that the inspection report is written in sentences and not in the cryptic grunts of
PowerPoint.
There is a research design problem or a control group problem here: are we seeing cracked
foam or inspections of cracked foam? Perhaps every launch of the 114 has had some foam
debris shedding, and we're only seeing small pieces and cracks now because the intensity of
inspections has increased since the Columbia. Or maybe not.
-- Edward Tufte, July 3, 2006
From 9.00-9.30 pm Monday, William Gerstenmaier, NASA's head of Space Operations,
gave an informative and smart news conference on the foam issue. He provided a summary
of the evidence and answered questions from the space press. There's no problem as a result
of the foam liberation incident. Weather permitting, the Discovery will launch tomorrow.
-- Edward Tufte, July 3, 2006
Response to PowerPoint Does Rocket Science (and the upcoming Discovery flight)
Did the news conference present the PP slides, or did they a different medium to convey the
details?
-- Allan T. Grohe Jr. (email), July 5, 2006
They usually give a brief talk and then answer questions in a straightforward and intelligent
manner, accompanied by occasional physical props, such as the broken-off piece of foam or
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 21 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
manner, accompanied by occasional physical props, such as the broken-off piece of foam or
model of the external tank. They did not use PP in the 8 to 10 press conferences I've
viewed. You can see the press conferences and the launch by going to nasa tv at
http://www.nasa.gov/multimedia/nasatv/
Apparently the PP Flight Readiness Review for the foam (reviewed above) was something
of a leak; the other FRRs at the meeting are not going to be made available. Keith Cowing,
who runs NASA Watch, sent me an email saying that I might be interested in the foam FRR
that he had posted at his website. You can see more on this at
http://www.nasawatch.com/
I think the press conferences are excellent, assisted by a well-informed space press. After the
flight, the head of NASA Michael Griffin was asked at the press conference if he felt
"vindicated" by his decision to launch. He said not at all, if anything, it was vindication for
the scientific method--that is, looking at the evidence and the numbers at hand. What a
wonderful thing for the Director of NASA to say. This contrasts to the PP cognitive style,
which often seems to encourage presenters to pitch rather than present evidence.
-- Edward Tufte, July 5, 2006
The FRR-foam summary slide is now shown about 5 contributions up (in my review of that
presentation).
And, also added, immediately above, a press conference photo of William Gerstenmaier
showing the foam chip.
-- ET, July 6, 2006
Below, a link to a good account of the Discovery inspections by John Schwartz of the New
York Times on the problem of distinguishing useful evidence from additional evidence, a
problem that also occurs with newly developed exquisitely sensitive measurements (for
example, PSA tests and the monitoring of contaminants of drinking water).
John Schwartz, New Scrutiny for Every Speck on the Shuttle, New York Times, 11 July
2006.
These issues can lead to quite subtle consequences, as my Yale colleague Alvan R. Feinstein
suggested in many studies, including this one in the Archives of Internal Medicine: ". . .
many breast cancers found by mammography screening have excellent prognosis not just
because of early detection, but also because many of the cancers are relatively benign,
requiring minimal therapy."
Sandra Y. Moody-Ayers, MD; Carolyn K. Wells, MPH; Alvan R. Feinstein, MD, MS,
"Benign" Tumors and "Early Detection" in Mammography-Screened Patients of a Natural
Cohort With Breast Cancer, Arch Intern Med. 2000;160:1109-1115.
(Thanks to Niels Olson for the NYT permalink above.)
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 22 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
-- Edward Tufte, July 10, 2006
Here's a clear technical report and press release, using a 4-page format (similar to A3 or 11"
by 17", folded in half). If the report were printed as a 4-pager folded-in-half, then the June
2005/April 2006 images would fall somewhat closer together, which would facilitate
comparison (although both images can be seen vertically adjacent simultaneously on, for
example, a 30" monitor). From the Institute for Science and International Security (ISIS):
http://www.isis-online.org/publications/southasia/newkhushab.pdf
http://www.isis-online.org/
Here are links to the Guardian and the Washington Post accounts of the ISIS report:
http://www.guardian.co.uk/pakistan/Story/0,,1828058,00.html
http://www.washingtonpost.com/wp-dyn/content/article/2006/07/24/AR2006072400995.html
Several other ISIS reports are distinguished by their sourcing, detail, use of satellite
photographs, and estimates of uncertainty. See, for example, Chinese Military Plutonium
and Highly Enriched Uranium Inventories, by David Albright and Corey Hinderstein:
http://www.isis-online.org/global_stocks/end2003/chinese_military_inventories.pdf
-- Edward Tufte, July 24, 2006
Interesting presentations with full-screen dynamic graphics by Hans Rosling:
http://www.gapminder.org/
http://www.ted.com/tedtalks/tedtalksplayer.cfm?key=hans_rosling&flashEnabled=1
(These links were provided by Kindly Contributor Cesar Martin.)
-- Edward Tufte, July 25, 2006
From Nature, 13 July 2006, still more on PP, this from Martin Kemp, an Oxford art
historian:
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 23 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
-- Edward Tufte, July 26, 2006
Here is a well-designed technical report:
http://evo.bio.psu.edu/printclock
It is about 7 pages long. Note the excellent illustrations, integration of text and images,
documentation, careful citations, and different types of evidence. Note also the use of
sentences and paragraphs and flowing text, not the grunts of hierarchical bullet points on
slides.
This is a very high standard for a technical report, but why not start at that level?
-- Edward Tufte, August 1, 2006
In your workshops, you describe how to replace PowerPoint presentations with 11 x 17
sized reports, and provide many good arguments for why "engineering by PowerPoint"
doesn't work very well.
This is a website that might interest you and your audience. It describes a process for
creating A3 sized technical reports, and using them to make better decisions.
http://www.coe.montana.edu/IE/faculty/sobek/A3/index.htm
Dr. Durward Sobek of Montana State University spent six months in Japan as a grad
student, interviewing and observing Toyota engineers to uncover the reasons why Toyota
was able to develop cars much more quickly than other auto makers and also maintain high
standards for reliability.
Toyota uses these A3 reports extensively in their engineering processes. They believe that
the discipline required to accurately capture a problem on a single sheet forces the author to
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 24 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
the discipline required to accurately capture a problem on a single sheet forces the author to
express the issue with both clarity and conciseness. They emphasize using visual models to
express ideas rather than a lot of text, and value the ability to have all of the pertinent
information within a single field of vision. The engineers are also required to bring their
supporting documentation, so that the team can dive into the details when necessary.
Since then, Dr. Sobek has taught many engineers how to use A3 reports to make better
technical decisions. I can tell you from my personal experience with this technique that it is
amazingly powerful. By using one of these reports, we solved a technical problem within a
single meeting that we had literally wrestled with for years through engineering by
PowerPoint. By forcing us to make our knowledge about the problem visible in a systematic
way, the tool helped us come to a deeper understanding that led to the solution.
Best regards,
Katherine Radeka
-- Katherine Radeka (email), August 2, 2006
The "A3 Process" described above begins with a good idea and then dilutes it into a
Business Methodology Fad. BMFs are characterized by a germ of a good idea, but also by
over-reaching, over-simplifying, excessive focus on a single idea, pitchy and enthusiastic
over-simplified examples, and pretentious names ("The Toyota Method," "The Long Tail,",
"The Genghis Khan Guide to Mastering the Universe," "The Takahari Guide to Infinite
Profits," and so on).
In the Beautiful Evidence chapter on corrupt techniques in evidence presentations, the
section on over-reaching concludes with this: "When a precise, narrowly focused technical
idea becomes metaphor and sprawls globally, its credibility must be earned afresh locally by
means of specific evidence demonstrating the relevance and explanatory power of the idea
in its new application." (p. 151)
The A3 method, which at its heart is a good idea, requires some down-in-the-trenches
detailed and complex examples. And it should avoid bullet lists in describing the method.
-- Edward Tufte, August 11, 2006
A remarkable account of "Death by PowerPoint," as the phrase takes on new meaning:
http://armsandinfluence.typepad.com/armsandinfluence/2006/08/death_by_powerp.html
-- Edward Tufte, August 11, 2006
From Thomas Ricks' book Fiasco:
"[Army Lt. General David] McKiernan had another, smaller but nagging issue: He couldn't
get Franks to issue clear orders that stated explicitly what he wanted done, how he wanted to
do it, and why. Rather, Franks passed along PowerPoint briefing slides that he had shown to
Rumsfeld: "It's quite frustrating the way this works, but the way we do things nowadays is
combatant commanders brief their products in PowerPoint up in Washington to OSD and
Secretary of Defense...In lieu of an order, or a frag [fragmentary order], or plan, you get a
bunch of PowerPoint slides...[T]hat is frustrating, because nobody wants to plan against
PowerPoint slides." That reliance on slides rather than formal written orders seemed to some
military professionals to capture the essence of Rumsfeld's amateurish approach to war
planning. "Here may be the clearest manifestation of OSD's contempt for the accumulated
wisdom of the military profession and of the assumption among forward thinkers that
technology-- above all information technology--has rendered obsolete the conventions
traditionall governing the preparation and conduct of war," commented retired Army Col.
Andrew Bacevich, a former commander of an armored cavalry regiment. "To imagine that
PowerPoint slides can substitute for such means is really the height of recklessness." It was
like telling an automobile mechanic to use a manufacturer's glossy sales brochure to figure
out how to repair an engine."
This raises some of the same issues discussed in the report by members of the NASA Return
to Flight Task Force in the contribution at the top of this thread.
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 25 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
-- Edward Tufte, August 11, 2006
I have just found and bookmarked this brilliant set of material - good work and great dialog
by all.
I've had my share of problems with PP and it's predicessors. Managing marketing groups in
Silicon Valley, I always tried to get product managers to keep it simple. My mantra was "no
more than three bullets" and "if it takes more than three, you need another slide". This
approach was a recognition that presentation software was going to be used no matter what.
I would say that my success rate was less than 10%. The prevailing culture often obsessed
with creating slide templates that allowed as many bullets on a page as possible.
One earlier post illustrates a similar "can't see the wood for the trees" communications
problem. Clyde Smithson does a creditable job of describing the efficiency of coding
systems (Oct 2005) but misses that larger point. He takes a one-dimensional approach to
value in communications - bandwidth is expensive - and advocates everything possible to
reduce the useage of this expensive resource. This kind of thinking comes from the early
days of communciations and microwave engineering. In a society dominated by Moore's
Law, how valuable is that saved bandwidth compared to using a more verbose coding
system like UTF that alows us to communicate in most languages on the planet?
Clearly, the message is about communication, less about cost. This kind of thinking has us
guiding 100 million dollar aircraft around the sky using restricted-voice bandwidth
technology that dates back to early phone systems and carbon radio microphones. Anyone
who has used a full-spectrum voice system like Skype knows that fewer mistakes and mishears occur when all of the audio information is present. Should the lives of hundreds of
passengers be placed at higher risk because the radio designer and regulatory agency saved
20 dollars on a radio costing a few hundred?
More power to your biting analysis and critical assessment of communication.
-- Brian MacLeod (email), August 20, 2006
Having just completed ET's course yesterday, I was curious to see if PP could handle
mathematical expressions (operators, superscripts, subscripts...). It can. Maybe not that
easily, but with a little work, the pivotal Boeing slide could have looked much better. This
means that those who are required to use PP should take the extra time to make their slides
the best they can.
Of course, we would all be better off with an 11 x 17 paper document with real analysis...
-- Andy Orr (email), August 25, 2006
A follow up to the quote from Thomas Ricks' book _Fiasco_ and an indication that things
can change. Again, from Thomas Ricks' _Fiasco_:
"Col. McMaster also challenged the U.S. military, all but banning the use of Power Point
briefings by his officers. The Army loves these bulleted briefings, but McMaster had come
to believe that the ubiquitous software inhibits clarity of thinking, expression, and planning."
pg 421
-- Dave Froberg (email), August 29, 2006
Engineering software problems at NASA:
http://www.hq.nasa.gov/office/oig/hq/audits/reports/FY06/S06012.pdf
http://images.spaceref.com/news/2006/NAC.Science.Cmte.report.05.18.2006.pdf
-- Edward Tufte, September 3, 2006
The Harvard School of Public Health Instructional Computing Facility may be capable of
writing witless PP designer stylesheets, however, I suspect that much of the problem is
generated from above by corporate "identity management" requirements. Consider this
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 26 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
generated from above by corporate "identity management" requirements. Consider this
advice on how to generate materials which inform their audiences about "world class"
research. http://www.leeds.ac.uk/identitymanagement/styles/research.htm
-- John McMillan (email), September 6, 2006
The War on Prose
9/11 Five Years Later: Successes and Challenges:
http://www.whitehouse.gov/nsc/waronterror/2006/index.html
That's not writing, it's PowerPoint (apologies to Truman Capote).
-- Alexey Merz (email), September 7, 2006
I've worked at several large software companies ($1B+). At both companies I worked to
help improve business processes, planning, reporting, and decision making.
The unfortunate truth in my experience is that the executive team and their business
managers didn't want more than the pablimized bullets. I sat in one meeting where a senior
executive chastised product management for "not writing business plans." But then, he'd
never asked for one ... so why would anyone ever have written one ... and frankly if
someone had written one, I'm not sure he would have read it.
It's easy to blame the tool ... but I think .ppt is just a symptom of something else. There are
two sides: authentic presentation and authentic listening. We need both.
I appreciate all the work Tufte has done on authentic presentation ... who's working on
authentic listening?
-- Dennis Allen (email), September 12, 2006
One more piece: Lousy PowerPoint presentations: The fault of PP users?
-- Edward Tufte, September 15, 2006
A well designed single page technical report from Science:
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 27 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
-- Edward Tufte, September 25, 2006
POWERPOINT GOES TO IRAQ! Here is an excerpt from today's (September 29, 2006)
New York Times article on Bob Woodward's latest book "State of Denial":
"The book describes an exchange in early 2003 between Lt. Gen. Jay Garner, the retired
officer Mr. Bush appointed to administer postwar Iraq, and President Bush and others in the
White House situation room. It describes senior war planners as having been thoroughly
uninterested in the details of the postwar mission.
"After General Garner finished his PowerPoint presentation -- which included his plan to
use up to 300,000 troops of the Iraqi Army to help secure postwar Iraq, the book says -there were no questions from anyone in the situation room, and the president gave him a
rousing sendoff."
Now, would you not love to see that presentation?
-- Bruce Post (email), September 29, 2006
You may wish to take a look at what Microsoft considers good presentation design:
http://www.microsoft.com/atwork/getworkdone/presentations.mspx
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 28 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
Those who make the tool seem quite out of touch with the needs of those who use it. I've read far better
advice even on sites that uncritically accept PowerPoint as a valuable tool. The slides Microsoft presents as
models, I would present as examples of what to avoid.
In particular, take a look at the 'better' slide at:
http://www.microsoft.com/library/media/1033/atwork/images/getworkdone/55686_375x325_basicbetter2_.jpg
Sigh.
Dominic Brown
-- Dominic Brown (email), October 11, 2006
Dr. Tufte,
I attended your recent one-day course in Chicago. You have advised the 11 x 17 "folded in
half" suggestion to help with spatial adjacency (great for handouts when presenting). I'm
seriously looking at a variety of printers that support printing to larger sizes including 11 x
17 (also known as "Tabloid" size) and its very close relative in the ISO standard world
known as A3 which others and yourself have further commented on aforementioned on this
web page. The prices of nice printers with oversized support seems to be gradually dropping
(such as the HP 2800dtn).
Its one thing to start shopping for a printer that supports oversized paper and duplex
printing, yet another thing to figure out what tool options exist especially those that support
open (non-proprietary) formats which lends well to document collaboration and preservation.
My background is not in design or graphic arts, but I'm determined after attending your
course to become a little more renaissance-like and develop some decent design skills (my
technical writing skills are already very good according to my peers so my goal is to get the
design part down nicely and then let the content flow - no bullet points coming out of my
keyboard). That being said, I'm a big proponent of open source tools because I do not like
having a proprieter being in control of my destiny. I'm also a big proponent of open formats
such as the OpenDocument format. For group collaboration and sharing of knowledge, I
truly believe that the formats have to be open and free of any proprietor's control. In my
opinion, Microsoft serves up the double whammy for 1.) introducing the PowerPoint
cognitive style, and 2.) holding billions of people hostage for decades due to the closed and
proprietary Office formats.
What applications can you recommend that people use such that the open source / open
document format criteria is met, which support decent layout and design of A3 (preferably
A3 since its an ISO standard and not a U.S. - influenced alleged standard) or 11 x 17?
Clearly there are tools Adobe Illustrator but an open source equivalent would be nice (and
one that would work cross platform including Mac OS X, Windows (which isn't going to
disappear overnight) and I would not rule out Linux Desktop (closely monitoring the
burgeoning success of Ubuntu Linux as a Desktop alternative). Document format-wise, it
would seem to me that the W3C proposed standard SVG would be ideally suited for what I
have described. SVG would also be great for Sparklines but the browsers need to catch up
and make SVG standard (and I'm not sure that Microsoft, once again, is committed to SVG
despite Microsoft taking a seat on the W3C's committee). Any further thoughts?
-- Eddie V. (email), October 20, 2006
An easy way to reproduce the 11 by 17 or A3 report (resulting in a 4-page report when
folded in half) is to print out 4 pages of 8.5 by 11 paper and then to copy the pages on a
copier that runs 11 by 17 paper (with pages 2 and 3 on one side, and then in a second pass
through the copier (pages 1 and 4 on the other side). Then fold. We do this all the time.
This is especially good for mass production because rather than using the usual printer
linked to the computer a high-speed copier goes to work. If you are making more than a few
hundred copies of your technical report, then going to a copy shop will prove economical
and fast. They can really bang the copies out. The cost for printing and paper for each copy
is about .12 euros or 15 cents. If you have color images, things get considerably more
expensive. A good use for slideware is to project full-screen color images during your talk.
Since copiers often now take direct digital input, your report can be directly linked to the
copier without the intervening 8.5-by-11 page waltz.
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 29 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
copier without the intervening 8.5-by-11 page waltz.
There are surely straightforward ways to do the page imposition necessary for the 4-page
format in word processors or in any page layout program (but you'll still have to turn the
paper over for the backside printing in most printers).
Can some Kindly Contributors help with this question, suggesting easy and inexpensive
methods for producing the 4-page/1-piece-of-paper technical report? The idea is not to get
caught up in an elaborate computer problem when what's really important is to devote all
your preparation time to perfecting your content.
-- Edward Tufte, October 20, 2006
Maybe a bit off topic, but a simple way of folding a single sheet of paper with content into
an eight page booklet.
http://www.pocketmod.com/
-- Andrew Nicholls (email), October 20, 2006
Excellent example of a public technical report
Here's an excellent public technical report sent to all the customers of the South Central Connecticut Regional
Water Authority. Note the straightforward text, the names of those responsible, and the competent data tables.
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 30 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
4/17/10 10:17 PM
Page 31 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
4/17/10 10:17 PM
Page 32 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
4/17/10 10:17 PM
Page 33 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
4/17/10 10:17 PM
Page 34 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
-- Edward Tufte, October 24, 2006
Coaches Use Laminated Game Outlines for Any Situation
The New York Times
By JUDY BATTISTA
Published: October 27, 2006
Double-sided, 11 by 14 inch technical report--or project management chart--here.
-- Edward Tufte, October 26, 2006
THIS MATERIAL IS FROM OUR THREAD PP AND MILITARY INTELLIGENCE,
WHICH HAS ADDITIONAL CONTRIBUTIONS ON THE SLIDE BELOW.
Central Command Charts Sharp Movement of the Civil Conflict in Iraq Toward Chaos
By MICHAEL R. GORDON, The New York Times
Published: November 1, 2006
See the NYT news story here
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 35 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
Here are some preliminary comments on the slide "Iraq: I&W of Civil Conflict."
It appears that "I&W" means "Indications and Warnings." Replacing the acronym in the
slide title does pep up meaning to outsiders: "Iraq: Indications and Warnings of Civil
Conflict," but maybe it wouldn't fit on the slide.
Only this single slide was leaked (by the military? by DoD?), and so maybe some of the
analytical problems are better handled on accompanying slides. Maybe.
Doing competent political analysis, epidemiology, nation-building, and war planning (all of
which they're trying to do) in a chaotic situation is impossible, and not much good social
science and epidemiology can be expected in chaos and from a military entangled in Iraq. In
real-time chaotic situations, the data-collection is going to be sloppy because people have
more important things to do. (Recall, for example, the gross errors in counts of 9/11 deaths,
as the count went from 6,000 to 2,800 in a few weeks.) And what's taking place is in
profoundly different cultures and in different languages from those of the non-local military
in Iraq. But sloppy data does not justify analytical sloppiness in reporting. In fact, sloppy
data requires greater analytical precision of thought.
The slide reports performance data--a list of phrases, with each phrase accompanied by a
measure of performance. This is what the tables in the sports section, mutual fund page, and
weather page of newspapers do very well. Those designs are much better for reporting
performance data than the slide format here. In sports and stock market tables, each phrase
is accompanied by multiple measures of performance, often over varying time-periods. All
that won't fit on the slide; this suggests that we should use better reporting method than PP,
instead of abbreviating the evidence to fit the slide. As the millions of readers of sports
tables each day demonstrate, people can easily manage large tables of information. Thus
those being briefed in the military should ask: Why are our presentations operating at 2%
of the data richness of routine tables found in the sports section? Let the viewers read and
explore through a range of material; different eyes will search for different things in the
evidence. The metaphor should be the cognitive style of the sports section (or weather or
financial newspaper pages) not the cognitive style of PowerPoint.
There is no cloud of uncertainty or error history associated with the editorializing color. At
times, such color codings suggest an excess of certainty.
The Iraq slide above provides some relevant but thin and overly short-run time-comparisons:
2 arrows on the left showing "change since last week," and the "Index of Civil Conflict
(Assessed)", which sort of compares "Pre- Samarra" with "Last week" and "Current". And
there's a potent time-comparison in words: ". . . violence at all-time high, spreading
geographically."
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 36 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
To get more time comparisons on the 14 "Reads" and "Additional Indicators," 14 sparkline
time-line histories for the last year (week by week, if available) would be useful as a overall
but detailed summary. This would reduce the snapshot tone of the 14 reads and indicators.
In our thread Sparklines: theory and practice, there are (at the top of the thread) data tables
with sparklines that report daily and longterm financial data; one such table shows 14,000
numbers, many of them accurate to only 2 digits (not much for financial data) under the
philosophy of "Try to be approximately right rather than exactly wrong." The short-run
weekly jitters and non-reports need to be smoothed out to see (and compare with)the longrun trends. Weekly data cooperate with the notorious recency bias, whereby way too much
weight is given to the most recent piece of data, just because it is recent. These weekly
reports should be in the context of longer run information to reduce the chances that analysis
will be dancing around only with today's news.
The list style, surely one damn thing after another here, is merely descriptive and thus
preliminary to policy analysis. That analysis might have been done on the other slides or
maybe this report is merely meant as a scorecard. If it is a scorecard, it is grossly
impoverished compared to sports, weather, and financial tables.
The current fashion (it, too, shall pass) in government is the stoplight style (green, yellow,
red), which tends to dequantify data. With categories of this sort, there's always a concern
with how the breaks among categories are chosen and with the meanings of the categories. It
will often be better to provide some evidence or numbers, and then a separate editorialjudgmental color about the number.
The slide contains odd uses of the color-words: for example, a green dot indicating "routine"
next to the exciting phrase "unorganized spontaneous mass civil conflict". Shouldn't "routine
unorganized spontaneous mass civil conflict" be red-critical? After Hiroshima, would
Nagasaki get the routine green dot for nothing different than what happened three days
earlier? It looks like weekly wiggles get too much attention, and longrun levels of
seriousness too little attention on this slide, as chaos becomes routine week by week and bit
by bit. Monthly rather then the sketchy weekly reports might be better for policy analysis.
Or at least provide a monthly aggregations over a period of many months (even the entire
war) in a scorecard along with the weekly incidents.
The leaking of the slide makes a point about the differences between the government's secret
analysis and the public reports by the Administration, a common theme of the insider books
on Iraq policy-making (most recently Colin Powell's book). At some time, "reality must take
precedence over public relations," as Richard Feynman remarked about the shuttle
Challenger accident.
A good many comments by our contributors are on-point but are not taken into account
here.
Note the measurements, definitions, and comparisons to standards in the customer scorecard
in the "Report of the South Central Connecticut Regional Water Authority" (above).
Finally, the over-riding metaphor of all this--"the war on terror"--is a big conceptual
problem. Once it's a war, then it almost necessarily invokes large-scale military action and
searching for a locale (some place, any place, Iraq) for large-scale military action. But
terrorists are more like the Mafia or gangs than they are like armies. Perhaps a better
metaphor is that terrorism is largely a law-enforcement problem (requiring focused and
clandestine local action, informants, endless detective work, detailed knowledge of the local
languages and cultures).
Among the grand truths about human behavior, surely the principles of "the unintended
consequences of purposive social action" and "it's more complicated than that" are among
the top five. Sometimes unintended consequences are largely virtuous or benign (as in
market allocation of goods and services if externalities are mitigated) and sometimes the
unintended consequences are appalling. That's because it's more complicated than that.
-- Edward Tufte, November 2, 2006
It occurred to me partway thru the discussions of NASA presentations that it was not always
that way at NASA. In the 60's, NASA reps regularly visited elementary schools across the
nation. They were good speakers, and they brought many physical props (rocket models,
liquid oxygen, various active electronics). What they did NOT bring was any sort of slides
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 37 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
liquid oxygen, various active electronics). What they did NOT bring was any sort of slides
or overhead projector. Their shows were wildly popular.
-- Carl Witthoft (email), November 14, 2006
At the completion of my undergraduate studies (1999), I had to present a thesis I wrote as a
requirement for latin honors. The thesis dealt with a software API I had written, and I
wanted to demonstrate my work by running an executable program on a laptop connected to
a projector.
Simple as that may sound, I had no success in communicating the nature of my requirements
to the college secretary. The discussion when something like this...
Beau: Can you provide a computer and projector so that I could run a computer program to
demonstrate my work to the audience?
Secretary: You'll have to talk with OTR if you want to do a PowerPoint.
Beau: I don't want to use PowerPoint, I just want to briefly run a program I wrote myself for
the audience.
Secretary (bewildered): You've got to talk with someone from OTR if you want to do a
PowerPoint. Shelly is doing a PowerPoint and she called up OTR weeks ago to schedule the
equipment. You have to check out a laptop with a copy of PowerPoint and a projector. You
can't just ask for these things three days before the presentation... there's a lot of setup that
has to be done!
Beau: Well, if you could just make sure there is a rudimentary PC-compatible computer at
the head of the table I can run my program. The room is small so I guess I don't really need
the projector.
Secretary: You'll have to call OTR if you want to do a PowerPoint.
I ended up using plastic transparencies with screen shots. I was never on the PowerPoint
bandwagon, but from that point on I developed an acute disdain for the product.
I am proud to say that I have never presented a PowerPoint presentation. And I have done a
great many presentations... I have presented profesionally to groups of managers, presented
data at technical conferences, served as MC for large awards banquets, etcetera.
On some occasions I have used HTML to present information. It lends itself remarkably well
to this task, especially with the browser in full-screen mode.
As far as the simple bullet / agenda type data that PowerPoint handles well is concerned, I
just pass out hard copies. It is amazing how few people do that.
If I want to show a bunch of pictures in sequence, I'll either use a slide projector of I'll just
open the pictures up in sequence, e.g. from Windows Explorer. If the content of my
presentation won't stand on its own merit despite the fact that I (unprofessionally) had
Explorer open in front of the audience for a few seconds, then there's a larger problem that
cannot be solved by PowerPoint.
To me, PowerPoint is yet another example of a phenomenon that has become epidemic to
the software industry: the answer to a question no one asked.
-- Beau Wilkinson (email), November 15, 2006
I've taken your suggestions to heart in the presentations I've given recently, and I thank you
for the good advice.
My questions is: how can this advice be applied to a multi-day training course? I'm about to
design a three-day technical course, and am determined to not assault the students with days
of PP junk.
My plan right now is do divide the course into 3-hour sections, each with a single-page
handout. Slides would be mostly graphics. But I'm concerned how students will react to the
lack of a text book (yes, usually just a set of slide printouts). I'm also trying to design the
course so that others can teach it, but the lack of text slides makes this more problematic.
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 38 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
course so that others can teach it, but the lack of text slides makes this more problematic.
I'm guessing I'll need a teaching guide that contains a prose walkthrough of each section.
In any case, I'd just like to get some opinions about how to best present a multi-day
technical course.
-- Patrick Paulin (email), November 30, 2006
The above advice about the 4-page technical report applies to multi-day courses as well.
You probably should average about one 4-page technical report per hour or two; that report
should contain graphics, tables, text, and whatever it takes to explain something. Use a slide
projector only for full-screen color images and videos. Use the handout technical reports for
everything else. In short, your presentation program is Word not PowerPoint. Use slideware
only as a projector operating system for full color images and videos.
It is useful to hand out all the day's technical reports at the beginning. If your audience goes
through them before you get to the material, you're already a success and you should be
thankful--for you have an alert active audience using their own cognitive style to look at
your stuff.
-- Edward Tufte, December 1, 2006
An opportunity missed
Here is an unfortunately typical recommendation: that data graphics intended as feed-grain
for talks presented to nonscientists be stripped of (among other things) error bars.
The better option is, of course, showing raw data along with the means (or other markers of
central tendency). This would not needlessly complicate the presentation, and it would allow
even the untutored to evaluate consistency or variability in the data - presumably, key
parameters in a study of human behavioral responses, the example shown.
If the study is a good one, the raw data will underscore that the conclusions presented sit
atop a substantial observational foundation. As Dr. Tufte and many others have pointed out,
any audience that can understand a typical sports or business page is going to be
underwhelmed by the data-paucity of a typical data-summary bar graph. A dozen dinky little
earthtone bars? You fellas got grant support for that?
In short, graphs of raw data clarify how observations are accrued and interpreted: the very
mechanics of science. Scientific process - every bit as much as the conclusions - should be
the central goal of communicating science to a lay audience. Otherwise, the growing fears
and suspicions that science is merely an empty belief system are reinforced.
-- Alexey J. Merz (email), February 7, 2007
Agreed.
The goal of the summary statistics and consequential graphic must be to show the
distribution of the responses. Measures of central tendency rarely tell the whole story
(excepting of course when discussing binary data). What are needed are summaries that tell
us the whole story; if the mean doesn't do the trick, then we need more.
The data (through the order statistics) are always jointly sufficient statistics, regardless of the
underlying distribution. That's why showing all the data works. Consider this as an example
displaying the perils of using only rudimentary summary measures. In general, error bars
that were removed in the above blog are a poor solution to showing the distribution. At least
for continuous data, boxes and whiskers, or the like, must be greatly preferred; the ink on
the page ought to map us to the data. 'Dynamite plots', showing means as thick bars and
standard deviations (or standard errors?) as error bar whiskers, violate this principle: at the
bottom with have ink that represents no data; then moving up, we have ink that represents
data; above the bar, we have blank space that represents data; then way up high, we have
blank space that represents no data. No wonder people are confused.
In the end, we are quite rarely interested in the measure of central tendency, regardless of
how easy it is to compute. What matters is the distribution; show the atoms.
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 39 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
Rafe
-- rafe donahue (email), February 8, 2007
I've read (or, at least, skimmed) the range of cogent comments and responses on the topic of
over-utilization of PowerPoint and am surprised to find little (if any) mention of the factor
that constitutes my own violent objection to its prevalent use, that being that increased
reliance on PP and other visual aids in education, training, project and crisis management
fundamentally undermines the active involvement and cognitive engagement of the target
audience/students/team members by converting the desemination of information to a passive
presentation. I'm by no means anti-tech and absolutely could not do my job (major
infrastructure electrical engineering and project managment) without the transforming
efficiency of CADD, CPM scheduling, electronic communications and spreadsheets.
However, I consider it my good fortune to have been educated, through college, in the prepersonal computer (and PowerPoint) era of blackboards and handwritten notes. Is anyone
aware of any formal studies on the pedagogical effects of wholesale conversion of
knowledge transmission from the "active" lecture/blackboard and individual note-taking
model to the "passive" distribution of pre-written and pre-organized information? To
illustrate my, perhaps, unclear point: imagine if the NASA briefing on the renegade foam
chunk had included, not a series of well-intentioned but arguably biased and confusing PP
"slides" but, instead, a presenter with a handful of notes reading aloud the issues of concern
and outlining them on a black or whiteboard. I guarantee that the convened participants
would have been actively: (1. taking more copious notes (hence, paying closer attention) (2.
asking questions and requesting clarifications (3. organizing and formulating their OWN
heirarchies of concern (4. forcing, by their active participation and feedback, the presenter to
adjust and expand his blackboard presentation and illustration of the issues to incorporate
the group feedback.
I would further argue that an individual's depth of comprehension and longterm functional
recall of any information is greatly enhanced by the simple act of physically transcribing it.
When people are provided, as passive recipients, with pre-organized information and know
they will have access to hard or electronic copy of same, they do not intellectually absorb
nor process it at the highest level nor are they as motivated to focus their attention.
Anyone can test this on themselves (I have.) Next time your organization mandates noncritical HR or IT informational "training" , have them run sessions with and without PP or
other graphic aids and handouts and require the attendees at the "without" sessions to take
notes. Collect the notes at the end and give both groups the same quiz on the topic. Care to
guess the results?
The outcome can be even more profound in crisis management meetings (again, I've tested
this). PARTICULARLY in a crisis situation the last thing you need for dynamic and
creative problem-solving is information distilled/arranged/codified/biased by one or a few
people (which is what use of PP inescapably tends to do) and presented in a format that
encourages intellectual passivity.
-- Kerry Parslow (email), February 9, 2007
Iran influence in Iraq - the 11 Feb briefing
http://www.talkingpointsmemo.com/docs/iran-in-iraq/?resultpage=1&;
-- David Person (email), February 12, 2007
As a high school student, PP is by far the most common way my peers give presentations.
While some take the slides as an outline for a more meaningful conversation with the class,
others read the bullets verbatim. I guess part of the problem with PP being used
professionally is that it's inculcated as the most efficient at a very young age. This is one
problem.
A more troubling problem is that my physics teacher teaches the class with PP slides. In
some ways I feel cheated, and I can tell the difference because I've been fortunate to take the
same Physics class at University. The University professor taught with a chalkboard, and, as
cheesy as it sounds, really brought the subject to life and made it interesting. Even though I
understand there is a difference in quality for high school teachers and college professors, I
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 40 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
understand there is a difference in quality for high school teachers and college professors, I
feel that I was taught Physics by the professor, while I am self-learning the subject that I am
taking in high school. It's even more disconcerting to find that some college professors (I've
seen an MIT physics class with PP notes) are beginning to use PP slides to teach. In a
rigorous mathematical course like Physics, I feel that it is a mistake.
But, I'm only a student, what can I do?
-- High school student, February 24, 2007
How is PP like a software house?
I have just finished reading the PP essay, which I thoroughly enjoyed and agreed with, but
one statement in it bothered me. In mentioning the history of PP, you make the claim that
the structure of PP presentations mirrors that of a software house; I fail to see the reasoning
here. If you're referring to the hierarchical division of a software house into departments and
so forth, than how does this differ from the structure of, say, a university? On the other
hand, the operation of a software house (not to mention software itself) is a very complex
thing, and surely not a model for dumbed-down presentations.
-- Erez Volk (email), March 18, 2007
For what I wrote about PP's metaphor, see our thread Metaphors for Presentations: Conway's
Law Meets PowerPoint.
The opening section of that thread is from my 32-page PP essay.
For more on design and bureaucracy, see my Visual Explantions, pages 146-149.
-- Edward Tufte, March 18, 2007
Related to my post on Dr Vigh's zoomimage use of Quicktime VR of 9 December 2005, the
Raskin Center has a similar demo. Of note is that the Raskin Center's demo is based on the
small web format (swf), so more people can use it. Quicktime is not natively supported on
Linux and other open-source distributions. The Raskin SWF also implements the obvious by
putting many thoughts on one page.
-- Niels Olson (email), May 24, 2007
Other PP threads
NOTE: Above is one of our major threads on PowerPoint. The substantial PP threads on this
board are:
PowerPoint does rocket science--and better techniques for technical reports
Account of the role of PP in the shuttle Columbia accident,
followed by many good alternative methods and examples for technical presentations.
Cancer survival rates: tables, graphs, and PP
Comparisons of methods for presenting cancer survival rates.
Plagiarism detection in PowerPoint presentations
An intriguing but under-explored topic.
PowerPoint and military intelligence
Mainly recent examples of leaked PP slides in the Iraq war.
Metaphors for presentations: Conway's Law meets PowerPoint.
Teaching and scientific papers are better metaphors for presentations
than marketing and computer programming.
Apple's Keynote vs. Microsoft's PowerPoint
Don't get your hopes up.
Lousy PowerPoint presentations: The fault of PP users?
A look at a rich and complex question:
What are the the causes of presentations?
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 41 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
What are the the causes of presentations?
A detailed analysis of PP is my 32-page booklet The Cognitive Style of PowerPoint:
Pitching Out Corrupts Within,
which also appears as a chapter in Beautiful Evidence.
-- ET, May 29, 2007
Teaching : blackboards, tablets, and powerpoint
I read something recently which I thought you might like:
http://www.upscale.utoronto.ca/PVB/Harrison/BlackboardPptTablet/BlackboardPptTablet.
pdf (linked from http://www.upscale.utoronto.ca/GeneralInterest/Pedagogy.html )
This is a study of how to teach physics to a large class, either by using multiple blackboards
(the traditional way, but hard to enlarge for a huge room) or powerpoint (a disaster, of
course) or by writing on a tablet PC.
They understand clearly what I haven't seen said elsewhere: first, that having the last few
minutes' material still visible is crucial, you're always a little behind the speaker and need to
refer back. Flipping back in powerpoint doesn't work, having a printout doesn't work well,
having the original still visible is wonderful. They achieve this with the tablet by having
multiple screens and using them in turn.
Second, that making the lecturer write in real-time provides a useful limitation on how much
detail can be included, forcing him (or her) to focus on what's really important, and also
communicates all sorts of extra clues about how he's thinking about the problem. For
instance, one often writes an equation in a several steps, starting with what's important and
then working backwards to fill in minor numerical factors and such. This is invisible in the
final result (unless elaborately faked in powerpoint) but unavoidable writing it by hand.
-- Michael (email), June 13, 2007
The first half of this TED talk is very reminiscent of Dr Vigh's zoomimage and the Raskin
Center demo above; then Blaise Aguera y Arcas takes it to the next level by reconstructing
and navigating the Notre Dame cathedral using Flickr imagery.
-- Niels Olson (email), June 24, 2007
PowerPoint Does Tunnel Design
I saw today that the NTSB announced their suspected cause of the Big Dig tunnel collapse. I
teach writing for engineers and I wanted to see if the reports of their investigation would be
useful for my classes.
Though they do have a synopsis written in sentences and paragraphs all of their
presentations are in PowerPoint:
http://www.ntsb.gov/Events/2007/BostonTunnel-MA/presentation.htm
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 42 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
Another example for my students on how not to present evidence. Will these government
agencies ever learn?
Bill Wolff
-- Bill Wolff (email), July 10, 2007
I work at an Insurance company and I can't tell you how much I enjoyed the essay and the
comments -- A cavalcade of ...what? I'm not sure what to call it. In my blog, Rhetoricia, I
wrote about our PPT practices in Biting the Bullet, but it's not very serious -- and there is
much to say. I'll be mining these links for some time. Thanks to Anonymous, who directed
me here.
-- Abby Shaw (email), July 25, 2007
Update to link on October 23, 2005 post
External Tank Tiger Team's Interim Report, Fact Sheet
http://www.nasa.gov/pdf/136219main_FS_ET_Tiger_Team_Report.pdf
ET Tiger Team Report - Part I : Interim Rev C 07 Oct 05 80 pages
http://www.boeing.com/defensespace/space/returntoflight/reports/136149main_ET_tiger_team_report.pdf
-- Wiley Holcombe (email), October 27, 2007
More matter with less art
In the current Communications of the ACM, Robert Gaskins essentially recommends that
most people not use chartjunk. I'm not sure how this advice, retroactively applied, would
have helped the NASA slides, which are very plain but highly hierarchical and confusing.
http://www.gbuwizards.com/files/gaskins-ppt-at-20-cacm-vol50-no12-dec-2007-p15-p17.pdf
-- Jason Catena (email), November 30, 2007
How can PowerPoint be used for serious documentation?
Recall from my essay that opens this thread that NASA used PowerPoint as engineering
documentaton for the shuttle Return to Flight project (about $2 billion).
The dangers of PP for serious work are indicated in this CNET/NY Times report "Office
2003 Update Blocks Older File Formats:" here.
This is particuarly poignant because today (Janaury 4, 2008) NASA was undertaking repairs
on the shuttle liquid fuel tank that was the major element in the Return to Flight project.
Was the liquid fuel tank guage connector--today's issue, which has delayed the current
launch for about a month--documented in PP?
[Update: Microsoft grants access to your files]
-- Edward Tufte, January 3, 2008
PowerPoint Does Science - Not
Reaching the end of a career in bureaucracy, I have experienced the rise and rise of PP and
have sat through thousands of PP presentations, most of them about science.
For presenting images and movies, where seeing something really is better than just hearing
about it, and pictures help the audience understand, PP is mighty convenient.
But words on slides? Fergeddaboudit. Too many people still read from their slides, so
boring when you've already read the words on the whole slide while they are getting to the
middle of the first point. You end up hating them for torturing you this way. If the slides
have only a few words, they just distract you as you try to make the connection between
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 43 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
have only a few words, they just distract you as you try to make the connection between
what you're hearing and what's on the slide.
And those 70-slide presentations - scientists always want to say more than you want to hear.
You want to shoot yourself at around slide #10. Orange text on magenta background - PP
presentations provide irrefutable evidence that colour-blindness is more prevalent among
scientists than in the general population. Teeny tiny invisible fonts. Indecipherable charts.
Laser pointering each word. PP doesn't make bad communicators better - it just gives them
the tools to be truly ghastly.
If people say: "Can I have a copy of your slides?", it's no compliment. You should have
come prepared with a proper handout to take home instead.
-- Catherine Kraina (email), February 27, 2008
As a former software engineer who now does mostly software sales and client relationship
management, I have been subjected time and time again to demands for PP slide decks.
100% of these demands come from business partners and executive management. 0% come
from current or potential customers.
Honestly, how often has your audience begged for PP?
Given the choice, I never use PP. Instead, I show actual working software and have
analytical conversations about it and the needs of my clients. That this is vastly more
efficacious should come as no surprise to readers of this thread.
PP at its best is a meeting agenda. 'Never go to a meeting without an agenda'. OK, so I
produce and distribute an agenda ahead of time and pass it out at the meeting.
As a bonus, people can write notes and questions on their hard copies of the agendas.
Displaying said agenda in luminous leaden lockstep with PP is a good bit less than useless;
it steals time and energy from human-to-human interaction.
So, I heartily encourage our competition to continue using PP!
-- Andre Lockhart (email), March 5, 2008
A very *modest* contribution, regarding the matter of typing in all caps: The explanation
may be as simple as the fact that not everyone can touch-type. People get scolded for
"shouting" online when I'm pretty sure half of them do it because switching between caps
and lowercase letters would be so slow they could never finish a thought. It happens less
now, as everyone learns to use computers, but I remember back in the Stone Age, when I
was a secretary, it was considered *unmanly* in many business circles to know how to
type! Seriously.
Thank God we've gotten past *that*!
-- Suzie Elliott (email), March 18, 2008
OTHER THAN ALL CAPS
but the natural default would be all lower case, not all caps.
i get email routinely these days in all lower case.
maybe ALL CAPS doesn't get through the spam filters anymore.
-- edward tufte, March 20, 2008
How about this for analysis by powerpoint?
http://www.epa.gov/climatechange/downloads/s2191_EPA_Analysis.pdf
Nearly 190 pages of powerpoint slides by the EPA that clearly should be a technical report.
Given the weight of the topic (Carbon legislation) and its impact upon our environment and
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 44 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
Given the weight of the topic (Carbon legislation) and its impact upon our environment and
economy.
There are so many opportunities to discuss bad information design within this document. I'd
love to hear thoughts from this Forum on the "report".
-- Lee (email), March 25, 2008
EPA POWERPOINT
Lee - that is awesome. It's a parody, right?
HAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHA
-- Catherine Kraina (email), March 26, 2008
As Lee says, there are many opportunities to discuss bad information design. I'll just
mention a few that occur to me immediately.
1. The unsuitability of PowerPoint (or any sort of presentation software) for this sort of
information is so obvious that it's not really worth discussing.
2. Typographical ugliness. They packed so much text onto slide 6 that the subscript in the
bottom line is almost lost. Perhaps they forgot that black isn't easy to read against a
dark blue background.
3. Come to that, the blue frame on every slide is an example of what ET calls chartjunk
4. Also on slide 6, what is the merit of abbreviating "and" to "&" and "continued" to
"con't" when there is room to write them in full?
5. Slide 18. All that white space, but not enough room, apparently, to spell out
"Greenhouse gas".
6. Slide 19 illustrates something that I hate. Instead of labelling the different bands on the
bands themselves (if necessary with little arrows) you have to refer to tiny little
coloured squares at the bottom of the slide. It's not obvious to me why the blue for
CO 2 is not uniform. Less noticeably, the other colours also fade as one moves to the
right: is this supposed to mean that the figures become more speculative towards the
right? If so, where does it say so? As they seem to know how do subscripts, why don't
they use them in CO 2 etc. at in the key to the colours?
7. Slide 29 is a better illustration of the problems with forcing the reader to go
somewhere else to find out what the colours mean. When 20 different colours are used
it is unrealistic to expect people to be able to match the key to the graph. Speaking for
myself (though I think others on this board will agree) I prefer much more sobre
colours than those used.
8. Slide 31. If my interpretation above of what the non-uniform colours mean is correct,
are we to assume that the amount of international credits is almost unknown for 2012,
but very precisely known for 2050?
9. Slide 61. In a long a complicated table faint grey backgrounds can be useful for
guiding the eye along a long line, but this table isn't complicated enough for that to be
an issue, and the backgrounds are too dark.
10. Slide 67. Although more or less the same brownish orange is used on both the left and
right halves of the slide, it means "allocated allowances" on the left but "States" on the
right. (The green is more logical, meaning "Auctioned allowances" on the left and
"Auctions" on the right.)
11. Slide 76. Did the people who prepared these slides not have outline maps of Alaska
and Hawaii, or did they think those Sates have little importance? Although in this slide
they mostly use nice sobre colours, the boundaries between the States would be better
in white.
I think that's (more than) enough from me.
-- Athel Cornish-Bowden (email), March 26, 2008
I looked at the EPA greenhouse gas presentation closely enough to learn only this: In slides
104 through 189 you will find the six appendixes.
That's right, the appendixes.
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 45 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
To a slide show.
-- Cliff Tyllick (email), March 26, 2008
Can you imagine sitting through a presentation of all 189 slides? And imagine that the
presenter is as bad at public speaking as he/she is at preparing PowerPoint slides. I cringe.
-- Miklos Z. Kiss (email), March 27, 2008
Every college student for the last decade can imagine sitting throught 189 powerpoint slides.
They sit through that much almost every day.
-- Niels Olson (email), March 29, 2008
Over 20 years in business and as a strategy consultant, I've seen lots of horrible powerpoint
in Australia. And I freely confess lots of it was mine. While I understand the process of
writing good business plans and information memorandums (which I do as key part of my
consulting work, they are the business equivalent of scientific technical papers), I had failed
to make the same magic happen in powerpoint.
I understand the power of the argument "just don't use the tool", however it is a key part of
business decision making processes. How convenient were it that George Orwell's
propaganda 'Ignorance is Strength' were true and I could ignore it - but I can't. So I decided
to try and find a way of making the process better, for me and for my audiences. I had heard
of Extreme Presentation by Dr Andrew Abela, http://extremepresentation.typepad.com/ , and
in desperation one evening as I faced my own unedifying, long, confused, muddled PP pack,
I took the time to go and visit the site and read the blog. It helped. So much that I flew from
Australia to Atlanta, Georgia to do the one-day course. I have had to let go of lots of old
baggage about how to use powerpoint. But that wasn't what helped me most - it was the
discipline of organising the argument, understanding the problem, the audience and the
medium's limitations (PP) before writing a single slide.
For me, Peter Kaplan said it best in his October 7, 2006 comment (which I can't find now I
want to reference it...) that his good presentations were when he thought about the problem
and its relationship to the data BEFORE writing PP slides, and his worst were when he
started with the Header Slide. That has been my experience too. I was a poor user of PP
because I had bad habits in using it, and did not understand and respect its limitations for
engaging an audience. I used it to try to cover my weaknesses. These days I try to get the
argument and evidence right and then choose my presentation tools - which may include PP,
but not always.
After reading the thread of all the arguments, I feel the need to confess, as the PP user, that I
have (hopefully now in the past) contributed to the problems of PP as a poor communication
tool. Thanks from DownUnder, for the examples and further background on PP limitations.
Willow
-- Willow Forsyth (email), April 2, 2008
Working in Higher Education, I agree that PowerPoint is almost always pure evil.
At a confeence a few years ago, the speaker (I believe it was Graham Webb) started off with
the usual "death-by-bullets" presentation and then about halfway through he acknowledged
that this was completely worthless and from then on all the slides were holiday pictures of
his family! The audience immediately warmed to him and listened to everything he had to
say. I just wish the hundred or so presenters who had preceded him that week had been half
as original...
-- Daniel (email), April 18, 2008
It's not only scientists who abuse powerpoint, but lawyers too.
Recently, I gave a one hour presentation on presentation skills for lawyers, in Melbourne (no
need to fly to the US for one day of training to receive training in how not to abuse
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 46 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
need to fly to the US for one day of training to receive training in how not to abuse
Powerpoint).
Even though I had mentioned I would be using my Macintosh and not be providing my
slides in Powerpoint for the convenience of the organisers, on the day this came to nought.
Indeed, it came at great personal expense to NOT conform and use something other than
Powerpoint (Apple's Keynote).
My presentation was due on the last day of the 4-day conference at a leading Melbourne
convention centre, where lawyers were attending for mandatory professional development
points. It was placed under "Business skills" and was booked out a few days before I was
due to present.
Two days before, I attended the venue to view the room, and seek answers to some basic
A/V issues I required: wireless mic for roaming, VGA cable to allow me to hook up my
Mac at the front of the room, assessment of the projector's resolution, and placement of the
screen.
I also sought permission to view a live presentation just to confirm my suspicions about
how lawyers present, believing they, being knowledge workers, would be no better than my
colleagues in psychology. I was amply rewarded by what I experienced to convince myself
that my presentation - challenging the social conformity and tradition of using the cognitive
style of powerpoint - was likely to be "on the money".
The A/V people had convinced the organisers that all slides should be made available to
them as a Powerpoint file by all presenters to load onto an IBM Thinkpad at the back of the
hall, hooked into a mixing panel for recording purposes. This was allegedly to speed up
presentations in a seminar of several presenters. My small request to use my own laptop and
speak not from a dais above the audience, but on the floor with the audience, produced a
"Shock, horror, but we can do it for a fee" reaction.
In reality, all it needed was a stretch of VGA cable, and a length of audio cable (all
inexpensive standard equipment surely at the disposal of a top flight convention centre).
But no. So fixated were they at doing things one and only one way, to speed up the
transition of speakers on a multi-speaker panel (even though I had an hour to myself
following by morning tea of 15 minutes) that the A/V company had to "hire" in special
equipment to hook up my Mac, whose cost was to come from my fees. (Oh, plus $1500 for
the morning tea that followed which also came from my fees.) I was quite castigated by the
organisers for upsetting the apple cart, so to speak, by using my own equipment. It was to
be the Powerpoint-way, or take the highway. My toll for taking the highway was a $750 fee
reduction from my speaker's fee for the "special equipment" hire cost.
At session's end (it was a fast-paced hour's consolidation of a full day workshop I conduct) yes, Mr. Tufte and Ted.com was given centre-stage for a good proportion of time - people
came up to me to thank me for showing another way of presenting, and not killing them
(their words) with powerpoint. Even though their daily work lives are filled with words,
when it comes to presentations, it seems even lawyers have their limits.
I will be most curious to see the evaluations, which appear to be taking a long time to get to
me. Even if they are high, somehow I don't imagine I'll be invited back, despite delivering a
"full house - SRO" performance. Pride before the fall.
-- Les Posen (email), April 19, 2008
I have been to numerous software engineering conferences over my long career and where
my primary objective is simply to learn about new technologies, languages and tools which
I can leverage in my software development work. The conferences I generally have attended
include several concurrent tracks which focus on specific content areas such as testing,
languages, methodologies, tools, etc.
As a personal rule, if the talk begins with a powerpoint presentation, I will generally get up
and explore the other rooms and find out if there are better presentation being given else
where. My personal experience is that where engineering related information is being
presented, I learn much more from the presenters who simply use the development tool or
language during the live presentation explaining how something works by showing it to the
audience. Unfortunately, at work where attendance may not be optional, I simply must
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 47 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
audience. Unfortunately, at work where attendance may not be optional, I simply must
endure the heavy use of PP along with all the other engineers required to attend these
meetings. So, basically where I am empowered to make a choice, PP does not stand a
chance of holding my attendance.
-- Richard Cunday (email), May 14, 2008
After completing a PowerPoint presentation for a meeting tomorrow, I felt impelled to
search for Tufte's words on the problems of the cognitive style of PowerPoint.
Amen!
I wish that I could find examples of project post-mortems from industry where the cognitive
style of PowerPoint contributed to the failure of a project. Dead astronauts are one set of
things, but showing that a company has lost money would really catch the attention of
corporate executives.
I strongly suspect that such examples could be found by a suitably motivated industrial
sociologist, with access to company email archives.
In my own experience I suspect that it is not a coincidence that the last major advance in
CPU microarchitecture, the transition to out-of-order execution with the Intel P6 (and AMD
K7, etc.) occurred just prior to the transition to PowerPoint.
Tufte, and many others, describe well how PowerPoint is inferior to formal or informal
technical reports. Complex ideas are just too darned hard to fit within the constraints of a
PowerPoint slide.
I'd like to mention additional problems with PowerPoint:
In many cases, PowerPoint presentations have replaced brainstorming, informal technical
discussions. Instead of a group of engineers meeting by a whiteboard, to flesh out ideas via
drawings that they later document with more formal written descriptions and more accurate
drawings, it has now become commonplace for someone to make a proposal as a et of
PowerPoint slides. Thus, a meeting of peers discussing an idea has been changed into a
presentation, with a proposer and a set of reviewers.
Or, perhaps more than one proposal is prepared as PowerPoint slides. Thus, instead of
cooperation, we start off with competing proposals; as often as not, competition between
different organizations within a company.
A simple trick can help here: use two slide projectors. Project both proposals (if there are
only two) side by side. Compare and contrast - informally, not as a hatchet job. Merge.
Unfortunately, two slide projectors in the same conference room is quite rare; more often
than not, the PowerPoint slides are not projected, but are viewed upon computer screens,
and there is seldom enough screen area to display two PowerPoint presentations
simultaneously. (That's another cost: you can split screens horizontally to display sections of
text frm two different reports simultaneously, but this doesn't work with PowerPoint unless
you have very good eyes, or are fortunate enough to have multiple monitors.)
The rise of PowerPoint seems to be related to "virtual teams" - organizations that are
geographically dispersed, e.g. with team members in Oregon, California, Boston, Israel,
India, etc. We so often attend phone meetings for such virtual teams. Lacking good virtual
reality conference rooms, we listen to each other on the phone. We may use conferencing
software such as NetMeeting or LiveMeeting, but overall it is usually just voices on a
telephonce conference call, with PowerPoint slides on people's screens. PowerPoint slides
provide a way of focussing such a conference call. Perhaps PowerPoint is a necessary evil
for such virtual teams. But I wonder if this justifies the productivity lost due to the
shortcomings of the cognitive style of PowerPoint.
A final point:
(Let's see how much trouble this gets me into.)
I have spent most of my career at Intel, in circumstances that made me especially aware of
the transition to PowerPoint.
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 48 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
But I also spent 2 years, 2002-2004, at AMD, during a period of team in which AMD was
widely perceived as being ahead at Intel (no credit me). I observed that AMD was
significantly *behind* Intel with respect to its adoption of Microsoft tools such as
PowerPoint and Word. Going from Intel to AMD in 2002 was like going back in time 7
years at Intel, to a time before PowerPoint. Most AMD engineers used UNIX tools and
FrameMaker. Virtual teams where less common at AMD, and less needed given AMD's
smaller size; but where they occurred AMD used videoconferencing. I attended
videoconferences on almost a weekly basis at AMD; at Intel I have not attended a
videoconference in more than 3 years.
However, I did observe, around the time I left AMD, eventually to return to Intel a few
months later, that PowerPoint and other Microsoft tools were being rapidly adopted by
AMD.
I suspect that it is not coincidental AMD has, in the intervening years, lost what at the time
appeared to be a commanding lead over Intel in terms of technology.
I.e. I suspect that Intel adopted the cognitive style of PowerPoint earlier than AMD, and
suffered a number of bad project decisions that led to AMD "taking the lead" over Intel. But
AMD eventually adopted the cognitive style of PowerPoint, and I suspect that this, in part,
has led to some of the problems that have allowed Intel to regain the lead. I.e. I suspect that
both Intel and AMD are equally crippled by the cognitive style of PowerPoint now.
If the cognitive style of PowerPoint is so bad, why is it so successful? I hypothesize that
PowerPoint is bad for the organization, but good for the PowerPoint presenter. PowerPoint
looks good. If a PowerPoint proposal is competing against a proposal embodied in a
technical report, other things being equal the PowerPoint proposal is more likely to win, and
the PowerPoint user more likely to be promoted. If a PowerPoint proposal is competing
against a proposal that contains both PowerPoint and a formal tech report, one might hope
that the latter might win; but if the same manpower is used for each, the slides-only
proposal will be more polished. I.e. the payoff per hour spent is higher with PowerPoint
than it is in technical writing - from the point of view of the proposal writer, although not
the organization.
This might be somewhat ameliorated by making it easier to prepare both a presentation and
a technical report. Back in the pre-PowerPoint days, I used Framemaker templates and
conditional text to accomplish this, allowing me to prepare slides and a technical report from
the same document source code. Unfortunately, I do not know how to do this with Word
and PowerPoint; perhaps it can be done with Visual Basic. I have hopes of using wikis to
accomplish this unification of reports and presentations, and have made some progress; but
it is by no means a finished toolchain.
In the meantime I use PowerPoint; I hate it when I vae to distort my ideas to fit into
PowerPoint; and I woonder what incorrect conclusions I may be drawing when I receive
PowerPoint.
- Andy "Krazy" Glew
"Mr. P6", the OOO, SpMT, and MLP guy, Computer architect (currently at Intel)
-- Andy Glew (email), May 20, 2008
Regarding ET's post: "Can some Kindly Contributors help with this question, suggesting
easy and inexpensive methods for producing the 4-page/1-piece-of-paper technical report?"
For a recent presentation on converting paper documents to hypermedia, I experienced some
challenges printing an 11x17 layout; I hope these suggestions prove helpful to others.
Software: a combination of MS Word, Adobe Photoshop, and Adobe InDesign. For free
options, consider OpenOffice, GIMP, and Scribus. Inkscape also looks interesting.
Workflow: edit the intro in MS Word; edit graphics in Photoshop; create two-page 11x17
layout in InDesign; import text and graphics to InDesign; save as print-quality PDF.
The PDF was copied to a USB flash drive and taken to Staples for printing. They were able
to print a test page and complete the entire run while I waited for ~10 minutes. Cost was
reasonable and the quality was high.
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 49 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
Speaking of quality: I attempted ET's above suggestion to print once and then copy multiple
times. However, none of the self-service copiers were able to reproduce light-colored
"callouts." Varying the lightness/darkness on the copier would force the callouts to either
disappear or display too heavily. The prices at some big-box copyshops are similar for selfservice copying and full-service printing. Be sure to check around, as the machines behind
the counter will produce much higher quality prints.
Readers of the presentation will note that I took ET's advice to "steal from the best" in the
center spread callouts.
-- Travis Thompson (email), June 20, 2008
ET: "Can some Kindly Contributors help with this question, suggesting easy and
inexpensive methods for producing the 4-page/1-piece-of-paper technical report?"
pdfnup may do what you want. For those using macs (and who have LaTeX installed)
PDFNupMaker does the same thing with a graphical interface.
BW: "On some occasions I have used HTML to present information. It lends itself
remarkably well to this task, especially with the browser in full-screen mode."
Plainview is a full screen browser (Mac only). As well as allowing web browsing in the
normal way, it has a presentation mode in which a hot key can be used to step through a
series of preselected webpages.
-- James Scott-Brown (email), August 12, 2008
In this week's New Yorker magazine, there is a profile of General David Petraeus.
http://www.newyorker.com/reporting/2008/09/08/080908fa_fact_coll
Disciples of Edward Tufte will wince (or worse) upon reading this paragraph:
Petraeus is a professional briefer, and with a PowerPoint slide before him he will slip into a
salesman's rapid-fire patter. He illustrates his remarks with a laser pointer; he will swirl a
bright dot of emerald light around a particular sentence fragment until a listener risks
succumbing to hypnosis. Petraeus and his staff will discuss at length the shading of colors
on a slide, or the direction of arrows depicting causality. When I asked, in a skeptical tone,
about this passionate use of PowerPoint, the General responded in the staccato of the
medium: "It's how you communicate big ideas--to communicate them effectively."
I'm hopeful that someone(perhaps ET himself) will come up with a good letter to the editor
on this one. He certainly tee'd it up for us.
-- Roderick Jones (email), September 7, 2008
For those struggling to make A3/B-size layouts:
Install a PDF writer if you don't already have one. CutePDF (Windows) works well and is
free.
Start your word-processing or page layout program (Word, Writer, CAD, etc. For maximum
irony you can use PowerPoint and use the drawing tools, text boxes, etc.!) and select the
PDF writer as the printer. You should now be able to select from a wide variety of paper
sizes. Select the size you want.
Do your layout.
Print to PDF and check that the page layout turns out OK.
Print from Adobe Acrobat, select the paper size you want and select "scale to fit paper".
You can proof it on A4/letter and email it out to a print shop if required.
-- Donncha Butler (email), December 29, 2008
Use sentences, not bullet grunts
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 50 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
He Wants Subjects, Verbs and Objects The New York Times, April 26, 2009. Richard
Anderson, the chief executive of Delta Air Lines, strives to run efficient meetings and look
for the intangibles of leadership when hiring executives.
http://www.nytimes.com/2009/04/26/business/26corner.html
-- Edward Tufte, April 27, 2009
Perhaps the Delta Airline executive wants subjects, verbs, and objects from his team. His
own speech, however, is not inspiring, interspersed as it is with "you have to...", "you want
to...", and "you know".
I also noticed his emphasis on problem solving, with no acknowledgement of how important
it is to first understand the problem or, more likely, the system of interrelated problems. I
would refer him to the writing and thinking of Ron Heifetz (adaptive vs. technical
challenges) and Russell Ackoff (systems thinking).
-- Steve Byers (email), April 30, 2009
PowerPoint does blackholes?
John Wheeler, eminent physicist, died in 2008 at the age of 96. To commemorate his contributions to physics, astronomy,
cosmology, quantum mechanics and more, the April 2009 issue of Physics Today contains several wonderful articles about
Wheeler.
The article, "John Wheeler, relativity, and quantum information", by his former-students Misner, Thorne and Zurek (all
prominent physicists themselves, now) contains this photograph taken by Kip Thorne. The caption reads, "John Wheeler
lecturing at a conference in Cambridge, UK, in 1971. Wheeler's style was to cover the blackboard with inspirational coloredchalk diagrams and phrases before the lecture, then work his way through them, one by one."
The blackboard looks like a precursor to today's PowerPoint presentation, but not poor PowerPoint full of bullet grunts. Instead,
a sequence of diagrams and key phrases to guide the audience, and the speaker, through the material. Judging from the photo,
this must have been quite the lecture...
Peter
-- Peter (email), May 1, 2009
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 51 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
The picture of Professor Wheeler prompted me to finally
stitch together pictures of two blackboards I saw at Baylor
School of Medicine in 2005. To this day I have no idea who
drew them, but it is one of the Baylor Biochemistry professors.
UPDATE 14 May 2009: Juan Ruiz-Hau was kind enough to prepare these excellently corrected versions from the originals.
-- Niels Olson (email), May 4, 2009
One fax = 2,500 PowerPoint slides
Here's one of my exhibits in The Drawing Room's FAX show:
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 52 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
-- Edward Tufte, May 11, 2009
There was an interview with Steve Balmer, Microsoft CEO, in the Sunday New York Times
(Sunday Business, Page 2, "Meetings 2.0, At Microsoft"). When asked what it's like to be in
a meeting run by Steve Balmer he says that he decided that what he calls the "long and
winding road" meeting style of a few years ago at Microsoft isn't productive. He says that
for most meetings, he now gets the materials in advance and he reads them. For the meeting
he comes in and says "I've got the following four questions. Please don't present the deck".
Incredible!
-- Sam Perry (email), May 17, 2009
"Eight PowerPoint Train Wrecks"
The first link below provides an interesting set of slides that makes some similar points to
"Cognitive Style," and some very different ones. Shares the view of bullet points and silly
transitions as boring and useless, but also eschews data density in favor of capturing
attention and making simple points. Some of their very bad presentation graphics are not bad
engineering diagrams, for the very same reason: their density.
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 53 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
http://www.cio.com/special/slideshows/2009/02/bad_powerpoint/index
Also see http://www.slideshare.net/thecroaker/death-by-powerpoint
These are dramatic illustrations of how the goal of the presentation shapes the way we
present "data" (to the degree that marketing slides are "data").
Are we trying to get a point across, or are we trying to make the data speak for itself and
promote clear thinking.
kind regards,
Todd
-- Todd I. Stark (email), June 1, 2009
PowerPoint and scientific fraud
July 14, 2009, Wall Street Journal Blog
"The oversight [of scientific data] is now vastly diminished. Even within the laboratory
environment, many students and post-docs and scientists are not showing raw data anymore.
They're showing PowerPoint presentations. That gives the individual, if they're so inclined,
the ability to manipulate data right up-front. Unless a mentor is vigilant, there's a real
breakdown."
--John Dahlberg, director, Division of Investigative Oversight in the Office of Research
Integrity, Dept. Health and Human Services
http://blogs.wsj.com/health/2009/07/14/qa-how-do-you-find-scientific-fraud/
-- Prem Thomas (email), July 14, 2009
Here is a fine example of the four-page handout:
http://www.sirlin.net/storage/articles/balance/GDC%202009%20sirlin%20handout6.pdf
Everything in paragraph form, fairly dense (but not overwhelming) content, and clear
illustrations.
Originally from this post:
http://www.sirlin.net/blog/2009/3/30/handout-from-my-gdc-lecture.html
-- Daniel F. Thornton (email), July 21, 2009
For the past ten years, we've been running a very successful technical conference without
ever resorting to a single PP slide:
http://www.ayeconference.com/wiki/scribble.cgi?read=WhyWeDoNotUsePowerPoint
We're all fans of Dr. Tufte's, and hundreds of people have benefited from the elimination of
this communicable disease, PP.
Many thanks.
-- Gerald M. Weinberg (email), July 21, 2009
A3 Printing in MSWord
I work in local government finance and performance monitoring and evaluation.
Communicating complex financial and interrelated performance data to managers and
elected officials is always challenging. I have migrated to the single 11x17 layout over the
years, encouraged by a Tufte seminar years ago, and now use it frequently.
A straightforward way to produce an A3, or 11x17, document in MS Word works well for
up to four discrete pages. First, write the report in Word set up for standard US Letter, 81/2
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Page 54 of 55
Ask E.T.: PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
4/17/10 10:17 PM
up to four discrete pages. First, write the report in Word set up for standard US Letter, 81/2
x 11. Print and proof as needed. Then, go to Page Setup, on the Margins tab find Multiple
Pages and set that to Book Fold. Set Orientation to Landscape. On the Paper tab select A3
or 11x17, as your printer is configured. It's helpful to set the printer to double-sided. Click
okay, then print. This is fast and straightforward, but experiment some to get your content
to flow correctly! Pagination is not always what you expect at first, but easily controlled
with practice.
I use this to encourage my staff to write brief, high-density reports and remove the technical
hurdles. However, it does not handle a full A3 spread layout very well - and those are very
useful when we want to present more complex graphs with supporting data.
-- Randy Webster (email), December 22, 2009
Contribute an answer
Threads relevant to PowerPoint:
Apple's Keynote vs Microsoft's PowerPoint: Don't get your hopes
up
Don't get your hopes up.
Cancer survival rates: tables, graphics, and PP
Comparisons of methods for presenting cancer survival rates.
Columbia Accident Investigation Board: The Boeing PowerPoint
Slide
How to make presentations: techniques, handouts, display
technologies
Lousy PowerPoint presentations: The fault of PP users?
A look at a rich and complex question: What are the the causes of
presentations?
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB
Metaphors for Presentations: Conway's Law Meets PowerPoint
NASA seeks to curb "PowerPoint engineering"
New edition of "The Cognitive Style of PowerPoint"
Plagiarism detection in PowerPoint presentations
An intriguing but under-explored topic.
PowerPoint and Military Intelligence
Mainly recent examples of leaked PP slides in the Iraq war.
Page 55 of 55