Eyetrack III - Home Page What We Saw Through Their Eyes By Steve Outing and Laura Ruel What do people see when they view a news website or multimedia feature? Is it what the site's designers expect? ... Perhaps not. The Eyetrack III study literally looked through the eyes of 46 people to learn how they see online news. In this overview article, we review the study's key findings. story> NEWS HOMEPAGES: IT IS POLITE TO STARE MULTIMEDIA: WHEN TO USE IT TO TELL STORIES ONLINE ADS: WHY PLACEMENT AND SIZE MATTER What does Eyetrack III say about news website design? Jay heard from a team of industry insiders. story> Here's what one news multimedia editor learned about his craft from Eyetrack III. story> Eyetrack III quantifies just how quickly and intently readers view an ad -- valuable intelligence in a performance-driven business. story> By Jay Small Homepage Design Design Comparisons Eye Viewing Patterns Headlines and Blurbs Headline Size Font Size Navigation Compact & Extended Pages Images Sizes and Faces http://www.poynterextra.org/eyetracking2004/9/2/2004 9:48:07 AM By Jeff Glick Article-Page Design Overview Summary Paragraphs Paragraph Length Number of Columns Subheads Text & Images Multimedia Results Overview Recall/Comprehension Multimedia Features By Kinsey Wilson Advertising Results Overview Avoiding Ads Barriers to Seeing Ads Text Ads Ad Size Ads in Text Body Mouseovers Animation Page Position Pop-ups Eyetrack III - About the Research MORE ABOUT EYETRACKING •Welcome to Eyetrack III •Eyetrack history •FAQ •How people read •Eyetrack is not a solution •How accurate is eyetracking? •The Eyetrack III team •Discussion forums Multimedia specials •Heatmaps for test homepages •Graphic: Understanding a heatmap •Graphic: Understanding an individual user session •Interactive quiz •Videos: Individual eyetracking sessions EYETRACK III RESULTS •Overview •Homepage design •Advertising •Article-page design •Multimedia content INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter Welcome to Eyetrack III How much do we really know about how people read news websites? We can track their behavior clicking through a site visit. We can collect personal information. We can ask them questions. But that presents a small part of the full picture. To get the rest, we need to climb inside their heads and look through their eyes as they view online news sites -- to peer into their minds and see patterns that even they don't consciously see. That, remarkably, is what we've done. The Poynter Institute, the Estlow Center for More than pretty colors: This Journalism and New Media, and Eyetools Inc. "heatmap" generated by Eyetools software from the in late 2003 took 46 Internet users and Eyetrack III study shows looked through their eyes -- utilizing aggregate eye fixations and sophisticated and non-intrusive viewing of a news website "eyetracking" equipment -- as they each homepage. (Click image to learn spent an hour reading news websites and about heatmaps.) multimedia news content. We used the Eyetools Analysis Solution Suite to capture and process the data and looked to the company's experts to help us compile the initial findings. What we learned is the subject of this website. So what might you do with all these findings? Can you extrapolate from the behavior of our 46 participants to the much larger audience of the website you manage? The simple answer: "No, putting this research to use is more complicated than that." We've done our best to simplify the complications, though. First, we recommend you read Howard Finberg's characterization of Eyetracking as a tool rather than a solution. Then we suggest you seek a more detailed understanding of how and what Eyetracking actually measures from Eyetools president Colin Johnson. Finally, browse the FAQ -– and additional links on the left side of this page –- in pursuit of additional info. Please send your additional questions to the researchers by using the Ask the Eyetrackers page. Written by Steve Outing and Laura Ruel, project managers; research and tools by Colin Johnson, Greg Edwards, and Leslie Kues of Eyetools Inc. http://www.poynterextra.org/eyetracking2004/about.htm (1 of 2)9/2/2004 9:48:13 AM Eyetrack III - About the Research http://www.poynterextra.org/eyetracking2004/about.htm (2 of 2)9/2/2004 9:48:13 AM Eyetrack III - Ask the Eyetrackers a QUESTIONS ANSWERED ON THIS PAGE •Coming soon Ask the Eyetrackers We've provided lots of information on this site about Eyetrack III, and we know you may have questions. Use the form below to send the project team your questions, and we'll answer them publicly on this page. Your name: Your e-mail: Your company: Your question: Send The Eyetrackers Respond: This area will include answers from the Eyetrack III researchers to readers' questions. Ask us a question and get things started! http://www.poynterextra.org/eyetracking2004/formpage.asp9/2/2004 9:48:14 AM Eyetrack III - What You Most Need to Know MORE ABOUT EYETRACKING •Welcome to Eyetrack III •Eyetrack history •FAQ •How people read •The Eyetrack III team •Eyetrack is not a solution •How accurate is eyetracking? •Discussion forums Multimedia specials •Heatmaps for test homepages •Graphic: Understanding a heatmap •Graphic: Understanding an individual user session •Interactive quiz •Videos: Individual eyetracking EYETRACK III RESULTS •Overview Homepages: •5 homepage designs •Eye viewing patterns •Headlines & blurbs •Headline size •Font size •Navigation •Photos & images •Compact & extended pages Other findings: •Advertising •Article-page design •Multimedia content The Best of Eyetrack III: What We Saw When We Looked Through Their Eyes By Steve Outing and Laura Ruel Eyetrack III project managers News websites have been with us for about a decade, and editors and designers still struggle with many unanswered questions: Is homepage layout effective? ... What effect do blurbs on the homepage have compared to headlines? ... When is multimedia appropriate? ... Are ads placed where they will be seen by the audience? The Eyetrack III research released by The Poynter Institute, the Estlow Center for Journalism & New Media, and Eyetools could help answer those questions and more. Eyetracking research like this won't provide THE answer to those questions. But combined with other site metrics already used by news website managers -- usability testing, focus groups, log analysis -- the Eyetrack III findings could provide some direction for improving news websites. In Eyetrack III, we observed 46 people for one hour as their eyes followed mock news websites and real multimedia content. In this article we'll provide an overview of what we observed. You can dive into detailed Eyetrack III findings and observations on this website -- use the navigation at the top and left of this page -- at any time. If you don't know what eyetracking is, get oriented by reading the Eyetrack III FAQ. Let's get to the key results of the study, but first, a quick comment on what this study is and is not: It is a preliminary study of several dozen people conducted in San Francisco. It is not an exhaustive exploration that we can extrapolate to the larger population. It is a mix of "findings" based on controlled variables, and "observations" where testing was not as tightly controlled. The researchers went "wide," not "deep" -- covering a lot of ground in terms of website design and multimedia factors. We hope that Eyetrack III is not seen as an end in itself, but rather as the beginning of a wave of eyetracking research that will benefit the news industry. OK, let's begin. ... At the core: Homepage layout While testing our participants' eye movements across several news homepage designs, Eyetrack III researchers noticed a common pattern: The eyes most http://www.poynterextra.org/eyetracking2004/main.htm (1 of 9)9/2/2004 10:12:48 AM Eyetrack III - What You Most Need to Know INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter often fixated first in the upper left of the page, then hovered in that area before going left to right. Only after perusing the top portion of the page for some time did their eyes explore further down the page. Depending on page layout, of course, this pattern can vary. The image above is a simplistic representation of the most common eye-movement pattern we noticed across multiple homepage designs. (In other words, don't take what you see above too seriously.) Now also consider another Eyetrack observation: Dominant headlines most often draw the eye first upon entering the page -- especially when they are in the upper left, and most often (but not always) when in the upper right. Photographs, contrary to what you might expect (and contrary to findings of 1990 Poynter eyetracking research on print newspapers), aren't typically the entry point to a homepage. Text rules on the PC screen -- both in order viewed and in overall time spent looking at it. A quick review of 25 large news websites -- here's a list of them -- reveals that 20 of them place the dominant homepage image in the upper left. (Most news sites have a consistent page design from day to day; they don't often vary the layout as a print newspaper would.) We observed that with news homepages, readers' instincts are to first look at the flag/logo and top headlines in the upper left. The graphic below shows the zones of importance we formulated from the Eyetrack data. While each site is different, you might look at your own website and see what content you have in which zones. http://www.poynterextra.org/eyetracking2004/main.htm (2 of 9)9/2/2004 10:12:48 AM Eyetrack III - What You Most Need to Know [Read more on what Eyetrack III says about homepage layout here.] Want people to read, not scan? Consider small type The Eyetrack III researchers discovered something important when testing headline and type size on homepages: Smaller type encourages focused viewing behavior (that is, reading the words), while larger type promotes lighter scanning. In general, our testing found that people spent more time focused on small type than large type. Larger type resulted in more scanning of the page -- fewer words overall were fixated on -- as people looked around for words or phrases that captured their attention. This was especially the case when we looked at headline size on homepages. Larger headlines encouraged scanning more than smaller ones. (Note: We are not advocating that you run out and reduce the size of your font across the board. You should make sure that people can read the font size you select in order to achieve the appropriate balance.) Particularly interesting was people's behavior when there were headlines and blurbs used on homepages. Eyetrack III test participants tended to view both the headline and blurb when the headline was bold and the same size as blurb text and immediately preceded the blurb on the same line. With a headline larger than the blurb and on a separate line, people tended to view the headlines and skip the blurbs; they scanned the headlines throughout the page more than the group that looked at the smaller headlines. Researchers believe that it is the contrast in type size that accounts for this behavior, as well as the type size itself. When a headline is larger than its accompanying blurb text, it's perceived as the important element of the headline-blurb block -- so people appear to decide that viewing the headline is http://www.poynterextra.org/eyetracking2004/main.htm (3 of 9)9/2/2004 10:12:48 AM Eyetrack III - What You Most Need to Know sufficient and they skip the blurb. Underlined headlines discouraged testers from viewing blurbs on the homepage: This may be related to a phenomenon that we noted throughout the testing: visual breaks -- like a line or rule -- discouraged people from looking at items beyond the break, like a blurb. (This also affects ads, which we address below.) When we look at news websites, we find that the vast majority of them (22 out of 25) use blurbs to accompany headlines on their homepages. It's the rare ones that use only headlines: CNN.com, NYPost.com, and ProJo.com. In terms of headline size, we observed about an even split between using larger type size for headlines vs. smaller type. We found that 12 out of 22 news sites that use blurbs on their homepage put rules under their headlines. [Read more on what Eyetrack III says about type and blurbs here and here.] Partial viewing of headlines, blurbs found to be common We found that when people look at blurbs under headlines on news homepages, they often only look at the left one-third of the blurb. In other words, most people just look at the first couple of words -- and only read on if they are engaged by those words. Here's a heatmap of a blurb demonstrating this. (A heatmap is an aggregate view of all the eye fixations of our test subjects. Below, the orange area was viewed the most, the blue areas the least.) With a list of headlines on a homepage, we can see where people looked with eyetracking -- and again, most often it's the left sides of the headlines. People typically scan down a list of headlines, and often don't view entire headlines. If the first words engage them, they seem likely to read on. On average, a headline has less than a second of a site visitor's attention. For headlines -- especially longer ones -- it would appear that the first couple of words need to be real attention-grabbers if you want to capture eyes. The same goes for blurbs -- perhaps even more so. Our findings about blurbs suggest that not only should they be kept short, but the first couple of words need to grab the viewer's attention. http://www.poynterextra.org/eyetracking2004/main.htm (4 of 9)9/2/2004 10:12:48 AM Eyetrack III - What You Most Need to Know On the 25 news websites we reviewed, there's considerable variety in blurbs. Average blurb length varies from a low of about 10 words to a high of 25, with most sites coming in around 17. [Read more on what Eyetrack III says about blurbs here.] What creates "hot spots"? In Eyetrack III, we tested several homepage designs, watching where on the page people looked. As you would expect, lower parts of the page -- especially areas you have to scroll to view -- receive modest viewing. But that doesn't mean you can't get people to look at content low on a scrolling page. On a couple of our test homepages, we found "hot spots" for some stories. Perhaps because our testing took place in San Francisco, research subjects were drawn to one story about the site "Craig's List" (a local online community popular since its inception in 1995). The headline for that story had an inordinate number of eye fixations compared to surrounding content, even though it was below the first visible screen of the page. We observed a similarly high number of eye fixations on a headline about clothing maker FCUK, which was placed far down on a page with a long list of headlines and blurbs. We think this spells good news for those websites with homepages that extend well beyond the initial screen view. Eyetrack III found that people do typically look beyond the first screen. What happens, however, is that their eyes typically scan lower portions of the page seeking something to grab their attention. Their eyes may fixate on an interesting headline or a standout word, but not on other content. Again, this points to the necessity of sharp headline writing. [Read more on what Eyetrack III says about homepage design here and here.] Where's your navigation? While testing several homepage designs, we varied the placement of a navigation element: top (under the flag or logo), left column, and right column. Navigation placed at the top of a homepage performed best -- that is, it was seen by the highest percentage of test subjects and looked at for the longest duration. In a survey of 25 top news sites, we found 11 that used top position navigation. The other 14 used left navigation. Seven of the 25 used left and top navigation elements. None of the 25 sites we surveyed used right side navigation. It's rare, but you can find right navigation in the news website world. It might surprise you to learn that in our testing we observed better usage (more eye fixations and longer viewing duration) with right-column navigation than left. While this might have been the novelty factor at play -- people aren't used to seeing right-side navigation -- it may indicate that there's no reason not to put navigation on the right side of the page and use the left column for editorial content or ads. http://www.poynterextra.org/eyetracking2004/main.htm (5 of 9)9/2/2004 10:12:48 AM Eyetrack III - What You Most Need to Know [Read more on what Eyetrack III says about navigation here.] What about article layout, writing style? Eyetrack III results suggests various characteristics of article writing and layout can affect a reader's viewing behavior. For example, let's take average paragraph length. Most news sites run articles with medium-length paragraphs -- somewhere (loosely) around 45-50 words, or two or three sentences. In a survey of 25 top news sites, however, we did find seven that routinely edited articles to make paragraphs shorter -- often only one sentence per paragraph. Shorter paragraphs performed better in Eyetrack III research than longer ones. Our data revealed that stories with short paragraphs received twice as many overall eye fixations as those with longer paragraphs. The longer paragraph format seems to discourage viewing. Most news website article pages present stories in a single column of text, but a handful of sites -- like IHT.com and TheHerald.co.uk -- mimic newspaper layout and present articles in two or three side-by-side columns. Is this as readable as the traditional (for the Web) one-column article format? Eyetrack III results showed that the standard one-column format performed better in terms of number of eye fixations -- in other words, people viewed more. However, bear in mind that habit may have affected this outcome. Since most people are accustomed to one-column Web articles, the surprise of seeing three-column type might have affected their eye behavior. What about photos on article pages? It might surprise you that our test subjects typically looked at text elements before their eyes landed on an accompanying photo, just like on homepages. As noted earlier, the reverse behavior (photos first) occurred in previous print eyetracking studies. Finally, there's the use of summary descriptions (extended deck headlines, paragraph length) leading into articles. These were popular with our participants. When our testers encountered a story with a boldface introductory paragraph, 95 percent of them viewed all or part of it. When people viewed an introductory paragraph for between 5 and 10 seconds -- as was often the case -- their average reading behavior of the rest of the article was about the same as when they viewed articles without a summary paragraph. The summary paragraph made no difference in terms of how much of the story was consumed. Just over 20 percent of the leading news websites regularly use summary paragraphs with articles. [Read more on what Eyetrack III says about article layout here.] Advertising http://www.poynterextra.org/eyetracking2004/main.htm (6 of 9)9/2/2004 10:12:48 AM Eyetrack III - What You Most Need to Know Eyetrack III tested a variety of ad placements and formats across our various hompages and article-level pages. The first thing we noticed is that people often ignore ads, but that depends a lot on placement. When they do gaze at an ad, it's usually for only 0.5 to 1.5 seconds. Good placement and the right format can improve those figures. We found that ads in the top and left portions of a homepage received the most eye fixations. Right side ads didn't do as well, and ads at the bottom of the page were seen, typically, by only a small percentage of people. Close proximity to popular editorial content really helped ads get seen. We noticed that when an ad was separated from editorial matter by either white space or a rule, the ad received fewer fixations than when there was no such barrier. Ads close to top-of-the-page headlines did well. A banner ad above the homepage flag didn't draw as many fixations as an ad that was below the flag and above editorial content. Text ads were viewed most intently, of all the types we tested. On our test pages, text ads got an average eye duration time of nearly 7 seconds; the best display-type ad got only 1.6 seconds, on average. Size matters. Bigger ads had a better chance of being seen. Small ads on the right side of homepages typically were seen by only one-third of our testers; the rest never once cast an eye on them. On article pages, "half-page" ads were the most intensely viewed by our test subjects. Yet, they were only seen 38 percent of the time; most people never looked at them. Article ads that got seen the most were ones inset into article text. "Skyscraper" ads (thin verticals running in the left or right column) came in third place. Reviewing 25 leading news websites, we discovered that there's a preponderance of small banner ads on homepages. And it's exceedingly common to find ads in the right column of news homepages. About half of the 25 sites we reviewed inset ads into article text. [Read more on what Eyetrack III says about advertising here.] Larger online images hold the eye longer than smaller images News homepages typically use templates, many of which employ a predetermined size for a main image. Although the value of using a templatedriven design can (and should) be debated, what we learned about photo size in Eyetrack III may be helpful to those who are wondering just how big a spot to leave for images. Although we learned that most of our test participants did not look at images first, we also observed that images received a significant number of eye fixations. We also learned that the bigger the image, the more time people took to look at it. One of our test pages had a postage-stamp sized mug shot that was viewed by 10 percent of our participants. Compare that with an average-sized photo (about 230 pixels wide and deep) that drew gazes from about 70 percent. http://www.poynterextra.org/eyetracking2004/main.htm (7 of 9)9/2/2004 10:12:48 AM Eyetrack III - What You Most Need to Know We found that images that are at least 210 x 230 pixels in size were viewed by more than half of the testers. Our research also shows that clean, clear faces in images attract more eye fixations on homepages. Article-level pages seem to follow suit. Again we found that the larger the image, the more users were drawn to it. In reviewing 25 news websites, we found that about 20 percent routinely use small images on their homepages. Four out of five sites routinely place their homepage main photo in the upper left. And here's an interesting research tidbit: We noticed that people often clicked on photos -- even though on our test pages that got them nowhere (and indeed, clicking on photos does nothing on many real news sites). [Read more on what Eyetrack III says about images here.] Text for facts; multimedia graphics for unfamiliar concepts Overall, we observed that participants were more likely to correctly recall facts, names, and places when they were presented with that information in a text fomat. However new, unfamiliar, conceptual information was more accurately recalled when participants received it in a multimedia graphic format. So what does this mean? While overall we did see a slight, although not statistically significant, increase in information recall from text stories, we should note that most of our recall questions were about facts, names, and places. Story information about processes or procedures seemed to be comprehended well when presented using animation and text. A stepby-step animation we tested supported this idea. We also observed that most participants attended to only two forms of media at a time. For example, in one of our testing situations users were presented with audio, still images, and written captions. We observed that they directed their attention to the audio and images. Important information in the photo captions were not read by many. The bottom line is that the best journalists working in multimedia environments know how to make good choices about the presentation of story information. As demonstrated in this research, some information is best conveyed by the use of good, descriptive writing. Other information is better explained graphically. [Read more on what Eyetrack III says about multimedia comprehension here, and read additional general multimedia observations here.] We've covered some of the highlights in this article, but there's lots more, so please spend some time exploring this website. Use the navigation devices at the top of this page and in the left column. http://www.poynterextra.org/eyetracking2004/main.htm (8 of 9)9/2/2004 10:12:48 AM Eyetrack III - What You Most Need to Know Elizabeth Carr provided research assistance for this article. http://www.poynterextra.org/eyetracking2004/main.htm (9 of 9)9/2/2004 10:12:48 AM Eyetrack III - Designs for Eyetrack III Test Websites Homepage designs in this report •Style No. 1 •Style No. 2 •Style No. 3 •Style No. 4 •Style No. 5 Add/view feedback on homepage reports MORE HOMEPAGE RESULTS ARTICLES •5 homepage designs •Eye viewing patterns •Headlines & blurbs •Headline size •Font size •Navigation •Photos & images •Compact & extended pages OTHER EYETRACK III FINDINGS •Overview •Advertising •Article-page design •Multimedia content INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter Designs for Our Mock News Websites In the world of the Web, news sites are not often regarded for their innovative and effective homepage designs. Some experts complain about cluttered, difficult-to-navigate homepages on most news sites. EYETRACK III FINDINGS This report is one of many from the Eyetrack III study of broadband-era news websites. 46 people were tested for one hour each in December 2003 by Eyetools Inc. in partnership with the Poynter Institute and the Estlow Center. During the test period, each test subject viewed mock news websites created for research purposes and real-world multimedia news features. Results were published in August 2004. In this round of Eyetrack III, we decided to test what we considered five "typical" homepage designs. In future studies, we hope to explore how more innovative, less typical design fare, but in this preliminary round we wanted to have our participants interact with today's usual Web offerings. At the time when our inital research began (more than a year ago), we identified five "typical" homepage designs that were in use by viewing more than 250 news homepages and categorizing them. Below are examples of the styles we identified (along with images of the actual pages we tested), with a "sample" webpage for each style. Although we acknowledge that these categories are not comprehensive, we do feel that they reflect some of the typical news-site designs in use today. Each of the five mock homepage designs we used had two versions -- with one controlled variable -- in the actual testing. Half of our group of test subjects saw one version of a homepage; the other half saw the other version. (For example, half the group saw homepage No. 1 with only headlines; the other half saw that page with headlines and blurbs.) For observational purposes, we also have included some data about how our participants interacted with each design. (The stats represent the average of both controlled-variable versions of each homepage.) ● ● ● Average time: This number is simply an average of the amount of time participants spent with a particular homepage design model. Average fixations: This number indicates the average number of times participants' eyes stopped (for a least a fraction of a second) and noticed something on the homepage. Average clicks per person: http://www.poynterextra.org/eyetracking2004/homepages.htm (1 of 9)9/2/2004 10:13:02 AM Eyetrack III - Designs for Eyetrack III Test Websites This number indicates the average number of times our test participants clicked on the page. Although not conclusive, these data do provide food for thought. (Note: The test websites we created were designed to look like real ones, obviously. The names are fictitious, though some may sound similar to real news websites.) Style No. 1 This style of news homepage is what we determined to be the most commonly used by news websites. In general, the flag reflects the organization's offline presence, navigation is to the left, photos take up less than 20 percent of the homepage, and overall presentation is fairly information-dense. These are the mock homepages we tested: Participant data for this page design: ● ● ● Average time: 15.2 seconds Average fixations: 20.7 Average clicks per person: 2.5 We based the design of this homepage style on the following example: http://www.poynterextra.org/eyetracking2004/homepages.htm (2 of 9)9/2/2004 10:13:02 AM Eyetrack III - Designs for Eyetrack III Test Websites Style No. 2 Distinguishing features of this homepage design are its use of structured white space, lack of contrast in font size, simple color pallette, and fairly plain or "simple" design.. These are the mock homepages we tested: http://www.poynterextra.org/eyetracking2004/homepages.htm (3 of 9)9/2/2004 10:13:02 AM Eyetrack III - Designs for Eyetrack III Test Websites Participant data for this page design: ● ● ● Average time: 16.6 seconds Average fixations: 21.6 Average clicks per person: 1.9 We based the design of this homepage style on the following example: http://www.poynterextra.org/eyetracking2004/homepages.htm (4 of 9)9/2/2004 10:13:02 AM Eyetrack III - Designs for Eyetrack III Test Websites Style No. 3 Immediately apparent with this design is its use of little or no photography, its horizontal feel, and "stacked" layout. These are the mock homepages we tested: Participant data for this page design: ● ● ● Average time: 12.1 seconds Average fixations: 17.1 Average clicks per person: 2.2 http://www.poynterextra.org/eyetracking2004/homepages.htm (5 of 9)9/2/2004 10:13:02 AM Eyetrack III - Designs for Eyetrack III Test Websites We based the design of this homepage style on the following example: Style No. 4 The use of sans serif type, section labels reversed in horizontal bars, relatively large amounts of white space, and "stylish" look distinguish this page design. These are the mock homepages we tested: http://www.poynterextra.org/eyetracking2004/homepages.htm (6 of 9)9/2/2004 10:13:02 AM Eyetrack III - Designs for Eyetrack III Test Websites Participant data for this page design: ● ● ● Average time: 12 seconds Average fixations: 18.2 Average clicks per person: 2.2 We based the design of this homepage style on the following example: http://www.poynterextra.org/eyetracking2004/homepages.htm (7 of 9)9/2/2004 10:13:02 AM Eyetrack III - Designs for Eyetrack III Test Websites Style No. 5 Among the distinguishing features of this design are the use of a dark background color, its treatment of the the lead story -- usually an image with type superimposed on it -- and a "multimedia magazine" look. These are the mock homepages we tested: http://www.poynterextra.org/eyetracking2004/homepages.htm (8 of 9)9/2/2004 10:13:02 AM Eyetrack III - Designs for Eyetrack III Test Websites Participant data for this page design: ● ● ● Average time: 16.4 seconds Average fixations: 22.3 Average clicks per person: 2.4 We based the design of this homepage style on the following example: Add/view feedback on homepage reports Written by Laura Ruel and Steve Outing, project managers; research and tools by Colin Johnson, Greg Edwards, and Leslie Kues of Eyetools Inc. http://www.poynterextra.org/eyetracking2004/homepages.htm (9 of 9)9/2/2004 10:13:02 AM Eyetrack III - Multimedia MULTIMEDIA RESULTS ARTICLES •Recall & comprehension •Multimedia observations MORE EYETRACK III RESULTS •Overview Homepages: •5 homepage designs •Eye viewing patterns •Headlines & blurbs •Headline size •Font size •Navigation •Photos & images •Compact & extended pages Other findings: •Advertising •Article-page design INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter The Eyes' View of Multimedia News Features Now that we're in the broadband-Internet era, news websites can safely go beyond text and static images -- and many routinely do. Audio-narrated slide shows, video, interactive animated info-graphics ... you'll find those and more while surfing for news these days. The Eyetrack III researchers decided to take a through-the-eyes look at what people see and how they behave when they encounter multimedia news features. Our testing wasn't comprehensive; rather, it represents the beginning of a more serious look at how to design multimedia editorial content so that viewers best find and comprehend the information contained. For this multimedia section of Eyetrack III, we conducted two exercises: Multimedia Recall & Comprehension We took two multimedia features (from NYTimes.com) and created text-only versions. Each of our testers viewed one multimedia feature and one text feature on different subjects, then we asked follow-up questions to measure their recall of facts presented in the features. We compared their performance. And we tracked their eyes throughout the test, so we could match comprehension with that data to determine whether they looked at the part of the page containing the answer. story> Observations on Multimedia News Features Next, we gave our test subjects a list of realworld multimedia news features and 10 minutes to view whatever interested them. The eyetracker rolled as we observed how they interacted with the features they chose to view. This informal test revealed interesting observations and gave us clues about what to test for in future multimedia-content eyetracking studies. story> http://www.poynterextra.org/eyetracking2004/multimedia.htm9/2/2004 10:13:12 AM Eyetrack III - Article-Level Page Design Observations in this report: •Summary paragraphs are ready by most •Carefully read summaries = little time with story •Shorter paragraphs mean more reading •1-column articles read more than multiple-column •Subheads don't affect how much is read •Eyes fixated on text before images •Tips Article-Level Page Design: What Matters Inside? Once you've piqued readers' interest enough so that they click to an article-level page, how do you keep them there? Many news organizations are trying a number of devices to enhance the display and readability of news stories online. In this example from MSNBC.com, an italic-type introductory paragraph is used to provide some context before readers delve into the meat of the piece. Add/view feedback on this report MORE EYETRACK III RESULTS •Overview Homepages: •5 homepage designs •Eye viewing patterns •Headlines & blurbs •Headline size •Font size •Navigation •Photos & images •Compact & extended pages Other findings: •Advertising •Article-page design •Multimedia content Some sites, such as the International Herald Tribune's, utilize a more newspaperlike look for the text of their article-level pages online: http://www.poynterextra.org/eyetracking2004/articlepages.htm (1 of 8)9/2/2004 10:13:16 AM Eyetrack III - Article-Level Page Design INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter And many sites take advantage of some of the lessons learned in print design -such as utilizing subheads, short paragraphs, and bulleted text -- in an effort to make copy more scannable and readable. EYETRACK III FINDINGS This report is one of many from the Eyetrack III study of broadband-era news websites. 51 people were tested for one hour each in December 2003 by Eyetools Inc. in partnership with the Poynter Institute and the Estlow Center. During the test period, each test subject viewed mock news websites created for research purposes and real-world multimedia news features. Results were published in August 2004. Eyetrack III put a few of these techniques to the test. We inserted a number of typical writing and/or design devices into test pages to see if these affected how readers interacted with content on article-level pages. Some observations: ● ● ● ● ● ● When readers encountered a story with an introductory paragraph, 95 percent of them read all or part of the introductory paragraph. Those who spent time carefully reading the introductory paragraph of a story on article-level pages typically spent little time with the full story. Those who gave the intro paragraphs little time usually spent even less time with the story text. Shorter paragraphs encouraged testers to continue reading. Story text in one-column format was read more extensively than story text presented in a "newspaper-like" multiple-column format. Subheads in online stories had little affect on how much of the first or top portion of the story was read when the reader's interest was strongest. However, subheads increased reading for "skimmers" and for those whose attention in a story was beginning to wane. When readers got to an article-level page, they seemed to be there to view the text. Overall, participants' eyes fixated on the story or other text elements before the accompanying image. http://www.poynterextra.org/eyetracking2004/articlepages.htm (2 of 8)9/2/2004 10:13:16 AM Eyetrack III - Article-Level Page Design (Please note that in this article we cite some statistics. Keep in mind that in this particular part of the Eyetrack III testing, we did not tightly control the variables -- as we did in other parts of the research. So use the numbers with caution.) Observation: When participants encountered a story with an introductory or summary paragraph, 95 percent of them read all or part of the introductory paragraph. Eyetracking data seem to show that introductory paragraphs (that is, long blurbs) are thought of as essential elements of a story. Overall, there were 58 occasions when our test participants encountered introductory paragraphs. In 55 of these instances, they read at least some of the text. This indicates an overwhelming pattern, and really does show that readers place value on these forms of story presentation. Once again, we are able to tell that participants were reading, not by what they told us, but by the number of eye fixations on these introductory paragraphs. Observation: Overall, the test subjects who spent the most time with a story's introduction (or summary paragraph) spent the smallest amount of time with that same story's body text. On a number of prototype websites we inserted summary or introductory paragraphs before the main text of a story on an article page. Most of these paragraphs were about 2-3 sentences, and were slightly darker and larger in size than the story text type. Here is an example: Lessons from print design teach us that these introductory paragraphs are good devices to create points of entry and generate interest. However, our research http://www.poynterextra.org/eyetracking2004/articlepages.htm (3 of 8)9/2/2004 10:13:16 AM Eyetrack III - Article-Level Page Design observations suggest that people interact differently with these elements online. For example, when participants read the summary paragraph for less than 5 seconds, they read even less of the story than if they had spent between 5-10 seconds with the introduction. This behavior indicates that their level of interest in the story may have been small to begin with. Their quick glances at display type, before moving on, seemed to confirm this interpretation. Overall, the distribution of their behavior mirrors that of reading behavior when no introductory paragraph is available. In instances where people read an introductory paragraph for 5-10 seconds, their average reading behavior of the article itself appears equivalent to that of articles without an introduction. Reading the summary paragraph made no difference, on average, in terms of how much of the story was consumed for this group of participants. Upon closer inspection, we see that this group evenly divides into one camp that abandons the text with very little review, and a second camp that reads the articles extensively. In a similar vein, when a participant read the introduction very carefully (for more than 10 seconds), it appears that their behavior also resulted in two typical patterns. The most common pattern (for about 80 percent of participants) was that they abandoned the text of the article after very little review. In short, it appears that most readers got what they wanted from the introduction and decided to move on. It is interesting to note that in a minority of the cases, however, the opposite occurred. About 20 percent of the subjects who read the introduction carefully went on to read all or nearly all of the article. For those not completely interested in the topic, but engaged enough to inform themselves in a cursory way, the introduction seems to serve as "Cliffs Notes" for the full text. If not intending to read the article, readers seem to rely completely on the introduction (reading it for more than 10 seconds).. Observation: Shorter paragraphs encourage readers to continue reading. It appears that shorter paragraphs on the test pages of the Eyetrack III study compelled our participants to read a larger percentage of any given article. What do we mean by "shorter" paragraphs? Well, we determined the length of paragraphs by the number of sentences or thoughts contained in them. Generally, a short paragraph was one or two sentences (or one thought). Medium paragraphs were the length of about three short paragraphs. Long paragraphs were the length of three medium paragraphs and nine short ones. Here are two examples: Short paragraphs: http://www.poynterextra.org/eyetracking2004/articlepages.htm (4 of 8)9/2/2004 10:13:16 AM Eyetrack III - Article-Level Page Design Medium to long paragraphs: By examining the number and duration of eye fixations on these paragraphs, we were able to discern how much of the stories containing each of these paragraph lengths was read. The bottom line is that stories with shorter paragraphs got more than twice as many overall eye fixations than those with longer paragraphs. These data suggest that the longer-paragraph format discourages reading and that short-paragraph format overwhelmingly encourages reading. Observation: Story text in a one-column format gets read more extensively than story text presented in a "newspaper-like" multiple-column format. Because we observed a few news sites imitating print-style layout grids on their webpages, we decided to put this type of design to the test. So, some of our inside text pages presented stories in a multiple-column format: http://www.poynterextra.org/eyetracking2004/articlepages.htm (5 of 8)9/2/2004 10:13:16 AM Eyetrack III - Article-Level Page Design Other pages presented stories in a more typical (for online) one-column fashion: Once again, by examining the number and duration of eye fixations, we were able to discern how much of the stories were read by our participants. Strong results show that the three-column format did not compel people to read as extensively as the one-column format. Our participants read more of the story text presented in one column. Close to three-quarters, or 72 percent, of the story text presented in a one-column format was read. This was markedly higher than the 56 percent of story text read presented in a three-column format. Observation: Subheads in online stories have little affect on how much of a story is read in the first or top portion of the story when the reader's interest is strongest. However, subheads can increase reading for "skimmers" and for those whose attention in a story is beginning to wane. http://www.poynterextra.org/eyetracking2004/articlepages.htm (6 of 8)9/2/2004 10:13:16 AM Eyetrack III - Article-Level Page Design There is no substantive difference in the amount of an article that is read on average whether or not there are subheads. The data in Eyetrack III suggest that readers glance down a bit more than half (about 68 percent) of the length of articles when there are no subheads. (Articles of all lengths were considered here.) For articles with subheads, the numbers are not all that different. Readers glance down a bit more than half (about 62 percent) of the length of articles with subheads. It is important to note an observation that we were able to make only anecdotally, in numbers too small to do statistical analysis. We identified that a subhead can help hold or recapture attention that is beginning to wane. We saw subheads increase reading for test participants who skimmed (or scrolled) down an article page. The story subheads seemed to encourage eye fixations for those giving the text a quick glance. Observation: When readers get to an article-level page, they seem to be there to read. Overall, participants' eyes fixated on the story or other text elements before moving to an accompanying image (when there is one). When participants reached an article-level page they made about seven "stops," or eye fixations, before looking at the photo. Here's the "Top 10" list. Average viewing sequence of article-page elements: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Article introduction Headline Paragraph 1 Byline Category or section label Subheadlines Paragraph 2 Image (photo or graphic) Paragraph 3 Image caption Keep in mind, however, that this is an average of all the viewing patterns of all our test participants. Most readers probably don't jump around as much as this list suggests, or exactly in that order. Tips ● ● ● Be aware that introductory paragraphs are almost a guaranteed "read" for online-news readers. So make them count! Rather than treating them as an afterthought, realize the power they have to make the news more interesting and accessible to your audience. Recognize that introductory paragraphs may be the only thing many readers view. Also remember that those who want to read the text of a story will do so, whether or not there is an introduction present. Keep paragraphs short. Web writing and usability experts (like Crawford http://www.poynterextra.org/eyetracking2004/articlepages.htm (7 of 8)9/2/2004 10:13:16 AM Eyetrack III - Article-Level Page Design ● ● ● Kilian and Jakob Nielsen) have been saying this for some time, but here is evidence. Long descriptive paragraphs that work in print probably need to be rewritten for the Web if your goal is to use all means to encourage readers to stick with the story. Although the data suggest that a single-column article format is read more extensively than story text presented in a multiple-column format, those results may be determined by habit as much as -- or more than -- ease of use. Most webpages utilize a single-column format on article-level pages, and thus, most readers probably have been conditioned to read this way. Understanding this distinction is important. On the topic of subheads, remember that initial interest, or lack thereof, seems to drive reading behavior at the top and throughout the length of any online news story. A subhead can help hold or recapture attention that is beginning to wane, but it does not appear to create interest in a story. Keep the article-page average viewing sequence observed in Eyetrack III (text first; images later) in mind, but understand the variability within it. Remember that display words and the first few paragraphs are key elements for the reader. Add/view feedback on this report Written by Laura Ruel and Steve Outing, project managers; research and tools by Colin Johnson, Greg Edwards, and Leslie Kues of Eyetools Inc. http://www.poynterextra.org/eyetracking2004/articlepages.htm (8 of 8)9/2/2004 10:13:16 AM Eyetrack III - Frequently Asked Questions MORE ABOUT EYETRACKING •Welcome to Eyetrack III •Eyetrack history •FAQ •How people read •Eyetrack is not a solution •How accurate is eyetracking? •The Eyetrack III team •Discussion forums Multimedia specials •Heatmaps for test homepages •Graphic: Understanding a heatmap •Graphic: Understanding an individual user session •Interactive quiz •Videos: Individual eyetracking sessions EYETRACK III RESULTS •Overview •Homepage design •Advertising •Article-page design •Multimedia content INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter Frequently Asked Questions BACKGROUND 1. What exactly is "eyetracking"? 2. Don't eyetracking subjects have to wear funny headgear with cameras, and wouldn't that alter the reading experience such that it's not very realistic? 3. Can you track people who wear eyeglasses? 4. What does an eyetracking user session image look like? 5. What's a "heatmap"? 6. Can you track time spent on a page? 7. Is this the first Eyetrack study of news websites? 8. Can you help me understand how people read? THE EYETRACK III STUDY 1. What equipment did you use? 2. What was the testing environment like? 3. Where did testing take place? 4. Who performed the actual testing? 5. How many participants were tested in this study? What were their ages, sex, etc.? 6. How were participants in the study recruited? 7. How long was the testing period per individual? 8. What did participants do during the session? 9. What were the goals of the study? 10. Were all the data collected "good"? 11. What's with the "comprehension" testing? That's not really eyetracking, right? 12. How accurate are the findings from this research? What's the "margin of error"? 13. What's the difference between a "finding" and an "observation" as found in the various Eyetrack III reports? 14. What is the concept of "counterbalancing" and how was this incorporated in the study? 15. What does it mean to "control variables," and how was this achieved during the testing? 16. What variables did you test in Section 1 of the Eyetrack III testing? 17. How did you create and design the 10 mock news websites? 18. I'm a member of the press. Who can give me more information about Eyetrack III? 1. What exactly is "eyetracking"? http://www.poynterextra.org/eyetracking2004/faq.htm (1 of 11)9/2/2004 10:13:21 AM Eyetrack III - Frequently Asked Questions Eyetracking is research that tracks where a person's eyes look while reading, then analyzes the data to reveal patterns. By combining and reviewing data from multiple individuals during testing, you can discover representative patterns that apply to most of the population. For the Eyetrack III study we examined viewing patterns of prototype news websites, but you can use eyetracking to study how people view printed newspapers and magazines (editorial content and/or advertising), to gauge effectiveness of various forms of advertising, product packaging, and computer applications; it can be used in flight simulators, and even to track what people look at on shelves when grocery shopping. Here's a more precise definition, from our friends at Eyetools, the company that conducted the Eyetrack III study: "Eyetracking is a monitoring technology that determines where a person is looking. Special cameras called 'eye trackers' can watch a person's eye and capture fixations and eye movements with a remarkable degree of accuracy (typically accurate to 1 cm on a standard computer screen) without requiring any special headgear." In other words, it's like getting inside of a person's head and watching what they see -- with the advantage that a computer is recording every eye movement and fixation for later compilation and analysis. 2. Don't eyetracking subjects have to wear funny headgear with cameras, and wouldn't that alter the reading experience such that it's not very realistic? It used to be that eyetracking research required people to wear headsets with small cameras that tracked the movement of their eyes and matched that to what they were viewing. The technology has now improved, so that in Eyetrack III our test subjects did not have to wear anything on their heads. They simply sat in a desk chair and looked at a standard size (17 inch) computer monitor. It wasn't a typical monitor, though. Current-generation eyetrackers put a small video camera below the screen, which is calibrated and locked on to the test subject's gaze. As long as the person's head doesn't move outside of the camera's field of view (a region of space about a cubic foot -- more than enough leeway for typical usage), the eyetracker stays on target throughout the session. The technology has gotten so good that today there exists eyetracking equipment that can use a telephoto lens and track a stationary person's gaze from 20 feet away. (We didn't use such equipment for Eyetrack III.) 3. Can you track people who wear eyeglasses? In Eyetrack III, we chose to recruit only people who did not need to wear glasses. Eyeglasses can make it more challenging (though not necessarily impossible) to eyetrack. Contact lenses were OK. As eyetracking technology improves, we'll have less trouble with glasses and will do more testing with people who wear them. (Remember, too, that fewer people now wear glasses, thanks to the popularity of laser eye surgery.) Older camera-headgear systems often are able to get good data from people who wear glasses. For this study, we felt that the advantage of having a more natural online reading environment outweighed the limitation. And despite this limitation, we still were able to test some subjects over the age of 50. http://www.poynterextra.org/eyetracking2004/faq.htm (2 of 11)9/2/2004 10:13:21 AM Eyetrack III - Frequently Asked Questions 4. What does an eyetracking user session image look like? Single-user session Above is an image (click to enlarge) of a single test subject's journey through a single page (one of the homepages from the Eyetrack III test). It looks to most of us like a jumble of lines and dots, but eyetrack researchers can tell a lot from this. Here's a graphic (click to enlarge) showing the features found on an individual session: Individual-session explainer Notice lots of circles connected by thin lines. The circles are points of eye fixation -- where the person halted their gaze for at least a fraction of a second. The lines are "saccades" and indicate the path that the eyes took through the page connecting each fixation. Find the green dot; that's the entry point to the page. Follow the lines (if you can) and you'll see the eye path. The image also has numbers in black boxes; these numbers are timestamps and can help you follow the path more clearly. You'll also note blue lines on some parts of the image. This represents the general trend of viewing over the page. The thickness of the line varies according to how much time was spent there. The individual images show you what one person did. To see the viewing trends of an entire group, we change the visual metaphor from lines and circles to that of a "heatmap" where the warmer colors represent where a higher percentage of the group looked. 5. What's a "heatmap"? Heatmap Click the thumbnail image above to see an example. A heatmap is an aggregate view of all the individual user session images (like the one in question 4 above) for a single webpage on a single task. Researchers combine all the individual page sessions to create a single view of a page, revealing eye patterns from the group of test subjects. Here's a graphic explaining the various features of a heatmap: http://www.poynterextra.org/eyetracking2004/faq.htm (3 of 11)9/2/2004 10:13:21 AM Eyetrack III - Frequently Asked Questions Heatmap explainer The red/orange/yellow areas are where the larger percentages of the group looked (that is, where their eyes fixated for at least a fraction of a second). The dark blue areas are where the smaller percentages looked. There's a key at the top of the image to guide you. The image also has "X" marks throughout, indicating where participants clicked. (The numbers signify specific test subjects.) Also note the red horizontal lines on the page. These tell us how far down the page the test subjects went before leaving the page. In this example, the largest group scrolled all the way to the bottom of the page, but a fair number of people didn't scroll down below what was visible initially. By looking at a heatmap image of a page, you might see that a particular ad, image, or headline hardly gets viewed, for example. You might find that the graphic that an artist spent many hours producing is hardly glanced at. Heatmaps are especially interesting when you start analyzing them in relation to each other. Heatmap images of two differently designed homepages, for instance, can tell you which one does a better job of attracting attention to the content, the ads, or other key areas of the page. There's much you can learn simply by comparing these images from different pages. 6. Can you track time spent on a page? Yes. (And we can track time spent on any component of the page.) Time spent on a page gets interesting when you're comparing different page designs. For example, in one part of Eyetrack III we had two nearly identical homepages, but one contained only headline links to inside stories while the other had headlines plus blurbs. The amount of time spent on each of those pages tells the researchers something. Eyetools researchers combine the heatmap images along with time spent and other factors to determine trends that you will read about in this Eyetrack III website. 7. Is this the first Eyetrack study of news websites? This is the third Eyetrack study of news conducted by The Poynter Institute and its partners. The first studied newspaper print editions; the second studied firstgeneration news websites. See "Eyetrack: A History of News Consumer Behavior." 8. Can you help me understand how people read? In understanding eyetracking results and data, it's helpful to understand the process of human reading. People typically don't look at individual letters of each word, but rather recognize words as a whole, and they often look at more than one word at once. According to psycholinguist Keith Rayner of the University of Massachusetts at Amherst, your eyes do not move smoothly across the text as you read. Instead, the typical reader behavior is to look at a word or several words in a group, then pause your eyes there briefly; this is called a "fixation," and it takes about 0.25 seconds on average. After a fixation, you move your eyes to the next word or group of words; this movement is called a "saccade" and http://www.poynterextra.org/eyetracking2004/faq.htm (4 of 11)9/2/2004 10:13:21 AM Eyetrack III - Frequently Asked Questions takes only 0.1 seconds. (People often skip over short or predictable words such as "of," "in," "a," etc.) After this pattern is repeated once or twice, you pause to comprehend the phrase you just looked at (which on average takes 0.3 to 0.5 seconds). According to Rayner, all these fixations and saccades result in 95 percent of college-educated people reading between 200 and 400 words per minute when reading an article; 300 words per minute is the average. As you read about Eyetrack III findings, you'll often see references to "fixations." Simply remember that a fixation is a brief but measurable pause in looking at a word, phrase, or image. "Saccades" are the paths between these fixations. THE EYETRACK III STUDY 1. What equipment did you use? The eyetracker used was a Tobii Technologies model ET-17. To the user, the device looks like a standard 17-inch monitor and keyboard. The only difference is a small video camera positioned at the bottom of the monitor. The test subject is not required to wear headgear of any kind, and there are no other hardware components to cause distraction. The monitor and keyboard are black. The Tobii ET-17 eyetracker Tobii stationary video camera According to the company, the device will tolerate fairly large and fast head movements. This is achieved without use of moving cameras. The device is calibrated for each test subject, but once set there's no need to recalibrate -even in the event of a break to get a cup of coffee. Tracking is resumed instantly. (This is a major improvement over eyetracking procedures that require headgear or sensors.) 2. What was the testing environment like? http://www.poynterextra.org/eyetracking2004/faq.htm (5 of 11)9/2/2004 10:13:21 AM Eyetrack III - Frequently Asked Questions Testing took place in a small office with no distractions or clutter. Test subjects sat in a comfortable office chair; the Tobii eyetracking monitor was positioned on a spartan desk. After initial instruction by an Eyetools representative, the test subjects were left on their own to work through their assigned tasks -- which took about an hour, including posttesting questioning. Test administrators monitored each participant from another room through a Web-enabled moderator interface that tracked their progress. They could call for help if they had a question or ran into a problem. 3. Where did testing take place? All testing for Eyetrack III took place in San Francisco, California, in the offices of Eyetools Inc. 4. Who performed the actual testing? Eyetrack laboratory testing was performed by Eyetools Inc., a San Francisco, California-based software company that specializes in eyetracking analysis solutions for these types of user studies. The company was founded by Greg Edwards, who was the chief researcher at Stanford University during the Stanford-Poynter Eyetrack II study conducted in 1999-2000. Eyetools personnel involved in this particular project included Edwards, CEO Colin Johnson, and research manager Leslie Kues. 5. How many participants were tested in this study? What were their ages, gender, etc.? We tested 46 people in this phase of the study. (Actually, we ran 51 through the process, but were forced to toss out five people because of bad data; the shape of some people's eyes prohibit accurate tracking.) They ranged in age from 19 to 60, and represented a cross-section of the adult, Internet-using population. To participate in the research, test subjects must have reported that they were regular Internet users and had spent time recently reading news websites. The majority of participants were between the ages of 19 and 45, with only six older than 45. Here's the breakdown: Age group Gender No. tested 18-24 M 5 18-24 F 5 25-34 M 5 25-34 F 6 35-54 M 10 35-54 F 10 http://www.poynterextra.org/eyetracking2004/faq.htm (6 of 11)9/2/2004 10:13:21 AM Eyetrack III - Frequently Asked Questions 55 & up M 2 55 & up F 3 In terms of education level, the test group broke down this way: high school, 13%; associate degree, 11%; bachelor's degree, 45%; master's degree or Ph. D., 27%. So, our test pool was better educated than the general population. The ethnicity breakdown was about 75 percent anglo/caucasian, with other groups split among the remaining. Part of the reason for such a preponderance of whites has to do with the limits of the eyetracking technology we used. The Tobii ET-17 eyetracker we employed was not ideally suited to track a variety of ethnicities; it's more difficult to track the eyes of some ethnic groups, especially some Asians, because of differences in the shape of the eye. The latest release of the no-headgear Tobii eyetracker, the model 1750, tracks all ethnicities equally well -- and we would hope to utilize it in future studies. We are not publishing any analysis of Eyetrack III data based on ethnicity or educational level. The numbers of participants are not large enough to be able to break the groups into even smaller categories and still have meaningful segmentations on the data that are statistically relevant. 6. How were participants in the study recruited? Recruiters used market-research databases and placed ads on online community sites to find the participants for Eyetrack III. The recruiters then screened each person on the phone for suitable characteristics (had visited a news website recently; Internet user; not eyeglass wearer), and scheduled them for a specific test date. Each participant was paid a fee of $75 as compensation for his or her time. 7. How long was the testing period per individual? One hour. 8. What did participants do during the session? First, they received a brief overview of what to expect during the session. The moderators introduced them to the eyetracker and the testing interface. The interface automatically served them a pre-scripted combination of easy-tofollow tasks separated into four sections: SECTION 1: Catching up on the news •Participants were asked to catch up on the news by visiting five different news sites. These sites were mock news website prototypes, populated with unique real news stories, photographs, and multimedia content. (See box at right.) SECTION 2: Comparison of comprehension for multimedia vs. text •Participants first read a control text news article, then answered a series of multiple-choice questions that nominally gauged their comprehension but which were really intended to put the participants in a frame of mind that was better suited for answering the questions of the following two features. (Answers for this control question were not tabulated.) http://www.poynterextra.org/eyetracking2004/faq.htm (7 of 11)9/2/2004 10:13:21 AM THE TEST WEBSITES Here are links to the news websites* used in the study. Half the participants saw Homepage 1-5, and half saw Homepages 6-10. ● Homepage 1 ● Homepage 2 ● Homepage 3 ● Homepage 4 ● Homepage 5 ● Homepage 6 ● Homepage 7 ● Homepage 8 ● Homepage 9 Eyetrack III - Frequently Asked Questions •The group viewed an editorial feature that was ● Homepage 10 presented either in text or in multimedia. They * The sites were optimized were told ahead of time that they would be asked for viewing on the Windows questions afterward. Half of the group viewed this version of Internet Explorer feature in its multimedia format, and the other half 6.0. They do not display viewed the feature in its text format. Both groups correctly in some other were asked the same set of questions to gauge browsers. their comprehension of this first feature. •The group then viewed a second editorial feature that also was presented in both text or in multimedia. They were told ahead of time that they would be asked questions afterward. The group that viewed the first feature in text format viewed the second feature in multimedia format. Similarly, the group that viewed the first feature in multimedia format viewed this second feature in text format. Both groups were asked the same set of questions to gauge their comprehension of this second feature. Counterbalancing feature formats (text vs. multimedia) in this way allowed us to cancel out a bias affect that might have occurred if everyone had seen the same thing in the same order. SECTION 3: Multimedia editorial features •Participants received five minutes of discretionary time to select one or more multimedia features of their choosing from a list of eight options across a variety of topics. SECTION 4: Demographic questions •Participants concluded the session by answering a few personal demographic questions. 9. What were the goals of the study? Each section of the study had distinct goals related to reading news online. Below are the goals for each section. SECTION 1: •Compare viewing across different news design elements (e.g., font size, use of blurbs, number of headlines) •Compare viewing across different news website styles and layouts that we modeled after current high-traffic news sites on the Web. •Compare viewing of different article page layouts and writing styles SECTION 2: •Compare comprehension of material in multimedia format vs. text format SECTION 3: •Gather preliminary information on how people view multimedia/interactive free-form articles 10. Were all the data collected "good"? Not all, but most of it. The eyetracking equipment is fairly accurate and robust. It is designed to automatically track a person's eyes within a region of about a one-foot cube. That translates into a region large enough to track the eyes of a user who sits comfortably still in front of a 17-inch computer screen. If the person moves around more than normal or leans outside of the camera's field of view, it would stop collecting data. In a case like this where there is a lot of data loss, we drop the data for the task so that the data's absence does not negatively bias the results. In those cases, we occasionally would lose part of the session, while other parts remained fine. If a participant could not be tracked at all, we dropped the session so that it would not bias the aggregate images. (Eyetracking hardware is constantly evolving, with new equipment offering better performance. The no-headgear eyetracker we used was reasonably good about minimizing bad data, but subsequent revisions of that brand of http://www.poynterextra.org/eyetracking2004/faq.htm (8 of 11)9/2/2004 10:13:21 AM Eyetrack III - Frequently Asked Questions eyetracker are considerably better.) Of the 51 people we invited to take part in the eyetracking testing, five of them proved to be not trackable, leaving us with 46 test subjects. The chart below shows the number of test subjects whose data was used on the 10 homepages that we tested as part of the study. Any conclusion that is based on data from a homepge was based on this sample size. Half our test subjects viewed one homepage in a set, the other half viewed the other matching homepage -- e.g., HP1 and HP6 are the same design with one variable. When comparing the controlled variable between the two homepages in a set, the researchers used data from the number of people listed here. Article pages were created that matched each of these five sets. Any observation or finding based on an element in an article page is generated from data from a total possible number of people included in the set. As an example, large pictures on article pages were part of the HP5/HP10 set. Any data generated came from a total group size of 36 (20 + 16). Findings on recall in text and multimedia were based on questionnaire responses from 44 of the 46 individuals tested. Observations on multimedia news features were based on smaller samples that are reported with each observation. 11. What's with the "comprehension" testing? That's not really eyetracking, right? A major part of the Eyetrack III study involved examining how news-website readers interacted with multimedia editorial content (interactive graphics, photo slideshows, audio, video, etc.) -- the newest form of journalistic storytelling. We wanted to learn whether multimedia editorial content was comprehended as well as, better than, or worse than text articles. So we devoted a small part of the hour-long testing period to text-vs.-multimedia comprehension using multiple-choice questions. We also kept the eyetracker going during these tests, which gave us additional eyetracking data to add to our other results. 12. How accurate are the findings from this research? The answers to that question are lengthy, so we've created a separate page to address it, written by Eyetools CEO Colin Johnson. Succinctly, for those who want to move on more quickly: •The equipment used in Eyetrack III is very accurate. It can capture and report what people are looking at within a centimeter's distance when you measure a person's point of gaze on a 17-inch computer screen. •The aggregate images (heatmaps) of eye fixations and movements for the group of test subjects are a more accurate representation of viewing patterns in areas of the images shown by warmer colors (where more people looked), and less so for the cooler colors (where fewer people looked). The data in the images emphasize where people focus and what they might perceive through their peripheral vision. http://www.poynterextra.org/eyetracking2004/faq.htm (9 of 11)9/2/2004 10:13:21 AM Eyetrack III - Frequently Asked Questions 13. What's the difference between a "finding" and an "observation" as found in the various Eyetrack III reports? As you read through the reports on this website about Eyetrack III, you'll notice that some are labeled "findings" and others are "observations." The distinction between these two is that the "findings" are more rigorous and have been appropriately tested using tightly controlled variables. "Observations" are just that, observations made by the researchers that have not been as vigorously tested. For example, we report "findings" on homepage headline size, because we tested two matching homepages that were different in only one way: One used small headlines, the other used larger headlines. Half of our test group saw one homepage, the other half saw the other page. The single variable allowed us to report "findings" about headlines size. For our report on news website advertising, we didn't use such tightly controlled variables. We tested five website designs, and each design contained different ad types, sizes, and placements. We could observe interesting behaviors of our test subjects as they looked at the various ads, but these are reported as "observations," because the testing protocol was not as stringent as in the headline-size example. 14. What is the importance of "counterbalancing" and how was this incorporated in the study? If a person sees the same thing numerous times, she will begin to change his or her viewing patterns because he or she grows increasingly accustomed to it. We call this phenomenon "the order effect." When we study viewing behavior in eyetracking studies, we prefer to mix up the order in which we present the stimuli so that we can cancel out that effect. When we mix up or randomize the order of the tasks, we call that "counterbalancing." If we cancel out the bias from order effect through counterbalancing and then discover that there are differences in the data, we can be certain those differences are caused by the stimuli and not by the order. We used Counterbalancing in Section 1 and Section 2 of Eyetrack III. 15. What is the importance of controlling variables, and what does that mean for designing testing materials? By controlling variables, researchers can more effectively pinpoint the causes of differing behavior. In Eyetrack III, specific design changes on the news websites were controlled to pinpoint how viewing behavior differed across different design elements. For example, homepage No. 1 and homepage No. 6 contain the same content, but homepage No. 6 includes blurbs below the headlines. Because everything else on the pages is the same, we believe that any differences in the data between the two pages may be caused by the thing that is different -- the presence or absence of the blurbs. When there are multiple variables that are different across two or more versions that are being compared, it becomes more difficult to establish how each of the variables contributes to the difference. This challenge is particularly difficult when the number of test participants is small. By limiting the differences to just one variable, we can be more assured of the cause of the difference. 16. What variables did you test in Section 1 of the Eyetrack III testing? Homepage variables: http://www.poynterextra.org/eyetracking2004/faq.htm (10 of 11)9/2/2004 10:13:21 AM Eyetrack III - Frequently Asked Questions •Homepage templates (five designs commonly used by news websites) •Headline links only vs. headline-plus-blurb links •Headline quantity (more vs. fewer headlines) •Headline size (small vs. larger) •Text font size (small vs. larger) •Navigation placement (left, top, right) •Ad placement and types of ads (including text, animated, and rollover-expand ads) •Photo size (small vs. average vs. large) Article-page variables: •Design templates (five designs, related to five homepage templates) •Navigation placement (left, top, right) •Photos (none vs. average size vs. large size) •Text layout (1 column vs. 3 column) •Paragraph length (short vs. average vs. long) •Subheads vs. no subheads •Bulleted text vs. no bullets •Use of summary paragraph vs. none •Ad size and placement (including large "half-page," animated, and inset into article text) 17. How did you create and design the 10 mock news websites? The Eyetrack III team first identified five news website designs most commonly used by the news industry. The team then worked with designers at Morris Digital Works -- who kindly volunteered their services -- to outline and develop the mock news websites to match the needs of the study. The mock websites were designed and produced by Morris designer Nik Wilets; they were hosted on the servers at Morris Digital Works. Content was drawn from a variety of Web news sources, and was chosen to be of interest when the actual testing took place. (That is, we chose mostly "evergreen" content that wouldn't appear dated, rather than deadline news.) 18. I'm a member of the press. Who can give me more information about Eyetrack III? The following people can speak to you about this research: Steve Outing, project manager (senior editor, Poynter Institute) E-mail: steve@poynter.org Phone: 303-543-7810 Laura Ruel, project manager (former executive director, Estlow Center for Journalism and New Media; assistant professor, University of North Carolina School of Journalism and Communication) E-mail: lruel@email.unc.edu Phone: 303-596-4691 Colin Johnson, CEO, Eyetools Inc. E-mail: cjohnson@eyetools.com Phone: 415-235-0418 http://www.poynterextra.org/eyetracking2004/faq.htm (11 of 11)9/2/2004 10:13:21 AM Eyetrack III - Advertising Observations in this report: •People avoid looking directly at most ads •Visual breaks serve as barriers to seeing ads •Ads that blend into the look and feel of the page perform better -especially text ads •Size matters; "halfpage" ads perform well •Ads inset within article text get seen more •Mouseover-expand ad viewed more •Animation helps somewhat, but results are mixed •Ad real estate diminishes the further it is away from top left headline •Ad creative quality, content affect viewing behavior •Small pop-ups are quickly viewed, then closed or hidden •Tips Observations on Advertising An analysis earlier this year by the Internet advertising-services firm Doubleclick revealed that only 4-5 people out of a thousand click on Internet ads -- the continuation of a long downward click-through trend. Figures like those once fed skepticism about the effectiveness of advertising online, particularly when the dot-com bubble burst. EYETRACK III FINDINGS This report is one of many from the Eyetrack III study of broadband-era news websites. 46 people were tested for one hour each in December 2003 by Eyetools Inc. in partnership with the Poynter Institute and the Estlow Center. During the test period, each test subject viewed mock news websites created for research purposes and real-world multimedia news features. Results were published in August 2004. Not anymore. These days, advertisers count click-throughs where they matter -- such as in sponsored links like those that Google provides -- and elsewhere they assess impact based on brand awareness and by using other measures. This is one of the reasons that Internet revenue is increasing. The observations below from Eyetrack III are a different indicator of where and how ads are effective, based on what the researchers saw as 51 participants viewed mock news website homepages we created specifically for research purposes. Five website designs contained a variety of ad formats and presentations, and we could compare their performance. (Unlike other parts of Eyetrack III, the advertising testing was not conducted using tightly controlled variables. Ergo, we offer these observations rather than scientifically sound findings.) Add/view feedback on this report MORE EYETRACK III RESULTS •Overview Homepages: •5 homepage designs •Eye viewing patterns •Headlines & blurbs •Headline size •Font size •Navigation •Photos & images •Compact & extended pages Other findings: •Advertising Observation: People avoid looking directly at some ads while looking at others; performance depends on placement. The so-called "invisibility effect" of Web advertising is no myth. We found among our test subjects that there were often instances when they did not look directly at ads, even for a fraction of a second. That doesn't mean they didn't see them at all -- in some cases eyes fixated close enough to the ads to be able to view them in peripheral vision; in other cases they looked at ads directly; and in many cases, they didn't see them at all. And placement of ads mattered a lot. That being said, let's put the viewing numbers within context of click-through rates, which are typically in the low-single-digit percentages. The ad viewing http://www.poynterextra.org/eyetracking2004/advertising.htm (1 of 15)9/2/2004 10:13:30 AM Eyetrack III - Advertising •Article-page design •Multimedia content INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter percentages in Eyetrack III range between 14 and 68 percent. This is good news for advertising that focuses on branding, because it says that even if the units are not getting clicked on, the percentages that they are being viewed are higher than click tracking might suggest. Take a look at the images below (click to enlarge). The left image is how one of the mock homepages (No. 8) looked. The colorful image on the right is a heatmap or aggregate representation of the eye fixations of all the participants who looked at this page. Colors indicate the percentage of participants who looked at specific parts of the page. Red-orange means most people set their gaze in that spot; blue-to-gray means fewer did. (There's a key along the top of the heatmap image.) Homepage No. 8 Heatmap - homepage No. 8 Notice how the heatmap image shows that the majority of the group we tested focused on the editorial content (in this case, headlines and blurbs) but the direct gaze largely avoided the large ad on the left. You can see the significantly smaller number of participants who looked directly at the ad (at least briefly), and in this case it was a very large ("skyscraper") ad. Between 50 and 60 percent of the group glanced directly at the left-column ad at least once, which isn't bad (though still much less than performance of the nearby editorial content). However, the results were not so good for the top banner ad on that page; it was mostly "invisible," with only 20-30 percent of the group ever looking at it. That particular homepage was unusual because we put the navigation elements in the right column (a practice that's uncommon, but not unheard of, among news websites) and an ad in the left column. Our experiment suggested that when ads are put in the left column, they perform better than right-column ads. (This is supported by Eyetrack III observations on homepage viewing behavior, which show that normal initial eye movement around the page focuses on the upper left portion of the screen.) In examining ad-viewing behavior across four distinct homepage designs, we found that on average, most ads were seen by less than half of the group. Here's a table showing the percent looking directly at banner ads based on their placement on the page. Overall, based on looking at all the homepage designs tested, advertising placed on top and in the left column of the page was viewed most. Ads placed at the bottom of the page were viewed least. When viewing homepage No. 8, more than 50 percent of participants moved their eyes all the way down the page to the last headline. However, fewer than 25 percent http://www.poynterextra.org/eyetracking2004/advertising.htm (2 of 15)9/2/2004 10:13:30 AM Eyetrack III - Advertising looked beyond that headline to the ad beneath it. Observation: Visual breaks in design serve as barriers to seeing ads. As seen in the heatmaps (aggregate images) produced for our collection of homepages, participants tended to avoid ads when a visual barrier of either white space or a border sat between the ad and the editorial content. A border or rule, or a visible area of white space, seemed to stop many people from viewing an ad. An example of this was seen with homepages No. 4 and 9 (which are variations of the same design). Click the images below to see these pages. Homepage No. 4 Homepage No. 9 Most participants who saw these homepages stopped at the end of the headline and blurb under the "Lifestyle" heading; they did not tend to see the VW ad beneath, in the lower right of the page. What's interesting is that on homepage No. 9, there was more white space between the blurb and ad -- and that page had less viewing of the VW ad than did No. 4. More white space between editorial and ad content seems associated with less eye action on the ad. Below are closeups of this section of homepage No. 4: http://www.poynterextra.org/eyetracking2004/advertising.htm (3 of 15)9/2/2004 10:13:30 AM Eyetrack III - Advertising Here are closeups of this section of homepage No. 9: http://www.poynterextra.org/eyetracking2004/advertising.htm (4 of 15)9/2/2004 10:13:30 AM Eyetrack III - Advertising Click the images below to see full-page heatmaps of those two pages. Heatmap - homepage No. 4 Heatmap - homepage No. 9 Observation: Ads that blend into the look and feel of the page -especially text ads -- draw more eyes. The Eyetrack III researchers found that ads that blended into surrounding editorial content on a news homepage attracted people's eyes more often than ads that featured contrasting colors or designs. If an ad was approximately the same background color as the rest of the page, it received more eye fixations. An example of an ad element that was viewed more -- perhaps because it had the same background color as the site itself -- appeared on homepages No. 4 and 9. The large horizontal ad for VW ran the full width of the page and was positioned above the flag/masthead, with a white background. Here's a detail of this ad, followed by its heatmap: http://www.poynterextra.org/eyetracking2004/advertising.htm (5 of 15)9/2/2004 10:13:30 AM Eyetrack III - Advertising A contrasting background color -- instead of the predominant white -- likely would have been seen less, based on what we observed on other pages tested. We believe this ad also generated a greater percentage of people viewing it than some ads due, in part, to visual bleed (spillover) from the editorial content, specifically the nearby site flag/masthead. In this study, the site flag/masthead on all the homepages received considerable visual traffic. Researchers speculated that this may have been a result of the fact that this trial featured fictitious prototype news sites that were modeled after real sites, but which were unfamiliar to participants. When looking at the viewing sequence across the group for each of these homepage designs, the participants overwhelmingly looked at the flag early in their visit to each site. Since the flag is the "editorial content" that brought visual attention within the vicinity of the banner ad, it is possible that regular users of a favorite site would not focus as much on the flag because they would already be familiar with the site. For homepages No. 2 and 7, we included text advertising rather than web banner ads. These were job listings for the San Francisco area (where Eyetrack III testing was conducted). Here's a detailed view of the text ads for homepage No. 2. (The corresponding text ads for page No. 7 were identically designed, but extended further down the page.) http://www.poynterextra.org/eyetracking2004/advertising.htm (6 of 15)9/2/2004 10:13:30 AM Eyetrack III - Advertising These text ads had the highest viewing of all ads on all of our homepages, by far. Overall, 82 percent of people who visited homepages No. 2 and No. 7 looked at the text job ads. (The next highest performer was a "skyscraper" ad, which was viewed by 68 percent of participants.) On homepage No. 2 -- a compact page with fewer content choices than other homepages -- the text ads attracted viewing from 90 percent of study participants. The text ads on these pages recorded the longest average duration of viewing. On average, participants spent 6.9 seconds perusing the text ads. The longest a banner ad was viewed, on average, was 1.6 seconds. Observation: Size matters; "half-page" ads perform well. Larger ads are seen before smaller ones, our observations suggest. The larger ads on homepages do not, however, get viewed for any longer than smaller ads. With the exception of text ads, as noted above, all banners of all sizes on homepages were seen, on average, for between 0.6 and 1.6 seconds. That doesn't sound like http://www.poynterextra.org/eyetracking2004/advertising.htm (7 of 15)9/2/2004 10:13:30 AM Eyetrack III - Advertising a lot of time for a website advertiser to make an impression, but for branding purposes that might do. We included some extra-large ads on some of our article pages -- "half-page" banners, sized at 368 x 850 pixels. (Here's an example.) On those pages, the half-page ads received more average fixations per person than any other ad size presented alongside articles, as the chart below shows. As you can also see from this chart, the next best-performing ads were 300 x 250 pixels (example) -- medium size, placed within article text -- followed by large "skyscraper" ads (example) placed in the left column of some of our article pages (160 x 800 pixels). Smaller ads placed on article pages -including the ubiquitous 468 x 60 banners -- did poorly in comparison. Indeed, the 468 x 60 banners just barely out-performed smaller right-column ads (184 x 90 pixels). Observation: Ads inset within article text are seen more than most other ads. Notice in the chart in the item above this that a 300 x 250 pixel ad inset into text of an article performed better even than "half-page" (368 x 850) ads on article pages, in terms of the percentage of people who actually looked at the ad while viewing the page. http://www.poynterextra.org/eyetracking2004/advertising.htm (8 of 15)9/2/2004 10:13:30 AM Eyetrack III - Advertising On visits to article pages with the ad inset into the text, 56 percent of our test participants looked at the ad -- giving it an average of 4.6 eye fixations while they were viewing these pages. No other size ad on article pages got looked at by more than half of our test pool. The half-page ads did get more fixations per person on average, though, among those who looked at them -- that is, the people who looked at them were more engaged with the half-page ads. This means that both the half-page and the text-inset ads performed well, but each achieved different results. Observation: Mouseover-expand ad were viewed more than other banner ads. On homepages No. 5 and 10, we presented identical designs with one difference: One page had a static 468 x 60 pixel banner ad at the top of the page below the flag/masthead; the other page had the same size and position ad, but when the user moved his/her mouse over the ad, it expanded downward (temporarily covering editorial content) to reveal a much larger ad. (There was no clue on the mouseover ad that it would expand. Click the thumbnail below to see the expanded-view ad.) Half of our participants saw homepage No. 5, the other half saw No. 10. Article pages associated with these homepages also used either the standard banner or the mouseover-expand version. Homepage 10 - expanding ad view http://www.poynterextra.org/eyetracking2004/advertising.htm (9 of 15)9/2/2004 10:13:30 AM Eyetrack III - Advertising The mouseover-expand ad performed very well in comparison to nonexpanding ads. Only 40 percent of the people who looked at homepage No. 5 fixated their eyes on the 468 x 60 pixel ad, while 93 percent of people who visited homepage No. 10 looked at the expanding ad in the same position (but not that many triggered the mouseover expansion). Part of the explanation may be that the ad on page No. 10 included a close-up of a handsome young man's eyes. Other observations from our research have established the draw of images of the human eye. The non-expanding ad on page No. 5 had a less-compelling image. On homepage No. 10, 64 percent of people triggered the expanding part of the ad by moving their mouse over the banner -- which is significantly more than the number who saw the ad at all (40 percent) on homepage No. 5. There was no clue that this ad expanded; the only way that a person would discover the expand behavior was to move the mouse over it. It might have been the model's eyes that led to this common mouse movement. More likely, it was the common user behavior of concentrating on the upper left portion of most homepages (as demonstrated by the homepage viewing patterns findings elsewhere in Eyetrack III). The expanding ad was viewed for almost 2 seconds, on average, by those who saw it. That makes it the best-performing banner ad in our test -- though it's a distant second to the right-column text ads (job listings) that we published on one set of homepages. The mouseover-expand ad also ran with article pages associated with homepage No. 10, and we saw similar behavior. The mouseover and non-mouseover banner ads on homepages No. 5 and 10 were placed below the masthead/flag, which we believe improved the ads' performance. These ads did better in terms of eye fixations than the same size (468 x 60 pixel) ads positioned above the masthead/flag. We believe that this placement close to the editorial content of the page improved the chances of being viewed. Observation: Static ads vs. animated ads revealed mix results. For the homepages portion of Eyetrack III, we mostly used static ads. On one homepage where we did use an animated ad -- a "skyscraper" in the left column of homepages No. 3 and 8 -- we noticed a modest, but nevertheless significant, increase in spillover from the adjacent editorial content to the ad, which we believe was mostly a result of the animation pulling the eye over to investigate. For our article pages, we included a mix of animated and static ads. Perhaps surprisingly, we couldn't identify a clear trend. As you can see from the chart below, best performance of static vs. animated ads varied by ad type. Static ads got a higher number of average fixations for most ad types, but not for "skyscraper" ads. Static ads were seen by more test subjects for most ad types, but animated ads inset in article text reversed that trend; animated intext ads were seen by four times as many people as static ads in that position. http://www.poynterextra.org/eyetracking2004/advertising.htm (10 of 15)9/2/2004 10:13:30 AM Eyetrack III - Advertising Observation: Ads closer to the top left part of the page are generally seen before ads elsewhere. Ads located at the bottom or bottom right of a homepage get fewer viewings than ads positioned elsewhere. Ads closer to the upper left tend to be seen much more quickly -- within the first 5 seconds, on average. Ads in the right column were first seen between 12 and 45 seconds after a participant had entered a homepage. And two ads that we placed at the bottom of some of our homepages weren't seen for 29 and 96 seconds, respectively. (These numbers include only those people who did see an ad, with their time-to-view averaged. The bottom ad that took 96 seconds to be seen was seen by only 10 percent of participants.) Observation: Ad creative quality, content affect viewing behavior. Differences in the content of banner ads can affect how they are viewed. Some images in ads perform better than others. (We saw that above in the mouseover-expand ad where the closeup of a handsome man drew more and longer fixations than another, less visually interesting ads placed in the same position.) On article pages in our Eyetrack III testing, we rotated several creatives in some placements. These were not tightly controlled variables, but we can still offer some observations about what we saw. "Skyscraper" ads. On one set of our test article pages, we included a 160 x 600-800 pixel ad in the left column. Ads rotated included: ● ● ● ● American Express/Marriott: cream-colored, with text and a small image of a couple two-thirds down the ad (not visible without scrolling); animated text scroll IBM: bright blue background with text only, no images; animated text scroll IBM: bright orange; type plus graphic images; animated text scroll Auto Europe: static ad with several photos of a castle and various automobiles, plus type; blue background The Auto Europe ad (the only one not animated) got 3-6 more viewings than the other ads. Thirty-two percent of people who visited article pages containing this ad looked at the ad; those who looked at the ad averaged 3.1 fixations. http://www.poynterextra.org/eyetracking2004/advertising.htm (11 of 15)9/2/2004 10:13:30 AM Eyetrack III - Advertising The ad that got the most fixations (among those who looked at the ad at all) was the blue IBM ad, with 6.8. That ad contained nothing but text on a color background. We hesitate to draw conclusions from this, but it would appear that the photographs in the Auto Europe ad drew more eyes in the first place, but didn't hold them as long as did the animated graphical text ad. Half-page ads. On one set of our test pages, we included "half-page" ads (336 x 850 pixel) at the article level (but not on the homepage). We rotated three ads through these pages: ● ● ● Exxon: white background; mostly text, with animation; small static image halfway down ad British Airways: dark blue background; text "above the fold"; series of illustrated graphics; with animation of text and illustrations HBO's Carnivale: colorful animated graphic and text The HBO ad generated the most intense viewings; among those who looked at the ad, there were an average of 9.6 fixations. That compared to only 2.7 and 3.0 fixations for the British Air and Exxon ads, respectively. The HBO ad seemed, to us, to be the most visually interesting. However, only 10 percent of people who visited pages including the HBO half-page ad ever looked directly at it to record a single fixation. British Air was looked at by 37 percent; Exxon was looked at by 30 percent. We noticed this pattern elsewhere. It's fairly often the case that ads so visually compelling that you'd expect people to look at them are avoided entirely by the majority of people exposed to the page -- but those that do look directly at them spend more time exploring their content. Less visually interesting ads seem to attract more eyes, but less intently. Could it be that the "louder" ads lead people to avoid them altogether, whereas subtler and less flashy ads draw more people in? Ads inset into article text. On one set of our test pages, we included ads (336 x 850 pixel) inset into article text. We rotated four ads through these pages: ● ● ● ● Einstein: image and graphic; animated text; question and radio buttons Jennifer Aniston: Text and three photos of women; photos move up and down for several seconds to attract attention Mercedes: Text and images of two cars; animated Atkins: Text and photo of woman in bikini from behind; no animation The best performing of these ads in terms of average number of eye fixations (among those who looked at the ads at all) was the one featuring Jennifer Aniston, with 6.6 (followed by the Atkins bikini ad at 5.7). This fits with results from our testing that showed the strong draw of people's faces. This ad featured head shots of three well-known actresses, and the images moved up and down. When we look at these ads in terms of percentage of people visiting the page, we again see an inverse order. That is, while the Aniston ad was examined most intently by those who did look at it, it had the smallest percentage of http://www.poynterextra.org/eyetracking2004/advertising.htm (12 of 15)9/2/2004 10:13:30 AM Eyetrack III - Advertising page visitors looking at it at all (15 percent). The Mercedes ad had the lowest average number of fixations, but the highest percentage looking at it (30 percent). Again we see the ads we consider least visually interesting looked at most often and less intently, and the most compelling ads seen less often, but -- when looked at -- they are looked at most intently. Observation: Small pop-ups are quickly viewed, then closed or hidden. On homepages No. 4 and 9, a small pop-up ad (200 x 200 pixels) appeared the first time a participant saw the homepage. (On subsequent visits, the ad was programmed not to reappear.) The interaction with the ad across the two groups of people who saw it was very similar. Approximately 70 percent of participants did see (look directly at) the pop-up. The other 30 percent never looked at the ad. The ways in which people dealt with the pop-up ad typically fell into two categories. The most common participant behavior was to close the pop-up window within 3 seconds of it becoming visible; typically, the person looked at it just long enough to close it. The other common behavior was to ignore it -not to try to close it, but not to look at it either -- and then after about 25 seconds (on average), to click somewhere else on the page, which caused the pop-up to disappear under the viewed webpage. Of the participants who did look at the pop-up, on average they had 2-3 fixations on it for a total viewing time of, on average, 1 second. This is within the same average viewing time range as other banner ads on the homepages. The pop-up didn't perform any better or worse. No participant clicked on the ad -- only on the "close" box to get rid of it. It's worth noting that this was a small pop-up ad. Participant viewing patterns might have been different on a larger pop-up. One unexpected consequence of the pop-up ad was that it brought attention to the top of the page by appearing on top of the banner ad above the flag/ masthead. Even though the pop-up ad itself did not receive significant viewing, it did draw attention to the VW banner ad placed immediately underneath it. In at least one notable example, the person was so drawn into the banner that he/she read it completely. The image below is a single-session page detail of the VW ad as viewed by this individual. (The orange lines indicate deep reading of the content; thin lines indicate a quick gaze path between two fixation points.) As a sole bit of data, it's not particularly useful -- but it is fascinating to see how someone interested in an ad viewed it over a period of a few seconds. Perhaps this person needed a new car? http://www.poynterextra.org/eyetracking2004/advertising.htm (13 of 15)9/2/2004 10:13:30 AM Eyetrack III - Advertising Tips Here are some tips based on what we observed about advertising in this part of the research. You might consider these Eyetrack III advertising observations when placing and pricing ads on your homepage. ● ● ● ● ● ● ● If you are responsible for creating the ad content for your advertiser clients, think about making sure that your design can deliver its message in a single glance, because that might be all you'll get. If you want to insert more text on an ad unit than can be consumed in a single (lessthan-a-second) glance, then assume that the unit will have one glance to hook the reader's attention. Once hooked, you have the opportunity to draw the reader in closer, but only if that initial hook is effective. Consider designing news homepages so that ads are not set apart from editorial content too much with horizontal or vertical rules and excessive white space, which can act as barriers to viewing ads. The researchers' observations suggests that you'll get better viewing for banner ads that do not contrast too severely with surrounding editorial content. An ad that broadcasts "I'm an ad!" by using bright, contrasting colors sometimes has the opposite of the intended effect. (Of course, the content of a contrasting ad can be compelling enough to counter this tendency; as is so often the case, the quality of the content can override other factors.) We are NOT recommending that ads be presented as camouflaged editorial content. While that may attract more visual traffic, this practice would diminish your credibility. Text ads work, in part, because they look similar to editorial content, and that helps bring visual traffic to them. It also helps if they are in close proximity to editorial content, which helps draw eyes. Size isn't always the dominant factor in Web ad performance. To get the most people to actually look at an ad (for them to fixate on it for at least a fraction of a second) on an article page, insetting it into the text flow seems to work better than any other placement. But in-text placement may not give you the most intense user engagement with an ad; sheer size appears to perform better in this regard. You might consider using expandable banner ads if you want better performance than static ads offer. If you do, you might want to let viewers know the banner will expand. It's not always a great idea to surprise users with this sort of behavior. If you're going to use a mouseover-expand ad, we suggest positioning it in a normal path of user mouse movement as a way to get the expanding part of an ad seen by a lot of people. (Note: We only tested a mouseover-action ad, but many sites now use ads that automatically appear on top of editorial content and must be closed by the user. This would make for an interesting future eyetracking test. We can make an educated guess based on these findings that such an ad would be seen by most if not all users. The trade-off is that such ads annoy some users.) Should you choose to use pop-up ads on your homepage, be aware of their poor performance relative to other ads. http://www.poynterextra.org/eyetracking2004/advertising.htm (14 of 15)9/2/2004 10:13:30 AM Eyetrack III - Advertising Add/view feedback on this report Written by Steve Outing and Laura Ruel, project managers; research and tools by Colin Johnson, Greg Edwards, and Leslie Kues of Eyetools Inc. http://www.poynterextra.org/eyetracking2004/advertising.htm (15 of 15)9/2/2004 10:13:30 AM Eyetrack III - Jay Small MORE ABOUT EYETRACKING •Welcome to Eyetrack III •Eyetrack history •FAQ •How people read •The Eyetrack III team •Eyetrack is not a solution •How accurate is eyetracking? •Discussion forums Multimedia specials •Heatmaps for test homepages •Graphic: Understanding a heatmap •Graphic: Understanding an individual user session •Interactive quiz •Videos: Individual eyetracking EYETRACK III RESULTS •Overview Homepages: •5 homepage designs •Eye viewing patterns •Headlines & blurbs •Headline size •Font size •Navigation •Photos & images •Compact & extended pages Other findings: •Advertising •Article-page design •Multimedia content When It Comes to Homepages, It Is Polite to Stare By Jay Small If it's your job to design the homepage for a newspaper website, you already deserve sympathy. The organization chart may show you have one boss. But you know better. You must drive traffic from that one page to everything else on the site. So everyone else at your company whose job depends on that traffic becomes your boss when design decisions affect his or her interests. OUR HOMEPAGE UMPIRES You didn't want just one interpretation of the Eyetrack III findings, did you? Of course not. So these experts cheerfully sent their insights by email in reply to questions about the research: Site Manager Lisa DeSisto was named vice president and general manager of Boston. com in February 1999 and has been with the site since its launch in October 1995. Lisa has held various positions at Boston.com including director of marketing and vice president of strategy and marketing. She is president of the New Media Federation, Newspaper Association of America, for 2004. Usability Experts Jakob Nielsen is principal of the Nielsen Norman Group, a consultancy on usability and user experience matters. Until 1998 he was a Sun Microsystems Distinguished Engineer. Nielsen founded the "discount usability engineering" movement for fast and cheap improvements of user interfaces. He holds 73 U.S. patents, mainly on ways of making the Internet easier to use. Jared Spool, a software developer and programmer, founded User Interface Engineering in 1988. He has more than 15 years of experience conducting usability evaluations on a variety of products, and is an expert in low-fidelity prototyping techniques. UIE's conferences and road shows routinely draw hundreds of Web developers to hone their user interface skills. Industry Researcher Rusty Coats is director of new media for MORI Research in Minneapolis, and has led new media research projects for the Newspaper Association http://www.poynterextra.org/eyetracking2004/jaysmall.htm (1 of 7)9/2/2004 10:13:55 AM Eyetrack III - Jay Small INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter of America and many of the top news media companies. He also has served as content manager for several newspaper Web operations in his career, including sites for the Star-Tribune in Minneapolis and The Sacramento Bee. Developer Adrian Holovaty is lead developer for World Online, the Web division of the Lawrence Journal-World in Kansas. His background is evenly mixed between newspaper/Web journalism and computer programming. He formerly served as assistant database editor and product developer at ajc.com, the Web site of the Atlanta Journal-Constitution. Design Expert Alan Jacobson, president of Brass Tacks Design, has more than 20 years experience in the communications industry working as a photographer, designer, editor, project manager, and publishing systems analyst. He has redesigned more than 50 newspapers and numerous Websites, winning multiple awards from the Society for News Design. Stalwart news managers expect you to maintain the storied "church/state separation" of journalism and advertising matter. Meanwhile, ambitious sales and marketing managers pepper you with requests for just one more tile here, a few more links there. Oh, and Circulation asked to move that subscription promo box higher on the page. Features wants to promote a comics survey with an illustration. Did you finish the animated Newspaper In Education teaser yet? How about the oddshaped logos, selection lists, and search boxes for jobs, cars, and homes? That one webpage bears all the promotional burdens that would typically be spread through an entire printed edition of your newspaper. Your homepage begins to look as though a dozen designers from different departments each built their own piece. In fact, maybe they did: "Hey, Ma, your friends called from the quilting bee -- they're sewing like crazy but they can't find the pattern!" Now come the findings of Eyetrack III research from The Poynter Institute. Marvelous, right? Here's some new ammunition to help bring the homepage design to a standard somewhere north of "early American minor-league outfield wall." But, you ask, whose ammunition is it? How much real firepower does it pack? Whose design objectives does it support? In short, what good is it? That depends on whom you ask. I asked people I knew would bring widely varying perspectives to their analysis (see profiles in the accompanying box, Our Homepage Umpires) On some points, these experts came to similar conclusions; on others, they disagreed. Gotta Have the Nav Whether your navigation menus cascade from a left-side bar or run across the top of the page in a compact stripe, their effectiveness makes or breaks your site. http://www.poynterextra.org/eyetracking2004/jaysmall.htm (2 of 7)9/2/2004 10:13:55 AM Eyetrack III - Jay Small Which nav formats get more notice? Eyetrack III participants noticed and fixated on top navs more often than other placements. And they checked right navs more often than left. "I've read that one reason right-side navigation is better than left-side navigation is because browser scrollbars are typically on the right side of browser windows," said Adrian Holovaty of World Online, "and having all the navigation widgets on one side of the screen requires less mouse movement." Holovaty's not the only one who's read that. But before you fire up Dreamweaver and start laying out a right-side nav bar, whoa, Nellie! Others in our expert group say you should think about whether Eyetrack III participants' fixation on a nav bar is a good thing no matter where you put it. "I would argue that the more time people spend with the nav bar, the less effective the nav bar is," said Alan Jacobson of Brass Tacks Design. "If you have to stare at it to figure it out, it must be a bad nav bar." Jakob Nielsen was even more adamant. "Having users spend more time on a task is not an indication of a better design, it's an indication of a worse design," said Nielsen, principal of Nielsen Norman Group. "Since people are used to finding the navigation on the left or the top, that's where it should stay. Instead of forcing users to spend more time on deconstructing your page layout and navigation features, it's better to have them spend the time on engaging with your content." A Picture Paints How Many Words? Plenty of news Web designers, whether they came from the print side or not, have had the concept of "dominant art" drilled into their heads: big picture + middle of screen = the eye-grabber. Eyetrack III findings don't line up with that formula. Participants tended to focus on the dominant headline of a homepage first, not the main photograph or image. That sounds about right to Nielsen. "This confirms many usability studies over the years, where the text generally determined the majority of users' actions," he said. "Pictures are less important, but that doesn't mean that they should be eliminated, only that they should be seen as supplements to the text." In Rusty Coats' research, however, users tell him a different story. "This flies against a lot of what I've seen in usability tests and other ethnographic/observational research," said Coats, of MORI Research. "In those tests [we] conducted, photos were the first items on a homepage noticed, even more so when the photograph was dynamic. The biggest complaint we hear from online users is that there is too much text on the page and that they'd prefer seeing more art, even if it's icon-sized." At least one big news site sees usage patterns that align with the Eyetrack findings. "We estimate the upper-left, or lead spot, on our homepage -- which generally has the largest headline -- is worth a click-through premium of 50-70 percent http://www.poynterextra.org/eyetracking2004/jaysmall.htm (3 of 7)9/2/2004 10:13:55 AM Eyetrack III - Jay Small over other placements on the page," said Lisa DeSisto, Boston.com general manager. "The photo we typically run at the top center of our homepage carries a premium, too, but it varies hugely depending on its strength." This finding ultimately speaks to the need for compelling wording in primary headlines and strong imagery in primary photographs. Or, if in doubt, play a design trump card: package the lead headline and main image together in a "poster" format, a la MSNBC.com. Blurring The Lines on Blurbs Eyetrack III's bottom line on homepage "blurbs" (a.k.a. "teasers," "decks" or "readouts") seems to be that they sometimes get noticed, but usually only under certain complex circumstances, and sometimes at the expense of time users might spend scanning other headlines. Huh? Just remember this: You can find reasons in the study either to use or not to use blurbs. Our experts' experiences provide additional context for either choice. "We've found in usability studies that blurbs are only good for main-story treatment and perhaps a feature breakout down-page," Coats said. "How the blurbs are written -- dynamic and catchy vs. dry-bone -- has a lot of impact on main-story readership but very little on down-page blurbs. Readers skip over those blurbs like a flat rock on a lake." DeSisto's site team didn't wait for Eyetrack III to make plans. "We're in the middle of a modest homepage redesign and during our research we discovered that the content of our headlines and blurbs is too often redundant," she said. "We are considering eliminating blurbs and just offering headlines." With headlines only, Boston.com might ponder another Eyetrack III finding, that participants' eyes tended to fix more often and longer on the first word or two of headline links. Holovaty sees that finding as a rationale for some content management tweaks. "Here's yet another good reason news sites shouldn't be sucking in headlines automatically from the print product," he said. "Site maintainers should take the time to craft Web-specific headlines -- if only for homepage stories. And publishing systems should allow multiple headlines for a single story, based on the story's context." Your Ad Here -- Call 555-BLIND Most newspaper sites tend to place banner ads across the very top of pages, and tile or tower ads down the right side, because ... well, just because. But in Eyetrack III, the display ads that participants noticed most ran between site branding and remaining first-screen content, or down the left side. By "noticed most" the researchers mean "saw for about a second and a half, at best." That duration may sound insignificant, but as DeSisto notes, low yield is still yield. "At Boston.com, it is essential we offer positions throughout the page because of the sheer volume of pageviews the homepage generates. So while http://www.poynterextra.org/eyetracking2004/jaysmall.htm (4 of 7)9/2/2004 10:13:55 AM Eyetrack III - Jay Small click-through yield on ads below the fold is lower, the overall number of clicks is high because of the volume." Only text ads seem to escape banner blindness. But is that just the novelty of the format, Dr. Nielsen? "Other ad types have registered higher click-through rates in the first few months after they were introduced only to suffer declining performance in later periods," Nielsen said. "Since text ads have been doing well for more than three years now, I now think that they have survived past the novelty stage and that they may be here to stay." Coats said text ads work best in the proper context. "Text ads rock on pages where users are already in a 'reading' mode," Coats said. "It's just an extension of their reading experience, whereas blinking tiles are not." Not Where Eyes Go, But Why Before taking these findings to heart or attempting to apply them in hopes of making better homepages, note that some of our experts expressed concerns about the overall research methodology and validity of the results. They mostly admired the fundamentals of the eyetracking method itself. Eyetracking, some said, could enrich data gathered in many other forms of user research. Their beefs about Eyetrack III, in particular, focused on these other factors. "Neutered" content: "The big flaw in this study is that it assumes that the actual news has no effect on the reader. But, we all know that isn't true. And study after study shows that it isn't true," said Jared Spool of User Interface Engineering. "This is amplified in this particular study because they explicitly 'chose mostly "evergreen" content that wouldn't appear dated, rather than deadline news.' Yet it is that very deadline news that drives people to online news sources on a regular basis. "I'm having trouble with every finding in this study because the mock-up content was so badly neutered and no controls or measures are reported about the relationship that the users have with the mockup content," Spool said. Sample sites: "I think it's safe to say that most readers of news sites are habitual," Holovaty said. "If the news site I work for were to participate in such a study, I'd want the sample site to be the actual site, and I'd want some of the subjects to be experienced users of the site. How much of the test subjects' eye movements were a result of never having seen the test site before?" Location: "Fifty-one participants in San Francisco? And that's representative of what?" Coats said. "I would not redesign a news website in Kansas City, Atlanta, or Davenport, Iowa, based on what 46 people said in the ultra-wired, ultra-Web-savvy, ultra-dot-com Bay Area." Motivations and reactions: "This data only tells us what people see when they use a site that looks like the test site," Jacobson said. "It doesn't tell us if they like the site; if the site is easy or difficult to use; what kind of site they would prefer; or how they would react to a site that is different in appearance. http://www.poynterextra.org/eyetracking2004/jaysmall.htm (5 of 7)9/2/2004 10:13:55 AM Eyetrack III - Jay Small "I'd prefer to get these questions answered by testing today's typical models against designs that are much different than current industry standards. Maybe eyetracking plus usability studies is the answer," he said. Nielsen extended that thought. "Eyetracking and traditional user testing share one important limitation in being reactive: they can only evaluate designs that already exist," he said. "Usability evaluation remains a definite necessity since we are very far from having websites that are truly easy to use. "But we also need to go further to discover features that users will want and that will allow them to do things that are currently not possible," he said. "Unfortunately, we can't just ask people what they want ... You can't even rely on people's predictions of whether or not they will use a proposed new feature or how much they would be willing to pay for it." The research coordinators, Steve Outing and Laura Ruel, acknowledged that Eyetrack III measured what participants looked at, but could not always gather why. That's why, they reminded, the best uses of and results from eyetracking may be yet to come. "We could only do so much in this study, which we've all along considered to be a preliminary look at broadband-era news websites," Outing said. "I hope that subsequent eyetracking research, whether done by Poynter or others, can dig deeper and address alternative views that we didn't have the time or resources to do." Until then, in addition to our heartfelt sympathies, homepage designers can still take away some useful nuggets from this research: ● ● ● ● Content compels a glance as much as anything in the designer's toolbox. You can catch a gaze with a surprising word or phrase (note the "FCUK" example) even downscreen on an ugly page. But even a page full of objects designed precisely to attract attention will draw yawns if the audience finds nothing worthy of holding attention. Navigation? The only safe thing to say is it's a bad idea to place primary navigation three screens down. You can take that to the bank. A headline on the Web serves as a call to action, in a different way from print. Heads written to fit print layouts may or may not succeed as Web enticements to click through. When in doubt, rewrite -- and if you do, consider putting the "power words" in front unless it's too awkward. With the possible exception of text ads, both Eyetrack III and the Magic 8-Ball seem to offer the same forecast for display advertising on homepages: "Outlook Not Good." Even as standard formats grow in size, and even as designers try them in new positions on the page, eventually those blinking, flashing commercial messages fade into site visitors' peripheral vision. Unless they contain the term "FCUK," of course. See? Made you look! [ ADD A COMMENT/READ COMMENTS ON EYETRACK HOMEPAGE DESIGN RESULTS HERE ] Jay Small is the director of product development for Belo Interactive, which runs websites for local newspapers, including The Dallas Morning News and The Providence Journal, and TV stations, including KING in Seattle and KHOU in Houston. He runs a weblog and e-newsletter on Internet design at smallinitiatives.com. http://www.poynterextra.org/eyetracking2004/jaysmall.htm (6 of 7)9/2/2004 10:13:55 AM Eyetrack III - Jay Small http://www.poynterextra.org/eyetracking2004/jaysmall.htm (7 of 7)9/2/2004 10:13:55 AM Eyetrack III - Jeff Glick MORE ABOUT EYETRACKING •Welcome to Eyetrack III •Eyetrack history •FAQ •How people read •The Eyetrack III team •Eyetrack is not a solution •How accurate is eyetracking? •Discussion forums Multimedia specials •Heatmaps for test homepages •Graphic: Understanding a heatmap •Graphic: Understanding an individual user session •Interactive quiz •Videos: Individual eyetracking EYETRACK III RESULTS •Overview Homepages: •5 homepage designs •Eye viewing patterns •Headlines & blurbs •Headline size •Font size •Navigation •Photos & images •Compact & extended pages Other findings: •Advertising •Article-page design •Multimedia content When, How to Tell Stories with Text, Multimedia By Jeff Glick Deputy Managing Editor, South Florida Sun-Sentinel Is a multimedia approach to online storytelling better then a textbased approach? You'll find no easy answers in the latest Eyetrack III research released by The Poynter Institute, Estlow Center for Journalism & New Media, and Eyetools. You will find the study offers insight into how to consider and approach multimedia storytelling presentations. Here is what I take away from the latest research: ● ● ● ● ● Navigation will make or break your presentation. Not every story will benefit from multimedia. Presentations that showcase real people and real things will always do better with users than those that don't. Anticipate users' needs based on the content presented. Interactivity can add to the user experience. The research validates a lot of the successes and mistakes we've made along the way at the South Florida Sun-Sentinel in our attempts to tell stories online. One of our first mistakes was making navigation too complicated (Haiti: The Eroding Nation). The Eyetrack observations clearly indicate that users do rely heavily on text and clear queues on where to click. Navigation should be simple, easy to use, and obvious (Marine Attractions: Below the Surface). Leveraging sound to draw attention or define navigation actions may be helpful. Although there's no current research to back it up, our philosophy is to not reinvent the wheel on every project. We believe that over time users will become familiar with the navigation, making it easier for them to focus on the story, not the technology. Future studies may want to look at whether users benefit from repeated exposure to a standard multimedia interface, and whether comprehension increases as the user becomes comfortable with the style. Consider the User Because most of us have limited resources and limited time to produce print, online, and on-air graphic presentations, we have to choose our projects wisely. Not every story should employ multimedia. The study shows that in many cases text may be the more effective approach. As journalists and http://www.poynterextra.org/eyetracking2004/jeffglick.htm (1 of 3)9/2/2004 10:14:06 AM Eyetrack III - Jeff Glick INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter storytellers we've got to know how and why users will benefit from multimedia before we attempt to produce it. What will the user take away from the experience? As a rule of thumb, subjects that would be treated visually in a newspaper are great starting points for multimedia presentations. The same graphics and photos that explain unfamiliar concepts or show what can't normally be seen can be turned into powerful multimedia. In our shop, we try very hard to choose subjects, scenarios, or events that don't illustrate the obvious (Voting Booth Simulator). If it's obvious, a nonmultimedia text presentation is probably the best way to inform. That is probably why Eyetrack's testing of a New York Times graphic that explains what a cupola is (a furnace that melts tons of scrap) scored so well with participants. Another consideration should be the human factor. As seen in Eyetrack's heatmap results, people look at people. The human form is a powerful visual image. Some of the projects (electoral tracker) my staff perceives to be great work were not as lauded as were the image and audio-oriented projects (AIDS in the Caribbean) that focused on people. Many websites utilize daily picture galleries to attract and inform their users. We do on our site, and it has become one of our most viewed features. When our team decides on a multimedia approach, we always ask: what tools (audio, video, text, sound effects, animation, and interactivity) do we employ and how do we integrate them into a cohesive story presentation? Eyetrack shows that when quizzed, those that viewed the multimedia version of the "Dangerous Business" story had a lower retention rate than those that read it as text. But it also makes the observation that "multiple streams of conflicting information" such as an animated photo, text, and audio, may be too much for a user to take in all at once. This is particularly true if the user isn't given any control of how to view the story. In a standard printed or online text presentation, a reader can stop, pause, or reread any word, line, or paragraph until it's understood. And they're never forced to pay attention to multiple items at once. Multimedia should be no different in that it should give the user control of the pace. The reader is experiencing the story and should have the ability to stop, pause, and replay an animation, text, or audio. That is why it is so important for us as creators of these presentations to understand the tools and know how and when to employ them. Good Design Is Universal Multimedia should be an experience, not a download. Besides navigation, the look and feel contributes to user expectation. Sites such as the New York Times, El Mundo, and MSNBC.com do a great job creating a cohesive presentation style that bends, shifts, and morphs to accommodate all kinds of content. In print design we talk about concepts such as hierarchy, contrast, and flow. These concepts are universal and can and should be applied to multimedia. Many of our online presentations pace users through a project (Spiegel Grove), http://www.poynterextra.org/eyetracking2004/jeffglick.htm (2 of 3)9/2/2004 10:14:06 AM Eyetrack III - Jeff Glick often playing an animation with audio to completion before the text appears. And when text is on the screen, the project waits for the user to signal that he or she is ready to proceed. The use of color, position of elements, and the timing of the content stream are critical in directing users through a presentation. Creating options for linear or non-linear viewing allows users to view things the way they want. There are times when a presentation and the interactivity it provides for the user can override the need for options. One such example is a simulator we built to give users a fun and educational idea of what it might have been like to operate the Civil War era submarine H.L. Hunley. In this project, the user is able to actually pilot the sub on a mission, and although we don't have any data on information retention rates, the sheer number of positive comments we received legitimized the approach in our eyes. Looking Ahead This study validates some successful practices, such as: ● ● ● ● ● Repurposing content is good. Web purists don't want to hear it, but most newsrooms put out an enormous amount of great content, much of which never gets used or used in a way that has impact and helps to better tell a story. In print, we may publish several photos with the story. Online, we can publish photos enhanced with audio captioning or narration that can stand alone. On-air, we may run a two-minute linear video package. Online, we can provide more detailed video packages that can be viewed in a non-linear way. In print, we can illustrate complex subject matter. Online, we can animate it and actually show it. Give users choices. Multimedia and text presentations can and should coexist. We now package the complete stories within our multimedia presentations at the Sun-Sentinel. Don't create in a vacuum. Integrate the multimedia workflow into the normal story planning process of your newsroom. Team tackle. Most newsrooms don't have large interactive resources. So develop very close relationships between online, on-air, and print newsrooms. The Eyetrack III study just begins to scratch the surface of multimedia storytelling potential and what we must consider as we continually evolve the medium. [ ADD A COMMENT/READ COMMENTS ON EYETRACK MULTIMEDIA RESULTS HERE ] http://www.poynterextra.org/eyetracking2004/jeffglick.htm (3 of 3)9/2/2004 10:14:06 AM Eyetrack III - Kinsey Wilson MORE ABOUT EYETRACKING •Welcome to Eyetrack III •Eyetrack history •FAQ •How people read •The Eyetrack III team •Eyetrack is not a solution •How accurate is eyetracking? •Discussion forums From Banner Blindness to Text Ads, Placement and Size Matter By Kinsey Wilson VP and Editor in Chief, USAToday.com Editors rely on an imperfect blend of news judgment, style conventions, audience metrics, usability studies, and occasional reader feedback to come up with a smart website design. But on the business side, publishers have a much more direct way to measure the effectiveness of ad placements on their sites. Multimedia specials •Heatmaps for test homepages •Graphic: Understanding a heatmap •Graphic: Understanding an individual user session •Interactive quiz •Videos: Individual eyetracking Money. EYETRACK III RESULTS •Overview Homepages: •5 homepage designs •Eye viewing patterns •Headlines & blurbs •Headline size •Font size •Navigation •Photos & images •Compact & extended pages Other findings: •Advertising •Article-page design •Multimedia content Armed with surveys and statistics that help them measure the short- and longterm performance of their Internet ad buys, many advertisers have already learned how to optimize the placement, appearance, and positioning of their creative -- whether it is a simple text link at the bottom of a page or a fullmotion video interstitial. Better ad treatments generally command more ad dollars. So publishers -particularly those whose sites rely on display advertising for the bulk of their revenue -– have become highly attuned to the nuances of ad positioning and design. Eyetrack III quantifies for the first time just how quickly and intently readers view an ad -- valuable intelligence in a performance-driven business. But the findings seem likely to reinforce what advertisers and publishers already have learned through hard experience and through data already at their disposal. Michael Zimbalist, executive director of the Online Publishers Association (OPA), a trade group representing many big-name news websites, says the real value of the study, at least for advertisers, may lie more in the details than the broad strokes: the finding, for example, that small visual breaks, even white space, can serve as unintended barriers to viewing ads. But at the end of the day, Zimbalist said, "The level of engagement people have with ads depends on how relevant it is and how creative the work is." The push to improve the performance of Internet advertising has been on ever since the tech bubble burst. Remember "banner blindness"? When venture money dried up and businesses began demanding real returns on their Internet ad buys, a chorus of complaints rose over the ubiquitous rectangular ads that most news websites parked at the top and bottom of their pages. Small in size (468 x 60 pixels), and walled off from the rest of the content, they http://www.poynterextra.org/eyetracking2004/kinseywilson.htm (1 of 4)9/2/2004 10:14:11 AM Eyetrack III - Kinsey Wilson INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter were easily ignored by readers. In what had suddenly become a buyers' market, advertisers began pressing for bigger bolder units. C|Net, a technology news site dependent upon online revenues for its survival, was one of the first to pioneer the use of so-called poster units, placing them square in the middle of its news pages. Readers could scarcely miss them. As other sites began to follow suit, the Interactive Advertising Bureau (IAB), an industry trade group, devised what it termed the "universal ad package": a standardized assortment of more commanding ad units. In addition to posters, the package included a vertical banner and a new oversized horizontal banner, renamed a "leader board." On most sites, posters are situated in the middle of the content -- where Eyetrack found they command the reader's attention -- while banners and leader boards are generally relegated to the margins, where the findings suggest they are more easily ignored. The market already has taken these differences into account. The New York Times, for example, charges a $20 CPM (cost per thousand ad impressions) for old-style banners, a $30 CPM for vertical banners and leader boards, and $40 for its poster units. That latter highest rate also covers the half-page ad, pioneered by the Times in April 2003 and mimicked by other sites -- consistent with Eyetrack's findings that size and proximity to content yield better performance. The study findings also line up with other industry trends. Eyetrack found that ads superimposed on the page -- mouseovers and pop-ups were the two examples tested in the study -- perform better than traditional banners, something advertisers figured out early in the game. Sites like USAToday.com and CBS MarketWatch initially met the demand for these more intrusive ads with one-of-a-kind custom solutions. But as demand grew, entire businesses were spawned -- Unicast and Eyeblaster are examples -- that could deliver cost-effective out-of-the-box solutions on any site, with no custom engineering. As the Eyetrack study suggests, these ads command attention but don't hold the reader for very long. "I thought it was encouraging that ads that were superficially flashy got more attention but less involvement," said Dave Morgan, CEO of Tacoda Systems, which produces software to help websites manage online ads. "Having something that stands out is not what you want to be doing on the Web. What seemed to be performing well were ads that had a consistency to the style and flow of the page." That's one reason many large news websites have imposed frequency caps and other limitations on intrusive ads that obscure content or momentarily interrupt the reader experience. At the other end of the ad spectrum, Eyetrack found that simple text ads had the highest viewing of all ads on all of the homepages it tested. Again, no surprise to the ad community. So-called sponsored link or paid search advertising -- text-only ads targeted to match the subject matter on the page -- has emerged as a significant ad category, surpassing online classified http://www.poynterextra.org/eyetracking2004/kinseywilson.htm (2 of 4)9/2/2004 10:14:11 AM Eyetrack III - Kinsey Wilson with $1.3 billion in revenue in 2003, according to JupiterResearch. In part, this may be because text links more closely resemble -- and may even be mistaken for -- editorial content. But it's also because these ads are engineered to match the reader's interest and intent: A story about Google's initial public offering, for example, will likely contain links to companies selling analyses of pending IPOs. Because of that precision targeting, "Many of us have known for a long time that text ads have a higher response rate than graphic ads," said Steve Yelvington, who is responsible for audience development strategy for Morris Digital Works, the Internet division of Morris Communications. (Disclosure: Steve Yelvington and Morris Digital Works provided technical support for the Eyetrack III research.) "This is the first study I've seen that examines what happens among people who don't click," Yelvington said, a finding he hopes will "buttress the argument that legible, simple advertising is a powerful model that we should embrace instead of being lured away by bad ideas like pop-ups, pop-unders, and banners that talk." Morgan echoes that sentiment and suggests that advertisers might do well to give greater visibility to simple text ads, which usually are consigned to the lower regions of a page. "What we see as the Web evolves is that a well-written sentence is more important than an animated GIF," Morgan said. Perhaps. But, for now, publishers see text ads and banners representing two very different ad models. The latter, discreet, targeted, action-oriented, is the online equivalent of direct mail. And the rates charged are directly pegged to the response rate (measured in click-throughs). Banner and poster advertising, at its best, is intended to engender broad-based brand awareness, and the rates charged for those ads increasingly reflect not click-through metrics (which tend to be low) but the overall product lift. With the help of groups like the IAB and the OPA, news sites have sought to demonstrate that online advertising can be a powerful, cost-effective part of the overall media mix and actually improve the effectiveness of TV ad buys -– not just drive direct-response marketing. The ability to generate that kind of lift only improves as publishers refine their ability to target specific demographic and behavioral segments of their audience, something most large sites are already doing. Finally, what about Eyetrack's finding that people avoid looking directly at many ads on most news homepages, adding to what the findings term the "socalled 'invisibility effect' of Web advertising"? Zimbalist is dismissive of this finding, saying an argument can be made that advertising on the Web performs far better than it does on television or in print, where people may throw away entire newspaper sections (and their paid ads) without ever looking at them. "We know that TV commercials are viewed by only 9 percent of the audience" watching the show in which the commercials appear, he said, citing a Forrester Research study. By contrast, in the Eyetrack study, he said. "People very http://www.poynterextra.org/eyetracking2004/kinseywilson.htm (3 of 4)9/2/2004 10:14:11 AM Eyetrack III - Kinsey Wilson clearly observed the offer in the Amex/Marriott skyscraper. They [just] didn't do so with the intensity with which they scanned the news headlines." Even here, the market may have already accounted for these findings. Homepage placements -- based on sheer volume alone -- can generate a lot of cash. But as a rule, the rates charged for those ads are lower than the same ad that appears on a more narrowly focused inside page. After all, says Bowen Dwelle, chairman of AdMonsters, a trade group for online advertising operations and technology, "I'm not going to The New York Times to see what the latest ad is on the page ... There may be some advertising that catches my eye ... [But] I'm going for the content, so I look at the content." [ ADD A COMMENT/READ COMMENTS ON EYETRACK ADVERTISING RESULTS HERE ] Kinsey Wilson is Vice President & Editor in Chief of USAToday.com and a member of The Poynter Institute's National Advisory Board. http://www.poynterextra.org/eyetracking2004/kinseywilson.htm (4 of 4)9/2/2004 10:14:11 AM Eyetrack III - Homepage Viewing Patterns Findings in this report: •Eyes fixate in the upper left •Users look at the first few words of headlines •Users read five headlines before clicking •Tips Add/view feedback on homepage reports MORE HOMEPAGE RESULTS ARTICLES •5 homepage designs •Eye viewing patterns •Headlines & blurbs •Headline size •Font size •Navigation •Photos & images •Compact & extended pages OTHER EYETRACK III FINDINGS •Overview •Advertising •Article-page design •Multimedia content INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter Viewing Patterns for Homepages Once users visit your homepage, where do they look first, and what drives them to look there? Not surprisingly, our left-to-right reading behavior in Western culture seems to greatly influence these responses. In Eyetrack III, we observed that the upper left corner of a page seems to be the preferred starting point for most online news users. However, the location of key elements -- such as headlines and the flag -- also seem to be powerful forces in determining reader attention. EYETRACK III FINDINGS This report is one of many from the Eyetrack III study of broadband-era news websites. 46 people were tested for one hour each in December 2003 by Eyetools Inc. in partnership with the Poynter Institute and the Estlow Center. During the test period, each test subject viewed mock news websites created for research purposes and real-world multimedia news features. Results were published in August 2004. Overall, the upper left-hand side of a homepage -- especially when dominated by headlines -- seems to be the consistent spot where online news users look first. We observed that several factors contribute to this scanning pattern, but among the most influential are the placement of main story headlines and the website's flag. After viewing content in the top left corner or the main headlines on our prototype homepages, participants' eye movements usually involved scanning back and forth across the homepages and ending in the upper right-hand quadrant. Our participants scanned headlines that were in the upper left first. However, once there, they generally took only a quick look at the first few words before moving on. We found that the first 1-2 inches of each headline is where most eyes fixated, suggesting that the first few words are key in drawing reader attention. This situation is particularly true when headlines are in a list format or when headlines and blurbs have a similar look and feel. As seen on all heatmaps of our test homepages, concentration of viewing is leftsided when participants are perusing a list. (A heatmap is an aggregate view of all participants' eye fixations on a page; a fixation is where the eye paused to look for at least a fraction of a second.) Data also suggest that approximately five headlines are viewed on a homepage before viewers move on. This observation does not seem dependent upon the number of headlines available to view. http://www.poynterextra.org/eyetracking2004/viewing.htm (1 of 9)9/2/2004 10:14:20 AM Eyetrack III - Homepage Viewing Patterns Observation: When viewing homepages, eyes initially tend to fixate in the upper left and finally move to the lower and upper right. TAKE AN EYETRACKING INTERACTIVE QUIZ Think you can predict where most people look -- and don't look -- on a news webpage? Take this interactive quiz and find out. Information from Eyetrack III suggests that this pattern may be partially caused by the learned instincts of readers and partially caused by the design of the pages they view. We believe users tend look to this area for two primary reasons: 1. The larger, top headlines (or links to the biggest news stories) are located here. The way the site's editors ranked the stories appeared to be important to our participants (though we can't know that for sure). 2. The news organization's flag (or branding element) is located here. Our participants appeared to have found it important to assess the credibility of their information source. The researchers surmised that perhaps because the sites our participants viewed were all new to them, they were drawn to look at the page flag first. The diagram below shows an average overall viewing sequence compiled from eyetracking data on our test homepages. http://www.poynterextra.org/eyetracking2004/viewing.htm (2 of 9)9/2/2004 10:14:20 AM Eyetrack III - Homepage Viewing Patterns As the diagram indicates, the upper-left quadrant gets a fairly complete glance. Not only is it the viewing start point in this area, but viewers also tend to peruse the entire area before moving on. Next stop -- a cursory look at the lower part of the upper right hand quadrant, followed by the same in the upper part of the lower left-hand quadrant. The eyes seem to move back and forth again to outlying areas of these two quadrants. Finally, users scan the right-hand side of the page before leaving. The sequence is fairly logical when considering that Westerners read from left to right. It also is consistent with traditional teachings of readers to scan printed pages in a Z-shaped pattern. Another way to think about these sections is to break them into priority zones. Given the designs tested, the researchers determined that some regions within the page have more immediate prominence than others. http://www.poynterextra.org/eyetracking2004/viewing.htm (3 of 9)9/2/2004 10:14:20 AM Eyetrack III - Homepage Viewing Patterns Thought about in a more general way, we can see how the upper left is viewed more quickly, on average, then the next layer out, then the outer periphery. However, it is important to remember that viewing patterns are affected by what page elements exist in particular locations. Headline placement, images, and blurbs are strong factors in determining viewing patterns. Our homepage designs that contained the largest lead headlines closer to the center of the page received the most initial gazes in that area. Remember, the visualization above is the average of eight different page designs. To illustrate this point, let's review the specific pages themselves and take a closer look. More observations: When analyzing viewing sequences on individual pages we observe: 1. Readers most often view the top headlines (wherever they may be) and the site's flag (or logo). 2. Visual elements such as blurbs can sometimes override this tendency. 3. The visual flow tends to move out from the starting point of top headline and flag/logo combination to the periphery of the page. Our first homepage complies with the pattern in the overall average. Participants view the top-left quadrant first, then move to the periphery. The top-left quadrant on this homepage is where the top headlines and flag/logo reside. http://www.poynterextra.org/eyetracking2004/viewing.htm (4 of 9)9/2/2004 10:14:20 AM Eyetrack III - Homepage Viewing Patterns This tendency changes, though, when blurbs are introduced. It appears the blurbs pull the readers down the page and keep them in this area longer -- long enough to discover the picture, before the natural tendency to check the flag/ logo kicks back in. See how this page has individuals looking to the picture before reviewing the flag/logo. http://www.poynterextra.org/eyetracking2004/viewing.htm (5 of 9)9/2/2004 10:14:20 AM Eyetrack III - Homepage Viewing Patterns In other homepages, we see the same tendency -- people look to the top headlines and/or the flag/logo interchangeably as their first item of viewing. We suspect that this is driven, in part, because they are on a news site to read news, hence their interest in the headlines. But before they dive into the headlines too deeply, they appear to want to be sure that they know the source of these headlines. Can the source be recognized or trusted? If these readers had visited their favorite news site rather than the mock brand that we gave them, we expect that they would bring to that browsing session familiarity with the site and the trust that they have already established with the organization. In future studies, it will be interesting to investigate whether readers in that situation would still look at the flag/logo or would ignore it because they already are familiar with a site. It is important to note here that our data suggest that the placement of elements on a page goes hand-in-hand with the content of those elements when determining a typical viewing sequence for online news readers. For example, on one of our test pages, the main headlines were placed more toward the center of the page, and for that page, viewing tended to begin in the center. Take a look: http://www.poynterextra.org/eyetracking2004/viewing.htm (6 of 9)9/2/2004 10:14:20 AM Eyetrack III - Homepage Viewing Patterns Observation: When viewing homepages, users generally look at only the first few words of headlines before moving to another headline or page element. Online news readers concentrate viewing on the left-hand side of headlines (the first 1-2 inches). This data indicate that the first few words of a headline are critical in drawing user attention. This situation is particularly true when headlines are in a list or when they have a similar look and feel. Take a look at the heatmap of this sample test page below. (A heatmap is an aggregate image showing overall eye activity on a webpage. Red-orange areas indicate the most eye activity, blue-black the least.) It is fairly representative of the group in this respect. (Click to enlarge.) Heatmap -- homepage No. 3 Notice the deep red and orange tones concentrated on the left-hand side of the headlines. These colors indicate that almost all subjects looked at these parts of http://www.poynterextra.org/eyetracking2004/viewing.htm (7 of 9)9/2/2004 10:14:20 AM Eyetrack III - Homepage Viewing Patterns the test page. Also notice the deliberate vertical viewing down the stack of headlines. This observation is typical. Many of our test pages with stacked headlines seemed to generate user behavior where eyes moved down the lefthand side of the list. Note that the red and orange colors do not extend the full width of each headline, indicating a drop off in the number of users who read headlines from start to finish. The majority of participants seem to spend time with only the first few words of the headlines they view. Another finding worth mention concerns use of an unusual keyword in the first few words of a headline. As the page above also shows, the use of "FCUK" (the name of a controversial clothing manufacturer) as an initial word in a headline far down on a homepage is an eye-grabber. We believe two factors contribute to this catchiness. First, the increase in viewing of "FCUK" is likely because of its perceived shocking content. Secondly, viewing may be strongly influenced by the fact that it is a short word in all capital letters. These make the text different than all the surrounding text and visually stand out in peripheral vision, thereby drawing attention to it. Eyetools researchers refer to elements such as these as "widows and orphans." Similar to the meaning of these terms in newspaper design, they are points of visual discontinuity. Observation: Users read a minimum of about five headlines, regardless of the number of headlines on a homepage. The average number of headlines viewed varies with the number of total headlines on the homepage. (This number includes repeat visits to a specific homepage during a user's session.) However, there appears to be a minimum number of headlines viewed, on average. As seen in the graph below, the number of headlines viewed increases with the number of headlines available. However, a minimum of about five headlines, on average, are viewed even when the number of headlines available is small. The number of viewed headlines does not rise significantly higher than five until the number of headlines available reaches about 20. Subjects in this study may have been scanning pages longer and more deliberately, though, than they normally would because they were in a testing environment. http://www.poynterextra.org/eyetracking2004/viewing.htm (8 of 9)9/2/2004 10:14:20 AM Eyetrack III - Homepage Viewing Patterns Tips ● ● ● Readers are first drawn to the flag/logo and your top headlines, especially when they are located in the upper left of your page, so you may want to place important content in these areas. You might use other elements such as blurbs to override this instinct if there is some other content that you wish your readers to see before all else. Also remember that large headlines tend to attract more eye fixations sooner than some other page elements. Understand that the first few words in a homepage headline are crucial in engaging the largest number of users. Also note that unusual initial words and words in all capital letters may influence how many eyes fixate on a particular headline. Users will give you a minimum of five chances to engage them with headlines. Excellent headline writing in this environment can make a difference. Add/view feedback on homepage reports Written by Laura Ruel and Steve Outing, project managers; research and tools by Colin Johnson, Greg Edwards, and Leslie Kues of Eyetools Inc. http://www.poynterextra.org/eyetracking2004/viewing.htm (9 of 9)9/2/2004 10:14:20 AM Eyetrack III - Headlines & Blurbs Findings in this report: •Blurbs encourage scrolling •Blurbs boost reading across the page •Blurbs read half the time •Clicks per user same with/without blurbs •People view left third of blurbs •Tips Add/view feedback on this report MORE HOMEPAGE RESULTS ARTICLES •5 homepage designs •Eye viewing patterns •Headlines & blurbs •Headline size •Font size •Navigation •Photos & images •Compact & extended pages OTHER EYETRACK III FINDINGS •Overview •Advertising •Article-page design •Multimedia content Using Headline & Blurbs on News Homepages The vast majority of news websites' homepages use a combination of headlines and accompanying blurbs to entice site visitors to click through to stories. This is standard practice at nearly all major news websites, and the majority of smaller ones. (Examples: WashingtonPost.com and Guardian.co.uk.) EYETRACK III FINDINGS This report is one of many from the Eyetrack III study of broadband-era news websites. 46 people were tested for one hour each in December 2003 by Eyetools Inc. in partnership with the Poynter Institute and the Estlow Center. During the test period, each test subject viewed mock news websites created for research purposes and real-world multimedia news features. Results were published in August 2004. Another approach is to stick to just headlines, without blurbs. While not as common, you can find this homepage approach on websites of small newspapers and TV stations, especially. (Examples: the CJOnline, Daily Triplicate, and KAALTV news sites; such sites often highlight a single story, then use headlines only for the rest of the page.) Just about every news website falls somewhere in between. In one part of the Eyetrack III research, we created two mock websites with homepages that were identical except for a single variable: one included headline/ blurb combos for all article links, the other included only headlines. Click on the thumbnails below to see the pages. Homepage No. 1 INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter Homepage No. 6 Here's what we observed. Finding: Blurbs encourage reading and scrolling on homepages. We found a statistically signficant difference in the time spent on homepage No. 1 (headlines only) and No. 6 (headlines + blurbs). Our test subjects spent 70 seconds on average viewing the blurbs homepage, and only 51 seconds with the http://www.poynterextra.org/eyetracking2004/blurbs.htm (1 of 8)9/2/2004 10:14:30 AM Eyetrack III - Headlines & Blurbs headlines-only page. In an effort to try to explain the difference in the time-on-page values, we discovered a marginally statistically significant difference in the percentage of headlines read vs. the percentage of headline + blurb combinations read. The homepage with blurbs exhibited more entries viewed, more reading, and more scrolling overall than the headlines-only page. Take a look at the heatmaps of homepages No. 1 and 6 below. (A heatmap is an aggregate image showing overall eye activity on a webpage. Red-orange areas indicate the most eye activity, blue-black the least.) Click the thumbnails to enlarge. Heatmap homepage No. 1 Heatmap homepage No. 6 As you can see from those overall page images, more people browsed down the entire page of headlines -- and saw more story links -- when there were blurbs. On the no-blurbs page, viewing concentrated mostly on the topmost list of headlines, without spreading as much through the rest of the page. (The purple X's in the images record where individual study participants clicked.) Overall, people scrolled more on the blurbs page, as can be seen on the heatmap of that page by the increased coverage of the elements on the right column. This was due at least in part to the fact that, on average, readers on the blurbs page chose to return to the homepage one more time during their allotted viewing period for this particular mock website (three times) than did those on the headlines-only page (two times), giving them more opportunity to scroll and peruse the blurb homepage's offerings. It's interesting to note that even if we adjust for the fact that more scrolling occurred on the blurbs page, there still exists a marginally statistically significant difference in the degree to which the headlines + blurbs combinations are read more. Homepage No. 1 - No blurbs http://www.poynterextra.org/eyetracking2004/blurbs.htm (2 of 8)9/2/2004 10:14:30 AM Homepage No. 6 - Blurbs Eyetrack III - Headlines & Blurbs © Eyetools, Inc. Less than 50 percent of the group looked to the headlines below the bottom of the first screen. © Eyetools, Inc. As much as 75 percent of this group sought more information on the homepage in the headlines below the bottom of the first screen. Also, it's worth noting that clicks were more dispersed throughout the page on homepage No. 1 (the headlines-only page). On homepage No. 6 (the headlines + blurbs page), about 50 percent of the clicks on the page were for the top two headlines. http://www.poynterextra.org/eyetracking2004/blurbs.htm (3 of 8)9/2/2004 10:14:30 AM Eyetrack III - Headlines & Blurbs The table above shows the number of clicks on headlines by all the participants while they looked at either homepage No. 1 or No. 6. (Half of our participants viewed No. 1, the other half viewed No. 6; no participant saw both pages.) You'll notice that the average number of clicks per participant is exactly the same for each homepage: 1.6. (This figure represents participants viewing the homepage more than once during the 5 minutes we allotted them to view this homepage and its accompanying article pages.) It's worth stepping back for a moment here to reflect on the significant power that visual design and content elements have to define the reader's experience. With the addition of a simple, seemingly innocuous blurb element, we shifted the flow of a reader's visual review of the page and redefined the content that readers selected. Finding: Blurbs boosted overall reading across the entire page. The chart below shows the percentage of people seeing each of the headlines on homepages No. 1 (headlines only) and No. 6 (headlines + blurbs). You'll notice that the two bar charts are somewhat similar, though the chart for the blurbs page shows generally higher percent-viewing figures across the board. A notable exception to this is headline Nos. 4 and 5, which received less viewing on the headline + blurbs page because the existence of the blurbs pushed these stories http://www.poynterextra.org/eyetracking2004/blurbs.htm (4 of 8)9/2/2004 10:14:30 AM Eyetrack III - Headlines & Blurbs further down the screen, causing them to be viewed less because of that placement. The images below (click the thumbnail to enlarge) show the order of the headlines cited in the chart above, for reference. Homepage No. 1 Homepage No. 6 Finding: When blurbs are available, people use them half the time as they make a selection. We found that 49 percent of participants who clicked a particular headline on the headline + blurbs homepage actually looked at the accompanying blurb. The other half never looked at the blurb, just the headline. Finding: Average number of clicks per person were identical for headlines-only and headlines + blurbs homepages. It's interesting that the average number of clicks per participant was the same for http://www.poynterextra.org/eyetracking2004/blurbs.htm (5 of 8)9/2/2004 10:14:30 AM Eyetrack III - Headlines & Blurbs homepage No. 1 (headlines only) and No. 6 (headlines and blurbs): 1.6. As noted above, this represents the total number of times that our participants clicked away from the homepage in the 5 minutes we allotted for them to view this particular mock website. A distinction between the two was that, on average, homepage No. 6 had one more return to the page than did homepage No. 1. That would seem to indicate, when considered in light of the same average number of clicks on each page, that the headlines-only page caused people to decide to stop the task of viewing this mock website during reading of an article -- whereas the headline + blurbs page motivated people to return to it one last time before signing off for the task. The time we gave participants to view this website was enough to browse multiple sections, skim several articles, and/or fully read one or two. Finding: People focus primarily on the left third of the text in blurbs. When blurbs are seen at all (as they were by about half the test participants who viewed our page with blurbs), viewing is primarily on the left third of the blurb. Below is a series of close-ups of article blurbs from heatmaps of homepage No. 6. The red-orange-yellow areas indicate the highest viewing, which is concentrated on the left side of the blurbs. http://www.poynterextra.org/eyetracking2004/blurbs.htm (6 of 8)9/2/2004 10:14:30 AM Eyetrack III - Headlines & Blurbs Tips ● ● ● Using blurbs with headlines rather than headlines-only seems to help disperse interest throughout a homepage (down the page). Recognize that a list of headlines-only high on the page might not get people to look as much on lower portions of the page. The use of blurbs does not appear to affect the number of clicks per headline -- it just redistributes the clicks. If you have some stories that you want to get people to more than others, you might want to use blurbs with those headlines and place those stories near the top of the homepage. If you're going to use blurbs, remember that the first few words may matter most. Our findings indicate that very few people go to the trouble of reading all of even short blurbs. Most people don't invest much time in deciding whether or not to click through to an article, so keep head/blurb combos succinct. Add/view feedback on this report Written by Steve Outing and Laura Ruel, project managers; research and tools by Colin Johnson, Greg Edwards, and Leslie Kues of Eyetools Inc. http://www.poynterextra.org/eyetracking2004/blurbs.htm (7 of 8)9/2/2004 10:14:30 AM Eyetrack III - Headlines & Blurbs http://www.poynterextra.org/eyetracking2004/blurbs.htm (8 of 8)9/2/2004 10:14:30 AM Eyetrack III - Headlines Size Findings in this report: •Smaller headlines with blurbs led to more scrolling •Smaller headlines make readers "work" harder •"Hot" words catch eyes of skimmers •Readers consume long, small-headline homepages in chunks •Large headlines make it "too easy" to read homepage •Tips Headline Size on News Homepages Most news website homepages have a set font size for headlines. (Only a small percentage of sites allow users to specify the type size they see.) In Eyetrack III, we tested how headline size affected viewing patterns. The findings below may help you in designing or redesigning your website for readability and comprehension of information. EYETRACK III FINDINGS This report is one of many from the Eyetrack III study of broadband-era news websites. 46 people were tested for one hour each in December 2003 by Eyetools Inc. in partnership with the Poynter Institute and the Estlow Center. During the test period, each test subject viewed mock news websites created for research purposes and real-world multimedia news features. Results were published in August 2004. Add/view feedback on this report MORE HOMEPAGE RESULTS ARTICLES •5 homepage designs •Eye viewing patterns •Headlines & blurbs •Headline size •Font size •Navigation •Photos & images •Compact & extended pages OTHER EYETRACK III FINDINGS •Overview •Advertising •Article-page design •Multimedia content Finding: Smaller headlines integrated with blurb text resulted in participants scrolling further down the page. For this piece of our research, we created two news homepages (and accompanying "inside" or article pages) that shared the same design but differed in one principal characteristic: headline size. In homepage No. 3 (below; click to enlarge), article headlines were smaller and on the same line as a short blurb. In homepage No. 8, headlines were larger and on a separate line from the blurb; the headlines also included a hyperlink underline to highlight them. One-half of our test participants viewed one of the pages; the other half viewed the other page. Homepage No. 3 Homepage No. 8 What we found was that aggregate viewing went further down the page on the version with smaller headline type -- that is, those users saw more of the page. Seventy percent of test subjects viewing that page scrolled down the page, vs. only 59 percent who scrolled down on the page with large-format headlines -- a statistically significant difference. This occurred despite there being a similar http://www.poynterextra.org/eyetracking2004/headlinesize.htm (1 of 4)9/2/2004 10:14:45 AM Eyetrack III - Headlines Size INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter number of clicks on the first 10 headlines of each page. You can see this on the heatmap images below. (A heatmap is an aggregate image showing overall eye activity on a webpage. Red-orange areas indicate the most eye activity, blueblack the least.) Small-headlines homepage Large-headlines homepage Participants who saw the larger-headlines page (homepage No. 8) spent more time with the navigation elements on the page. The researchers speculate that this could reflect that users on the larger-headlines page where less satisfied (perhaps because they skimmed more and missed content that small-headlines visitors found). It could be for this reason that large-headline visitors looked to the navigation for more information. (This is just an observation, and not a rigorously tested finding.) In this case, the navigation was in the right column. They also saw less of the page than did the group that was given the smaller-headlines page to review. Finding: Smaller headlines support users seeing and reading more text. Researchers noticed that on homepage No. 8 (larger underlined headlines), people treated the headlines as separate units. They tended to look at just the headlines and ignore the blurbs (which were on a separate line on this homepage). In contrast, the small headlines that blended in with the blurbs (on homepage No. 3) tended to attract more viewing and reading over the entire headline and blurb block of text. http://www.poynterextra.org/eyetracking2004/headlinesize.htm (2 of 4)9/2/2004 10:14:45 AM Eyetrack III - Headlines Size That is, the stand-alone blurbs (which appeared on a separate line below the headline link) were ignored more often; the small headlines integrated with blurbs tended to be viewed more as a package. The size and treatment of homepage headlines, then, had an unexpected impact on overall viewing of the page. In homepage No. 3 where the headline font was small and the headlines blended in with the blurbs, the headline/blurb combination was viewed more completely. Finding: "Hot" words can catch the eye of skimmers. An interesting finding comes from homepage No. 8, which had larger headlines on a separate line from the blurbs. As noted above, this page encouraged more skimming behavior. Toward the bottom of this (rather lengthy) page is this headline: "FCUK: Innocent Label or Thinly Disguised Profanity?" The provocative word "FCUK" (the controversial name of a clothing line) is something that obviously catches the eye. As seen on the heatmap image of homepage No. 8, that word caught a lot more eyes than anything else that far down the page. This observation suggests that a large percentage of people looking at this page were scanning, looking for something that would grab their attention. In this case, FCUK was both surprising and in all capital letters, resulting in a visual discontinuity with surrounding content. The same hot spot can be seen on homepage No. 3, which had smaller headlines integrated with blurbs. That page also shows more intense viewing throughout the page; the "FCUK" attention-grabber is more pronounced on the larger-headlines page where there's more scanning behavior. Observation: There seems to be a "dip" in reading the full length of titles at the bottom of the first screen (and subsequent screens) on small-headline pages. On homepage No. 3 -- the one with small headlines integrated into blurbs -- we noticed that there was less viewing on headlines that fell at the bottom of the first screen. The same thing happened with subsequent screenfuls as people moved down this fairly long page in chunks, each time exhibiting less viewing at the bottom of the screen. These intervals appear to correspond to the bottom of the screen had the test subjects moved down the page one full screen at a time. In other words, more of them clicked the scroll bar to move down the page a chunk at a time (or hit the "Page Down" key) than slowly scrolled down the page. Finding: Large headlines may make it easy to read a homepage (perhaps "too easy"). While large headlines make it easy for an audience to scan your homepage http://www.poynterextra.org/eyetracking2004/headlinesize.htm (3 of 4)9/2/2004 10:14:45 AM Eyetrack III - Headlines Size seeking something of interest, there's a downside to that: They may miss much of the other information you put on the page. Tips ● ● ● When headlines stand out alone by virtue of larger font size and/or by being on a separate line from a blurb, people tend to read the headline and ignore the blurb. It may make more sense, if you want readers to see blurbs on homepages, to have the headline and blurb run together. The testers on the homepage with larger headlines appeared to do more skimming or spot-checking than they did reading. Larger headlines seem to encourage readers to skip blurbs and easily scan down the page. So, the headline size you use may depend, in part, on what you want your homepage to do for readers. Do you have serious, thoughtful readers who you want to absorb all the information you put on your homepage? Or do you have readers who you expect to quickly scan your homepage? The decision you make on headline size may depend upon your audience and strategy. Be aware that certain words or phrases in headlines can draw lots of eyeballs -- which might be useful in drawing attention to other content elements, if you place them in proximity to predicted hot spots. Add/view feedback on this report Written by Steve Outing and Laura Ruel, project managers; research and tools by Colin Johnson, Greg Edwards, and Leslie Kues of Eyetools Inc. http://www.poynterextra.org/eyetracking2004/headlinesize.htm (4 of 4)9/2/2004 10:14:45 AM Eyetrack III - Font Size Findings in this report: •Smaller font size results in more careful viewing of the page •Disparity in font size seems to make a difference between scanning and reading behavior •Sometimes, content trumps format and presentation •Tips Add/view feedback on this report MORE HOMEPAGE RESULTS ARTICLES •5 homepage designs •Eye viewing patterns •Headlines & blurbs •Headline size •Font size •Navigation •Photos & images •Compact & extended pages OTHER EYETRACK III FINDINGS •Overview •Advertising •Article-page design •Multimedia content INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter Font Size on News Homepages Most news website homepages do not allow readers to adjust the font size of the page they are viewing. What you see is what you get. In part of the Eyetrack III testing, we designed two identical homepages and included overall font size as a variable. Half of our participants saw one homepage with larger type, the other half saw a homepage with smaller type. Homepage No. 4, below (click to enlarge), used smaller type. EYETRACK III FINDINGS This report is one of many from the Eyetrack III study of broadband-era news websites. 46 people were tested for one hour each in December 2003 by Eyetools Inc. in partnership with the Poynter Institute and the Estlow Center. During the test period, each test subject viewed mock news websites created for research purposes and real-world multimedia news features. Results were published in August 2004. Homepage No. 4 Homepage No. 9, below, used larger type. Homepage No. 9 This particular homepage design featured a mix of headlines with and without blurbs. Finding: Smaller font size results in a little more careful viewing of the page. Our other Eyetrack findings suggest that the overall size of type used on a homepage (both blurb text and headlines) has a marginally significant impact on viewing: Smaller type causes readers to read more and get drawn into actually reading the blurbs. On pages with larger type, there's more scanning going on and less concentrated viewing (less reading) of the words. http://www.poynterextra.org/eyetracking2004/fontsize.htm (1 of 4)9/2/2004 10:14:52 AM Eyetrack III - Font Size Beyond statistical testing, researchers inspected the data for behavioral clues to help us decipher what might have accounted for the slight differences between the two pages. We observed on homepage No. 9 (large type) that more people spot-checked the page and looked at just the headlines, and then selected headlines on that basis alone. In contrast, we observed on homepage No. 4 (small type) that more people viewed the blurbs before clicking on a story, often reading them. Our confirmation of this theory came from two additional inspections that we conducted. First, we speculated that people would read the blurbs more often on homepage No.4 (small type) than on No. 9 (larger type). We found that 88 percent of the group that looked at homepage No. 4 read blurbs at least once before clicking; 62 percent of the group that looked at homepage No. 9 read blurbs at least once. To test that this affect was real, we confirmed that the inverse might also be true -- that more people would read only a headline on No. 9 than on No. 4. We found that 30 percent of the group on No. 9 clicked to article pages after reading only the headline without ever reading a blurb; that figure was 6% of the group on No. 4. While this sample size is too small to claim that this tendency can be extrapolated to a general population, we found it interesting to uncover trends here that warrant additional study. The heatmaps below for homepages No. 4 (small type) and 9 (large type) exhibit the visual patterns we saw. (A heatmap is an aggregate image showing overall eye activity on a webpage. Red-orange areas indicate the most eye activity, blue-black the least.) Click the images to enlarge. Heatmap - homepage No. 4 Heatmap - homepage No. 9 Finding: Disparity in font size seems to make a difference between scanning and reading behavior. Where viewing of text (especially blurbs) falls off is where there's a wide disparity between font sizes. The larger the headlines are in relation to blurbs the less likely it is that blurbs will be viewed or read. When headlines and blurbs are the same size, and that size is small, then both are viewed or read more often. An interesting point -- brought out in our comparison of homepages No. 4 and 9 as well as elsewhere in our testing -- is that readers seem willing to make the effort to adjust their reading style to overcome whatever strain might be associated with use of smaller font sizes. With larger font size and especially larger headlines, people seem to be less likely to "work hard" in viewing content beyond the headline. Eyetrack researchers surmise that once a person is focused on granular information that is not easily skimmed (i.e., small type), it is easier to continue http://www.poynterextra.org/eyetracking2004/fontsize.htm (2 of 4)9/2/2004 10:14:52 AM Eyetrack III - Font Size in that mode to view whatever text might be close at hand (like a blurb). Finding: Sometimes, content trumps format and presentation. On homepage No. 9, the one with larger font size, the lower portion of the page was not read nearly as much as the lower portion of homepage No. 4, the smaller-font page. But, on No. 9 there was one story positioned low on the page that attracted a lot of interest: the headline "Craigslist: Now It's a Movie, Too." Now, Craigslist is a San Francisco online institution, and Eyetrack III testing took place in San Francisco. It seems fair to believe that Craigslist has high recognition for many residents of the city. Participants in this study looked at the blurb heavily on this story -- on both the small-font page and the large-font page. On the small-font homepage (No. 4), this heatmap detail shows high viewing rate for several article blurbs On the large-font homepage (No. 9), the heatmap detail shows low viewing for article blurbs, except for the Craig's List story (largest orange area). The way eyes found "Craig's List" is an example of how content can override the format of a page and its presentation. The people who mostly scanned the large-font page (No. 9) found this story of high interest to them, then zoomed in to look at the detail (the accompanying blurb). Tips http://www.poynterextra.org/eyetracking2004/fontsize.htm (3 of 4)9/2/2004 10:14:52 AM Eyetrack III - Font Size Here are some tips based on what we found in this part of the research: ● ● ● Is the intent with your website's homepage to encourage scanning or to encourage concentrated viewing? If you want your readers to fix their eyes on the homepage for longer, you might want to use smaller type in headlines and blurb copy. If you want to encourage scanning behavior, larger headlines may help. If your desire is to have website readers examine all or most of the type on your homepage, you may want to keep headline size and blurb size comparable. For pages using larger type and contrasting-size type, the words on a headline are important. With an audience that you know is scanning, the choice of words is critical. Add/view feedback on this report Written by Steve Outing and Laura Ruel, project managers; research and tools by Colin Johnson, Greg Edwards, and Leslie Kues of Eyetools Inc. http://www.poynterextra.org/eyetracking2004/fontsize.htm (4 of 4)9/2/2004 10:14:52 AM Eyetrack III - Navigation Observations in this report: •Homepage: Top nav captures most viewing •Article level: Left nav seen most •Not much time spent looking at navigation •Nav used most on compact homepages •Tips Add/view feedback on this report MORE HOMEPAGE RESULTS ARTICLES •5 homepage designs •Eye viewing patterns •Headlines & blurbs •Headline size •Font size •Navigation •Photos & images •Compact & extended pages OTHER EYETRACK III FINDINGS •Overview •Advertising •Article-page design •Multimedia content INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter Navigation Placement on News Homepages The majority of news websites carry navigation -- links to other major sections of the site -- in the left columns of their homepages. Second most common is the top navigation bar, tucked under the site's nameplate and above page content. Some even carry left AND top navigation, like WashingtonPost.com. Others use 3-way wrap-around navigation -- that is, navigation elements are repeated at top, left, and right -- like Newsday.com. EYETRACK III FINDINGS This report is one of many from the Eyetrack III study of broadband-era news websites. 46 people were tested for one hour each in December 2003 by Eyetools Inc. in partnership with the Poynter Institute and the Estlow Center. During the test period, each test subject viewed mock news websites created for research purposes and real-world multimedia news features. Results were published in August 2004. In part of our Eyetrack III testing of news website user behavior, we had test participants view five sets of homepage designs created by Nik Wilets, chief information architect for Morris Digital Works. (There were 10 homepages total, with each set of two being identical except for one variable that differed.) Among those sets, the designs used variable navigation placement. Looking across the eye movements tracked on all those pages, researchers were able to discern some patterns relating to navigation placement. Because the navigation elements were not tightly controlled variables in this particular part of our testing, we present below some observations about navigation placement on news websites (not definitive findings). Click on the thumbnails below to bring up images of the homepage designs. Left navigation: Homepage No. 1 Homepage No. 6 Homepage No. 5 Homepage No. 10 http://www.poynterextra.org/eyetracking2004/navigation.htm (1 of 5)9/2/2004 10:14:58 AM Eyetrack III - Navigation Top navigation: Homepage No. 2 Homepage No. 7 Homepage No. 4 Homepage No. 9 Right navigation: Homepage No. 3 Homepage No. 8 Observation: On homepages, top navigation captures more views than left or right navigation. A common finding throughout this eyetracking research is that items placed in close proximity to major headlines are viewed more because of "visual bleed." An item close to a prominent headline will receive more direct views (or "fixations") than an item that is separated by white space or a visual barrier like a rule (or line). We observed the same pattern with top navigation, which is closest of all the navigation placements to a homepage's top headlines and also received the most views of the three navigation placements. As you can see in the heatmap images below, a greater percentage of people looked at the top navigation element than looked at left or right navigation. (A heatmap is an aggregate image showing overall eye activity on a webpage. Redorange areas indicate the most eye activity, blue-black the least.) Heatmap - homepage No. 2 Heatmap homepage No. 7 Now, compare those pages to the one below, which has left navigation. Notice how the navigation receives considerably less viewing. Heatmap - homepage No. 6 http://www.poynterextra.org/eyetracking2004/navigation.htm (2 of 5)9/2/2004 10:14:58 AM Eyetrack III - Navigation There is some fear among news Web designers that top navigation -- because it's typically a thin bar running horizontally across the page that doesn't occupy as much screen real estate as a typical left navigation scheme -- won't be as visible. Perhaps these eyetracking observations will help to ease those fears. This chart shows the percentage of people who saw the navigation component of each of our homepages (that is, who looked directly at it, or in eyetracking terminology had a fixation). The top performers were the top-navigation elements on homepages No. 2 and 7, followed by the top navigation on homepage No. 4. (Take these numbers with a grain of salt; this particular part of the research did not use tightly controlled variables.) When it comes to left and right navigation on homepages, the percent-seeing figures are very close for both. The right navigation actually has a very slight advantage. Of course, this might have been the novelty factor at play; it's unusual to find right navigation on a website, so that might have affected viewing behavior. Observation: At the article level, left navigation is viewed most frequently. Our test subjects also viewed a variety of article pages, with navigation placement corresponding to the homepages they saw before clicking on the article link. What we found was that with articles, left navigation was viewed 18 percent more often by our testers than either top or right navigation placement. Our test participants viewed top and right navigation about equally. But the right navigation received more attention time (more eye fixations) on average than either left or top -- though it was seen less. People who looked at right navigation spent more time focusing on it than navigation placed elsewhere. (That could have been because of the novelty of seeing navigation on the right side of the page.) Top navigation performed better than left in terms of number of average fixations -- mirroring top-placement's better performance on homepages. Remember, page design varied along with navigation placement; this particular part of the Eyetrack III testing didn't use tightly controlled variables. So use these observations with caution; the figures cited above would not hold up to statistical analysis. http://www.poynterextra.org/eyetracking2004/navigation.htm (3 of 5)9/2/2004 10:14:58 AM Eyetrack III - Navigation Observation: People didn't spend much time looking at navigation on our homepages. On average, our participants spent 1.2 seconds looking at the homepage navigation element, out of a total average of 14.5 seconds spent viewing a homepage. This was fairly consistent between the various homepage designs, with the navigation element receiving only a second or two of most test subjects' eye time. The only exception was on a compact homepage, which is discussed in the item below this. Observation: Navigation gets used most on compact homepages. Navigation is critical to compact pages and less critical for extended pages with lots of content. Forty percent of participants clicked on the navigation bar when viewing homepage No. 2, which was a compact-design page with only five headline links and did not require scrolling to see the entire page. This was far more than any other page. The least-clicked navigation was on homepage No. 7 (which was the same design as No. 2, but with 17 headlines). On that homepage, only 7 percent of participants clicked on the navigation. When we look at all the clicks on a page, we find that on homepage No. 2 (compact), nearly 30 percent of the clicks were on the navigation. No other page came anywhere close to that; the average for all other pages was 8 percent of all clicks going to navigation elements. The least-clicked navigation was on homepage No. 7 -- the expanded version of the No. 2 page design. This suggests that homepages with lots of content and options for clicking on articles may see modest usage of navigation elements, but rather can expect users to click on article links as navigation inside the site. The majority of news websites in the real world -- at this writing -- have expanded homepages with lots of content. We might expect, then, that navigation on such pages isn't directing many people to inside sections; instead, homepage content is. Tips: Here are some tips based on what we observed on navigation in this part of the research: ● ● ● Do not take this report to indicate that top navigation placement is THE way to go for homepages. Rather, take it to mean that there's probably no reason based on these observations to avoid it. The advantage to top placement, of course, is that it can be compact, opening up a left column for editorial and/or advertising content. As for left and right navigation, it appears that right placement -- which is unusual among news websites -- is a viable option. The performance of right-nav placement was very similar to left. The navigation strategy you employ depends, in part, on how you view your page. Does your homepage provide a brief summary of headlines (in which case our observations suggest users rely on navigation more) or a comprehensive extended table of contents to the materials inside (in which case our observations suggest users rely on navigation less)? For a compact homepage -- especially one that fits entirely on a single screen http://www.poynterextra.org/eyetracking2004/navigation.htm (4 of 5)9/2/2004 10:14:58 AM Eyetrack III - Navigation -- the navigation is more heavily used by readers. If you have an expanded homepage, be aware that the navigation may not be viewed or used much, since readers are clicking on links to browse through the site. So, if you want users to go to your business section, for example, be sure there are business article links and/or blurbs on your homepage. Add/view feedback on this report Written by Steve Outing and Laura Ruel, project managers; research and tools by Colin Johnson, Greg Edwards, and Leslie Kues of Eyetools Inc. http://www.poynterextra.org/eyetracking2004/navigation.htm (5 of 5)9/2/2004 10:14:58 AM Eyetrack III - Compact vs. Extended Homepages Findings in this report: •On extended pages, content up high not affected •With extended pages, top portion is consumed first •Headlines "below the fold" deliver additional content without penalty •Tips Add/view feedback on this report MORE HOMEPAGE RESULTS ARTICLES •5 homepage designs •Eye viewing patterns •Headlines & blurbs •Headline size •Font size •Navigation •Photos & images •Compact & extended pages OTHER EYETRACK III FINDINGS •Overview •Advertising •Article-page design •Multimedia content Comparing Compact & Extended News Homepages Look around the Web these days and you'll see many news websites that have homepages containing lots and lots of links and content (we refer to them in this article as "extended"). Sites like WashingtonPost.com and LATimes.com, for instance, typically have well over a hundred links, and all sorts of photos, charts, and ads. EYETRACK III FINDINGS This report is one of many from the Eyetrack III study of broadband-era news websites. 46 people were tested for one hour each in December 2003 by Eyetools Inc. in partnership with the Poynter Institute and the Estlow Center. During the test period, each test subject viewed mock news websites created for research purposes and real-world multimedia news features. Results were published in August 2004. Less common are news websites with fewer links that require little or no scrolling to see the full content of the page (we refer to them as "compact") -for example, Observer-Reporter.com and CJOnline.com. It probably makes sense to position yourself on the compact-extended continuum based on your audience's needs. In Eyetrack III we tested two similarly designed homepages: one compact (homepage No. 2, below) that fits on a screen with no scrolling required, and the other (homepage No. 7) an extended page. Click the thumbnails below for enlarged views. Homepage No. 2 INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter Homepage No. 7 Here's what we observed. Finding: Adding more content to make a longer, extended page did not affect viewing of editorial page elements "above the fold." With the extended homepage that had content going beyond the first screen, we found no difference in viewing behavior on editorial content on the top part of the page. What we saw on the top portion of the extended page was similar http://www.poynterextra.org/eyetracking2004/compactextended.htm (1 of 5)9/2/2004 10:15:04 AM Eyetrack III - Compact vs. Extended Homepages to what we saw with the compact page (all of which was visible without scrolling). One major difference was in use of the horizontal navigation bar on these two pages. Viewers of the compact page used 30 percent of their clicks on the navigation, vs. only 3 percent of clicks for viewers of the extended page. (Half of our test group saw the compact homepage; the other half saw the extended page.) That makes sense; with fewer options on the compact page, you'd expect the navigation to play a more important role in guiding people to inside content. In terms of people viewing the navigation at least once (even if for a short period), it was close for both: 80 percent of the compact-page viewers saw the navigation; 87 percent of the extended-page viewers saw it. (The difference between those figures is not statistically significant, however.) On the extended page, the navigation bar was the second-most viewed page component, following the topmost headline. On the compact page, four items were seen by more people than the navigation bar: the right-column ads, the page flag/ masthead, and headline/blurb combos No. 1 and 2. Another major difference in how the compact page differed from the extended page was in the amount of viewing participants gave the advertising on the right. In the same way that the reduced options appeared to drive users to the navigation bar, so too did the reduced options drive participants to look at the advertising on the right. Since so many news homepages extend well below the first screen (requiring scrolling), it's encouraging to see that on the extended test homepage, nearly 60 percent of participants did view at least some headlines "below the fold." You can see some of the behaviors described above by looking at the heatmaps of homepages No. 2 (compact) and No. 7 (extended) below. (A heatmap is an aggregate image showing overall eye activity on a webpage. Red-orange areas indicate the most eye activity, blue-black the least.) Click images to enlarge. Heatmap - homepage No. 2 Heatmap - homepage No. 7 Finding: When more choices are available on a homepage, people still tend to consume the top portion of the page first. When test participants looked at the extended page (No. 7), they still tended to review the top portion of the homepage first before moving on to the bottom section. In this particular design, a weather feature served to split the homepage into distinct sections, but when viewing the page initially, the bottom section was visible -- so participants would have been able to realize that there was more to the page. Most people moved their gaze and attention around the top part of the page first, then later moved down the page. You can see this by looking at the average viewing sequence graphic for the extended page (No. 7) below. (The graphic shows the order in which parts of the page -- what the researchers term "zones of interest" -- are viewed on average by all the people who viewed this particular page.) http://www.poynterextra.org/eyetracking2004/compactextended.htm (2 of 5)9/2/2004 10:15:04 AM Eyetrack III - Compact vs. Extended Homepages Below is the average viewing sequence graphic for the compact homepage, No. 2. You can see how people tend to bounce around the page. Finding: Headlines placed "below the fold" deliver additional content without penalty to performance of the upper part of the page. In looking at statistics for viewing headlines on the top of both pages, we see http://www.poynterextra.org/eyetracking2004/compactextended.htm (3 of 5)9/2/2004 10:15:04 AM Eyetrack III - Compact vs. Extended Homepages that the numbers are fairly consistent for both groups (compact-page viewers and extended-page viewers). If you look at statistics for headlines further down the extended page, they typically fall in the 30-45 percent range. The extended-page viewers focused on a larger number of headlines than the group viewing the compact page. So, even though much of the viewing of the extended page was up top, plenty of people also explored down the page (about 87 percent). The chart below shows the percentage of people who looked at or saw the various components of the compact and extended homepages. Blank cells in the spreadsheet are there because the compact homepage did not contain some elements that the extended page contained. Tips Here are some tips based on what we found in this part of the research: ● On a compact page, navigation is used more. On an extended homepage, the navigation is likely to be used less, so be sure there are content links http://www.poynterextra.org/eyetracking2004/compactextended.htm (4 of 5)9/2/2004 10:15:04 AM Eyetrack III - Compact vs. Extended Homepages ● ● ● on the homepage to areas of the site you want people to visit. By removing content from the homepage and making it compact, you can change user behavior relative to how the user navigates the site as a whole -- from one of grazing content in the homepage text to find a link to click, to one of using the navigation to dive into the site for more content. By limiting the amount of content on a homepage, you can drive viewing to advertising on the page (presuming, of course, that the ad is positioned effectively), the researchers suggest. The research indicates that people view the top part of a news site's homepage first (predictably). Participants did read headlines "below the fold," though, which is good news for most websites, since very few restrict the length of their homepages to the first visible screen. Add/view feedback on this report Written by Steve Outing and Laura Ruel, project managers; research and tools by Colin Johnson, Greg Edwards, and Leslie Kues of Eyetools Inc. http://www.poynterextra.org/eyetracking2004/compactextended.htm (5 of 5)9/2/2004 10:15:04 AM Eyetrack III - Online Images Observations in this report: •Photo size impacts number of users seeing photos and the time they spend with them •Faces in photos draw users' eyes •Multiple faces in photos attract more viewers •Users often click on photos •Tips Add/view feedback on this report MORE HOMEPAGE RESULTS ARTICLES •5 homepage designs •Eye viewing patterns •Headlines & blurbs •Headline size •Font size •Navigation •Photos & images •Compact & extended pages OTHER EYETRACK III FINDINGS •Overview •Advertising •Article-page design •Multimedia content Online Images: Faces, Size Attract Take a look at most news site homepages and you'll generally find that photos consume only a small portion of the screen. Some sites, such as South Florida's Sun-Sentinel.com or Las Vegas' ReviewJournal.com, provide users with a "click to enlarge" option that results in a pop-up window with a larger photo size than the homepage "postage-stamp" provides. Other sites, such as WashingtonPost. com and MSNBC.com, seem to give photos a bit more homepage real estate. EYETRACK III FINDINGS This report is one of many from the Eyetrack III study of broadband-era news websites. 46 people were tested for one hour each in December 2003 by Eyetools Inc. in partnership with the Poynter Institute and the Estlow Center. During the test period, each test subject viewed mock news websites created for research purposes and real-world multimedia news features. Results were published in August 2004. But when it comes to images and online-news sites -- either on homepages or interior pages -- is bigger really better? Initial Eyetrack III observations suggest that the answer is yes. Image size does seem to affect the number of photos noticed by online news consumers. It also affects the time they spend looking at pictures. Our observations also indicate that people's faces draw users' eyes the most. For the purposes of this study we assumed that our images varied only in size and in the number of people depicted in them. Images were not controlled for visual quality or consistency. Future studies ought to consider how different qualities of photographs translate into different viewing behaviors. Within this scope of study, we observed: ● INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter ● ● ● On both homepages and article-level pages, photo size affects the number of users seeing photos and the time they spend with them. Faces in photos draw users' eyes. Multiple faces in photos attract more viewers. A minority of users routinely click on photographs. Observation: Larger photo size increases the percentage of users seeing photos and the time they spend looking at them. For this study, we tested three common sizes of photos: small (about 80x80 http://www.poynterextra.org/eyetracking2004/photos.htm (1 of 12)9/2/2004 10:15:15 AM Eyetrack III - Online Images pixels), medium (210x230 pixels), and large (365x240 pixels). Thumbnails of the five images we used appear below. Small photo Medium photo Medium photo http://www.poynterextra.org/eyetracking2004/photos.htm (2 of 12)9/2/2004 10:15:15 AM Eyetrack III - Online Images Large photo Large photo We then placed these photos on the mock webpages we created for this study and observed eye fixations and movement. Take a look at these sample pages (click to enlarge): Small photo size: Medium photo size: http://www.poynterextra.org/eyetracking2004/photos.htm (3 of 12)9/2/2004 10:15:15 AM Homepage no. 3 Homepage no. 1 Eyetrack III - Online Images Large photo size: Homepage no. 4 When comparing homepages with small photos to homepages with average-size photos, the percentage of users who noticed the images (that is, whose eyes fixated on the image for at least a fraction of a second) almost doubled. Smaller photos just don't attract viewers -- they are often ignored entirely. However, there does seem to be a point of diminishing returns as photo size increases. Although large photos do get the greatest percentage of viewers and the greatest duration of viewing, the difference in the percentage of viewers who notice the medium photos vs. the large photos is not all that substantial. At almost double the page real estate of the average-size photo, the large photos attracted only slightly more people to look at them than the average-size photo. However, differences in the length of time spent with larger photos remains substantial. The chart below provides specifics. Photo viewing patterns on homepages Percent of participants seeing photo Percent increase over next-smaller size Amount of time spent viewing photo (in seconds) Percent increase over next-smaller size Small photo 43 N.A. 0.6 Average photo 80 86% 1.5 Large photo 88 10% 2.3 N.A. 150% 53% Both of the large images used in this study had headlines superimposed on the image. It is important to note that this fact may contribute to -- or account for -the increase in viewing duration. On article-level pages, the same observations seem to hold true. Larger images (approximately 500x300 pixels) receive almost twice as many fixations as medium-sized pictures (approximately 300x200 pixels). Article-level page with large image: http://www.poynterextra.org/eyetracking2004/photos.htm (4 of 12)9/2/2004 10:15:15 AM Eyetrack III - Online Images Article-level page with medium image: Observation: Faces in photos draw users' eyes. The homepages used in Eyetrack III all contain photos with images of people. We learned that with every image -- whether a real photo or a graphic image -users zeroed in on faces. Here are a few examples. The homepage below contains a screen shot of an image from a video game. On this heatmap you'll see that those who looked at this image zeroed in on the face. That's where about 60-70 percent of the users' eyes focused. (A heatmap is an aggregate image showing overall eye activity on a webpage. Red-orange areas indicate the most eye activity, blue-black the least.) http://www.poynterextra.org/eyetracking2004/photos.htm (5 of 12)9/2/2004 10:15:15 AM Eyetrack III - Online Images Now look at this homepage, which contains a main photograph of two people. Again, the "hot spots" are the faces, with more than 60 percent of the group focused there. http://www.poynterextra.org/eyetracking2004/photos.htm (6 of 12)9/2/2004 10:15:15 AM Eyetrack III - Online Images Finally, look at this homepage image, taken from a distance. The faces of the individuals are a small percentage of the image, and still, users found their way to at least one of them. The third example introduces an additional variable -- the headline of the lead story is placed inside the photo and above the faces. Taking into account our observation that users eyes tend to fixate on main headlines (see our article on eye viewing patterns), we cannot ignore that the location of the headline type on the photo could have contributed to eye fixations in that area of the page. We also must remember that our test images were not controlled for visual quality or consistency. In the case of this homepage image (which is grainy and amateurish in its composition), these qualities could have had a negative effect on user interest. A final observation in this area concerns the effect of an image's clarity and readability. It seems logical that on a busy homepage with a lot of clutter, or "visual noise," that a tight image with a strong focal point would draw gazes. Our data seem to support this observation. Compare the two pages below again. Both images are of two people and both have headline type placed on top of them. However, the headline gets fewer gazes on the first image, where the http://www.poynterextra.org/eyetracking2004/photos.htm (7 of 12)9/2/2004 10:15:15 AM Eyetrack III - Online Images faces are clearer and larger, than in the second example, where the faces are not as close up. To state this observation another way: The clear, readable image of the two men may have such stronger drawing power than the distant image of the man and woman. http://www.poynterextra.org/eyetracking2004/photos.htm (8 of 12)9/2/2004 10:15:15 AM Eyetrack III - Online Images Observation: Multiple faces in photos attract more viewers. Our data in this study suggest that viewers spend more time with images that contain more than one face. Comparing the images on the two homepages below illustrates this point best. First, take a look at the image on Homepage No. 7, which has only one face clearly visible. http://www.poynterextra.org/eyetracking2004/photos.htm (9 of 12)9/2/2004 10:15:15 AM Eyetrack III - Online Images Then, take a look at Homepage No. 6, which has the same size image, but with two faces clearly visible. As the chart below indicates, there is a substantial difference in the percentage of users who view the image on homepages No. 6 and 7, and the amount of http://www.poynterextra.org/eyetracking2004/photos.htm (10 of 12)9/2/2004 10:15:15 AM Eyetrack III - Online Images time they spend with the image (more than three times greater). Image interest and time spent gazing Percent of participants seeing image Amount of time spent with image (in seconds) Homepage No. 7 (Image has one face visible.) 67 0.7 Homepage No. 6 (Image has two faces visible.) 94 2.5 Observation: People routinely click on photos. Most news websites publish photographs that are not links -- that is, if you clicked on them, nothing would happen. Some websites do make homepage images into links -- mostly to accompanying articles, but sometimes to enlarged views of smaller images placed on the homepage. Some test subjects (a minority, but a substantial percentage) clicked on photos, even though there was no indication that clicking might produce a result. The heatmap image at right shows clicks on an image from three people out of 25 who viewed this page. (The purple X's are clicks; the numbers identify individual test subjects). You'll see this behavior throughout the heatmaps for this study. The researchers have observed this behavior on many studies prior to Eyetrack III, so this isn't new. But judging by how seldom real-world news sites make photos links, perhaps it's not widely recognized by the industry. Tips ● ● ● ● Images that are least 210x230 pixels seem to keep viewers most engaged. Remember that on both homepages and inside pages, the larger the image, the more it will be noticed and the longer a viewer will engage with it. Understand that online-news users' eyes move toward faces when viewing photos of people. When choosing images with people for homepages, tighter images where faces are easy to read may work best to draw reader attention. When choosing photos of people for homepages, remember that viewers tend to look at and spend more time with images that have more than one face clearly visible. Since it's common for some users to click on photos, perhaps it's worth considering making homepage images links. What to link to? Consistency throughout a site is most important, but a photo click could always bring up a larger version of the image; or from the homepage it could always lead to the full article associated with the photo. http://www.poynterextra.org/eyetracking2004/photos.htm (11 of 12)9/2/2004 10:15:15 AM Eyetrack III - Online Images Add/view feedback on this report Written by Laura Ruel and Steve Outing, project managers; research and tools by Colin Johnson, Greg Edwards, and Leslie Kues of Eyetools Inc. http://www.poynterextra.org/eyetracking2004/photos.htm (12 of 12)9/2/2004 10:15:15 AM Eyetrack III - Multimedia vs. Text Comprehension Findings in this report: •Names, places recalled better with text •Process, procedures recalled best with multimedia •Factual info recalled best with text •No gender difference in recall •No statistically significant difference in recall between text and multimedia •Tips Add/view feedback on multimedia reports MORE MULTIMEDIA RESULTS ARTICLES •Recall & comprehension •Multimedia observations MORE EYETRACK III RESULTS •Overview Homepages: •5 homepage designs •Eye viewing patterns •Headlines & blurbs •Headline size •Font size •Navigation •Photos & images •Compact & extended pages Other findings: •Advertising •Article-page design Recall of Information Presented in Text vs. Multimedia Format With many news organizations investing time, money, energy, and human resources into online multimedia presentations, the Eyetrack III team was curious: Does the presentation of editorial content in multimedia format help readers understand and remember more story information? EYETRACK III FINDINGS This report is one of many from the Eyetrack III study of broadband-era news websites. 46 people were tested for one hour each in December 2003 by Eyetools Inc. in partnership with the Poynter Institute and the Estlow Center. During the test period, each test subject viewed mock news websites created for research purposes and real-world multimedia news features. Results were published in August 2004. To put this concept to the test, we utilized two distinct story presentation styles. With the help of NYTimes.com, we edited text versions of two news stories to 3-5 minute reads. We then edited existing multimedia presentations of these stories to a 3-5 minute experience. Here are links to the presentations we used: "Dangerous Business" story: ● ● Text Multimedia "Al Hirschfeld" story: ● ● Text Multimedia Half of our test participants (approximately 25 people) experienced one of the stories in text and the other in multimedia. The other half of our participants experienced the opposite formats. (They all saw a control article beforehand, as well.) After they read or viewed a story, we gave the same recall quiz to both groups. Here are links to copies of the quizzes: ● ● Dangerous Business recall quiz Al Hirschfeld recall quiz http://www.poynterextra.org/eyetracking2004/multimediarecall.htm (1 of 10)9/2/2004 10:15:28 AM Eyetrack III - Multimedia vs. Text Comprehension INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter Special care was taken to ensure that participants had not viewed these stories before the day of the testing. We also specifically asked participants not to guess at answers, but rather to answer "I don't know" if they were unsure. In addition to recording their answers, we were able to observe eye movements and fixations for participants, which added a new dimension to this comprehension/recall study. What we found: ● ● ● ● Overall, we found a slight, marginally significant difference in how test subjects correctly recalled story information that was presented in text vs. using multimedia. When asked to recall information about names and places, participants who received information in text were more likely to answer questions correctly. However, information about a process or procedure that was unfamiliar to them was more correctly recalled when participants received it in a multimedia graphic format. Users who received information in text form seemed to have better recall of specific factual information. There was no significant difference between men and women when it came to recall of information presented in text or multimedia format. Observation: There was a slight difference in recall of story information between participants who received a text version vs. participants who received a multimedia version. Text resulted in better recall overall. We observed that participants were, on average (for both stories in multimedia and text), 5 percent more likely to correctly answer recall questions about story information if they read a text version of the story. On average, the group that received the story information in text answered about 68 percent of the comprehension questions correctly, versus 62 percent correct answers for the participants who received the information in multimedia. The result has marginal statistical significance. While the difference between these groups is intriguing, further research is warranted to confirm this result. (To view statistical testing results, click here.) While considering these observations, it is beneficial to look at the eyetracking data that accompanies it. Tracking the behavior of participants' eyes at the time when the information in the recall test was presented provides interesting insight and perspective. For example, in the "Dangerous Business" recall quiz, participants were asked, "Who is Michelle Sankowsky?" Sankowsky is a nurse who works for Tyler Pipe, the company that is profiled in the story. Twenty percent of the participants who saw the multimedia version answered this question correctly, while 62.5 percent of the participants who read the text version responded correctly. In this case, the observation holds up to statistical rigor, and the difference is significant. (To view statistical testing results, click here.) In the text version, the answer appeared in this paragraph: http://www.poynterextra.org/eyetracking2004/multimediarecall.htm (2 of 10)9/2/2004 10:15:28 AM Eyetrack III - Multimedia vs. Text Comprehension Applying the heatmap key to this paragraph reveals that between 70 and 80 percent of those who viewed the story read Sankowsky's name and occupation. (A heatmap is an aggregate image showing overall eye activity on a webpage. Red-orange areas indicate the most eye activity, blue-black the least.) Now, let's compare that to the tracking data from the multimedia. At the time Sankowsky was identified, viewers were shown a photo of her face while they heard audio of her talking.. Her identification is in the text below the photo: Applying the heatmap key here reveals that 70-80 percent of participants looked at Sankowsky's face, and only about 40 percent read the part of the caption that contained Sankowsky's name and occupation. This is an area where further study is required before any solid conclusions can be drawn, but we do see evidence here to suggest that multiple, potentially conflicting streams of information (graphic, audio, and text) might impede some types of recall rather than help it. Observation: When asked to recall information about names and places, participants who received information in text were more likely to answer questions correctly. Our eyetracking data indicate that participants re-read story information that could be perceived as shocking or unusual. Using the "Dangerous Business" story as an example, we observed that correct http://www.poynterextra.org/eyetracking2004/multimediarecall.htm (3 of 10)9/2/2004 10:15:28 AM Eyetrack III - Multimedia vs. Text Comprehension recall of dramatic or shocking information is high. For example, part of this story is a description of worker Marcos Lopez's on-the-job injury. Pictured below is the tracking heatmap data for that part of the story. As the heatmap indicates, 60-70 percent of participants read the entire three paragraphs, with portions read by 80 percent and above. Moreover, our tracking shows that people read (and re-read) the paragraph describing what happened right after the injury (Lopez was not taken to the hospital or even to have an X-ray). We noticed that correct recall of this information was higher for those who received the information in text. Ninety-six percent who received the information in text got the answer right. Seventy percent who received the information in multimedia answered the question correctly. The results hold up to statistical testing, demonstrating that this difference is significant. (To view statistical testing results, click here.) The multimedia piece presented most of this information in audio and/or text. Here is a heatmap for the multimedia frame where Lopez's injury is described: http://www.poynterextra.org/eyetracking2004/multimediarecall.htm (4 of 10)9/2/2004 10:15:28 AM Eyetrack III - Multimedia vs. Text Comprehension While audio of Lopez talking about the injury plays, the image of him is displayed. The text below the photo is a succinct description of what happened: "Mr. Lopez, 45, slipped and fell while working. He suffered a severe compression fracture to his spine." Again, notice that the text below the photo is read by only about 60-70 percent of those who encounter this presentation. Just about everyone looks at Lopez's face, though. The audio does not appear to compensate, in terms of information recall, for the fact that the participants did not read the caption. Observation: New, unfamiliar information about processes or procedures was more correctly recalled when participants received it in a multimedia graphic format. When asked to recall information about a process or procedure or to define vocabulary related to a new process that few participants were likely to have ever heard of before, participants who received information in multimedia graphic formats appeared to learn more effectively and were more likely to answer questions correctly. For example, part of the "Dangerous Business" story helped readers understand how iron pipes are made. We found that most comprehension questions about this process where more likely to be answered correctly by those who received this information in an animated graphic than by those who received it in text. When participants carefully reviewed a moving illustration of this process, they seemed better able to recall it. Here is how two of the questions and answers broke down. Recall test question Text readers who answered question correctly Multimedia viewers who answered question correctly What is a cupola? Answer: A furnace that melts tons of scrap metal. 8 percent 65 percent Statistical testing demonstrates that this difference is significant. (To view statistical testing results, click here.) The data explain the process that people used to learn this new term. http://www.poynterextra.org/eyetracking2004/multimediarecall.htm (5 of 10)9/2/2004 10:15:28 AM Eyetrack III - Multimedia vs. Text Comprehension When the cupola first appeared on the screen, participants read about the term in the paragraph that accompanied the graphic; they also viewed the image with nearly universal consistency. When it came time to answer the question, these participants had successfully learned the term. The responses about how pipes are formed did not reflect as dramatic a difference. Recall test question Text readers who answered question correctly. Multimedia viewers who answered question correctly. How is molten iron formed into pipes? Answer: It is poured into spinning cylinders. 71 percent 80 percent The statistical data suggests that the mode of delivery -- text or multimedia -did not have a significant impact on whether our participants answered this recall question correctly. (To view statistical testing results, click here.) Here's the heatmap for the portion of the multimedia presentation that presented that information: http://www.poynterextra.org/eyetracking2004/multimediarecall.htm (6 of 10)9/2/2004 10:15:28 AM Eyetrack III - Multimedia vs. Text Comprehension It is important to note that, here again, there are two modes of communication at work -- text and moving illustrations. Further study needs to be done to see if this behavior changes when audio is added to the mix. Observation: Users who received information in text form seemed to have better recall of specific, factual information. Most of the questions on our recall tests asked participants to remember specific factual information, rather than concepts. The majority of these types of questions were answered correctly by participants who received the information in text form. (To view statistical testing results, click here.) Here are some examples: Recall test question Text readers who answered question correctly Multimedia viewers who answered question correctly Which statement most accurately depicts what happened to Lopez after he received an X-ray? Answer: The results of the X-ray were not divulged to him. 96 percent 70 percent What is the average air temperature inside the Tyler pipe foundry? Answer: 130 degrees. 88 percent 60 percent http://www.poynterextra.org/eyetracking2004/multimediarecall.htm (7 of 10)9/2/2004 10:15:28 AM Eyetrack III - Multimedia vs. Text Comprehension Which group of individuals most considered it an honor to be drawn by Hirschfeld? Answer: Stage performers. 95 percent 38 percent Statistical testing supports that the differences in these responses are significant, and show that text was a more effective delivery system in terms of users' recall. However, as noted in the section above, new, unfamiliar information (defining a "cupola") was better recalled when participants received it in a multimedia form. One other question also had a statistically significant advantage for those who received the information in multimedia form: Recall test question Text readers who answered question correctly. Multimedia viewers who answered question correctly. What hours did Hirschfeld usually work? Answer: 10 a.m. to 5 p.m. 60 percent 92 percent Statistically, this result is significant. The presentation of the information in multimedia form had a positive effect on the accuracy of the participants' recall. To understand this performance disparity, we took a closer look at how the information was presented in both versions and found some interesting differences. When we looked to the article, we saw that this information is available in one place -- in a sentence at the end of a paragraph. By contrast, the video segment entitled "Video: Work Habits" has two distinct differences: 1. It has moving images of Hirschfeld at his drawing board accompanied by a personal narration from his manager about how he spends his time during a typical day. 2. It dedicates about 75 percent of this one-minute clip to the topics of how and when he works. The red line drawn in the progress bar section of this clip (pictured below) indicates the segment during which this discussion plays. http://www.poynterextra.org/eyetracking2004/multimediarecall.htm (8 of 10)9/2/2004 10:15:28 AM Eyetrack III - Multimedia vs. Text Comprehension It is difficult to know whether it was the length of time dedicated to his work habits in the video or the mode of delivery (audio narration and images) that contributed to this significant recall advantage for multimedia. Again, this is an area where further study is required before any solid conclusions can be drawn. However, we do see evidence here to suggest that a 45-second audio narration with images could encourage recall more than a sentence of text. Observation: We found no significant difference between men and women when it came to recall of information when presented in text vs. multimedia. Contrary to our hypothesis that men might perform better with multimedia content (based on many younger males' predilection for computer games), when we analyzed our data for gender, we saw no significant difference in men's and women's abilities to correctly recall story information presented in text vs. multimedia. Gender Correct answers when story was received in multimedia Correct answers when story was received in text Male 62 percent 68 percent Female 63 percent 68 percent Although the multimedia world of video games seems to be male dominated, we learned that women and men absorb information in similar ways when presented with news online. In future studies it would be worthwhile to look at other demographic information such as race and economic status. We also should consider user satisfaction of the experience and see how that crosses demographic lines. (To view statistical testing results, click here.) http://www.poynterextra.org/eyetracking2004/multimediarecall.htm (9 of 10)9/2/2004 10:15:28 AM Eyetrack III - Multimedia vs. Text Comprehension Tips ● ● ● ● With only a marginally significant percentage of improved recall from text, it is difficult to offer a firm suggestion about the benefits or drawbacks of multimedia. However, it is important to be judicious in decisions about when to invest time and energy into multimedia projects. It also is important to remember that information recall probably is not your only goal. User satisfaction and overall understanding also are important to any good piece of journalism, and these aspects were not tested in this study. Because facts, names, and places seem to be recalled best when presented in text, designers may want to factor this into their choices about what projects are best suited for multimedia. Animated graphics seem to be an effective way to teach readers about processes and procedures or to acquaint them with new terms and concepts. Remember that clear, concise writing in the accompanying text is essential, because it is heavily read. The best journalists working in multimedia environments know how to make good choices about the presentation of story information. As demonstrated in this research, some information is best conveyed by the use of good, descriptive writing. Other information is better explained graphically. Add/view feedback on multimedia reports Written by Laura Ruel and Steve Outing, project managers; research and tools by Colin Johnson, Greg Edwards, and Leslie Kues of Eyetools Inc. http://www.poynterextra.org/eyetracking2004/multimediarecall.htm (10 of 10)9/2/2004 10:15:28 AM Eyetrack III - Multimedia Features Multimedia features reviewed in this report: •Enrique's Journey •Camp Heartland •The Big Picture •Obesity in America •Smart Growth •50 Years of Corvettes •Saddam's Sons Add/view feedback on multimedia reports MORE MULTIMEDIA RESULTS ARTICLES •Recall & comprehension •Multimedia observations MORE EYETRACK III RESULTS •Overview Homepages: •5 homepage designs •Eye viewing patterns •Headlines & blurbs •Headline size •Font size •Navigation •Photos & images •Compact & extended pages Other findings: •Advertising •Article-page design Observations on Multimedia Features As part of the Eyetrack III research, we had test participants view a variety of multimedia editorial features. Each of the 51 people we tested was given the same list of several features and told to look at any they wanted. Then we tracked their eye movements during the 10 minutes they had for this task. EYETRACK III FINDINGS This report is one of many from the Eyetrack III study of broadband-era news websites. 46 people were tested for one hour each in December 2003 by Eyetools Inc. in partnership with the Poynter Institute and the Estlow Center. During the test period, each test subject viewed mock news websites created for research purposes and real-world multimedia news features. Results were published in August 2004. We didn't control the subject matter; we didn't control any variables; we didn't have a hypothesis about what we might find. Rather, this portion of the test was strictly observational. We wanted to see if we could spot any patterns that would lead us to better understand how people interact with multimedia editorial content; to spot some multimedia best practices that seemed to engage people; and to give us guidance for future, in-depth eyetracking research on how online users interact with different forms of multimedia content. The observations we've made in this article are qualitative, as if we had gathered a mini-focus group of people who are not regular readers of a website. (If we had tested regular readers of a website, they might have responded differently.) The features we selected for this test varied in format and scope. Some were award winners, some were not. They represent a typical cross-section of multimedia features found on news websites. Not every participant viewed every feature tested, so the sample size for each one is modest. Here's what we observed. Multimedia Story 1: Enrique's Journey - LATimes.com "Enrique's Journey" is a Pulitzer Prizing-winning package from the Los Angeles http://www.poynterextra.org/eyetracking2004/multimediafreeforall.htm (1 of 15)9/2/2004 10:15:40 AM Eyetrack III - Multimedia Features INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter Times, published in the print edition, and also published online with a significant number of multimedia enhancements. Our Eyetrack III testing of this project suggests that few people saw the multimedia elements in this package, most likely because of the way it was designed. Nine of our 51 test subjects viewed this multimedia package as part of our testing. Our first clue that they didn't become engaged is the heatmap of the main page, below, which shows that the page did not guide visitors' eyes in any coherent fashion to key elements of the page. (A heatmap is an aggregate image showing overall eye activity on a webpage. Red-orange areas indicate the most eye activity, blue-black the least.) Four of the nine visited the main Enrique page then backed out without clicking on anything. http://www.poynterextra.org/eyetracking2004/multimediafreeforall.htm (2 of 15)9/2/2004 10:15:40 AM Eyetrack III - Multimedia Features Of the nine people who spent time with this package, the average number of seconds spent was only about eight, and it averaged only nine eye fixations per person. Among the nine people viewing the main page of the Web package, only four clicks resulted (from everyone combined). Below the central art element of the feature's main page is a small video window featuring the reporter speaking about the project. Our test subjects ignored this element entirely; not only did they choose not to run the video, they barely even looked at it. We noticed that with the main page, Eyetracking revealed no common area where people first cast their gaze which suggests there's no dominant entrance point. At the article level (after clicking through from the main page), neither of the two people who viewed an article saw the multimedia options available in the right column; they just read text. Possible lessons to be learned from this package: ● ● It's important to promote entrance to multimedia elements in ways that provide the user with room to view and/or click. Not to do so means they may never be seen -- not seen then ignored, but rather not noticed at all. With an enterprise package with lots of components, layout is especially important. Designers need to think about how readers will be drawn to a particular element. Read LATimes.com editor Richard Core's reaction to this eyetracking review. http://www.poynterextra.org/eyetracking2004/multimediafreeforall.htm (3 of 15)9/2/2004 10:15:40 AM Eyetrack III - Multimedia Features Multimedia story 2: Camp Heartland - Time.com This stand-alone multimedia feature started with an auto-running introduction. Five out of 51 Eyetrack III participants viewed this project. Three of them watched the intro, and two of them hit the "Skip intro" button. Most of this feature includes photos in the right side of the window, which are a combination of user-controlled and auto-running. The reader must click an arrow to move to the next photo, but some of the frames are auto-running to show several pictures in succession -- then the reader has to click again to get to the next batch of photos. Everyone looked at the images, and the typical behavior was to read the text at the left, then look to the photo, back to the text, and back and forth as photos changed. http://www.poynterextra.org/eyetracking2004/multimediafreeforall.htm (4 of 15)9/2/2004 10:15:40 AM Eyetrack III - Multimedia Features As you can see from the heatmap image above, most of the eye fixations were on the text. For eye fixations on the photos, as we often found with photos throughout Eyetrack III, fixations tended to be on human faces, and overlap less than on text. None of our five readers of this page used the small thumbnails to navigate to other "chapters" of the feature. (Note on the heatmap above how few eye fixations landed upon them. A heatmap is an aggregate image showing overall eye activity on a webpage. Red-orange areas indicate the most eye activity, blue-black the least.) After clicking through the first nine photos of the first section, most of them didn't realize that there were other chapters. They thought they were "done" with the graphic when, in fact, there was much more available. An alternative way to continue to the next chapter was to click the arrow after the first group of photos, but only one person figured that out. The "9/9" sequence indicator fooled them into thinking that they were done, when they could have reached the next chapter by clicking the arrow one more time. Possible lessons to be learned from this package: ● ● Subtle navigation appears to be challenging for readers. It's important to make navigation obvious, because most people don't spend much time looking intently at multimedia pages. Small thumbnails don't get looked at much, it appears. (We noticed the same thing on MSNBC.com's Big Picture feature described below.) Multimedia story 3: The Big Picture/Oscars - MSNBC.com http://www.poynterextra.org/eyetracking2004/multimediafreeforall.htm (5 of 15)9/2/2004 10:15:40 AM Eyetrack III - Multimedia Features MSNBC.com is a news site that's long been pushing the envelope with multimedia editorial content and its "The Big Picture" series of features pushes the hardest. The Big Picture combines video, audio, still images, text, interactive quizzes, and background material. An innovative feature is a video narrator who guides you through the features. We tested The Big Picture Oscars 2003 package. (Note: MSNBC.com changed the template somewhat for Big Picture packages published after the 2003 Oscars package.) Six of our 51 test subjects opted to spend time with this feature. This multimedia project seemed to engage those who spent time with it. The average time spent viewing it was 189 seconds; on average, people who viewed the feature had 298 eye fixations throughout the viewing period, making it the second-most intently viewed of our eight multimedia features. The Big Picture Oscars is, upon being launched, an auto-running feature. You can simply sit back and watch as the various segments of the presentation are shown to you. Or you can click on one of the navigation elements on the left side and control where you go and what you see and hear, changing the default order if you wish. All six viewers of this feature let the auto-run play for at least a little while upon entering the presentation. However, five of the six chose to use their mouse to interact with content, to control its flow. In terms of where people looked on the page initially, all six people exhibited the same behavior. Within the first seven seconds, everyone looked at the left navigation area and the main content window. Five of six people looked at the "time remaining" graphic below and to the left of the main content window, as you can see in the heatmap image below. For the various video segments of the feature, the counter gave viewers a quick read on how long the segment would last. http://www.poynterextra.org/eyetracking2004/multimediafreeforall.htm (6 of 15)9/2/2004 10:15:40 AM Eyetrack III - Multimedia Features An interesting observation involved the left side of the feature, where there are a series of small icons next to text links to the various sections ("Welcome"; "Vote: Best Actor"; etc.). As you can see from the heatmap above, which aggregates all six participants' data across their entire visit to the piece, most of the people (four of the six) did not look at the icons, but only at the text descriptions. Of the six viewers of this feature, two spent only a short time with it before going elsewhere (less than half a minute); three spent a medium amount of time (from a couple to several minutes); and one spent a lot of time (9 minutes) looking at it and examining most parts of the presentation. The two who left the feature quickly never visually engaged with the female narrator in the lower right. The rest did look at the narrator, moving their eyes back and forth between the main video/image window and the small narrator area when she was talking. Of the four people who did look at the narrator, three of them looked only at her face; the other looked at her face and upper body. Finally, note that on the feature there is a small "Hyundai" sponsor link just above the narrator's face. That small, subtle ad received a lot of eye fixations; five of six people looked directly at it. This fits with observations elsewhere in our research that ads in close proximity to often-viewed editorial content also receive lots of direct views. Possible lessons to be learned from this package: ● ● The innovative technique of having a video narrator guide viewers through a complicated graphic seems to attract readers. People looked at her and paid attention to her face. Our small sample of viewers opted to control the flow -- bouncing around to specific auto-run segments -- rather than rely solely on the length-offeature auto-run capability of this multimedia project. While the sample isn't big enough to make a sweeping conclusion, our observations do http://www.poynterextra.org/eyetracking2004/multimediafreeforall.htm (7 of 15)9/2/2004 10:15:40 AM Eyetrack III - Multimedia Features point to the notion that interactivity is important to Web users. Read MSNBC.com senior producer for broadband productions Ashley Wells' reaction to this eyetracking review. Multimedia story 4: Obesity in America - Associated Press This AP multimedia graphic is simpler than some of the other features we included in this part of the Eyetrack research. It contains a number of interactive charts and infographics outlining the obesity problem in the U.S., presented sequentially (but no photos, slideshows, audio, or video). One of the first things we noticed was that the navigation element -- text links across the bottom of the feature -- wasn't used much, even though most people did look at it. Of 11 out of 51 test subjects who viewed this feature, only two used the navigation. Of those who did not use the navigation, seven of them did look at it, the eyetracker revealed; two people never looked at it. (The alternative to navigating the content of the feature from the bottom links was to click through a series of arrows at the bottom of the screen.) The visuals of this feature were charts and graphics -- no photographs. As you can see from the heatmap image below, fixations were fairly equal overall between text and graphics, with a slight bias toward graphics. http://www.poynterextra.org/eyetracking2004/multimediafreeforall.htm (8 of 15)9/2/2004 10:15:40 AM Eyetrack III - Multimedia Features This feature included several frames of content; you had to click through multiple screens to see everything. Intensity of reading fell off as the feature went on. Most of the readers looked heavily at both text and graphics. Of the light readers of this feature -- about one-quarter of the group -- the tendency was to skim the graphics and not read the text. Several frames into the feature was an interactive Food Pyramid graphic; users could click on the various components to get additional information. Six of our group of 11 did get that far into the graphic, but only three of them clicked to reveal the hidden information. There was also an interactive U.S. map earlier on, which showed obesity rates during different years. The map auto-ran to show changes, then the user could click on a year to see the map change again. Four people clicked on the map after the auto-run feature ended. Possible lessons to be learned from this package: ● ● Charts and graphs appear to have eye-pulling power. In fact, among people who did look at them, they drew more eye fixations than most photos tested elsewhere in Eyetrack III. Interactive components of a multimedia graphic may not be overtly obvious to the user, and when they are too subtle, people may overlook an interactive opportunity. Multimedia story 5: Smart Growth - KQED.com This multimedia feature takes the form of a play-along graphic. You read some text instructions, then "build" your own community -- using illustrated http://www.poynterextra.org/eyetracking2004/multimediafreeforall.htm (9 of 15)9/2/2004 10:15:40 AM Eyetrack III - Multimedia Features components like roads, buildings, and parks -- based on land-use choices. At the end of the short game, you receive a score and a page of advice (presented in text). This game held the attention of all nine of the 51 test subjects who viewed the feature. Every one of them played through to the end. Six of the nine read most of the text on all the frames of the feature thoroughly. The text blocks were all short, except for the final score-and-advice page. About half the group thoroughly read the final score and advice text; the other half skimmed it. http://www.poynterextra.org/eyetracking2004/multimediafreeforall.htm (10 of 15)9/2/2004 10:15:40 AM Eyetrack III - Multimedia Features Possible lessons to be learned from this package: ● It would appear that interactive games -- at least short ones with clear instructions -- can hold people's attention. Multimedia story 6: 50 Years of Corvettes - NYTimes.com This multimedia feature is typical of the Web slideshow genre. It features a series of slides of Corvettes through the years, with audio narration by a New York Times automobile correspondent. http://www.poynterextra.org/eyetracking2004/multimediafreeforall.htm (11 of 15)9/2/2004 10:15:40 AM Eyetrack III - Multimedia Features Upon loading the page, the audio starts and the feature auto-runs to show several car photos and accompanying captions. The only user-control option is to stop the sound temporarily. Once the feature has run completely through, the user can click thumbnails below the main image pane to review the other cars. The heatmap below shows that the main parts of the feature -- the large photo and the accompanying captions (which changed along with the photos) -- were looked at about equally by the six of our 51 test subjects who viewed. http://www.poynterextra.org/eyetracking2004/multimediafreeforall.htm (12 of 15)9/2/2004 10:15:40 AM Eyetrack III - Multimedia Features What's interesting about this heatmap (which compiles viewing data from this feature's entire run-time across all participants who visited it) is that it shows several people clicking on the large photo, as though they expected the click to do something. (You'll note that one person -- ID no. 1540 -- not only clicked the photos several times, but also clicked the caption text.) That behavior is quite common, we found. People often click photos, and a minority of people routinely click random points on the page as they look around. Another interesting observation is that while the audio was running, people tended to read the captions less. This feature included a banner ad underneath the editorial content, and five of the six viewers looked at the ad. Overall, however, this feature didn't seem to hold viewers. Two left after seeing the first car; two left on the second car; one left on the third. Only one person looked at most of the car photos, and that was by switching between photos via manual control. No one let the feature auto-run in its entirety. Possible lessons to be learned from this package: ● ● Auto-run slideshows perhaps should have a user-control option. No one let this auto-run through to completion. We saw a similar desire to control the flow of content in the MSNBC Big Picture feature described above. When there's audio narration, as in this feature, there can be too much stimulation when the viewer is expected to listen to the narration, look at a photo, and read a caption. Perhaps it makes sense to build in pauses in the audio to allow for text reading. Multimedia story 7: Saddam's Sons - CBSNews.com In a fairly small screen footprint, this feature includes lots of content: photo slideshows, video, infographics, maps, and text. Navigation through all the content options occurs by clicking a list of topics from the main page, and from inside pages by a pop-up menu in the upper right. Eight people in our test pool of 51 viewed this multimedia project. http://www.poynterextra.org/eyetracking2004/multimediafreeforall.htm (13 of 15)9/2/2004 10:15:40 AM Eyetrack III - Multimedia Features The majority of pages of this feature included an image or photo slideshow in the main content window, with explanatory text on the right side. Overall, the eyetracker showed that people spent more time looking at the text than the images. The heatmap image below is typical of what we saw as people viewed this feature, with heaviest eye fixation on the text. Of the text blocks, many were too long to fit in a single screen and included a scroll bar for further reading. Six of eight people did get to a page where text scrolled, and five of them did scroll to read further at least once. However, in the majority of instances they did not bother to scroll -- rather just reading or skimming the visible text. Because the overall size of the feature is small, the photographs are of modest http://www.poynterextra.org/eyetracking2004/multimediafreeforall.htm (14 of 15)9/2/2004 10:15:40 AM Eyetrack III - Multimedia Features size, too (most often 370 x 278 pixels). Perhaps that partly explains why photos weren't viewed much and text dominated the eye activity on this feature. The only time photos were looked at more heavily was when people viewed the slideshows, which required clicking through to see a series of images. However, only two people viewed a slideshow. This feature also included a banner ad at the bottom of the screen. Only two people ever looked at the ad; most concentrated their gaze on the editorial content and never ventured far enough down to look directly at the ad, even for a fraction of a second. Possible lessons to be learned from this package: ● We wonder if the small footprint of this feature and the resulting modest size of the images led to the dominance of text. Add/view feedback on multimedia reports Written by Steve Outing and Laura Ruel, project managers; research and tools by Colin Johnson, Greg Edwards, and Leslie Kues of Eyetools Inc. http://www.poynterextra.org/eyetracking2004/multimediafreeforall.htm (15 of 15)9/2/2004 10:15:40 AM Eyetrack III - A History of News Eyetracking MORE ABOUT EYETRACKING •Welcome to Eyetrack III •Eyetrack history •FAQ •How people read •Eyetrack is not a solution •How accurate is eyetracking? •The Eyetrack III team •Discussion forums Multimedia specials •Heatmaps for test homepages •Graphic: Understanding a heatmap •Graphic: Understanding an individual user session •Interactive quiz •Videos: Individual eyetracking sessions EYETRACK III RESULTS •Overview •Homepage design •Advertising •Article-page design •Multimedia content Eyetrack: A History of News Consumer Behavior The latest Eyetrack study on news websites is the third examination of news consumer behavior conducted by The Poynter Institute and its partners over the last 13 years. Through Eyetrack research, we've learned about how news consumers interact with print editions of newspapers (Eyetrack I, 1990-1991) and first-generation news websites (Eyetrack II, 1999-2000). 'Eyes on the News': Print Eyetracking - 1990-91 Way back in the late 1980s, The Poynter Institute first got interested in learning more about news consumers' behavior through eyetracking research. Poynter affiliates Mario Garcia and Pegie Stark Adam, working with researchers and EYE-TRAC Research technology from Gallup Applied Science of Princeton, New Jersey, conducted the first eyetrack studies of printnewspaper readership. Their findings were published in 1991 in a book, "Eyes on the News" (available from the Poynter Bookstore for $5). The research was groundbreaking. This was the first time that a significant independent study had been conducted for the newspaper industry using eyetracking technology. Focusing especially on the use of color, the research produced some findings that startled and surprised the news industry. In his introduction to "Eyes on the News," Poynter's Roy Peter Clark wrote: "As I read (the findings), I sat scratching my head as myth after myth about newspaper reading fell by the wayside." Among the study's key findings: ● INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter ● ● ● ● Color photos do not automatically draw readers. Content, size, and placement are more important. Readers will enter a newspaper page wherever the most powerful element is -- and are willing to follow trails that editors lay for them. Readers look at facing pages as single units. Readers are willing to accept bold, even outrageous color experiments. Color does not detract from a reader's acquisition of visual information. The 1991 research was conducted in three U.S. cities: Santa Ana, Calif.; Minneapolis, Minn.; and St. Petersburg, Fla. Realistic 20-page prototypes of the major daily newspapers in those cities were created for the testing. A total of 90 people were selected for the study, and their precise eye movements recorded throughout their sessions. http://www.poynterextra.org/eyetracking2004/history.htm (1 of 4)9/2/2004 10:16:00 AM Eyetrack III - A History of News Eyetracking Subjects in the Eyetrack I newspaper testing had to wear awkward dual-camera headgear. (From Eyes On the News.) Back then, eyetracking technology wasn't as sophisticated as it is today. Test subjects sat at a desk with a bulky headpiece containing two video cameras and a reflective visor, with wires attaching the unit to a computer. This was as "realistic" a reading environment as could be created given the limits of the technology at the time. Videotape of each participant's session was analyzed to record where the reader entered a page and the number of elements -- headlines, photos, cutlines, and text -- looked at along the path through the page. By seeing how long the eye remained on any one element, the researchers determined to what degree the material was processed. Online Eyetracking - 1999-2000 In 2000, Stanford University and Poynter researchers published data from the first eyetracking study of news websites, which offered some surprising results. Stanford researchers, who had been videotaping news readers in their homes and offices for several years, began to use eye-tracking equipment in a laboratory setting to analyze the behavior of 67 test subjects as they viewed a variety of real news websites (though a technical problem resulted in the loss of data from some of the volunteer testers). http://www.poynterextra.org/eyetracking2004/history.htm (2 of 4)9/2/2004 10:16:00 AM Eyetrack III - A History of News Eyetracking Participants in the 2000 Online Eyetrack study wore slightly sleeker headgear than participants in the 1990 Eyetrack research. The testing procedure for the 2000 study (which took place the year prior) involved having volunteers don a piece of headgear -- strung with awkward wires -- that had a small camera attached, then calibrating the equipment. It was an improvement over the equipment used in the Eyetrack I study, but a far cry from Eyetrack III (the latest study), where volunteers had no headgear; a camera mounted in the computer monitor tracked their eye movements, making for a far more realistic news-reading experience. The software for analyzing the resulting data from the tests was crude in comparison to what's available today, remembers Greg Edwards, research associate for Stanford during the 2000 study and co-founder of Eyetools, the eye-tracking company that conducted the latest round of research. In the 2000 study, participants were told to view any news websites they wished, and given no specific instructions. The test was free-form, and researchers pored over the data afterward trying to determine trends. (By contrast, the 2003 testing was highly structured, to better learn about patterns for specific web-page characteristics.) Results of the 2000 study surprised many people in the still-fledgling onlinenews industry. One finding was that online-news readers look first to text (especially briefs and photo captions), not images -- which was opposite the behavior of print news readers, as observed in the 1991 eyetracking study. (Critics explained the discrepancy this way: While the test used a high-speed Internet connection where web-page images appeared on screen quickly, most users at that time were used to dial-up connections where text appeared first and photos scrolled slowly on the page. Perhaps users were conditioned to look for text first and were repeating patterns that had become familiar to them.) Another surprising finding was that banner ads indeed did catch online readers' attention -- a notion that went against current thinking. Some 45 percent of banner ads were viewed by test subjects, for an average fixation time of one second, which is long enough to perceive the ad. The ads actually were viewed more often than editorial graphics. (Even in 1999, web banner ads weren't doing well, so this finding became suspect among critics.) Here are a few more findings from the 2000 Online Eyetrack study: ● Thirty-year-olds were more likely to read local news than either 60-year- http://www.poynterextra.org/eyetracking2004/history.htm (3 of 4)9/2/2004 10:16:00 AM Eyetrack III - A History of News Eyetracking ● ● ● ● olds or 20-year-olds. And 20-year-olds read more science and sports news than did other age groups. Virtually all ages read opinion articles in healthy proportion to their total article reading. Eighty percent of all participants read crime and disaster coverage. Women were very slightly more likely to read this category, but men were more likely to read more items. (These judgments are based on proportion of each gender to its number in the study: 30 women, 37 men.) Sports, surprisingly, was read equally by males and females -- 70 percent of the total for each gender. But, no female read heavily in this category, while 11 percent of the men did. In fact, the heavy sports readers were likely to exceed the number of items read per person than in any other category. A higher proportion of women read local news than did men, and by a tiny margin also read more heavily in this category. Overall, 48 percent of all participants read local news. Somewhat more men than women read national news and by a small margin, also read more items. Overall, 67 percent of all participants did some reading of national news. You can read more about the 2000 Poynter-Stanford Online Eyetrack study here. Eyetrack III: 2003-04 Eyetrack III, the subject of this website, is Poynter's third media eyetracking research project. Scroll to the top of the page to find navigation links to the rest of the most recent study. Written by Steve Outing and Laura Ruel, project managers; research and tools by Colin Johnson, Greg Edwards, and Leslie Kues of Eyetools Inc. http://www.poynterextra.org/eyetracking2004/history.htm (4 of 4)9/2/2004 10:16:00 AM Eyetrack III - How We Read MORE ABOUT EYETRACKING •Welcome to Eyetrack III •Eyetrack history •FAQ •How people read •Eyetrack is not a solution •How accurate is eyetracking? •The Eyetrack III team •Discussion forums Multimedia specials •Heatmaps for test homepages •Graphic: Understanding a heatmap •Graphic: Understanding an individual user session •Interactive quiz •Videos: Individual eyetracking sessions EYETRACK III RESULTS •Overview •Homepage design •Advertising •Article-page design •Multimedia content The Basics: How We Read In understanding eyetracking results and data, it's helpful to understand the process of reading. People typically recognize words as a whole; we don't look at individual letters in each word. According to psycholinguist Keith Rayner of the University of Massachusetts at Amherst, eyes do not move smoothly across the text as we read. Instead, the typical reader behavior is to look at a word or several words in a group, and pause our eyes there briefly. This is called a "fixation," and it takes about 0.25 seconds on average. After a fixation, we move our eyes to the next word or group of words; this movement is called a "saccade" and takes only 0.1 seconds. (People often skip over short or predictable words such as "of," "in," "a," etc.) After this pattern is repeated once or twice, we pause to comprehend the phrase just viewed (which on average takes 0.3 to 0.5 seconds). According to Rayner, all these fixations and saccades result in 95 percent of all college-level people reading between 200 and 400 words per minute; 300 words per minute is the average. As you read about Eyetrack III findings on this website, you'll often see references to "fixations." A fixation is a brief, but measurable, pause of the eye when looking at a word or phrase. "Saccades" are the paths between these fixations Written by Steve Outing and Laura Ruel, project managers; research and tools by Colin Johnson, Greg Edwards, and Leslie Kues of Eyetools Inc. INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter http://www.poynterextra.org/eyetracking2004/howweread.htm (1 of 2)9/2/2004 10:16:09 AM Eyetrack III - How We Read http://www.poynterextra.org/eyetracking2004/howweread.htm (2 of 2)9/2/2004 10:16:09 AM Eyetrack III - Eyetrack Is Not a Solution MORE ABOUT EYETRACKING •Welcome to Eyetrack III •Eyetrack history •FAQ •How people read •Eyetrack is not a solution •How accurate is eyetracking? •The Eyetrack III team •Discussion forums Multimedia specials •Heatmaps for test homepages •Graphic: Understanding a heatmap •Graphic: Understanding an individual user session •Interactive quiz •Videos: Individual eyetracking sessions EYETRACK III RESULTS •Overview •Homepage design •Advertising •Article-page design •Multimedia content INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter Eyetrack Is Not a Solution By Howard Finberg Research is a tool, not a solution. Eyetrack III is a tool, not a solution. Eyetrack is an effort to show how online news users process information on a webpage. We look through the eyes (literally) of a group of consumers as they view sample news websites, multimedia editorial content, and advertising. This is the third eye-tracking study conducted by Poynter since 1991. For media company managers and staff, this Eyetrack research will provoke important discussions and may help guide sites in conversations about redesign and navigation decisions. It is likely that there will be some findings that will raise questions and concerns. Fundamentally, however, the Eyetrack results are just one more tool to help journalists do their jobs better. Research tools, like the garden tools lying in the backyard shed, don't really mean much without two important factors: ● ● A strategic vision of where you want your site to go. The integration of all other elements -- your market research, your technical abilities, and content resources. Finally there's the sweat and hard work to make it come together. It is your job to put all of the factors -- research, resources, vision -- to good use. We work in a society that often seems like it is governed by research. There are statistics for just about any topic you want to address. And there is research that can pretty much prove any point you want to make. For many news media managers, research is a strange and unknown commodity. We embrace it when it proves our point; we dismiss it when it makes us uncomfortable. Research that breaks new ground often challenges our preconceptions. And to that I say "Hooray." What research should not do, however, is govern our actions. Research can not tell us what to do any more than the Magic 8 Ball can provide managerial decisions. Jakob Nielsen, principal of Nielsen Norman Group, a user research consulting http://www.poynterextra.org/eyetracking2004/solution.htm (1 of 3)9/2/2004 10:16:15 AM Eyetrack III - Eyetrack Is Not a Solution company, has been creating and studying Web usability research for years. He agrees that research can only highlight an issue or problem. "Research is a reality check. It tells you what really happens when people use computers. We can speculate on what customers want, or we can find out. The latter is the more fruitful approach," he said in an e-mail interview. "Unfortunately, research doesn't tell you what to do. It shows what works and what causes problems, but it doesn't say 'this is how to build the next release of your website.'" Mike Donatello, director of Survey Solutions for comScore Networks, Inc., has this caution about any eye-movement study: "Folks ought to realize that it's very granular measurement that indicated how readers' near-automatic parsing of on-screen information is affected by presentation elements. Unless attitudinal or recall data are included in the study, no one should necessarily make the leap that just because folks read/glance at a page in a certain way means that they remember or comprehend the information any better," he said in an e-mail interview. "Studies like this are best used in a diagnostic capacity or to provide a general framework, rather than strict guidelines or rules." But what happens when you disagree with what's being presented? Rusty Coats, director of New Media at MORI Research, a company that works with many newspaper websites, likes the idea of disagreement. "Disagreement can be the best thing for research, because it forces us to look harder at the facts rather than preconceptions and emotions. First, make sure the data is sound. Research is like cooking a cake: There are a lot of ingredients, and they all have to be exactly right. If you disagree with the data, dig into it," Coats said. He urges examination of research in the same manner as an objective reporter looks for truth. "Sometimes, people disagree with research because of a belief they've had, a dogma they've established, a purely emotional response. In any business, these can be false idols," he said in an e-mail interview. The issue of emotional response to new research is an important one. Journalists advance through their careers by learning the stories of their organizations. We learn the mythology of the places we work and of the leaders we follow. This mythology shapes our beliefs and values. Sometimes it is very hard to surrender those beliefs when new data arrives. "If your intuition disagrees with research findings," Nielsen says, "you should view this as a learning opportunity to improve your insights in the future. Design is not religion. You don't have to defend the beliefs of your forefathers to the bitter end. Design is a business decision, so you should follow the data and do what works best for your company." Coats looks at the disagreement issue this way: "No one is forcing anyone to agree with the research. If the data is sound and you still disagree, that's certainly your prerogative, though ultimately it might not be a very wise one. There's no profit in making customers wrong, though it can be very tempting at times." http://www.poynterextra.org/eyetracking2004/solution.htm (2 of 3)9/2/2004 10:16:15 AM Eyetrack III - Eyetrack Is Not a Solution One of the biggest challenges to readers of the Eyetrack III research will be the volume of information available. And while the authors have done a great job in organizing the material, there's a lot to absorb. Here are some suggestions from Nielsen, Coats, and myself on how to cope with this problem: ● ● ● ● ● ● ● ● ● ● ● Process the information in small bites. Take a finding and look at other sites, at first, rather than your own. Give yourself time to absorb the information before acting or reacting. Print out a copy of your homepage and circle design elements that contradict a research finding. Assemble a diverse group of staff members and have them discuss different findings. Find similar research; compare and contrast. Join the online discussion at Poynter Online and ask questions. Assume nothing; start from a position called "zero sum budgeting" where every rule is open to challenge and change. Ask for help from your company's research department. Write down questions and then look for answers within the data. Go back and read the research again. Coats says one of the best places to look for help is "a study's authors and authors of similar studies. And then ask everyone -- all the wickedly bright people you know -- 'What do you think of this? How are you applying this? What makes sense to you?' Sometimes the best thing research generates is conversation." http://www.poynterextra.org/eyetracking2004/solution.htm (3 of 3)9/2/2004 10:16:15 AM Eyetrack III - How Accurate Is Eyetracking? MORE ABOUT EYETRACKING •Welcome to Eyetrack III •Eyetrack history •FAQ •How people read •Eyetrack is not a solution •How accurate is eyetracking? •The Eyetrack III team •Discussion forums Multimedia specials •Heatmaps for test homepages •Graphic: Understanding a heatmap •Graphic: Understanding an individual user session •Interactive quiz •Videos: Individual eyetracking sessions EYETRACK III RESULTS •Overview •Homepage design •Advertising •Article-page design •Multimedia content INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter How Accurate Is Eyetracking? By Colin Johnson, President & CEO, Eyetools Inc. "How accurate are the findings from this Eyetrack III research?" There are three ways to address this question depending on the degree of abstraction with which you're thinking about this study: at the level of the individual fixation, at the level of the individual page, and at the level of a test participant's overall experience. 1) The first way to answer this question focuses on the equipment and the most granular view of the data. Are the data and findings from the equipment accurate? To this question the answer is yes, the findings are accurate. An eyetracker can capture and report what people are looking at to within a centimeter's distance when you measure a person's point of gaze on a typical computer screen. The manufacturers of eyetracking devices report this accuracy in units called "visual angle." The Tobii eyetracker that we used in this study is accurate to within 1 centimeter of visual angle. This means that if you draw an imaginary line from a person's pupil to the item she is viewing, the machine will report what it thinks she is looking at within 1 degree of that imaginary line. To explain it in geometric terms, the line between the eye and the point of gaze is the long leg of a right triangle. The short leg of the right triangle is the distance between the true point of gaze and where the eyetracker thinks the person is looking. The visual angle is measured between the long leg of the right triangle and the hypotenuse of the right triangle drawn by the joining of the long leg and the short leg. When the long leg is about 50 centimeters (approximately 2 feet), a typical distance between a computer monitor and a person's eye, one degree of visual angle drawn between the long leg of the triangle and the hypotenuse translates into a short leg of the triangle that is about one centimeter long. When you consider that the human eye can perceive the immediate surrounding of its point of gaze through its peripheral vision, the 1 degree of error from the eyetracker is lost in the noise of how the human eye works anyway. When Eyetools displays individual viewing behavior, it has developed a patented visualization that incorporates a region around the fixation point that looks like a halo. This halo illustrates peripheral vision. As the distance increases away from the point of gaze, the acuity gradient or "halo" becomes dimmer. This signifies how the acuity of our peripheral vision degrades the farther away one measures from the point of gaze. Think about it yourself. As you are staring at this word on the screen, you can see that there are other words on the screen. Without moving your eyes, you can probably make out the couple words in front of and the couple words http://www.poynterextra.org/eyetracking2004/accuracy.htm (1 of 4)9/2/2004 10:16:23 AM Eyetrack III - How Accurate Is Eyetracking? behind the word you're reading right now. You can probably also make out the words on the line above, or even the line above the line above this one. The same holds for the lines below. Chances are, however, that you can't make out specific words that are a couple paragraphs away. Now, without moving your eyes, try to become aware of the computer screen and the desk. How about the room at the periphery of your field of vision? So you see, at the individual level, the data accurately reflects what people see on a screen. While there is a margin of error of plus or minus 1 degree of visual angle, this error falls within the margin of error of the natural function of the human eye. (As an example, that is to say that it is completely natural for people to focus just above or just below the line of text that they are actually reading.) To further compensate for this, the representation of the data also reflects this so that one can intuitively appreciate the various options that might be the focus of a person's gaze given a fixation point drawn on a page. 2) The second way to answer this question is to think not about the accuracy of the individual fixations but rather to contemplate the accuracy of displaying the viewing behavior of groups of people upon a single page. Let's explore in greater detail how the aggregate images or "heatmaps" are generated and what they represent. The aggregate image is another patented visualization from Eyetools. The basic aggregate image represents the viewing of a specified group of individuals on a given page. The warmer colors at specific points on the page signify that a larger percentage of the group viewed those points. One specifies whose data to include in a given aggregate image, and then the Eyetools Analysis Solution calculates all the relative percentages and displays them in their respective colors. The key to the accuracy of the aggregate images is that they account for the acuity gradient in the same way that the individual images represent each test participant's peripheral vision. That means that the aggregate images represent where the participants of the group fixate as well as the areas of the page that could potentially have been perceived through the participants' peripheral vision as well. When an aggregate image includes data from eight people or more, the smoothing of these relative percentages of acuity gives a representative display of all the areas on the page where the individuals' viewing patterns overlapped. Through the algorithms used to generate this data visualization, the portions of the page that only a minority of the group viewed are de-emphasized. The warm colors indicate over-arching consistency in viewing patterns across a group. Viewing behavior represented by cool colors is not as consistent. While these aggregate images do not tell the whole story of each participant's visual interaction with each page, it does give an accurate representation of which pixel by pixel region draws the attention of the group. Now that we've explained the mechanics of the aggregate image, let's shift gears back to how these images are used in analyzing design performance. To do this, it is first important to highlight a phenomenon that few people realize exists. There are distinct differences between how a person views a page when seeking to evaluate it and how a person views a page when attempting to use it. Those people who are responsible for the design or content of a page give the page serious scrutiny and often virtually commit to memory every single pixel of that page. When they think about how their target user will experience the page, it is difficult to imagine what the experience is of a visitor who comes to that page only once or twice, and if that much, only briefly. It can be difficult for design professionals to recognize that the infrequent visitor to the page only http://www.poynterextra.org/eyetracking2004/accuracy.htm (2 of 4)9/2/2004 10:16:23 AM Eyetrack III - How Accurate Is Eyetracking? sees a limited amount of information on the page and consequently is able to commit only a small fraction of that information to active memory. If it is not in the data, it is not in the minds of your customers. The aggregate image is helpful for these professionals because it gives a single-glance depiction of what information they can assume visitors have in their minds after visiting a page. It is critical to emphasize that if the data says that people did not look at a given area of a page, then it's true. They really did NOT look at that area. The data represents what visitors focus on AND what they might perceive through their peripheral vision. If they didn't look, that area is not in their active memory. Because it is not in their active memory, they do not consider it when deciding what to do next or when they determine if they got the information that they require. The technical term for what people have in their minds and can act from is called their "consideration set." A case that illustrates this point was an independent trial that Eyetools researchers conducted in January 2001 on an early version of the E*TRADE website. The center section of the homepage at that time featured promotional offers designed to entice prospective customers to sign up. The researchers selected a group of six participants who fit the profile of potential customers of online brokerage services and asked each to visit the E*TRADE site and evaluate whether or not they might consider signing up for the service. The researchers tracked each participant's eye movements and then analyzed the data. Much to their surprise, they discovered that virtually none of the test participants more than glanced at the very promotions designed to draw them in and entice them to sign up. These promotions were virtually ignored completely. The researchers wanted to determine if the data could predict what future users to the site might include in their consideration set. To test this, they cached the homepage, converted the text to complete gibberish on all the areas of the page where they determined that visitors ignored, and then invited another six participants to repeat the same task of evaluating whether or not they might consider becoming customers. The only difference was that upon completion of the task, the researchers asked each participant if they thought there was anything weird or unusual about the site. What they found was that not only was the viewing data essentially the same as the first round of testing, but that not one of the participants found anything odd about the site. "No, no, the site was great. No problem at all. Everything seemed fine." The researchers then tested the site against a much larger sample of participants and found the same exact reaction. Only one person in 20 identified that there was anything strange about the site, suggesting that the visual presentation had failed to achieve the primary purpose of the homepage -- to draw in new customers. This example illustrates how important it is to understand what users include in their consideration set, and then to proactively modify the visual presentation of underperforming pages if the data suggests that a page's design is not performing as effectively as it could. The aggregate image represents this well and has been proven to be an accurate way to highlight potential problem spots on a page. 3) The third way to address this question is to consider whether or not the findings from eyetracking analysis are accurate representations of the overall experience of one's users. This is one of the most exciting areas of user research and one of the elements that we hope to explore further in Eyetrack http://www.poynterextra.org/eyetracking2004/accuracy.htm (3 of 4)9/2/2004 10:16:23 AM Eyetrack III - How Accurate Is Eyetracking? III. Simply put, the field of eyetracking research in website evaluation is still young. Researchers in this area are only just beginning to uncover the connections between eye movement and overall user experience. Until recently, the challenge of capturing and analyzing data from users browsing websites was nearly impossible for most researchers because no tools existed to help them correlate the output from an eyetracker to scrolling webpages. The Stanford/Poynter Eyetrack 2000 study (Eyetrack II) was one of the first of its kind. When he was still a researcher at Stanford working on Eyetrack II, Eyetools founder Greg Edwards had to create a software application from scratch to meet this challenge because nothing existed in the marketplace for him to use. The code from that original software application became the foundation of the Eyetools Analysis Solution which we used to do the data processing and analysis for Eyetrack III. This is what we do know. How people use their eyes is an important indicator of what people think about things, what they like and dislike, where they succeed and how they fail when using websites. Psychologists have known for years that eye movement reflects both the unique drivers in an individual's conscious thought as well as the unconscious drivers that are consistent across a population. The challenge has been to capture the right data and then efficiently segment and analyze the data in a way that uncovers those drivers. Because of this, the quality of one's analysis tools determines how much insight one can expect to glean from analyzing this data. The team at Eyetools believes that effectively interpreted eye movement can reveal more refined insights into one's customers than virtually any other indicator. The current Eyetools Analysis Solution represents a great advance in the analytical capabilities available to researchers and practitioners. To make its adoption easier, we have designed it to seamlessly integrate into conventional user testing as an augmentation to what companies and universities are doing today. As the solution enables ever-growing levels of analysis automation, users of the solution will find it easier and easier to use eyetracking data as a way to understand overall customer experience. http://www.poynterextra.org/eyetracking2004/accuracy.htm (4 of 4)9/2/2004 10:16:23 AM Eyetrack III - The Team MORE ABOUT EYETRACKING •Welcome to Eyetrack III •Eyetrack history •FAQ •How people read •Eyetrack is not a solution •How accurate is eyetracking? •The Eyetrack III team •Discussion forums Multimedia specials •Heatmaps for test homepages •Graphic: Understanding a heatmap •Graphic: Understanding an individual user session •Interactive quiz •Videos: Individual eyetracking sessions EYETRACK III RESULTS •Overview •Homepage design •Advertising •Article-page design •Multimedia content INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter The Eyetrack III Team Eyetrack III is a joint effort by The Poynter Institute, the Estlow International Center for New Media and Journalism at the University of Denver, and Eyetools, a California-based eyetracking software and services company. Morris Digital Works provided considerable technical support during the project. Project managers: Steve Outing, senior editor, Poynter Institute Laura Ruel, former executive director, Estlow Center; assistant professor of multimedia, School of Journalism and Mass Communication, University of North Carolina at Chapel Hill ● ● Editing and additional assistance: Julie Moos, news editor, Poynter Institute ● Bill Mitchell, online editor/marketing director, Poynter Institute ● Howard Finberg, interactive learning faculty, Poynter Institute ● Robin Sloan, interactive learning producer, Poynter Institute ● Elizabeth Carr, online reporter, Poynter Institute ● Design, graphics: Anne Conneen, design editor, Poynter Institute ● Larry Larsen, multimedia editor, Poynter Institute ● Eyetools team: Colin Johnson, CEO ● Greg Edwards, founder and CTO ● Leslie Kues, research manager ● Daesub Yoon, research support and statistical testing ● Morris Digital Works: Nik Wilets, mockup webpage designs. ● Thanks! The Eyetrack III team also would like to thank Steve Yelvington and Ed Coyle of the Morris Digital Works team, whose contributions made this research possible. We also were helped on the multimedia comprehension phase of the study by the multimedia and public relations staffs of NYTimes.com. Special thanks to Geoff McGhee and Christine Mohan. And of course, thanks to all of our colleagues at Poynter and the University of Denver's Department of Mass Communications and Journalism Studies who took the time to review our work and advise the Eyetrack team. http://www.poynterextra.org/eyetracking2004/team.htm (1 of 2)9/2/2004 10:16:27 AM Eyetrack III - The Team http://www.poynterextra.org/eyetracking2004/team.htm (2 of 2)9/2/2004 10:16:27 AM Eyetrack III - Discussion Forums MORE ABOUT EYETRACKING •Welcome to Eyetrack III •Eyetrack history •FAQ •How people read •Eyetrack is not a solution •How accurate is eyetracking? •The Eyetrack III team •Discussion forums Multimedia specials •Heatmaps for test homepages •Graphic: Understanding a heatmap •Graphic: Understanding an individual user session •Interactive quiz •Videos: Individual eyetracking sessions Eyetrack III Discussion Areas We hope the Eyetrack findings will generate lots of conversation and that you will join in and share your thoughts. Here are some discussion areas, separated into categories based on the findings. You can also submit questions to Ask the Eyetrackers. ● ● ● ● Homepage design ❍ Headlines & blurbs ❍ Headline size ❍ Font size ❍ Navigation ❍ Photos & images ❍ Compact & extended Article design Advertising Multimedia EYETRACK III RESULTS •Overview •Homepage design •Advertising •Article-page design •Multimedia content INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter http://www.poynterextra.org/eyetracking2004/discussforums.htm (1 of 2)9/2/2004 10:16:29 AM Eyetrack III - Discussion Forums http://www.poynterextra.org/eyetracking2004/discussforums.htm (2 of 2)9/2/2004 10:16:29 AM Eyetrack 2004 Heatmap http://www.poynterextra.org/eyetracking2004/heatmap.htm (1 of 2)9/2/2004 10:16:45 AM Eyetrack 2004 Heatmap http://www.poynterextra.org/eyetracking2004/heatmap.htm (2 of 2)9/2/2004 10:16:45 AM Poynter Online - Heatmap explainer http://www.poynter.org/content/resource_popup_view.asp?id=27204 (1 of 2)9/2/2004 10:18:53 AM Poynter Online - Heatmap explainer Eyetools, Inc. http://www.poynter.org/content/resource_popup_view.asp?id=27204 (2 of 2)9/2/2004 10:18:53 AM Poynter Online - Individual-session explainer Eyetools, Inc. http://www.poynter.org/content/resource_popup_view.asp?id=272159/2/2004 10:19:03 AM Eyetrack 2004 Fixation Quiz http://www.poynterextra.org/eyetracking2004/FQ.htm (1 of 2)9/2/2004 10:19:15 AM Eyetrack 2004 Fixation Quiz http://www.poynterextra.org/eyetracking2004/FQ.htm (2 of 2)9/2/2004 10:19:15 AM Eyetrack III - Eyetracking-Session Videos MORE ABOUT EYETRACKING •Welcome to Eyetrack III •Eyetrack history •FAQ •How people read •Eyetrack is not a solution •How accurate is eyetracking? •The Eyetrack III team •Discussion forums Multimedia specials •Heatmaps for test homepages •Graphic: Understanding a heatmap •Graphic: Understanding an individual user session •Interactive quiz •Videos: Individual eyetracking sessions EYETRACK III RESULTS •Overview •Homepage design •Advertising •Article-page design •Multimedia content Videos of Individual Eyetracking Sessions Ever wonder what your eyes really see when viewing a website? You may think you know, but the mind works fast -- and your eyes move around a page quickly, with a series of very short fixations on page elements and words. It's a good bet that you are not completely conscious of all the movements your eyes make when viewing a webpage. INDIVIDUAL VIDEOS ● ● Video No. 1 Video No. 2 In the videos on this page, we show you what it looks like through the eyes of a web user. The person tracked in these videos is viewing some of the mock websites created for Eyetrack III. However, this person was not one of our test participants; our guinea pig is a college-educated white male in his early 30s. Don't take what you see too seriously. Everyone views a page slightly differently, and this is just a look through one person's eyes. Nevertheless, it's fascinating viewing. A note about what you see: Blue dots represent fixations; as a fixation gets longer, the dot grows. Lines between the blue dots are saccades, which are the paths between fixations. RELATED: Eyetrack III researchers also reviewed static representations of individual-user page sessions. Here's an explanation. INDUSTRY REACTION •Jay Small: With homepages, it's polite to stare •Jeff Glick: When to tell stories with multimedia, text •Kinsey Wilson: Online ads: Why placement & size matter http://www.poynterextra.org/eyetracking2004/videos.htm (1 of 2)9/2/2004 10:19:25 AM Eyetrack III - Eyetracking-Session Videos http://www.poynterextra.org/eyetracking2004/videos.htm (2 of 2)9/2/2004 10:19:25 AM