E-Surveys Made Easy Using Visitor Email Addresses to Conduct Destination Research Prepared by: Jerry Henry Corporate Director of Research Herschend Family Entertainment 1 Presentation Overview • • • • • Advantages and Limitations of E-Surveys Average Response Rates Do They Really Work? Real-Life Project Examples How To Get Started? 2 Definition • An “e-survey” in this presentation refers to travel surveys that are posted on the web and completed by a destination’s (or attraction’s) visitors. • This is accomplished by sending email messages to past customers, inquiries or website visitors offering an invitation for the respondent to click on a survey link and complete a specific online questionnaire. • While other types of online surveys can be completed by consumers browsing a destination’s website, this type of study differs in that it is completed only by those targeted in the initial email blast—either visitors, past visitors, inquiries, non-visitors, etc. 3 Advantages Less Intrusive Pictures, Sound Clips and Video Clips More Detailed Responses Logical. Telephone Response Rates are Dropping Mainstreaming of the Internet Faster Response Time Less Expensive Free From Interviewer Bias Better Recall 4 Limitations Privacy Legislation Self-Administered Not Everyone is Online Cannot Ensure Desired Respondent Spam Filters Zap 20% of Commercial Emails Non-Response Satisfaction Ratings May Vary Cannot Ask Follow-Up Questions Professional Survey Takers 5 Conduct a Parallel Test • The only way to find out how e-survey results compare to your current methodology is to try it! This can best be accomplished by conducting a parallel test—e.g. running your survey using both methods simultaneously and then comparing the data from both. Although the results are rarely identical (for the reasons mentioned previously), it is important to see if the core measurements line up. • The following metrics should be the same if the online research is to replace another type of research: 1. Is the top-of-mind (#1) mention the same using both methods? 2. Is the rating of products or concepts at a similar level, using both methods? 3. Is any market movement equally indicated using both methods? 4. Are stated behaviors (previous brand usage, previous category involvement) similar using both methods? 6 Metrics May Differ But, Online Surveys May Actually be More Accurate • Unaided awareness: because online respondents generally take more time to think about their responses, they will generally be able to recall more brands, more ads, more experiences, etc. This can make the results MORE accurate than other approaches. • Complex questions: because online respondents can read and re-read questions, they are more likely to respond accurately. In other approaches, respondents rarely ask the interviewer to reread the question for them. (Note: this can be eliminated in either case if the questions are kept very simple). • Detailed questions: because online respondents do not feel the time pressure to give an answer, they may actually investigate the answers before they respond. 7 Results from Herschend Family Entertainment’s Parallel Tests • In 2001, Herschend Family Entertainment (HFE) launched our pilot e-survey program by running six different properties’ parallel e-surveys concurrent with telephone surveys asking the same questions. (Silver Dollar City, Dollywood, Stone Mountain, White Water, Showboat and Splash Country) • Phone and Email survey responses were compared on party composition, age of party members, household income, last visit to the property, other area activity participation, and satisfaction. • In all, we found that in 70% of the line item demographic data elements there was no significant difference in the results. The largest disparities came in the areas of satisfaction and age. 8 Celebration City Packaging Research Example • However, our most conclusive parallel test came as we were preparing to launch Celebration City. The purpose of this study was to determine which combination of properties, days and pricing was most attractive to our customers. • A total of 1,500 phone interviews began simultaneously with an e-survey that asked the same questions. Both studies were conducted among past Silver Dollar City visitors. • How did they compare? The 1,500 phone calls were completed in only 18 days at a cost of $14,000, a bargain by most standards. However, over 1,700 email responses were received in just under 4 days at an incremental cost of less than $1,000. (programmers and analysts are already on the payroll). • And, the results? See for yourself…… 9 Internet Results Ticket Package Tradeoffs 53% 3 Days, 2 Pks, $55 47% 21% Family of 4, $299 79% 54% 4 Days, $59 47% 61% 3 Pk, S. Pass, $89 3 Pk, S. Pass, $99 4 Days, $59 3 Pk, S. Pass, $99 39% 4 Days, $69 3 Days, 2 Pks, $55 76% 24% 1 Day, 1 Park, $32 4 Days, $69 77% 23% Family of 4, $299 1 Day, 1 Pk, $32 76% 24% Family of 4 w/ Hotel 48% 3 Pk, S. Pass, $99 20% Family of 4 w/ Hotel -50% 79% 0% 10 3 Pk, S. Pass, $89 1 Day, 1 Park, $32 3 Days, 2 Parks, $55 51% 21% Family of 4 w/ Hotel n = 584 73% 50% 3 Pk. S Pass, $89 -100% 80% 27% Family of 4, $299 4 Days, $69 52% 50% 4 Days, $59 100% Phone Results Ticket Package Tradeoffs 61% 3 Days, 2 Pks, $55 30% Family of 4, $299 4 Days, $69 1 Day, 1 Pk, $32 24% 72% 28% 76% 24% 56% n = 267 -50% Family of 4 w/ Hotel 4 Days, $69 74% 0% 11 3 Pk, S. Pass, $89 1 Day, 1 Park, $32 3 Days, 2 Parks, $55 49% 26% Family of 4 w/ Hotel Family of 4, $299 68% 51% 3 Pk. S Pass, $89 1 Day, 1 Park, $32 75% 32% Family of 4, $299 4 Days, $69 44% 25% Family of 4 w/ Hotel 4 Days, $59 3 Pk, S. Pass, $99 41% 76% 3 Pk, S. Pass, $99 -100% 51% 59% 3 Pk, S. Pass, $89 3 Pk, S. Pass, $99 70% 49% 4 Days, $59 3 Days, 2 Pks, $55 39% 50% 4 Days, $59 100% Survey Facts & Figures • Over the past three years, HFE has conducted 58 different esurvey projects and distributed more than 340,000 email survey links. Of those, about 88% or 300,000 actually made it into consumer homes. • Response rates among past visitors have averaged 11.5% over that time with a range from from 4.4% to 42.3% (based upon the gross e-blast). However, HFE properties has a significant affinity among their customers. Typically the greater the affinity for the destination or attraction, the greater the typical response rate. Destinations tend to have lower response rates in the 5% to 10% range. • Response rates from rental lists are generally much lower. Typically, only 5%-7% of these kinds of surveys get opened and response rates average only 0.5% on the gross e-blast. However, in a recent Springfield study we got a 2.5% response. 12 Herschend Family Entertainment E-Survey Statistical History No of Projects 58 GROSS Emails Distributed 341,543 100.0% LESS: Undeliverables (37,911) 11.1% LESS: Suppressions (3,756) 1.1% 299,876 87.8% 39,277 11.5% NET Emails Delivered Average Response Rate Target Sample Size 400 Est. Email Blast Required 3,335 13 Examples • • • • • • Springfield Visitor Profile Springfield Conversion Study Research Templates Silver Dollar City Non-Visitor Silver Dollar City Brand Assessment (incl. Consumer Mapping) Celebration City Concept Study 14 Springfield Visitor Profile • The Springfield CVB began conducting guest profile studies via email in 2001. Intercept or phone surveys were cost prohibitive at the time. Given that well over two-thirds of all travelers (and closer to 80% non-VFR travelers) have internet access, the methodology seemed appropriate even in 2001. • Visitor email addresses are collected by area partners, hotels, the visitor center and the airport. E-survey links are distributed each month. Over the years annual sample size has ranged from 700 to 1,200. • Traveler data is downloaded and processed into tables on demand. Year over year trends appear normal in every data category. 15 Springfield Visitor Profile • This information has helped define who visits Springfield, what they do, when they plan their visit, where they’re from, why they visit, and how they spend in the area—among many other things. 16 Party Composition Springfield Visitor Profile Example Variance from 2003 (Points) Other 2% 4% 3% Families 32% 2.9% 2% 1% 0% -1% Adults 66% -0.8% -2% -1.8% -3% Families 17 Adults Other Age Breakdown Springfield Visitor Profile Example Variance from 2003 (Points) 55+ yrs 25% Under 18 yrs 18% 8% 5.7% 6% 4% 2% 0.7% 0% 18-34 yrs 20% -2% -0.6% -4% -6% -5.8% -8% 35-54 yrs 37% Under 18 18 18-34 35-54 55+ Last Visit to Springfield Springfield Visitor Profile Example Variance from 2003 (Points) 25% First Time 30% Over 2 Years 33% 19.7% 20% 15% 10% 8.2% 5% 0% -5% -10% -15% -13.7% -20% ETY/LY 37% ETY/LY = Earlier this year or last year First-Time 19 ETY/ LY Over 2 Years Distance Traveled Springfield Visitor Profile Example Variance from 2003 (Points) 0-100 mi 11% 8% 6.9% 6% 4% 2% 301+ mi 50% 0% 101-300 mi 39% -2% -1.1% -4% -6% -5.8% -8% 0-100 mi 101-300 mi 20 301+ mi % Spending the Night in Springfield by Visitor Segment Springfield Visitor Profile Example 80% 77.9% 77.3% 78% 76% 75.0% 74% 72% 72.9% 71.9% 71.8% 72.1% 70.8% 70% 67.6% 68% 66% 64% 62% Families Adults 0-100 mi 300 mi 301+ mi 21 1st Time Visitors Pleasure Business Average Length of Stay by Visitor Segment Springfield Visitor Profile Example 7.0 5.8 Number of Nights in Spfd 6.0 5.0 4.0 3.6 3.6 3.3 3.1 2.9 3.0 2.0 2.5 1.9 1.5 1.0 0.0 Families Adults 0-100 mi 300 mi 301+ mi 22 1st Time Visitors Pleasure Business Spending Per Party Per Trip by Visitor Segment Springfield Visitor Profile Example $900 $824 $800 $700 $600 $719 $689 $674 $638 $568 $495 $476 $500 $400 $312 $300 $200 $100 $0 Families Adults 0-100 mi 300 mi 301+ mi 23 1st Time Visitors Pleasure Business Springfield Conversion Study • E-surveys can also help one understand the impact of the advertising materials sent out, how prospects made their decision to visit one destination over another, what converted visitors did while in the area, etc. • BUT, conversion studies conducted using ONLY EMAIL have additional limitations. Email surveys are great for finding out more about a destination’s visitors, brand or past visitors. But a conversion study must inherently attract an objective ratio of the number of inquirers who visited. Hence, you must also have a representative group of non-visitors as well --and that is difficult to accomplish with an e-survey. • For that reason, I recommend conducting an additional short phone survey that asks… 1) Did you visit? and 2) How much did you spend?—by medium. 24 Conversion Rate Comparison 1998-2003 Springfield Conversion Study Example 60% 50% 48.3% 42.9% 40% 44.8% 46.9% 32.7% 30% 25.4% 20% 10% 0% 1998 *Included Reader Service 1999 2000* 2001 25 2002 2003 2000-2003 Conversion Rates by Advertising Campaign Springfield Conversion Study Example 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 76.5% 51.3% 56.9% 53.1% 44.5% 56.3% 47.2% 43.4% 38.8% 32.8% Unclassified Misc/ Other MDT 2003 26 Television 2002 Show-Me Revenue Per Visitor Party by Advertising Campaign Springfield Conversion Study Example $900 $800 $700 $600 $500 $824 $806 $847 $663 $597 $840 $614 $644 $606 $582 $400 $300 $200 $100 $0 Unclassified Misc/ Other MDT 2003 Overall Revenue Per Visitor Party = $ 649 27 2002 Television Show Me Springfield Total Revenue Generated by Advertising Campaign ($000s) Springfield Conversion Study Example $10,000 $9,000 $8,000 $7,000 $6,000 $5,000 $4,000 $3,000 $2,000 $1,000 $0 $8,727.2 $8,007.5 $6,951.4 $6,166.3 $4,472.0 $4,315.5 $3,655.4 $2,281.8 $507.6 $311.6 Unclassified Misc/ Other MDT 2003 Total Revenue Generated = $21,998,462 28 2002 Television Show Me Springfield Return on Gross Investment by Advertising Campaign Springfield Conversion Study Example $500 $439.49 $400 $300 $200 $280.62 $285.97 $255.86 $196.10 $105.47 $121.96 $120.34 $100 $0 Unclassified Misc/ Other MDT 2003 2002 Overall Revenue Generated for Every Ad Dollar = $213.23 29 Television* Show Me Springfield Research Templates a.k.a. Research-in-a-Box • Some destination marketing organizations (DMOs) take responsibility for helping smaller destinations, attractions or festivals help themselves. • They are starting to create “research templates” that smaller entities can use to collect data for themselves. This is essentially a research study in-a-box. Each template includes a standard questionnaire, input file, legend, data summaries and applicable charts. • The survey can be launched, have the data collected and input by virtually anyone with a computer. Once the data is input, the template processes the data automatically and creates both a set of data tables and relevant charts. 30 Silver Dollar City Non-Visitor Study • E-surveys work great for a wide variety of types of surveys conducted among visitors. But, at Silver Dollar City we have found that even our past customers have a passion for helping us improve our products and services. Response rates among old visitors is nearly as good as it is among active customers. • Silver Dollar City has used these kinds of non-visitor assessments to determine why some people just don’t come back. These kinds of surveys work wonderful for open-ended responses giving respondents an opportunity to write exactly what’s on their minds. • Over the past couple of years we learned that our park had begun to experience, “brand drift.” We are now taking action to correct some of the issues we identified. 31 Reasons for Not Visiting Sampling of Actual Customer Responses SDC Non-Returning Visitor Study Example • • • • • • • • It has become too amusement park oriented. It used to be that the park was full of crafters, but that seems to have given way to amusement rides. We went every year for about 15 years to the Fall National Crafts Fair, but within the last 8 years or so, that has become a joke. There used to be crafters and artists lined up and down the sidewalks of the park, with the count at least at 200. Now the National Crafts Fair has pretty much fallen the same way 76 Boulevard has - very few crafts and crafters, and more commercial shops, restaurants, and hotels. It is boring and always offers the same stuff for an elevated cost It seems that it has been the same for some time. My children wanted something new for a change. It's changed---not for the better It's gotten "old". Nothing new, and we have done it so many years in a row. Less music. The combining of weeks such as music with crafts. Seems that creativity and variety have been sacrificed for moneys sake. No new roller coasters. We chose to spend our money on one long vacation instead of several quick trips to Branson 32 Rate Reasons for Not Returning to SDC Inactive Visitor Responses SDC Non-Returning Visitor Study Example Kids too old / grown 2.04 Don't think about it 2.19 Don't like park rides 2.21 Traffic Congestion 2.33 Nothing new of interest at SDC 2.39 Prefer Branson shows 2.55 No kids on last visit 2.56 Been before / Doing other things 2.62 Too expensive/ cannot afford 3.27 Have not visited Branson 0.00 3.34 0.50 1.00 33 1.50 2.00 2.50 3.00 3.50 4.00 Silver Dollar City Brand Assessment • E-surveys also work very well at helping decision makers assess the value of their brands and measure how satisfied their visitors are with those aspects of the visit that are most important to them, called consumer mapping. 34 Describe Silver Dollar City SDC Brand Assessment Example • • • • • • • • A great place for clean family fun. The only place I know that still has christian values. We LOVE that!!!! "A wholesome fun experience for your entire family!" 1800 Theme Park with some rides and musical shows. Craft work for sale A "SAFE" PLACE FOR THE ENTIRE FAMILY!!!! A christian, family oriented place that I can take anyone and have a good time. I like the fact that there are rides that aren't too wild that I (as an adult) can ride and have fun along with my children (who like the wilder rides) A family oriented park where one can feel safe. The park employees for the most part are very courteous. The food is tasty. A fun place to visit A great place for couples and families 35 “Silver Dollar City's craftsmen and entertainers seem to be more friendly today than they were a few years back” SDC Brand Assessment Example BRAND QUESTION 35% 30% 29.0% 25% 20% 20.7% 19.3% 19.2% 15% 10% 5% 0% Visitors Non-Returners % Agree % Disagree 36 “I/we are more likely to visit Silver Dollar City today than we were a few years ago” SDC Brand Assessment Example 40% BRAND QUESTION 36.9% 35% 30% 32.0% 26.9% 30.2% 29.2% 25.6% 25% 20% 15% 10% 5% 0% Visitors Non-Returners % Agree % Disagree 37 Inactives “As SDC's prices escalate I find myself choosing different entertainment options for my family” SDC Brand Assessment Example BRAND QUESTION 70% 60.9% 60% 52.7% 50% 40% 38.7% 31.9% 30% 19.2% 20% 15.8% 10% 0% Visitors Non-Returners % Agree % Disagree 38 Inactives “Silver Dollar City is more unique and attractive to my family today than it was a few years back” SDC Brand Assessment Example BRAND QUESTION 35% 30.1% 30% 29.9% 29.2% 26.9% 25% 22.1% 20% 17.3% 15% 10% 5% 0% Visitors Non-Returners % Agree % Disagree 39 Inactives “Knowing the storylines behind SDC's rides and attractions make them more enjoyable for my family” SDC Brand Assessment Example 70% BRAND QUESTION 60.8% 60% 54.9% 51.6% 50% 40% 30% 20% 18.8% 13.6% 12.5% 10% 0% Visitors Non-Returners % Agree % Disagree 40 Inactives Consumer Mapping • Consumer Mapping entails combining attribute importance and performance to define the target issues and strengths of the bank. • It provides information about the relative strengths and weaknesses among attributes of the attraction or destination only, rather than a specific destination relative to the competition. • Those attributes that most impact loyalty and whose performance is better than average are called Strengths. Those issues that are high impact where performance is lower are designated Target Issues. • Secondary Opportunities are those where the attraction or destination’s performance is lower than average but the attributes have lower impact on loyalty. • Similarly, Low Priority Issues have low impact on loyalty and are better in performance for the destination. 41 Issue Targeting Secondary Opportunities Satisfaction More Satisfied Strengths These are the "primary strengths" of the brand. These attributes are not crucial. Immediate focus should be on target issues. Customers’ needs are being met, though these attributes are not important. Potential for resource misallocation. These are "target issues" to improve customer loyalty. The brand is performing below average and these attributes are important. Less Satisfied Low Priority Less Important Target Issues Importance 42 More Important Consumer Mapping Better -EXAMPLE- Secondary Opportunities Strengths Friendliness Safety/ Security Variety Performance Number of Restaurants Ease of Getting There Value for the Dollar Scenic Beauty Low Priority Worse Less Important Target Issues Importance 43 More Important SDC Brand Assessment Example Consumer Mapping 5.00 Wholesome Atmosphere Friendly 4.50 Craft Demos Know ledgeable Emps Trams Satisfaction Wide Paths Availability of Stroller Rentals 4.00 Variety of Attractions Kid-Friendly Mdse Diaper Changing Stations Quality of Stroller Rentals Kiddie Attractions Parking Lot Exp. Adequate No. of Air Cond. Restaurants Interactive Activities Availability of 1st Aid Stations Interaction w / Characters Well-Marked Smoking Areas Stroller Maps Availiability of Kids Crafts Place to Heat Baby Bottles Quality of Restrooms Signage/ Maps Making Child Feel Special Meeting Spot for Lost Family Mbrs 3.50 Cleanliness Qty of Restrooms Benches & Places to Sit Adequate Restaurant Seating Kid-Friendly Dining Kid-Friendly Meals Healthy Food Choices Availiability of Locker Storage Availiability of Kid Height Water Fountains Availiability of Nursing Stations Price of Stroller Rentals Availiability of Family Restroom 3.00 Ease of Maneuvering Strollers Qty of Water Fountains Entertainment in Lines 2.50 1.50 2.00 2.50 3.00 3.50 Importance 44 4.00 4.50 5.00 Celebration City Concept Study • E-surveys are ideal for testing new ideas, e.g. advertising positioning, headlines, or new concepts. Perhaps the most important types of research HFE conducts by e-survey is our concept testing program. Before we add a new festival or build a new ride, we ask our customers to tell us what they think about the options we are considering. • You cannot blindly ask people what they would like to see us add because they typically don’t know or cannot verbalize it. BUT, they can review several different options and tell you which ones they like, which one’s they don’t, and which one’s would be most likely to cause them to visit. • At HFE, we’ve conducted 21 such studies and tested nearly 60 different concept ideas over the past four years. 45 Concepts’ Overall Appeal Families with Children Celebration City Concept Evaluation Example 8.0 7.0 6.8 6.9 7.1 6.9 6.0 6.6 5.4 5.0 4.0 3.0 2.0 1.0 0.0 MGM Grand Finale 10-point scale Hallmark Magic Moments Nick-at-Nite 46 ET's Light Up the Stars No Limits Pop Rocks by Coke INTENT-TO-VISIT These Concepts Families with Children Celebration City Concept Evaluation Example 8.0 7.0 6.9 7.0 7.2 7.1 7.0 No Limits Pop Rocks by Coke 5.9 6.0 5.0 4.0 3.0 2.0 1.0 0.0 MGM Grand Finale 10-point scale Hallmark Magic Moments Nick-at-Nite 47 ET's Light Up the Stars % Concepts Perceived as Being MOTIVATIONAL HOOKS Families with Children Celebration City Concept Evaluation Example 40% 35% 30% 37.3% 36.3% 29.0% 29.7% 27.0% 25% 17.7% 20% 15% 10% 5% 0% MGM Grand Finale Top-Box Ratios Hallmark Magic Moments Nick-at-Nite ET's Light Up the Stars 48 No Limits Pop Rocks by Coke Summary of Appeal and Likelihood of Visiting Concept Ratings Celebration City Concept Evaluation Example APPEAL Concept LIKELIHOOD OF VISITING Families Overall Families Overall Overall MGM 6.8 6.9 6.9 6.8 Hallmark 6.9 6.8 7.0 6.7 Nick-at-Nite 7.1 6.8 7.2 6.9 ET 5.4 5.1 5.9 5.6 No Limits 6.9 6.7 7.1 6.7 Pop Nights 6.6 6.3 7.0 6.7 49 Favorite Concept of All Families with Children Celebration City Concept Evaluation Example Favorite Concept-Group 1 Hallmark 30% Favorite Concept-Group 2 Hallmark 11% Pop Nights 24% MGM 20% No Limits 31% ET 5% Nick-atNite 38% Nick-atNite 41% 50 NET: FAVORITE Less LEAST FAVORITE Families with Children Considering the net results of “favorite” minus “least favorite” concept, this chart shows that Nick-at-Nite generated the largest margin of favorite “votes”compared to least favorite “votes.” 30% 20.7% 20% 10% 2.0% 5.8% 0% -10% -4.9% -1.8% -20% -30% -40% -40.9% -50% MGM Grand Finale *Ratio of Favorite: Least Favorite Hallmark Magic Moments Nick-at-Nite ET's Light Up the Stars 51 No Limits Pop Rocks by Coke How Can You Get Started? • Call Your IT People. I started with the idea and didn’t know anything about programming, web design, etc. • Order WebSurveyor.com or SurveyMonkey.com. It’s pretty easy to learn and they do the tough part. • Pay a Consultant. For less than a few hundred dollars you can have your survey blast to your customer list, posted on the web, and the results downloaded into an input file. For less than several thousand you can have the whole thing done for youturnkey. • Hire a Research Company. This will likely be quite a bit more expensive. For whatever reason, research companies are typically charging nearly as much for online surveys as they do for phone surveys. 52 Summary • E-Surveys are typically faster, less expensive, and not as intrusive as traditional phone surveys. While not everyone has internet access, travelers are far more likely to have access than the average U.S. citizen. Hence, they provide a very attractive alternative for most destination marketing organizations. • Response rates today rival phone surveys and because people can complete such surveys on their own time, responses tend to be more thorough. • Bottom line, e-surveys are effective and affordable. They offer DMO’s the unique opportunity to gain a better understanding of their customers without breaking the budget. • For many it is the difference between being able to conduct customer research and continuing to operate with one’s best guesses. 53