© 2005 CORNELL UNIVERSITY
DOI: 10.1177/0010880405275966
Volume 46, Number 3 344-362
10.1177/0010880405275966
Text Mining for the
Hotel Industry
by KIN-NAM LAU, KAM-HON LEE, and YING HO
With the availability of huge volumes of text-based
information freely available on the Internet, text mining can be used by hoteliers to develop competitive
and strategic intelligence. Although the software
needed to analyze online text files remains comparatively expensive, the cost will inevitably fall. A demonstration of text mining compiled information about
Hong Kong hotels and about would-be travelers. Such
relatively invariant information as statistics on competitors’ facilities is fairly easy to assemble, but variant
information, such as room rates, can be elusive. Customers’ demographics and attitudes can be mined
with reasonable accuracy from newsgroup postings
and the like.
Keywords: text mining; Hong Kong hotels; competitor intelligence; marketing intelligence
B
usiness intelligence plays an important role
when executives make strategic and operational
decisions. Accurate and timely competitor and
customer intelligence enhances hotel effectiveness
and customer satisfaction. In the era of information
explosion, hospitality practitioners are overloaded by
data. They often complain that they have too much
data but they do not have enough understanding. 1
344
Cornell Hotel and Restaurant Administration Quarterly
Hoteliers have long recognized the benefits—and
expense—of installing information technology (IT)
systems.2 One particular focus for IT has been on using
data-mining techniques to extract meaningful patterns
and build predictive customer-relationship models
from numeric data. Although widely used, data mining
is applicable only to structured, numeric databases.3
However, a majority of business information exists in
the form of unstructured or semistructured text documents, such as those stored in a hotel’s internal databases or in Web-based data sources. The traditional
way of processing text information involves human
actions in information gathering, analysis, and dissemination. This requires substantial investment of
money, time, and human resources.
Moreover, it is difficult to combine qualitative text
data with quantitative numeric data in business analyses. Therefore, there is a pressing need to develop a
method that can accurately extract business intelligence from large text collections and integrate the
fragmented information into business intelligence
databases.
This article proposes text mining as a means of
information management. Text mining can analyze the
voluminous textual information that can be found in a
hotel’s internal databases and external sources. This
AUGUST 2005
TEXT MINING FOR THE HOTEL INDUSTRY
article demonstrates an exploratory data
analysis that derives new and relevant
information from large text collections.4
Although the current state of technology
makes it unlikely that the text-mining
technique can be readily implemented,
hoteliers should be aware of its potential. Based on the demonstration study,
this article discusses potential uses and
technological limitations of text mining.
It also addresses the implementation
cost and possible future technological
advancement.
Concept of Text Mining
Similar to data mining, text mining
explores data in text files to establish valuable patterns and rules that indicate trends
and significant features about specific topics.5 Unlike data mining, text mining
works with an unstructured or semistructured collection of text documents
(e.g., corporate documents, Web pages,
newsgroup postings).6 The text-mining
process starts with a keyword search in
text collections. Current text-processing
technology allows a search technique beyond simple Boolean searches by using
natural-language queries. Since search
engines can recognize any of thousands of
keywords and phrases but not the concepts
behind the text, it is necessary for researchers to construct a dictionary that
acts as the knowledge base to associate
keywords with specific concepts.7 The
dictionary is then used to translate the
unorganized text into meaningful figures
and indexes, which may provide valuable
business intelligence to hoteliers. For
example, the dictionary may identify one
hundred keywords and phrases to be associated with the concept of “room type.” If
the keywords identified in a text document
match those specified in the dictionary, a
positive recognition of the room-type concept can be concluded. Failures in identi-
AUGUST 2005
COMMENTS
fying the keywords for a certain concept
will result in missing values or data for
8
that specific concept. Text-search results
can then be further analyzed and converted into an actionable knowledge
database.
Hoteliers may find the text-mining
technique useful in many areas of their
operations, including
1. environmental scanning of customer intelligence by analyzing customer newsgroups,
online bulletin boards, and online customer
surveys;
2. acquiring customer intelligence by analyzing personal home pages, customer comment cards, and qualitative survey data; and
3. improving efficiency of internal knowledge
management by analyzing e-mail, patent
databases, and corporate documents.
A number of powerful text-mining
products are available to analyze textual
information such as e-mails, customer
surveys, corporate documents, and patent
databases (see Exhibit 1). Industries such
as tourism, financial services, insurance
services, and manufacturing have successfully applied the text-mining technique in
their data-management functions (see
Exhibit 2). Users applaud the efficiency
of these tools in managing huge amounts
of text data. To the best of our knowledge,
the hotel industry has not yet adopted text
mining, but it may improve the industry’s
information-management efficiency.
What’s Different about
Text Mining
The text-mining concept is similar to
traditional survey research in a number of
ways. The process of constructing a keyword dictionary is similar to constructing
a questionnaire. The identification of keywords and phrases for a specific concept is
analogous to questionnaire design in
which questions are set, aiming at capturing certain concepts about the inter-
Cornell Hotel and Restaurant Administration Quarterly
345
COMMENTS
TEXT MINING FOR THE HOTEL INDUSTRY
Exhibit 1:
Text-Mining Software
Product Name
Vendor
Intelligent Miner for Text IBM
SAS Text Mine
SAS
SPSS LexiQuest Mine
GoodNews
SMART Text Miner
WordStat
TextAnalyst
SemioMap
ClearResearch
Copernic Summarizer
CATPAC
dtSearch Text
Retrival Engine
Statistical Text Mining
SPSS
Robotics Institute
SMART
Communications
Provalis Research
Megaputer Intelligence
Semio Corp.
ClearResearch Corp.
Copernic Technologies
Terra Research &
Computing, Inc.
dtSearch Corp.
Enkata Technologies
ExactAnswer
INFACT
ISYS search solution
Klarity
Leximancer
Lextek
Matchpoint
InsightSoft-M
Insightful Corp.
Odyssey Development
Intology
Leximancer
Lextek International
Triplehop Technologies
MindServer
Recommind Inc.
Online Miner
TextQuest
Temis Group
Social Science
Consulting
Processor Information
Technology
Xanalys
Search Technology
Readware Information
Management
Xanalys Indexer
VantagePoint
VisualText
Free and shareware
INTEXT
Spy-EM
Text Analysis
International, Inc.
Social Science
Consulting
University of Illinois
URL
http://www-3.ibm.com/software/data/iminer/fortext
www.sas.com/technologies/analytics/datamining/
textminer/
www.spss.com/lexiquest/lexiquest_mine.htm
www-2.cs.cmu.edu/~softagents/text_miner.html
www.smartny.com/miner.htm
www.simstat.com/
www.megaputer.com/products/tm.php3
www.informationweek.com/683/83iumin.htm
www.clearforest.com/Products/Tags.asp
www.copernic.com/en/products/summarizer/
www.pbelisle.com/library/reviews/catpac6.htm
www.dtsearch.com/PLF_Features_2.html
www.enkata.com/products/statistical_text_
mining.html
www.insight.com.ru/products.html#3
www.insightful.com/products/infact/default.asp
www.isysusa.com/products/index.html
www.intology.com.au/20products/
www.leximancer.com/overview.html
www.lextek.com/onix/
www.triplehop.com/product_demos/
matchpoint.html
www.recommind.com/english/solutions/
default.asp?url=Products
www.temis-group.com/
www.textquest.de/eindex.html
www.readware.com/prod_infoproc.asp
www.xanalys.com/solutions/indexer.html
www.thevantagepoint.com/pages/
whitesheet_1.html
www.textanalysis.com/Products/products.html
www.intext.de/eindex.html
www.cs.uic.edu/~liub/S-EM/S-EM-download.html
Source: www.google.com.hk; www.kdnuggets.com/software/text.html.
346
Cornell Hotel and Restaurant Administration Quarterly
AUGUST 2005
TEXT MINING FOR THE HOTEL INDUSTRY
COMMENTS
Exhibit 2:
Business Applications of Text-Mining Tools
1. Tourism
a
Example: regional tourism board of Piedmont region (northwest Italy)
The project aims at accessing textual data from guidebooks for use in marketing
campaigns. The project’s goals are to build a database of restaurants, to compare Piedmont with the other Italian regions in terms of number of high-quality
restaurants, to study customer preferences, and to get an estimate of restaurant
patronage during different periods of the year. The text-mining tool converts
unstructured data into structured data, reduces the dimensionality of data while
keeping relevant information, and jointly analyzes quantitative and qualitative
data.
2. Financial services
b
Example: The Principal Financial Group
The Principal Financial Group implemented a document management and presentation application that allows company employees to use their Web browsers to access current customer statements on its intranet. The text-mining tools
provide the company with a high-performance document storage, retrieval, and
presentation solution. As a result, customers get questions answered immediately because finding the right statement is fast and convenient for service
representatives.
3. Insurance
c
Example: Fireman’s Fund Insurance Company
The Fireman’s Fund faces a number of mundane threats: claims fraud, rising
workers’ compensation costs, unnecessary payouts, and intense competition.
To tackle these issues, the fund uses text-mining software to systematically
review and analyze diverse claims databases. Special checks and flags alert the
company to possible fraud or to opportunity for subrogation. The fund built predictive models that identify useful patterns in the data and use text mining to
spot words associated with fraud.
4. Manufacturing
d
Example: Hewlett-Packard (HP)
Hewlett-Packard wants to make better decisions based on what customers want
and expect in the products and services they need. They use text-mining tools to
analyze e-mail, customer surveys, warranty claims forms, and feedback. Textmining tools enable them to (1) uncover underlying themes contained in large
document collections, (2) automatically group documents into topical clusters,
(3) classify documents into predefined categories, (4) integrate text data with
structured data to enrich predictive modeling endeavors, and (5) base business
decisions on insightful customer information.
a. Source: www.sas.com/success/agenziaturistica.html.
b. Source: www-306.ibm.com/software/success/cssdb.nsf/csp/navo-4vpvek.
c. Source: www.sas.com/success/firemans.html.
d. Source: www.sas.com/success/hp.html.
AUGUST 2005
Cornell Hotel and Restaurant Administration Quarterly
347
COMMENTS
TEXT MINING FOR THE HOTEL INDUSTRY
viewee. In the process of text search,
the search engine is the “interviewer,”
while the individual text document is the
“interviewee.”9
Although the two approaches are similar, they do differ in the following ways.
First, traditional research seeks to understand the target population through statistical inference from a representative sample. In comparison, the objective of text
mining is to study the whole population
instead of just a sample. Second, survey
questions in marketing research are developed according to the researchers’specific
interests, usually based on a theoretical
framework, and respondents then provide
answers to the questions. In comparison,
data in the text-mining approach are selfrevealed according to the owners’ preferences, and researchers set inquiries without regard to what kinds of information
are available. Finally, while we have
humans as interviewers and interviewees
in survey research, the “interviewers” and
the “interviewees” in text-mining research
are, as we said, the search engines and the
individual text documents.
Online Text Mining
Online text mining, searching through
the volumes of material available on the
Internet, is gaining increasing attention
from both researchers and practitioners.10
The Internet generates and stores considerable amounts of business information in
online databases, such as company Web
sites, customer newsgroups, and online
focus groups. For instance, knowledge
about potential customers (e.g., gender,
age, marital status, interests, and hobbies)
is available on personal home pages,
which can be converted into consumer
intelligence databases for marketing purposes.11 Discussions in newsgroups and
online bulletin boards may serve as abun-
348
Cornell Hotel and Restaurant Administration Quarterly
dant sources of market intelligence (e.g.,
consumer preferences, evaluation of existing services, and customer complaints).
Key “firmographics” (e.g., number of
hotel rooms, price plans, services, and
facilities) can be extracted from hotel Web
sites, which help managers to better understand business partners and competitors. In sum, online text data represent a
rich reservoir of business intelligence that
is potentially valuable to hoteliers.
Online text mining involves the following four steps:
1. Definition of mining context and concepts.
It is crucial at the beginning to identify the
types of information being sought. Key
aspects of information (labeled “concepts”)
in the hotel industry may be room price,
hotel facilities and services, and customers’
preferences and concerns.
2. Data collection. Hoteliers need to establish
which text documents will be analyzed. For
example, if they intend to analyze Web
pages, they may send Web crawlers to a
user-defined set of URLs or a Web space
to collect data from text and HTML files.
They then store metadata collected from
target Web sites in a database for further
analysis.
3. Dictionary construction. As we explained
above, researchers must construct a dictionary to associate search terms with specific
concepts.
4. Data analysis. With the help of the dictionary, researchers translate unorganized text
into meaningful figures and indexes, which
can be used for data-mining purposes. In
this way, the qualitative text data can be
combined with quantitative data for more
comprehensive business analyses. Computer programs then interpret results for
managerial decision making.
Since a large proportion of competitive
intelligence and customer intelligence
resides in online databases, we illustrate
how hoteliers may use the text-mining
technique to convert Web data into actionable competitive intelligence and
customer-intelligence databases.
AUGUST 2005
TEXT MINING FOR THE HOTEL INDUSTRY
Demonstrating the
Potential of Text Mining
We carried out three studies to illustrate
the potential use of the text-mining technique in the hotel industry. We conducted
a hotel-profile analysis (Study 1) and a
room-price analysis (Study 2) to show
how hoteliers may obtain invariant and
variant competitive intelligence from
hotel Web sites. In addition, we studied
travel-related newsgroups (Study 3) to
illustrate how hoteliers may extract customer intelligence from online discussion
groups. The illustrations do not imply that
the text-mining technology is readily applicable in the hotel industry at the moment. The purpose of these illustrations is
to show what might be possible with the
text-mining technique. We used one of the
well-known text-mining products as the
primary tool for text analysis, supplemented by some computer programs.12
Competitive Intelligence
In this illustration, we used the textmining technique to analyze hotel Web
sites for competitive intelligence. In
August 2002, we gathered Web pages of
seventy-four member hotels of the Hong
Kong Hotels Association in an HTML
document.13 After preliminary screening,
we removed seven hotels from further
analysis because they contained mostly
nontext files (e.g., flash and jpeg), which
cannot be analyzed by our text-mining
tool. As a result, a total of sixty-seven
hotels were retained for text-mining
analyses.
The first step of text mining is defining
the research context and concepts. As we
indicated above, competitive intelligence
in the hotel industry can be classified as
invariant and variant. Invariant competitive intelligence remains unchanged most
of the time and thus may not require close
monitoring for day-to-day changes. Ex-
AUGUST 2005
COMMENTS
amples include such attributes of a hotel’s
profile as room type, food, in-room amenities, business services, beauty services,
health and sports facilities, other services,
affiliated hotels, and contact information.
In comparison, variant competitive intelligence is frequently updated. Changes in
variant competitive intelligence can be
important indications of competitive
moves and should be closely monitored.
Examples include room price and promotion packages.
It is important to note that text-mining
results should accurately represent the
true meaning of the text documents being
analyzed. Text-mining success not only
hinges on a search engine’s effectiveness
but also requires the researcher to interpret
the results. This is particularly true for
words that may express entirely different
meanings (e.g., the word “deluxe” may
represent a room type or an adjective complementing hotel service) and for different
expressions that may convey the same
meaning (e.g., airport coach, shuttle service). Therefore, text-mining accuracy
should be defined as how well the textmining result is a true representation of
the text collection. To evaluate the accuracy of the text-mining result, we compared the computer’s output with the
human analyst’s results and calculated the
hit rate for each concept. We define hitrate percentage as the number of truly represented cases compared to all analyzed
cases, as follows:
Hit Rate (%) = (Correctly Classified Cases ÷ Total
Number of Cases) × 100%.
In the following section, we explain
dictionary construction and data analysis
in examining invariant competitive intelligence (i.e., hotel profile) and variant competitive intelligence (i.e., room price).
Cornell Hotel and Restaurant Administration Quarterly
349
COMMENTS
TEXT MINING FOR THE HOTEL INDUSTRY
Study 1: A Hotel Profile
(Invariant Information)
This study aims at constructing a database that contains profile information of
Hong Kong hotels (e.g., services and facilities, location, contact method). A hotel
profile database may be useful to hoteliers
in identifying opportunities and threats in
current and prospective markets. In the
dictionary-construction stage, we focused
on major hotel services and facilities, including the following examples:14
1. room type—single room, double room, triple room, suite;
2. food-service concepts—Chinese cuisine,
Indian food, buffet, seafood, vegetarian;
3. in-room amenities—in-house movie, inroom broadband, in-house play station;
4. business services—meeting room, secretarial service, translation service;
5. beauty services—body treatment, facial
treatment, solarium, sauna;
6. health and sport facilities—health center,
aerobic room, gymnasium; and
7. other services—shuttle service, handicap
facilities, helicopter service.
We also identified a property’s affiliated
hotels and contact information. Exhibit 3
shows examples of keywords used in analyzing room type and food concepts. The
hotel profile dictionary includes 18,340
search terms.
Exhibit 4 shows that room type and
food information is found in almost all of
the hotels we surveyed, although some
hotels contain missing values for certain
concepts. The problem of missing values
is most noticeable for beauty services and
affiliated hotels, where missing values
exceed 37 percent. Since hotel home
pages are mainly for promotional purposes, it is reasonable to assume that hoteliers are willing to disclose as much information as possible about their services and
facilities. Therefore, missing values for
some concepts probably mean that the
hotels do not offer these services or facili-
350
Cornell Hotel and Restaurant Administration Quarterly
ties. A further investigation into three concepts, in-room amenities, health and
sports facilities, and other services,
confirmed this rationale.
We calculated hit rates to evaluate the
accuracy of the text-mining technique.
Exhibit 5 shows the hit rates for the roomtype and food concepts. All concepts
except in-room amenities and contact
information have hit rates above 90 percent. Results for those two concepts are
less than satisfactory, with some subconcepts attaining hit rates as low as 70
percent. These results suggest that the
text-mining technique can generate reasonable accuracy for a hotel profile
analysis.
Two factors affect the levels of missing
values and hit rates. First, the dictionary
could be made more exhaustive with the
addition of more keywords. Taking food
as an example, the dictionary includes
only such terms as Chinese food and Thai
cuisine. A more comprehensive dictionary might include the names of popular
regional dishes, such as U.S. roast beef
and Italian cold-cut platter. Second, technical limitations exist for context-based
query and Web-table analysis.15 For example, our text-mining tool cannot recognize
sentence breaks except by conventional
punctuation, namely, periods, question
marks, or exclamation marks. Since much
information in hotel Web sites is displayed
in tables and lists that do not include punctuation, some data are overlooked, bringing in inaccurate results.
Study 2: Room Prices
(Variant Information)
In this section, we used the same data
set as in the hotel profile analysis to analyze the hotels’ pricing schedules. The
purpose of Study 2 is to construct a database containing room-price information
for Hong Kong hotels. We acknowledge
AUGUST 2005
TEXT MINING FOR THE HOTEL INDUSTRY
COMMENTS
Exhibit 3:
Keyword Examples of Hotel Profile Analysis
Room type
Single room
E.g., single room,a guestrooma single, single guest room,a standard single, single moderate, economy single, single executive, superior single,
single deluxe, suite single, single studio, occupancy single, single # of,
etc.
(58)
Double room
a
a
E.g., double/twin room, guestroom double/twin, double/twin guest
a
room, standard double/twin, double/twin moderate, economy double/
twin, double/twin executive, superior double/twin, double/twin deluxe,
suite double/twin, double/twin studio, occupancy double/twin, double/
twin # of, etc.
(103)
Triple room
E.g., triple room,a guestrooma triple, triple guest room,a standard triple,
triple moderate, economy triple, triple executive, superior triple, triple
deluxe, suite triple, triple studio, occupancy triple, triple # of, etc.
(57)
Guestroom
Room,a guest room,a guest-room,a guestrooma
(8)
Suite
Suitea
(2)
Food
Chinese cuisine
E.g., Anhui afternoon tea, Bao Tou beverage,a Beijing breakfast, Cantonese buffet,a dim-sum, yum cha, drink tea, Chinese tea, Hainese
chicken rice, dim sim, etc.
(3,933)
Indian cuisine
E.g., Indian breakfast, Indian buffet,a Indian catering, Indian catering
menu, Indian curry,a Indian bread, etc.
(54)
Japanese cuisine
E.g., Japanese seasonal specialties, Japanese set dinner,a Japanese
set lunch,a Sushi, sashimi, teppanyaki, shabu shabu, etc.
(55)
Singaporean cuisine
E.g., Singaporean afternoon tea, Singaporean beverage,a Singaporean
breakfast, Singapore noodles, Singaporean Satay, etc.
(53)
Thai cuisine
E.g., Thai seasonal specialties, Thai set dinner,a Thai set lunch,a Thai
style,a etc.
(51)
Italian cuisine
E.g., Italian savory, Bolognese seasonal recipes, Florentine seasonal
specialties, etc.
(306)
(continued)
AUGUST 2005
Cornell Hotel and Restaurant Administration Quarterly
351
COMMENTS
TEXT MINING FOR THE HOTEL INDUSTRY
Exhibit 3:
(Continued)
Portuguese cuisine
E.g., Portuguese afternoon tea, Portugal beverage,a Portuguese breakfast, etc.
Bar
Bar,a pub,a café,a cafe,a lounge,a coffee shopa
(102)
(12)
Buffet
Buffeta
(2)
Seafood
Seafood
(1)
Vegetarian food
Vegetarian
(1)
Note: The numbers in parentheses indicate the total number of keywords used during text search.
a. Different forms of the keyword (e.g., plural, past tense, adjective, etc.) are included during the search.
Exhibit 4:
Text-Mining Result of Hotel Profile Analysis
Concept
Room type
Single room
Double room
Triple room
Guestroom
Suite
Missing
Food
Chinese cuisine
Indian cuisine
Japanese cuisine
Singaporean cuisine
Thai cuisine
French cuisine
Italian cuisine
Portuguese cuisine
Bar
Buffet
Seafood
Vegetarian food
Missing
Frequency Count
Percentage of the Total
46a
49
5
67
63
0
69.00
73.00
7.00
100.00
94.00
0.00
42
1
20
2
2
8
9
1
64
52
26
63.00
1.49
30.00
3.00
3.00
12.00
13.00
1.49
96.00
78.00
39.00
46.00
1.49
1
a. In this instance, 46 means that single-room keywords are found in 46 hotel home pages.
352
Cornell Hotel and Restaurant Administration Quarterly
AUGUST 2005
TEXT MINING FOR THE HOTEL INDUSTRY
Exhibit 5:
Validation Result of Hotel Profile
Analysis
Concept
Room type
Single room
Double room
Triple room
Guestroom
Suite
Food
Chinese cuisine
Indian cuisine
Japanese cuisine
Singaporean cuisine
Thai cuisine
French cuisine
Italian cuisine
Portuguese cuisine
Bar
Buffet
Seafood
Vegetarian food
Hit Rate (%)
95.50
97.00
97.00
100.00
100.00
92.50
100.00
100.00
100.00
100.00
95.50
95.50
100.00
100.00
100.00
100.00
97.00
that hotel room prices may change continually in response to market supply and
demand. For simplification, we adopted a
static view of a hotel’s pricing schedule.
Current text-mining tools are unable to
search for room price, which can be any
number within a continuous range. We
used a supplementary program to identify
price information on hotel Web sites.
Before actual analysis, we processed Web
pages by
1. reformatting the source file of a home page
into a tag-free single-line sentence,
2. removing Chinese characters, and
3. converting prices given in U.S. dollars into
HK dollars.
We constructed 158 search terms for
the price study. Exhibit 6 provides examples of keywords used. As a simple illus-
AUGUST 2005
COMMENTS
tration, we define normal price as a room
price without additional date or promotion
16
information. The program remembered
positions of (1) room-type keywords and
(2) numbers, with or without a dollar sign.
It filtered irrelevant prices and numbers
such as restaurant meal prices and international direct dialing (IDD) cost. It also
excluded room prices with date or promotion keywords. We then matched the
room-type keyword with the nearest number following it. Proximity between a keyword and its corresponding number could
not exceed ten words. Program output was
a price range if we identified several prices
for the same room type. A sample output
of price program is as follows:
Studio
Standard
Superior
Deluxe
Suite
Extra bed
HK$2,080
HK$880
HK$1,180
HK$1,480
HK$2,380-$2,880
HK$400
Results reveal that we successfully
identified 83 percent of the normal price
information provided in hotel Web sites.
We checked the accuracy of identified
room types and their corresponding price
ranges. In identifying room types, all
room types have hit rates above 92 percent. In identifying price ranges, we
achieve hit rates above 90 percent for
practically all room types. The only
exception is harbor-view rooms, which
achieved a less satisfactory hit rate of 67
percent.
Study 2 focuses on stated room prices
at a particular time. It is important to note,
however, that the text-mining approach
generates most value in a longitudinal
analysis of price information because
changes in online price information signal
competitive pricing tactics. One can also
detect changes in other Web information
Cornell Hotel and Restaurant Administration Quarterly
353
COMMENTS
TEXT MINING FOR THE HOTEL INDUSTRY
Exhibit 6:
Keyword Examples of Hotel Price Analysis
Room type
Economy, Saver, Moderate, Standard, Regular, Studio, Premier, Superior, Deluxe, Grandluxe, Executive, Suite, Harbour, Triple, Extra, Third
(16)
Irrelevant price
Wedding, Meeting, Breakfast, Lunch, Dinner, Restaurant, Restaurants,
Food, Beverage, Table, Dining, Facial, Spa, Equipment, Transport, Transportation, Benefits, Supplement, Days, Nights, Weekly, Monthly, Month,
Week, Package, Packages, Long, Promotion, Promotions, Weekend,
Special
(31)
Irrelevant number
Office, Inc, Inc., Ltd, Ltd., Company, Mr., Mrs., Miss., Ms., Dr., Jr., Mr,
Mrs, Miss, Ms, Dr, Jr, Street, Road, Avenue, Total, Rooms, Guestrooms,
Years, Early, Wedding, Meeting, Breakfast, Lunch, Dinner, sq., sq.ft, min,
hrs, mins, volts
(74)
Excluded dates
January, Jan, February, Feb, March, Mar, April, Apr, May, June, Jun,
July, Jul, August, Aug, September, Sep, October, Oct, November, Nov,
December, Dec
(23)
Excluded promotion
Supplement, Days, Nights, Weekly, Monthly, Month, Week, Package,
Packages, Long, Promotion, Promotions, Weekend, Special
(14)
Note: The numbers in parentheses indicate the total number of keywords used during text search.
(e.g., new facilities, new promotional
plans) by identifying deviations from previous text-mining results. With changes
detected, hoteliers may assign management personnel to investigate the situation
further. In this way, the text-mining approach enables hoteliers to scan their industry environment and screen for critical
business intelligence. Hotels constantly
update their home pages with information
such as seasonal packages, promotional
campaigns, and new services. Therefore,
it is appropriate to monitor changes in various hotel Web sites.
Customer Intelligence
By analyzing newsgroup postings,
hoteliers may obtain up-to-date knowl-
354
Cornell Hotel and Restaurant Administration Quarterly
edge of potential travelers (e.g., their destination preference, information needs,
safety concerns). This information helps
hoteliers to understand the needs of
their potential customers, thus gaining a
competitive edge in running a lodging
business.
Study 3: Travel-Related
Newsgroups (Customer
Intelligence)
Travelers’ demographic profiles, preferences, and interests are valuable information for hoteliers. Customer intelligence is particularly crucial in case of
unusual occurrences, such as the SARS
outbreak in 2003. In the wake of Hong
Kong’s SARS outbreak, average occu-
AUGUST 2005
TEXT MINING FOR THE HOTEL INDUSTRY
pancy rates across all categories of hotels
and tourist guesthouses dropped by about
65 percent.17 To overcome the crisis, Hong
Kong hotel operators needed to apprehend
and react to the concerns of potential
inbound travelers. Checking travel-related
newsgroups would facilitate such data
gathering. Already, a number of studies
recognize the importance of generating
intelligence from newsgroup postings.18
Study 3 examined newsgroup postings
related to traveling in Europe. The objective is to understand the demographic profile, primary interests, and concerns of
potential travelers. In addition, we explored the association between a person’s
travel pattern and his or her demographic
characteristics.
We collected 4,393 newsgroup articles
that were posted from March 29 through
April 24, 2002. We examined the following discussion topics (that is, concepts):
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
Transportation
Accommodation
Price
Reservation
Ticketing
Currency
Food
Concerns—safety, regulations
Activities—water activities, night activities, eco-tourism, cultural and heritage
Destination—Eastern Europe, Southern
Europe, Western Europe, Northern Europe,
Central Europe
Sender’s gender
Sender’s country of origin
Sender’s marital status
Travel with or without children
Exhibit 7 provides examples of keywords and phrases that we used to build a
travel-related dictionary to translate unorganized text in newsgroup postings into
meaningful figures and indexes. This dictionary includes 6,657 travel-related
keywords for the analysis of newsgroup
postings. Text-search results were trans-
AUGUST 2005
COMMENTS
lated into a database containing key characteristics of each message sender.
After textual information was converted into figures and indexes, we used
chi-square analysis to identify possible
associations between concepts. We explored how gender may affect tourists’
travel patterns (e.g., activities and destination choice). We also investigated how
travelers’ accommodation concerns are
associated with their demographic characteristics. Results for gender, for instance,
indicate that female travelers tend to be
more interested in night-life activities,
sightseeing, and cultural tourism. Moreover, female travelers are more interested
in Southern Europe than they are in other
areas. As for accommodation issues, travelers from North America are more concerned about the quality and nature of
their accommodations, whereas those
from Western Europe are less concerned
about those matters. Travelers going to
Eastern, Southern, and Central Europe
also show more concerns about what
accommodations they will find. This
intelligence may help hoteliers to identify
what information travelers might want.
Newsgroup postings are typically
short, concise messages about a single
particular topic. As different messages
cover different issues, it is reasonable to
expect a large missing percentage for each
concept. Results show that the most commonly identified concepts are destination
and gender, which have missing value of
less than 41 percent. Reservations, activities, ticketing, marital status, and travel
with or without children are less often
mentioned by newsgroup participants.
Missing values of these concepts exceed
90 percent. Missing percentage of the
remaining concepts ranges from 65 percent for transportation to 89 percent for
currency. Exhibit 8 shows the missing
Cornell Hotel and Restaurant Administration Quarterly
355
COMMENTS
TEXT MINING FOR THE HOTEL INDUSTRY
Exhibit 7:
Keyword Examples of Newsgroup Analysis
Accommodation
Apartment, apartments, studio, studios, accommodation, accommodations, camp site, campe sites, campsite, campsites, where to stay, hostel, hostels, hostal, hostals, hotel, hotels, B&B, B and B, double room,
single room, private room, private rooms, twin room, twin rooms,
shower, double bed, double beds, queen-sized bed, queen-sized beds,
king-sized bed, king-sized beds, en-suite, ensuite, suite, suites, bath,
bathing, place to stay, places to stay, bed, beds, guest house, guest
houses, lodge, lodging, etc.
(49)
Concerns
Safety
Safe, safety, drug, drugs, mugged, stolen, steal, stole, pickpocket, pickpockets, pick-pocket, pick-pockets, crime, crimes, criminal, criminals,
security, disease, diseases, violence, violent, theft, thefts, die, kill, robbery, robberies, burglary, burglaries
(29)
Regulation
Custom, customs, duty, duties, passport, passports, entry visa, stay visa,
entry visas, stay visas, immigration, immigrations, overweight, regulation, regulations, declare, declares, declared, declaring, permit, permits
(21)
Phone
Calling card, calling cards, phone, phones, cellular, cellulars, dials,
dialed, dialing, dialled, dialling, roaming, GSM, GPRS, CDMA, TDMA,
SIM card, SIM cards, local call, local calls, incoming call, incoming calls,
outgoing call, outgoing calls, receive call, received call, receive calls,
received calls, receiving call, receiving calls, telecom, telecomm, telephone, telephones, celphone, celphones, cellphone, cellphones, triband, tri band, dual-band, dual band, telecommunication,
telecommunications
(44)
Rental
Rental, rentals, rent, rents, renting, rented, lease, leases, leasing, leased,
hire, hires, hiring, hired
(14)
Note: The numbers in parentheses indicate the total number of keywords used during text search.
percentages of accommodation and concerns concepts.
In terms of text-mining accuracy, all
concepts except gender attained hit rates
above 94 percent. Gender has a comparatively less satisfactory hit rate of 83 percent. This is mainly due to the difficulty
for the computer to distinguish between
356
Cornell Hotel and Restaurant Administration Quarterly
message texts of the current sender and the
previous sender(s). Exhibit 9 shows the hit
rates for accommodations and concerns.
As newsgroup contents change rapidly
over time, it is worthwhile for hoteliers to
keep track of changes in the topics being
discussed. Hoteliers may repeatedly apply
the text-mining process on updated news-
AUGUST 2005
TEXT MINING FOR THE HOTEL INDUSTRY
COMMENTS
Exhibit 8:
Text-Mining Result of Newsgroup Analysis
Concept
Frequency Count
Percentage of Total
688
3,705
15.66
84.34
215
210
175
220
3,665
4.89
4.78
3.98
5.01
83.43
Accommodation
Missing
Concerns
Safety
Regulation
Phone
Rental
Missing
Exhibit 9:
Validation Result of Newsgroup
Analysis
Concept
Accommodation
Concerns
Safety
Regulation
Phone
Rental
Hit Rate (%)
95.82
99.40
99.80
99.80
99.60
group data sets over time so that any significant modification in customer intelligence can be detected in a timely manner.
Implementation Costs
Implementation costs of the textmining approach may be a major concern
for hoteliers. Establishing and maintaining such a system incurs the fixed cost of
purchasing a text-mining tool and the variable cost of dictionary construction, program customization, and maintenance.
The prices of text-mining tools vary with
different vendors and functionality.19 Variable costs are driven by human labor to
build the dictionary and execute and maintain text-mining programs. This labor cost
escalates with increasing complexity and
execution period of the project.
AUGUST 2005
As a simple illustration, this study used
an IBM text-mining tool that costs about
20
US$91,000. We employed a research assistant and some undergraduate students
for dictionary design, program customization, and result validation, for a labor cost
of about US$14,000. Thus, the total project cost is roughly US$105,000—a substantial number. We speculate that the
fixed cost of text-mining software will
drop substantially in the near future. Indeed, in the two years since we acquired
the text-mining software, its price has
declined 29 percent.21 With increasing
need for text analysis and thus more demand for text-mining products, software
vendors will devote more resources in
developing text-mining software, eventually lowering its price.22
We acknowledge that at the current
stage of development, manual search
methods may be more cost-efficient for
small projects. However, text-mining
efforts create most value when firms adopt
a comprehensive view of their market
environment. As an example, in developing competitive intelligence, it is beneficial to consider indirect and potential
competitors, as well as one’s direct rivals.
Future threats may also materialize as new
markets open up. For example, Hong
Kong’s government recently minimized
Cornell Hotel and Restaurant Administration Quarterly
357
COMMENTS
TEXT MINING FOR THE HOTEL INDUSTRY
restrictions on self-guided tourists from
Mainland China. An influx of Chinese
tourists to Hong Kong induces competition between medium-price hotels and
budget guesthouses. With improving crossborder transportation and visa control,
mainland tourists might consider staying
outside Hong Kong (e.g., in Shenzhen,
Macau, or Zhuhai) while shopping daily
in Hong Kong. There are 122 hotels in
Shenzen, 71 in Macau, and 75 in Zhuhai.23
These 268 hotels are potential competitors
of those Hong Kong hotels serving mainly
tourists from Mainland China.
In the past, hotels’competitive analyses
focused primarily on monitoring and analyzing their direct competitors, most
likely because they lacked sufficient time
and human resources to analyze indirect
competitive challenges. Moreover, qualitative results from conventional customerfeedback channels (e.g., comment cards
and focus groups) are not combined with
other quantitative data when conducting
business analysis. When we broaden our
scope of analysis, we face voluminous
textual information that is impossible to
handle manually. Therefore, we need a
cost-efficient approach to extract timely
and accurate intelligence from vast
amounts of textual data.
To evaluate the cost efficiency of the
proposed text-mining approach, we need
to compare its costs and benefits. The
fixed cost can be considerable, but it is a
one-time investment. Once the textmining tool is installed, the continuing
variable costs of maintaining and using
the system are relatively small. On the
other hand, benefits of this system can be
substantial. For analyzing large quantities
of posted material, this approach excels in
saving time and cost. Text-mining tools
can process thousands of online documents in seconds. This greatly reduces the
time required in document collection, data
358
Cornell Hotel and Restaurant Administration Quarterly
analysis, and result reporting. In addition,
as human intervention is minimized, a
text-mining approach reduces the possibility of human errors in handling huge
amounts of data. This study demonstrates
that text-mining tools can attain satisfactory accuracy in analyzing Web contents.
The proposed text-mining approach is
cost-efficient because it converts voluminous market information into valuable intelligence in an accurate manner.
Current Limitations
Although text mining is still in its
infancy, this article shows that the technique holds great potential for the hotel
industry. We encountered different levels
of difficulty for different types of text collection. Analyzing newsgroup postings is
comparatively simpler than mining hotel
Web sites, but the concise newsgroup
postings disclose only limited amounts of
information compared to the hotels’ Web
pages. Mining hotel home pages is relatively easier than analyzing personal
home pages because data availability and
accuracy are rarely concerns for hotel Web
sites.24 As the primary objective of maintaining hotel Web sites is promotional,
hoteliers typically disclose rich and updated information in company Web sites.
Second, company Web pages are more
standardized in their expression of services and facilities (i.e., variations in sentence patterns and word usage is minimal).
Therefore, interpretation of text-mining
results can be accurate and be in the right
context.
The idea of text mining is not without
limitations. There are a number of potential concerns for applying the text-mining
technique in the hotel industry at the current stage of technological development.
Image files. As a matter of graphic design, hotel Web sites often contain graphic
AUGUST 2005
TEXT MINING FOR THE HOTEL INDUSTRY
files, the text in which current text-mining
technology cannot accurately analyze.
Technology is improving, however, and
researchers can now search and retrieve
useful information from images and videos.25 Future research may focus on how to
integrate text-mining and image- and
video-processing technologies.
Dynamic Web sites. Some Web sites
require Web surfers to input their queries
before displaying information. For instance, one may need to specify room
type, date of arrival, and length of stay
before obtaining a room price. In other
words, although search engines may point
to the dynamic databases (i.e., libraries),
they cannot search the contents therein.26
Nevertheless, crawler technology is showing promising improvements. According
to representatives from IBM Hong Kong,
we can improve on the Web-crawling
technology by constructing an applicationspecific solution to build a private copy, or
cache, from library content and functionally integrate it with data from static
online databases. The question of how to
integrate static and dynamic databases is
an interesting topic for future research.
Context-based analysis. Acquiring
accurate intelligence requires interpretation of search terms in the context that the
researcher has defined. Although textmining tools allow context-enhanced
searches that improve interpretation accuracy, the technical limitations of these
tools (e.g., with regard to Web tables) lead
to missing values and inaccurate results.
Nonetheless, computer technology is continually improving, and computer tools
are able to analyze structural aspects of a
Web table and extract attribute-value pairs
from it.27 This technique may solve the
table-mining limitation that we faced in
AUGUST 2005
COMMENTS
the current study. With the proliferation of
online materials and increasing demand
for text analysis, we expect continued
improvements in the capacity of textmining tools.
Region-specific dictionary. Dictionary
design has to be context-specific to obtain
accurate text-mining results. The dictionary used in this study is region-specific.
Some search terms are tailor-made for
Hong Kong (e.g., Hong Kong, Kowloon,
and New Territories). Therefore, modifications of the current dictionary are necessary if future research is to focus on the
hotel industry in other areas.
Conclusion
Text mining provides an alternative to
traditional human analysis of textual information. It may effectively reduce the
use of manual labor in identification, storage, and analysis of business intelligence.
In this article, we use three studies to illustrate how the text-mining technique may
help to translate online textual information into meaningful competitive and customer intelligence for managerial decision making.
In particular, hoteliers should pay
attention to Web mining, which involves
using Web crawlers for text mining. Web
mining is a much wider field than text
mining because it also involves other
elements such as multimedia and ecommerce data.28
Although the text-mining approach has
significant business implications, it is
appropriate to acknowledge the technical
difficulty as well as the complexity and
uncertainty of future technological development. First, existing text-mining tools
are not mature enough to accurately analyze dynamic Web sites and image and
animation files. Future research may ex-
Cornell Hotel and Restaurant Administration Quarterly
359
COMMENTS
TEXT MINING FOR THE HOTEL INDUSTRY
plore ways to integrate text-mining tools
with related technologies such as image
recognition and Web-table mining. Second, the proposed text-mining approach is
a hybrid method that combines efforts of
computer programs (profile and price
identification) and manual labor (Web
page preprocessing). The ideal option is a
fully automated system. Although a
hybrid data-collection model may not be
as cost-effective as a fully automated system, it is however definitely better than
doing everything manually, which is
almost impossible. Third, implementation
costs of the text-mining technique can be
substantial. Nevertheless, this amount is
r e l a t ive t o t h e cu r r e n t s t a g e o f
technological development.
The value of text mining is well
regarded by both academics and industry
practitioners because of the existence of
huge collections of untapped text documents. On the application side, wellknown software vendors such as SAS,
SPSS, and IBM have devoted considerable resources to developing and customizing text-mining software for industry
uses. On the technological side, computer
researchers are actively exploring solutions for the current limitations of textmining tools, for instance, by developing
ways to integrate image and voice recognition technologies with text mining. With
heightened interests in this area, we expect
that the technical limitations can be
resolved in the near future. We are also
optimistic that the current manual process
can gradually be automated, computing
time and capacity can gradually be improved, and operation costs can be lowered over time. Although the text-mining
technology may not be immediately applicable in the hotel industry now, it holds
great potential in business intelligence
management in the future.
360
Cornell Hotel and Restaurant Administration Quarterly
Endnotes
1. Cathy A. Enz, “Information + Analysis = Understanding,” Cornell Hotel and Restaurant
Administration Quarterly 42, no. 3 (June
2001): 3.
2. Hubert B. Van Hoof, Marja J. Verbeeten, and
Thomas E. Combrink, “Information Technology Revisited—International Lodging-Industry Technology Needs and Perceptions: A
Comparative Study,” Cornell Hotel and Restaurant Administration Quarterly 37, no. 6
(December 1996): 86-91.
3. Vincent P. Magnini, Earl D. Honeycutt Jr., and
Sharon K. Hodge, “Data Mining for Hotel
Firms: Use and Limitations,” Cornell Hotel
and Restaurant Administration Quarterly 44,
no. 2 (April 2003): 94-105.
4. Marti A. Hearst, “Untangling Text Data
Mining,” in Proceedings of ACL’99: The 37th
annual meeting of the Association for Computational Linguistics (New Jersey: Association for Computational Linguistics, June
1999), 3-10.
5. Tetsuya Nasukawa and Tohru Nagano, “Text
Analysis and Knowledge Mining System,”
IBM Systems Journal 40, no. 4 (2001): 967-84.
6. Alex Berson, Stephen Smith, and Kurt
Thearling, Building Data Mining Applications
for CRM (New York: McGraw-Hill, 2000), 461.
7. Concepts are topics that hoteliers are interested in.
8. We cannot identify concept-related information in a text file (e.g., company document,
Web page) if it does not contain the requested
information. This is similar to conducting survey research. A researcher cannot obtain the
requested information if the survey respondent
does not answer the survey question.
9. Kin-nam Lau, Kam-hon Lee, Pong-yuen Lam,
and Ying Ho, “Web-Site Marketing for the
Travel-and-Tourism Industry,” Cornell Hotel
and Restaurant Administration Quarterly 42,
no. 6 (December 2001): 55-62.
10. Berson, Smith, and Thearling, Building Data
Mining Applications for CRM, 461.
11. Kin-nam Lau, Kam-hon Lee, Ying Ho, and
Pong-yuen Lam, “Mining the Web for Business Intelligence: Homepage Analysis in the
Internet Era,” Journal of Database Marketing
and Customer Strategy Management 12, no. 1
(September 2004): 32-54.
AUGUST 2005
TEXT MINING FOR THE HOTEL INDUSTRY
12. We used the IBM text-mining product as the
primary tool of text analysis. For details, please
see www-128.ibm.com/developerworks/
d b 2 / l i brary/techarticle/0212tschaffler/
0212tschaffler.html. We used it to analyze room
type, food, in-room amenities, business services, beauty services, health and sports facilities, and other services. We used two computer
programs to identify affiliated hotels and contact information (i.e., address, e-mail, fax number, and telephone number). We also used one
computer program to examine hotel room price.
13. The Hong Kong Hotels Association had eighty
member hotels in 2002. We excluded three hotels because they did not have a valid Web address. In addition, since the Web crawler was
unable to distinguish hotels sharing the same
Web address, another three hotels were combined with their respective group hotels. As a
result, we collected Web pages of seventy-four
hotels from the Hong Kong Hotels Association
Web site.
14. The dictionary constructed for this study is
available upon request.
15. In particular, if a query requires the correct sequence of a search term to be found within a
sentence, HTML codes have to be well
formed. Every open tag has to be followed by a
close tag. For every HTML open tag without a
corresponding close tag, all information after
the open tag will be “invisible” to the textmining tool. Some information is lost in this
way which results in missing values for some
concepts.
16. We excluded special-period discounts (implied by date ranges) and promotional packages (implied by promotional keywords) in the
normal price analysis.
17. Hong Kong Tourism Board, “April Arrival
Figures Confirm Severity of Tourism Downturn,” Hong Kong Tourism Board Press Releases, May 29, 2003.
18. For examples of newsgroup studies, see
Rakesh Agrawal, Sridhar Rajagopalan,
Ramakrishnan Srikant, and Yirong Xu, “Data
Mining: Mining Newsgroups Using Networks
Arising from Social Behavior,” in Proceedings
of the 12th International Conference on World
Wide Web (New York: ACM, May 2003), 52935; and Andrew T. Fiore, Scott Lee Tiernan,
and Marc A. Smith, “Collaborative Filtering:
Observed Behavior and Perceived Value of
Authors in Usenet Newsgroups: Bridging the
AUGUST 2005
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
COMMENTS
Gap,” in Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems: Changing Our World, Changing
Ourselves 4, no. 1 (April 2002): 323-30.
For example, in 2004, the IBM DB2 Information Integrator for Content Processor cost about
US$65,001, while another product WordStat
(with Simstat 2.5) cost about US$525.
The estimated marketable price of IBM Enterprise Information Portal V7.1 in February
2002 was US$91,000.
IBM DB2 Information Integrator for Content
is formerly known as Enterprise Information
Portal.
Data source: www.fact-index.com/e/ex/
experience_curve_ effects.html.
Data sources: for Shenzhen, www.86hotel.
com; for Macau, official 2003 government statistics, from the Statistics and Census Service
of Macau SAR Government, Hotels and Similar Establishments Survey (Macau: The Statistics and Census Service, 2003), 5; and for
Zhuhai, www.86hotel.com.
For a discussion of difficulties associated with
analyzing personal home pages, please see
Gabriele Piccoli, “Web-Site Marketing for the
Tourism Industry: Another View,” Cornell Hotel and Restaurant Administration Quarterly
42, no. 6 (December 2001): 63-65.
For more information on analysis of digital imagery, please refer to www.dli2.nsf.gov/
internationalprojects/working_ group_rep o r t s / d i g i t a l _ i m a g e r y. h t m l # _ f t n 1 ;
w w w.i n f o r m e d i a . c s.c m u .e d u / d l i 1 / i ndex.html; and www.informedia.cs.cmu.edu/
dli2/index.html.
Lau, Lee, Lam, and Ho, “Web-Site Marketing
for the Travel-and-Tourism Industry,” 55-62.
Yingchen Yang and Wo-Shun Luk, “Web Mining: A Framework for Web Table Mining,” in
Proceedings of the 4th International Workshop
on Web Information and Data Management
(New York: ACM, November 2002), 36-42.
Jan H. Kroeze, Machdel C. Matthee, and Theo
J. D. Bothma, “Differentiating Data- and TextMining Terminology,” in Proceedings of the
2003 Annual Research Conference of the
South African Institute of Computer Scientists
and Information Technologists on Enablement
through Technology (Republic of South Africa: South African Institute for Computer Scientists and Information Technologists, September 2003), 93-101.
Cornell Hotel and Restaurant Administration Quarterly
361
COMMENTS
TEXT MINING FOR THE HOTEL INDUSTRY
Kin-nam Lau, Ph.D., is a professor in The School of Hotel and Tourism Management at the Chinese University of Hong Kong, where Kam-hon Lee, Ph.D., is a professor of marketing and director of the School
of Hotel and Tourism Management (khlee@cuhk.edu.hk) and Ying Ho is a project officer of the Center for
Hospitality and Real Estate Research.
362
Cornell Hotel and Restaurant Administration Quarterly
AUGUST 2005