Understanding Percentiles and Norms (Culture)
Overview
Denison Organizational Culture Survey results are presented in the form of percentiles. The purpose of
this document is to describe what percentiles are, why they are used, and how they are used by Denison
Consulting to help make culture survey results more understandable.
What are percentiles?
Percentiles show how an organization’s culture ratings compare to ratings of other organizations. They
indicate the percent of organizations that scored below a certain value. For instance, if your
organization’s “Core Values” index average is at the 57th percentile, it means that 57% of organizations
in the benchmark database had lower index averages than your company.
Why use percentiles?
The most difficult parts of conducting any type of evaluation are determining how well someone or
something actually performed and then communicating that information in a meaningful way. For
instance, suppose that Heather, a fourth-grader, knows 18 out of 30 words on a vocabulary test, or that
Steve, a high school senior, runs a mile in 7 minutes and 40 seconds. Most people would not find these
numbers to be very meaningful. Is knowing 18 out of 30 words an indication that Heather is falling
behind in vocabulary, or is she right where a fourth-grader should be? Is 7 minutes and 40 seconds a
poor time for a senior running a mile or should Steve consider running track in college? A logical
question to ask about the performance of Heather or Steve is “Is that good or bad?”
Both of these examples use what is known as a “raw score.” A raw score is the unadjusted value that
has been measured, whether it is a fourth-grader’s score on a vocabulary test, a high school senior’s time
in the mile run, an infant’s weight, a retail outlet’s sales revenue, or an IT consulting firm’s average
survey score on a five-point scale on the “Core Values” index. By themselves, raw scores may not be
very useful or informative for anyone other than a knowledgeable specialist. In short, raw scores often
cannot answer the question, “Is that good or bad?”
Percentiles to the rescue!
One way of addressing the “Is that score good or bad?” problem is to compare the score for one
organization with the scores attained by other organizations. This is known as “norm-referencing,” and
it is best known for its use in educational testing. Most parents with school-age children have probably
seen achievement testing results reported in percentile terms, particularly if their children are old enough
to have taken the PSAT, SAT, ACT, or any of a number of different state-administered achievement
tests given in the U.S. For instance, if Angie scores at the 82nd percentile on the “Math Concepts”
section of the Iowa Tests of Basic Skills this means that she scored higher than 82% of same-grade
students in a normative sample that took the same Math test. For Angie’s parents, this is a useful thing
to know. Relative to other fourth-graders, Angie’s progress in mathematics is extremely good. Unless
they were highly knowledgeable about the “Math Concepts” scale of the Iowa Tests, however, being
told that Angie had a raw score of 19 would not tell them very much about her progress.
The basis for comparison—norm groups
Reporting test scores or survey results in this manner requires that there be some group that sets the
standard. This group is known as the “normative sample,” “benchmark group,” or simply as the
“norms” and assessment specialists generally take great pains to make sure that this group is large and
representative enough to serve as an acceptable baseline. The Denison Organizational Culture Survey
database, for example, contains a normative sample of 707 companies to which an organization’s survey
results are compared. Whenever an organization participates in a new culture survey project, its mean
ratings on the 60 items and 12 indexes are compared to the item and index ratings obtained by the
organizations in this normative group.
How did Denison Consulting develop this benchmark group?
The database was built over seven years of conducting survey projects within organizations. The
database contains raw mean scores (averages of individual employees’ ratings) of items and indexes for
each organization. Organizations’ mean values were listed from lowest to highest for each item and
index and a statistical procedure was used to assign percentiles based on where those values fell in the
distribution of means.
The 2004 DOCS norms are based on 707 organizations as rated by over 280,000 total respondents.
Every organization included is rated by at least 25 and as many as 16,000 respondents. On average,
organizations in the benchmark were rated by 405 employees. Regardless of the number of respondents,
however, each individual organization is weighted equally in the norms. In other words, an organization
that surveys 3,000 people does not have a larger influence on the benchmark than an organization that
surveys 300. Organizations in the database were all assessed between 1995 and 2004, with the majority
(60%) surveying after 2000.
How does Denison change survey responses into percentiles?
For simplicity, this example is limited to the Vision index of the Denison Organizational Culture
Survey. Note that the process described here is identical for the other 11 indexes as well as all
individual survey items. Between 1995 and 2004, Denison Consulting collected Vision index ratings
from hundreds of companies and organized them into a database. This database contains average Vision
index ratings for each organization, listed from lowest to highest. A statistical procedure was performed
in order to determine which percentiles were associated with which raw scores. This procedure
“normalizes” the database, which puts all of the values on a normal curve (see next page).
A Normal Distribution of Raw Score Means for the
“Vision” Culture Index
50th
Percentile
80th
Percentile
Percentile
30%
5th
30%
95th
Percentile
Percentile
15%
2.62
Frequency of Means
20
th
15%
2.84
3.06
3.31
3.56
Raw Score Means
“Raw Score Means” refers to the original, unadjusted averaged values obtained from survey results. Raw score means are
arranged in this diagram in ascending order; as you move from the left to the right in this distribution, they become increasingly
large.
“Frequency of Means” refers to how common particular raw score means are in the distribution. The higher the curve at a
particular point, the larger the raw means are under that area. Means near the middle of the range, such as those between 3.30
and 3.60, tend to be most common. On the other hand, means in the two “tails,” or extreme ends, are much more unusual. Only
about 10% of all organizations have “Vision” indexes with raw means less than 2.98 or greater than 3.96.
A raw mean of 2.62 percentile is extremely low. It exceeds only about 5% of all raw means. In other words, it is smaller than
about 95% of means.
A raw mean of 2.84 is quite low. It exceeds 20% of all other raw means and is smaller than 80% of all other raw means.
A raw mean of 3.06 is at the 50th percentile, also known as the “median.” This value splits the distribution in half—it is larger
than about 50% and smaller than about 50%.of all raw means.
A raw mean of 3.31 is at the 80th percentile, which is higher than 80% and smaller than 20% of all raw means.
A raw mean of 3.56 is extremely high. It is higher than 95% and lower than just 5% of all raw means in the distribution.
After the normalization procedure was performed, each raw score mean was associated with a particular
percentile. Using this information, a new raw mean, such as an organization’s average on the Vision
index, can be converted into a percentile. At this point, the normative database was ready for use with
new survey projects.
How does this affect my survey project?
Organizations’ raw means on the Vision index can be converted into percentiles so stakeholders can find
out how well their organization is doing relative to the companies in the normative database.
Suppose for example that two companies, “Company A” and “Company B” both participated in culture
survey projects. Each company had a few hundred employees complete the Denison Organizational
Culture Survey. For each company, employees’ ratings were averaged to create raw means for each
item and index. The raw mean of the ratings provided by Company A employees on the Vision index
was 3.60. The raw mean of the ratings provided by Company B employees on this index was somewhat
lower at 3.27. These raw means are just a starting point—by themselves, they do not communicate
much information. Fortunately, because of the normative database it is possible to convert them into
something more useful. The raw means on the Vision index for Company A and Company B can be
compared to raw means on the Vision index obtained by the 707 organizations that contributed to the
normative database. The database includes not just raw means but also the percentiles associated with
those raw means. At this point, it is a simple matter of converting the raw means for these two
companies into percentiles:
“Vision” Normative Database
Company A
Company B
Raw Score Mean
Percentile
3.96
3.83
3.75
3.70
3.64
3.60
3.56
3.51
3.48
3.44
3.41
3.38
3.35
3.32
3.27
3.20
3.13
3.07
2.98
95
90
85
80
75
70
65
60
55
50
45
40
35
30
25
20
15
10
5
Note that the actual norms have percentiles ranging from 1 to 99.
To save space, this example table only shows every 5 th percentile.
These values are meant to serve as an example only and do not actually
represent normative data for the “Vision” index.
As the table shows, Company A’s raw mean of 3.60 is at the 70th percentile. This indicates that it is
higher than 70% of raw means obtained by the other organizations in the normative database. Company
A seems to be doing a better job than 70% of the companies in the benchmark group when it comes to
creating a vision for the organization.
On the other hand, Company B’s raw mean of 3.27 is only at the 25th percentile. Company B is in the
bottom quartile when it comes to establishing a clear, shared, long-term vision for the company—its raw
mean is higher than just 25% of the raw means in the normative database.
Company A and Company B both receive survey reports with their percentiles on the Vision index.
Their survey responses started off as raw scores, were averaged into raw mean scores, and now appear
as percentiles. The point of this whole process was to make the culture survey results easier to interpret
to give people a better understanding of their organizations’ relative strengths and weaknesses.