Developing a coding scheme for
content analysis
A how-to approach
Literature review
• As is pretty much always the case, finding out
what people have done in the past is the best
way to prepare for your own research
– Gives you the theoretical lay of the land
– Helps you see how others have approached your
own concerns
– Helps you determine what measures have been
used and how they have worked
– Allows you to identify areas where additional
study is needed
Constructs
• To carry out a successful study, you will need
to identify constructs of significance for your
research
– Remember our discussion of explication?
• From the literature review and your own
experience/interests, what are the constructs
you want to measure?
– Is there a theory here?
Constructs
• In order for your analysis to be valid, you must
first specify your constructs carefully
– What are the characteristics of your construct?
• What would be a case of sexism, for example?
– (Note: perhaps ‘genderism’ would be better).
• What would not reflect sexism in your definition but
someone else might think would?
• What would be an example of non- or anti-sexism?
An example:
• How would you define ‘Respect for authority’?
• What behavioral examples would reflect it?
• What examples would you say do not reflect
it, but some folks would say did?
• What examples of lack of respect or disrespect
for authority can you think of?
Based on your analysis of ‘respect for
authority,’ what examples can you
think of?
• Who are authorities?
• What behaviors show respect? Lack of
respect or disrespect? What behaviors might
someone else say show respect but you
exclude from your list?
Now identify features of content that
reflect your constructs
• What would we look for in romance films that
would reflect the concept of genderism?
–
–
–
–
–
Males making important decisions
Women wearing skimpy clothing
Females showing greater sensitivity
Men always driving the cars
Etc.
• Note: There are a great number of potential
indicators—you have to choose those that are the
best indicators and are practical to score
Example
• If a student does what the teacher in a TV
show says, is it an example of respect for
authority?
• What else might be necessary?
– “Whatever you say, you’re the boss.”
– “I’ll do it to save my A, but that’s the only reason.”
– “I really like you, but I think this is a dumb idea.”
Choosing indicators
• Content analysis is quantitative, so you need
to develop categories that can be assigned
numbers
– Use indicators that can be translated reasonably
into quantitative scores
• Avoid developing category schemes that call
for value judgments and/or too much
interpretation on the part of the coder
Objective v. subjective codes
• Subjective categories (“high, medium, low”;
“better, worse”) are read quite differently by
various coders so their use invites unreliability
• Try to construct objective categories as much
as possible
– “4 or more times” v. “often”
– “Over 250 pounds” v. “large”
Good coding categories
• Categories should be:
– Exhaustive
– Mutually exclusive
– Derived from a single classification principle
– Independent
– Adequate to answer the questions asked of
the data
Exhaustive
• There should be a coding category that each
recording unit can be placed in
– Can use “other” or “none” categories to make the
scheme exhaustive
Mutually exclusive
• Each recording unit should fit into only
one category on a given scoring
dimension
Derived from a single
classification principle
• Must keep conceptually different
dimensions of analysis separate
– Code separately for each dimension
• Example: Character presented as “fearful and
inactive” (separate fear and activity into
individual scoring dimensions)
Independent
• Each category should be independent of
other categories—seek an ‘absolute’
value for each category
– This will be violated if your categorization
scheme assigns units to categories according
to their relative position on some dimension
• “more biased,” “scarier”
Adequate to answer the questions
asked of the data
• Must cover the entire concept or nearly so
• Must exclude third variables/related concepts
that are not supposed to be measured by the
category scheme
– Don’t let influences other than those you are
studying creep in and affect your results
• Differences among categories must be
meaningful
– Large enough to matter
– Dimension is appropriate
The precision tradeoff
• The more narrowly tailored and precise your
categories, the better the test of your research
questions or hypotheses
• However, the finer the distinctions you ask your
coders to make, the more ‘mistakes’ you will
generate
– In this form of text analysis, when two coders put
the same coding unit into different categories, at
least one of the coders is ‘wrong’
Good practice
• Try not to make categories too narrow
• e.g., 10-12 years old, 13-15, 16-18, etc.
– Few instances
– Coders have difficulty making such fine distinctions
• Cover enough features to make valid judgments
– Use multiple indicators of each construct
• Use the indicators that are easiest to
measure/code
– Only go to less clear-cut, more complicated indicators
after the straightforward ones have been exhausted
Think of a critic when you
design your scheme
• What would someone who doesn’t believe
genderism exists say when I have completed
my study and present my findings? Someone
who is adamant that genderism is rampant in
the media?
• Anticipate and deal with the most significant
critiques prior to finalizing your coding
scheme