Text analysis for the political sciences
Winter Semester 2015/2016
Semester: Winter
Start and end dates: Jan 11 2016 – April 1 2016
Academic Program:
Master of Arts in Political Science;
Master of Arts in Political Science (2 years)
Name of instructor: Roel Popping
Learning Outcomes:
Text analysis is a data collection technique just like many others. Usually there are no direct responses
to a (research) question like agree, agree a little bit till disagree with an argument. The answers are
found in a piece of text, where the investigator should try to find them. In general these are not answers
to direct questions (like open-ended questions in a survey), but the texts contain opinions or facts on a
specific issue that are relevant for an investigator. This results especially in data that allow describing
certain developments in time, as development of democracy based on information in editorials in
newspapers. It is also possible to describe differences between groups, view of women versus men on
certain issues.
The course is an introductory one. It is meant as an introduction in the quantitative text analysis. In
there several qualitative elements do receive attention.
Generally social scientists use for their data collection some kind of survey research. Here the data
are collected from individual respondents. However, there are several other methods to collect data.
Text analysis is one of these. In such studies data might be sampled from individual respondents (via
diaries, letters, etc.), but more often the data come from articles in newspapers, documents from a
government, minutes of meetings, discussion groups, and so on. These articles usually are assumed to
represent the opinions that are hold by a broad public. At the end of the course the participants should
have an idea of how to perform a simple text analysis study and must be able to evaluate more complex
studies.
Assessment:
Students are expected to attend classes regularly, to read mandatory readings before classes, to be
active there, to write short position papers when asked, and, depending on the number of participants in
the class, to give a short presentation. Secondly, there will be an in-class, midterm test to be completed.
Finally, a 10 pages long (one-and-half spaced, without the references) final essay reporting a text
analysis study you performed yourself should be written, to be handed by the last class. Evaluation:
Activity and position papers 30%, Midterm test 20%, Final essay 50%.
Full description:
Master of Arts in Political Science Program - elective course
Master of Arts in Political Science (2 years) Program - mandatory course
In the course an overview is given of the actual state of the art of text analysis mainly within research
in sociology, political science, and communication studies. This overview is based on the literature.
After being confronted with the traditional content analysis or instrumental thematic text analysis
where concepts or themes are considered from the perspective of the investigator (Holsti, Krippendorff,
and Weber) the student will learn about the representational form of analysis, where the concepts are
considered from the point of view of the sender of the message. The instrumental and representational
approach are not only applied to the thematic analysis (with focus on frequency of occurrence and cooccurrences of concepts or themes), but also to the semantic (having attention for relations between
concepts: subject – verb - object) and network (text is transposed into networks) analyses. Finally
attention will be given to reliability and validity and some new developments will be mentioned.
At the end the student must be able to perform a simple (thematic) text analysis and must be able to
judge the more complex forms.
In the assignment a (small) study is performed in which the type of text analysis is performed that is
used by most investigators. During the course you will be asked to read some texts, we can discuss
together.
Information about the course
Familiarity
The student should be familiar with basics of methodology. Also some knowledge about statistics
would be very helpful.
Classes
The course will take place in two blocks: January 11 – February 5 and March 21 – April 1 2016.
Classes are on Tuesday from 17:20 h. till 19:00 h. and Thursday also from 17:20 h. till 19:00 h. in
FT809.
Way of working
During the teaching part of the courses the literature is worked out and clarified. Here the order as in
the reader is followed as much as possible. In the working part of the courses questions about literature
and assignments can be posed.
Assignment
There will be no final exam. The judgement is based on participation during the course, a midterm test
(February 3) and the final assignment. A sufficient grade is needed. The assignment is to write a
research paper / essay.
The final paper, which is an individual research paper, must be handed in on April 1 2016 before 17:00
hours. During the course the data needed for writing the paper will be elaborated in 3 steps (defining
the categories, defining the codebook, performing the analyses and reporting).
The idea is that the research for the paper and its writing is for the greater part done between February
8 and March 18. The topic of the papers is the comparison of some issues (to be agreed upon with each
of the students) is State of the Union speeches by the US Presidents Bush Jr. and Obama (so the
presidents are compared) or issues that are discussed in speeches outside Parliament by the Hungarian
Prime Minister. Here types of audience will be compared. For both topics data (texts) are available.
Evaluation of the assignment
To be decided after consultation.
Speaking hour
As I have no office at CEU the easiest way is to send an email: poppingr@ceu.hu.
Literature
Reader;
Lecture notes.
The reader, the PowerPoint presentations which will be made available and the content of the courses
give the background on which the assignments must be based.
Organisation and way of working
Way of studying
The course is organised around the literature and the assignments. The literature contains an
introduction into the quantitative text analysis (for a considerable part based on examples). In the
assignments it becomes clear how the text analysis can be used in a research situation.
Classes
The classes consist of a teaching part and a working part. The teaching part is meant to make the study
of the literature easier. The working part offers the possibility to discuss the assignment, so is focussed
on applying the things that are taught.
Global course outline with required readings
January 12
Introduction to text analysis
January 14
Instrumental thematic text analysis
Roberts, Carl W. (2000). “A conceptual framework for quantitative text analysis: On joining
probabilities and substantive inferences about texts.” Quality and Quantity, 34 (3): 259-274.
January 19
Representational thematic text analysis,
assignment / explanation computer programs.
Namenwirth, J.Z. (1969) 'Marks of distinction: An analysis of British mass and prestige newspaper
editorials.' American Journal of Sociology, 74 (4): 343-360.
Bligh, Michelle C., Kohles, Jeffrey C., & Meindl, James R. (2004). "Charting the language of
leadership: A methodological investigation of President Bush and the crisis of 9/11." Journal of
Applied Psychology, 89 (3): 562–574.
January 21
Semantic text analysis
Roberts, Carl W. (1997) “A Generic Semantic Grammar for Quantitative Text Analysis: Applications
to East and West Berlin Radio News Content from 1979.” Sociological Methodology, 27: 89-129.
January 26
Modality text analysis
Roberts, C.W., Zuell, C., Landmann, J. & Wang, Y. (2010). “Modality analysis: A semantic grammar
for imputations of intentionality in texts.” Quality and Quantity, 44 (2): 239-257.
Popping, Roel (2013). “What about the leader? What should the Hungarian Prime Minister do after he
lied?” Quality and Quantity, 47 (3): 1323-1335.
January 28
Assignment + what was left
February 2
Network text analysis + mid term test
Van Cuilenburg, Jan J., Kleinnijenhuis, Jan, & De Ridder, Jan A. (1986). “A theory of evaluative
discourse: Towards a graph theory of journalistic texts.” European Journal of Communication, 1 (1):
65–96.
Tambayong, Laurent, & Carley, Kathleen M. (2013). “Network text analysis in computer-intensive
rapid ethnography retrieval: An example from political networks of Sudan.” Journal of Social
Structure, 13 (2): 1-24.
February 4
Reliability and validity
Popping, Roel & Roberts, Carl W. (2009). “Coding issues in semantic text analysis.” Field Methods, 21
(3): 244-264.
Popping, Roel. (2010). “Ag09. A computer program for interrater agreement for judgments.” Social
Science Computer Review, 28 (3): 391-396.
March 22
Instrumental thematic text analysis (2)
Pennings, Paul, & Keman, Hans (2002). "Towards a new methodology of estimating party policy
positions." Quality and Quantity, 36 (1): 55-79.
March 24
Instrumental thematic text analysis (3)
Grimmer, Justin, & Stewart, Brandon M. (2013). “Text as data: The promise and pitfalls of automatic
content analysis methods for political texts.” Political Analysis, 21 (3): 267-297.
March 29
March 31
Coding
Final
Assignment / final questions on reporting. Hand in
assignment