Vianden, J. (2012).
The critical incident technique in student affairs research and practice.
Journal of Student Affairs Research and Practice, 49(3), 333–346.
doi:10.1515/jsarp-2012-6441
Available at http://journals.naspa.org/jsarp/vol49/iss3/art7/
Innovations in Research and Scholarship Feature
The Critical Incident
Technique in Student
Affairs Research
and Practice
Jörg Vianden, University of Wisconsin-La Crosse
Since the 1950s, the critical incident technique (CIT) has aided researchers in
the collection, analysis, and interpretation of qualitative data. In the student affairs literature only a very small number of CIT studies exist. Using the two most
recent publications as examples, this article provides a detailed overview of
the CIT, highlights its benefits and limitations, and offers specific recommendations for future applications of the CIT in student affairs research and practice.
In 1954, John C. Flanagan, professor of industrial psychology at the University of Pittsburgh,
and one of the founders of the American Institute for Research (AIR), published his landmark article on the critical incident technique (CIT)—a qualitative research method. The CIT was an outgrowth of the Aviation Psychology Program of the U.S. Army Air Forces, which Flanagan was commissioned to lead in the early 1940s. In spite of its original military applications, the CIT has been
widely heralded and used in several fields and professions, including counseling, psychology, nursing, communications, medicine, and social work (Butterfield, Borgen, Amundson, & Maglio, 2005).
The marketing literature, in particular, has witnessed an explosion of CIT-related articles—more
than 150 studies have been published since the early 1990s (Gremler, 2004).
Jörg Vianden, Assistant Professor, Student Affairs Administration in Higher Education, University of Wisconsin-La Crosse.
JSARP 2012, 49(3)
© NASPA 2012 http://journals.naspa.org/jsarp doi:10.1515/jsarp-2012-6441
Brought to you by | University of Wisconsin La Crosse
Authenticated | 138.49.99.65
Download Date | 1/16/13 11:04 PM
333
Critical Incident Technique
Although the use of the CIT in education-related fields has been well documented, studies
using the CIT in student affairs are practically nonexistent. This is surprising given the ubiquity of
innovative qualitative methods in higher education and specifically student affairs research. Existing CIT studies related to higher education focus on the instructor-student relationship, specifically in-class encounters and instructor fairness (Houston & Bettencourt, 1999), instructor quality or behavior (Davis & Swanson, 2001; Frankel & Swanson, 2002; Voss, 2009; Voss, Gruber, &
Reppel, 2010), or satisfaction with the college classroom or with the overall university experience
(Douglas, McClelland, & Davies, 2007; Menges & Kulieke, 1984). All of these studies, however, surfaced in the marketing and service literatures; and most of them took place outside the American
higher education context.
A search for CIT-related studies in the online repositories for the Journal of College Student
Personnel/Development (1965 to present), the NASPA Journal (1970–2009), and the Journal of Student
Affairs Research and Practice (2010 to present) yielded a total of four studies using the CIT (Belch
& Marshak, 2006; Bohan & Humes, 1986; Moores & Popadiuk, 2011; Schmelzer, Schmelzer, Figler,
& Brozo, 1987). In addition, Aamodt, Keller, Crawford, and Kimbrough (1981) published a study
on resident assistant job satisfaction in a nonstudent affairs journal. Despite the applicability of
the CIT to the higher education and student affairs context, why have so few studies chosen this
specific method?
This article focuses on introducing and critiquing the CIT, its methods, and its applications
to student affairs research and practice. In a review of the methods and findings of the two most recent publications, Belch and Marshak (2006) and Moores and Popadiuk (2011), the current study
illustrates the utility of the CIT. Such a summary and methodological critique of the CIT does not
exist in a student affairs context; in fact, I assume most student affairs professionals have not heard
of this method. Finally, I explore the CIT as an appropriate technique to collect data for student affairs research as well as practice. It is important to note that Dwayne Gremler, professor of marketing at Bowling Green State University, published a groundbreaking review in 2004 of over 100 CIT
studies used in the marketing and service literatures. The present article consults Gremler’s work
to provide an overview of the utility of the CIT in student affairs research and practice.
Overview of the CIT
Background
During World War II, Flanagan (1954) developed the CIT to effectively measure combat pilot
performance. Flanagan asked pilots to describe incidents that were particularly helpful or harmful
in their missions, with the goal of developing a list of components critical for task performance and
training (Woolsey, 1986). After the war, the development of the CIT continued at the AIR, of which
Flanagan was a founding member (Butterfield et al., 2005). Over the years, researchers employed
the CIT to develop ethical standards for psychologists, measure task proficiency, select and classify
334
doi:10.1515/jsarp-2012-6441 http://journals.naspa.org/jsarp © NASPA 2012
Brought to you by | University of Wisconsin La Crosse
Authenticated | 138.49.99.65
Download Date | 1/16/13 11:04 PM
JSARP 2012, 49(3)
Critical Incident Technique
staff, design job operating procedures and equipment, recognize attitudes and leadership behaviors, and identify factors effective in counseling (Woolsey, 1986).
Flanagan (1954) defined the CIT as follows:
[A] set of procedures for collecting direct observations of human behavior in such a way as
to facilitate their potential usefulness in solving practical problems and developing broad
psychological principles. . . . [an incident describes] any specifiable human activity that is
sufficiently complete in itself to permit inferences and predictions to be made about the person performing the act. To be critical the incident must occur in a situation where its consequences are sufficiently definite to leave little doubt concerning its effects. (p. 327)
To add to the original definition, Bitner, Booms, and Tetreault (1990) described a critical incident as
an observable human activity comprehensive enough to make predictions and conclusions about
the person displaying the behavior. A critical incident makes a significant positive or negative contribution to an activity or phenomenon (Gremler, 2004) or reflects a specific satisfying or dissatisfying interaction between customers and service providers (Bitner et al., 1990). Using the CIT,
some higher education research has classified student-faculty interactions inside the classroom as
service encounters (e.g., Voss, 2009).
Although incidents can be gathered in many ways, the universally accepted approach is for
participants to qualitatively relate an experience, a memorable event, or an incident that in some
way shaped or changed their thinking or behavior. After the collection of incidents, the CIT defines
specific methods for classifying the observations of human behavior.
Description of the CIT Method
Five specific steps are followed in a critical incident study: (a) determining the aim of the
activity under study, (b) making plans and setting specifications, (c) collecting data, (d) analyzing data, and (e) interpreting the data and reporting results. Although this section outlines the
five components, Flanagan (1954) or Woolsey (1986) provide more detail on the specific procedures.
Aim of the activity. Early CIT studies explored and described a specific activity of employees performing a task. As a result, determining the aim of that activity is a basic step for the
researcher before proceeding with the remainder of the study. Questions the researcher needs to be
asking in this first step are: (a) What is the objective of the activity? and (b) What is the person engaging in the activity supposed to accomplish (Butterfield et al., 2005)? An evaluation of whether
the activity or behavior is effective cannot be completed unless the researchers set the expectations
for the behavior at the outset (Flanagan, 1954). However, Flanagan warned that objectivity may not
be reached in conceptualizing the general aim of the activity if a number of leaders or constituents
want input in the activity and what is to be expected. A commonly accepted description of the
functional activity, used by all stakeholders, is critical for data collection. According to Flanagan,
JSARP 2012, 49(3)
© NASPA 2012 http://journals.naspa.org/jsarp doi:10.1515/jsarp-2012-6441
Brought to you by | University of Wisconsin La Crosse
Authenticated | 138.49.99.65
Download Date | 1/16/13 11:04 PM
335
Critical Incident Technique
such descriptions center on a simple phrase or are slogan-like in character (e.g., “appreciation,”
“efficiency,” “development,” or “service”). For example, the general aim of a residence hall director
might be to develop resident assistants who appreciate diversity. Likewise, the general aim of a
vice president of student affairs may be the effective and efficient management of fiscal and human
resources of the division. In summary, establishing the general aim of an activity should include a
brief description obtained from the experts (e.g., researchers, teachers, managers, participants) in
the field. This description, expressed in simple yet agreeable terms, focuses the objectives to be accomplished with the activity (Flanagan, 1954).
Plans and specifications. At this second stage Flanagan (1954) outlined the specific
training of observers. Such training is necessary for objectivity and consistency in making and
documenting observations of participant behavior. This stage also defines who will do the observing (e.g., supervisor or participant), which persons or groups will be observed, and what specific
participant behaviors will be observed (Kain, 2004). Initially, Flanagan suggested that supervisors
are the best participants for a CIT study because of their broader perspective and their likelihood
of witnessing or hearing about multiple critical incidents. On the other hand, the supervisee likely
only recalls his or her own critical incidents. Conversely, in more recent studies (see Kain, 2004)
researchers advocated collecting critical incidents from the employee directly (e.g., resident assistant) rather than the supervisor (e.g., residence hall director). Supervisors may not directly observe
all the possible behaviors of interest to a researcher studying effective resident assistant confrontation techniques or meaningful diversity appreciation programming. The bottom line in step two of
the CIT is to establish a sound procedure so the researcher can clearly determine who will provide
what information about a specific group of participants (Kain, 2004).
Data collection. Collecting data is the third component of a critical incident study. Flanagan (1954) initially established two main methods of data collection: (a) direct observations of
participant behavior by trained experts and (b) asking participants to recall critical incidents from
their memory. Although at first Flanagan felt direct observation was the preferred method of data
collection, he later suggested that authentic and accurate results can emerge from recalled incidents when participants are prompted to use precise detail to present the incident (Butterfield et
al., 2005). The main methods of data collection for the CIT are individual or group interview, as well
as open-ended surveys or questionnaires; see Voss, Gruber, and Reppel (2010) for a suitable CIT
study conducted with an open-ended online survey. Well-conceived online surveys enable researchers to reach hundreds of participants using a qualitative approach, which is impractical using
face-to-face data collection methods. To determine sample size, CIT researchers observe the number
of precisely recalled critical incidents rather than the number of participants (Butterfield et al.,
2005). During an interview, the researcher simply invites participants to tell a story about a critical event, experience, or incident; and asks them to explain why the specific story was significant or
memorable. Not all CIT studies follow the same blueprint when asking questions of participants.
336
doi:10.1515/jsarp-2012-6441 http://journals.naspa.org/jsarp © NASPA 2012
Brought to you by | University of Wisconsin La Crosse
Authenticated | 138.49.99.65
Download Date | 1/16/13 11:04 PM
JSARP 2012, 49(3)
Critical Incident Technique
In some studies researchers ask participants to recall positive and negative incidents
(Aamodt et al., 1981), while other studies focus on only either positive (Moores & Popadiuk, 2011)
or problematic incidents (Belch & Marshak, 2006). According to Kain (2004), some CIT researchers ask participants to recall as many events as possible during an interview, while others encourage participants to only recall one or two (either negative or positive or both). Kain (2004) pointed
out that as few as 16 to as many as 4,000 incidents have been reported as adequate samples in published CIT studies. Flanagan (1954) offered the general rule that incidents are sufficiently reported
or exhausted when collecting another 100 incidents only yields 2 to 3 more critical behaviors or
experiences.
Some researchers invite participants to recall all incidents of a specific activity over time
while others set the timeframe in the more immediate past. According to Voss (2009), the most effective time period for authentic recall of critical incidents should not extend more than four to six
months from the incident. The aim of the study, however, is crucial here: younger participants or
those bound by time or by a specific institutional environment (e.g., students at a two-year college)
may require the opportunity to recall incidents that reach farther back than the suggested optimal
four to six months.
As with other qualitative methods, CIT researchers should be carefully aware of how they
word questions during data collection. Investigators should consider how participants may perceive the researcher’s affiliation with the general aim of the activity, the context of the questions
asked, and how follow-up questions are worded. In general, most CIT scholars suggest that collecting critical incidents from respondents in interviews should always include the following specific
questions or slight variations of these:
•
Exactly what happened during the incident?
•
Exactly who did or said what?
•
Who or what was the subject of the incident?
•
Where and when did the incident take place?
•
What does the participant take away from the incident?
•
What specifically about the incident was positive or negative?
•
What has been the participant’s response to the incident?
Some researchers incorporate Likert Scales into their interviews to assess the significance of impact of the incident on the participant (e.g., 1–7, least to most significant). Others ask whether or
not participants have talked about the incident with someone else to suggest positive or negative
word-of-mouth communication (Voss et al., 2010).
In summary, most important for the third component of the CIT are that (a) participants
report actual incidents and behaviors, (b) the relationship of the reporter to the behavior is clear,
JSARP 2012, 49(3)
© NASPA 2012 http://journals.naspa.org/jsarp doi:10.1515/jsarp-2012-6441
Brought to you by | University of Wisconsin La Crosse
Authenticated | 138.49.99.65
Download Date | 1/16/13 11:04 PM
337
Critical Incident Technique
(c) the incidents provided are sufficiently relevant to the general aim of the activity, (d) the participant clearly identifies what makes the incident critical, and (e) the reasons for this identification
are clear to the researcher (Kain, 2004).
Data analysis. The fourth section of a CIT study is data analysis. Often considered the
most important and difficult step, the classification of hundreds of incidents can be complicated
(Butterfield et al., 2005). Data analysis in the CIT follows inductive procedures and results that
emerge from the data. Flanagan (1954) held that analysis of critical incident studies should be practical so the results could be used in training or evaluation of employees. In this stage, the researcher
creates a categorization scheme, which describes and summarizes the data usefully without taking away their validity, comprehensiveness, and specificity (Flanagan, 1954). To this end, Flanagan
proposed three specific steps of data analysis. First, the researcher determines a frame of reference
(the classification system). In establishing the classification of the data, researchers must establish
the purposes for which they will use the data. The second step in data analysis is the creation of the
categories that reflect the frame of reference. Flanagan (1954) noted the subjective nature of this
formation of categories, as it required experience and judgment from the researcher. This inductive analysis “is a tentative, interactive process, where the researcher sorts incidents into groups
derived from the frame of reference” (Kain, 2004, p. 76). The final step in data analysis involves the
selection of specificity on a continuum from general to specific. Although one could report hundreds of specific incidents, researchers advocate for listing fewer, more universal categories that
inform the general aims of the activity (Flanagan, 1954; Kain, 2004). Practical considerations about
the use of the research should drive the level of specificity or generality of the classification scheme.
The headings should be logical and clear-cut, and Flanagan suggested that they should be easily
discernable by the consumer of the research.
Data interpretation and reporting results. The final component of critical incident studies involves decisions about interpretation and reporting. CIT reports should be “vivid
and evocative” (Woolsey, 1986, p. 251), conveying an image of the critical incidents in each category.
The purpose of the report drives the amount and type of information to be included. Some authors
simply include the categories, providing brief descriptions of each, while others explicate each category in detail, using in vivo quotes from participants to make incidents come alive (Woolsey, 1986).
Kain (2004) suggested the report account for the steps the researcher took in the preceding
four phases, enabling the reader to discover how the results could be generalized to other contexts
or studies. Researchers should keep in mind that qualitative data do not aim at generalization to
larger contexts, but achieve reader-based transferability of results (Lincoln & Guba, 1985). Most
important in this last step is the researcher’s explication of the credibility and value of the findings.
To determine these components, Flanagan (1954) encouraged researchers to explain what was most
salient about the findings under initial aim of the activity.
338
doi:10.1515/jsarp-2012-6441 http://journals.naspa.org/jsarp © NASPA 2012
Brought to you by | University of Wisconsin La Crosse
Authenticated | 138.49.99.65
Download Date | 1/16/13 11:04 PM
JSARP 2012, 49(3)
Critical Incident Technique
Benefits of the CIT
One of the most significant advantages of the CIT is its connection to real-world problems
and situations provided through the words of the participant, thus limiting the subjectivity of
the researcher (Kain, 2004). CIT studies collect data from the participants’ perspective; and participants decide which incidents, situations, events, or activities are the most critical, memorable,
or salient. Perhaps no other qualitative method allows participants as wide a range of responses
within a given research study (Gremler, 2004). CIT studies do not present the participant with
preconceived questions, definitions, or constructs; and researchers do not force participants to provide experiences within a given framework, so participants freely develop the context using their
own perspective. This allows cultural neutrality (Gremler, 2004), as the participants are invited
to share their perspective without the cultural boundaries of methods such as surveys, questionnaires, or interviews. When recalling incidents, participants openly use their language, and the researcher’s questions typically do not include jargon or technical language.
Another major advantage of the CIT is its usefulness in the early stages of exploring an understudied phenomenon (Kain, 2004). As an inductive method, the CIT researcher avoids hypotheses and operates with a flexible set of rules to let themes or theories emerge directly from the data.
Thus, the CIT is valuable in pilot studies or in research where determining all variables a priori is
difficult or impossible (Gremler, 2004).
Third, CIT studies provide accurate, in-depth, firsthand, and significant portraits of events
from a variety of participants. To this end, the CIT uses a narrative approach, one of the most “accessible human discourse avenues” (Kain, 2004, p. 78). This is specifically vital in higher education
and student affairs contexts. Authentic accounts provide meaningful insights into how a group
of participants experience a particular institutional environment, course, or student service. The
efficacy of such stories in customer relations positions is not to be underestimated. Critical incidents can be instructive for management when training frontline personnel on which behaviors
to perform or to avoid when interacting with customers (Gremler, 2004). In a higher education
context, this is vital for faculty and student affairs staff who have more daily contact with students
than any other group on campus.
Limitations of the CIT
Although the advantages of using the CIT are substantial, the CIT is not without potential
drawbacks. Despite its longevity of well over 50 years, the CIT has seemingly not broken into the
cadre of renowned methodologies, including ethnography, phenomenology, case study, or grounded theory designs. The paucity of student affairs research using the CIT perhaps underpins that
claim.
According to Kain (2004), the CIT may suffer from an identity crisis evident in both terminology and worldview. The term critical incident technique is the most commonly used; however,
JSARP 2012, 49(3)
© NASPA 2012 http://journals.naspa.org/jsarp doi:10.1515/jsarp-2012-6441
Brought to you by | University of Wisconsin La Crosse
Authenticated | 138.49.99.65
Download Date | 1/16/13 11:04 PM
339
Critical Incident Technique
other researchers have used variations including critical incident analysis, critical event technique,
critical incident study technique, critical incident report, and critical incident reflection (Butterfield et al., 2005). Such inconsistent use of the name and definition of the CIT does not guarantee
its universal recognition and appreciation. Further, the CIT has been characterized as a research
approach not firmly grounded in either qualitative or quantitative paradigms. Developed during
a time when positivist paradigmatic approaches dominated social scientific research, many CIT
scholars today still attempt to use descriptive statistics to count category membership of critical
incidents (Kain, 2004). Such efforts appeal to the quantitative worldview, which is often still assumed superior to qualitative methods. The strength of the CIT does not lie in inferential or even
descriptive statistics, however, but in the attention paid to authentic richness of detail in participants’ accounts of a phenomenon (Kain, 2004).
Another limitation centers on how researchers should best gain trustworthiness of the
methods and findings in CIT studies. Many criticize the fact that researchers can misinterpret or
misunderstand reported incidents, or potentially misclassify or wrongly code incidents into specific categories. Along with these possible flaws originating with the researcher, others criticize
the CIT for its retrospective nature and the possible recall bias of participants. Researchers ask
participants who have to remember past incidents to report truthfully and accurately. As a result,
potential lapses in memory or reinterpretation of events are critical issues for the CIT researcher
(Gremler, 2004).
Third, data collection in CIT studies relies on specific stories and incidents participants consider critical. However, participants may not be adept at telling or writing such a story. This results
in a potential drawback: stories about critical incidents may be insufficiently rich in detail, thus
yielding a limited dataset. Data devoid of rich detail may not present enough salient implications
for practice or behavior in a specific aim under study.
The Utility of the CIT Method in a Student Affairs Context
The application of the CIT in student affairs has been very limited; only five studies exist
with methods and findings that are instrumental for future CIT researchers. To highlight instructive examples for the CIT in student affairs research, I focus on the two most recent publications
(Belch & Marshak, 2006; Moores & Popadiuk, 2011) in the following section.
Belch and Marshak (2006) explored critical incidents related to college students with psychiatric disorders. Rather than collecting incidents from student participants, Belch and Marshak
studied these incidents from the perspective of 62 senior student affairs officers (SSAOs). The purpose of the study was to elucidate common issues that the participants identified as particularly
problematic when attempting to work with students who have psychiatric concerns.
Belch and Marshak (2006) created a questionnaire, based on a variation of the CIT, to elicit
responses from the participants and to reach a wider audience. The researchers disseminated the
340
doi:10.1515/jsarp-2012-6441 http://journals.naspa.org/jsarp © NASPA 2012
Brought to you by | University of Wisconsin La Crosse
Authenticated | 138.49.99.65
Download Date | 1/16/13 11:04 PM
JSARP 2012, 49(3)
Critical Incident Technique
questionnaire to 400 SSAOs of NASPA member institutions. This questionnaire was intentionally
open-ended to avoid researcher-biased responses while also allowing for detailed unambiguous accounts of critical incidents. The authors asked two specific questions of the SSAOs: (a) to briefly
describe an incident with a student with a psychiatric disorder that SSAOs or their staff considered problematic and (b) specifically what the respondents found difficult about the incident. The
CIT is clearly an appropriate approach for such an aim of activity; however, two specific issues are
worthwhile to point out for future student affairs researchers interested in using the CIT.
First, as chief student affairs officers on campus, SSAOs likely are too far removed from firsthand encounters with students with psychiatric disorders, particularly at larger institutions. As a
result, all critical incidents they report are prone to be secondhand information. Selecting directors
of campus counseling centers or actual counselors as participants may have been more appropriate because of their closer proximity to critical incidents with students with psychiatric concerns.
Studying those more immediately involved with the aim of an activity likely yields incidents richer
in detail and, therefore, more salient implications for practice.
Second, the questions Belch and Marshak asked on the questionnaire were perhaps not specific enough to elicit more detailed incidents. The authors encouraged participants to offer a “brief”
description of the incident without asking more explicitly who said or did what, where and when
the incident occurred, or the subsequent response of the participant or student to the incident.
This lack of specificity potentially contributed to a relatively low response rate of 16%. Further, it
is unclear how many useable critical incidents resulted from the study. Informing the reader about
the specific sample of critical incidents is helpful and instructive for future CIT users and aids in
gauging transferability of results.
In their 2011 study, Moores and Popadiuk focused on the transition of seven international
students to a Canadian university. The authors collected critical incident data in one-hour, semistructured, face-to-face interviews. Not considering any negative events, the researchers gathered
134 positive incidents, yielding an average of 19 critical transition incidents per student. This leaves
potential questions about the level of detail of each incident and the depth of the stories participants can offer. In addition, the authors provided their participants a list of key questions prior to
the interviews—a technique not discussed in other CIT studies. On the one hand, CIT researchers
aim to avoid having participants relay incidents that sound rehearsed or unauthentic because they
were prepared beforehand. On the other, having the opportunity to think about critical incidents
ahead of time may aid the recall process and yield richer data.
Moores and Popadiuk (2011) offered comprehensive information about their data analysis
techniques serving as an instructive method for future CIT researchers. The authors completed
their data analysis in three stages: (a) incident extraction from the data, (b) category formation,
and (c) explication of trustworthiness. Matching the techniques of other CIT researchers, Moores
and Popadiuk considered only those incidents that participants discussed in sufficient detail,
those whose events were clearly identified, and those that clearly related to the focus of the study.
JSARP 2012, 49(3)
© NASPA 2012 http://journals.naspa.org/jsarp doi:10.1515/jsarp-2012-6441
Brought to you by | University of Wisconsin La Crosse
Authenticated | 138.49.99.65
Download Date | 1/16/13 11:04 PM
341
Critical Incident Technique
During incident categorization, the authors described their strong collaboration, which included
meeting on several occasions and frequent e-mail correspondence to review and develop the categories reflecting the incidents. Moores and Popadiuk indicated that they categorized data by the
incident itself, rather than the source of the incident or its outcome.
CIT researchers face difficulties when participants relate incidents without clearly demarcating whether the outcome or source is the incident in the mind of the participant. For instance,
in Moores and Popadiuk’s (2011) context, is a well-conceived orientation for international students
the source or the outcome of the activity? Asking clarifying questions during data collection can
ameliorate this concern. Moores and Popadiuk identified eight major categories of incidents and
nine subcategories. This process included a shaping and reshaping of the categories until only relevant incidents had been placed in each category and until all relevant incidents had been included.
Moores and Popadiuk (2011) detailed strategies to achieve trustworthiness of their findings
using techniques explained by Lincoln and Guba (1985), including prolonged engagement with
participants, peer debriefing with colleagues, member checks, and persistent observation. Moores
and Popadiuk also employed two tests of inter-rater agreement between researchers and an assessment of the comprehensiveness of the categories using a 10% sample of incidents initially not
included in the categorization. These techniques can be easily followed by future CIT researchers
who want to achieve a high level of trustworthiness. Moores and Popadiuk’s study is an apt example
for how the CIT can be used in a student affairs context. Although the study leaves some unanswered questions, the authors offered a thorough analysis of methods and data analysis within a
CIT framework.
As this brief discussion of CIT studies has shown, variations of the CIT exist in student affairs research. Scholars who want to use the CIT in subsequent studies should design their studies carefully from the outset and record their specific steps during the process so the explication
of methods and findings is clear for a potential submission for publication. Although the studies
discussed here provide a satisfactory level of detail about how to use the CIT, Gremler (2004) presented outstanding instructions about study design and data analysis.
Suggestions for Applying the CIT in
Student Affairs Research and Practice
As previously stated, it is surprising that the CIT has not seen more frequent use in student
affairs research. Its relative ease of data collection and analysis, the ability to record and provide
authentic lived experiences of participants, and the capacity to let the participant decide what is
most salient in a given experience make the CIT attractive to the student affairs researcher. Aside
from scholars who aim to create new knowledge about phenomena, the CIT provides practitioners
a meaningful assessment tool to get a clearer picture about how constituents perceive their servic-
342
doi:10.1515/jsarp-2012-6441 http://journals.naspa.org/jsarp © NASPA 2012
Brought to you by | University of Wisconsin La Crosse
Authenticated | 138.49.99.65
Download Date | 1/16/13 11:04 PM
JSARP 2012, 49(3)
Critical Incident Technique
es, programs, or activities. The CIT is a near-perfect method for gathering evaluative and summative
information about all student affairs functions and units.
Previous research has shown the utilization of the CIT in human resources or management
contexts (Butterfield et al., 2005). In student affairs, supervisors should employ the CIT to assess
the level of training needed, skill attained, and the competencies mastered by staff members at all
levels. Undergraduate staff members in orientation or residence life, for instance, may be excited to
participate in such a study because they decide which incidents are most critical and engage more
authentically in the study. In addition, the CIT can be used to supplement performance evaluations
of staff members and of supervisors. In a 360-degree evaluation, for instance, staff should be invited
to report satisfactory and unsatisfactory incidents in interactions with other staff or supervisors. If
conducted by an impartial consultant, such evaluations can remain anonymous to yield authentic
information.
Another potential application of the CIT involves the development and evaluation of leadership behaviors or professional ethics. Student activities officers could use the CIT to gather data
about which behaviors or actions are considered critical by program participants or staff, or gather
data from student affairs leaders on campus about their perception of critical behaviors to subsequently devise new programs or training modules. Given the recent endorsement of new professional student affairs competencies by the ACPA and NASPA, opportunities abound for the CIT
to help discover whether a given student affairs unit meets the competencies or moves staff in the
direction of mastery.
Flanagan (1954) initially used the CIT to assess which tasks combat pilots perceived as critical under stressful situations. In student affairs, stressful situations occur in crisis response and
emergency management. First, the CIT presents an invaluable tool to explore effective emergency
response and crisis management skills of staff by identifying critical training and development
deficiencies. Second, the CIT can be used as a counseling technique to help staff identify which
events or situations are critical during crisis response. When well-designed, this use of the CIT may
provide staff a cathartic outlet to share stressors, while providing the supervisor with salient data
about emergency response training. Woolsey (1986) pointed out the CIT’s capacity to offer a counseling-like setting between the researcher and participant. Such an effect of the CIT on a stressed
student affairs professional should not be underestimated.
Diversity education and awareness potentially represent a large area for application of the
CIT in the student affairs context. Practitioners should use the CIT to gather data from diversity
trainers, program participants, students, or faculty about the levels of civility on campus. The CIT
offers a welcome break from typical quantitative campus climate studies that fail to understand
critical events from a lived experience chosen by and expressed in the words of participants. This
may yield more authentic data about diversity concerns on campus. On the other hand, practitioners should employ the CIT to unearth the most satisfying incidents on campus relative to diver-
JSARP 2012, 49(3)
© NASPA 2012 http://journals.naspa.org/jsarp doi:10.1515/jsarp-2012-6441
Brought to you by | University of Wisconsin La Crosse
Authenticated | 138.49.99.65
Download Date | 1/16/13 11:04 PM
343
Critical Incident Technique
sity appreciation. Quotes gathered from such assessments fit well into marketing strategies or advertisements for services or educational programs. When discrimination or harassment occur on
campus, professionals may use the CIT to gather specific data about what victims consider most
critical in the event. Should a concern of a cultural nature exist for the victims, the CIT allows them
to relate an experience in their own words, instead of the researcher using typical scripted questions based on a potentially different cultural foundation.
Finally, student affairs practitioners can use the CIT to determine how student affairs professionals influence college student success. The learning outcomes movement urges college educators to unequivocally identify how they or their services affect student success; however, professionals in student affairs have not yet followed suit universally, due to time constraints or to
inadequate training in assessment methods or data analysis. The CIT gives student affairs professionals the opportunity to empirically identify which staff, events, or programs students perceive
most critical to their learning and success. Such a study would go beyond merely reporting student
satisfaction; student participants can explicate incidents in which student affairs staff affected
their success critically, and not simply those who made a student happy.
This article aimed to encourage all student affairs researchers and practitioners to understand, utilize, and develop the CIT in their research or during outcomes assessment. One word of
caution: as with all qualitative methods, CIT studies take a significant amount of time. Using the
CIT to produce trustworthy critical findings means the student affairs professional should be meticulous in design, data collection, analysis, and reporting. Without giving the CIT due diligence,
the outcome may be a hasty collection of unreliable data with little value for student affairs research and practice.
Conclusion
Although the CIT has been sparingly used in student affairs research and practice, it presents an invaluable method of collecting authentic data about the lived experiences of participants.
The relative ease of use by following five specific steps in the design phase, its benefits in collecting and analyzing data, and its capacity to report vivid and evocative incidents in the words of
the participants make the CIT an invaluable research technique. Student affairs researchers and
practitioners alike should begin to explore the CIT because it provides a meaningful, culturally
sensitive, and authentic data-gathering strategy to study the effectiveness of any student affairs
unit, service, or program.
344
doi:10.1515/jsarp-2012-6441 http://journals.naspa.org/jsarp © NASPA 2012
Brought to you by | University of Wisconsin La Crosse
Authenticated | 138.49.99.65
Download Date | 1/16/13 11:04 PM
JSARP 2012, 49(3)
Critical Incident Technique
References
Aamodt, M. G., Keller, R. J., Crawford, K. J., & Kimbrough, W. W. (1981). A critical-incident job analysis of the university housing resident assistant position. Psychological Reports, 49(3), 983–
986.
Belch, H. A., & Marshak, L. E. (2006). Critical incidents involving students with psychiatric disabilities: The gap between state of the art and campus practice. NASPA Journal, 43(3), 464–
483.
Bitner, M. J., Booms, B. H., & Tetreault, M. S. (1990). The service encounter: Diagnosing favorable and
unfavorable incidents. Journal of Marketing, 54(1), 71–84.
Bohan, D. K., & Humes, C. W. (1986). Assessment of the integration of physically handicapped college students into extracurricular activities. Journal of College Student Personnel, 27(1), 53–57.
Butterfield, L. D., Borgen, W. A., Amundson, N. E., & Maglio, A. T. (2005). Fifty years of the critical incident technique: 1954–2004 and beyond. Qualitative Research, 5(4), 475–497. doi:
10.1177/1468794105056924
Davis, J. C., & Swanson, S. T. (2001). Navigating satisfactory and dissatisfactory classroom incidents. Journal of Education for Business, 76(5), 245–250.
Douglas, J., McClelland, R., & Davies, J. (2007). The development of a conceptual model of student
satisfaction with their experience in higher education. Quality Assurance in Education, 16(1),
19–35. doi: 10.1108/09684880810848396
Flanagan, J. C. (1954). The Critical Incident Technique. Psychological Bulletin, 51(4), 327–358.
Frankel, R., & Swanson, S. R. (2002). The impact of faculty-student interactions on teaching behavior: An investigation of perceived student encounter orientation, interactive confidence,
and interactive practice. Journal of Education for Business, 78(2), 85–91.
Gremler, D. D. (2004). The Critical Incident Technique in service research. Journal of Service Research,
7(1), 65–89. doi: 10.1177/094670504266138
Houston, M. B., & Bettencourt, L. A. (1999). But that’s not fair! An exploratory study of student perceptions of instructor fairness. Journal of Marketing Education, 21(2), 84–96. doi:
10.1177/0273475399212002
Kain, D. L. (2004). Owning significance: The critical incident technique in research. In K. B. deMarrais & S. D. Lappan (Eds.), Foundations for research: Methods of inquiry in education and the social
sciences (pp. 69–85). Mahwah, NJ: Erlbaum Associates.
Lincoln, Y. S., & Guba, E. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage Publications.
Menges, R. J., & Kulieke, M. J. (1984). Satisfaction and dissatisfaction in the college classroom. Higher
Education, 13(3), 255–264.
JSARP 2012, 49(3)
© NASPA 2012 http://journals.naspa.org/jsarp doi:10.1515/jsarp-2012-6441
Brought to you by | University of Wisconsin La Crosse
Authenticated | 138.49.99.65
Download Date | 1/16/13 11:04 PM
345
Critical Incident Technique
Moores, L., & Popadiuk, N. (2011). Positive aspects of international student transitions. Journal of
College Student Development, 52(3), 291–306.
Schmelzer, R. V, Schmelzer, C. D., Figler, R. A., & Brozo, W. G. (1987). Using the critical incident technique to determine reasons for success and failure of university students. Journal of College
Student Personnel, 28(3), 261–66.
Voss, R. (2009). Studying critical classroom encounters: The experiences of students in German college education. Quality Assurance in Education, 17(2), 156–173. doi: 10.1108/09684880910951372
Voss, R., Gruber, T., & Reppel, A. (2010). Which classroom service encounters make students happy
or unhappy? Insights from an online CIT study. International Journal of Educational Management, 24(7), 615–636. doi: 10.1108/09513541011080002
Woolsey, L. K. (1986). The critical incident technique: An innovative qualitative method of research.
Canadian Journal of Counseling, 20(4), 242–2
346
doi:10.1515/jsarp-2012-6441 http://journals.naspa.org/jsarp © NASPA 2012
Brought to you by | University of Wisconsin La Crosse
Authenticated | 138.49.99.65
Download Date | 1/16/13 11:04 PM
JSARP 2012, 49(3)