CPED FIPSE Research Project
Data Analysis Plan
This document contains a description of the data analysis plan for the CPED-FIPSE
research grant. It is meant to be a quick reference guide as you and your team member
embark on analyzing your case study data.
Research questions: The research questions guiding this project are:
1) What has been CPED’s impact on doctoral preparation?
a. What is CPED?
b. How has CPED influenced change at Participating Institutions of Higher
Education?
c. How has CPED influence on doctoral education moved beyond
participating institutions?
2) What do the Professional Practice Doctorates look like and how do they differ
from what was offered before?
a. Organizational structure
b. Learning environments
c. How do faculty and students engage with one another
d. Curriculum and teaching
e. How do these programs align with CPED principles?
f. What are the commonalities across the members?
3) How did the College/School make these changes?
a. What was the process?
b. Who was involved?
c. How did CPED influence the process?
4) What are the lessons learned?
a. How have changes crossed institutional boundaries?
b. What is a Professional Practice Doctorate (PPD or redesigned EdD)?
c. What do the PPDs look like?
d. What organizational structures support faculty as they redesign PPDs?
e. How does this differ from what offered before?
Levels of Data Analysis
This is a multiple-case study (Yin, 2003) across 22 CPED institutions. The study will
engage two levels of analysis.
Level 1: The first level of analysis will focus on the department/college that has
undertaken the redesign of their EdD/PPD utilizing CPED design-concepts and
principles. This analysis will reveal the change process at each institution and provide a
story of change that will answer research questions #1-4.
1
Level 2: The second level of analysis will focus on all 22 institutions that have
participated in the study and take place after individual cases have been written. This
cross-case analysis will provide answers to research question #5.
Theoretical Framework
Rogers’ (1995) Diffusion of Innovation Model. This model examines how
innovative ideas are disseminated through an understanding of the innovation, the
communication channels through which the innovation is described, the influences of the
social system on the process, and the time it takes for a decision to adopt the innovation is
made. Theoretical propositions from this framework have been used to develop the initial
codes for data analysis.
Data to be Collected and Analyzed for Final Case Report
This study employees a mixed methods design to enhance the findings of the study. Data
to be collected and analyzed by each person in each team are in red and include:
 Pre-visit data (institutional, college, EdD information) (collected by CPED and
used by each research team to describe the institution)
 Institutional and CPED artifacts (PI at each institution collects and each research
team uses to support the themes and patterns that emerge)
 Semi-structured interviews using protocols (each research team collects and
analyzes)
 Observations using protocol (each research team collects and analyzes)
 Quantitative online survey (PI at each institution distributes and a sub-team
analyzes)
Level 1 Analysis Procedure for Individual Case Studies
Timeline for Analysis
You may begin your analysis as soon as your documents, transcripts and other data are
loaded into the Google site. We will continually update teams on timelines for
completion during 2012.
2
At a Glance Analysis Plan
Review Raw Data
(Read raw transcripts—confirm you have all of your transcripts; remove names and identifiers; note any initial thoughts
about your case.)
D
e
d
u
c
t
i
v
e
C
o
n
t
i
n
u
e
Initial Coding
Code transcripts (via your choice) utilizing the coding list that were developed from Rogers theoretical propositions
Second Coding
Have your partner cross-check your coding and discuss any discrepancies
Categorical Analysis
A. Develop categories and subcategories and define their properties and dimensions (Strauss & Corbin, 1998, p. 124).
Move data into these categories
B. Relate categories and subcategories to one another to identify the conditions, actions, interactions and consequences
of each phenomenon (Glauser & Strauss, 1967)
Reveal Themes and Patterns
Create relational statements that link the themes and patterns
I
n
t
e
r
p
r
e
t
i
v
e
u
n
t
i
l
d
a
t
a
a
r
e
Making Assertions
Relate themes and patterns to theoretical propostions and research questions to make bold assertions about emerging
themes and patterns
s
a
t
u
r
a
t
e
d
Memoing
Write up ideas to identify the interactions among themes and statements (Miles and Huberman, 1994)
Narrative Case Report*
Follow the given case report frame in order to clarify the shape and direction of emergent themes and patterns
*See Yin (2003) resource regarding use of Chronologies for this report
Specifics for the Analysis Plan
Step 1. Review your raw data
 Read your transcripts
 Check that all documents you collected are uploaded (if documents are missing
contact Barb Valalik or Jill Perry immediately)
 Check transcripts to ensure they are accurate. Make corrections as necessary.
 Remove names and identifiers
 Note any initial thoughts about your case
3
Step 2. Begin your initial coding
Use the predetermined codes (based on Roger’s Theory and the research questions) to
code a your interviews and observations. (Note: computer programs such as NVivo,
Hyper Research, OneNote, or Atlas.ti are suitable tools to use).
Step 3. Perform a second coding
Have your partner cross-check your coding and discuss any discrepancies. Miles and
Huberman (1994) recommend that the consistency of coding be in agreement at least
80% of the time. After consistency is reached continue coding the rest of your
documents.
Step 4. Perform a categorical analysis
After documents are coded perform a categorical analysis. To do this, develop
categories and subcategories and define their properties and dimensions (Strauss &
Corbin, 1998, p. 124). Move your data into these categories.
Next relate your categories and subcategories to one another to identify the conditions,
actions, interactions and consequences of each phenomenon (Glauser & Strauss, 1967)
Use this deductive process (Steps 3 and 4) until the data has “run its course—when all
of the incidents can be readily classified, categories are saturated, and sufficient
numbers of regularities emerge” (Miles & Huberman, 1994, p. 62).
Step 5: Reveal themes and patterns
Once your data coding is saturated, create relational statements that link the themes and
patterns.
Step 6. Interconnect relational statements and make assertions.
Interconnect your statements and make assertions that can be used to answer each
research question.
Step 7. Memoing
Use memoing to write up ideas that emerge from examining the interactions among
themes and statements (Miles & Huberman, 1994).
Step 8. Write your case report
Follow the given case report frame in order to clarify the shape and direction of emergent
themes and patterns
*See Yin (2003) resource regarding use of Chronologies for this report
4
On-going Things to do Through the Entire Analytic Process to
Ensure Validity and Reliability
Establish that your analysis and the assertions from it are credible and trustworthy.
Ways to establish credibility and trust include:

onstantly compare your codes/themes to your data to make sure there is not a
drift in your analysis or shift in your interpretation of codes

eriodically share your findings with your research partner as you are
coding/interpreting your data

riangulate data sources

se thick/rich description

onform to the analysis plan so we can document consist procedures and have an
clean audit trail
c
p
t
u
c
We aim for:
External validity tests whether or not the findings of a study can be generalized to others
(Yin, 2003). The tactic typically used to establish external validity is that the study is
grounded in a broader theory. This study meets these criteria because its questions and
codes are grounded in Rogers’ Diffusion of Innovations model. Data collected will be
coded using theoretical propositions derived from this model and findings will be
compared to Rogers’ model to ensure comparability of results or to explain outliers.
Construct validity is tested during the data collection phase and seeks to ensure that the
researcher has established “correct operational measures” (Yin, 2003, p. 34) for the
phenomenon being studied. Case study methodology typically tests construct validity by
using multiple sources of evidence, establishing a chain of evidence and having key
informant review draft case study reports. In this study of change processes several
methods of data collection were employed.
Reliability demonstrates that the “operations of a study can be repeated” (Yin, 2003, p.
34). Demonstrating that others could potentially repeat the study eliminates researcher
bias and errors in a research study. Reliability is tested during the data collection and
analysis phases and is reinforced by using a case study protocol and a case study
database. This study is reliable because it uses semi-structured interview protocols and
structured procedures for collecting and analyzing data.
Internal validity is achieved during the data analysis phase and establishes a “causal
relationship” (Yin, 2003, p. 34). Internal validity is verified by creating patterns in codes
and building explanations from those patterns--theoretical propositions as initial codes
5
and developed categories and relational statements to find patterns and themes in the data
that explain the change process at each institution.
6