Initial Protocol for a Systematic Literature Review of
Governance Innovation for Security and Development
Adapted from http://www.dur.ac.uk/ebse/resources/templates/SLRTemplate.pdf
Table of Contents
Change Record
Background
Search Strategy
Five steps:
1. Broad Sampling
2. PDS Framework
3. Study Eligibility
4. Source Evaluation
5. Validity, reliability and applicability of the studies (VRA)
Time period covered by this review
How the search process will be evaluated
Study Quality Assessment
Data Extraction
Synthesis
Study Limitations
Reporting
Schedule
Change Record
dates and changes embodied in each version of the protocol and (where appropriate), the reasons for these, in this
Please be aware that the document history is open to all, to verify changes and authorship:
Change
Section
Reason
Author
Affiliation
Time
Stamp
Example: Modified the
Change Record
descriptions
1.1.
Table format works better
than a list
Mark
Nelson
Stanford
21/6/2013
Removed “with initial
focus on 38-G”
Title
Too narrow focus
KG
NPS
10/21/2013
Struck sections of
protocol;
throughout first pass at simplifying
protocol document; see
Google doc revisions history
MSQ
Stanford
10/24/2013
Updated table
table
KG
NPS
10/29/2013
Clarify content
Background
NPS/IMSG/Stanford’s Governance Innovation for Security and Development project’s preliminary needs, in order
Specifically, we need a fast initial survey of literature on Civil Affairs/Military Governance broadly speaking, the
ule of law, safe and secure environment, sustainable economy, social well-being) and homeland integration.
rvey--and update-- the current state of civil affairs and military governance in the various sectors?
addressed
CA/MG?
urnal articles, case studies, manuscripts)?
ous stakeholders and ourselves for scholarly work?
f military operations is well-known to be vital to mission success in locales as diverse as Afghanistan, the Balkans,
remains little understood. Military support to governance in particular has suffered from years of neglect due to
military communities – despite specific responsibilities under international law and US DoD polcy - are skittish
upation” and wary of “military government” responsibilities that stem from it; the literature on governance is largely
ttention to or understanding of military norms and rules; the literature on military operations is generally focused
ttention to or understanding of civil domain factors. In short, the literatures that are available do not effectively
ver, the subject area is extremely complex, as state-society relations and expectations vary from place to place, and
cluding new roles for social media in civic participation and empowerment, and new ways of monitoring political
ng developed and/or refined.
e, a significant knowledge gap in civil affairs, is as complex as it is essential. The need for understanding military
cularly acute today, given the number of missions and organizations now concerned with civil-military planning and
omain has become much more diverse with the advent of new technologies – hardware and software – that enable
ize, and share information. Civil Information is a core task for Civil Affairs. Today, civil affairs are at the forefront of
ssion of the United States military and its relationship to civilian agencies.
Search Strategy
Five steps:
1. Broad Sampling
The first stage is broad sampling from search engines such as ProQuest, Lexis, JSTOR, Google
Scholar, NPS Bosun, to define an initial corpus of published articles, working papers, and reports.
Automated search terms used (initial proposed)
1. civil affairs; military governance.
2. security, stability, etc.
3. governance, etc.
4. social well being, humanitarian assistance, disaster relief, social capital, etc.
5. rule of law, justice, etc.
6. economy, infrastructure, reconstruction, etc.
7. human behavior, etc.
8. civil information management, technological enablers, etc.
As researchers extend the review with ancillary searches and interviews, they should also add
search terms
Compound automated search terms used (initial proposed)
Compound terms here are key words that are not relevant in and of themselves but serve as useful
modifiers and constraints:
 civil society, capacity building
Please record results of any prototyping of the search strings in the table below:
Search String Variants Search Engine or Library Results
2. PDS Framework
Then, we focus consideration on the population, dimension, and study design (PDS Framework).
Specifically, to pass to the next stage the articles need to meet three criteria:
2.1
Is the study relevant for understanding the GISD topic of interest?
2.2 Is the topic (military government/social well-being, etc.) independent or outcome variable in
the study?
2.3
Is the study original and not a review of original studies?
3. Study Eligibility
The studies that pass the second stage are then reviewed full text to establish the eligibility. In this
stage we take into consideration the following characteristics:
3.1
3.2
3.3
Does the study use data including civil affairs or its synonyms?
Does the study use a documented/recognised data/evidence source for civil affairs?
Is CA/MG (or synonyms) independent or outcome variable in the study?
4. Source Evaluation
Then, we evaluate the sources as follows:
4.1
Does the study have a clearly set out theoretical framework linked to qualitative evidence?
(For case studies)
4.2
Does the study have a clearly set out empirical framework tested with quantitative
evidence? (For empirical studies)
5. Validity, reliability and applicability of the studies (VRA)
The final stage assesses the validity, reliability and applicability of the studies (VRA). The validity
aims to assess the methodological rigour; reliability assesses the extent to which the findings of the
study are robust and reproducible; and applicability captures whether the findings can be
generalised to other countries. Five questions guide the process:
5.1
Validity of construct: Is the role of civil affairs theorised/modelled coherently and in the
light of existing literature?
5.2
Validity of method: Is the method of analysis informed by existing theory/theories?
5.3.1
Reliability of data/evidence (1): Is data/evidence documented and its reliability discussed?
5.3.2 Reliability of data/evidence (2): Is the evidence related to causal mechanisms postulated in
the analysis?
5.3.3 Reliability of findings: Are the findings related to relevant methodological or
theoretical/analytical literature?
The studies that passed the final stage are included in the review.
Time period covered by this review


The CA/MG general review will cover historical documents from 1776 to the present
Topical reviews will cover the period from 2000 to the present, exceptions will be noted
Synthesis
on 1 - Cross-tabulation
Initially we propose to apply a cross-tabulation survey structure as follows, extensible at all
levels and allowing for both additional detail and additional category types as needed. We
will then regularly review and optimize the current analysis, structure, and methodology, as
part of each future iteration of the survey.
Missions
SCA/TSC/TA
What is
MSG
Gov
Sec
Econ
SOC
RoL
HI
Maritime
ity
Stakeholders
38G skills and
certifications
Gaps Tech/Beh Metrics
y
Study Limitations
Reporting
studies in other domains
Schedule
Work Period
Time estimates for each major step
By 31 October
 Search Strategy
 Selection Criteria
By 10 December
 Study Quality Assessment
By 13 January
 Summary review