Protocol Outline
Title of Dataset: Water Quality for Unconventional Shale Oil & Gas Development
Date of Protocol: June 16, 2013, August 16, 2013
Goal: To compile statutes, rules, and regulations pertaining to water quality that is relevant to the
development of unconventional shale oil/gas.
I.
Primary Data Collection
a. Data collection activities: May 20, 2013 to August 16, 2013
b. Dates covered in the data set: May 1972 to March 1, 2014
c. Data Collection Methods
i. During the summer of 2013, six researchers (four law students and two
undergraduates) worked on this project. With 11 states in the original
research project, the research duties were divvied up with four of the
researchers taking on two states each, two researchers teamed up on two
states, and the project leader had one state. Upon completion of primary
research, researchers swapped states and conducted redundant research.
d. Search terms and data bases used
i. Due to the wide scope of examining water quality regulations pertaining to
oil and gas development, search terms were highly specialized to each
identified segment of oil and gas development category: Permitting,
Design & Construction, Well Drilling, Well Completion, Production and
Operation, and Reclamation. The search terms used for each category and
questions are listed below with the search terms on the far right column of
the table. Several legal databases were used LexisNexis and WestLaw.
Also, databases unique to each state were also utilized:
1. Colorado: http://cogcc.state.co.us/;
http://www.lexisnexis.com/hottopics/colorado/
2. Montana: http://bogc.dnrc.mt.gov/rulesregs.asp;
http://leg.mt.gov/bills/mca_toc/
3. North Dakota: https://www.dmr.nd.gov/oilgas/rules/rulebook.pdf;
& http://www.legis.nd.gov/information/acdata/html/Title43.html;
http://www.legis.nd.gov/general-information/north-dakotacentury-code
4. New Mexico:
http://www.nmcpr.state.nm.us/nmac/_title19/T19C015.htm;
http://public.nmcompcomm.us/nmpublic/gateway.dll/?f=templates
&fn=default.htm
5. New York: http://www.dec.ny.gov/regulations/regulations.html,
Chapter V - Resource Management Services & Chapter X Division of Water;
http://public.leginfo.state.ny.us/menugetf.cgi?COMMONQUERY
=LAWS
6. Ohio: http://codes.ohio.gov/
7. Pennsylvania:
http://www.legis.state.pa.us/WU01/LI/LI/US/HTM/2012/0/0013.H
TM; http://www.pacode.com/secure/data/025/chapter78/chap78toc
.html
8. Texas: http://www.rrc.state.tx.us/rules/rule.php;
http://www.statutes.legis.state.tx.us/
9. Utah: http://www.rules.utah.gov/;
http://le.utah.gov/UtahCode/title.jsp
10. West Virginia: http://www.legis.state.wv.us/WVCODE/Code.cfm;
11. Wyoming: http://wogcc.state.wy.us/;
http://legisweb.state.wy.us/titles/statutes.htm
e. Search Terms for Each Category
Search Terms: Water Quality – Permitting, Design, and Construction
Public
Description
Search Terms
1(P)
Is baseline
water source
testing
required?
Yes
Yes, but only for injection
wells
Only In Specific
Jurisdictions
No
Baseline water
source testing
“baseline”
“monitoring”
“testing”
“sampling”
“water”
1.1(C)
What location
requirements
for water
source testing
are specified?
None
Distance from Operations
Other
Location
requirements
for water source
testing
“baseline”
“monitoring”
“testing”
“sampling”
“water”
1.2(C)
What water
sources must
be tested?
Not specified
Water Wells
Springs
Surface Water
Other
Water sources
to be tested
“baseline”
“monitoring”
“testing”
“sampling”
“water”
Question
Question
Possible Answers
Water source
testing
parameters
“baseline”
“monitoring”
“testing”
“sampling”
“water”
“chemical”
“biological”
“physical”
Water source
testing
techniques
“baseline”
“monitoring”
“testing”
“sampling”
“water”
Follow-up water
source testing
“baseline”
“monitoring”
“testing”
“sampling”
“water”
Comparison to
baseline testing
“baseline”
“monitoring”
“testing”
“sampling”
“water”
Yes
No
Public
disclosure of
test results
“baseline”
“monitoring”
“testing”
“sampling”
“water”
“disclosure”
“presumption of
liability”
“liability”
“presumption of
liability”
“liability”
1.3(C)
What
parameters for
water source
testing are
specified?
1.4(C)
What
techniques for
water source
testing are
specified?
1.5(C)
Is follow-up
water source
testing
required?
1.5.1(G)
Does follow-up
testing use
identical
requirements to
baseline
testing?
1.6(C)
Are water
source testing
results publicly
disclosed?
2(P)
Is there a
presumption of
liability against
operators when
pollution is
found in a
nearby water
source?
Yes
No
Presumption of
liability for
pollution of
nearby water
sources
2.1(C)
At what
distance from
oil and gas
operations
0 feet - 1,500 feet
1,501 feet - 2,500 feet
2,501 feet - 5,000 feet
More than 5,000 feet
Presumption of
liability distance
Not Specified
Chemical Indicators
Biological Indicators
Physical Indicators
Not Specified
Number of Samples
Method Used
Yes
No
Yes
No
does this
presumption
apply?
Variable
2.2(C)
What are the
defenses/exem
ptions for
operator
liability?
Baseline testing
Water source owner
refused baseline testing
Other
Defenses/excep
tions for
presumption of
liability
“presumption of
liability”
“liability”
“defense”
“exemption”
3(P)
What
subsurface
features are
operators
required to
identify?
Other
Aquifers
Seismic Hazards
Subsurface
feature
identification
“aquifer”
“underground”
“subsurface”
“seismic”
3.1(C)
Are there any
exemptions?
Yes
No
Subsurface
identification
exemptions
“aquifer”
“underground”
“subsurface”
“seismic”
4(P)
What are the
siting
restrictions for
oil and gas
wells?
Other
Exclusion Areas
Water Source Setback
Restrictions
Well-density Restrictions
Oil and gas well
siting
restrictions
“siting”
“spacing”
“well-density”
“setback”
“location”
4.1(C)
Are there
specific siting
restrictions if
the well is to be
hydraulically
fractured?
Well fracturing
siting
restrictions
“siting”
“spacing”
“well-density”
“setback”
“location”
“hydraulic
fracture”
“horizontal”
Yes
No
4.2(C)
Are there any
exceptions/wai
vers to the
siting
restrictions?
Yes
No
Exceptions/waiv
ers to well siting
restrictions
“siting”
“spacing”
“well-density”
“setback”
“location”
“exemption”
“waiver”
5(P)
What are the
siting
restrictions
None
Other
Exclusion Areas
Material storage
siting
restrictions
“pit”
“tank”
“pool”
specific to pits,
holding tanks,
and other
material
storage?
“pond”
Water Source Setback
Restrictions
5.1(C)
Are there any
exceptions/wai
vers to these
restrictions?
Yes
No
Exceptions/waiv
ers to material
storage siting
restrictions
“pit”
“tank”
“pool”
“pond”
6(P)
What are the
siting
restrictions
specific to
injection wells?
Other
Exclusion Areas
Water Source Setback
Restrictions
Well-density Restrictions
Injection well
siting
restrictions
“injection”
“class II”
6.1(C)
Are there any
exceptions/wai
vers to these
restrictions?
Yes
No
Exceptions/waiv
ers to injection
well siting
restrictions
“injection”
“class II”
7(P)
Is there any
limitation on the
size of the well
pad
disturbance?
Well-pad size
limitations
Oil and Gas,
Disturbance,
acre/mile, limit,
well site, well
pad, size
8(P)
Other design
limitations?
Design
limitations
Oil and Gas,
Prohibited, Ban,
Limit,
9(P)
What water
quality
protection
technologies
and practices
are
recommended
or required?
Closed-Loop Drilling
Pit Liners
Other Best Management
Practices
Storage Tank
Requirements
Other
None Identified
10(P)
Who must
operators notify
as part of the
permitting
process?
State Regulatory Agency
Local Government
Surface owners
Mineral Owners
Neighboring Landowners
Yes
No
Yes
No
Oil and Gas,
Closed-Loop
Drilling, Holding
containers, Best
Required
Management
technologies/pr
Practices,
actices
Storage Tank, Pit
liner,
Recommended
practices
Operator
notification
requirements
“notify”
“notification”
Other
11(P)
Is the source or
type of water to
be used in
operations
regulated?
Yes
No
Water used in
operations
12(P)
Do operators
need a
stormwater/ero
sion mitigation
plan as part of
the permitting
process?
Yes
Varies by Size of
Development
No
Stormwater/ero
sion mitigation
“stormwater”
“erosion”
13(P)
Is a
reclamation
plan required to
receive a
permit to drill?
Yes
No
Reclamation
plan
requirement for
permit
“reclamation”
“abandonment”
“plugging”
“permit”
Search Terms: Water Quality – Well Drilling
Question
Number
Question
1(P)
Does your state specify well-bore
integrity standards for drilling?
2(P)
Does your state require
verification that well-bore surface
casing was implemented
properly?
3(P)
Are operators required to use air
or freshwater-based drilling fluid
when drilling through fresh water
aquifers?
Possible Answers
Yes
No
Yes
No
Air
Freshwater-based drilling
fluid
Both
Neither
Public
Description
Search
Terms
Specifies wellbore integrity
standards
well bore,
well-bore,
drilling
Surface casing
surface
casing
Drilling through
fresh-water
aquifers
groundwater,
drinking
water, fresh
water,
aquifers, air
cooling,
drilling fluid
4(P)
When is well-bore cementing
required?
4.1(C)
Is stage cementing allowed as an
alternative when standard cement
casing is uneconomical?
5(P)
Is blowout prevention equipment
(“BOPE”) required?
6(P)
Are cement-bond logs required to
be recorded and filed?
6.1(C)
With whom must the logs be
filed?
7(P)
Does your state specify
mechanical integrity testing
requirements?
7.1(C)
Is annulus pressure monitoring
and testing required?
Subsurface conditions
unknown
Subsurface conditions
known
Other
Yes
No
Yes
No
Yes
No
Regulatory authority
Well operator
Other
Yes
No
Yes
No
Conditions
necessitating
cement casing
well-bore
cementing,
casing
Stage cementing
Stage
cementing
Blowout
prevention
equipment
blowout
prevention
equipment,
BOPE
Cement-bond logs
cement-bond
logs,
recording,
monitoring
Where to find the
logs
monitoring,
recording,
[name of
state
regulatory
authority],
cement bond
logs
Mechanical
integrity testing
mechanical
integrity,
standards,
testing,
monitoring
Annulus pressure
testing
Annulus,
annular,
pressure,
testing,
monitoring
7.2(C)
Does the state require
compliance with American
Petroleum Institute
recommending practices for
testing?
8(P)
Does your state specify a
compliance reporting process
during well drilling?
Yes
No
API RP
compliance
American
Petroleum
Institute,
recommendin
g practices,
API RP,
testing
Yes
No
Compliance
requirements
reporting,
compliance,
monitoring
Who is
responsible for
compliance
reporting,
compliance,
monitoring,
(name of
regulatory
authority)
Horizontal-drilling
requirements
Horizontal
drilling
Horizontal-drillingrequirement
specifications
horizontal
drilling, well
site spacing,
bottom-hole,
permitting
Operator files reports
Regulators inspect well
site
Other
8.1(C)
What is the process?
9(P)
Does your state specify horizontal
drilling requirements?
9.1(C)
What horizontal-drilling
requirements must an operator
follow?
Yes
No
Spacing requirements
Bottom-hole requirements
Permitting requirements
Search Terms: Water Quality – Well Completion
Question
Number
1(P)
Question
Is public disclosure of
chemicals/additives in
fracturing fluid
required?
Possible
Answers
Yes
No
Public
Description
Search Terms
Disclosure of
chemicals in
fracturing fluid
Detailed information
Stimulation fluid source
Additive type
Concentration rates
Chemical compound
name
Chemical Abstract
Service (CAS) number
Percentage by weight
Percentage by volume
Pounds per gallon
Gallons per thousand
gallons
Parts per million / parts
per billion
Disclosure
Trade names
Hydraulic fracturing
fluid
Disclosure
Oil and Gas
1.1(C)
When are operators
required to disclose
chemicals/additives in
fracturing fluid?
Before Use
Within 30 days of
use
Within 31-60 days
of use
Other
Timing of
chemical
disclosure
1.2(C)
Are operators required
to report the total
volume of fracturing
fluids used?
Yes
No
Fluid volume
reporting
1.3(C)
Is disclosure of
concentration of
chemicals/additives in
the fracturing fluid
required?
Overall
Chemical/Additive
Concentration
Individual
Chemical/Additive
Disclosure of
concentration of
chemicals/additiv
es in fracturing
fluid
Before well simulation
As soon as possible
No later than
Days following
completion
Days after completion
Detailed information
Stimulation fluid source
Additive type
Concentration rates
Chemical compound
name
Chemical Abstract
Service (CAS) number
Percentage by weight
Percentage by volume
Pounds per gallon
Gallons per thousand
gallons
Parts per million / parts
per billion
Disclosure
Oil and gas
Hydraulic fracturing
Fracturing fluid
Well completion
Proposed
concentration
Each additive
Overall concentration
Percentage by weight
Concentrations
None
1.4 (C)
Is there an exemption
from disclosure
requirements for trade
secrets?
Yes
No
1.4.1 (G)
Who may access the
trade secret
information?
Oil & Gas State
Agency
Other State
Agency
Health Care
Professionals
Other
Nobody
1.5 (C)
How does the public
access disclosure
information?
State website
Fracfocus.org
Other
2(P)
Does your state
regulate the reuse
and/or recycling of
flowback/produced
water at well
completion?
Encourages reuse
Requires reuse
Specifies
conditions of
reuse
No
Percentage by volume
Pounds per gallon
Gallons per thousand
gallons
Parts per million / parts
per billion
Disclosure
Oil and gas
Hydraulic fracturing
Fracturing fluid
Well completion
Exemptions for
trade secret
Disclosure to
specific agencies
or others
Public access to
disclosure
information
Reuse or
recycling of water
Trade Secret
Privileged information
Confidential
Protection
Exempt
Trade secret
Emergency
Trade secret exemption
Public health official
Diagnosis or treatment
Health care
State agency
Oil and gas
commission
Public access
State website
Fracfocus
Access information
Feasible
Solids removal
equipment
Drilling fluid waste
Encourages the
recycling of drilling
Drilling fluids intended
for recycling
Recycle of flowback
Reuse for fracturing
operations
Well completions
Oil and gas
Hydraulic fracturing
fluid
2.1(C)
3(P)
4.1(C)
Does your state
require reporting of the
volume of
reused/recycled
water?
Does the state have
requirements for the
collecting, holding,
and disposing of
flowback water?
What possible
methods of disposal
does the state allow
for flowback water?
Reporting of
water
reuse/recycle
Yes
No
Yes
No
Class II well
Commercial waste
disposal facility
Land
treatment/applicati
on
Pits/tanks
Total volume of
recycled
Collecting,
holding, and
disposing of
flowback water
Well stimulation fluid
Hydraulic fracturing
fluid
Storage
Protective of
groundwater
Tanks
Lined pits
Migratory birds
Oil and gas
Possible
methods of
disposal of
flowback water
Class II well
Commercial waste
disposal facility
Land
treatment/application
Pits
Tanks
Road spreading
Land farming
Disposal of well
stimulation fluid
Search Terms: Water Quality – Production & Operations
Question
Number
1(P)
Question
Does the state have ongoing
well inspection/monitoring
requirements?
Possible
Answers
Yes
No
Public Description
Search Terms
Well inspection
requirements
(Westlaw) “oil
and gas,
inspection,”
(within
regulations)
“inspections,”
“monitoring”
1.1(C)
Does the state have
frequency requirements for
well inspections?
Yes
No
Well inspection
frequency requirements
“well
inspection,
frequency,”
“frequency
requirement”
“inspections,”
“monitoring”
1.2(C)
Does the state have
requirements for selfinspection/monitoring of
wells?
Yes
No
Self-inspection
requirements
“inspection,”
“reporting,”
“monitoring”
2(P)
What authority does the
state’s governing agency
have regarding
inspection/monitoring of oil
and gas sites?
Right to inspect
Right of entry
Access to well
records
Other
Agency authority to
inspect
“inspections,”
“agency
authority to
inspect”
Does the state regulate the
reuse and/or recycling of
produced water during well
operations?
Encourages reuse
Requires reuse
Specifies
conditions of
reuse
No
Reuse or recycling of
water
“oil and gas,
recycled
water,” “oil
and gas,
produced
water”
Reporting of water
reuse/recycle
“oil and gas,
water volume,”
“oil and gas,
measurement
of water”
Collecting, holding, and
disposing of produced
water
Drilling,
disposal
requirements,
Collecting
Holding
Produced
water
3(P)
3.1(C)
4(P)
Does your state require
reporting of the volume of
reused/recycled water?
Does the state have
requirements for the
collecting, holding, and
disposing of produced
water?
Yes
No
Yes
No
Methods of
collecting/holding
produced water
drilling,
disposal
requirements,
produced
water,
pits, tanks,
pipeline
4.2(C)
What methods of disposal
does the state allow for
produced water?
Class II Injection
Well
Evaporation
Centralized
Waste Treatment
facility
Publicly Owned
Treatment Works
Roadspreading
Other
Disposal of produced
water
drilling,
disposal
requirements,
Class II
Injection Well
Evaporation
Centralized
Waste
Treatment
facility
Publicly Owned
Treatment
Works
Roadspreading
5(P)
Does the state have
requirements for reporting
spills and accidents?
Yes
No
Spills and accident
reporting requirements
oil and gas,
spills and
accident
reports
5.1(C)
Does the state require
notification be given to the
surface owner after a spill?
Notification to surface
owner after spill
oil and gas,
spills and
accident
reports,
notification
5.2(C)
Is the type of notification
dependent on the type or
volume of the spill or
accident?
Yes
No
Notification dependent
on volume/type of spill
oil and gas,
spills and
accident
reports,
notification
5.3(C)
Is the reporting timeframe
Yes
Reporting time frame
oil and gas,
4.1(C)
What methods of
collecting/holding produced
water does the state allow?
Pits
Tanks
Pipelines
Other
Yes
No
dependent on the type or
volume of the spill or
accident?
5.4(C)
Are reporting requirements
dependent on the type or
volume of the spill or
accident?
6(P)
Does the state have
requirements for the
storage, handling,
transportation, treatment,
recycling, or disposal of
Exploration and Production
waste?
No
spills and
accident
reports,
notification
Yes
No
Reporting requirements
dependent on type of
spill
oil and gas,
spills and
accident
reports,
notification
Yes
No
Exploration and
Production Waste
Management
“exploration
and production
waste,” “waste
disposal”
Search Terms: Water Quality – Reclamation
Question
Number
1(P)
Question
Is interim reclamation
required?
Possible Answers
Public
Description
Search Term
Yes
No
Interim
reclamation
Oil, gas, interim,
reclamation,
restoration,
disturbed,
remediate: These
baseline search
terms could/should
be used for every
child question
below.
abandoned, shut-in
wells
minimum
1.1(C)
Is interim reclamation
required for temporarily
abandoned or shut-in wells?
Yes
No
Interim
reclamation
during
temporary
abandonment
1.2(C)
What is the minimum
No time given
Minimum
timeframe within which
interim reclamation must
begin?
1.3(C)
Is removal of debris
required?
Dependent on life cycle
Less than 3 months
3 - 6 months
Greater than 6 months to 1
year
Greater than a year
timeframe to
begin interim
work
timeframe, land,
surface, plugged,
days, months, year
Yes
No
Debris
removal
during interim
reclamation
debris, removal
backfill, pits, holes
1.4(C)
Is backfilling of unnecessary
holes and pits required?
Yes
No
Unnecessary
holes & pits
backfilled
during interim
1.5(C)
Must grasses, forbs, shrub
and/or trees be planted?
Yes
No
Planting
during interim
reclamation
planting,
vegetation,
grasses, forbs,
shrub, tree
1.5.1(G)
Do the grasses, forbs, shrub,
and/or trees have to be
native to the surrounding
area?
Yes
No
Native
plantings
during interim
reclamation
planting,
vegetation,
grasses, forbs,
shrub, tree, native,
indigenous
Reclamation
plan
requirement
for permit
Oil, gas, interim,
reclamation,
restoration,
disturbed,
remediate,
application for
permit to drill, plan
2(P)
Is a reclamation plan
required to receive a permit
to drill?
3(P)
Are financial assurances
required for new wells in
case proper reclamation is
not achieved?
Yes
No
Financial
assurances
required
Oil, gas, financial,
security, bond,
assurance, surety,
reclamation,
restoration,
remediate
What is the financial
assurance required for a
single new well?
$0
$1-5,000
$5001-10,000
$10,001-20,000
More than $20,000
Financial
assurance
required for a
single well
Oil, gas, financial,
security, bond,
assurance, single
new well
3.1(C)
Yes
No
3.2(C)
What is the financial
assurance required to cover
the entire state?
$0; $1-5,000
$5001-10,000
$10,001-20,000
More than $20,000
Financial
assurance
requirement
for statewide
coverage
Oil, gas, financial,
security, bond,
assurance,
statewide, blanket
3.3(C)
Is the financial assurance
uniform regardless of the
type of new well?
Yes
No
Uniform
financial
assurances
Oil, gas, financial,
security, bond,
assurance, uniform
Oil, gas, final,
reclamation,
restoration,
disturbed,
remediate,
plugged,
abandoned: These
baseline search
terms could/should
be used for every
question below.
Are final reclamation tasks
required when a well is
plugged and/or abandoned?
Yes
No
Final
reclamation
tasks
required
4.1(C)
What is the timeframe for all
final reclamation tasks to be
completed?
No time given
Less than 3 months
3-6 months
Greater than 6 months to 1
year
Greater than a year
Time to
complete final
reclamation
tasks
timeframe, tasks
4.2(C)
Is removal of debris
required?
Yes
No
Debris
removal
during final
reclamation
debris, removal
backfill, pits, holes
4(P)
4.3(C)
Is backfilling of unnecessary
holes and pits required?
Yes
No
Unnecessary
holes & pits
backfilled
during final
reclamation
4.4(C)
Must grasses, forbs, shrub
and/or trees be planted?
Yes
No
Planting
during final
reclamation
planting,
vegetation,
grasses, forbs,
shrub, tree
4.4.1(G)
Do the grasses, forbs, shrub,
and/or trees have to be
native to the surrounding
Yes
No
Native
plantings
during final
planting,
vegetation,
grasses, forbs,
4.5(C)
4.6(C)
II.
area?
reclamation
Does the state require an
evaluation of reclamation
efforts before releasing the
bond?
Evaluation
before
releasing
financial
assurance
evaluation, release,
return, forfeiture,
state inspection
Factors when
evaluating
final
reclamation
evaluation, release,
return, forfeiture,
state inspection,
vegetative cover,
percentage, weed
control, stormwater
management,
water, pits, holes,
backfilled, filled,
removed
What factors are considered
when evaluating final
reclamation?
Yes
No
Vegetative covers of
disturbed lands
Weed control
Stormwater controls
Removal or burial of pits
Other
Not specified
shrub, tree, native,
indigenous
Coding
a. Background information
i. The initial two weeks of the research project consisted of providing a
framework and opportunities for the researchers to gain subject matter
knowledge. An introductory meeting was held with the six researchers to
provide information about the scope of the project and hand out initial
background reading assignments about unconventional oil and gas
development. The entire team then received a presentation from a
University of Colorado Law professor about shale gas and oil
development and impacts. Each research was then tasked with collecting
grey literature pertaining to water quality and shale oil/gas development.
ii. Throughout the course of the research project, Google Drive was utilized
to collect and share information. A general outline of the stages of oil and
gas development and potential statutory and regulatory subjects was built
for the researchers. The outline was a starting point in order to identify
and develop proper water quality questions. The stages and potential
statutory and regulatory subjects had been developed in coordination with
our partners, the Environmentally Friendly Drilling program. Six stages
were initially identified, and each stage was assigned to a researcher. The
potential statutory and regulatory subjects for each stage were analyzed by
the researchers to determine their adequacy; subjects were added and
removed depending on that research. Potential statutory and regulatory
subjects were presented to entire group and finalized. At this point, each
researcher was assigned a number of states and began collecting statutes
and regulations for their states that corresponded with the finalized
subjects. Upon completion of this initial collection, the researchers
swapped states and checked, at a minimum, five questions per state as a
quality control measure. Simultaneously, while this work was being
conducted, each researcher was developing draft questions for the
LawAtlas database. The drafts were written utilizing the Workbench User
Guide. Finally, a day-long, group editing session was held to finalize the
questions for the database. A group session was held to finalize the
questions because each of the researchers had spent considerable amount
of time concentrating on one stage of the oil and gas development process
and because the process of collecting statutes and regulations for
individual states had provided a basis for the nuances of regulatory
structure across jurisdictions and development stages.
iii. Once the questions were coded and entered into LawAtlas, each researcher
began transferring the appropriate statutes and regulations into the legal
text box and answering the questions. During this process, questions and
answers were edited with consent from the entire group. The nuances of
state laws and regulations pertaining to water quality required tinkering
with the questions and answers in order to properly capture the
information.
b. Coding methods
i. The coding is done on an individual basis with each researcher responsible
for assigned states. Upon completion of coding, the researchers swapped
states and a minimum of five questions were randomly double checked. If
concerns arose during this portion, questions and suggestions were shared
with the entire team. Weekly meetings were also held to hammer through
any discrepancies and concerns.
c. Test with Naïve Coder
i. Naïve coders were brought on toward the end of the coding process to
code 13 randomly selected records (15% of 88 records). Once the naïve
coding was finished, the project leader analyzed the divergences between
the original coder and the naïve coder. Conversations were held with the
naïve coders to better assess the divergences and clarification to questions
and answers followed.