Research Methods for the CREW Diagnostic
Country Reports, August 9, 2013 Zambia
Nathan Economic Consulting India Pvt. Ltd.
 This study aims to assess the state of competition and benefits of
competition reforms on consumers and producers in selected product
markets and countries, especially focusing on the impact that past and
NATHAN INDIA
Introduction
existing public policies, regulations, private sector business practices,
institutional arrangements and other factors have had on consumer and
producer welfare*.
 The goal of the project is to better demonstrate measurable benefits from
effective competition reforms in DCs, for ensuring long-term support for
competition.
*Due to discussions and concerns about the term “producer welfare” we recommend
using the term producer benefits.
2
 The first phase of this project is the Diagnostic phase, which started with the
selection of four countries and two sectors for the study:
Country
Staple Food
Ghana
Maize/Rice
India
Wheat
Philippines
Rice
Zambia
Maize
Bus Transport
NATHAN INDIA
Introduction: Three Phases
Inter-City transport
and Intra-City
transport routes
 The next stage in the Diagnostic phase is to identify a standardized framework in
which the analysis is to be conducted through Diagnostic Country Reports (DCRs).
 In the Design phase, developing the toolkit to do the impact analysis.
 The impact analysis will be done and quantified in the Validation phase.
3
Diagnostic
Design
NATHAN INDIA
We are currently still in the Diagnostic Phase
Validation
4
NATHAN INDIA
Conceptual Framework for Phase I
5
1.
At the first level it will provide a detailed background on the market structure of the 2
sectors in the 4 countries, including the regulatory structure, using the DFID
Competition Assessment Framework to answer questions about the competitive
NATHAN INDIA
Objectives of the DCR
landscape of the market.
2.
Next, the DCR will provide a criteria for identifying the competition-enhancing reforms
that have already taken place, or could take place, which would promote competition.
3.
Third, after identifying the reforms, the DCR will provide the analysis framework to:
1.
define the relevant market,
2.
identify players and stakeholders who will/could be affected including both
consumers and producers,

3.
frame the hypotheses for testing and define the counterfactual,.
4.
identifying the data required, data sources, and methods to test the hypotheses.
It is important to remember that the DCR just provides a framework and the actual analysis comes
later.
6
 The DFID Competition Assessment Framework will be the basis for assessing the competitive
landscape in the chosen sectors in each of the four countries.
 Identifying the markets and competitors
NATHAN INDIA
Step 1-Competition Analysis
 Examining the market structure
 Looking for barriers to entry (or exit)
 Looking for anti-competitive conduct
 Considering vested interests and the principal beneficiaries
 Identifying government policies or institutions that limit competition
 Use “Porter’s Five Forces of Competition” to recommend industries’ strengths and weaknesses:
1. Nature of competition in the industry
2. Potential of new entrants into industry
3. Power of suppliers
4. Power of customers
5. Threat of substitute products
7

Identify elements of competition reforms to be examined in each sector starting
with identifying reforms which are targeted or likely to have an impact on
competition. “Reforms” include:

Enabling government policies that promote competition in markets.

Appropriate regulatory framework and institutions for promoting competition in
NATHAN INDIA
Step 2-Identifying Competition Enhancing Reforms
sectors including sector-specific regulations


Effective competition enforcement institutions
The criteria for selecting the reforms/recommended reforms will be based on the
OECD’s Competition Assessment Toolkit looking at whether the
reforms/recommended reforms:

o
Remove barriers to entry and increase number and range of suppliers
o
Allow suppliers to compete
o
Incentivize suppliers to compete
o
Increase investment
Ex-post vs Ex-ante analysis
8
Defining the relevant market
 The relevant market has to be defined so that impact analysis can be focused.
 Definition of the relevant product market
NATHAN INDIA
Step 3 – Impact Analysis Framework
 Definition of the relevant geographic market
 Will likely be a qualitative analysis
 Next, identify the stakeholders that could be affected:
 Consumers
 Focusing on only End Consumers
 Producers
 Incumbents
 New entrants
9
A non-exhaustive list for discussion:
CONSUMERS
PRODUCERS
OTHER
General public (preferably
differentiated by income group)
Farmers
Think Tanks/University Research
Centers
Exporters
Millers
Government regulatory bodies
Distributors (Public or private)
Farmers/Business Associations
Input Suppliers (such as fertilizer
or seed suppliers)
Pre-production: banks/MFIs
lending to farmers
NATHAN INDIA
Identification of Stakeholders – Staple Food
Consumer Advocacy Groups
10
CONSUMERS
PRODUCERS
OTHER
General Public
Bus operators
(Public and Private)
Regulatory Bodies
Input suppliers (e.g. tires, petrol/diesel)
Bus terminal operators
NATHAN INDIA
Identification of Stakeholders – Bus Transport
Think Tanks/University
Research Centers
Consumer advocacy groups
Map relevant stakeholders along the supply chain to clearly illustrate the
particular stage of the pre-production, production, or distribution stages where
the impact of a particular reform measure is likely to be felt.
11
A pertinent hypothesis should ask to what extent a reform measure affected
competition in the market, and how it affected consumers and producers.
Some potential hypotheses:
NATHAN INDIA
Determination of Hypotheses – Staple Food
Staple Food
NULL HYPOTHESIS FOR STAPLE FOOD
The effect of reform has no impact on the quantity of goods sold.
The effect of reform has no impact on the quality of goods sold.
Improvements in the quantity/quality of goods rendered have no impact on the health outcomes.
The reform has no impact on intermediaries’ (millers, traders, etc) business by sales volume.
A reduction (increase) in prices has no impact on the quantity of goods sold.
The reform has no impact in number of players in the market.
The effect of entry of a new player does not have an impact on the price of the goods provided.
The effect of entry of a new player does not have an impact on the quality of the goods provided.
The reform has no effect on innovations in the sector.
12
Bus Transport
NULL HYPOTHESIS FOR BUS TRANSPORT
The reform has no significant impact in the quality of service.
The reform has no significant impact on consumer satisfaction.
NATHAN INDIA
Determination of Hypotheses – Bus Transport
The reform has no significant impact in the frequency of services (inter/intra state).
The reform has no significant impact in the consumers income level/other demography indicators
The reform has no significant impact in the bus manufacturers/operators financial indicators.
The reform has no impact on the price of services rendered.
The reform has not reduced bus related injuries and fatalities.
It is also important to define the Counterfactual –would the result have
been the same regardless of the reform?
13
 Impacts on consumer welfare will focus on the end consumer
 On the producer side, the project will look at reforms throughout the
various levels of the supply chains
NATHAN INDIA
Definition of Benefits to Consumer and Producers
 Impact on producers should be measured as benefits accruing to them
from:
 Ease of entry,
 Ease of doing business,
 Operational efficiencies.
14
 Potential variables capturing consumer welfare and benefits to producers include:
BENEFITS TO CONSUMER/CONSUMER WELFARE
BENEFITS TO PRODUCERS
Access to goods/services
Access to essential services
Quality
Free movement of goods and services
Price changes
Predictability of regulatory actions
Choices of new goods/services
Cost savings
Time savings
Fair market processes
Access to financing
Productivity gains
NATHAN INDIA
Identification of Variables and Data to be Collected
Level-playing field
Transparency in market
Profit
15
 Potential sector -wise data to collect from primary and secondary sources:
STAPLE FOOD
BUS TRANSPORT
Price data
Price data
Costs of production
Costs of production
Quantities produced and sold
Frequency and Number of buses
Availability of substitute products
Number of passengers
Changes in health outcomes
Available roads and infrastructure
Number of distribution channels
Substitute products and their details
NATHAN INDIA
Identification of Variables and Data to be Collected
Number of players in the market
Changes / reduction in fatalities and injuries
Passengers per bus/capacity of bus
Length of bus routes
16
 Data is available from a combination of
 Secondary sources (census, government records, international statistics)
 Primary sources (survey, focus groups)
NATHAN INDIA
Data Collection - Data Sources
 Secondary sources may include:
• Competition Authority, if existent
• Ministry of Commerce/Trade
• Government censuses
• Business Associations
• International donor agencies such as the World Bank
• Local research institutes
• Consumer Protection Agencies
• Publicly available online sources
 If no secondary data are available, turn to primary data collection, such as surveys or focus
group discussions.
 Keep in mind the variables to be measured and budget/resource constraints
17
 A single methodology should be employed across countries when possible
 Probability sampling: random selection of sample elements from sampling frame
o
Simple Random Sampling
o
Random Sampling
o
Stratified Random Sampling
NATHAN INDIA
Primary Data Collection: Sampling Methods and Strategies
 Non-Probability sampling: should be used as a secondary option when probability sampling is
not possible
o
Accidental sampling (such as ‘man on the street’)
o
Snowballing
o
Purposive sampling
 A broad strategy including a stratified random sampling method, with clustering to control
costs if the geographic areas are large, could be utilized in this study.
o
Stratification divides elements into population subgroups, ensuring representation from
each; strata here could be defined based on region and urban/rural
o
Clustering lowers costs when populations are spread across large geographic areas
 For bus transport a non-probability sampling method might be most feasible
 Nationally representative sample vs. cost constraints
18
 The sample size should be large enough for the analysis to be statistically significant while
balancing survey and project costs.
 Formulas calculate the sample size (n) based on the following parameters:
o Population of the sampling frame (N)
NATHAN INDIA
Primary Data Collection: Sampling Size
o Degree of variance (P)
o Precision desired (α)
o Confidence interval (Z)
o Response rate (R)
 Below is an illustrative example:
EXAMPLE 1
EXAMPLE 2
100,000
500
Degree of variance (P)
0.5
0.5
Precision desired (α)
0.05
0.05
Confidence interval (Z)
1.96
1.96
Response rate (R )
0.7
0.7
Sample size (n)
547
310
Total population of people in the sampling frame (N)
19
 Primary data for the analysis will be collected through a survey.
 Questionnaires will be designed for both producers and consumers in each sector
o Slight modifications , including translation to local languages, will be made on a
NATHAN INDIA
Primary Data Collection: Survey Design
country-specific basis, but the questions in general should be similar across countries
 Survey canvassing via a combination of email, phone and in person interviews
 Data collection using either pen and paper interviewing (PAPI) or computer assisted
personal interviewing (CAPI).
o CAPI via PDAs is recommended
 Surveyor training
 Piloting
 Follow-up and spot checks for quality control
20
 Quantitative Analysis
o Time Series Variation
o Spatial Variation Analysis
NATHAN INDIA
Types of Analysis
o Difference-in-Difference
o Cost-Benefit
 Qualitative Analysis
o Focus Group Discussions
o In-depth Individual Interviews
o Case Studies
 Cross Country Analysis
21
 Time Series Analysis compares data before and after a regulatory reform is
introduced
o Data availability limitations
NATHAN INDIA
Quantitative Analysis Techniques
o Carefully choose data time periods
o Use econometric rigor to avoid estimation errors
• Spatial variation is a method that compares two sets of markets which are identical
or similar in all respects, except for the regulatory constraint introduced.
o Produces more precise estimates of the effect of policy reform
o Very data intensive.
o Often difficult to find two sample sets that are characteristically similar,
except for the regulatory reform being studied.
o Difference-in-Difference combines time series analysis with spatial variation to
compare two different markets at two time periods, with a reform occurring in one
o Use multiple regression techniques to control for external factors
22
 Regression analysis:
o Methods include ordinary least squares (OLS), two-stage least squares (2SLS),
and limited dependent variable techniques (logit or probit models).
NATHAN INDIA
Quantitative Analysis Techniques (cont’d)…
o Dependent variable such as price, rate of technical change
o Modeled as a function of exogenous independent variables
o Dummy variable indicates the “regulated” period; the sign and coefficient of the
dummy corresponds to the impact and size impact of the regulation.
 If data constraints don’t allow for regression analysis, other tests many be used:
 Parametric Tests
 Non-parametric tests
23
 Measures the impact of an intervention in terms of the
 Economic benefits it generates or gains to the economy relative to
 Actual and opportunity costs of the intervention or losses to the economy
NATHAN INDIA
Quantitative Analysis – Cost Benefit Analysis
 Steps to undertake
• Quantify the impacts, positive and negative, tangible and intangible
• If the reform has not yet occurred, estimate the likely impact of the expected
reform
• Calculate of social costs, both the tangible and the intangible.
• Determine which are attributable to the reform
• Convert all costs and benefits to USD
• Discount the future value of costs and benefits as they accrue over time
• Compare of the costs and benefits to determine the net social rate of return
• Use a qualitative approach whenever benefits would be intrinsic and not measurable
in monetary terms such as improvements in quality, innovations, and new products.
24
• Used in the absence of sufficient data to undertake a purely quantitative
evaluation
• Poses open-ended questions about reform strengths and shortfalls to provide a
NATHAN INDIA
Qualitative Analysis
source of new information on benefits or potential ideas for future reforms.
• Cover all levels of stakeholders
• Focus Group Discussions (FGD) and In-depth Individual Interviews
• Discussions with an individual or sample group in order to understand
their in-depth experiences
• Provides insight explaining the results of the quantitative analysis
• Provide a basis for case studies
• Case study: an individual study highlighting the experience of an individual,
group, or company to give context and a personal touch to the numbers
25
Data unavailability
Cost of collecting nationally representative data samples
How to measure:
NATHAN INDIA
Potential Issues
 Improvements in quality, innovations, and new products
 Tradeoffs between price and quality
 Tradeoffs between producer and consumer welfare
Determining pass-on of benefits between various levels of consumers
How to treat transfers of producer to consumer surplus and how to define
net welfare
Weighing short-run vs. long-long tradeoffs of welfare
Controlling for external factors
Attributing impact to a certain reform
Access to consistent measures across countries
26
Nathan India
Nathan Economic Consulting India Private Ltd.
Chennai
No. 45, George Ponnaiya Building,
TTK Road, Alwarpet,
Chennai 600 018
T: +91 44 4293 7700
F: +91 44 4293 7777
Nathan Associates Inc.
2101 Wilson Boulevard
Arlington, VA 22201
Washington DC Metro Area
T: 703-516-7700
F: 703-351-6162
e-mail: nathanindia@nathaninc.com
web: www.nathaninc.com
New Delhi
Level 4, Rectangle 1,
Commercial Complex D-4, Saket,
New Delhi 110 017
T: +91 11 4051 4076
F: +91 11 4051 4052
NATHAN INDIA
Contact Us