Is Mandatory Nonfinancial
Performance Measurement
Beneficial?
Susanna Gallani
Ranjani Krishnan
Takehisa Kajiwara
Working Paper 16-018
Is Mandatory Nonfinancial Performance
Measurement Beneficial?
Susanna Gallani
Harvard Business School
Takehisa Kajiwara
Kobe University - Japan
Ranjani Krishnan
Michigan State University
Working Paper 16-018
Copyright © 2015 by Susanna Gallani, Takehisa Kajiwara, and Ranjani Krishnan
Working papers are in draft form. This working paper is distributed for purposes of comment and discussion only. It may
not be reproduced without permission of the copyright holder. Copies of working papers are available from the author.
Is Mandatory Nonfinancial Performance Measurement Beneficial?
SUSANNA GALLANI
Michigan State University
gallani@bus.msu.edu
TAKEHISA KAJIWARA
Kobe University - Japan
kajiwara@people.kobe-u.ac.jp
RANJANI KRISHNAN*
Michigan State University
krishnan@bus.msu.edu
Acknowledgments: We thank Jeff Biddle, Clara Chen, Leslie Eldenburg, Matthias Mahlendorf, Melissa
Martin, Pam Murphy, Steve Salterio, Greg Sabin, Daniel Thornton, Stephanie Tsui, Jeff Wooldridge, and
workshop participants at the 2013 Canadian Accounting Association Meeting at Montreal, 2014 AAA
Management Accounting Section Meeting at Orlando, University of Arizona, Erasmus University,
Michigan State University, Queen’s School of Business, and Wilfrid Laurier University for their valuable
comments and suggestions. We thank Kenji Yasukata, Yoshinobu Shima and Chiyuki Kurisu for their
support in the collection of the data used in this study. * Corresponding author address: N207, North
Business Complex, Broad College of Business, Michigan State University, East Lansing, MI 48824, Ph:
517-353-4687, Fax: 517-432-1101.
Data availability: data are available from the authors upon request.
Is Mandatory Nonfinancial Performance Measurement Beneficial?
ABSTRACT
We use value of information theory and examine the effect of regulation requiring mandatory
measurement and peer disclosure of nonfinancial performance information in the hospital industry. We
posit that mandatory nonfinancial performance measurement has an information effect and a referent
performance effect. The information (referent performance) effect arises because the new performance
signals induce more precise posterior beliefs about individual (relative) performance. Using panel data
from the Japanese National Hospital Organization, we analyze performance improvements following
regulation requiring standardized measurement and peer disclosure of absolute and relative patient
satisfaction performance. After controlling for ceiling effects, bounded dependent variables, and
regression to the mean, results show that mandatory nonfinancial performance information measurement
and peer disclosure improves overall performance (information effect) with larger improvements for
poorly performing hospitals (referent performance effect). These effects are found even in the absence of
any compensation-based incentives to improve performance.
Keywords: Value of information, Patient Satisfaction, Mandatory performance measurement, Health
care.
2
Is Mandatory Nonfinancial Performance Measurement Beneficial?
1. Introduction
Improving health care outcomes, quality, and cost are topics of high policy interest. Systematic
collection and disclosure of reliable, consistent, and comparable health outcome information is a popular
policy tool that has been proposed in many countries. For example, in the US, the Center for Medicare
and Medicaid Services (CMS) and the Agency for Health Care Research and Quality (AHRQ) developed
a patient satisfaction survey titled Hospital Consumer Assessment of Healthcare Providers and Systems
(HCAHPS) with an aim to produce and publicly disseminate “comparable data on patient's perspective on
care that allows objective and meaningful comparisons between hospitals on domains that are important
to consumers”, to “create incentives for hospitals to improve their quality of care”, and “enhance public
accountability in health care by increasing the transparency of the quality of hospital care provided in
return for the public investment” (http://www.hcahpsonline.org/home.aspx).
Such regulations generate information that was previously unavailable to the decision maker and
should therefore improve decisions. However, the requirement for public disclosure may cloud the value
of the new information for improved internal decision making. For example, decision makers may
become so highly focused on the public’s response to the information that they could exert more effort to
manage the information rather than use the insights provided by the new information. Additionally,
incentive mechanisms may further encourage managers to improve reported performance rather than
focus on the actual drivers of performance (Dranove and Jin 2010). Mandatory requirements for new
information that are bundled with public disclosure therefore do not allow for the assessment of the value
of the new information to the decision maker absent any rewards or penalties arising from the public
disclosure. An interesting question that arises is whether the mere availability of new information to
decision makers has an independent effect on performance; that is, absent public disclosure or obvious
monetary benefits or penalties, will information affect performance?
Most economists and decision theorists would agree that information is largely beneficial to a
decision maker. For example, value of information (VOI) is commonly defined using Bayesian methods
3
as the difference between the expected utility of an action based on the posterior probability given a new
information set, and the expected utility of the action given only the prior information set (Pratt et al.
1995). However, as stated earlier, outside of a laboratory, one rarely encounters situations where access
to new information is not confounded with other factors that impinge on the decision maker’s choices
with the new information. In this study, we examine the value of information using a unique, quasiexperimental, Japanese hospital setting where new regulation requiring standardized measurement and
peer disclosure (as opposed to public disclosure) of patient satisfaction was imposed. The new
information was not previously collected by the hospital or available from other agencies. Further,
performance on the new information metrics was not tied to incentive compensation or other pecuniary
payoffs. We posit that mandatory performance measurement has two effects: an information effect that
arises from the value of the new information about the firm’s own performance, and a referent
performance effect that arises from the value of the new information signal about the firm’s performance
relative to the peer group. Our empirical settings allow us to disentangle these effects without the
confounding influences of performance pressures that could arise from incentive contracting or public
disclosures.
The Japanese hospital industry introduced regulation in 2004 requiring hospitals belonging to the
Japanese National Health Organization (NHO) to be subject to an annual patient satisfaction survey using
a standardized questionnaire. A neutral external agency surveys inpatients and outpatients about their
satisfaction with a number of aspects of their hospital experience, including medical treatments and
procedures, physician and staff behavior and attitudes, and hospital infrastructure. The results of the
survey containing performance information on the level as well as relative rank of individual member
hospitals are disseminated to all hospitals within the NHO.
We analyze patient satisfaction panel data from all 145 NHO-member hospitals over a period of
seven years (2004-2010). We first conduct a factor analysis of the survey responses and identify two
satisfaction constructs, which we label as staff /treatment, and logistics/infrastructure. We then examine
whether the patient satisfaction information results in an improvement in performance on each construct
4
in subsequent years, and whether there are differences in the extent of improvement based on initial
relative performance.
Our data are right- and left-bounded, which is typical of surveys that use a Likert-type scale. That
is, the patient satisfaction score is constrained by both a floor and a ceiling, corresponding to the
minimum and maximum values of the scale used. The boundary observations create problems in
econometric estimation if standard models are used (Papke and Wooldridge 1996). Further, poorly
performing hospitals have greater available range for potential improvement than highly performing ones.
Consequently, performance improvements are likely to be non-linear. To address these issues, we use a
fractional response econometric model, which accounts for non-linearity in the data and corrects for floor
and ceiling effects. Our analyses control for regression to the mean, i.e., hospitals that have low (high)
performance on patient satisfaction in a particular year could have high (low) performance in future years
simply because of the nature of the distribution rather than actual changes in performance (Cook and
Campbell 1979).
Results indicate that mandatory measurement of patient satisfaction has an information effect.
Firm-fixed effects using the fractional response method indicate an overall increase in inpatient as well as
outpatient satisfaction. This increase is of larger magnitude in the year immediately following the release
when the information is new compared to subsequent years. Trend analysis indicates that this effect is not
driven by ceiling effects or diminishing marginal utility of effort. The improvement is found for all
hospitals and not merely the poorly performing hospitals (i.e., hospitals in the lowest quartile of
performance in 2004). We conclude that the patient satisfaction information has value. We also find
evidence of the referent performance effect - hospitals that were performing poorly during the first year of
the information release (i.e., in the lowest quartile of performance in 2004) have greater improvement in
performance following the release. We conclude that the new information about relative performance
facilitates improved goal-directed effort.
Our study contributes to the literature in several ways. First, our unique, quasi-experimental
design enables the assessment of the value of new information, which to our knowledge has not been
5
studied in an archival health care setting. Second, we study the value of information in the absence of
confounding factors such as incentive compensation or performance pressure arising from public
disclosures. Prior research finds that public disclosures of health care quality can motivate hospitals and
providers to improve quality. For example, Evans et al. (1997) finds that mandated public disclosures of
hospital mortality performance lead to subsequent improvements in mortality for hospitals that were
performing poorly during the initial period. Lamb et al. (2013) finds that voluntary reporting of quality of
ambulatory care motivated physician groups to improve quality. In both these aforementioned studies,
the hospitals or physicians were already collecting the quality information and the only change was to
make the disclosure public. That is, the information was not new to the decision maker but only the
disclosure was new. Third, our empirical method attempts to disentangle the value of information about
individual performance and the value of information about relative performance. Fourth, research in
accounting has explored the importance of nonfinancial measures such as satisfaction in driving future
financial performance (e.g. Ittner and Larcker 1998; Nagar and Rajan 2005). These studies have stressed
the importance of nonfinancial information due to its role as a lead indicator of future financial
performance, rather than the value of such information in improving decision making. Finally, our study
has policy implications. Recently, there has been an increasing recognition that engaging patients in their
own care is a cornerstone of successful health system reform (Hibbard and Greene 2013). From October
2012, the Patient Protection and Affordable Care Act of 2010 requires that hospitals collect and report
patient satisfaction or face a reduction in reimbursement by up to 2.0 percentage points. The results of this
study indicate that such regulation requiring mandatory collection and of satisfaction information has the
potential to improve hospital performance on the reported measures, even in the absence of public
disclosures.
The following section summarizes the theory and extant literature. Next, we present a description
of sample characteristics and the methods. A discussion of the results follows. The last section concludes.
2. Prior literature and hypotheses development
2.1 Institutional background of Japanese NHO
6
The Japanese National Health Organization (NHO) is the oversight agency for Japanese
government hospitals. Headquartered in Tokyo, it comprises of 145 hospitals, which represent 3.5% of
the total number of hospitals in Japan. Like other Japanese Independent Administrative Institutions (IAI),
the NHO is the result of the separation between political and operational responsibilities for public
services. NHO hospitals are grouped into two categories: general hospitals, which have discretion on the
type of service they provide, and sanatoriums, which, in addition to offering services similar to general
hospitals, are required to supply particularly expensive and risky medical services. Both types of hospitals
provide inpatient as well as outpatient treatment. Funding for NHO hospitals comes from three sources:
patients’ copayments for service rendered, reimbursements by the National Health Insurance or
Employees’ Health Insurance, and public funding through government grants and subsidies.1 Patient
copayments are received directly by the hospital, while insurance reimbursements are received through a
claims process (Figure 1). Public funding allocation to each hospital within the NHO is dependent upon
periodic performance evaluations of medical outcomes (e.g. mortality and morbidity) and assessment of
the hospital’s need for resources. Prices for healthcare services are determined by the Japanese
government. Price is, therefore, not a driver of patients’ choice of healthcare provider.
--- Insert Figure 1 here --In 2004 the NHO introduced an annual patient satisfaction survey for every hospital within the
NHO. Patients treated in NHO hospitals are required to complete a standardized questionnaire, which
assesses their satisfaction with many aspects of their hospital experience, including medical treatment, the
behavior of the staff and the quality of the infrastructure. The survey is conducted by an independent
university research agency, which is unrelated to the NHO or its hospitals. The university research agency
compiles and analyzes the results of the annual survey, and feedback reports are disseminated to all
1
Health Insurance in Japan is compulsory for all citizens and can be obtained either through the employer
(Employees’ Health Insurance) or, in the case of self-employed individuals and students, through the
National Health Insurance system. Special insurance programs are in place for elderly citizens (over 75
years). Patients pay for 30% of the cost of medical services, with the remaining 70% being reimbursed to
the hospital by the insurer or the government. Medical costs exceeding the equivalent of $600 in a month
are fully reimbursed by the insurer or the government.
7
member hospitals. The feedback reports include the average scores of each hospital on the various
categories and each hospital’s ranking within the NHO. There is no explicit monetary incentive tied to
patient satisfaction performance.2
Consistent with standard models of health economics beginning with the seminal research of
Arrow (1963) and the more recent model of Kolstad (2013), we assume that health care providers obtain
utility from non-financial outcomes such as patient satisfaction, independent of monetary compensation.
We further partition the utility from patient satisfaction into two parts: one that arises from the hospital’s
absolute performance level, and the other that arises from performance relative to peers. We then test
whether the information provided by the patient satisfaction survey has value, in that it affects the choice
of actions leading to improvements in the reported performance.
2.2 Information effect of Patient Satisfaction Information
Adopting an expected value of information framework (Demski 1972; Bromwich 2007), suppose
an organization is considering an action choice from a vector of potential actions given by A{a}, and a
potential set of uncertain states S={s}. The expected utility of a particular action is U(s, a) and the
organization maximizes its expected utility, that is
|
∗
∈
,
∅
. The relationship
between the action a and the expected outcome is based on subjective probability distributions based on
past events (Feltham 1968). Suppose the organization obtains an additional information signal y from an
information system ƞ. The information signal y allows for an improved assessment of the state and
appropriate action choices, and it has value if it affects the decision that the organization would have
made without the signal, thus leading to improved utility. The organization’s expected utility including
the new information signal is
, ƞ,
∗
∈
,
∅ | , ƞ . The expected value of the new
2
Physicians and medical staff at the NHO are compensated on a fixed wage basis and are not provided
performance-contingent bonuses. Physicians and staff obtain raises based on general macro-economic
conditions. Section 3 examines physician compensation at NHO hospitals in greater detail.
8
information signal y is the difference between
, ƞ,
∗
and
|
∗
. 3 If this expected value is
positive, then the new information has value to the decision maker.
We first assess whether the patient satisfaction measure provides new information. NHO sources
indicated that before 2004, neither the NHO nor individual member hospitals were collecting information
on patient satisfaction. Without this information, hospitals only had noisy priors about their own
performance. Because of the tendency for individuals to be overconfident about their ability and
overestimate their effort levels (Camerer and Lovallo 1999; Benoit and Dubra 2011; Kruger and Dunning
1999, 2002), health care providers likely concluded that their patient satisfaction performance was above
average. Additionally, hospitals were unable to accurately assess the payoffs from their efforts to increase
patient satisfaction. Thus, on average, hospitals are likely to have exerted less than optimal effort toward
increasing patient satisfaction. With the introduction of the mandatory patient satisfaction survey,
hospitals received an additional information system (corresponding to ƞ referred to earlier). ƞ contains
two new signals - absolute patient satisfaction level (y1) and patient satisfaction relative to peer hospitals
(y2). These two new signals are based on data collected systematically by an independent universitybased research center using a scientific, standardized, survey instrument. The new signals enable hospitals
to revise their priors and obtain more accurate posterior beliefs about the relation between their actions
and patient satisfaction, which influence future effort allocations and decisions, i.e. to move from utility
function
|
∗
to
, ƞ,
∗ 4
The information on individual performance level (y1) provides a
more precise signal of performance, which allows for guided effort that is better suited to the
circumstances (Morris and Shin 2002; Bandura and Jourden 1991; Ederer 2010). The positive weight
associated with patient satisfaction within the healthcare provider’s utility function drives the extent to
3
Although VOI is sometimes interpreted rather narrowly as the amount a decision maker would be
willing to pay for higher quality information, the analytical models of VOI are generic and refer to
“value” in a flexible sense that allows for non-pecuniary interpretations (Bromwich 2007; Demski 1972;
Raiffa 1968).
4
This assumes that hospitals are Bayesian, i.e., they use new information to update their prior beliefs,
which is a standard assumption in decision theory (Pratt et al. 1995).
9
which the information is internalized and used in the subsequent decision making process. This leads to
the following prediction about the information effect of nonfinancial performance information:
H1. Introduction of mandatory nonfinancial performance measurement improves subsequent
performance.
2.3 Referent performance effect of Patient Satisfaction Information
The second signal contained in the new information system ƞ is patient satisfaction relative to
peer hospitals (y2). Economic theory recognizes referent performance as an important driver of
individuals’ and firms’ utility functions (Kolstad 2013; Sugden 2003).5 Referent performance is
particularly important for strategically interdependent competitive firms (Lant and Hewlin 2002). If a
valuable relative performance signal indicates poorer performance relative to a referent group, it prompts
organizations to increase effort as well as search for new strategies that can enhance relative performance
(Bandura and Jourden 1991) especially if decision makers have flexibility to respond to the new
information (Abernethy and Bouwens 2005). Organizational theory posits that decision makers pay more
attention to activities that fail to meet targets compared to those that succeed (Levitt and March 1988).
Evidence indicates that poor relative performance is a higher motivator of performance than good
performance. For example, Ramanarayanan and Snyder (2012) find that information disclosure in the
dialysis industry is associated with reduction in mortality for poorly performing firms, but do not find
comparable effects for highly performing firms.
The relative performance signal y2 generated by the new information system ƞ eliminates
idiosyncratic uncertainty creating a level field to assess performance. The new information signal y2
increases the accuracy of the posterior belief function about the mapping between effort and output
relative to the organization’s peer group. The noise reduction value of relative feedback is higher for
poorly performing hospitals because they likely expected to be above average in the pre-regulatory
5
In a reference-dependent utility model, preferences between decisions are influenced both by the
expected outcome of the decision and by a reference point, which could be performance of a peer or
competitor (Sugden 2003).
10
period, and therefore the relative information represents an unpleasant surprise.6 This serves as a
motivation for poorly performing hospitals to increase effort to improve performance (Ederer 2010).
Based on the above, we predict that firms with lower initial relative performance on patient satisfaction
will have greater future performance improvements.
H2. Lower initial performance on nonfinancial performance measures is associated with higher
magnitude of subsequent improvements.
3. Data and analyses
3.1 Data and descriptive statistics
The sample used in this study includes patient satisfaction data from 145 NHO hospitals for the
period 2004 to 2010. Appendix A provides the list of the survey questions. The standardized patient
satisfaction survey is administered every year during the months of June and July in all NHO general
hospitals and sanatoriums. Separate surveys are administered to inpatients and outpatients and contain 15
questions (outpatients) and 19 questions (inpatients) respectively. The survey questionnaires contain
multiple items related to quality of medical treatment, behavior of the staff, quality of infrastructure and
facilities, waiting periods, etc. Ten additional questions capture the comprehensive assessment of the
patient’s overall satisfaction. All the questions use a 5-point Likert-type scale, where 1 indicates “strong
dissatisfaction” and 5 indicates “strong satisfaction.” Data are collected and processed by a university
research center, which is unrelated to the NHO. Feedback reports are subsequently distributed to each
member hospital. These reports indicate the average score for each of the questions included in the
questionnaires and the ranking of the hospital within the NHO based on the overall satisfaction results.
Table 1, Panel A contains the descriptive statistics. Hospital size is measured as the number of
staffed beds. A dummy variable is used to capture whether the hospital is a general hospital (value = 1) or
a sanatorium (value = 0). The sample includes 58 general hospitals and 87 sanatoriums. The number of
6
Prior literature finds that in the absence of information, individuals and firms tend to hold optimistic
beliefs about their ability and therefore are overconfident about their performance relative to competitors
(Kahnemann et al. 1982).
11
government and private hospitals per 100,000 inhabitants in the hospital’s prefecture (a geographical unit
equivalent to a county) is used as a proxy for competition. Panel B provides the descriptive statistics for
the overall satisfaction scores for each of the seven years included in the analysis.
--- Insert Table 1 here --3.2 Variable reduction
Patient satisfaction is a multidimensional construct (Chen 2009). To obtain a measure of the
underlying dimensions, we conduct a factor analysis using principal component analysis with oblique
rotation.7 Untabulated results show that for inpatients and outpatients alike, the items load on two factors
(factor loadings > 0.6) and no variable cross-loads on more than one factor. Based on the items’ loadings
we identify two constructs: staff /treatment, and logistics/infrastructure. Cronbach’s alpha values are
greater than 0.9 for both inpatients and outpatients. Overall patient satisfaction is computed as the average
of the scores reported for the ten overall satisfaction questions for inpatients and outpatients.8
3.3 Analysis of inpatients and outpatients
We conjecture that inpatients and outpatients may differentially weigh the importance of each
factor in formulating their assessments of overall satisfaction with the hospital.9 Therefore, we examine
the extent to which each of the factors influences overall inpatient and outpatient satisfaction (Krishnan et
al. 1999). We analyze the relationship between the aggregate overall patient satisfaction and each of the
two satisfaction factors using OLS regressions with heteroskedasticity-robust standard errors clustered by
firm. Separate regressions are performed for inpatients and outpatients. The results of this analysis are
7
The oblique rotation method allows for the possibility that the factors are correlated.
The factor analysis on the ten questions related to overall satisfaction resulted in all questions loading
onto one factor (factor loadings > 0.9 and Cronbach’s alpha > 0.97).
9
A recent survey conducted by the Japanese Ministry of Health, Labor and Welfare explored the major
drivers of hospital choice for inpatients and outpatients. The sample consisted of more than 150,000
respondents, randomly selected from the patient population of all Japanese Hospitals. Overall, outpatients
(inpatients) identified the following drivers of hospital choice: 38% (34.9%) prior experience at the same
hospital, 37.6% (29.9%) physical closeness to their residence, school or place of work, 33.2% (49%)
recommendation by doctors, 31.4% (34.7%) kindness of doctors and nurses, and 28.7% (25.5%)
size/technology of the hospital. (Japanese Ministry of Health, Labour and Welfare. (2011). Patients
Behavior Survey. from http://www.mhlw.go.jp/english/new-info/2012.html).
8
12
reported in Table 1, Panel C. The results indicate that while both factors are significant drivers of overall
inpatient satisfaction, satisfaction with staff and treatment is a bigger driver (coefficient = 0.172) than is
logistics and infrastructure (coefficient = 0.075, p-value of difference in coefficients < 0.10). For
outpatients, only the staff and treatment factor contributes to overall satisfaction (coefficient = 0.185)
while the logistics and infrastructure factor (coefficient = -0.004) is not significantly different from zero
(p-value > 0.50). These results suggest that the relationship with the staff and the experience during
treatment has a primary role in the patient’s satisfaction assessment. Therefore, for the remainder of the
analysis, we do not examine the drivers of logistics and infrastructure outpatient satisfaction.
3.4 Estimation issues: Regression to the mean and fractional response model
The hypotheses relate to the change in patient satisfaction subsequent to the introduction of the
new regulation. Unobserved hospital-level characteristics may bias the results. That is, hospitals whose
initial patient satisfaction measure is high are likely to keep performing well due to characteristics other
than the patient satisfaction. A regression analysis of the satisfaction scores for each year on the
satisfaction measure of the previous year confirms the presence of significant first-order firm fixed effects
(untabulated). Consequently, all the subsequent analyses in this study control for firm fixed effects and
estimate robust standard errors clustered by firm.
Because of correlations between repeated measures, and standard deviations that decrease over
time, it is necessary to consider the extent to which results may be a manifestation of regression towards
the mean rather than actual improvements in patient satisfaction. That is, poorly performing hospitals may
improve in performance simply because of the nature of the behavior of extreme values in a statistical
distribution rather than an actual improvement.10 We are able to reject the hypothesis of regression to the
mean based on the following. First, Table 1, Panel B shows that the overall mean patient satisfaction
10
Note that regression to the mean is primarily an issue when the analysis consists of only two
observations, such as two variables measured on one occasion (e.g. control and treatment group in an
experiment) or one variable measured on two occasions (e.g. pre-test post-test comparison after an
experimental intervention). Regression to the mean is not a phenomenon that is relevant to multiple
observations over time (Nesselroade et al. 1980), such as our longitudinal panel data.
13
measure increases over time, which rules out aggregate mean stability, a necessary condition for
regression to the mean (Cook and Campbell 1979; Zhang and Tomblin 2003). Second, the overall
satisfaction never decreases significantly below the initial (2004) levels for hospitals in the highest
quartile of performance, which is further evidence against the aggregate mean stability condition. Third,
we estimate the correlation between subsequent satisfaction measures after controlling for firm fixed
effects. Untabulated results show non-significant correlations coefficients, thus rejecting the possibility
that the improvement over time is purely a result of regression towards the mean (Cook and Campbell
1979; Zhang and Tomblin 2003). Finally, all analyses of satisfaction change include the initial (2004)
satisfaction rates, which is standard practice for controlling for regression to the mean (Evans et al. 1997;
Kolstad 2013). Consequently, we conclude that regression towards the mean has a non-significant
influence on the results of this study.
Patient satisfaction measures in our sample are bounded at both extremes. A patient who is
extremely unhappy with the medical services provided by the hospital can, at most, assign a score of 1.
Similarly, an extremely satisfied patient cannot assign a score higher than 5. With bounded data,
traditional linear models, like OLS regressions, may misrepresent important characteristics of the
relationship between outcomes and explanatory variables. For example, use of standard linear regression
models could lead to predicted values outside of the response scale interval. Perusal of our data reveals
that a non-trivial number of observations are at the extremes of the scale. Models involving non-linear
transformations of the dependent variable, like the log-odds ratio, are likely to fail in the presence of
response variables that take values at the extremes (Papke and Wooldridge 2008).
Additionally, traditional linear functional models do not consider the diminished scope for
improvement available to the firm when customer satisfaction is already at a high level. A linear model
assumes that the distances between two response scale items are constant. However, an improvement
from 4 to 5 on a five point scale is much harder to obtain than an improvement from 3 to 4.
Transformations such as log-odds or beta distributions have been used in similar circumstances. However,
14
the problem with each of these alternative distributions is that they are not defined at the extremes of the
scale unless an additional ad-hoc transformation is applied to the extreme values.
A fractional response model circumvents the above issues with the application of Bernoulli quasilikelihood parameters estimation methods within generalized linear model settings. A fractional response
model does not require special data adjustments for observed variables at the extremes and performs
robust and efficient estimations of parameters using the assumptions of generalized linear models (Papke
and Wooldridge 1996). To account for time-constant unobserved effects that could be correlated with
explanatory variables (Wagner 2003), we specify our model as an unconditional fixed-effects fractional
response model by adding indicator independent variables for each of the hospitals included in our
sample. Our fractional response model takes the following form (Papke and Wooldridge 1996)
|
where
∙ is a cumulative distribution function (cdf) and
(1)
⁄1
satisfies 0
1 for all . The predicted values of G need to lie on the interval between l (the lowest end of
the response scale) and h (the highest end of the response scale). Estimation of a fractional response
model requires that the dependent variable is in the form of a proportion in the range of [0 ,1], with a
positive probability mass on the extremes. Therefore we transform each response scale variable into a
proportion.
3.5 Compensation practices at NHO
Our interest is in examining the value of information even in the absence of incentive
compensation tied to performance improvement. The vast majority of rank and file workers receive
minimal to zero incentive compensation and it is important to examine if improving their information set
can improve decisions. Prior research by (Banker et al. 2000) finds improvement in nonfinancial
performance following the implementation of an incentive plan that includes nonfinancial performance
measures. In their setting, although customer satisfaction measures were tracked before the
implementation of an incentive plan that includes such measures, these measures did not have a
performance effect until it was used explicitly for incentive purposes. Similarly, (Kelly 2007)
15
experimentally demonstrates that “though providing feedback on nonfinancial measures alone may
improve managerial decisions over time, the key to enhancing decision quality may be providing both
feedback and incentives on nonfinancial measures.” In Evans et al. (1997), even though the hospitals
previously collected mortality data, performance improvements were only observed when the mortality
measures were disclosed to the public. This suggests that disclosure to external stakeholders, rather than
the information per se, has a performance effect. Whether or not increased patient satisfaction information
improves performance in the absence of an explicit link to incentive compensation, and when the
information is disseminated to internal as opposed to external stakeholders is an open empirical question.
We aim to study the value of information unfettered by the effect of financial incentives. Consequently,
we first examine physician compensation practices at NHO hospitals using field and archival data to
determine if there is any link between patient satisfaction performance and compensation.
Field evidence of compensation practices at NHO
We conducted interviews with hospital administrators at the NHO headquarters to glean
information about physician compensation practices. These interviews revealed that there is no explicit
link between physician compensation and patient satisfaction performance. Essentially, Japanese NHO
physicians are government employees. Each physician, nurse, and paramedic is classified into a particular
grade based on a hierarchy, and each grade is provided compensation as per the government salary
schedule. The typical compensation package includes: a monthly salary, allowances for cost of living,
overtime and travel. Appendix B contains information on employment, compensation, and promotion
systems at NHO.
Archival evidence of compensation practices at NHO
To empirically examine whether there is any link between patient satisfaction and physician
compensation, we estimated the following model of physician bonus as a function of patient satisfaction:
Physician Bonust =α+β1 Overall Inpatient Satisfactiont-1 +β2 Overall Outpatient Satisfactiont-1
+ β3-7 Year Dummy +β8 Competition+ β9 Hospital Dummy+ β10 Size (2)
16
where Bonus is the total annual bonus payout by the hospital to physicians. We include control variables
for competition, hospital type, and size. We control for competition because a large body of research in
economics, management, and strategy finds that competition intensity increases firms’ performance
pressures (Banker and Mashruwala 2007; Chen et al. 2013; Scherer and Ross 1990), especially in
industries that compete on non-price aspects such as healthcare (Joskow 1980; Pautler and Vita 1994;
Robinson and Luft 1985). Additional information may be more valuable in more competitive industries
as suggested by Raju and Roy (2000). We measure competition intensity as the number of private and
public hospitals per 100,000 people in the geographical area (prefecture) where each NHO hospital is
located.
The Japanese National Health Organization is comprised of two types of hospitals, general
hospitals and sanatoriums. General hospitals are similar to private hospitals and are allowed greater
discretion in the choice of healthcare services they offer. Sanatoriums are expected to provide not only the
services that are provided by general hospitals, but, in addition, they are required to provide special
services “that cannot be dealt with properly by Non-National Hospital Organizations due to historical and
social reasons.”11 These include treatment of expensive, long-term, risky, or communicable ailments such
as tuberculosis, AIDS, Alzheimer’s, ALS, complex mental illnesses, and invasive or terminal cancer.
Sanatoriums face a different set of pressures. We include a control indicator variable Hospital, which
takes the value of 1 for general hospitals and 0 for sanatoriums.
If patient satisfaction were taken into consideration in bonus payouts, the coefficient on
and
would be positive. We use lagged satisfaction scores because the satisfaction scores for year t are only
released to member hospitals in the following year and therefore likely to only be incorporated into the
following year bonus payments. The results of estimating equation 2 (Table 2, Column 1) indicate no
significant association between bonus and either inpatient or outpatient satisfaction. These results suggest
that patient satisfaction is not taken into consideration in the determination of bonus payments. Thus, we
11
National Hospital Organization (Independent Administrative Institution) page 1;
http://www.mof.go.jp/english/filp/filp_report/zaito2004e-exv/24.pdf.
17
conclude that improvements in patient satisfaction are not driven by the motive to increase incentive
compensation.12
--- Insert Table 2 here --3.6 Patient satisfaction and hospital revenues
Grant revenues
It is possible that patient satisfaction is driven by the desire to increase grant revenue for NHO
hospitals. If so, although there is no explicit tie of satisfaction performance and incentive compensation,
hospitals may have a pecuniary benefit to improving satisfaction. To test this, we estimate the following
regression model:
Hospital Grant Revenuet+1 = α +β1 Overall Inpatient Satisfactiont-1 + β2 Overall Outpatient Satisfactiont-1
+β3-7 Year Dummy +β8 Competitiont + β9 Hospital Dummy+β10
(3)
In equation 3, Hospital Grant Revenue is the lagged amount of grant revenues. We use two-year
lagged grant revenues because conversations with hospital managers indicate that the review of grants
happens mid-term over a five-year program (i.e., every 2.5 years). Results are provided in Table 2
(Column 2) and indicate no statistical association between patient satisfaction and hospital grant revenue.
Our conversations with NHO administrators revealed that hospital grants are based on the research output
of the hospitals rather than patient satisfaction.13
Patient revenues
Prior research finds a positive relationship between customer satisfaction and subsequent
revenues (Ittner and Larcker 1998; Hallowell 1996; Chen et al. 2009). The mechanisms by which
customer satisfaction influences revenues include customer attraction, customer loyalty, and word of
mouth (Rust et al. 2002; Szymanski 2001), which also operate in the non-profit health care industry
(Stizia and Wood 1997; Gemme 1997). When patients are satisfied with the hospital, they tend to return.
12
We re-estimated equation 2 using two-and three-year lags and did not find significant results.
We re-estimated equation 3 using three-, four- and five- year lags and did not find significant results.
Similar analysis of changes in grant revenues did not yield significant results.
13
18
Additionally, their opinion influences others’ choices (Gemme 1997).14 A successful patient satisfaction
program can therefore result in increased revenues. To test the relationship between patient revenues and
satisfaction, we estimate the following regression model:
Hospital Patient Revenuet =α+β1 Overall Inpatient Satisfactiont-1 +β2 Overall Outpatient Satisfactiont-1
+β3-7 Year Dummy +β8 Competitiont + β9 Hospital Dummy+β10 Sizet (4)
In equation 4 we use lagged values of satisfaction in order to allow for word of mouth and other
mechanisms to take place. We find a statistically significant association between patient satisfaction and
lagged revenues (Table 2, Column 3), which indicates that patients respond to hospitals’ efforts to
improve patient satisfaction.15
Evidence that patient satisfaction increases hospital revenues introduces potential for physicians to
obtain pecuniary benefits from increasing satisfaction if compensation is tied to hospital revenues (Finkler
1983). We test the relationship between physician bonus and patient revenues and find no association.16
Prior studies on the role of nonfinancial measures as lead indicators of financial performance are
generally subject to the confounding effects of incentive compensation. Our setting allows us to
contribute to this body of literature by providing evidence of a positive relationship between satisfaction
and future revenue even in absence of compensation-based incentives.
4. Results of hypotheses tests
4.1 Information effect of patient satisfaction
H1 predicts a positive effect of patient satisfaction information. That is, the release of patient
satisfaction information increases nonfinancial performance in subsequent years on the disclosed
measures due to the incremental value of the information signal regarding the level of performance. If the
14
Gemme (1997) reports survey results indicating that 90% of patients’ choice of health care provider are
influenced by other patients’ opinions, and that 40% of surveyed subjects had consulted a patient who had
used the service of the organization they had chosen to use.
15
We re-estimated equation 4 using contemporaneous patient satisfaction data and found similarly
significant associations with patient revenue.
16We tested the association between bonus and patient revenues, and between bonus and lagged patient
revenues and did not find significant results (p-value of coefficient on bonus term > 0.35 in all
estimations).
19
patient satisfaction signal has information value, then we would expect the following: first, patient
satisfaction performance will increase for all hospitals following the first release of satisfaction
information. Second, the increase in satisfaction will be greatest during the first year following the initial
release and the rate of increase in satisfaction will be lower in the subsequent years relative to the first
year because the value of new information is most salient in the year subsequent to the release.17
Univariate analysis
Table 3, Panel A provides information on the mean inpatient and outpatient satisfaction for each
year subsequent to the information release partitioned by the level of hospital satisfaction performance
during the first year of the release (2004). It can be seen that in the year following the release (2005), the
average satisfaction for the full sample of hospitals increased significantly for both outpatients (the t value
of difference in mean outpatient satisfaction of 4.043 in 2005 and 3.962 in 2004 is 3.13, p<0.001) as well
as inpatients (the t value of difference in mean inpatient satisfaction of 4.330 in 2005 and 4.204 in 2004 is
4.416, p<0.001). Examination of the performance of the hospitals that were in the highest performing
quartile in 2004 (top performers) reveals a positive but insignificant increase in 2005 outpatient (t=0.203,
p=0.83) as well as inpatient performance (t=0.129, p=0.89). These results are represented in Figure 2
Panel A (inpatient satisfaction) and Panel B (outpatient satisfaction). Top performers did not significantly
decrease their performance in 2005 (or any of the subsequent years) despite the pressure faced by other
hospitals following the patient satisfaction information release, nor did the mean of top performers drop
below the full sample mean during any year. Indeed, the top performers continued to perform
significantly above the mean during the entire period (all p-values for the difference between top
performers and whole sample are > 0.05).18
Table 3, Panel B reports the annual changes in satisfaction and shows that the improvement in
17
Conversations with NHO hospital managers indicated that prior to 2004, hospitals were not
systematically collecting patient satisfaction information.
18
This is further evidence that regression to mean is not a problem with the data. If regression to the
mean was occurring, top performers would have significantly deteriorated in the subsequent years and
attained lower performance than the sample mean.
20
both inpatient and outpatient satisfaction for the entire sample was highest in the year following the
release, relative to subsequent years. For example, in 2005, the average change in outpatient (inpatient)
satisfaction relative to 2004 was 0.081 (0.126). In 2006, the average change in outpatient (inpatient)
satisfaction relative to 2005 was 0.024 (0.063). The average change in 2005 over 2004 was significantly
higher than the average change in 2006 over 2005 for both outpatients (t=16.01, p<0.001) and inpatients
(t=17.69, p<0.001). Similarly, the average change in 2005 over 2004 was higher than the change in any
subsequent years for both inpatients and outpatients.19 In sum, the highest improvement in outpatient and
inpatient satisfaction was obtained during the first year of release.20 This provides univariate support for
the information effect (H1).
--- Insert Table 3 here ----- Insert Figure 2 here --Multivariate analysis and fractional response model
We test H1 using the following fractional response model:
Satisfaction= α+β1-6 Year Dummy + β5 Competition + β6 Hospital Dummy+ β7 Size
(5)
We estimate five models, one for the overall satisfaction score for inpatients and outpatients
respectively, one for each of the two satisfaction factors i.e., staff/treatment and logistics/infrastructure for
inpatients, and the staff/treatment factor for outpatients. Satisfaction results for 2004 are used as baseline
and therefore 2004 is the omitted dummy. The model controls for firm fixed effects and calculates
heteroskedasticity-robust standard errors clustered by firm.
The dependent variable in equation 5, overall inpatient or outpatient satisfaction, spans the range
[1, 5]. To represent this in fractional terms, we subtract each individual response by 1 (the minimum) and
All p-values of differences in changes in mean satisfaction for subsequent years relative to the first
change in 2005 over 2004 are less than 0.10.
20An alternative explanation for the decay over time is that there are diminishing returns to patient
satisfaction efforts. However, over time hospitals will also be in a better position to identify successful
strategies and exert focused effort. Further, the scope for returns is different for hospitals depending on
their position in 2004. If diminishing returns were present, the low performing hospitals would not
experience decay in performance immediately following the first year. 19
21
divide by 4 (the range of the variable). Thus, a score of 1 on overall patient satisfaction would be
represented in fractional terms as (1-1)/4 = 0. A score of 3 would be represented in fractional terms as (31)/4 = 0.50. A score of 5 would be represented in fractional terms as (5-1)/4 = 1. We perform a similar
transformation on each of the factors because they suffer from the same bounded data problems.
The results in Table 4, Panel A indicate a positive coefficient for each of the year dummies. We
conclude that overall patient satisfaction as well as satisfaction on both factors is significantly higher in
each of the years following the introduction of the patient satisfaction (i.e., 2004), even after controlling
for time invariant characteristics and unobservable hospital level factors that may drive performance.
Because our model is non-linear, the marginal effect of each explanatory variable (x) depends on
the value of x. Therefore, we estimate the average partial effects (APEs), which represent the slope
coefficients estimated at the population average of x. To estimate APEs, we take the average of the partial
first order derivative of the dependent variable (satisfaction) with respect to each of the predictors (Papke
and Wooldridge 2008). Thus, the APE for the Year2005 variable will be estimated as
,
averaged across the distribution. The APE coefficients indicate the mean difference in the dependent
variable driven by each of the predictors. Table 4, Panel B reports the APE results, which indicate that the
introduction of the patient satisfaction has, on average, a positive effect on hospital patient satisfaction for
inpatients and outpatients alike for each of the six subsequent years following the first year of the patient
satisfaction information release. This is consistent with the prediction of the information effect (H1) that
patient satisfaction information improves performance.
--- Insert Table 4 here --4.2 Referent performance effect of patient satisfaction
H2 predicts that a lower initial performance on patient satisfaction is associated with higher
magnitude of subsequent nonfinancial performance improvements, arising from the value of the relative
performance signal. Table 3, Panel B indicates that the mean patient satisfaction change from 2004 to
2005 for hospitals in the lowest performance quartile in 2004 (poor performers) is higher than the change
22
for the full sample for both outpatients (0.159 versus 0.081, t-value of difference in change = 7.52,
p<0.01) and inpatients (0.317 versus 0.126, t-value of difference in change = 7.15, p<0.01). The higher
magnitude of improvement in outpatient satisfaction for poor performers continues to be significant for
each subsequent year change, except for the year 2009. For inpatient satisfaction, initial poor performers
increased their performance by a substantial magnitude in 2005, and continued to show significantly
(p<0.01) higher improvement for the 2007 change and the 2010 change.
To test H2, we test the following fixed-effects model with heteroskedasticity-robust standard
errors clustered by hospital.
Change in Satisfaction = α + β1 Satisfaction Score in 2004 + β2 Competition
+ β3 Hospital Dummy+β4 Size+β5-10 Year Dummies
(6)
where Change in Satisfaction is defined as the annual change in patient satisfaction, Satisfaction Score in
2004 is a continuous variable representing the initial satisfaction score in 2004, Competition, and Hospital
are as defined previously. Results presented in Table 5, Panel A indicate a negative coefficient for the
Satisfaction Score in 2004 for outpatients and inpatients alike and for each of the factors. These results
indicate that hospitals that had lower satisfaction scores during the first year of mandatory performance
measurement (2004) had higher increases in satisfaction. These results indicate support for the referent
performance value of information, consistent with H2. Because of the presence of negative annual
changes, the fractional response model is not applicable to this analysis without loss of observations. 21 To
account for the non-linearity of the changes and for the effects of compression at the top (that is,
improvements are harder to obtain when hospitals are already performing well), we re-estimate equation 6
using a scaled version of the dependent variable, defined as ((Satisfactiont – Satisfactiont-1)/ (5 –
Satisfaction2004)). Table 5, Panel B, shows that even considering for the smaller opportunities for
improvement available for higher performers, hospitals that had lower satisfaction scores in 2004
obtained a larger scaled improvement in satisfaction, which further supports H2. Further, the 2004 scores
21
In supplemental analysis, we discuss the robustness of the results to the application of a fractional
response model.
23
as controls reduce the likelihood of diminishing marginal returns arising from the ceiling effect.
--- Insert Table 5 Here --4.3 Control variables
From Table 4, Panels A and B, it can be seen that hospitals located in areas with higher
competition had higher levels of satisfaction for inpatients and outpatients. The results in Table 5, Panel A
also show a positive but non-significant coefficient on Competition for overall change in inpatient and
outpatient satisfaction. The coefficient on Competition is positive and significant for the two inpatient
factor regressions (Staff and Treatment and Logistics and Infrastructure).
Table 4, Panels A and B also report a positive and significant coefficient on the Hospital Dummy
for overall inpatient satisfaction as well as satisfaction with inpatient staff and treatment, which indicates
that sanatoriums correspondingly had lower levels of inpatient satisfaction. However, for outpatient
services, hospitals (sanatoriums) had lower (higher) levels of inpatient satisfaction. Results in table 5,
Panel A indicate that hospitals (sanatoriums) had a higher (lower) change in overall inpatient satisfaction
and satisfaction with inpatient staff and treatment. Sanatoriums did not have significantly lower change in
outpatient satisfaction relative to hospitals.
4.4 Robustness analyses
Trend regressions
We test the robustness of our results by using trend regressions. We estimate the following
model:
Change in Satisfaction = α+β1 Time Trend +β2 Low Performer Dummy
+β3 Time Trend*Low Performer Dummy+β4 Competition
+β5 Hospital Dummy+ β6 Size
(7)
where, Change in Satisfaction is defined as the yearly change in patient satisfaction (Yeart – Yeart-1), Time
Trend is a continuous variable, which takes the value of 1-7 corresponding to the years 2004-2010, Low
Performer Dummy takes the value of 1 if the hospital was in the lowest quartile of performance in the first
year of the information release (2004), and Competition, and Hospital are as defined previously. Results
24
are presented in Table 6, Panel A, Columns 1 and 2. We also re-estimate equation 7 using scaled changes
in satisfaction as our dependent variable (Table 6, Panel A, Columns 3 and 4).
The results reveal the following for overall inpatient satisfaction: (a) there is significant increase
in satisfaction following the introduction of patient satisfaction (indicated by the significant positive
intercept term, α, of 0.082 for unscaled changes and 0.097 for scaled changes), (b) the rate of increase in
satisfaction decreases over time (captured by the significant negative β1 coefficient of 0.017 for unscaled
changes and 0.024 for scaled changes), (c) hospitals that were performing poorly in the first year of
patient satisfaction had significantly higher rates of increase in performance (indicated by the significant
positive β2 coefficient of 0.147 for unscaled changes and 0.089 for scaled changes), and (d) there was a
declining rate of increase in performance for hospitals that were performing poorly in the first year of
patient satisfaction (indicated by the significant negative β3 coefficient of 0.027 for unscaled changes and
0.014 for scaled changes). These results are consistent with H1 and H2 and indicate that: (i) patient
satisfaction information is associated with an improvement in performance, (ii) the improvement in
performance was highest in the year following the introduction of the patient satisfaction, when the value
of new information was highest (the information effect of performance signal), (iii) patient satisfaction
information has a referent performance effect as evidenced by the higher rate of increase in performance
for initial poor performers even accounting for the ceiling effect, (iv) the referent performance effect also
decreases over time, indicating that the value of new information for referent comparisons is highest
during the first year, providing additional evidence of the information effect of patient satisfaction.
Results for the control variables indicate that general hospitals had higher rate of increase in performance
and correspondingly, sanatoriums had lower rates of increase in performance (captured by the significant
positive β5 coefficient of 0.017 for unscaled changes and 0.022 for scaled changes). The results for overall
outpatient satisfaction are qualitatively similar (Table 6).22
--- Insert Table 6 Here ---
22
The two inpatient factors (staff and treatment, logistics and infrastructure) and outpatient staff and
treatment factors also exhibit similar results.
25
Fractional response model
Recall that we could not use the fractional response model to estimate equations 6 and 7 because of
negative values on some of the dependent variable (change) observations. To further test the robustness of
our results, we re-estimated equations 6 and 7 after dropping the negative observations to make use of the
powerful adjustments for bounded data offered by the fractional response model. The results
(untabulated) are similar to the results in Table 6.
Split-sample analysis
Our sample includes general hospitals and sanatoriums. To test whether the results hold for each
type of hospital, we re-estimate equations 5, 6, and 7 separately for hospitals and for sanatoriums.
Untabulated results for equations 5 and 6 provide evidence that the information effect and referent
performance effect of patient satisfaction is significant for both hospitals and sanatoriums, thus providing
support for H1 and H2. Finally, we re-estimated the trend regressions separately for hospitals and
sanatoriums. Table 6, Panel B, reports the results for the estimation using unscaled changes as the
dependent variable, which are qualitatively similar to the results obtained on the full sample. Untabulated
results allow us to draw similar conclusions based on the estimation using scaled changes.
Pre-existing trend
It is possible that the change in patient satisfaction is merely due to the continuation of a preexisting trend in all the important firm-level variables. To explore this alternative interpretation, we
analyze the change in total cost. The results indicate, on average, (a) no statistically significant higher
change in cost per patient during the first year following the release of the patient satisfaction
information, and (b) no declining rate of change in cost performance. Therefore, we conclude that the
information effect and the referent performance effect are not due to a continuation of a pre-existing trend
in firm-level outcome variables.
5. Conclusions
Information generally has decision value, regardless of whether it is private or public. However,
most studies of the value of information are confounded with decision makers’ responses to pecuniary
26
incentives stemming from performance pressures or public disclosure pressures. In this study we examine
the effect of mandatory nonfinancial performance measurement information on subsequent nonfinancial
performance in a setting where there is minimal confound from public disclosures or incentive
compensation. In 2004, the Japanese National Health Organization introduced mandatory collection and
peer disclosure of patient satisfaction measures using standardized questionnaires. We use patient
satisfaction data for a sample of 145 Japanese public hospitals for a period of seven years (2004-2010) to
explore the effects of mandatory information on patient satisfaction. The information provided to each
member hospital contained two new signals – first, a signal about the absolute level of patient satisfaction
performance, and second a signal about the level of performance relative to a referent group. We posit
that the information effect arising from the value of the new information signal about the level of
performance induces a more precise posterior distribution of beliefs and facilitates decision making. We
also examine the referent performance effect arising from the value of the new information signal about
performance relative to a peer group. Our findings support the notion that patient satisfaction information
improves future satisfaction performance.
Some of the analysis use bounded variables; to mitigate problems that arise with bounded data,
we use a fractional response econometric model. Results indicate the following. First, after controlling for
firm fixed effects and heteroskedasticity, nonfinancial performance increases in the years subsequent to
the introduction of the patient satisfaction requirements. Second, the greatest degree of increase is
observed in the first year following the first release of information. The rate of increase in subsequent
years reveals a decreasing trend, suggesting that patient satisfaction information has the most impact
during the first year of release, consistent with the information effect. Third, hospitals that were poor
performers during the first year after the introduction of the new information system improve by a greater
extent than hospitals with higher initial satisfaction performance, consistent with the referent performance
effect. We control for the level of competition intensity and institutional pressures.
Information has value if it affects subsequent actions. Whether information is used in the decision
making process is a function of the relevance of the information for the decision maker. Mandatory
27
performance measurement systems are generally developed to facilitate better decisions. Prior research
suggests that when performance measurement systems are mandatory through legislative provisions,
subordinate organizations are likely to comply with the regulatory requirement, but make little use of such
information for internal decisional processes (Cavalluzzo and Ittner 2004). In our setting the subordinate
organizations appear to use the new information obtained from the mandatory measurement. Our results
support the notion that information generated by mandatory performance measurement systems is utilized
internally if it has decision value for the regulated firms. Although not empirically tested in this study, our
results indicate that hospital physicians and staff were likely able to respond to the new information and
assimilate it into their decisions, which has been identified by Abernethy and Bouwens (2005) as
important drivers of the success of new accounting systems.
Prior research on mandatory nonfinancial performance measurement in the healthcare industry
has primarily used publicly disclosed medical measures of hospital quality (such as mortality). The
general consensus in the literature is that public disclosures of performance influence future demand and
have the potential to assist consumers make informed choices. Two problems plague most studies in this
area. First, it is difficult to disentangle real improvements in quality from increases arising from gaming
behaviors by firms to artificially boost quality measures. In our setting, the information is collected and
analyzed by an independent third party, reducing the likelihood of gaming. Second, demand may increase
because of customers’ response to rankings as opposed to response to an actual increase in service quality
(Dranove and Jin 2010). We find an association between changes in patient satisfaction and patient
revenues even without public dissemination of performance information. We infer that patients respond to
the actual performance improvements rather than a mechanical response to rankings.
Substantial evidence exists regarding that firms should not neglect nonfinancial measures such as
satisfaction because they provide leading indicators of future financial value.23 These studies focus on the
23For example see Baiman and Baldenius 2009; Banker et al. 2000; Chenhall 2005; Feltham and Xie
1994; Hemmer 1996; Ittner et al. 2003; Ittner and Larcker 1998; Kaplan and Norton 1992; Kekre et al.
1995; Krishnan et al. 1999; Nagar and Rajan 2005; and Rajan and Reichelstein 2009.
28
pecuniary benefits of focusing managerial attention on nonfinancial measures. Indeed, Banker et al.
(2000) in their study of customer satisfaction in the hotel industry mention that without tying nonfinancial
incentive measures to compensation contracts, “managers did not recognize the true benefit of allocating
more effort and resources to improve customer satisfaction, and did not do so until the change in
compensation plan that focused their attention on improving the customer satisfaction measures” (p.90).
Evidence in this study builds on the conclusion in Banker et al. (2000) and empirically shows that
nonfinancial information has the ability to achieve improved outcomes even without including these
metrics in compensation plans. In the hospital industry, nonfinancial performance measure such as
patient satisfaction is especially important and forms a central feature of patient centered medicine
(Bardes 2012).
We conclude that our results indicate a significant effect of mandatory nonfinancial performance
measurement systems arising from the absolute as well as referent value of the additional information.
These findings offer significant contributions to both research, practice, and policy by showing that in
health care, nonfinancial information has a decision-facilitating role in addition to the previously
documented decision-influencing role. While it is important to ensure that health care decision makers
have access to information that facilitates decision making, there is debate on whether mandatory public
disclosure of information has the potential to reduce social welfare. Mandatory public disclosure can be a
double-edged policy instrument because it can cause over-reaction by the public, as well as neglect of
private information by decision makers (Morris and Shin 2002). Results of our study indicate that, in
some instances, mandating information collection without public disclosures can be beneficial.
29
References
Abernethy, M. A., and J. Bouwens. 2005. Determinants of accounting innovation implementation.
ABACUS 41 (3): 217-240.
Arrow, K. J. 1963. Uncertainty and the welfare economics of medical care. The American Economic
Review 53 (5): 941-973.
Baiman, S., and T. Baldenius. 2009. Nonfinancial performance measures as coordination devices. The
Accounting Review 84 (2): 299-330.
Bandura, A., and F. J. Jourden. 1991. Self-Regulatory mechanisms governing the impact of social
comparison on complex decision making. Journal of Personality and Social Psychology 60 (6):
941-951.
Banker, R. D., and R. Mashruwala. 2007. The moderating role of competition in the relationship between
nonfinancial performance measures and future financial performance. Contemporary Accounting
Research 24 (3): 763-793.
Banker, R. D., G. Potter, and D. Srinivasan. 2000. An empirical investigation of an incentive plan that
includes nonfinancial performance measures. The Accounting Review 75 (1): 65-92.
Bardes, C.L. 2012. Defining “Patient-Centered Medicine”. New England Journal of Medicine 366:782783.
Benoit, J.-P., and J. Dubra. 2011. Apparent overconfidence. Econometrica 79 (5): 1591-1625.
Bromwich, M. 2007. Economics in management accounting. In Handbook of Management Accounting,
edited by A. G. H. Christopher S. Chapman, Michael D. Shiends: Elsevier, 137-162.
Camerer, C., and D. Lovallo. 1999. Overconfidence and excess entry: an experimental approach.
American Economic Review 89 (1): 306-318.
Cavalluzzo, K. S., and C. D. Ittner. 2004. Implementing performance measurement innovations: evidence
from government. Accounting, Organizations and Society 29 (3-4): 243-267.
Chen, C. X. 2009. Who really matters? Revenue implications of stakeholder satisfaction in a health
insurance company. The Accounting Review 84 (6): 1781-1804.
Chen, C. X., M. Martin, and K. A. Merchant. 2009. The effect of timing on the association between
customer satisfaction measures and future financial performance. In SSRN:
http://ssrn.com/abstract=1196702.
Chen, C. X., E. M. Matsumura, J. Y. Shin, and S. Y. Wu. 2013. The effect of competition intensity and
competition type on the use of customer satisfaction measures in executive annual bonus
contracts. In AAA 2012 Management Accounting Section (MAS): SSRN:
http://ssrn.com/abstract=1910170.
Chenhall, R. H. 2005. Integrative strategic performance measurement systems, strategic alignment of
manufacturing, learning and strategic outcomes: an exploratory study. Accounting, Organizations
and Society 30 (5): 395-422.
Cook, T. D., and D. T. Campbell. 1979. Quasi-Experimentation: design and analysis issues for field
settings. Boston: Houghton Mifflin.
Demski, J. S. 1972. Information improvement bounds. Journal of Accounting Research 10 (1): 58-76.
Dranove, D., and G. Z. Jin. 2010. Quality disclosure and certification: theory and practice. Journal of
Economic Literature 48 (4): 935-963.
Ederer, F. 2010. Feedback and motivation in dynamic tournaments. Journal of Economics and
Management Strategy 19 (3): 733-769.
Evans, J. H. I., Y. Hwang, N. Nagarajan, and K. Shastri. 1997. Involuntary benchmarking and quality
improvement: the effect of mandated public disclosure on hospitals. Journal of Accounting,
Auditing and Finance 12 (3): 315-346.
Feltham, G. A. 1968. The value of information. The Accounting Review 43 (4): 684-696.
Feltham, G. A., and J. Xie. 1994. Performance measure congruity and diversity in multi-task
principal/agent relations. The Accounting Review 69 (3): 429-453.
Finkler, S. A. 1983. The hospital as a sales-maximizing entity. Health Services Research 18 (2): 117-133.
30
Gemme, E. M. 1997. Retaining customers in a managed care market. Marketing Health Services 17 (3):
19-21.
Hallowell, R. 1996. The relationship of customer satisfaction, customer loyalty, and profitability: and
empirical study. International Journal of Service Industry Management 7 (4): 27-42.
Hemmer, T. 1996. On the design and choice of "modern" management accounting measures. Journal of
Management Accounting Research 8: 87-116.
Ittner, C. D., and D. F. Larcker. 1998. Are nonfinancial measures leading indicators of financial
performance? An analysis of customer satisfaction. Journal of Accounting Research 36: 1-35.
Ittner, C. D., D. F. Larcker, and M. W. Meyer. 2003. Subjectivity and the weighting of performance
measures: evidence from a balanced scorecard. The Accounting Review 78 (3): 725-758.
Joskow, P. L. 1980. The effects of competition and regulation on hospital bed supply and the reservation
quality of the hospital. The Bell Journal of Economics 11 (2): 421-447.
Kahnemann, D., P. Slovic, and A. Tverski. 1982. Judgment under uncertainty: heuristics and biases.
Cambridge, MA: Cambridge University Press.
Kaplan, R. S., and D. P. Norton. 1992. The balanced scorecard - measures that drive performance.
Harvard Business Review: 71-79.
Kekre, S., M. S. Krishnan, K. Srinivasan. 1995. Drivers of customer satisfaction for software products:
implications for design and service support. Management Science. 41(9) 1456–1470.
Kelly, K. O. 2007. Feedback and incentives on nonfinancial value drivers: effects on managerial decision
making. Contemporary Accounting Research 24 (2): 523-556.
Kolstad, J. T. 2013. Information and quality when motivation is intrinsic: evidence from surgeon report
cards. In NBER: http://www.nber.org/papers/w18804.
Krishnan, M.S., V. Ramaswamy, M. C. Meyer, and P.Damien.1999. Customer Satisfaction for Financial
Services: The Role of Products, Services, and Information Technology. Management Science
45(9):1194-1209
Kruger, J., and D. Dunning. 1999. Unskilled and unaware of it: how difficulties in recognizing one's own
incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology 77
(6): 1121-1134.
———. 2002. Unskilled and unaware--but why? A reply to Krueger and Mueller (2002). Journal of
Personality & Social Psychology 82 (2): 189-192.
Lant, T. K., and P. F. Hewlin. 2002. Information cues and decision making: the effects of learning,
momentum, and social comparison in competing teams. Group & Organization Management 27
(3): 374-407.
Levitt, B., and J. G. March. 1988. Organizational learning. Annual Review of Sociology 14: 319-340.
Morris, S., and H. S. Shin. 2002. Social value of public information. The American Economic Review 92
(5): 1521-1534.
Nagar, V.and M. V. Rajan. 2005. Measuring Customer Relationships: The Case of the Retail Banking
Industry. Management Science 51(6):904-919
Nesselroade, J. R., S. M. Stigler, and P. B. Baltes. 1980. Regression toward the mean and the study of
change. Psychological Bulletin 88 (3): 622-637.
Papke, L. E., and J. M. Wooldridge. 1996. Econometric methods for fractional response variables with an
application to 401(K) plan participation rates. Journal of Applied Econometrics 11 (6): 619-632.
———. 2008. Panel data methods for fractional response variables with an application to test pass rates.
Journal of Econometrics 145 (1-2): 121-133.
Pautler, P. A., and M. G. Vita. 1994. Hospital market structure, hospital competition and consumer
welfare: what can the evidence tell us? Journal of Contemporary Health Law and Public Policy
10: 117-167.
Pratt, J. W., H. Raiffa, and R. O. Schlaifer. 1995. Introduction to Statistical Decision Theory. Cambridge,
MA: MIT Press.
Raiffa, H. 1968. Decision analysis; introductory lectures on choices under uncertainty: Addison-Wesley.
31
Rajan, M. V., and S. Reichelstein. 2009. Objective versus subjective indicators of managerial
performance. The Accounting Review 84 (1): 209-237.
Raju, J.S., and A. Roy. 2000. Market Information and Firm Performance. Management Science
46(8):1075-1084.
Ramanarayanan, S., and J. Snyder. 2012. Information disclosure and firm performance: evidence from the
dialysis industry. In SSRN: Working Paper # 2152796.
Robinson, J. C., and H. S. Luft. 1985. The impact of hospital market structure on patient volume, average
length of stay, and the cost of care. Journal of Health Economics 4: 333-356.
Rust, R. T., C. Moorman, and P. R. Dickson. 2002. Getting return on quality: revenue expansion, cost
reduction or both? Journal of Marketing 66 (October): 7-24.
Scherer, F. M., and D. Ross. 1990. Industrial Market Structure and Market Performance. Boston, MA:
Houghton Mifflin.
Stizia, J., and N. Wood. 1997. Patient satisfaction a review of issues and concepts. Social Science and
Medicine 45 (2): 1829-1843.
Sugden, R. 2003. Reference-dependent subjective expected utility. Journal of Economic Theory 111 (2):
172-191.
Szymanski, D. M. 2001. Customer satisfaction: a meta-analysis of the empirical evidence. Journal of the
Academy of Marketing Sciences 29(1): 16-35.
Wagner, J. 2003. Unobserved firm heterogeneity and the size-export nexus: evidence from German panel
data. Review of World Economics 139 (1): 161-172.
Yokota, F., and K. M. Thompson. 2004. Value of information analysis in environmental health risk
management decisions: past, present, and future. Risk Analysis 24 (3): 635-650.
Zhang, X., and J. B. Tomblin. 2003. Explaining and controlling regression to the mean in longitudinal
research design. Journal of Speech, Language and Hearing Research 46 (6): 1340-1351.
32
Appendix A
Survey instrument
Panel A: Overall satisfaction (Same questions asked separately for outpatients and inpatients)
Strongly
Dissatisfied
1
Somewhat
Dissatisfied
2
Neutral
3
Somewhat
Satisfied
4
Strongly
Satisfied
5
I am satisfied with the results of the treatment
1
2
3
4
5
I am satisfied with the period of the treatment
1
2
3
4
5
I am satisfied with treatment I have been taking
1
2
3
4
5
I am satisfied with the hospital
1
2
3
4
5
I think this hospital provides safe medical services
1
2
3
4
5
I think the explanations provided by the medical staff
were very clear
I think the treatment I have received was acceptable
1
2
3
4
5
1
2
3
4
5
I generally trust this hospital
1
2
3
4
5
I would like to recommend this hospital to family
members and friends
1
2
3
4
5
Strongly
Agree
Somewhat
Agree
Somewhat
Disagree
Strongly
Disagree
1
2
Neither
Agree nor
Disagree
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
I am generally satisfied with this hospital
Panel B: Questions for inpatients
I am not satisfied with the explanation by doctors
when I was hospitalized
I was unhappy with the procedure of medical
admission
I was unhappy with hospital's explanation about my
life during the hospital stay
I think that the doctors behave badly and use bad
language in this hospital
I was worried about some doctors' skills and
knowledge
I think that the nurses behave badly and use bad
language in this hospital
I was unhappy with the assistance received for daily
life activities
I think that medical staff such as doctors, nurses and
other medical staff lacked teamwork
I did not like today's medical tests (For patients who
accepted medical tests)
I did not like today's medical surgeries (For patients
who accepted medical surgeries)
I did not like today's medical treatment (For patients
who accepted medical treatment)
I did not like today's drip, injection, medicine, or
prescription (For patients who had a drip, injections,
medicine, or prescription)
33
I did not like today's rehabilitation (For patients who
had rehabilitation)
I am unhappy with the toilets and bathrooms in this
hospital
I think that passageways, stairs and elevators are
inconvenient
I am unhappy with my room
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
I am unhappy with the food in this hospital
1
2
3
4
5
I am unhappy with the other environment such as
stores, and interiors
I am unhappy with the hospital's explanation of my
discharge
1
2
3
4
5
1
2
3
4
5
Strongly
Agree
Somewhat
Agree
Somewhat
Disagree
Strongly
Disagree
I felt uneasy when I came to the hospital at the initial
visit
I think that this hospital is very inconvenient
1
2
Neither
Agree nor
Disagree
3
4
5
1
2
3
4
5
I have a bad impression about this hospital
1
2
3
4
5
I am unhappy with waiting time
1
2
3
4
5
I am unhappy with the waiting room
1
2
3
4
5
I think that doctors behave badly and use bad
language in this hospital
I was worried about some doctors' skills and
knowledge
I think that nurses behave badly and use bad
language in this hospital
I did not like today's medical tests (For patients who
accepted medical tests)
I did not like today's medical treatment.(For patients
who accepted medical treatment )
I did not like today's drip, injection, medicine, or
prescription (For patients who had a drip, injections,
medicine, or prescription)
I did not like today's rehabilitation (For patients who
had rehabilitation)
I am unhappy with the treatment room
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
I am unhappy with the other environment such as
shops, ATM, and interiors
I am unhappy with the procedures for payment
1
2
3
4
5
1
2
3
4
5
Panel C: Questions for outpatients
Notes to Appendix A: This appendix lists the questions used in the survey administered to Japanese
National Health Organization (NHO) general hospitals and sanatoriums. The translation from Japanese to
English aimed at maintaining the original meaning as close as possible.
34
Appendix B
Physician compensation at NHO
Salary schedule: Each NHO post is classified into a certain grade in a salary schedule. The classification
of the employee into a post is based on two factors: educational classification and experience. Most
Japanese government agencies have ten grades. Within each grade employees receive raises in steps,
which are based on time in grade. A sample of the pay scale for a Japanese government agency is
provided below.
Steps
1
5
9
13
17
21
25
29
33
37
41
45
49
53
57
61
65
69
73
77
81
85
89
93
97
1
135,600
140,100
144,500
149,800
155,700
161,600
172,200
178,800
185,800
191,600
196,900
202,000
207,100
211,600
215,400
219,200
223,000
226,900
230,100
233,000
236,100
239,000
241,900
243,700
299,600
2
185,800
192,800
200,000
207,000
214,600
222,000
229,300
236,100
242,100
248,000
254,200
259,700
265,200
270,100
275,200
279,700
283,500
287,200
290,400
292,300
293,800
295,300
296,800
298,200
349,500
Salary per month (Yen)
Grade
3
4
5
6
222,900 261,900 289,200 320,600
230,200 270,200 298,200 329,800
237,500 278,600 307,300 338,600
244,900 287,000 316,400 347,200
252,200 295,400 325,200 355,500
260,100 303,800 333,500 363,500
267,700 312,100 341,500 371,500
275,300 320,400 349,400 379,500
282,700 328,400 357,000 386,900
290,100 336,500 364,200 393,700
297,400 344,400 370,100 398,400
304,200 352,000 374,700 403,000
310,600 358,500 378,400 405,900
317,100 363,000 381,700 408,800
323,400 367,100 384,500 411,600
328,100 369,800 387,000 414,300
331,900 372,400 389,600 416,900
335,200 375,000 392,200 419,400
337,800 377,600 394,800 422,000
340,000 380,200 397,300 424,600
342,000 382,700 399,900
344,000 385,100 402,500
345,900 387,600
347,700 390,100
7
366,200
376,300
386,400
397,100
406,400
414,800
422,900
429,400
434,600
439,700
443,200
446,400
449,400
452,400
455,400
458,400
8
413,000
422,800
432,300
441,300
449,300
456,500
462,500
467,800
471,000
474,200
477,400
480,500
9
466,700
479,000
491,300
503,600
513,300
519,000
524,800
529,600
533,100
536,700
540,300
Composition of Salary: In addition to the monthly salary, government employees also get allowances
averaging at about 20% of base salary. The allowances include: living expenses (cost of living
adjustment), housing allowance, commuter allowance, overtime allowance, cold weather allowance, and
diligence allowance (typically based on the number of months of consecutive work in the previous 6month period). There are some compensation adjustments related to macroeconomic conditions.
Individual performance-based bonuses are not commonly found.
Figure 1 Medical expenses reimbursement process
35
10
532,000
544,700
554,700
562,100
568,100
572,900
Figure 1
Japanese Health Care System
Notes to Figure 1: Source: Japanese Ministry of Health, Labour and Welfare
http://www.mhlw.go.jp/english/wp/wp-hw4/dl/health_and_medical_services/P28.pdf )
36
Figure 2
Inpatient and outpatient satisfaction performance
Panel A: Mean inpatient satisfaction performance by year
4.600
4.400
4.200
4.000
3.800
3.600
3.400
2004
2005
2006
Best performers in 2004
2007
2008
2009
Worst performers in 2004
2010
All hospitals
Panel B: Mean outpatient satisfaction performance by year
4.400
4.200
4.000
3.800
3.600
3.400
3.200
2004
2005
Best performers in 2004
2006
2007
2008
Worst performers in 2004
2009
2010
All hospitals
37
Table 1
Descriptive statistics
Panel A: Hospital characteristics
Variable
Size
Competition
Hospital
Bonus
Grant
Inpatient
Revenue
Outpatient
Revenue
Total
Revenue
Description
Number of Staffed Beds (in
hundreds)
Number of Hospitals per
100,000 inhabitants in the
prefecture
1 if the Hospital is a General
Hospital and 0 if it is a
Government Sanatorium
Total bonus compensation at
the hospital-year level
(Billion Yen)
Government grant at the
hospital-year level (Million
Yen)
Medical revenue for
inpatient services at the
hospital-year level (Billion
Yen)
Medical revenue for
outpatient services at the
hospital-year level (Billion
Yen)
Medical revenue for
inpatient and outpatient
services at the hospital-year
level (Billion Yen)
N
1,015
Mean
3.950
Median
3.800
Std Dev
1.413
Q1
3.070
Q3
4.750
1,015
7.003
6.300
2.619
5.1
8.3
1,015
0.400
0
.490
0
1
1,007
0.334
0.295
0.157
0.231
0.400
1,006
0.035
0.023
0.047
0.003
0.048
863
4.190
3.528
2.585
2.462
5.081
863
0.848
0.531
0.843
0.234
1.199
1,006
5.029
4.107
3.362
2.683
6.155
38
Panel B: Overall satisfaction scores
Overall Patient Satisfaction - Outpatient
2004
2005
2006
2007
2008
2009
2010
Overall Patient Satisfaction – Inpatient
2004
2005
2006
2007
2008
2009
2010
N
142
142
142
142
141
140
140
Mean
3.962
4.043
4.067
4.086
4.112
4.103
4.113
Median
3.946
4.031
4.049
4.106
4.091
4.109
4.110
Std Dev
0.222
0.214
0.185
0.184
0.181
0.171
0.175
Q1
3.817
3.897
3.941
3.970
3.999
3.982
4.019
Q3
4.101
4.191
4.191
4.188
4.245
4.223
4.212
135
135
135
135
134
132
133
4.204
4.330
4.393
4.403
4.450
4.431
4.439
4.250
4.373
4.434
4.453
4.501
4.501
4.512
0.265
0.213
0.256
0.252
0.212
0.299
0.301
4.118
4.256
4.351
4.361
4.412
4.374
4.415
4.350
4.454
4.531
4.544
4.571
4.573
4.590
Panel C: Regression results: relationship between factors and overall satisfaction (two-tailed p
values in parentheses)
Constructs
Staff and Treatment
Logistics and Infrastructure
Adjusted R2
Dependent Variable
Overall Outpatient
Overall Inpatient
Satisfaction
Satisfaction
0.185***
0.172***
(p<0.001)
(p<0.001)
-0.004
0.075***
(p>0.01)
(p<0.001)
0.789
0.794
N = 1,004
N = 923
Notes to Table 1: (1) Panel A reports the mean values for the variables used in the analysis. (2) Panel B
contains the mean overall satisfaction scores by year of survey for the questions listed in Appendix A,
Panel A. Overall satisfaction scores are calculated as the mean of the reported scores for the ten overall
satisfaction questions that are common for both inpatients and outpatients. A factor analysis of these
scores resulted in all variables loading on a single factor with factor loadings greater than 0.9 and
Cronbach’s alpha greater than 0.97. (3) The coefficients in Panel C are based on regression analysis of the
overall satisfaction scores on the factor scores. The regression equation is:
1∗
2∗
. The
regression calculates robust standard errors clustered by hospital. (4) Similar regression estimation results
(untabulated) were obtained when considering hospitals only or sanatoriums only. (5) The factor scores
are based on principal component analysis with oblique rotation. (6) The factor scores are expressed in
standardized terms. (7) * = p-value <0.10; ** = p-value <0.05; *** = p-value <0.01.
39
Table 2
Analysis of the relationship between patient satisfaction performance and incentive
compensation, hospital grant and patient revenues
Variable
Inpatient Overall Satisfactiont-1
Outpatient Overall Satisfactiont-1
Year 2006
Year 2007
Year 2008
Year 2009
Year 2010
Competition
Hospital Dummy
Size
Adjusted R2
Dependent Variable
= Performance
Bonust
0.006
(0.005)
0.012
(0.013)
-0.006**
(0.002)
0.012***
(0.004)
0.009*
(0.005)
0.006
(0.004)
0.003
(0.005)
-0.001
(0.003)
0.138***
(0.018)
0.044***
(0.013)
0.708
N = 846
Dependent Variable
= Hospital Grant
Revenuet+1
0.001
(0.003)
0.002
(0.003)
0.007***
(0.001)
0.016***
(0.012)
0.023***
(0.003)
0.005**
(0.002)
-0.000
(0.003)
0.001
(0.007)
0.015***
(0.002)
0.179
N=704
Dependent
Variable = Patient
Revenuet
0.174*
(0.089)
0.574***
(0.171)
-0.112
(0.034)***
0.107**
(0.051)
0.144
(0.104)
0.385***
(0.072)
0.778***
(0.098)
-0.048
(0.056)
3.696***
(0.402)
-0.048
(0.056)
0.650
N=844
Notes to Table 2: (1) Robust standard errors are reported in parenthesis for each coefficient. (2) The
regression equation in column 1 is Physician Bonust = α + β1 Overall Inpatient Satisfactiont-1
+ β2 Overall Outpatient Satisfactiont-1 + β3-7 Year Dummy + β8 Competition + β9 Hospital Dummy +
β10
. (3) Patient satisfaction predictors are lagged one period for both inpatient and outpatient cases
for the Performance Bonus results in Column 1. (4) The regression equation in column 2 is
Hospital Grant Revenuet+1 = α + β1 Overall Inpatient Satisfactiont-1 +
β2 Overall Outpatient Satisfactiont-1 +β3-7 Year Dummy +β8 Competitiont + β9 Hospital Dummy +β10
.
(5) We use two-year lagged satisfaction in Column 2 because review of grants happens mid-term occurs
every 2.5 years. (6) The regression equation in column 3 is similar to column 1, with Patient Revenue as
the dependent variable. (7) Patient satisfaction predictors are lagged one period for both inpatient and
outpatient cases for the Patient Revenue results in Column 3. (8) All regressions use robust standard
errors and control for hospital fixed effects. (9) All financial measures are expressed in Billion Yen. (10)
* = p-value <0.10; ** = p-value <0.05; *** = p-value <0.01.
40
Table 3
Satisfaction performance by year based on initial performance
Panel A: Mean patient satisfaction by year based on initial performance
Whole Sample
Performance Quartile in 2004
Lowest Quartile
Highest Quartile
Overall Outpatient Satisfaction
N
Mean
N
Mean
N
Mean
2004
142
3.962
35
3.699
36
4.244
2005
142
4.043
35
3.858
36
4.255
2006
142
4.067
35
3.897
36
4.204
2007
142
4.086
35
3.924
36
4.196
2008
141
4.112
35
3.977
36
4.213
2009
140
4.103
34
3.957
36
4.178
2010
140
4.113
35
3.978
35
4.206
Overall Inpatient Satisfaction
2004
135
4.204
33
3.858
31
4.475
2005
135
4.330
33
4.170
31
4.482
2006
135
4.393
33
4.208
31
4.521
2007
135
4.403
33
4.231
31
4.509
2008
134
4.450
32
4.283
31
4.545
2009
132
4.431
31
4.206
31
4.537
2010
133
4.439
31
4.290
31
4.529
Panel B: Mean annual change in satisfaction scores based on initial performance
Whole Sample
Performance Quartile in 2004
Lowest Quartile
Highest Quartile
Overall Outpatient Satisfaction
N
Mean
N
Mean
N
Mean
2005-2004
142
0.081
35
0.159
36
0.010
2006-2005
142
0.024
35
0.038
36
-0.051
2007-2006
142
0.019
35
0.028
36
-0.008
2008-2007
141
0.025
35
0.053
36
0.017
2009-2008
140
-0.010
34
-0.020
36
-0.035
2010-2009
139
0.008
34
0.016
35
0.022
Overall Inpatient Satisfaction
2005-2004
135
0.126
33
0.317
31
0.008
2006-2005
135
0.063
33
0.035
31
0.039
2007-2006
135
0.010
33
0.066
31
-0.012
2008-2007
134
0.051
32
0.033
31
0.036
2009-2008
132
-0.023
31
-0.070
31
-0.008
2010-2009
131
0.004
30
0.040
31
-0.008
Notes to Table 3: (1) Hospitals are identified as lowest initial performers (lowest quartile) and highest
initial performers (highest quartile) based on the satisfaction scores of 2004, which is the first year of the
mandatory collection of satisfaction performance information. (2) In Panel B, change is defined as
satisfaction in Yeart – Yeart-1.
41
Table 4 Fractional response model with firm fixed effects: analysis of patient satisfaction level
Panel A: fractional response model.
Dependent variable = patient satisfaction (robust standard errors in parentheses)
Variable
Year 2005
Year 2006
Year 2007
Year 2008
Year 2009
Year 2010
Competition
Hospital
Dummy
Size
R2
Inpatient Satisfaction
Overall Inpatient
Staff and
Logistics and
Satisfaction
Treatment
Infrastructure
0.208***
0.117**
0.150***
(0.043)
(0.051)
(0.039)
0.308***
0.239***
0.213***
(0.034)
(0.053)
(0.039)
0.371***
0.309***
0.211***
(0.041)
(0.052)
(0.041)
0.453***
0.369***
0.304***
(0.039)
(0.050)
(0.038)
0.376***
0.359***
0.385***
(0.051)
(0.055)
(0.045)
0.338***
0.348***
0.381***
(0.080)
(0.054)
(0.056)
0.032***
0.034***
0.027**
(0.011)
(0.011)
(0.011)
0.224***
0.244***
0.077
(0.055)
(0.053)
(0.056)
0.033
0.043*
0.005
(0.023)
(0.024)
(0.020)
0.116
0.154
0.108
N = 993
N = 923
N = 923
Outpatient Satisfaction
Overall Outpatient
Staff and
Satisfaction
Treatment
0.124***
0.105***
(0.022)
(0.017)
0.174***
0.187***
(0.026)
(0.021)
0.197***
0.185***
(0.026)
(0.022)
0.222***
0.235***
(0.027)
(0.022)
0.209***
0.229***
(0.027)
(0.023)
0.226***
0.234***
(0.027)
(0.021)
0.014**
0.011*
(0.006)
(0.006)
-0.120***
-0.156***
(0.032)
(0.032)
-0.010
-0.016
(0.013)
(0.012)
0.132
0.173
N = 1,007
N = 1,004
42
Panel B: Average partial effects
Dependent variable = patient satisfaction (delta-method standard errors in parentheses)
Variable
Year 2005
Year 2006
Year 2007
Year 2008
Year 2009
Year 2010
Competition
Hospital
Dummy
Inpatient Satisfaction
Overall Inpatient
Staff and
Logistics and
Satisfaction
Treatment
Infrastructure
0.028***
0.014**
0.028***
(0.006)
(0.006)
(0.007)
0.041***
0.028***
0.039***
(0.004)
(0.006)
(0.007)
0.049***
0.036***
0.039***
(0.006)
(0.006)
(0.007)
0.061***
0.043***
0.056***
(0.005)
(0.005)
(0.007)
0.050***
0.042***
0.071***
(0.007)
(0.006)
(0.008)
0.045***
0.041***
0.070***
(0.010)
(0.006)
(0.010)
0.004***
0.004***
0.005**
(0.001)
(0.001)
(0.002)
0.029***
0.028***
0.014
(0.007)
(0.007)
(0.010)
Outpatient Satisfaction
Overall Outpatient
Staff and
Satisfaction
Treatment
0.022***
0.019***
(0.003)
(0.003)
0.031***
0.035***
(0.004)
(0.004)
0.035***
0.035***
(0.005)
(0.004)
0.039***
0.044***
(0.005)
(0.004)
0.038***
0.043***
(0.005)
(0.004)
0.040***
0.044***
(0.005)
(0.003)
0.002**
0.002*
(0.001)
(0.001)
-0.022***
-0.029***
(0.005)
(0.005)
Size
0.004
(0.003)
0.005*
(0.003)
0.001
(0.004)
-0.002
(0.002)
-0.003
(0.002)
R2
0.116
N = 993
0.154
N = 923
0.108
N = 923
0.132
N= 1007
0.173
N = 1004
Notes to Table 4: (1) Results are based on a fractional response econometric model (Papke and
Wooldridge 1996) of the following form: Satisfaction= α+β1-6 Year Dummy + β7 Competition
+ β8 Hospital Dummy+ β9 Size. The dependent variable (patient satisfaction) is scaled as a percentage of
the range of possible outcomes. (2) Year 2004 is the dropped dummy. (3) The models control for firm
fixed effects and uses robust standard errors clustered by firm. (4) The standard errors in the estimation of
the average partial effects (APE’s) are calculated using the delta-method. (5) The regression estimation
results (untabulated) were similar when considering only hospitals or only sanatoriums. (6) Statistical
significance of the estimated coefficients is indicated with * = p-value <0.10; ** = p-value <0.05; *** =
p-value <0.01.
43
Table 5
Drivers of change in patient satisfaction
Panel A: Dependent variable = unscaled change in patient satisfaction (robust standard errors in
parentheses)
Variable
Satisfaction
score in 2004
Competition
Hospital
Dummy
Size
Year Dummies
R2
Overall
Inpatient
Satisfaction
-0.087***
(0.025)
0.001
(0.001)
0.013*
(0.007)
-0.003
(0.003)
Yes
0.207
N = 802
Inpatient Satisfaction
Staff and
Logistics and
Treatment
Infrastructure
-0.130**
(0.051)
0.012***
(0.004)
0.037*
(0.023)
-0.003
(0.010)
Yes
0.061
N = 748
-0.155***
(0.027)
0.010*
(0.005)
0.047
(0.032)
-0.004
(0.013)
Yes
0.078
N = 748
Outpatient Satisfaction
Overall
Staff and
Outpatient
Treatment
Satisfaction
-0.103***
-0.086***
(0.015)
(0.013)
0.001
0.001
(0.001)
(0.004)
0.001
-0.002
(0.004)
(0.022)
0.002
0.006
(0.002)
(0.009)
Yes
Yes
0.143
0.085
N = 846
N = 843
Panel B: Dependent variable = change in patient satisfaction scaled by initial potential for
improvement (robust standard errors in parentheses)
Inpatient Satisfaction
Outpatient Satisfaction
Variable
Overall
Staff and
Logistics and
Overall
Staff and
Inpatient
Treatment
Infrastructure
Outpatient
Treatment
Satisfaction
Satisfaction
Satisfaction
-0.066**
-0.034***
-0.040***
-0.135***
-0.172***
score in 2004
(0.027)
(0.013)
(0.009)
(0.042)
(0.005)
Competition
0.002
0.004***
0.002
0.002*
0.001
(0.002)
(0.001)
(0.002)
(0.001)
(0.001)
Hospital
0.018*
0.029**
0.021*
-0.001
-0.002
Dummy
(0.011)
(0.014)
(0.011)
(0.006)
(0.005)
Size
0.001
0.001
0.002
0.002
0.002
(0.004)
(0.044)
(0.005)
(0.003)
(0.003)
Year Dummies
Yes
Yes
Yes
Yes
Yes
R2
0.085
0.031
0.049
0.059
0.057
N = 796
N = 748
N = 748
N = 846
N = 843
Notes to Table 5: (1) Results in Panel A are based on a fixed effects regression model of the following form:
Change in Satisfaction= α+β1 Satisfaction Score in 2004+β2 Competition
+β3 Hospital Dummy+ β4 Size+β5-10 Year Dummies.(2) In Panel A change in satisfaction is defined as satisfaction in
Yeart – Yeart-1, while in Panel B, change in satisfaction is scaled by the potential for improvement assessed in 2004
((Satisfaction in Yeart – Satisfaction in Yeart-1)/(5 – Satisfaction in 2004)). (3) Year 2004 is the dropped dummy. (4)
The model controls for firm fixed effects and uses robust standard errors clustered by firm. (5) Similar estimation
results (untabulated) were obtained when considering hospitals only or sanatoriums only. (6) Statistical significance
of the estimated coefficients is indicated with * = p-value <0.10; ** = p-value <0.05; *** = p-value <0.01.
44
Table 6
Trend regressions for change in patient satisfaction
Panel A: Dependent variable = change in patient satisfaction (robust standard errors in
parentheses)
Unscaled change
Variable
Overall change
(Constant)
Trend Variable
Low Initial Performer
Dummy
Trend*Low Initial
Performer Dummy
Competition
Hospital Dummy
Size
R2
Overall Inpatient
Satisfaction
0.082***
(0.021)
-0.017***
(0.002)
0.147***
(0.033)
-0.027***
(0.009)
0.001
(0.001)
0.017**
(0.007)
-0.002
(0.003)
0.051
N =802
Overall
Outpatient
Satisfaction
0.031**
(0.015)
-0.009***
(0.002)
0.079***
(0.016)
-0.015***
(0.004)
0.001
(0.001)
0.011**
(0.005)
0.003
(0.002)
0.039
N =846
Change scaled by initial potential for
improvement
Overall Inpatient Overall Outpatient
Satisfaction
Satisfaction
0.097***
(0.026)
-0.024***
(0.004)
0.089***
(0.029)
-0.014*
(0.008)
0.001
(0.001)
0.022**
(0.011)
0.001
(0.004)
0.043
N = 796
0.007
(0.028)
-0.006**
(0.003)
0.062***
(0.017)
-0.012***
(0.004)
0.001
(0.001)
0.015**
(0.007)
0.004
(0.004)
0.019
N = 846
45
Panel B: Dependent variable = unscaled change in patient satisfaction (robust standard errors in
parentheses)
Variable
Overall change
(Constant)
Trend Variable
Low Initial Performer
Dummy
Trend*Low Initial
Performer Dummy
Competition
Size
R2
Hospitals only
Overall Inpatient
Overall
Satisfaction
Outpatient
Satisfaction
0.071***
0.069***
(0.024)
(0.020)
-0.019***
-0.014***
(0.003)
(0.004)
0.155***
0.046**
(0.033)
(0.019)
-0.027***
-0.009*
(0.008)
(0.005)
0.002
-0.001
(0.001)
(0.001)
0.003
0.003
(0.116)
(0.003)
0.107
0.085
N = 346
N = 340
Sanatoriums only
Overall Inpatient Overall Outpatient
Satisfaction
Satisfaction
0.101***
(0.032)
-0.015***
(0.004)
0.152***
(0.039)
-0.029***
(0.011)
0.001
(0.002)
-0.009
(0.006)
0.047
N = 456
0.021
(0.021)
-0.007**
(0.003)
0.108***
(0.027)
-0.020***
(0.007)
0.001
(0.001)
0.001
(0.004)
0.025
N = 506
Notes to Table 6: (1) Results in Panel A are based on a fixed effects trend regression model of the
following form: Change in Satisfaction = α+β1 Time Trend +β2 Low Performer Dummy
+β3 Time Trend*Low Performer Dummy+β4 Competition+β5 Hospital Dummy+β6 Size.
(2) In Panel A, Columns 1 and 2, change in satisfaction is defined as satisfaction in Yeart – Yeart-1, while
in Columns 3 and 4, change in satisfaction is scaled by the potential for improvement assessed in 2004
((Satisfaction in Yeart – Satisfaction in Yeart-1)/(5 – Satisfaction in 2004)). (3) Panel B reports the
coefficients and robust standard errors for the regression of unscaled changes estimated separately for
hospitals and sanatoriums. Similar estimation results (untabulated) were obtained using scaled changes as
the dependent variable. (4) All models controls for firm fixed effects and calculate robust standard errors
clustered by firm. (5) Statistical significance of the estimated coefficients is indicated with * = p-value
<0.10; ** = p-value <0.05; *** = p-value <0.01.
46