ECHO SAB meeting
Brussels; 10-11th June 2013
Meeting Venue
"ORBAN" building DG Research (European Commission); Room number: 6A146 (6 th floor ORBAN building)
Square Frere Orban 8,
1049 Brussels, Belgium
SAB Members present
Josep Figueras (European Observatory for Health Systems)
Edward Kelley (WHO)
Nicolaas Sieds Klazinga (OECD)
Alberto Holly (Lausanne University)
Guillem Lopez-Casasnovas (International Health Economic Association)
Patrick Romano (University of California-Davis)
Dirk Van den Steen (European Commission)
President
Rapporteur
SAB Discussion Points
General comments, passed by the SAB, cover:

overall methodological concerns, addressing the need:
o to enrich presented analyses with contextual information, related to data measurement
issues (definitions of concept and measurement units) as well as country health systems /
local healthcare delivery practices.
o to ensure face validity of presented findings (e.g. cross-walks, cross-validation/calibration
using national aggregates from external sources).

satisfaction with dissemination strategy, specifically in view of the policy relevance of presented Atlas
dummy documents and the expected effectiveness of the proposed policy dialogue tool. An
additional comment.

concern on timely project delivery.
Specific comments, passed by the SAB address the same three aspects of the ECHO project: methodology,
policy relevance and project monitoring.
1. Methodological points

Definition of MAREs: provider-based resident attribution criterion (to counter-act possible extravariation) may imply supply-induced demand patterns can only be assessed to a lesser extent.

Hospital efficiency:
o Useful work on applicability of stochastic frontier analysis is welcomed by the SAB
o Productivity ratio approach will require further discussion of possible cross-country variation
in concept definitions (e.g. "hospital bed", "nurse") as well measurement unit issues (FTE vs
headcount approach). Possibility to cross-validate/calibrate using OECD/WHO/ESTAT data
should be explored. (Note Rapporteur: several SAB members indicated further comments in
writing to be be forthcoming)

Further(beyond hospital efficiency bullet above) points noted to address specific contextual/definition
issues:
o 4 hour emergency department target in England as a potential driver of higher admission
rates, which might be remedied by comparing LOS distributions across countries and
potentially excluding same-day discharges.
o Residual problems in ICD-9 to ICD-10 mapping, such as the lack of complete equivalence of
definitions for asthma and chronic obstructive lung disease.

Equity:
o Section on possible contextual confounders (national health system / local healthcare
delivery practices) would be welcomed linked to extensive section of related hypotheses for
future research.
1/2

Dissemination:
o Policy Dialogues (LDGs):
 the targeted outcome should bet to put project findings on the political agenda. In
this sense, further policy context (e.g. observed overall trend towards more
integrated care) should be highlighted and terminology/graph legends, etc. made as
accessible as possible. Also, sound preparation of LDGs will include scanning for
already available evidence at local level on geographic care utilisation variability.
 possibility to demonstrate / include guidance on usability of ECHO data for hospital
efficiency assessments should be explored.
o E-Handbook:
 The SAB regrets time was lacking to cover this point. Members will send comments
in writing.
o On-line tool:
 The SAB welcomes the promising work delivered, which is likely to be of great
benefit to the research community (e.g. sparking research hypotheses for countries
out of ECHO scope).
o Atlas "dummy" document:
 The SAB welcomes the presented output. The choice of interventions considered
(potentially avoidable hospitalisations) is well considered as are the dimensions
covered (regional, international, time trend and social gradient). Exploration of
question as to whether a consistent "degree of avoidability" can be found across
considered interventions could be strengthened (variability rank order within regions,
countries, across time, etc.).
o (Note rapporteur: Scientific publication strategy was discussed with SAB during oral
comments session and found to be largely satisfactory as regards targeted journals and
possible thematic clustering. The possibility to present overall project findings via a special
edition journal format focusing on sustainability of health systems in light of population
ageing.)
o Possible further media/platforms: specific mention of project presentation and related
exchange of ideas was made in view of WHO-worldbank co-operation on hospital efficiency.
2. Policy relevance of project results

The presented Atlas dummy (for use in LDG) stressed policy area of pressing relevance (see also
European Semester Country Specific Recommendations proposed by Commission that stress need
to make EU health systems less hospital centric).

Specific relevance was also identified in the national legal restrictions (possibly with technical
implications in addition) hampering the set-up of the DataWareHouse. It was suggested to try
address the issue in the EU eHealth Network 1, which has a one of its objcetives to "draw up
guidelines on (ii) effective methods for enabling the use of medical information for public health and
research"

(Note rapporteur: in the oral SAB comments sessions stress was put on presenting findings as a tool
to help and not a pillory. Further it was noted that updated analyses (2010-2012) would help buy-in
by politicians)
3. Project monitoring
1

The SAB would have welcomed a clearer breakdown of the noted delay ("9 to 12 months") into
underlying "delay drivers".

It is understood that the assembly/compilation of the DWH posed specific contractual/legal issues.

The noted delay in the cross-walk face validation process is worrisome as prima facie acceptance of
presented results in LDGs is crucial (cf. supra: related to specific issues noted under 3rd bullet of
methodological points section).
See http://ec.europa.eu/health/ehealth/policy/network/index_en.htm
2/2