Evaluation report on the activities
of the Economics and Research
Department of Banco de Portugal
2014
Vítor Gaspar
Fátima Barros
Ricardo Reis
João Santos
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Lisbon, 2014
Design Banco de Portugal | Administrative Services Department | Editing and Publishing Unit
ISSN (online) 2183-377X
Contents
Disclaimer | 7
Acknowledgements | 9
Executive summary: Conclusions and main recommendations | 11
Introduction | 16
PART 1. Activities
1. Participation in the Eurosystem | 18
Box 1: Decision-making in the Eurosystem | 18
Box 2: Preparation of macroeconomic projections | 20
2. Policy advice to the Governor of Banco de Portugal | 20
3. Research | 21
3.1. Publications in academic journals | 22
Box 3: ECB points system for publications | 22
3.2. Working Papers | 24
4. Regular publications and communication | 25
4.1. Regular publications | 25
4.1.1. Regular publications produced by DEE | 25
4.1.2. Release and impact of DEE’s publications in the media | 26
4.2. DEE’s other institutional publications | 28
Box 4: Book A Economia Portuguesa no Contexto da Integração Económica,
Financeira e Monetária (The Portuguese Economy in the Context
of Economic, Financial and Monetary Integration) | 30
4.3. Seminars, courses, conferences and workshops | 31
4.4. DEE’s internal communication | 32
5. Institutional cooperation | 32
5.1. Other departments of Banco de Portugal | 32
5.2. Domestic institutions | 33
5.3. International institutions | 34
PART 2. Special topics
6. The Portuguese economy | 35
6.1. Economic indicators | 35
6.2. Forecasting | 36
6.3. Structural studies | 37
7. Create a new Financial Intermediation Division in the DEE | 38
7.1. DEE’s past Financial Intermediation related activities | 39
7.2. The case for a Financial Intermediation Division within the DEE | 40
7.3. Proposed Financial Intermediation Division’s responsibilities | 41
PART 3. Organisation and Human Resources
8. Organisation | 43
9. Demography | 44
10. Recruitment process | 44
11. Careers | 46
12. Training | 49
13. Mobility | 50
PART 4. Recommendations and suggestions
Main recommendation 1: Set up an accountability framework | 51
Main recommendation 2: Improve human resource management | 51
Main recommendation 3: Identify financial intermediation
and European financial integration as priorities for the next three years
and create a financial intermediation division | 53
Main recommendation 4: Pursue excellence in research | 54
Main recommendation 5: Rethink communication | 54
6. Other relevant recommendations | 55
Charts and Tables
Table 1.1 International institutional representation of Banco de Portugal in 2013 | 56
Table 1.2 International institutional representation of the DEE in 2013 | 56
Chart 3.1 Developments in the total number of articles published in peer-review journals
and in articles weighted by ECB ranking | 57
Chart 3.2 JEL CODES of the articles published in peer-review journals (number of articles) | 57
Chart 3.3 Concentration of the number of articles (by economist) | 58
Chart 3.4 Working Papers published in the Banco de Portugal series | 58
Table 3.1 List of publications according to the Point System for Publication of the ECB | 59
Chart 4.1 Evolution of the total number of news items relating to the DEE and a breakdown
by publication | 60
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Chart 4.2 Evolution of the total number of news items relating to the Quarterly Economic
Bulletin and a breakdown by media type | 60
Chart 4.3 Breakdown of the total news items relating to the DEE’s publications (for 2013) | 61
Chart 4.4 Number of pageviews of the Quarterly Economic Bulletin through Banco de
Portugal’s institutional website (from 2012; data per publication) | 61
Chart 4.5 Frequency of consultation of the DEE’s regular publications | 62
Chart 4.6 Breakdown of OTSs per DEE publication | 62
Chart 4.7 Evolution of the total number of news items relating to the Annual Report and a
breakdown by media type | 63
Chart 4.8 Evolution of the total number of news items relating to the Monthly Economic
Indicators and a breakdown by media type | 63
Chart 4.9 Number of pageviews of the Monthly Economic Indicators through Banco de
Portugal’s institutional website (from 2012; data per publication) | 64
Chart 4.10 Number of pageviews of the Coincident Indicator through Banco de Portugal’s
institutional website (from 2012; data per calendar month) | 64
Chart 4.11 Evolution of the total number of news items relating to the Financial Stability
Report and a breakdown by media type | 65
Chart 4.12 Number of pageviews of the Financial Stability Report through Banco de
Portugal’s institutional website (from 2012; data per publication) | 65
Chart 4.13 Evolution of the total number of news items relating to the DEE and a
breakdown by media type | 66
Chart 4.14 Each media type’s share of news items relating to the DEE (for 2013) | 66
Chart 4.15 Shares of each support type used to consult the DEE’s publications | 67
Chart 4.16 Number of pageviews of the Book A Economia Portuguesa no Contexto da
Integração Económica, Financeira e Monetária (The Portuguese Economy in the
Context of Economic, Financial and Monetary Integration) through Banco de
Portugal’s institutional website (from 2012; data per calendar month) | 67
Chart 4.17 Seminars and courses organised by the DEE | 68
Table 9.1 DEE staff (as of 31 December 2013 of each year) | 68
Table 12.1 Examples of specific internal initiatives focusing on technical issues | 69
Table 12.2 DEE’s external training (2011-2013) | 69
Table 13.1 Internal mobility – DEE (2011-2013) | 70
Table 13.2 External mobility – DEE (2011-2013) | 70
Annexes
Annex I. Independent Commission’s Mandate for assessing DEE’s activities | 71
Annex II. Acronyms and abbreviations | 72
Annex III. Sources of information used | 72
5
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Disclaimer
The report is signed by Professor Vítor Gaspar (Chairman), Banco de Portugal, Professor Fátima
Barros, ANACOM – Autoridade Nacional de Comunicações, Professor Ricardo Reis, Columbia University and Professor João Santos, Federal Reserve Bank of New York. This is the composition of
the independent evaluation commission appointed by Banco de Portugal to assess the activities
of the Economics and Research Department (from here on “DEE” or “Department”).
The views expressed are those of the authors and do not necessarily reflect those of Banco de
Portugal, Eurosystem, ANACOM – Autoridade Nacional de Comunicações, the Federal Reserve
Bank of New York or the Federal Reserve System.
7
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Acknowledgements
The members of the Evaluation Commission want to thank the Management team of the Economics and Research Department for their outstanding cooperation during the process and, in
particular, for all the information made available and their willingness to have extended exchanges of views with the Evaluation Commission.
We would also like to thank Jorge Correia da Cunha (advisor to the Management team of the
Economics and Research Department) and DEE’s division heads for all their collaboration. The
members of the Evaluation Commission also thank all the staff of DEE for their availability to have
interviews with the Commission and sharing their views on the functioning of the Department.
We also want to thank all of those in Banco de Portugal who read the report and made insightful
comments and suggestions and to those who were available to have interviews with the Commission and answer to the questionnaires that were sent.
The Commission also thanks all the support work provided by Hélia Mendes (Secretariat to the
Commission).
We want to thank the current and former Governors and the other Members of Banco de Portugal Board of Directors for all their collaboration and for sharing their opinions about the Department.
9
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Executive Summary:
Conclusions and main recommendations
Our overall assessment of the Economics and Research Department of Banco de Portugal in the
period under review (2000-2013) is positive.
DEE provides analysis, information and policy advice to the Governor especially in his role as
member of the ECB’s Governing Council. The Department actively participates in the work of the
Eurosystem and the European System of Central Banks. DEE’s contributions are highly appreciated by relevant stakeholders. Contributions to the ECB’s monetary policy strategy, including the
role of monetary aggregates and economic forecasts were particularly relevant.
DEE produces reference work. Its publications are considered by economists as important for
their analysis of the Portuguese economy. The Quarterly Economic Bulletin and the Annual Report
were, in the period under review, the flagship publications. Gradually the Quarterly Economic
Bulletin, which includes economic forecasts for the Portuguese economy, has become the most
important publication. It now enjoys significant prominence and is widely used by the press, the
general public, professional economists and researchers. That is clear from media reporting,
internet access, and opinions surveyed. In general the visibility of work on the Portuguese economy seems to be increasing. This trend benefited from a substantial rise in the importance of
electronic access and the use of Banco de Portugal’s publications.
A good example is provided by The Portuguese Economy in the Context of Economic, Financial and
Monetary Integration released in 2009. This book brings together diversified contributions from all
strands in the Department. It effectively uses synergies between research and policy work. It relies
on research that was previously published in some of the best academic journals. In 2013 the book
was still accessed more than 20,000 times on the institutional website of the Banco de Portugal.
DEE produces analytical work up to the standards of leading peer-reviewed academic journals.
Between 2000 and 2013, DEE’s economists have published 172 papers in academic journals.
19 of these made it to top-ranked journals. Over time there is a clear upward trend. DEE is ranked
among the top economic research institutions in Portugal. It has established a reputation for
excellence in research inside the Eurosystem. The main fields of publication are unsurprisingly
monetary economics and macroeconomics. A strong strand in the Department makes use of
large micro datasets to investigate topics such as financial intermediation, labour markets, product markets, and international trade. This places DEE at the cutting edge of on-going research.
Over the years DEE has made relevant contributions in the area of financial intermediation and
financial stability. The regular publication of the Financial Stability Report since 2005 is particularly
noteworthy1. The Department has also successfully participated in the Eurosystem’s Household
Finance and Consumption Network.
In the 2004 Evaluation Report of Research Activities at the ECB, the evaluators wrote: “Academically oriented economists provide additional benefits to a central bank ( ) such economists can
model economic circumstances not ordinarily, if ever, encountered in reality, such as financial
instability or deflation, and develop strategies for dealing with them”. The ability to adapt to
unforeseen circumstances has been illustrated, in the context of the financial crises, by the pro-
1.In 2013, the Banco de Portugal decided to create the Financial Stability Department to help the Bank promote and safeguard financial system stability.
The new department was entrusted with the responsibility of coordinating the Bank’s financial stability related activities and policies.
11
12
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
duction of the Funding and Capital Plans and the Stress Tests.
DEE has been able to attract and maintain outstanding staff. In the last 10 years staff has averaged 77. In 2013 it declined to 73 as some of its staff moved to the newly established Financial
Stability Department. The share of PhD economists has increased, from 1/5 in 2000 to 1/3 in
2013. The Department has a strong deontology based on sound analysis and research. That
allows staff to work with remarkable cohesion even under stressful conditions.
Our overall assessment of DEE is positive. Nonetheless, the Economics and Research Department Evaluation Commission (from here on “Commission”) is of the view that there is potential for
improvement. Given the quantity and quality of resources available, DEE can move closer to the
efficiency frontier. In order to do so it should address gaps in some areas. Firstly, management
practice could improve by following a top-down approach to strategic planning and priority-setting. Systematic evaluation of results against targets should also be adopted as standard practice.
Secondly, attracting, retaining and developing talent is a key goal. Hence, the recruitment strategy
should clearly target the Department’s needs. Thirdly, financial factors will be key drivers of Portugal’s performance going forward. Institutional changes linked to Banking Union will shape the role
of Banco de Portugal. Hence, financial intermediation and European financial integration should
be key priorities over the medium term. Fourthly, excellence in research should be continued.
Finally, the Department is at the center for Banco de Portugal’s institutional publications. Publications should be clearly structured, targeted, focused, and streamlined.
Hence the Commission submits a number of recommendations and suggestions. The main recommendations are included in this Executive Summary. The complete list of recommendations
and suggestions is included at the end of the Report. The main recommendations concern the
management and internal organization as well as some of the Department’s main outputs.
Main recommendation 1
Set up an accountability framework
Given the long lead times associated with research and analytical work DEE should develop an
accountability framework. The framework would allow the Department to account for its use of
resources in fulfilling its tasks and functions. The accountability framework should encompass
the full programming cycle: planning, monitoring, evaluation, and communication. Annual planning of activities should build on the priorities defined in Banco de Portugal’ strategic planning
and define deliverables and key performance indicators to evaluate progress. Research activities
and analytical work should be programmed on the basis of clear rules and taking into account
the Bank’s priorities. Information on the planning and programming of activities should be available to all staff.
The accountability framework requires that metrics are available to measure the quantity and
quality of work produced. In many cases it will be necessary to ensure systematic feedback from
relevant stakeholders. That will be certainly the case for policy work and the provision of consultancy work by DEE to other departments of the Bank.
Conducting regular external evaluations of DEE could be part of the accountability framework.
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Main recommendation 2
Improve human resource management
Human resources are the Department’s key asset. Attracting, developing, and retaining talent is a
key managerial challenge. Human resource management should be integrated with the accountability framework so as to ensure full alignment of purpose. Such alignment should include the clear
setting of performance goals so as to ensure objectivity in the ex post evaluation of performance.
In the Commission’s view it is important to develop a framework allowing for professional development and career progression. Banco de Portugal could consider the possibility of opening an
advisory career path that would parallel the current managerial path. The advisory career path
would be open to excellent professionals. It would be open to a select few focusing on policy
work, as well as to highly productive researchers. In any case, the Commission is of the view that
the rules applying to promotion of researchers should be clearly defined and communicated.
One important aspect is that sufficient research time should be guaranteed. In particular, economists recruited directly after completing their PhD should be given a period of time where they
would be (almost) exclusively focused on getting their papers published.
The Banco de Portugal could consider setting up a system aimed at identifying high potential staff.
The management of these careers should involve internal and external mobility.
In the view of the Commission efficiency in research and analytical work could substantially improve
if DEE would make use of Research Analysts who could perform a large number of routine tasks,
thereby enabling staff to concentrate on more demanding tasks.
Main recommendation 3
Identify financial intermediation and European
financial integration as the priorities for the next three
years and create a Financial Intermediation Division
This Commission is of the view that financial intermediation and European financial integration
will be crucial for Banco de Portugal and for the country in the next years. Firstly, the changing
financial structure will be a fundamental factor shaping trends and performance in the Portuguese economy. Secondly, Banking Union will imply that decision-making on micro and macro
prudential will approach the Eurosystem standard for monetary policy.
Consistent with this, “Financial stability in the European context” is the first of four strategic guidelines set by Banco de Portugal for the next three years. The Department can contribute to the
strategic goal in a number of different ways.
It should enhance the analysis and forecasting of the Portuguese economy through the explicit
modeling of the financial factors. It should also contribute to the estimation of monetary and
credit conditions in Portugal and the understanding of the monetary transmission mechanism.
A broader issue is the evolution of financial intermediation in Portugal in the context of a financial
transformation in Europe. For this purpose it is crucial to explore synergies inside DEE.
In the area of financial stability it is particularly critical that policy advice by Banco de Portugal staff
is organized as an inter-departmental process so as to ensure that information, knowledge and a
13
14
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
diversity of views are adequately reflected in the process of reaching an institutional position. DEE
should thus cooperate intensively with other business areas of the Bank. It should produce relevant
research and provide analytical instruments and policy analysis to the macro and micro prudential
departments. It should also be an active participant in the editorial process leading up to the Financial Stability Report, which should continue to be led by the Financial Stability Department.
In order to allow DEE to build the necessary capacity to contribute to these key priorities, the
Commission recommends the creation of a Financial Intermediation Division in DEE.
Main recommendation 4
Pursue excellence in research
On research the Commission is of the view that the path of excellence should be pursued further.
Research should aim at publication in leading edge scientific journals. Clear career incentives for
productive researchers are fundamental. Another important contribution would be the adoption of a point-system for classifying scientific publications and to quantify research work. Such
indicator would be used when assessing staff performance. The Commission would suggest
that a system similar to the one used by the ECB could serve this purpose. It is also important
to explore synergies between research and policy work.From this viewpoint, clear signaling of
priority areas for Banco de Portugal as a policy-making institution would be particularly valuable.
Main recommendation 5
Rethink communication
On the contribution of DEE to external communication, the Commission is of the view that some
re-thinking would be appropriate in order to improve targeting and focus. In particular communication policy should identify the targeted groups, their demands and the purposes of communication. Quarterly Economic Bulletin has become the Banco de Portugal’s flagship publication. It
is now the main vehicle of dissemination of the central bank’s views on the current situation and
prospects for the Portuguese economy. It is the main vehicle of communication with the general
public. The Commission suggests one senior economist of the Department should be entrusted
with the task of editor of Quarterly Economic Bulletin. That would facilitate the transition to a
simpler and more direct style of communication. The production process should be associated
with internal procedures aimed at early debate of the overall assessment and relevant policy
messages. Quarterly Economic Bulletin articles should be accessible to a wide audience and focus
on relevant topics for the policy debate.
The Commission is also of the view that the Annual Report could make a complete transition to
a standard corporate Annual Report format. In the same vein Indicadores de Conjuntura could be
limited to factual tables and charts. The release of Coincident Indicators could be accompanied by
a devoted press release.
On the research side, Working Papers should be the series to disseminate research done at Banco
de Portugal. The aim would be early dissemination of analytical work that would potentially be
published in top-level scientific journals. Working Papers should be submitted for publication in
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
academic-refereed journals. The series should be the responsibility of DEE but open to submissions from staff of other departments.
Other analytical work could be disseminated in a new series: Staff Technical Notes. This new series
would contain the background analytical work behind some Quarterly Economic Bulletin articles to
be published. It would also replace the Occasional Papers that would be discontinued.
15
16
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Introduction
The Report is the first strategic review of the Economics and Research Department of Banco
de Portugal. Strategic reviews of research activities in policy-making organizations are becoming
standard practice. In a number of cases, it has been decided that such strategic reviews will be
conducted on a regular basis.
The scope of the Report is wide. It reflects the mandate given to the Commission. The report
covers research activities conducted in the Economics and Research Department of Banco de
Portugal. Research is here understood as the production of pieces of analysis with the potential
to be published in good academic refereed journals. In doing so it follows a considerable number
of recent evaluation reports. Excellence in research is important in any policy-making institution.
It is particularly so for national central banks (from here on “NCB”) participating in the Eurosystem.
In times of crisis, high quality research is even more important as the conduct and operation of
policy deviates from well-trodden paths.
For research publications and for institutional publications the Commission assesses their quantity, quality, relevance and impact. For research activities the evaluation of quantity, quality and
relevance follows a practice that has become standard in recent years.
But the mandate of the Commission is considerably broader than research. It includes the evaluation of other activities that are at the core of the Department. Namely, policy advice in the
context of the Eurosystem and the European System of Central Banks (in particular on monetary
policy); advice on financial stability (both at national and at European level); advice to the Government on the conduct of economic policies in Portugal; and other pertinent issues of central
banking policy. The mandate also involves evaluating the Department contribution to the Bank’s
institutional publications (including the Annual Report and the Quarterly Economic Bulletin). The
Department represents Banco de Portugal in a large number of Eurosystem structures. It also
ensures representation in the European Union (from here on “EU”), Organization for Economic
Co-operation and Development (from here on “OECD”) and the Eurostat. Last but not least the
Department contributes to the capacity of Banco de Portugal by attracting, forming and retaining
human resources.
The Banco de Portugal is a member of the Eurosystem. The primary objective of the Eurosystem
is to maintain price stability. In order to do so, the Eurosystem defines and implements the single monetary policy. The Eurosystem is governed by the decision-making bodies of the ECB: in
particular the Governing Council and the Executive Board. The NCB, including Banco de Portugal,
are an integral part of the Eurosystem and act in accordance with the guidelines and instructions
of the ECB.
The financial architecture of the euro area and of the EU was transformed by the Global Crisis.
The creation of the Single Supervisory Mechanism places the ECB at the center of banking regulation and supervision. The European Systemic Risk Board has a complementary role in the area
of macro-prudential policies. In any event the Governing Council is likely to be at the crux of the
new procedures. Hence, there is now some parallelism between the decision-making procedures
for monetary policy and financial stability. Given that the Governor is member of the Governing
Council the organization of decision-making puts the Governor at the center of Banco de Portugal’s participation in the Eurosystem.
According to the most recent version of its organic law, the Banco de Portugal has extensive
domestic competences in the area of financial stability and banking. Specifically, Banco de Portugal is responsible for the stability of the Portuguese financial system; Banco de Portugal is
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
responsible for the definition and implementation of macro-prudential policy; Banco of Portugal
is responsible for banking supervision; and, finally, Banco de Portugal is the national resolution
authority in the country.
Last but not least, Banco de Portugal also has the task of advising the Portuguese government in
matters of economic and financial policy.
The international institutional activity of Banco de Portugal is concentrated in Europe. About 2/3
of the total number of international institutional meetings in which the Banco participates are
meetings of the various structures of the Eurosystem. The share increases to about 90% when
other EU structures are added. The remainder meetings refer to various committees and working
groups that function in the context of the International Monetary Fund (from here on “IMF”), the
BIS and the OECD.
The Economics and Research Department contributes to the international representation of
Banco de Portugal. In particular, the Economics and Research Department contributes to the
policy work of Banco de Portugal in the context of the Eurosystem. It provides advice on monetary
policy. It participates in the Monetary Policy Committee and its Working Groups. In this context
it participates in the Eurosystem Staff Macroeconomic Projections exercises (see Box 2). In the
area of financial stability, the Economics and Research Department produces extensive analysis
pertaining to macro and micro adjustment and stability.
Domestically, Banco de Portugal cooperates with other financial supervisors. In this domain the
National Council of Financial Supervisors, chaired by the Bank, is particularly important. Banco
de Portugal cooperates with the Statistics Portugal (from here on “INE”) as part of the National
Statistical System. Banco de Portugal also acts as economic adviser to the Government in matters
of economic and financial policy. It also cooperates with the Ministry of Finance and the Court of
Auditors in matters relative to fiscal policy.
In the context of the sovereign debt crisis in the euro area and the Portuguese adjustment programme (2011-2014), Banco de Portugal contributed constructively to the various pillars of the
programme. That role was particularly focused on financial stability, in general, and on the banking system, in particular.
The Economics and Research Department also produces in-depth analysis and research of structural features of the Portuguese economy. The goal is to contribute to better information and
understanding, improving the quality of the policy debate, and, ultimately, contributing to better
policy-making. That role was particularly intense during the run-up and execution of the adjustment programme.
For policy, representation and advisory work the Commission based its evaluation and recommendations on publicly available material especially the institutional publications of Banco de Portugal.
This basic information was supplemented by a number of initiatives. Specifically, the Commission:
• Conducted a large number of structured interviews. It interviewed key stakeholders and all
Department staff;
• Relied on questionnaires sent to users of publications released by Banco de Portugal;
• Used a media survey as a measure of broad public relevance and impact;
and
• Used data on access to the various documents from Banco de Portugal’s web site.
The Commission findings are presented in the following sections.
17
18
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
PART 1. Activities
Banco de Portugal has three main functions. Firstly it is a member of the Eurosystem. This membership derives from its role as central bank of the Portuguese Republic. Secondly, it has far-reaching
responsibilities in the area of financial stability and supervision of credit institutions. Thirdly, it advises
the Government on economic and financial policy. The Economics and Research Department contributes to all these areas of Banco de Portugal’s activity.
1. Participation in the Eurosystem
Banco de Portugal is a member of the Eurosystem. This brings a set of responsibilities that are
defined in the Statutes of the European System of Central Banks and of the European Central Bank
(from here on “Statutes”) and in the Organic Law of Banco de Portugal.
Box 1: Decision-making in the Eurosystem2
The Eurosystem is comprised of the European Central Bank (from here on “ECB”) and the
NCB of the EU Member States that have adopted the euro.
In accordance with Article 8 of the Statutes, the European System of Central Banks (from
here on “ESCB”) it is governed by the decision-making bodies of the ECB. The decision-making process is thus fully centralised. Article 9.3 of the same document defines that the
ECB’s decision-making bodies are the Governing Council and the Executive Board. Articles
10 and 11 of the Statutes of the ESCB and of the ECB describe how each decision-making
body operates and Article 12 defines their responsibilities. Under Article 12.1 “The Governing Council shall adopt the guidelines and take the decisions necessary to ensure the performance of the tasks entrusted to the ESCB under these Treaties and this Statute. The
Governing Council shall formulate the monetary policy of the Union (...). The Executive Board
shall implement monetary policy in accordance with the guidelines and decisions laid down
by the Governing Council. ( )” Article 12.2 indicates that the Executive Board prepares the
agenda for the Governing Council’s meetings.
The Eurosystem’s decision-making process comprises three key phases: the preparation
phase, the decision phase and the implementation phase.
The decision phase involves both the Executive Board and the Governing Council. The
Governing Council comprises all the members of the Executive Board and the governors of
the Eurosystem NCB.
The NCBs, as such, have an active role in the preparation and implementation phases.
The preparation phase involves the technical contributions of the ESCB / Eurosystem’s staff
and Committees. The goal of this phase is to gather information and achieve consensus.
Each Committee has a set mandate that defines what contribution should be prepared as
input for the decision-making process. The Committees are authorised to delegate work of
a more technical nature to Working Groups with a specific mandate or to task forces that
remain operational until the task is complete. The Economics and Research Department
represents Banco de Portugal at the Monetary Policy Committee and its sub-structures.
In the implementation phase, the NCB carry out the Governing Council’s decisions. The Executive Board is responsible for giving the NCB the instructions needed for this. DEE is not
involved in the implementation phase.
2. This Box was based on Occasional Paper No 79 published in the ECB’s Occasional paper series (“The Workings of the Eurosystem – Monetary Policy Preparations and Decision-Making – Selected Issues”, by Phillipe Moutot, Alexander Jung and Francesco Paolo Mongelli).
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Banco de Portugal’s international activity focuses mainly on the ESCB / Eurosystem, as would be
expected (Table 1.13). In 2013, Banco de Portugal took part in 480 face-to-face meetings and
275 conference calls in that particular field, which represents approximately two-thirds of the
total.
DEE’s international activity has an important role for Banco de Portugal. It took part in 53 meetings and two conference calls with the main bodies of which it is a member, also focusing on the
ESCB / Eurosystem (Table 1.2). In total, DEE’s representation work is spread across approximately
30 groups and over 20 representatives.
Among these is the work in the Monetary Policy Committee (from here on the “MPC”). The MPC is
one of the technical Committees making up the Committee structure of the ESCB / Eurosystem.
As mentioned in “Box 1”, the MPC takes part in the preparation phase for the Governing Council’s
decisions. In this regard, the most important decisions are those relating to monetary policy.
“Box 2” looks at the preparation of macroeconomic projections, one of the elements used by the
Governing Council in taking monetary policy decisions. The MPC supports the Eurosystem in setting the single monetary and exchange rate policy for the euro area, contributing among other
things to:
• Analysis of topics related to the conduct of monetary policy in the euro area;
• Research on the instruments underlying the analysis of the current economic, monetary and
financial situation;
• Preparation of a set of macroeconomic projections for the euro area as a whole;
• Exchange of opinions and information on the economic, monetary and financial situation from
a euro area perspective;
• Analysis of the overall performance of the Eurosystem’s monetary policy operational framework.
• There are three Working Groups under the aegis of the MPC, namely, the Working Group on
Public Finance (WGPF), the Working Group on Forecasting (WGF) and the Working Group on
Econometric Modeling (WGEM). DEE also takes part in certain research networks and in some
research groups on a voluntary basis.
In order to assess DEE’s activity, surveys were conducted, which were submitted to some members of the Eurosystem staff4. According to their responses5, DEE’s participation is deemed to be
of high quality in its contribution (i) to the technical debate at regular meetings, (ii) to the development of new methodologies and (iii) to the peer-review of macroeconomic and fiscal projections. The analysis and presentation of the developments in the Portuguese economy were also
assessed as positive, specifically in terms of (i) the quality of the statistical information and the
transparency of the information provided, (ii) the technical quality of the analysis of past developments and the macroeconomic and fiscal projections and (iii) the ability to provide insights into
the macroeconomic, financial and fiscal developments in the Portuguese economy.
3.See Charts and Tables.
4.This group includes staff that participates in Working Groups and Committees with DEE representatives.
5.The conclusions drawn should take into account the low number of responses to this survey.
19
20
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Box 2: Preparation of macroeconomic projections
Preparation of the macroeconomic projections by the Eurosystem staff is one example
of the way the Eurosystem Committees contribute to the preparation of the monetary
policy decisions.
The macroeconomic projections for inflation, GDP, fiscal variables and other macroeconomic variables for the euro area are produced twice a year by the Eurosystem staff,
with ECB staff exclusively producing two further projections a year. The macroeconomic
projections are an analytical tool that condenses a broad information set on current and
future economic developments. They provide an insight into the most probable scenario
for the euro area over a two-year time horizon6. The preparation of the macroeconomic
projections involves complex interaction between the ECB and the NCB.
2. Policy advice to the Governor of Banco de Portugal
As explained in “Box 1”, the Eurosystem is managed by the decision-making bodies of the ECB: the
Governing Council and the Executive Board. The Governor of Banco de Portugal is a member of the
Governing Council. In this capacity, the Governor operates fully independently, in accordance with
the ECB’s mandate and objectives.
DEE’s tasks include providing the Governor of Banco de Portugal with high-quality advice on monetary and financial policy. The quality and relevance of the arguments produced is a key element for
the Governor’s performance in the Governing Council.
The Governing Council usually meets twice a month. In the first meeting of each month it analyses the economic and monetary developments and takes monetary policy decisions. In the second meeting usually it tackles issues related to other tasks and responsibilities of the ECB and the
Eurosystem7. Furthermore, the Governing Council also schedules steering debates on background
issues, for example the examination of the monetary policy strategy in 2003.
The policy advice activities undertaken by DEE may be broken-down into those that are carried out
as part of the preparation for the Governing Council meetings and a heterogeneous set of other
policy advice activities. The latter may be considered in the context of the advice given to the Government on economic and financial matters.
DEE’s support activities related to the Governor’s participation in the Governing Council include: (i)
the monthly monetary policy briefing, (ii) the bimonthly discussion of monetary policy topics and (iii)
the quarterly discussion of the projections for the Portuguese economy.
Each month a briefing with the Governor takes place as part of the preparation for the Governing
Council’s monetary policy meetings. At this meeting, DEE summarises the developments of the prior
month and suggests the direction to follow in the conduct of monetary policy. The presentation is
based on a document called the “Euro area policy brief” and a booklet with a statistical appendix
(“Background information”), which includes a broad set of economic indicators. In the current version, the briefing’s format also allows for a discussion between the Governor and the DEE team.
6.This horizon was extended to three years in March 2014.
7.In July 2014 the ECB adjusted the schedule of meetings of the Governing Council. The Governing Council meetings dedicated to monetary policy will
change to a new six-week cycle, from January 2015. The meetings not dedicated to monetary policy will continue to be held at least once a month.
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Indeed, following a direction from the Governor8, since February 2012, a significant change took
place in the organisation of the technical debate on monetary policy. Special sessions were included
on monetary policy topics. These sessions are now held six times a year.
Interviews held with Governors and members of the Board of Directors of Banco de Portugal have
indicated that the monthly briefing held by DEE is a valuable contribution. The support documents
are timely produced and well organised.
In certain cases, DEE’s contribution to the policy debate was particularly relevant. Two examples: (i)
the critique of the role of the M3 monetary aggregate in the ECB’s monetary policy strategy and (ii)
the risk assessment methodology associated with the macroeconomic projections for the euro area.
DEE also holds a quarterly presentation and discussion on projections for the Portuguese economy
with the member of the Board of Directors responsible for DEE and thereafter with the Governor. This
discussion is based on the forecast report (Relatório de Previsão). This document presents the international context assumptions for the projection exercise and the comparison with the prior projection
exercise. Furthermore, the document sets out the current projections and those of the prior projection
exercise (and the difference between them), for a set of macroeconomic and fiscal variables.
Other policy advice activities include: (i) the weekly report on economic and fiscal developments, (ii)
the quarterly presentation of the unsigned text for the Quarterly Economic Bulletin, (iii) the discussion of topics on the Portuguese economy, (iv) the discussion of the articles presented at Jackson
Hole and (v) the presentation of the Board of Directors’ Annual Report on the Portuguese economy.
The weekly report on economic and fiscal developments is presented at the Internal Markets Meeting with the Board of Directors. This report is based on a data set that is updated whenever new
information is released. A booklet providing economic information (Informação de Conjuntura)
is updated with data released by INE, among other sources. The booklet “Information on public
finances” is updated using the Directorate-General for the Budget’s monthly bulletin and INE’s quarterly and annual national accounts.
Concerning the unsigned text for the Quarterly Economic Bulletins and the Board of Director’s Annual
Report on the Portuguese economy, DEE prepares presentations highlighting the key points of the
texts. These are delivered respectively to the member of the Board of Directors responsible for DEE
and the Board of Directors of Banco de Portugal.
The discussion of topics on the Portuguese economy takes place twice or three times per year.
3. Research
Analytical work at DEE takes different shapes. Some of it is closely tied to the policy debates of the
day and is best described as policy work because it provides input to the policy advising function.
Its quality is partly assessed by the usefulness that this analysis has to other bodies and structures
of Banco de Portugal. Another part consists of applied research on Portugal, which is closely linked
to Banco de Portugal advisory role on economic and financial policies. Much of this research ends
up being published in academic journals, but its ultimate goal is to advance our knowledge of the
Portuguese economy. Finally, there is academic research, whose ultimate intent is to advance economic knowledge, and so whose success is measured by publication in peer-reviewed scientific
journals and subsequent citations. Academic research helps to establish the Bank’s reputation,
8.These meetings were introduced because the monthly monetary policy briefing meetings were insufficient to debate relevant monetary policy issues.
21
22
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
it often provides the foundations for the other two research goals, it allows the DEE to attract the
top-notch researchers that can do the other tasks, and it provides an important public good for the
country. Moreover, and this should be emphasized, it is the commitment to research that provides
the unifying deontological perspective that unites the whole Department in its multiple functions.
All of these types of analytical work are mixed within the DEE. There is no clear distinction between
either the people who are supposed to engage in one versus other forms of work, or between how
these different activities should be differentially stimulated. In many ways, this is a good setup, and
some level of mixing is surely optimal. Central banks are precisely institutions where interaction
between these three strands of research work should exist and be fostered, with gains for the quality
of each of the three types.
Moreover, when we surveyed economists at the bank, 90 % replied that the regular publications of
the Banco de Portugal are either essential or important for their work. An overwhelming number
reported consulting these publications, and may do so very often. Therefore, even internally, there
is great demand for the Bank’s publications, and acknowledgment of their usefulness.
For the purposes of this report, we separate research stricto senso as analytical work aiming at
meeting the requirements for publication in good academic refereed journals. Therefore, in this
section we concentrate on publication in academic journals and working papers. This creates some
difficulties with evaluating the DEE, since many of its staff members perform analytical work that is
not ultimately published in academic outlets. Because there is no clear separation within the Department between researchers and others, it is hard to measure research output per researcher,
as opposed to research output per economist, as we will do below.
3.1. Publications in academic journals
Between 2000 and 2013, the economists at the DEE published 172 papers in refereed journals.
Chart 3.19 shows the evolution of those publications over time, both in raw terms, as well as weighting each publication by the quality ratings of different journals used by the ECB. Box 3 describes
briefly the grading system of the ECB. Chart 3.1 shows an upward trend, with the years since 2008
having significantly more articles published. On average, the number of publications as a ratio of
number of staff members with a PhD is in line with the output at the ECB.
Box 3: ECB points system for publications
The ECB follows a points system for publications. Publications in what are deemed top
general interest journals are placed in tier 1* and given 12 points (6 journals), with top field
journals in core policy areas and other very good general interest areas placed in tier 1 and
receiving 10 points (7 journals). After that, a publication in a tier 2 receives 5 points and
includes other good general interest journals as well as top field or policy journals in relevant areas (24 in total), tier 3 has 2 points for other journals in not core areas and other field
and general interest journals, and finally 1 point for tier 4 which includes all other journals.
The full list of ranking of economic journals is in Charts and Tables. The rankings are revised
every three years (we used the 2011 categories) and co-authorship is discounted by weighting the points by 2/(1+N), where N stands for the number of authors. Finally, “papers and
proceedings” or non- or lightly-refereed contributions to journals are ranked one tier below
the journal itself. The list of economic and financial journals can be found in Table 3.1.
9.See Charts and Tables.
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
There are quite a few high quality publications, with 19 out of the 172 articles published in tier 1
and tier 1* journals. Moreover, well below half of the articles (62) were published in tier 4 journals.
The economists at the Bank are being adequately ambitious, sending their papers mostly to visible journals, instead of yielding to the temptation to publish wherever.
Turning next to measures of impact of these papers, we were able to find citations in Web of Science to 63 % of these published articles, again a respectable number. The total number of citations
is 767, but only 32 articles received more than 5 citations, and only 10 have more than 20 citations.
Again, this is not an unusual pattern for research departments, including academic ones.
We can also look at the most widely cited articles per year since publication to ensure that they
are in topics of relevance to the Bank. We find that they study the unemployment rate (Blanchard
and Portugal), the distribution of firm entry (Mata and Portugal), and the use of single or multiple
banks by firms (Farinha and Santos)10. All of these use Portuguese data and inform important
policy debates in Portugal. They are all very good examples of what central-bank based research
can and should produce.
An alternative assessment of impact is provided by the website research repository IDEAS. It produces rankings of the top institutions producing economic research in Portugal, based on number and quality of publications, as well as on downloads and citations. The Banco de Portugal consistently appears as one of the top two institutions in Portugal, together with the Nova School of
Business and Economics, and with the third-ranked institution behind by a considerable distance.
In terms of fields covered in these publications, Chart 3.2 shows that, unsurprisingly, macroeconomics and monetary economics is the main field of focus. The closely associated field of mathematical and quantitative methods comes second, while the Bank of Portugal seems to have a
specific strength in labour economics, which is less typical in central banks. This strength is also
evident by looking at the keywords in the articles published where, after the boilerplate “econometric / statistical”, the most prevalent keywords were “employment / unemployment”. When
asked in a survey what were the main areas where the working papers were more influential and
useful, the Bank’s staff points to three main areas: employment / unemployment, macroeconomics and public finances.
Looking at the distribution not across fields, but across authors, Chart 3.3 shows a high concentration of research output. As usual, a few economists account for a large fraction of the total
number of publications and citations of the Department: Pedro Teles, Pedro Portugal and Isabel
Horta Correia alone account for 136 of the total 393 points in the ECB scale for the entire DEE.
This should not be a concern, as the market for research output is well known to have a “star”
structure. In terms of the internal organization of the Department, the 10 top economists (not
all of which are predominantly researchers) account for 56 % of all the Web of Science citations
accrued.
The fact that 28 of the 76 economists who worked in the Department during the period under
analysis have not published a single article deserves some attention. While it is normal to have
a few members that are more devoted to other tasks, or that are at a different stage in their
careers, it is not desirable such a large share of non-publishing economists in an economics and
research department.
10. The article by Blanchard and Portugal came out as working paper 1998-03 and published in the American Economic Review, 2001; the one by Mata
and Portugal was WP 2001-01 and published in the Strategic Management Journal in 2002; the one by Farinha and Santos never came out as Working
Paper and was published in the Journal of Financial Intermediation.
23
24
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
3.2. Working Papers
The working papers series of DEE published 227 working papers between 2000 and 2013. Graph
3.4 shows the evolution of the number of published working papers per year. There isn’t as pronounced an upward trend as in the case of peer-reviewed publications indicating a slight uptick
in the number of working papers that are eventually published in journals.
Compared with the typical Federal Reserve Bank, the number of publications is smaller but not
by much: at one extreme we have the Federal Reserve Bank of New York, which produced regularly more than 60 working papers per year over the past decade, and at the other extreme,
the Federal Reserve Banks of Atlanta, with approximately 24, and Kansas City with approximately 14. Relative to other central banks in the Eurosystem, the performance is quite good.
For countries similar in population size, the Bank of Finland produced 31 working papers on
average over the last decade, the National Bank of Belgium 18, the Bank of Greece 16, the Oesterreichische Nationalbank 10, and De Nederlandsche Bank 9. At roughly 20 working papers
per year for, on average, slightly less than 50 economists, DEE produces one paper every two
and a half years. By comparison, at the recent evaluation of the Banco de España, concern was
expressed over there being on average one working paper put out per researcher every two
years. In the view of the Commission, this is not so much an indication of low productivity by
the economists at DEE, but rather that they are not all publishing their research work in the
working paper series.
Looking at the journals where working papers were eventually published, we detected two patterns that deserve some attention. First, less than half (75) of the peer-reviewed publications by
DEE economists (172) were included in the Banco de Portugal working paper series. Working
papers should be the main outlet for the academic output of the DEE, and would be a natural
metric to judge their output in real time, as they are more immune to delays in publications.
Second, 92 of the 211 working papers were never published elsewhere, neither in peer-reviewed
journals, nor in other Bank publications, and excluding publication in other working paper series,
the number rises to 102. As the title indicates, a working paper should reflect work in progress
that is going to be published shortly in a final outlet. Yet, some economists at DEE use the working papers as a final stop for their research. It should be noted that this is not only a problem at
DEE. The IMF’s most recent evaluation report also noted that only about one third of their working
papers were published within three years of being released. The comparable numbers at the ECB
or at the Federal Reserve Banks in the United States are significantly higher.
The Commission selected a random sample of the working papers and read them to assess their
contribution. The quality was variable, with some of the work being of top-notch quality, either as
pieces of academic or applied research, while some other working papers were of distinctly lower
quality, and were not publishable either at an academic journal or at Quarterly Economic Bulletin.
Papers are accepted for publication at the working paper series without being subject to refereeing, and after some very mild editing by one editor external to the Bank.
From the Commission’s conversations and interviews with different members of the Bank, it was
clear that there is some confusion about the role of the working papers. Some criticized them
for dealing with topics that are not relevant to Portugal. Yet, it is the nature of the exploratory
work in academic research that some of it may be more theoretical, or even applied to other
countries, as knowledge is being accumulated on an area before it reaches a stage where it
can be applied to the country’s problems. Others worried that the conclusions in the working
papers might sometimes be in contradiction with the official policies of the Bank. Again, this is to
be expected of the more academic inclined work, and should never be discouraged at the risk
of losing the DEE’s important role of exploring policy alternatives. Finally, some worried that the
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
working papers did not have any relevant impact in public debates. This is connected to the use
of working papers as final stops for research. If, as it should be, the release of a working paper
is only an intermediate step in the research process before its ultimate publication elsewhere,
this issue would be moot.
4. Regular publications and communication
4.1. Regular publications
DEE produces various publications on a regular basis, with different audiences in mind. In this section we assess the impact of these publications on these various audiences, and DEE’s contribution
to raising the quality of the economic information and the public debate in Portuguese society.
4.1.1. Regular publications produced by DEE
In the period under review, DEE was responsible for producing the following regular publications (i) the Board of Directors’ Annual Report – The Portuguese economy11 (from here on the “Annual
Report” or the “AR”), (ii) the Quarterly Economic Bulletin (from here on also “QEB”), (iii) the Financial
Stability Report (from here on also “FSR”) and (iv) the Monthly Economic Indicators (from here on
also “MEI”). These publications are made available through Banco de Portugal’s institutional website. Paper versions of the QEB and the AR are also published. The MEI have been available in
electronic format only since January 2014.
The Annual Report presented a detailed analysis of the Portuguese economy’s development over
the previous year, with the goal of framing that behaviour within a structural perspective and
comparing it to other economies.
The Quarterly Economic Bulletin is a publication that aims to present regular analyses of the Portuguese economy, including “Special Issues” and Boxes which discuss relevant and timely economic
policy themes. The QEB also contains signed articles, frequently of a more analytical nature.
In 2014, alterations were made to the April Quarterly Economic Bulletin and the Annual Report.
The April 2014 Quarterly Economic Bulletin includes detail on the performance of the Portuguese
economy in 2013, replacing the longer text that was traditionally published with the AR. Furthermore, the release of the 2014-2016 projections was brought sooner to March12, to comply with
the Eurosystem’s publication rules13. The April QEB thus simply reproduced the March projections.
In turn, the Annual Report, published at the end of May, focuses mainly on structural issues, as the
developments in 2013 had already been analysed in the Quarterly Economic Bulletin of April. A first
part aims to explain the low trend growth in the Portuguese economy in the most recent period
and also to present three issues that are particularly relevant in the current national context: the
11. Up to the 2010 edition, a single Board of Directors’ Annual Report was produced, containing two sections: a first part on developments in the Portuguese economy, drafted by the DEE, and a second part on Banco de Portugal’s activity and accounts, compiled by the Office of the Governor based on
departmental contributions regarding activity and from the Accounting Department regarding the accounts. From the 2011 edition onwards, two Board of
Directors’ Reports were produced: “Annual Report: Activities and Financial Statements” and “The Portuguese Economy”. The DEE retains responsibility for
producing the report on the Portuguese economy.
12. In March this year, the DEE published a separate document of projections for the Portuguese economy for the current year and the following two
years. This document will be published yearly.
13. Eurosystem rules do not allow national central banks to publish projections in the period running from two months before the publication of the Eurosystem Staff Board Macroeconomic Projection Exercise (BMPE) and one month after its publication (unless they are published simultaneously with the BMPE).
25
26
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
deleveraging process, the functioning of the labour market and recent institutional reforms in the
European Union. In the second part there is an overview of the Portuguese economy’s ongoing
adjustment process.
The Monthly Economic Indicators include a summary and tables and charts that cover the most important indicators for Portugal and the international economy (particularly in terms of the euro area).
Up to 2013, DEE also produced the Financial Stability Report. However, with the creation of the
Financial Stability Department this responsibility was transferred to the new department, despite
benefiting from DEE’s contributions. This publication includes the risk assessment for the Portuguese financial system.
4.1.2. Release and impact of DEE’s publications in the media
With the goal of measuring the impact and the users’ assessment of DEE’s publications, the Evaluation Commission commissioned an impact study on the media, carried out by the company
Manchete, surveyed the publications’ users, gathered information on the number of pageviews of
the publications on Banco de Portugal’s institutional website and held interviews themed on the
publications14. The study carried out by Manchete focused on Banco de Portugal’s publications
and ran from October 2006 to December 201315. The survey of the publications’ users aimed to
learn about a selected sample of the users of DEE’s publications and covered DEE’s publications
and a number of publications of the Statistics Department16. Data on pageviews relate to publications produced by DEE and cover the period from January 2012 to 30 April 201417.
Data reported and the respective conclusions must be read with caution, insofar as the analysis
contains a number of important limitations18. Still, certain relevant conclusions result from the
analysis of the information gathered.
Firstly, DEE’s publications have a high impact, which increased sharply in the period under review.
The total number of news items on DEE’s publications doubled from around 950 in 2007 to
around 1,900 in 2013 (Chart 4.119).
14. Some of the questions asked in the interviews focused on the usage frequency and relevance of the DEE's publications and any suggestions for
improving the publications.
15. Note that the last quarter of 2006 was excluded from the analysis in order to use full-year data only. The research includes publications for which the
DEE and the Statistics Department are responsible. The DEE's publications covered were: the QEB, the AR, the MEI, the FSR, the Bank Lending Survey and
Research / Working Papers. From the Statistics Department’s publications were included the Statistical Bulletin and the Central Balance-Sheet Studies. Some of
Banco de Portugal's publications were not covered by Manchete's research: the Market Conduct Supervision Report, the Payment Systems Report, the Bulletin
on Banknotes and Coins, among others. Manchete's study aims to understand developments in the exposure of Banco de Portugal's publications in the media,
assessing among other things the evolution of news produced over the period under review, the impact of the news produced and the kind of news media used.
16. The survey included questions on the following publications: the QEB, the AR, the MEI, the FSR, the Statistical Bulletin (from the Statistics Department),
the QEB and FSR articles and Working Papers. The questions aimed to analyse usage frequency of the publications, the format used for consultation, its
relevance and specific information consulted, among other aspects. Some questions were also asked about conferences and seminars held by the DEE
and on its relationship with other departments. In parallel, a few questions were sent to a group of Eurosystem staff members on DEE’s participation in
committees and working groups that DEE takes part in.
The number of surveys sent was 249 and the number of responses amounted to 71.
17. Information was analysed on: the QEB, the AR, the FSR, the MEI and the Coincident Indicator. Information for one complete year of pageviews was
used for the publications. From 16 February 2012 and 30 March 2012 there are no available pageview data.
18. The limitations of reported data concern Manchete’s survey, the surveys sent and pageview data.
With regard to Manchete’s survey: (i) the OTS measure is only a summary indicator of print media and does not measure the impact of other media and (ii) the
study only covers Banco de Portugal’s publications that are the responsibility of DEE and the Statistics Department, and excludes publications of other Departments.
With regard to the surveys sent: (i) the number of responses was small (71 out of 249 surveys sent) and (ii) a very significant number of responses were from
DEE’s staff (33) and academics (15).
As regards pageview data, information is only accessible from January 2012 onwards, and there are no available data between 16 February and 30 March 2012.
19. See Charts and Tables.
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
The number of pageviews for the AR, the QEB, the FSR and the MEI exceeded 100,000 in 2012.
For the period as a whole, the opportunities-to-see (from here on “OTS”20), a summary measure
for print media, reached 119 million.
According to the user survey, the publications are used mostly to obtain general information on economic and financial matters, to produce reports and for research. Around 90 % of the respondents
attribute a high or average importance to Banco de Portugal’s regular publications for their work.
Secondly, we concluded that in relative terms the Quarterly Economic Bulletin stands apart clearly as
the seminal publication. Its importance has increased over time across all media (Chart 4.2). The QEB
is the subject of more news items when compared to other regular publications (67 % of the total in
2013, versus 41 % in 2007) (Chart 4.3); it receives a high number of pageviews (around 7,000 per QEB)
(Chart 4.4); and is also mentioned by the users as the publication most frequently consulted, according to the survey of users of Banco de Portugal’s publications (85 % of respondents use the QEB at
least once a quarter) (Chart 4.5). Analysis of the OTS also suggests that the QEB is the publication with
the greatest impact, with 47 % of the OTS associated with news items in print media on DEE publications (Chart 4.6). The study and surveys indicate that the Quarterly Economic Bulletin is used by a broad
set of readers, notably the press, the general public, professional economists and researchers.
A third conclusion relates to the fact that the impact of the other regular publications seems to
be markedly lower than that of the Quarterly Economic Bulletin. The evidence for this is the lower
number of news items published, and the lower OTS level for the other publications.
With specific regard to the Annual Report, there has also been a significant fall in news items
published, reaching a relatively low level in the past few years (Chart 4.7). However, looking at
the number of pageviews on Banco de Portugal’s institutional website, the AR holds a prominent
position. The number of pageviews for the AR exceeded 11,000 for 2011. Therefore the low
coverage of the AR in the news may be related to the fact that it focuses on the description and
analysis of economic developments of the year before and is released late in the year. For this
reason the relevance for the media is low. By contrast, the high number of pageviews may be
due to the fact that the AR presents a detailed analysis of the annual trends in the Portuguese
economy, which makes it a useful source of information for consultation, and as a result, a reference document. According to the survey of users of Banco de Portugal publications, over 70 %
of respondents use the AR at least once a year. With the recent changes to Banco de Portugal’s
publications, specifically the AR and the QEB of April, the QEB will henceforth present this analysis
of the previous year.
For the Monthly Economic Indicators, the number of news items remained relatively stable, at a low
level per issue (Chart 4.8) but, even so, higher in overall terms than the other publications (apart from
the Quarterly Economic Bulletin). The user survey confirms the lower relative relevance of the MEI
versus the QEB (50 % of respondents use the MEI at least once a quarter). The number of pageviews
is lower too, but still significant (Chart 4.9). The number of pageviews of the MEI is around 3,000
per publication. The split between English-language and Portuguese-language versions is balanced.
This contrasts with the AR and the QEB where the Portuguese pageviews predominate. Analysing
the monthly average individualised consultations of the Coincident Indicator, we see that it is above
the average for the Monthly Economic Indicators (Chart 4.10). Furthermore, it is mainly the Englishlanguage version of the Coincident Indicator that is consulted on Banco de Portugal’s institutional
website. One may conclude that many users prefer to look exclusively at the Coincident Indicator,
and never go as far as looking at the Monthly Economic Indicators publication.
20. The OTS measure is calculated from the print media's circulation (data from the Portuguese Association for Print Run and Circulation Control). The
print media's circulation is an estimate of the number of copies sold (number of subscriptions plus newsstand sales).
27
28
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Regarding the Financial Stability Report, the number of news items increased slightly21 (Chart 4.11).
On average there were around 5,000 pageviews of the FSR per publication (Chart 4.12).
A fourth conclusion is that in the period under review, the internet took on increasing importance
as a means for releasing news about the publications (Chart 4.13), and as a means for consulting
the publications themselves. In 2013, almost 60 % of the news items on DEE’s publications were
released through the internet (Chart 4.14). The results of the user survey show that around twothirds of the respondents exclusively use the electronic version of the publication (Chart 4.15).
The rising relevance of the internet has mirrored a falling relevance of print media. In the print
media the specialized media stands out22 (the top six producers of news about Banco de Portugal
belongs to the specialised print channels, covering 26 % of the news generated about the publications researched by Manchete).
Furthermore, in interviews held with a number of relevant current and ex-stakeholders and members of staff of Banco de Portugal, other conclusions were also reached about the publications.
In general, the relevant stakeholders interviewed mention the fact that DEE’s publications are an
important vehicle for Banco de Portugal communication.
Some relevant stakeholders and members of staff mentioned the need to simplify the terminology and
the message transmitted in the publications, making the language clearer for the reader23. There were
also some references to the need for the texts to be more analytical and less descriptive. To this end,
both for the Quarterly Economic Bulletin and for the Annual Report, there were occasional references to
potential for improvement by providing a less extensive text, including an executive summary.
Regarding the Quarterly Economic Bulletin, various relevant stakeholders mentioned the importance of this publication for releasing economic projections on the Portuguese economy. Some of
the relevant stakeholders mentioned the need to improve the communication of the uncertainty
behind a projection exercise. Some respondents also suggested the possibility of the articles published in each QEB and FSR to be available separately. Regarding the Annual Report, this publication
was deemed to be relevant for a wide audience as it presents developments in the Portuguese
economy for the year before. Regarding the Monthly Economic Indicators, various respondents
highlighted the fact that it contains important statistical information, in particular the Coincident
Indicators. However, several interviewees mentioned that the publication could be replaced by
simplified ways of releasing information. Regarding the Financial Stability Report, it was said to be a
particularly important publication for the Portuguese banking system’s reference.
4.2. DEE’s other institutional publications
The set of publications falling under DEE’s responsibility include the Bank Lending Survey, Inquérito à Situação Financeira das Famílias (household finance and consumption survey), the Occasional Papers from the series edited by Banco de Portugal and the Research Newsletter. DEE
also produced the book A Economia Portuguesa no Contexto da Integração Económica, Financeira e
Monetária (The Portuguese Economy in the Context of Economic, Financial and Monetary Integration), published by Banco de Portugal (see Box 4).
21. Note that the FSR became a biannual publication in 2010.
22. However, the general media have a much greater share of total news than the specialised press, as they greatly outnumber the specialised press.
23. In the user survey, one of the suggestions that arose the most was writing publications for a non-technical audience.
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
The Bank Lending Survey, held quarterly, is part of the Eurosystem’s Bank Lending Survey and
aims to obtain more detailed information on the credit market.
The Inquérito à Situação Financeira das Famílias, started in 2010, is part of the European project
called the Household Finance and Consumption Survey24, carried out within the Eurosystem. This
project aims to describe in detail the financial situation of households such that the data are comparable across euro area countries. In Portugal, the Inquérito à Situação Financeira das Famílias is
the responsibility of Banco de Portugal and INE. In 2015, the results of this survey (held in 2013)
will be published again, and thereafter this publication will come out every two years.
The Occasional Papers series has the purpose of presenting pertinent economic policy topics.
These documents contain work by Banco de Portugal economists. Six papers have been presented to date.
The Research Newsletter, called Spillovers, is published twice a year through Banco de Portugal’s
institutional website. It was produced for the first time in October 2013 and presents selected
research in summary form, a list of recently published articles, new titles in Banco de Portugal’s
working papers series and reports from conferences held. In addition, it also announces the
upcoming conferences and seminars and includes short interviews with some of the economists.
The Newsletter has less impact than DEE’s regular publications but is nevertheless significant,
with pageviews of the first edition numbering slightly over 2,000.
24. Inquérito ao Património e Endividamento das Famílias (household wealth and indebtedness survey), the Inquérito à Situação Financeira das Famílias'
precursor, was published in 1994, 2000 and 2006/2007. The Inquérito ao Património e Endividamento das Famílias was a Portugal-only initiative, and was
also the responsibility of Banco de Portugal and INE.
29
30
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Box 4: Book A Economia Portuguesa no Contexto da Integração Económica,
Financeira e Monetária (The Portuguese Economy in the Context
of Economic, Financial and Monetary Integration)
In 2009, DEE published a book entitled “A Economia Portuguesa no Contexto da Integração
Económica, Financeira e Monetária” (The Portuguese Economy in the Context of Economic,
Financial and Monetary Integration). This work covers a broad set of topics on the Portuguese economy and its economic, monetary and financial integration into the euro area
and the EU.
It serves a wide audience. One indicator that measures the frequency of recent consultations of the book is the number of pageviews it has received on Banco de Portugal’s institutional website. For 2013 it received over 20,000 pageviews (Chart 4.16).
The book is an example of DEE economists with very different analytical profiles combining
their skills. This publication thus reflects the complementary relationship that has to exist
between research and policy advice.
The book’s chapters reflect a series of papers published by Banco de Portugal. Some of
these original contributions were published in top international scientific journals: The American Economic Review, The Review of Economic Studies, Journal of Economic Theory and Journal
of Labor Economics, among others.
The book highlights the great change in the Portuguese economy following the economic
integration brought by membership of the EU and the globalisation process. It shows that
the Portuguese economy has recorded rapid financial integration through its participation
in the euro area. Published in 2009, the book predates the financial fragmentation that
took place after 2010. The following themes are raised in the analysis of the Portuguese
economy: price and wage formation, developments in output and unemployment, external
trade, public finances, the financial system and the financing conditions facing households
and firms.
Three main objectives are identified in the book: (i) to present a comprehensive overview
of the Portuguese economy in the last two decades, (ii) to present empirical and theoretical
contributions in an integrated way, and a broad set of statistical information resulting from
analysis of existing and new databases and (iii) to encourage academic debate and future
research.
Three horizontal issues are covered by the book’s chapters: (1st) the importance of the
changes to the regime and incentive structure for economic agents, (2nd) the importance
of combining micro- and macroeconomic analyses, and (3rd) the challenge of economic
growth in Portugal. In relation to the first issue, the book illustrates four dimensions that
reveal the importance of the change to the regime and incentives: the ECB’s monetary
policy decisions, fiscal policy decisions, change to the comparative advantages of the Portuguese economy and the change to the regime following entry into the euro area. The book
identifies various examples in relation to the second issue: research on price and salary
rigidities (Chapter 3), analysis of export destination decisions by firms (Chapter 5), comparison between private-sector and public-sector salaries (Chapter 6) and assessment of
companies’ and households’ balance sheets and their consequences for macroeconomic
stability (Chapter 7). Regarding economic growth in Portugal, the book makes clear that it is
a particularly complex issue, which is a major challenge for the future.
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
4.3. Seminars, courses, conferences and workshops
DEE periodically organises various seminars, courses, conferences and workshops.
Regarding the set of seminars and courses carried out by the Department, during the period in
question, external seminars accounted for about 50 %, internal seminars approximately 43 %
and courses around 7 %.
Over the last five years the number of seminars and courses stabilised, with an annual average
of 40 (Chart 4.17).
The external seminars are aimed at specialists from Banco de Portugal and other domestic and
foreign institutions, principally university professors and journalists.
The internal seminars comprise seminars designed for DEE economists25. The presentations may
be given by the Department’s economists, visiting researchers or those on a research grant and
recruitment candidates26. From 2009 the number of internal seminars fell, becoming more infrequent. This is partly due to increased DEE activity following the economic and financial crisis and
the start of the Economic and Financial Assistance Programme to Portugal. In February 2014, a
series of internal seminars was started on a weekly basis, called Exchange and reserved for members of the Department and visiting researchers. These seminars have various purposes: the
presentation of papers, the discussion of specific topics, the advanced presentation of articles to
be published in the Quarterly Economic Bulletin and the presentation of the economists’ research
agenda, among others. The interviews held with DEE staff suggested that these seminars were
positively received by DEE economists, significantly boosting information exchange and the discussion of ideas.
The courses normally last a week and are presented by foreign academics essentially in the areas
of macroeconomics, microeconomics, econometrics and the financial system. They are open to
DEE economists, economists from other Banco de Portugal departments, university professors
and economists from other central banks. As they attract the participation of a broad range of
Banco de Portugal staff, they have proved to be a very efficient advanced training method.
From 2000 to 2013, DEE organised a total of 37 conferences and workshops, which involved a
limited number of participants and were by invitation only. The events organised included Conferences on the Labour Market and Monetary Economics, alternating every two years. Also taking place every two years are the ‘Portuguese Economic Development in the European Space’
Conferences. The conferences and workshops involve the participation of staff from Banco de
Portugal and other domestic and international institutions.
The survey of users of the regular publications also featured some questions about participation
in conferences and seminars. Most respondents often take part in conferences and seminars
organised by Banco de Portugal. These events are mainly used for research and to garner general information on economic and financial affairs. One of the main suggestions forthcoming was
to organise seminars and conferences on economic and financial themes of the day as a way to
improve the information released to the public.
25. From time to time, these seminars may be opened to other Banco de Portugal staff.
26. The recruitment seminars are one of the phases of the recruitment process in the PhD Job Market for those who have just graduated from doctorate
programmes, aiming to allow a restricted set of candidates pre-selected by departmental staff to present PhD Job Market Papers.
31
32
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
4.4. DEE’s internal communication
One of the ways DEE economists communicate within the Department is through the internal
seminars (see Section 4.3). Informally, there is also regular contact between economists to share
information and discuss work in progress. There are several partnerships within the framework
of research projects involving economists from DEE and external institutions.
DEE’s activities are now planned each year, in a document accessible to departmental staff.
However, the interviews held with DEE economists revealed that more regular communication is
needed on research projects in progress. This regular release of information will give the department a deeper understanding of DEE’s activity and in turn will generalise techniques developed
by individual departmental staff members.
5. Institutional cooperation
When performing its tasks, DEE keeps direct contact with various Banco de Portugal departments
and different domestic and international institutions.
5.1. Other departments of Banco de Portugal
DEE works in close cooperation with those departments linked to core aspects of Banco de Portugal’s mission, particularly with a view to ensuring price stability and contributing to financial system stability. In this vein, DEE currently collaborates regularly on the one hand with the Markets
and Reserve Management Department and on the other with the Financial Stability Department.
Collaboration with the Markets and Reserve Management Department involves preparing monthly monetary policy briefings for presentation to the Governor. In addition, DEE also collaborates
with the Markets and Reserve Management Department on a regular basis in terms of external
representation, in the Eurosystem committees where both departments participate. The Markets and Reserve Management Department also shares basic information for some DEE studies
and joint policy notes. With the financial crisis, collaboration between the two departments grew
markedly, with an increased need for the exchange of information.
Collaboration between DEE and the Financial Stability Department and to a lesser extent with the
Banking Prudential Supervision Department is linked to activities relating to financial system stability. Cooperation with the Financial Stability Department mainly involves the analysis of economic and
financial developments, while discussing macroeconomic forecasting, stress tests and risks to the
banking system. There is also a regular joint participation in meetings with investors and, where needed, in technical groups of the European Systemic Risk Board. In addition, DEE provides analytical instruments relevant for the Financial Stability Department’s regular activities. As regards the Financial Stability Report, DEE participates through signed articles. The DEE also contributes to the follow-up reports
on the banking system with empirical studies, which involve frequent collaboration between staff from
the two departments. Nowadays cooperation with the Banking Prudential Supervision Department is
more sporadic, with one-off contributions from DEE to the analysis of specific topics.
DEE also has a strong interaction with the Statistics Department, using data compiled by the latter and collaborating in order to promote quality statistics. Interaction between the two departments takes place at the level of databases and statistics, notably the Central Balance‑Sheet
Database, the Central Credit Register, credit market statistics, public finance statistics, banking
system-related statistics, and balance of payments statistics, among others.
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Collaboration with the International Relations Department takes place mainly in the preparation
of the briefing for the Governor and international cooperation initiatives, notably with Portuguese-speaking African countries. There is also an interaction with the Office of the Governor to
prepare supporting documentation on the Portuguese economy for presentation in seminars,
conferences and other events.
DEE’s collaboration with Banco de Portugal’s Porto Branch is targeted at developing the Laboratório de Investigação em Microdados (BPlim, a Microdata Research Laboratory) at the latter’s
premises. This structure will be a partnership among several departments: Statistics Department,
DEE, Financial Stability Department and the Banking Prudential Supervision Department.
DEE also collaborates with other departments of Banco de Portugal, although less regularly, such
as with the Payment Systems Department.
Some Heads of Department answered the surveys, mentioning collaboration with DEE in various
formats, notably in briefings and working groups, bilateral technical clarifications and joint external representation. They mentioned a number of areas where DEE could boost collaboration
with other departments. These included the organisation of conferences, seminars and training
courses and participation in joint research projects.
5.2. Domestic institutions
DEE has relationships with various domestic institutions, with a long-term close relationship with
INE and the Ministry of Finance. In 2012 it established an extended cooperation with the newly
created Portuguese Public Finance Council.
Collaboration between DEE and INE is frequent, namely as regards participation in technical
working groups, such as the Working Group on Labour Market Statistics and the Working Group
for the Development of Macroeconomic Statistics, operating in the framework of the High Council for Statistics. There is also a very intense formal and informal relationship with the National
Accounts Department for monitoring public finances and macroeconomic developments, focusing in particular on statistical data compiled by INE.
At the Ministry of Finance, DEE maintains a close relationship with the Office for Economic Policy and International Affairs (GPEARI). This involves some exchange of information and technical
opinions on the analysis and projection of macroeconomic and fiscal developments. DEE also
participates in joint international representation with the Ministry of Finance, notably in the Economic Policy Committee of the EU27 and the OECD.
Interaction between DEE and the Portuguese Public Finance Council includes exchanges of views
and information on public finance developments in Portugal, and support to the Portuguese
Public Finance Council staff training.
DEE also liaises on a non-regular basis with the Ministry of Economy to exchange data needed for
research projects or collaborate in joint projects. DEE also interacts with universities, participating in conferences, external seminars and courses organised by DEE and receiving students as
interns in the Department.
27. Note that in the Economic Policy Committee of the EU, the main representation belongs to the Ministry of Finance.
33
34
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Staff from a number of Portuguese institutions answered the surveys, mentioning a close collaboration at various levels, namely participation in joint working groups, analysis of economic
developments, collaboration in external representation and bilateral clarifications. It was mentioned that there could be an exchange of human resources and training courses with DEE, as
well as greater participation in joint research projects, as ways to step-up co-operative relations
with these institutions.
5.3. International institutions
In terms of international institutions, DEE collaborates with the IMF, the European Commission
(from here on “EC”) and the OECD.
DEE’s collaboration with the IMF occurs, under normal circumstances, pursuant to Article IV28 and
within the scope of the Financial System Assessment Programme (or FSAP).29 In this context, DEE
releases information and provides the relevant clarifications to IMF staff. With the worsening of
the economic and financial crisis and afterwards under the Economic and Financial Assistance
Programme, IMF’s missions to Portugal became more frequent and involved additional effort
from DEE. Also, DEE comments on the texts and interventions of the permanent Portuguese
spokesperson in the IMF. In addition, it comments on some documents produced by IMF staff
(notably staff reports).
As part of its relationship with the EC, DEE participates in working groups related namely to
national accounts and short-term forecasting. In addition, DEE participates on a sporadic basis
in a number of seminars organised by the EC and comments on texts produced by EC services.
DEE collaborates with the OECD within the scope of the Portuguese representation in Portugal
country reviews,30 Working Group No. 131 and the Economic Policy Committee.
DEE also interacts frequently with IMF, EC and OECD staff monitoring the Portuguese economy.
28. Under Article IV of its Articles of Agreement, the IMF maintains bilateral discussions with members, usually on an annual basis.
29. FSAP is conducted on a five-year or six-year basis.
30. The OECD produces a biannual report on Portugal, with a prior discussion with the Portuguese authorities on the topics under review..
31. Working Party No. 1 of the OECD's Economic Policy Committee meets twice a year to discuss structural and macroeconomic analysis research undertaken by the OECD's economists. This working group also analyses the recommendations issued to countries in order to identify best practice in economic
policy, namely in the framework of the report Going for Growth .
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
PART 2. Special topics
6. The Portuguese economy
A crucial function of DEE is to increase the knowledge regarding the Portuguese economy. Doing
so requires not just the academic type of studies on the economy, which we discussed earlier
in section 3, but also reporting recent developments in the economy, forecasting its near future,
and advising the Governor on structural issues that will pose policy challenges for Portugal within
the euro area. These functions are partly accomplished through three separate outputs: the
Monthly Economic Indicators (Indicadores de Conjuntura), the frequent macroeconomic forecasts
and the studies on topics of the Portuguese economy made in the Department.
6.1. Economic indicators
Any analysis of the Portuguese economy presupposes the existence of good, reliable, and frequent data on macroeconomic aggregates. The DEE has played an important role in building
quarterly series for the Portuguese economy, which are invaluable for researchers in this area.
The DEE has also provided long monetary and financial series that span many decades, and which
are likewise of great value. Moreover, by assembling every month the Monthly Economic Indicators, the DEE makes important data easily accessible to the other economists at the Bank and
to outside researchers and members of the informed public. These are one of the most often
downloaded Bank publications online, with almost 3,000 downloads per report.
Another important output is the coincident indicators for output and consumption. These are
well cited by the press, and carefully monitored by most observers of the Portuguese economy.
Producing them on a regular basis must continue to be a function of DEE. The data is already
released in a fairly consumer-friendly way, but efforts to improve their accessibility must be continuous.
A more important priority is to continue research and deepen the knowledge about the best
coincident indicators are and how to build new indicators. There has been much progress in
this area of economics and statistics in the past decade. The current indicators used by DEE are
not state of the art, but they are sufficiently close to allow a significant improvement with some
effort and resources. Some specific examples are the new approaches to nowcasting, methods
to detect turning points, and new efficient filters to use on large datasets, which the DEE has not
yet adopted. Given the success of the coincident indicators in Portugal, and the clear demand for
them from the private sector, there is some urgency in investing some research time to update
and innovate the methods in use.
The time for this research can be obtained by eliminating the text that accompanies the Monthly
Economic Indicators since most of it does little more than repeat what is in the tables, without any
appreciable analysis. Since the users of the publication emphasize that they are mostly interested
in the coincident indicators, a one-page press release that discusses their evolution, and which
the DEE already produces, seems enough. Staff in DEE reported that preparing the indicators
absorbs a significant amount of time, and simply publishing the numbers while discontinuing the
analysis around them would save valuable resources. These would be better directed to research
that generates new estimators.
35
36
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
6.2. Forecasting
The DEE produces the main forecasts of the Portuguese economy both for the short run and for
the medium term. These are published regularly in the Quarterly Economic Bulletin. The April issue
summarizes the developments of the past year, and its implications for the current year, while the
June Quarterly Economic Bulletin makes the projections for the following two years, which are then
updated in the December issue. The forecasts are especially detailed for inflation, as is normal at
central banks.
Forecasts are an essential output of a central bank. The DEE provides essential inputs to the euro
wide system forecasts, which are required both to guide monetary policy choices, given the lags
between decisions and their effects on the economy, and to ensure that the ECB as a central
bank is transparent about its choices and accountable to the public it must serve. Forecasts also
provide a public good to private agents, reducing uncertainty in the economy. Even at the Banco
de Portugal, for the staff in our survey, learning about the economic forecasts was the most often
mentioned reason for reading the institution’s publications. The forecasts from DEE fall into two
headings: short-term forecasting and business cycle forecasting. DSGE models are currently used
for long-term exercises and simulation analysis.
Short-term forecasting is done with bridge-equation modeling, where several macroeconomic series
are related through reduced-form relations in a way that uses past data in order to maximize shortterm forecasting success. The models used by the DEE perform satisfactorily at their main task.
At the same time, these forecasts can be improved in terms of the methods used. They do not
efficiently use all of the information that is available at any given date, and not enough work
seems to go into forecast evaluation. DEE has not adopted dynamic factor models, which have
become commonplace in many central banks, and does not do much in reporting distributions
with the forecasts, together with a more complete analysis of risks. Moreover, there is no clear
iteration between inserting judgment, updating models, and checking consistency of the different
variables being forecasted against basic economic models.
The DEE makes business-cycle projections using a macroeconometric structural model. They are
a major output of the Department’s work and take a prominent role in economic debate in Portugal. They are perceived as perhaps the most reliable and credible estimates of the evolution of
the Portuguese economy. The model’s performance at forecasting since 2008 was recently evaluated in the April 2013 issue of the Quarterly Economic Bulletin, and compared with the forecasts
from the EC, the IMF and the OECD. The forecasts had relatively large errors, as is true for most
models during the 2008-09 crisis. The Bank’s forecasts perform better than those in the comparison list, although not by a large margin.
Here, again, the methods used at DEE could be improved. Over the past 15 years, forecasting at
many central banks has moved from the classical macroeconometric models into models that
include elements that are more common to DSGE modeling. The research frontier, as represented perhaps by the work done at the Federal Reserve, uses an interaction between judgment
and models that ensure that projections across series and over time are internally consistent.
Moreover, the models are crucial to consider alternative scenarios in a systematized way.
The staff at the DEE has built one DSGE model, labeled PESSOA. This is a fine small open-economy
new Keynesian model, which closely builds on the Global Integrated Monetary and Fiscal model
(GIMF) developed at the IMF by Michael Kumhof and Douglas Laxton. It has been sensibly adapted and applied to Portugal, and this required considerable effort to set up that the DEE deserves
much credit for. One recent useful application of the model was to calculate fiscal multipliers for
Portugal, when there was great debate over the topic both domestically and internationally.
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
At the same time, while PESSOA was a good first step, it is not today at the frontier in the literature. A comparison between PESSOA and the DSGEs used at other small open economy
(for instance the Reserve Bank of New Zealand or the Bank of Canada) or in Europe (the ones at
the Banco de España or the ECB) makes this evident. To start, PESSOA is calibrated, not estimated
to fit the Portuguese economy. Moreover, PESSOA does not have an operative financial sector,
or an account of the role of collateral constraints or other frictions when lending from abroad, so
it has difficulty accounting for the sudden stops of capital that pose some of the greatest policy
challenges for the Bank. Like most DSGE 6 or 7 years ago, but less so today, PESSOA omits banks
and the determinants of credit and interest rate spreads. A case in point comes from the application of PESSOA to evaluate fiscal multipliers. The model lacks a feedback between developments
in public finance and the interest rate that the country faces from abroad. It also assumes that
labour income taxes are the main adjustment tool to pay for debt, when in the data changes in
VAT have been important.
As important as updating PESSOA is to develop alternative models within the Bank. Combination
of forecasts from different perspectives is often an effective way to generate superior forecasts,
and there are techniques to do so in a disciplined way. At the frontier of central banks’ practices,
diversity of models is the key.
One fruitful starting point might be to draw on research done at other parts of DEE. Not having
more of an interaction between the economists that are developing models, and the forecasters providing forecasts, is a negative for both. To give one example, the Bank has a comparative
strength in labour economics. Detailed models of flows in the labour market could, and should,
be used to produce or discipline forecasts about the labour market. The forecasts for unemployment produced by DEE seem rudimentary when compared with the models the same DEE
uses to study the evolution of unemployment in its more academic publications. Likewise, the
DEE already produces an appreciable amount of research on the financial sector, and this will
likely increase. Incorporating that knowledge into the forecasts should improve their quality while
effectively exploiting synergies.
In sum, the DEE has all the ingredients to produce good forecasts that mix current data analysis
with nowcasting, that use many models, and that combine judgment with the internal consistency
of a model. For each of these ingredients, current work is not quite at the research frontier, and
it does not seem to combine the different parts of current analysis and forecasting in an optimal
way. At the same time, one can see through the working papers produced by the economists at
the Bank that they are aware of these issues, and some have even produced good research on
them. Within the regular publications of the Banco de Portugal, there are good articles developing dynamic factor models for forecasting GDP in Portugal, integrating some financial factors into
PESSOA, and working to develop some other DSGE models in the context of the Eurosystem and
its new EAGLE model. Some of the members of the staff are aware of the limitations of its current forecasting models and they have the ability to work in the right directions to improve them.
The missing step is to devote more resources and be more ambitious in applying some of this
knowledge in a systematic way in order to produce better regular forecasts. Therefore, with more
guidance and effort, the Commission is optimistic that significant improvements to forecasting
could be achieved.
6.3. Structural studies
DEE should provide studies on structural issues of the Portuguese economy that alert the Bank to
emerging challenges, and inform its policy choices in this regard. This is done through many out-
37
38
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
puts, of which at the core are the working papers, but which involve occasional books and regular
conferences sponsored by the Bank. The Commission chose to assess this output by asking how
had the research at the Bank informed three key questions over the past decade.
The first question, that dates to the start of the euro and which proved crucial during the recent
crisis, is whether current account imbalances are important in a monetary union. As Portugal
accumulated large current account deficits in the first years of the euro, this was perhaps one of
the most important questions facing the Banco de Portugal between 2000 and 2007. The Governor gave speeches on this topic, and the Bank organized a conference where Olivier Blanchard
delivered a now celebrated paper arguing that the large Portuguese current account deficits
were a cause for concern. Yet, we see little in the Quarterly Economic Bulletin or in the working
papers showing that there was significant research output from the DEE on this topic. The staff
reports regular discussions, but there is little in terms of research output.
A second question that became particularly relevant between 2007 and 2014 was the role of
financial developments in the economy, and especially their interaction with fiscal problems.
Here, the DEEs output is satisfactory. In 2007, in the context of the IMF Financial Assessment
Program, the Banco de Portugal started using bank loan microdata to study fluctuations in credit
and loan defaults. Even before that, there were already some theoretical studies on the interaction between monetary policy and financial factors. After the crisis of 2008, there are several high
quality research pieces on how financial factors affect the financing of firms, or how being in a
currency union may constrain monetary policy but not fiscal policy. The DEE has also systematically produced good research on the state of public finances in Portugal, and on the expected
evolution of the Social Security system. Finally, the bi-annual conference in monetary economics,
together with the research conference of Banco de Portugal have had in their programmes many
presentations of very high quality on these topics.
A third structural question is to what extent can we use micro data to answer macroeconomic
questions on the evolution of the labour market in Portugal, or the behaviour of inflation. Here,
the performance of the Bank is extremely good. Both in explaining job flows, and in understanding how prices are set by Portuguese firms, economists at the Bank have been ahead of most central banks in the use of detailed micro price data to inform important policy questions. This has
resulted in clear useful insights for policy, from understanding how the expansion of the social
safety net in Portugal over the last 25 years may have affected unemployment, to accounting for
earning gaps across genders and sectors.
7. Create a new Financial Intermediation Division in the DEE
Traditionally, central banks are associated with the conduct of monetary policy. The most recent
financial crisis showed that central banks are also valuable for their role in maintaining financial
stability. The financial crisis highlighted the importance of micro- and macro-prudential supervision for central banking. It also demonstrated the value of having advanced financial-stability
analytical capabilities.
The establishment of the Banking Union made it even more important for central banks in the EU
to boost their financial-stability capabilities. The Banking Union will shape financial intermediation
in the EU, and will drive European financial integration going forward. Additionally, by setting up
the Single Supervisory Mechanism and by placing the Governing Council of the ECB at the center
of the policy-making process, the Banking Union will transform the architecture of the institutions
responsible for financial stability in the EU and the euro area.
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
As a result, the role of national central banks in the various areas of financial stability will approach
current practice in the area of monetary policy. This will make it critical for central banks, in particular
those of smaller countries, to complement their micro- and macro-prudential capabilities with strong
analytical capabilities in financial intermediation. Excellence in financial analysis and research will add
to central banks’ reputation, and provide them with the necessary knowledge and strategies to preserve influence in the design of financial stability policies. It will help central banks with their microand macro-prudential responsibilities. Lastly, given the growing evidence on the importance of the
financial sector for the economy at large, it will help central banks in their broader responsibilities.
7.1. DEE’s past Financial Intermediation related activities
Over the years, the DEE has made an outstanding contribution to the Banco de Portugal, in the
area of financial stability. This is apparent in the volume and importance of the work it carried out
as documented below. It is also apparent in the pioneering role the Department has played in a
number of initiatives (e.g. launch of the Financial Stability Report), and in its ability to address new
problems and challenges, as evidenced by its contributions to the adjustment strategy of the Portuguese banking sector that followed the sovereign debt crisis. The main financial related activities
of DEE in recent years included:
• Brief the Governor, and other senior members, including the member of the Advisory Technical
Committee (ATC) (the Head of the Department or the Head of the Financial Stability Division as
an alternate), and the Head of the DEE, on various topics related to financial stability.
• Contribute on finance-related topics to the Banco de Portugal’s publications, including the
Financial Stability Report, Annual Report, Quarterly Economic Bulletin¸ Monthly Economic Indicators,
and Bank Lending Survey.
• Participate in initiatives of the European Systemic Risk Board and European Banking Authority,
including meetings of the Analysis Working Group and meetings of the Instruments Working
Group, and in meetings and activities related to the stress testing exercises of the European
Banking Authority.
• Participate in Eurosystem working groups and task forces, including the Macro-prudential
Research Network, the Household Finance and Consumption Network, the Bank Lending Survey, and ad-hoc task-forces put together for the preparation of the Structural Issues Report.
• Organize the Banco de Portugal biannual conference on Financial Intermediation, participate
in academic conferences, and produce academic papers to be published in refereed journals.
• Other activities: Implement and monitor the quarterly Funding and Capital Plans of the eight
major Portuguese banking groups required under the Economic and Financial Assistance Programme; implement and draft of the reports on quarterly Stress Tests of the eight major Portuguese banking groups required under the Economic and Financial Assistance Programme;
implement the national Household Finance and Consumption Survey of the Eurosystem.
With the establishment of the Financial Stability Department32 at the Banco de Portugal, and
with the ongoing transformation in the EU on the institutional allocation of the responsibilities
for maintaining financial stability and for designing financial policy, it is timely for the Banco de
Portugal to reconsider its financial-analytical capabilities.
32. In 2013, the Banco de Portugal decided to create the Financial Stability Department to help the Bank promote and safeguard financial system stability.
The new department was entrusted with the responsibility of coordinating the Bank’s financial stability related activities and policies.
39
40
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
7.2. The case for a Financial Intermediation Division within the DEE
In the past, most of the financial-related contributions of the DEE came from its Financial Stability
Division. With the establishment of the Financial Stability Department, some of the DEE’s financialrelated responsibilities were transferred to this new department. Notwithstanding this transference of responsibilities, given the increased importance of research and analytical capabilities
on financial-related issues highlighted above, the Commission recommends that the Banco de
Portugal creates within the DEE a new division – the Financial Intermediation Division – with the
set of responsibilities described below. In our view, the establishment of this division is critical for
the Banco de Portugal for the following reasons:
• As we discussed above, ongoing changes in the euro area will require a more active participation by the Banco de Portugal in euro area policy debates on topics related to financial intermediation and financial stability. An effective participation in these debates will require a great deal
of analytical and research support on issues that are increasingly complex. Importantly, it will
also require the development of strategies specifically designed to help the Banco de Portugal
advance its goals. These functions, by virtue of their technical demands and complexity and
because of their dependence on well-designed strategies, are best performed by economists
with expertise and research agendas on financial intermediation and financial stability.
• The dynamics of the macroeconomic adjustment in the euro area have been, and will likely to
continue to be, dominated by financial factors. Similarly, the challenges Portugal will continue
to face for many years, as a result of elevated levels of government, corporate, and household
debt, place financial factors at the center of the macroeconomic adjustment for the country.
Additionally, financial intermediation in Portugal will continue to be a very important determinant of monetary conditions and of the monetary transmission mechanism. Placing the economists with financial expertise in the DEE will allow for synergies between analysis and monitoring of financial intermediation and macroeconomic analysis and forecasting, and consequently
address the aforementioned challenges more effectively.
• Placing the economists with expertise and research agendas on financial-related matters in
the DEE will also facilitate the hiring of financial economists, as many economists from the best
schools, particularly in the rookie market, are more willing to join research departments where
they can continue to undertake research.
• Financial research and the resulting publications will provide the Banco de Portugal with a reputation of an institution capable of producing top-notch research in the areas of financial intermediation and financial stability. This will help the Banco de Portugal advance its agenda on
financial-stability related objectives both domestically and in the euro area. It will also help the
DEE provide high-quality technical support to other departments of the Bank on issues related
to financial intermediation and financial stability. The benefits of synergies with other economists and peer-pressure suggest that placing financial economists in the DEE will help with their
research and have positive externalities for the other economists in the DEE.
• Placing financial economists in a new division of the DEE will increase the likelihood of success
of these objectives. This will raise the prominence of financial economists within the DEE, putting them at par with the other divisions of the Department, with positive effects on their working incentives. It will increase the chances of financial economists exploiting the synergies that
derive from their similar interests and expertise, with positive effects on their performance and
contribution to the Bank. Lastly, it will facilitate the management of financial economists and will
improve their carrier prospects, which will also have a positive effect on their performance and
commitment to the Banco de Portugal.
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
7.3. Proposed Financial Intermediation Division’s responsibilities
The Commission recommends that the new Financial Intermediation Division to be established
within the DEE be staffed with the necessary resources (in quantity, quality, and with the appropriate background skills) to meet the following list of responsibilities related to financial intermediation and financial stability.
(i) Provide analytical support to the Governor and Senior Management
The move of the ECB Governing Council to the center of policy-making on financial stability is a major
change. The same applies to the establishment of the Single Supervisory Mechanism. These important transformations pose both new demands and challenges for the Banco de Portugal, in general,
and for the Governor, as member of the Governing Council. An effective participation in these new
arrangements will require not only a great deal of technical support, but also the development
of strategies specifically designed to give the Banco de Portugal the opportunity to be active and
influential player in the decision making process that will be established to deal with financial matters. The Commission recommends that the Banco de Portugal establishes processes that include
the joint participation of the relevant departments of the Bank, but be under the leadership of the
Financial Intermediation Division, to develop the aforementioned technical support and strategic
thinking and to make both of these available to the Governor and Senior Management.
(ii) Contribute actively to the Financial Stability Report
The development of the Financial Stability Report is a responsibility of the Financial Stability
Department. However, given that the analytical and research skills on matters pertaining to financial intermediation and financial stability will continue to reside in the DEE, the Commission is of
the view that the DEE, and in particular the Financial Intermediation Division, should participate
actively in the production of the report. To that end, the Commission proposes that a formal process for the production of the report be put in place, and that this process renders the DEE the
opportunity not only to contribute with content but also to comment on the entirety of the report.
Further, the Commission suggests that the aforementioned process includes the setting up of an
editorial committee chaired by the Financial Stability Department but with representatives from
DEE (and other departments). The editorial committee would oversee the production of the FSR.
(iii)Provide technical support to other departments of the Banco de Portugal
The Financial Intermediation Division should be open to offer technical support to other departments of the Banco de Portugal on issues related to financial intermediation and financial stability. In general, this support should be provided on demand and subject to approval by the Financial Intermediation Division / DEE Management team. However, in the cases of micro supervision
of financial institutions and macroprudential supervision, given the complexity of these functions
and the increased demands and scrutiny of the single supervisory mechanism, the Commission recommends that formal processes be put in place promoting an active participation of the
Financial Intermediation Division economists in these activities. These partnerships will provide
an opportunity for the Banco de Portugal to lever up the analytical capabilities that will reside in
the Financial Intermediation Division.
(iv)Undertake policy analysis on topics deemed important by the Banco de Portugal
There is growing interest in understanding the implications of the ongoing changes both in the
institutional allocation of the responsibilities for financial stability, and in the architecture of finan-
41
42
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
cial intermediation in Europe. Similarly, there is a growing interest in understanding the challenges that high levels of indebtedness of the public and private sectors will pose to policymakers in Europe. These are just some examples of important questions in need of analysis. The
Banco de Portugal, and in particular the Financial Intermediation Division that the Commission
proposes be introduced in DEE, is uniquely positioned to make a valuable contribution to these
questions. The Bank has an incentive to investigate these issues as they are closely linked to the
Bank’s mission. It has the independence and credibility that are critical for participating in these
debates. It has access to sources of information necessary to undertake these analyses, and with
the establishment of the Financial Intermediation Division, it will be equipped with the intellectual capabilities needed to address these and other financially-related policy questions the Bank
deems important.
The Commission is of the view that the Banco de Portugal could make a valuable contribution
to these debates by producing a book on financial structure, financial integration and financial
stability. This project would also afford and opportunity to promote horizontal cooperation within
the Banco de Portugal as it would build on contributions from the various departments of the
Bank. This cooperative work could be led by the Financial Stability Department. Additionally, it
would provide an opportunity for the Banco de Portugal to identify the key areas it should develop strategies in order to have an active and influential role in the new arrangements in the EU
and the euro area.
(v) Monitor the financial conditions in Portugal
The Financial Intermediation Division should monitor the financial conditions in Portugal, in particular the availability of credit to the various economic sectors. The Financial Intermediation Division should also cooperate with other divisions in DEE in the analysis of the relevance of financial
conditions for economic developments in the Portuguese economy. Macro-financial modeling is
one of the fields where synergies are likely to be strongest.
(vi)Conduct research
The Financial Intermediation Division should produce research on topics related to financial
intermediation and financial stability with the aim at publishing in academic journals, particularly
in the top journals. This activity is very valuable, and therefore should be encouraged. It enhances
the reputation of the Banco de Portugal, adding credibility to its views, and strengthening its
bargaining power both at home and abroad. It puts pressure on the Financial Intermediation
Division economists to maintain both their skill sets and knowledge not only up to date, but also
at the highest levels of the profession. This will render economists the ability to provide better
advice and technical support to the Governor, Senior Management, and other departments of
the Bank. Lastly, the academic research will help DEE in the job market, as economists from the
best schools, particularly in the rookie market, do not view favorably institutions that do not value
research.
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
PART 3. Organisation and Human Resources
8. Organisation
The activities and functioning of DEE are closely linked to the mission of Banco de Portugal.
In organising its activities, DEE therefore pursues a number of objectives established in the threeyear strategic plan for Banco de Portugal.
DEE is particularly involved in two objectives established in the strategic plan for 2014/2016:
(1) to actively participate in the Eurosystem and other fora where Banco de Portugal participates
and effectively implement monetary policy decisions and (2) to contribute to a society better
informed about the country’s economic and financial reality and the activities of Banco de Portugal.
As regards the first strategic objective, DEE mainly focuses on strengthening the analytical quality and relevance of economic research. In addition, DEE is expected to contribute both to the
improvement of the support provided to the Governor and representatives of Banco de Portugal
when participating in Eurosystem meetings and other fora involving the Bank and the coordination of these involvements. As regards the second strategic objective, DEE is expected to participate in rationalising the Bank’s external communication and promoting the institution’s openness
to the outside world.
In terms of operations, a work plan (Plano de Atividades) has been established for DEE based on
the Department’s mission. This mission has three main objectives: (i) analysis of the domestic and
international economic situation and prospects, (ii) policy advice and (iii) research.
This work plan lists the Department staff’s main activities and projects throughout the year, covering (i) the Department’s regular activities, establishing, whenever possible, a timetable and the
person responsible for the task and (ii) a set of research activities that are either to be carried out
in the future, currently under development or being finalised, with the aim of setting deadlines
and publication goals.
In terms of its current organisational structure, DEE is made up of (i) two Heads of Department, (ii)
three research and economic analysis divisions, the Economic Environment and Forecast Division
(Área de Conjuntura e Previsão), the Public Finance and Structural Studies Division (Área de Finanças
Públicas e Estudos Estruturais) and the Monetary Policy Division (Área de Política Monetária), (iii) a
Data Management and Documentation Unit and (iv) a Support Office.
These three divisions of research and economic analysis carry out a number of operational and
regular activities and follow a research programme. As regards operational activities, the production of texts for regular DEE publications and the external representation of Banco de Portugal
within the Eurosystem and in Portuguese institutions and / or other international institutions are
common to these three divisions.
The mission of the Economic Environment and Forecast Division is to monitor, analyse and project macroeconomic developments in Portugal. In addition to the aforementioned tasks, the Economic Environment and Forecast Division is thus responsible for (i) preparing macroeconomic
estimates and forecasts, in cooperation with the Public Finance and Structural Studies Division,
(ii) developing macroeconomic models and conducting simulations, (iii) systemising and analysing data on the Portuguese economic environment, including developing coincident and leading
indicators and (iv) maintaining quarterly long time series for the Portuguese economy.
43
44
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
The task of the Public Finance and Structural Studies Division is to prepare research, namely
on the efficiency of public expenditure, external trade and the labour market. This division is
also responsible for monitoring fiscal outcomes both in Portugal and in European Union and for
developing analytical instruments to simulate and forecast fiscal policy developments, in cooperation with the Economic Environment and Forecast Division. In addition to these tasks, the Public
Finance and Structural Studies Division participates in regular Eurosystem forecast exercises and
internal DEE projections on public finance.
The main task of the Monetary Policy Division is to provide advice on euro area monetary policy
issues. In particular, this division pays special attention to price behaviour, the transmission of monetary policy decisions to the economy and, more recently, the implications of monetary policy for the
Portuguese economy. In addition, one of the main tasks of the Monetary Policy Division is to prepare
the monthly monetary policy briefing. In the past few years, the Monetary Policy Division has also
been assigned tasks related to monetary and financial conditions in the euro area and in Portugal.
The Support Office and the Data Management and Documentation Unit provide support to the
activities of DEE.
9. Demography
In the last 10 years the number of staff working in DEE was, on average, 77. The corresponding figure for economists was 51. In 2013 the number of economists decreased from 59 to 54
mainly due to some staff relocation to the Financial Stability Department (Table 9.133). In terms of
academic degrees, the large majority of economists working in the Department holds a master
degree. The share of economists holding a PhD increased substantially from 20 % in 2000 to 33 %
in 2013. In the same period there was a substantial reduction in the number of administrative
staff from 20 elements to only 8 in 2013.
50 % of the staff are less than 41 years old. The age mode is in the range 41-45 denoting a rather
mature Department. About 57 % of the staff has been with the DEE for more than 10 years, and
33 % work in DEE for at least 15 years. However, as of 2013, 30 % of the staff had been with the
Department for less than 5 years.
10. Recruitment process
Traditionally, DEE has recruited its economists through three different channels:
• Internships programmes mainly addressed to young graduates from top Portuguese universities.
• Open tenders for economists and open tenders for economists holding a PhD.
• Internal transfers of economists from other departments of Banco de Portugal.
The most relevant recruitment channels are internships and open tenders for economists holding a PhD (these account for 66 % of the recruitments in the period 2000-2013).
The internship programme is essentially a six months remunerated position and is a very important source of selection and recruitment of high potential economists. After six months of work-
33. See Charts and Tables.
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
ing experience in the Department, the top performers receive a job offer to join the Department
staff with a tenure contract. Until 2009 the recruitment of trainees was done among top graduates from undergraduate courses. Currently, due to the Bologna process, this recruitment is
done at the master level. In the period 2000-2013, 37 % of the DEE’s staff was hired through this
internship process.
In 2006, DEE introduced some changes in its process to recruit economists with a PhD. Following
international best practices, DEE started to recruit young doctorates in the international PhD job
market. Until 2006 the recruitment of economists with a PhD was done through an open tender
call that was essentially addressed to the domestic market and Portuguese nationals. With the
new recruitment policy the Department gained the ability to attract international researchers
with a PhD degree from top worldwide universities. The switch in the recruitment policy was a
relevant milestone in the Department, contributed to its internationalization, increased diversity
among its staff, and reinforced the group of researchers capable of conducting academic-style
research. Among economists that are currently in the Department, 12 % have been recruited in
the job market.
The recruitment process in the job market follows standard practice: the DEE publicizes, internationally and domestically, the call for applications to the available young PhD positions. In general
three economists from DEE (one or two members of the Management team of the Department
and one or two senior economists) participate in the US job market and interview and select a
short list of candidates who are invited to present a seminar at DEE. A committee, including senior economists belonging to the same scientific field of candidates and some senior researchers,
selects the candidates that will receive a job offer. The job offer is basically a tenure contract.
Since 2006, 41 candidates were invited to give a seminar at the Banco de Portugal; out of them
9 researchers (7 were non-Portuguese) accepted a job offer.
The other arrangement that the DEE uses to hire economists is through internal transfers. In the
period 2000-2013 only 15 % of the economists in DEE were recruited through this channel.
The recruitment policy seems to be adequate. However there are some aspects in the implementation process where in our view there is room for improvement.
Regarding the internships programme it seems that in the recent years almost all trainees
received a job offer, suggesting that the Department does not exercise the option to select the
best people. One could argue that this is the result of a very demanding ex-ante selection process (following the same procedures of tests and interviews as any regular recruitment) that
ensures high quality trainees. However, it can also occur because the pool of candidates is not
big enough or the ex-post selection criteria are not very demanding. In our opinion the internship
programme is a good practice to attract talented people. Nevertheless the number of trainees
should largely exceed the number of positions available to allow for a further strict selection in
the post-training hiring process.
With regards to the job market recruitment process, DEE should pay more attention to the information that it conveys to candidates. According to the information we gathered from staff interviews, the candidates do not have a very clear idea about the rules of the game. Some aspects
in the contract process are very unambiguous: candidates are offered a tenure contract and
they are informed about the remuneration conditions. They also get the information that after
5 years they have the opportunity of getting a substantial salary increase if they obtain a very
satisfactory appraisal conducted by an external assessment committee. The same can occur after
a subsequent five years period. According to the contract provisions, researchers are expected
to devote 70 % of their time to academic-style research and 30 % to policy oriented issues. However, there is a lack of information and transparency in what regards performance expectations:
45
46
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
researchers should be informed about the output they are expected to deliver during, let us say,
the first five years, before they are appraised by an external committee. They should also receive
clear information about the career path they can aspire to have at the DEE. We recommend the
Department to clarify these issues and to establish very clear and transparent rules that should
be presented to candidates during the job market interviewing process.
Another limitation of the current recruitment policy relates to the tenure contract offered to
researchers. According to the Banco de Portugal’s general recruitment policy (and to the Portuguese labour law), usually after a six month period experience any person gets tenure, (including
the junior staff hired after completing the internship programme). We consider that this policy is
against the best practices since does not provide the right incentives to researchers. Following
the traditional tenure track system in academia, whenever the circumstances and legal framework allow it, we suggest to offer young researchers (recruited in the job market) an initial threeyear contract (the maximum length allowed by law for a fixed term contract), with very clear
definition of objectives in terms of high level publications and policy contributions. After this
three-year period, the researcher should be evaluated by an external committee that comprises
international experts in his scientific field. If the assessment is positive the researcher should get
tenure otherwise the contract will expire. This standard tenure track policy provides the right
incentives for young PhDs to produce high quality research and publish in top journals and, at
the same time, reduces the risk to DEE since there is great uncertainty vis à vis the potential of
young doctorates to conduct high level academic-style research (despite the fact that they might
be graduates from top worldwide universities) or to integrate properly in the institution.
11. Careers
According to the Human Resources Department there is a single career for experts that apply to
the entire staff in Banco de Portugal. The concept of a research career does not exist in formal
terms. The career of each individual working in the institution is managed according to the set of
rules and procedures compiled in the Acordo de Empresa and other internal rules.
The large majority of economists in DEE follow a career path similar to other economists at Banco
de Portugal. This staff is dedicated to analytical policy work and policy supporting work, usually
as part of the Banco de Portugal current activities, like preparing regular reports and briefing
sessions to the Governor or addressing questions raised by other departments. These pieces
of analysis can be published either as working papers or in the Banco de Portugal’s reports as
articles, involving some analytical work to support policy discussions, or they can be occasional
papers based on a non-technical analysis. Many staff members also write academic papers for
peer-reviewed publication, and a few even reach top journals. Note, however, that some senior
economists particularly involved in policy advice and institutional representation actually have
little time to dedicate to academic research or even applied research.
DEE staff expects to have a smooth evolution in their careers, following a path of promotions that
lead to small steps upward in the salary scale. The evolution speed in the career is contingent
on the annual performance evaluation. However, the salary path is quite flat and staff only gets
a significant salary increase if they are appointed to coordinate a specific unit (coordenador de
grupo or coordenador de núcleo). At this level, coordinators do not have much managerial responsibilities; they just coordinate a group to perform a very well defined task. The next step in the
career, coordenador de área, also gets a salary increase but the incremental amount is not as
significant as at the level of coordenador de grupo or coordenador de núcleo. Coordenador de área
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
is, in fact, the key step in terms of assuming a clear management position involving, among other
tasks, definition of objectives and staff appraisal. Regarding the evolution in careers we suggest
a greater differentiation between economists, according to their performance and to the rules
of promotions.
According to our understanding of the current situation, there are no clear quality standards or
uniform review process to support the performance assessment of staff doing policy and analytical work. This is a weakness of the current situation since there is not a systematic evaluation
of the quality of output and there are no mechanisms in place to prevent low-quality papers
from being produced. The only information we could get regarding the quality of analytical policy
work (e.g. Quarterly Economic Bulletin articles), policy supporting work and briefings was gathered
through interviews that involved several people (including former Governors).
Another aspect relates to the formal annual performance evaluation of the economists in DEE.
In spite of the existence of annual plans, a clear definition of medium-term and short‑term objectives for the Department is crucial for setting down key performance indicators for staff and
for the Department as a whole. Most of the analytical work produced by staff is highly decentralized and is to a large extent the result of individual initiatives. Relevance of the research is
not assessed according to formal priorities settings or quality standards. Also, neither quality of
policy oriented output nor relevance of briefing sessions to the Governor are formally assessed.
This may introduce inefficiencies in the staff’s annual appraisal process since the tools that are
required to have a transparent evaluation of the individual’s performance are not in place.
It is important to stress that the definition of research among DEE’s staff is a rather confusing
concept. DEE’s staff does not distinguish between academic-style research and analytical work
and it tends to use the research concept as a broad definition. We consider of upmost relevance
to clarify what constitutes research and policy work and to include this classification in the personal objectives definition. The quality standards established for the assessment of each category
should also be defined and communicated to staff. It should also be made clear that individual
annual performance appraisal will be based on the quality assessment of all categories of work:
analytical policy work, policy supporting work, briefings and research.
Furthermore there is no formal career management process in the DEE. As a result, staff is not
managed according to a career perspective but rather to fulfill the urgent needs of the Department. A drawback of this situation is that in many cases individuals are compelled to accept management positions in order to attain higher salaries. Quite frequently this is an inefficient situation
because the appointed person might perform outstanding technical work but may not have the
adequate management profile. On these occasions, the institution looses a proficient economist
and gets a poor manager.
Frequently senior staff has an excessive administrative work load or is heavily occupied with routine analytical work that could be performed by junior staff. Given DEE’s limited resources it would
be desirable to alleviate senior staff’s workloads by recruiting research assistants to perform those
tasks that are analytically demanding but do not require deep knowledge or broad experience.
Research assistants could be hired, whenever the circumstances and legal framework allow it, for
a two years period, among outstanding master graduates from top universities. This procedure
would provide the Department with a pool of talented young people that could contribute to
improve the quality of output in some areas and, simultaneously, allow senior staff to perform
high level analytical work. We also suggest that research assistants as well as interns have the
opportunity of being recruited by other departments of Banco de Portugal if not chosen by DEE.
Staff working in the DEE must also respond to other departments requests, for example by supplying information on the Portuguese economy, by contributing with analysis and reports on
47
48
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
policy issues. However, there is very limited interaction and collaboration between DEE’s staff and
the staff of other departments; exceptions are often based on personal relationships.
Academic-style research oriented economists form a quite separate group inside the Department and they have, to some extent, a different treatment in terms of advancement in the career.
Currently there are about 15 economists distributed among the different divisions of the Department and essentially dedicated to this type of research. Of these, 9 were recruited through the
job market process and are ready to be submitted to a formal appraisal in order to be promoted.
According to the Human Resources Department in Banco de Portugal, these researchers are not
formally in a research career (because it does not exist, research is considered as a function not a
career) however they benefit from an advancement regime based on a case by case assessment.
So far these economists spend 70 % of their work time producing high level research and some of
them have been increasingly successful by academic standards, publishing in top peer-reviewed
journals. They are expected to generate new knowledge and techniques to expand and deepen
the understanding of policy settings, to support the development of policy strategies and to support the decision-making process. However, research activities are highly decentralized. In fact,
there is no mechanism in place to ensure a coherent linkage between the outcome of research
and the policy issues that are part of Banco de Portugal’s work programme. Research projects are
predominantly established in a bottom-up fashion, with a brief discussion with the Department
Management team and a final endorsement by the Board of Directors.
The remaining 30 % of these economists’ time is expected to be dedicated to policy supporting
work. There has been an increasing effort to involve these researchers in the current activities of
the different divisions where they are integrated whilst maintaining their main focus on research
oriented activities. Traditionally, there has been very limited interaction between this group of
researchers and the other members of DEE. For example, they did not participate in policy meetings and were not debriefed by the Department managers. This was a rather inefficient situation
since researchers and policy oriented people could benefit from a two-way communication: on
one hand researchers can have a relevant contribution to the discussion and preparation of policy
advice; on the other hand the debate of policy issues can raise very interesting questions that
might inspire and motivate new research projects and contribute to improve the policy relevance
of research output. One explanation to this limited interaction is related to a lack of aligned incentives: researchers tend to be more focused on their research since they will be evaluated on
the grounds of their scientific production. To overcome this situation the appraisal of researchers should explicitly include their participation in policy oriented activities. Currently the Department recognizes the importance of organizing internal seminars where researchers participate,
together with the other members of DEE, in the debate of policy issues. In our view, it is important
to clarify the career of researchers in the DEE and to establish very transparent rules for their
career management. A young PhD recruited in the job market must be aware of what the Department expects him to produce in terms of output: in the first five -year period these individuals
should concentrate 70 % of their time on research and 30 % on policy related issues. It is recommended to define an initial period for the PhDs recruited in the job market in which they should be
(almost) exclusively focused on their research, to ensure their thesis papers are published. The first
appraisal by an external committee will occur at the end of a period of a minimum of 5 years. If the
researcher gets a positive evaluation then he / she is promoted with a substantial salary raise and
the Department’s Management team, following the recommendations of the external committee,
may decide to maintain 70 % of his time dedicated to research for the next 5 years. However,
the researcher should be aware that the institution expects that more senior researchers will get
higher involvement in policy oriented work. Therefore, as long as researchers attain advanced
stages in the career they should gradually increase their contribution to policy oriented work.
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
In case where the first appraisal is non-satisfactory, meaning the researcher was not able to reach
the required level of publications, the Management team should change the profile of the researcher and increase the weight of policy-oriented analytical work to, at least, 50 % of his / her time.
Currently, as there is no career path in research, the promotion of researchers is dependent on
the discretionary power of the Head of Department. It is urgent to clarify the current situation
especially for those researchers recruited in the job market. Some of them have been waiting for
a formal appraisal and it is not clear when this will occur. The evaluation, and possible promotion, of the first cohort of researchers is crucial to consolidate the maturity of the system that is
in place since 2006.
12. Training
In DEE there is no formal training programme for its staff or any kind of mentoring programme
for the newcomers.
The Bank offers languages courses, but at the DEE, only non-Portuguese researchers enroll in
Portuguese language courses. There are also computer-skill courses available but the participation of DEE staff is almost zero. This is clearly due to the high level education and specialization
of people in the Department.
There are other institutional training offers like Management and Leadership, Team Work, Communication, Time Management, Presentation Skills, etc but they are not attractive to the highly
qualified staff in the DEE with few exceptions like a course on Ethics, or other specific courses as
Incidents Register, Institutional Integration and the New Orthographic Agreement.
DEE’s staff registered a larger participation in specific internal initiatives focusing on technical
issues (Table 12.134).
With regards to external training, in the period 2011-2013, DEE’s staff participated mainly in
academic conferences, workshops, summer schools and other courses offered in other central
banks, or in Portuguese and international universities (Table 12.2).
In terms of training the DEE’s staff also has the opportunity to enroll in a one-year Post-Graduation in Banking, Financial Regulation and Supervision offered by Nova School of Business and
Economics and sponsored by the Banco de Portugal.
According to the Human Resources Department the average number of training hours (18.2 in
2013) is significantly lower in DEE than in the whole bank (33.3 hours). However the Human
Resources Department does not register attendance at internal research seminars or other specialized seminars that are organized by the Department, which represent a significant number
of hours and have a large participation of DEE’ staff. We recommend the definition of suitable
metrics to compute the number of training hours that must include the attendance at internal
research seminars and other specialized initiatives, which are the appropriate training forums for
economists in DEE. These metrics should be communicated to the Human Resources Department.
We also recommend that staff in the DEE with management responsibilities should be compelled
to take training courses in management and leadership since economists usually do not have
training on these skills.
34. See Charts and Tables.
49
50
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
13. Mobility
According to the Human Resources Department, in the period 2011-2013 17 people were transferred from DEE to other departments. In 2013, due to the internal reorganization, 6 economists
were transferred from the DEE to the Financial Stability Department (Table 13.135). During the
same period only one person, an administrative staff member, was transferred to DEE.
According to the information provided by the Human Resources Department, there were some
intra-department moves in DEE in the 2012-2013 period: 7 economists changed from one division to another (horizontal mobility). In the same period 7 people were appointed to new functions (including the current Head of DEE), and among them 6 moved significantly up in the salary
scale.
In terms of external mobility, the DEE recruited 13 new staff members in the period 2011-2013;
in the same period 5 people returned to the Department after spending some time working in
another public institution or after a nonpaid leave (Table 13.2). Therefore the Department benefited from the entry of 18 economists. At the same time 3 staff members left the DEE due to
retirement and a total of 10 members are on leave (working in other public institutions or nonpaid leave). Overall, the difference between external entries and exits is five members representing a positive balance to the DEE.
In sum, the DEE has a rather stable internal structure and there is not much staff mobility. DEE staff
does not follow the practice of spending time at other departments of the Bank. Short visits of
DEE staff, including researchers, to other departments of Banco de Portugal or to other European
central banks should be encouraged since they can provide very positive learning experiences.
35. See Charts and Tables.
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
PART 4. Recommendations and suggestions
This section sets out the complete list of the Commission’s recommendations and suggestions.
List of recommendations and suggestions
Main recommendation 1: Set up an accountability framework.
Associated recommendations:
1.1. Design the accountability framework to encompass the full programming cycle: planning,
monitoring, evaluation and communication.
1.2. Inform all staff about the planning and programming of activities.
1.3. Evaluate progress of the accountability framework taking into account the goals and performance indicators defined.
1.4. Define key performance indicators (KPIs).
1.5. Execute and evaluate the planning and programming of activities building on Banco de
Portugal’s strategic planning.
1.6. Conduct regular external evaluations of DEE.
Main recommendation 2: Improve human resource management.
Associated recommendations:
2.1. Align human resource management with the accountability framework.
2.2. Set clear goals and performance indicators for the staff based on the accountability framework.
2.3. Assess DEE’s performance against performance indicators defined.
2.4. Clarify DEE’s work in terms of the following categories: (i) policy advice, (ii) briefings, (iii) policy
supporting work, (iv) analytical policy work and (v) research.
2.5. Base annual objectives and individual annual performance appraisals on the work categories
of Recommendation 2.4.
2.6. Define and communicate to DEE the quality standards for the assessment of each work
category.
2.7. Define profiles for the staff according to the work to be done.
2.8. Aim to publish the research produced in DEE in leading-edge scientific journals. (Recommendation shared with Main recommendation 4)
2.9. Adopt a points system for classifying scientific publications and quantifying research work, similar to the ECB’s system (see Box 3). (Recommendation shared with Main recommendation 4)
2.10.Establish a procedure for evaluating the analytical work, policy supporting work and briefings
to the Governor.
2.11.Evaluate the analytical work, policy supporting work and briefings to the Governor, other
members of the Board of Directors and other senior management, based on (i) relevance for
explaining the policy issue being analysed, (ii) appropriateness of the analytical tools, (iii) relevance of the data used, (iv) writing clarity and adequacy of document structure, (v) adequacy of
information to form an opinion and (vi) pertinence of conclusions for policy decisions.
51
52
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
2.12.The evaluation outlined in 2.11 could be conducted through surveys of DEE ‘clients’, including
the Governor, other Banco de Portugal departments, and even external entities that consider
the central bank reports as inputs for their policy decisions.
2.13.Another possibility could be to set up an annual prize for the best article published in the
Quarterly Economic Bulletin and the Financial Stability Report.
2.14.Banco de Portugal could consider a revision of the human resources policy, namely through a
formal career development and management process.
2.15.As part of the integrated analysis of careers launched by Banco de Portugal, staff could follow
an alternative career allowing access to top salary levels without managerial responsibilities:
the advisor career.
2.16.Candidates for the advisor career would be individuals with exceptional talent, for example,
specialists that make notable contributions to policy areas or researchers that maintain an
outstanding publications record.
2.17.The selection of candidates for the advisor career should be carried out by a committee comprising external members using very demanding criteria that should be set transparently.
2.18.Build a strategic recruitment plan for the next five years for young PhDs, with each DEE area
specifying the number of researchers and research profiles needed.
2.19.Select candidates in the job market each year according to that strategic recruitment plan.
2.20.Revise the current process for recruiting in the job market, setting clear and transparent rules
on the career and evaluation system for young PhDs in the job market. These rules must be
communicated to the candidates in the job market interviews.
2.21.Coordination of the job market recruitment process should be the responsibility of the DEE’s
Management team, in coordination with the Human Resources Department.
2.22.The job market recruitment committee should comprise the most research-active economists.
2.23.The job market recruitment committee could also comprise researchers from different areas
and, if appropriate, specialised staff from other Banco de Portugal departments.
2.24.The initial contract term for young PhD researchers recruited in the job market should be
three years, whenever the circumstances and legal framework allow it.
2.25.The contractual process should include a very clear definition of objectives in terms of publications and policy contributions to be carried out by researchers at the start of their careers.
2.26.After the three-year period, the researchers should be evaluated by an external committee
that comprises international experts in their scientific field.
2.27.If the evaluation is positive the researchers should get tenure, otherwise the contract should
expire.
2.28.It is recommended to define an initial period for the PhDs recruited in the job market in which
they should be (almost) exclusively focused on their research, to ensure their thesis papers
are published.
2.29.Create the position of research assistant.
2.30.Hire strongly analytical research assistants, for a two-year period, whenever possible within
the applicable legal framework.
2.31.Consider the possibility of a number of trainees greatly exceeding the number of positions
available to increase DEE’s selection capability.
2.32.Promote internal and external mobility as part of career management.
2.33.Define suitable metrics to compute the number of training hours in the DEE that must include
attendance at internal research seminars and other initiatives, such as scientific conferences
and workshops. These metrics must be communicated to the Human Resources Department.
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
2.34.Promote training courses in management and leadership for DEE staff with management
responsibilities.
2.35.Revise the visitor researchers programme in order to ensure that during their stay they take
part in projects of interest for DEE.
Main recommendation 3: Identify financial intermediation and European financial
integration as priorities for the next three years and create a financial intermediation division.
Associated recommendations:
3.1. Identify financial intermediation and European financial integration as key priorities for the
next three years.
3.2. Create a Financial Intermediation Division in DEE.
3.3. Establish processes that include the joint participation of the relevant departments, under
the leadership of the Financial Intermediation Division, to develop the technical support and
strategic thinking needed for the Governor and Board of Directors.
3.4. Retain in DEE analytical and research skills on matters pertaining to financial intermediation
and financial stability. The Financial Intermediation Division should focus research on this
area, for publication in top journals.
3.5. DEE should contribute to policy analysis on topics deemed important by Banco de Portugal,
with particular regard to (i) the implications of the ongoing changes both in the institutional
allocation of the responsibilities for financial stability, and in the architecture of financial intermediation in Europe and (ii) the challenges that high levels of indebtedness of the public and
private sectors will pose to policymakers in Europe.
3.6. Produce a book on financial structure, financial integration and financial stability in Portugal,
led by the Financial Stability Department, and promoting horizontal cooperation between the
Banco de Portugal departments involved in these areas.
3.7. The Financial Intermediation Division should monitor the financial conditions in Portugal, in
particular the availability of credit to the various economic sectors. This division should also
cooperate with other DEE divisions in analysing the relevance of financial conditions for economic developments in Portugal.
3.8. Provide technical support through the Financial Intermediation Division to other departments
of Banco de Portugal on issues related to financial intermediation and financial stability.
In general, this support should be provided on demand and subject to approval by the
Division’s or DEE’s Management team.
3.9. In the cases of micro supervision of financial institutions and macroprudential supervision,
formal processes could be put in place promoting the active participation of Financial Intermediation Division economists in these activities.
3.10.Set up a cooperation agreement between DEE and the Financial Stability Department, covering three key areas: (i) drafting of texts for Banco de Portugal’s publications, both regular and
occasional, (ii) preparation of policy briefings in the financial stability area and (iii) analytical
activities and / or research.
53
54
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Main recommendation 4: Pursue excellence in research.
Associated recommendations:
4.1. Aim to publish the research produced in DEE in leading-edge scientific journals. (Recommendation shared with Main recommendation 2)
4.2. Separate research work from analytical work more clearly within the tasks of each DEE economist, so that clear and differentiated expectations are set on each type of work for each
individual economist.
4.3. Adopt a points system for classifying scientific publications and quantifying research work, similar to the ECB’s system (see Box 3). (Recommendation shared with Main recommendation 2)
4.4. Identify the priority research areas for Banco de Portugal but in general, comprehensive
terms, giving substantial freedom to researchers regarding themes to explore in these areas.
4.5. Provide regular information to the staff on ongoing DEE activity, particularly through presentations related to research projects, in order to stimulate collaboration and cross fertilisation
between teams.
4.6. Ensure there is time for free research, rewarding those researchers who are the most successful, and taking a long-term approach to allow for the delays in publication in scientific
journals.
On modeling:
4.7. Update, revise and improve short-term forecasting models, incorporating new tools, in particular, dedicate research time to updating and innovating the methods used in the production
of Coincident and Leading Indicators.
4.8. Innovate in the forecasting methods used in the economic cycle projections, partly using
research work already carried out within DEE more actively and efficiently.
4.9. Increase rigour in forecasting tasks, comparing their execution and performance to that of
other central banks, and including forecasting tasks in the research objectives subject to regular appraisal.
4.10.Integrate forecasting more closely with research, allowing synergies between the two and
innovating in the use of models and data.
4.11.Develop a diversified set of forecasting tools.
Main recommendation 5: Rethink communication.
Associated recommendations:
5.1. Identify the Quarterly Economic Bulletin as the flagship publication.
5.2. Entrust one senior economist with the task of editing the Quarterly Economic Bulletin.
5.3. Associate the Quarterly Economic Bulletin’s production process with internal procedures aimed
at early debate of key issues.
5.4. Make the Quarterly Economic Bulletin articles accessible to a wider audience.
5.5. Autonomise electronic access to the following products in Banco de Portugal’s institutional
website: economic projections, signed articles, special issue texts, the Portuguese economy,
while nevertheless retaining the unity of the Quarterly Economic Bulletin as a publication.
5.6. Consider the possibility of issuing economic projections a few days before the release of the
Quarterly Economic Bulletin, with a devoted press release.
5.7. The special issues texts, published in the Quarterly Economic Bulletin, should be signed.
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
5.8. Consider changing the Annual Report to a standard corporate Annual Report format.
5.9. Remove the text accompanying the Montlhy Economic Indicators and limit this publication to
factual tables and charts.
5.10.Issue a devoted press release to accompany the Coincident Indicators.
5.11.Make clear that the Working Papers is a series and establish that they will be, as a rule, submitted for publication in academic-refereed journals.
5.12.Define Working Papers as a metric to assess DEE’s academic output in real time.
5.13.Keep Banco de Portugal’s Working Papers series as a responsibility of DEE but open it to submissions from the staff of other departments.
5.14.Create Staff Technical Notes as a means for releasing analytical work mainly related with the
texts of the Quarterly Economic Bulletin.
5.15.Discontinue the Occasional Papers.
5.16.Promote the research newsletter, Spillovers, in order to raise its profile.
5.17.Rethink the contribution of DEE to external communication, identifying the targeted groups,
their demands and the purposes of communication.
6. Other relevant recommendations.
6.1. Promote diversity of viewpoints and analytical tools in DEE debates.
6.2. Banco de Portugal could consider management processes enabling a matrix format management between departments and various areas in the same department.
6.3. Provide timely and detailed information on the models used in DEE, both to increase Banco
de Portugal’s transparency and to expose these models to healthy external scrutiny from
other researchers.
6.4. Banco de Portugal may consider broadening access to the databases created in the institution.
6.5. Update and expand the research support tools on a permanent basis, including IT support
and software purchasing (for example additional STATA and MATLAB licences).
55
56
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Charts and Tables
Table 1.1 International institutional representation of Banco de Portugal in 2013
Face-to-face meetings
Conference calls
ESCB / Eurosystem
480
275
European Union
358
72
60
1
174
31
39
34
15
3
IMF
2
-
BIS
13
-
OECD
15
1
Other
34
6
917
357
Of which:
European Commission and Eurostat
European Supervision Authorities
ESRB
Portuguese-speaking Countries
Total
Source: Banco de Portugal.
Table 1.2 International institutional representation of the DEE in 2013
Face-to-face meetings
ESCB / Eurosystem
Conference calls
39
2
35
2
4
-
10
-
4
-
53
2
Of which:
Monetary Policy Committee
Competitiveness Research Network
European Union
OECD
Total
Source: Banco de Portugal.
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Chart 3.1 Developments in the total number of articles published in peer-reviewed journals
and in articles weighted by ECB ranking
140
70
0
2000
2001
2002
2003
2004
2005
Number of Articles
2006
2007
2008
2009
2010
2011
2012
2013
Number of articles weighted by ECB ranking
Source: Banco de Portugal.
Chart 3.2 JEL CODES of the articles published in peer-reviewed journals (number of articles)
Macroeconomics and Monetary Economics
Mathematical and Quantitative Methods
Labour and Demographic Economics
International Economics
Financial Economics
Microeconomics
Industrial Organization
Economic Development, Technological Change and Growth
Public Economics
Business Administration and Business Economics, Marketing, Accounting
Agricultural and Natural Resource Economics,
Environmental and Ecological Economics
Regional, Real Estate and Transportation Economics
Health, Education andWelfare
Economic History
0
Source: Banco de Portugal.
45
90
135
180
57
58
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Chart 3.3 Concentration of the number of articles (by economist)
number of articles % of total*
100%
50%
0%
0
10
20
30
40
50
60
70
Number of economists
* Total includes the articles published in the 2000-2014 period.
Source: Banco de Portugal
Chart 3.4 Working Papers published in the Banco de Portugal series
30
15
0
2000
2001
Source: Banco de Portugal.
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Table 3.1 List of publications according to the Point System for Publication of the ECB
TIER 1*
Tier 3 (continuation)
American Economic Review
European Economic Review
Econometrica
European Financial Management
Journal of Political Economy
European Journal of Finance
Quarterly Journal of Economics
Financial Management
Journal of Finance
IMF Economic Review
Review of Economic Studies
Industrial and Labor Relations Review
International Journal of Industrial Relations
TIER 1
Journal of Banking and Finance
Journal of Business and Economic Statistics
Journal of Common Market Studies
Journal of International Economics
Journal of Comparative Economics
Journal of Monetary Economics
Journal of Economic Behaviour and Organisation
Journal of European Economics Association
Journal of Economic Dynamics and Control
Journal of Financial Economics
Journal of Economic Growth
Review of Financial Studies
Journal of Economic History
Review of Economics and Statistics
Journal of Economic Literature
Journal of Empirical Finance
TIER 2
Journal of Financial Markets
American Economic Journal – Macroeconomics
Journal of Financial Services Research
Biometrika
Journal of Financial Stability
Economic Journal
Journal of Forecasting
Frontiers of...: Berkeley electronic journals
Journal of Futures Markets
Games and Economic Behaviour
Journal of Industrial Economics
International Economic Review
Journal of International Money and Finance
International Journal of Central Banking
Journal of Law and Economics
International Journal of Forecasting
Journal of Law Economics and Organisation
Journal of Applied Econometrics
Journal of Macroeconomics
Journal of Development Economics
Journal of Mathematical Economics
Journal of Economic Perspectives
Journal of Policy Modelling
Journal of Economic Theory
Journal of Risk and Uncertainty
Journal of Econometrics
Journal of Time Series Analysis
Journal of Financial Econometrics
Journal of Urban Economics
Journal of Financial Intermediation
Journal of Population Economics
Journal of Financial Quantitative Analysis
Journal of Portfolio Management
Journal of Human Resources
Journal of the Royal Statistical Society (series A)
Journal of Labour Economics
Kyklos
Journal of Money Credit and Banking
Labour Economics
Journal of Public Economics
Land Economics
Journal of the American Statistical Association
Macroeconomic Dynamics
Quantitative Economics
Mathematical Finance
Rand Journal of Economics
NBER Macro Annual
Review of Economic Dynamics
Oxford Bulletin of Economics and Statistics
Oxford Economic Papers
TIER 3
Oxford Review of Economic Policy
Advances …: Berkeley Electronic journals
Public Choice
American Economic Journal – Microeconomics
Review of Income and Wealth
American Economic Journal – Applied Economics
Review of Finance
American Economic Journal – Economic Policy
Scandinavian Journal of Economics
Annals of statistics
Scottish Journal of Political Economy
Tier 3 (continuation)
Studies in Nonlinear Dynamics and Econometrics
Applied Economics
Theoretical Economics
Brookings Papers on Economic Activity
Economica
Cambridge Journal of Economics
Empirical Economics
Canadian Journal of Economics
Weltwirtschaftliches Archiv
Economic Inquiry
Economics Letters
TIER 4
Economic Theory
All other journals in EconLit, including …
Econometric Theory
German Economic Review
Econometric reviews
National Tax Journal
Econometrics Journal
Risk Magazine
Economic Modelling
59
60
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Chart 4.1 Evolution of the total number of news items relating to the DEE and a breakdown
by publication
1600
2.000
800
1.000
0
0
2007
2008
Total (right-hand scale)
2009
2010
2011
2012
QEB
MEI
AR
FSR
2013
Other publications
Source: Manchete.
Chart 4.2 Evolution of the total number of news items relating to the Quarterly Economic
Bulletin and a breakdown by media type
800
1500
400
750
0
0
2007
2008
Total (right-hand scale)
Source: Manchete.
2009
2010
Internet
2011
Print media
2012
Television
2013
Radio
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Chart 4.3 Breakdown of the total news items relating to the DEE’s publications (for 2013)
192
10%
128
7%
QEB
20
1%
MEI
275
15%
AR
FSR
1274
67%
Source: Manchete.
Chart 4.4 Number of pageviews of the Quarterly Economic Bulletin through Banco de
Portugal’s institutional website (from 2012; data per publication)
10.000
5.000
0
Mar. 12
Jun. 12
English
Source: Banco de Portugal.
Sep. 12
Dec. 12
Portuguese
Mar. 13
61
62
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Chart 4.5 Frequency of consultation of the DEE’s regular publications
100
50
0
Weekly
Monthly
QEB
Quarterly
Anually
FSR
Occasionally
AR
Never
MEI
Source: Survey completed by users of Banco de Portugal’s publications.
Chart 4.6 Breakdown of OTSs per DEE publication
20
16%
QEB
10
8%
56
47%
9
8%
MEI
AR
FSR
Other publications
25
21%
Source: Manchete.
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Chart 4.7 Evolution of the total number of news items relating to the Annual Report and a
breakdown by media type
80
140
40
70
0
0
2007
2008
2009
Total (right-hand scale)
2010
Internet
2011
Print media
2012
2013
Television
Radio
Source: Manchete.
Chart 4.8 Evolution of the total number of news items relating to the Monthly Economic
Indicators and a breakdown by media type
200
360
100
180
0
0
2007
2008
Total (right-hand scale)
Source: Manchete.
2009
2010
Internet
2011
Print media
2012
Television
2013
Radio
63
64
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Chart 4.9 Number of pageviews of the Monthly Economic Indicators through Banco de
Portugal’s institutional website (from 2012; data per publication)
4.000
2.000
0
Jan. 12
May. 12
Sep. 12
Jan. 13
May. 13
English
Sep. 13
Jan. 14
Portuguese
Source: Banco de Portugal.
Note: No data available between 16 February 2012 and 30 March 2012.
Chart 4.10 Number of pageviews of the Coincident Indicator through Banco de Portugal’s
institutional website (from 2012; data per calendar month)
7.000
3.500
0
Jan. 12
May. 12
Sep. 12
Jan. 13
English
Source: Banco de Portugal.
Note: No data available between 16 February 2012 and 30 March 2012.
May. 13
Sep. 13
Portuguese
Jan. 14
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Chart 4.11 Evolution of the total number of news items relating to the Financial Stability Report
and a breakdown by media type
140
250
70
125
0
0
2007
2008
2009
Total (right-hand scale)
2010
Internet
2011
2012
Print media
2013
Television
Radio
Source: Manchete.
Chart 4.12 Number of pageviews of the Financial Stability Report through Banco de Portugal’s
institutional website (from 2012; data per publication)
6.000
3.000
0
May. 12
Nov. 12
English
Source: Banco de Portugal.
May. 13
Portuguese
65
66
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Chart 4.13 Evolution of the total number of news items relating to the DEE and a breakdown
by media type
1200
2250
600
1125
0
0
2007
2008
2009
Total (right-hand scale)
2010
Internet
2011
Print media
2012
2013
Television
Source: Manchete.
Chart 4.14 Each media type’s share of news items relating to the DEE (for 2013)
4%
13%
Internet
Print media
Television
24%
Source: Manchete.
59%
Radio
Radio
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Chart 4.15 Shares of each support type used to consult the DEE’s publications
3%
31%
Paper
Internet
Paper and Internet
66%
Source: Survey completed by users of Banco de Portugal’s publications.
Nota: The publications included in this chart are: QEB, FSR and AR.
Chart 4.16 Number of pageviews of the Book A Economia Portuguesa no Contexto da
Integração Económica, Financeira e Monetária (The Portuguese Economy in the Context of
Economic, Financial and Monetary Integration) through Banco de Portugal’s institutional
website (from 2012; data per calendar month)
3.500
1.750
0
Jan. 12
May. 12
Sep. 12
Jan. 13
English
Source: Banco de Portugal.
Note: No data available between 16 February 2012 and 30 March 2012.
May. 13
Sep. 13
Portuguese
Jan. 14
67
68
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Chart 4.17 Seminars and courses organised by the DEE
70
35
0
2000
2002
2004
2006
External Seminars
2008
2010
Internal Seminars
2012
Courses
Source: Banco de Portugal.
Table 9.1 DEE staff (as of 31 December 2013 of each year)
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013
Total
65
72
74
77
74
71
74
75
79
82
80
80
79
73
3
3
3
3
3
3
3
3
3
3
3
3
3
2
42
51
53
57
56
54
57
58
63
67
66
66
68
63
Economists
35
43
45
47
46
43
46
47
52
55
54
55
56
52
Graduate
6
8
9
8
9
7
8
7
8
9
9
6
6
5
22
26
25
28
26
26
27
28
29
29
29
31
31
30
7
9
11
11
11
10
11
12
15
17
16
18
19
17
7
8
8
10
10
11
11
11
11
12
12
11
12
11
3
4
4
3
3
3
3
3
2
3
4
4
3
2
15
12
12
12
10
9
9
10
10
8
6
6
5
5
2
2
2
2
2
2
2
1
1
1
1
1
0
1
Management team
Staff
Masters and similar
Doctorates
Others
Specific / secretariat tasks
Administrative staff
Support staff
Source: Banco de Portugal.
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Table 12.1 Examples of specific internal initiatives focusing on technical issues
• Annual Meeting of the Banco de Portugal
• Financial Systemic Risk, Basel III, SIFIs
• Workshop on top-down stress tests
• Encontro de Lisboa
• The Dog that didn’t bark
• Desarollo de la economia de Colombia
• Seminar on PPP’s
• Dealing with Household Debt
• Orçamento do Estado para 2014
• Annual Summer Conference – CIRSF
• Iceland’s experience regarding financial crisis
• Conference on Financial Supervision Basel III
• Global Financial Stability Report FMI
• Banking Crises and Remedies
Source: Banco de Portugal.
Table 12.2 DEE’s external training (2011-2013)
Year
Training
Entity
2013
Economic Modelling and Forecasting
Central Bank of Malta
2012
Applied Bayesian econometrics for central bankers
Bank of England
2012
Euro Area Business Cycle Network Training School
Banca d’Italia
2013
Financial Stability and Macroprudential Policy
Bank of Korea
2013
2013 Econometrics Summer School using STATA
CEMAPRE / ISEG
2011
Macroprudential Regulation of Banks
Centro de Estudios Monetarios y Financieros
2013
Panel Data Econometrics
Centro de Estudios Monetarios y Financieros
2011
The Econometric Analysis of Mixed Frequency Data
EABCN – Euro Area Business Cycle Network
2011
Unemployment and Matching
EABCN – Euro Area Business Cycle Network
2012
Euro Area Business Cycle Network Training School
EABCN – Euro Area Business Cycle Network
2012
Forecasting Inflation
EABCN – Euro Area Business Cycle Network
2012
Global VAR modelling using the GVAR Toolbox 1.0
EcoMod Modeling School
2012
Stress Testing 2.0
Event Production s.r.o.
2013
A Course on Panel Data Models
Faculty of Economics, University of Cambridge
2012
Financial Stability, Systemic Risk, and Macro-Prudential Policy
IMF – International Monetary Fund
2012
Financial Stability, Systemic Risk, and Macro-Prudential Policy
IMF – International Monetary Fund
2013
Empirical Banking – Methodological Aspects
Barcelona Graduate School of Economics
2013
Simulation and Estimation of DSGE Models
ISEG – Lisboa School of Economics & Management
2013
PET 13
Católica, Lisbon
2013
PET 13
Católica, Lisbon
2011
Spring Course on Labor Econometrics
University of Coimbra
2012
Spring Course on Nonstationary Panel Time Series Methods
University of Coimbra
2013
Spring Course Bayesian Econometrics
University of Coimbra
2012
The Science and Art of DSGE Modelling
University of Minho
2013
Discrete Choice Models For Cross Section
University of Minho
69
70
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Table 13.1 Internal mobility – DEE (2011-2013)
Year
Entries
Exits
Total
2013
1
9
-8
2012
0
6
-6
2011
0
2
-2
Total
1
17
-16
Source: Banco de Portugal.
Table 13.2 External mobility – DEE (2011-2013)
Entries
Exits
End of
requisitions
2013
6
0
0
4
2
2012
3
3
2
3
1
2011
4
2
1
3
2
Total
13
5
3
10
5
Source: Banco de Portugal.
Retirements / Terminations
Start of
requisitions
Difference
entries / Exits
Recruitments
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Annexes
Annex I. Independent Commission’s Mandate
for Assessing DEE’s Activities
Banco de Portugal has the objective of creating and maintaining within its Economics and
Research Department the analytical and human capability to contribute to its tasks as a member
of the Eurosystem and as the Portuguese Republic’s central bank. To this end, Banco de Portugal
must be able to attract and retain talent of the quality level found in the best research departments of organisations responsible for conducting economic policy.
To assist with DEE’s strategic guidance, the Board of Directors of Banco de Portugal decided to
hold an independent assessment of DEE’s activities, through a Commission specially appointed for
the purpose. The Commission will be chaired by Vítor Gaspar and will have three further members.
The Commission will have DEE’s support in accessing data and documentation needed for the
assessment, as well as the support of a senior economist for the compilation of the information
gathered. The Committee is authorised to interview members of DEE and their counterparts outside the Department. This assessment will cover the following areas:
• The quality, relevance and impact of the policy advice within the framework of the Eurosystem;
• The quality, relevance and impact of the research into the Portuguese economy;
• The quality and quantity of the research published in scientific journals;
• The impact and relevance within the framework of Eurosystem and European System of Central
Banks bodies;
• The quality of the models, analytical tools and analysis of other Banco de Portugal departments’
activity;
• DEE’s contribution to Banco de Portugal’s communication, namely in the areas of information
dissemination and economic analysis;
• DEE’s contribution to Banco de Portugal’s ability to attract, retain and develop professional
talent of the quality level found in the best organisations in the field around the world.
The Commission will produce a set of recommendations based on the analysis. The Commission’s analysis and recommendations will be published after considering the comments of DEE
and Banco de Portugal’s Board of Directors.
The Commission’s final report must be submitted to Banco de Portugal’s Board of Directors by
30 June 2014.
71
72
Evaluation report on the activities of the Economics and Research Department of Banco de Portugal
Annex II. Acronyms and abbreviations
AR or Annual Report
Annual Report – The Portuguese Economy
Commission
Economics and Research Department Evaluation Commission
DEE or Department
Economics and Research Department
EC
European Commission
ECB
European Central Bank
ESCB
European System of Central Banks
EU
European Union
FSR
Financial Stability Report
IMF
International Monetary Fund
INE
Statistics Portugal
MEI
Monthly Economic Indicators
MPC
Monetary Policy Committee
NCB
National Central Banks
OECD
Organisation for Economic Co-operation and Development
OTS
Opportunities-to-see
QEB
Quarterly Economic Bulletin
Statutes
Statutes of the European System of Central Banks and of the European Central Bank
Annex III. Sources of information used
• The Treaty on European Union, the Statutes of the European System of Central Banks and of
the European Central Bank, the Constitution of the Portuguese Republic, the Organic Law of
Banco de Portugal;
• Banco de Portugal’s Company Agreement; Internal Rules and Regulations of Banco de Portugal;
Programming and Strategic Planning Documents of the ECB and the Eurosystem; Programming
and Strategic Planning Documents of Banco de Portugal;
• Information provided by DEE’s Management team and other managers in DEE and Banco de
Portugal;
• Face-to-face interviews with current and previous members of the Board of Directors of Banco
de Portugal, senior management of DEE and other Banco de Portugal’s departments, DEE divisions and unit heads and DEE staff;
• Written questionnaire from key internal and external interested parties;
• Research on the impact of the publication in the media carried out by the company Manchete;
• Survey of scientific publications produced by DEE staff;
• Consultation of scientific publications;
• Consultation of institutional publications;
• Evaluation reports on central banks and other entities involved in conducting policy.