Annual Review of
Strengthening Transparency, Accountability and Responsiveness
(STAR) in Ghana
Narrative Report
Taaka Awori
Richard Johnson
John Rowley
April 2012
Acronyms and Abbreviations
CB
Capacity Building
CEIA
Centre for Environmental Impact Analysis
CIDA
Canadian International Development Agency
CSO
Civil Society Organisation
CSP
Civil Society Platform
CSGF
Civil Society Governance Fund
DPs
Donor Partners
EOI
Expression of Interest
FC
Funders Committee
GESI
Gender and Social Inclusion
GP
Grant Partners
G-RAP
Ghana Research and Advocacy Programme
IT
Information Technology
KITE
Kumasi Institute of Technology and Environment
M&E
Monitoring and Evaluation
POSDEV
Pan-African Organization for Sustainable Development
PMT
Programme Management Team
QA
Quality Assurers
RAVI
Rights and Voice Initiative
RI
Results Initiative
SC
Steering Committee
SP
Service Providers
STAR
Strengthening Transparency Accountability and Responsiveness
TA
Technical Advisors
TOR
Terms of Reference
USAID
United States Agency for International Development
VFM
Value for Money
Table of Contents
1
Background
1
2
Review Objectives
2
3
Review Process
2
4
Progress Against Expected Results
3
5
Monitoring and Evaluation
9
6
Value for Money
10
7
Implementation Strategies
13
8
Governance and Management
16
9
Conclusion
19
STAR-Ghana
Annual Review Narrative Report
________________________________________________________
1. Background
STAR-Ghana is a five-year multi-donor pooled funding mechanism aimed at increasing
the influence of civil society and Parliament in the governance of public goods and
service delivery. The ultimate goal is to improve the accountability and responsiveness
of Ghana’s government, traditional authorities and the private sector. STAR-Ghana
builds on previous programmes such as:




Rights and Voice Initiative (RAVI) (2004-2010)
Ghana Research and Advocacy Programme (G-rap) (2005-2011)
KASA (2008-2010); and the
Civil Society Governance Fund (CSGF) (2004-2010).
Ghana has achieved significant reduction in income poverty and progress in
consolidating democracy. However, deficits in accountability institutions persist and the
responsiveness of these institutions to the provision of basic services is low. There is
also evidence of high public demand for accountability but this is insufficiently
aggregated. STAR-Ghana has therefore commenced at a very critical time in the
country’s progress towards middle income status. Oil revenues are likely to influence
foreign aid flows, particularly allocations to budget support. Windfall oil revenues
typically strain political institutions and reduce accountability incentives. Effective
momentum to bring about real pro-poor policy change requires the mobilisation of
sustained constituencies external to government. Following on from the achievements
of the four earlier funding mechanisms, there is need to institutionalise, strengthen and
improve further the role that CSOs and Parliament play in influencing the way
government does business. There is therefore the need to support Ghana’s objectives
to address accountability and responsiveness deficits even more broadly and in more
depth.
The programme impact of STAR is therefore to increase the accountability and
responsiveness of government, traditional authorities, and private enterprise to
Ghanaian citizens. The outcome is to increase the influence of CSOs and Parliament in
the governance of public goods and service delivery.
The Programme is focused on delivering the following Outputs:

Increase capability of CSOs to enable citizens- particularly women and excluded
groups to claim rights.



Enhanced civil society engagement in policy formulation, implementation and
monitoring
Increased use of civil society evidence in policy and practice
Improved representative oversight and lawmaking functions of selected
parliamentary committees
2. Review Objectives
The overall purpose of this first annual review of STAR was to assess and score the
progress that the programme has made towards achieving its purpose. More
specifically, the review was expected to:





assess progress against expected results as stated in the programme log-frame,
considering each indicator and respective Year 1 milestones and including
evidence and implications for the programme.
assess the effectiveness of the programme in capturing and monitoring results
and providing and using evidence of progress.
assess the programme’s performance on value for money, considering measures
put in place at - and between - programme and grant partners.
assess the overall effectiveness of programme implementation, drawing lessons
and making recommendations for future programming and implementation;
assess the efficacy of the programme’s governance and management
arrangements (including financial management).
3. Review Process
The review was conducted by a team of three consultants1 and a DFID Zambia
Accountability Adviser2 from January 16th to February 2nd 2012. The process included
consultations with the STAR Programme Management Team, Grant Partners, CSOs
who had applied for but did not receive STAR funding, Members of the Steering
Committee, all Development Partners, staff of Coffey (the lead Partner in the
implementing Consortium ), members of the other Consortium partners, and various
Service Providers. Meetings were held in Accra and in Tamale where many of the
Grant Partners are based. For a list of people and organizations met, please see Annex
5 to this report.
The process also consisted of a review of a significant amount of documentation from
STAR including PMT inception and monitoring reports, Grant partner reports, financial
data, reports from the service providers, media reports on GP activities, applications
and proposals of select GPs and evaluations and project completion reports of STAR’s
predecessors.
1
2
Taaka Awori, Richard Johnson and John Rowley.
Mulima Akapelwa.
The process of conducting the annual review has not been without its challenges.
STAR is a large and complex programme. The documentation is immense and the
stakeholders are many. There is only so much that can be covered in a mission period
of a little over two weeks. For this reason, this review and this narrative report in
particular only seek, under each of TOR areas, to capture significant achievements and
issues where corrective action is urgently needed. A deliberate effort was made to keep
this report short and simple. Further details of many of the issues raised herein can be
found in the Annexes on Monitoring and Evaluation, Value for Money and Governance
contained in Volume II to this report. It is strongly recommended that these Annexes
are carefully read by the Programme Management Team who will be responsible for
taking forward the recommendations. Other stakeholders such as the Development
Partners and Steering Committee who want to better understand specific issues raised
in this report, should read the relevant sections of the annexes.
4. Progress against Expected Results
Overall the Review team was impressed by the significant amount of work that has
gone into setting up STAR in a very short period of time. Significant achievements
include:
» Awarding grants to 40 organisations in the Results Initiative (RI) and Oil and Gas
Calls (during inception) and launching three additional calls on elections,
education and sustainability within education.
» Supporting organisations to conduct capacity self-assessments and following this
up with support from Quality Assurers, M&E Service Providers and Change
Facilitators to work with grant partners.
» Conducting comprehensive Political Economy Analyses (PEA) of Ghana in
general and the education sector in particular.
» Developing programme strategies for capacity building, sustainability, gender
and social inclusion (GESI) and work with Parliament and the media.
» Establishing governance structures which include a Steering Committee of
eminent Ghanaians with significant and diverse expertise.
» Developing an M&E framework and baseline for the log frame.
These achievements notwithstanding, DFID requires that the log frame is the framework
against which STAR’s progress is measured. While using the log frame to measure
progress is a commonly-used and objective benchmark for assessment, it is limited in
that the process can appear to be rather mechanistic, i.e. you are primarily asking
whether the milestone was met or not? In addition, the assessment is dependent on the
quality of the log frame and the milestones set. To address these limitations, the
recommendations provided in this section go beyond the narrow review of milestones to
look more broadly at the progress made towards the outputs.
a. Output 1: Capability of CSOs
Output 1: Capability of grant partners to enable citizens, particularly women, children,
and excluded groups, to claim rights increased.
The Review Team found that the Output “moderately did not meet expectation”3. The
milestones for the first and second indicators in terms of grant partners capability and
number of public actions to demand citizens’ rights were achieved. However, the
milestone was not achieved for the third indicator which is: “the number of grant
partners reporting increased number of constituents particularly women, children and
vulnerable groups claiming specific rights at local and national levels.”
This third milestone is at the heart of the output which in practice means citizens
increasingly claiming their own rights rather than civil society organizations (CSOs)
claiming rights on their behalf. A good example of this approach is reflected in the work
of the Center for Environmental Impact Analysis (CEIA) which empowered the residents
of Tarkwa Nsuaem Municipality and Prestea Huni Valley District to address the
detrimental effects of mining on their health. STAR and Grant Partner reports indicate
only 9 organisations made significant efforts to empower men and women across the
sector which was well below the rather ambitious target of 41. Included within the 9
organisations were the Kumasi Institute of Technology and Environment (KITE) who
trained and facilitated citizens of Jomoro District to contribute to the development of the
District Medium Term Development Plan and the Pan-African Organization for
Sustainable Development (POSDEV) which worked with queen mothers and other
women’s groups to become more involved in leadership within local government.
To ensure achievement of the output in the future, the Review Team recommends that
STAR should:
1. Include approaches and tools that nurture the direct agency of citizens,
particularly women and excluded groups, within GP capacity building. This
would include conscientization of constituents so that they see themselves as
agents of change and rights holders rather than victims or beneficiaries; training
and equipping constituents with sufficient information so that they can engage
directly with duty bearers and creating safe and empowering spaces and process
for the engagement between constituents and duty bearers.4 Best practices in
this respect could be shared amongst organizations during the thematic learning
festivals.
2. Ensure that CSO capability to nurture the direct agency of constituents is
included in the results framework of GPs.
3
This is the terminology used in the DFID annual review template which this narrative report has adopted to
ensure consistency.
4
For further information and examples of this see: Samuel, J (2002): What is People-Centered Advocacy, PLA
Notes, 43: 9-12 at www.planotes.org/documents/plan_04302.pdf.
b. Output 2: Grant Partner Engagement
Output 2: Grant partner engagement in policy formulation, implementation, and
monitoring enhanced.
The Review Team found that the Output moderately exceeded expectations as follows:
Indicator 2.1 - No. of partners demonstrating support to public access (including
private sector) to budget, policy, and performance information.
Year 1 Milestone - 25 Grant partners. Most of the 41 GPs have included public
awareness of government policies and/or budget as part of their activities.
Indicator 2.2 - No. of representations and submissions made by grant partners to
Parliament for changes to policy, implementation, and legislation particularly in ensuring
gender equality and social inclusion.
Year 1 Milestone - 7 representations and submissions to Parliament. 15 presentations
were made to Parliament by GPs.
Indicator 2.3 - Effective monitoring initiatives undertaken by grant partners that improve
accountability and responsiveness of government, private sector and traditional
authorities.
Year 1 Milestone - 11 initiatives undertaken by GPs. STAR and GP reports indicate 12
monitoring initiatives have been undertaken.
Undoubtedly, progress achieved against this output built on significant work undertaken
by CSOs before STAR. This is because the Results Initiative Call was designed to
support work that had begun under STAR’s predecessors but needed additional funding
for results to be achieved. Grant Partners who are beginning new initiatives may not be
able to achieve the same results in the grant period. Experience in Ghana and
elsewhere has shown that CSO influencing of Government is more effective when
engagement is consistent (rather than one-offs) and when trust is present in the
relationships. Ensuring consistent engagement and developing relationships is much
easier when CSOs have formal spaces for encounters. This also ensures that CSO
access to government is not dependent on the goodwill of one public servant.
Some of these spaces exist, like the education sector working group but are not
effectively utilized by CSOs. In the next year, STAR should encourage the education
grant partners to work with the other education CSOs to effectively use this space. At
the District level, STAR could explore with grant partners within a particular theme what
mechanism would be appropriate to ensure regular and consistent follow up on their
engagement with the District Assembly and how they could best lobby for the creation
of such a mechanism.
Furthermore, to ensure that CSO engagement with Government has an aggregate
impact, linkages have to be made between work at the national and the local level. In
all its calls, STAR has encouraged GPs to form coalitions and alliances to maximize the
impact of their work. However, GPs expressed concerns about these networks, which
some claim are primarily formed to access funding rather than formed around genuine
commitment to a shared vision. Either through the upcoming learning festivals or GP
conventions, STAR should encourage a frank dialogue amongst GPs about the
challenges in national-local networking. The dialogue should include lesson learning
from national local platforms which are commonly cited as having been successful, such
as the Civil Society Platform on Oil and Gas and the Ghana Federation of the Disabled
advocacy on the 2% of the District Assembly Common Fund (DACF) for Persons with
Disabilities. Individualized support to GPs should also be provided on this issue through
PMT technical support and the Quality Assurers.
In this respect, the Review Team recommends that STAR should:
1. Support GPs to effectively use or lobby for formal spaces for engagement with
Government at both national and local level where there can be consistent follow
up on issues raised and relationships of trust can be developed.
2. Encourage a frank dialogue and lesson learning amongst GPs on how to conduct
effective national - local networking.
3. Provide technical support to GPs to strengthen work in coalitions particularly
those involving national and local CSOs.
c. Output 3: Use of Civil Society Evidence
Output 3: Increased use of civil society evidence in policy and practice
The Review team found that the output met expectations as follows:
Indicator 3.1 - Number of GPs generating critical evidence on policy issues and
practices.
Year 1Milestone - 10 grant partners. STAR and GP reports indicate 15 GP generating
critical evidence.
Indicator 3.2 - Number of different actions undertaken by GPs to make available
evidence from their work to policy makers and implementers.
Year 1 Milestone - 35 actions undertaken. The STAR reports that 35 actions were
undertaken and a review of documentation during this annual review indicates that this
is about right.
Indicator 3.3 - Number of GPs whose evidence has been taken up in policy making and
implementation around STAR-Ghana’s thematic priorities.
Year 1 Milestone - 11 GPs. The STAR report indicates that 11 GPs have had their
evidence taken up and review of documentation during this annual review indicates that
this is about right.
Again, success in meeting the output milestones can be probably attributed to previous
work undertaken by the GPs.5 In order to sustain these achievements, STAR will have
to support the new grant partners to use more innovative and politically astute
strategies. STAR’s project memorandum recognizes that the following four factors are
important in determining CSO influence on policy outcomes: (i) the existence and
strength of alliances between organisations representing those affected, their support
groups, reformers within government and elected representatives; (ii) the quality of
research and documentation on problems and proposed solutions; (iii) a skilled
combination of popular mobilisation and public pressure (with lobbying and the use of
political access where it exists); and (iv) the employment of the media to influence
public opinion, increase awareness of rights and responsibilities and change the terms
of debate.
The evidence reflects that to date GPs have not effectively used the right combination of
popular mobilisation, media engagement and alliance building to achieve their advocacy
objectives. Particularly in the run up to elections where the attention of both Parliament
and Government are on campaigning, it will be critical for GPs to very strategic and
tactical in their choice of strategies. STAR can provide support to grant partners in this
respect through the learning festivals, peer learning and coaching from the Quality
Assurers.
In this respect, the Review Team recommends that STAR should:

Support GPs to use more politically astute strategies for influencing government
particularly in the run up to elections. This would include strategic engagement
with the political machinery and leadership of the 3-4 major political parties,
careful use of the media and reframing constituents as voters rather than
beneficiaries and encouraging constituents to engage with political aspirants as
voters on the critical issues.
d. Output 4: Strengthened Parliamentary functions
Output 4: Strengthened legislative, oversight and representative functions of Parliament
in relation to priority issues identified by STAR-Ghana.
The Review team found that Output substantially did not meet expectations as follows:
5
It should also be noted that the milestones set in this output were more realistic than in other outputs.
Output indicator 4.1 - Committees’ improved access to information for legislation.
Year 1 Milestone - Committee work plans in place. Committees have draft work plans
but these will require significant work to be completed.
Output indicator 4.2 - Committees have conducted increased oversight activities.
Year 1 Milestone - Committees have plans that contain efficient oversight actions.
Some oversight activities are included in the Committees’ work plans. However, as
noted previously, these plans still require significant work to be completed.
Output indicator 4.3 - Number and outcomes of Parliamentary interactions with
citizens at all levels.
Year 1 Milestone is 3 out of six committees are proactively engaging with citizens.
Since action plans are yet to be finalized, Committees have not begun to engage
citizens based on these plans.
The difficulty in achieving the targeted results for this output reflects how time
consuming it is to work with Parliament in a manner that is owned and driven by the
institution. Currently the main instrument STAR is using is Technical Assistance. The
review team was not convinced whether the current level and mix of local TA will enable
STAR to achieve its targets with respect to Parliament and there are similar concerns
within the PMT. There was insufficient time for the team to study the issue in detail and
propose an alternative set-up but we would encourage STAR, with support from the
international TA on Parliament to review current arrangements and develop a revised
plan for discussion at the next STAR global retreat.
The Addendum to STAR’s Parliamentary Strategy reflects a very wise re-consideration
of planned activities for this year given impending elections. Given the experience to
date, the list of planned activities still seems too ambitious. What may be more realistic
is to prioritise on the Committees work stream: (i) finalization of the work plans; and (ii)
reflections on the 5th Parliament. On the work stream with the Parliamentary Service,
STAR should concentrate on: (i) Preparation of the Induction Programme for new MPs;
(ii) Development of Committee Specific Resource Manuals which would be critical for
the new MPs; and (iii) the Speaker Breakfast Forum which could address pressing
legislative issues of current GPs such as the Right to Information Bill, the Mental Health
Bill and the Property Rights of Spouses Bill.
Finally, despite the efforts of STAR to ensure a connection between the Committees
and the Education GPs, the draft work plans don’t currently reflect a very strong linkage.
One option to address this gap is for STAR to facilitate an interface between GPs and
the committees on education and women and children in particular to present some of
the issues they are working on and explore how Parliament can be of support.
In this respect, the Annual Review Team recommends that STAR should:
1. Review the nature and level of TA required to achieve the Parliamentary work
stream objectives and present a revised plan during the next global retreat in this
respect.
2. Scale-down the level of planned activities for Parliament given impending
national elections.
3. Strengthen the link between Committees and the work of Education GPs.
5. Monitoring & Evaluation
The TORs for the Annual Review asked the Review Team to: “assess the effectiveness
of the programme in capturing and monitoring results and providing and using evidence
of progress.” The following section provides the key findings and recommendations in
this respect. A more detailed discussion of M&E can be found at Annex 2 to this report.
The Review Team found:
»Good
focus on results: Overall, the methods developed and applied oblige grant
partners to pay attention to results. The tools are effective and the focus on results that
appears in project proposals results frameworks and quarterly reports is appropriate.
»Significant
support to M&E. The focus on M&E as the first area for capacity
strengthening was justified. The Review team were able to talk to all four of the Service
Providers in M&E and a number of grant partners who had taken part in the training
provided. All the grant partners met during the review said that the support in M&E
from the Service Providers was excellent and much appreciated. The same very
positive appreciation was heard about the results days that were carried out in Accra
during the Review.
»Variable
quality in quarterly reports - The quarterly reports from GPs are a key
element of M&E and a major source of information for reporting on the log frame. The
management of the quarterly reports is therefore essential to the success of the M&E.
The reports seen during this review are variable in quality. The best are very good
whilst the poor reports contain all the common difficulties that are routinely encountered
in project reports. These include the confusion between impact indicators and the
completion of activities; assertions of impact with no corroborating evidence; and high
expectations that do not relate to the level of activity. It was also observed that a
significant number of quarterly reports contain sections that have not been completed.
Nevertheless some of the quarterly reports conform to the requirements of the form and
demonstrate a complete grasp of results based approaches and contain excellent
recording of milestones and accounts of changes. They are among the best results
based project reports seen and could be used directly at any level of reporting.
»Rejections,
budget slashing, reporting needs put pressure on relationships.
.
The question of management of relationships comes out of interviews with GPs which
raised a number of concerns over the quality of communications and understanding
between STAR-Ghana and GPs. The very demanding applications procedures; the
capacity self-assessments; the rejections of proposals; the budget slashing and the
demands of the reporting formats all caused GPs difficulties. These difficulties were not
mitigated by good relationships which might have helped communication and
understanding. The biggest concern comes from comments which suggest that GPs
may choose not to raise questions with STAR-Ghana for fear of negative repercussions.
»Log
frame seems incomplete and hard to use. There is a need for a further
reworking of the log frame which contains a number of difficulties. Overall there is a
need to follow the convention that the programme should be able to take responsibility
for success at output levels and that it should be easy to make the connection between
success at output and outcome levels. The question should be asked: Is a GP able to
do this? If yes, then it may be an output. If success requires others to act as well as the
GP then in may be an outcome. Currently there is a mix of indicators with some
apparently too close to the completion of activities and some too far removed (and too
close to being outcomes) which may create problems for the programme in later
reviews.
»Theory
of Change requires strengthening: The Inception Report describes Theory
of Change (TOC) which is centred on a description of a Government Business Cycle
(GBC) and shows where pressure can be applied to the theoretical and actual
processes involved in that cycle. The main point that comes out of the presentation of
the GBC is a concern to improve as many elements in the cycle as possible. This is to
avoid the perceived problem that too much CSO activity is focused on advocacy for
policy change which is not followed up by sufficient work on implementation and
monitoring of government performance. This description is not a complete theory of
change for STAR because it does not show how the various actions of the programme
lead to changes that lead to the impacts that the project is intended to contribute to.
Based on these findings, the Review Team recommends that STAR should:
1. Dedicate time in the next year, with professional help if needed, to reflect on
how to strengthen relationships between the various stakeholders of STAR,
particularly between the PMT, applicants and grant partners.
2. Review and develop for approval a revised Log frame and attendant Theory
of Change at the next Global Retreat.
6. Value for Money
The TORs for the Annual Review asked the Review Team to: “assess the programme’s
performance on value for money, considering measures put in place at - and between programme and grant partners.” As with the section on M&E, this section provides the
key findings and recommendations with respect to Value for Money. A more detailed
discussion of STAR’s performance on value for money is provided in Annex 3.
The Review Team found:
»STAR
has a deeply integrated approach to VfM and effectiveness. STAR has put
Value for Money at the core of its project design. This is reflected in its engagement
with projects, its assessment of management costs, and its assessment of effectiveness
within the log frame. The 3Es are used by STAR-Ghana in its approach of
“mainstreaming” VfM. These include:
 Economy – getting the best value inputs
 Efficiency – maximising the outputs for a given level of inputs
 Effectiveness – ensuring that the outputs deliver the desired outcome.
»STAR’s
drive for economy & efficiency puts effectiveness at risk. STAR has
achieved strong advances in understanding and applying principles of economy and
effectiveness. STAR has worked hard at economy and efficiency to get best value.
However, its application of these principles risks undermining project effectiveness and
value for money. For example, STAR reduced successful applicants’ proposed project
budgets from approximately $24 m to $10m, a cut of 60%, in the first 4 calls. All CSOs
encountered, however, reported that their budgets had been “slashed” without adequate
explanation or agreement. While the attempt to achieve economies is valuable, where
budgets are imposed without the agreement of the grant partner, the imposition is likely
to result in reduced output, causing loss of effectiveness and impact.
STAR has also looked hard at its own management costs. Similarly, one of the
strategies STAR used to improve the efficiency of grantees’ operations was to
encourage shared working between grant partners. In some instances, this has been
perceived as STAR requiring grantees to commit to enter an alliance as a condition of
accepting a grant. In this respect, while alliances between willing partners are
innovative, and offer both efficiency savings and improvements in effectiveness,
alliances between unwilling partners cannot be enforced, and may damage project
effectiveness and sustainability as well as efficiency. At the heart of this is that the
contribution of these efforts to VfM depends on the process used and the underlying
relationships between STAR and GPs. Recognizing the power that the project wields
with respect to potential and successful grant partners, it is critical that STAR’s
relationships with Grant Partners fosters a spirit of partnership and ownership by Grant
Partners.
»Cost
of management does not reflect its value for Programme Outputs. STAR
has conducted a useful benchmark study of programme management costs6. It is,
however, concerned that its management cost of £5.0m over five years appears high at
26.9% of grant spend, relative to a comparable range of 20-27%. However, the PMT
also contributes to the development of capacity within civil society, and this goes
beyond management. Budgets and accounting systems can record expenditure based
on inputs, activities or outputs. STAR currently presents its management expenditure as
inputs: fees for staff, consultants and associated expenses. An activity-based budget
would enable STAR to distinguish expenditure on administering grants and capacity
building. An Output-based budget would show the contribution by PMT and TA to
STAR’s Outputs. STAR has already made progress in this direction, by funding QA
under the Grants budget.
of a political economic analysis of civil society to determine STAR’s
strategy and legacy. STAR’s initial P.E.A. does not assess civil society in the same
depth as government. The absence of such an assessment increases the likelihood that
STAR could have an adverse as well as positive impact on civil society. STAR is widely
reported to be the dominant funder of civil society. Its creation, evolution and closure
can be expected to have major impact on civil society. Co-funding DPs and CSOs
expressed concerns about its dominance, and the fact that its replacement of bilateral
funders reduces opportunities for funding.
»Absence
To make an effective and sustainable contribution to civil society, STAR needs a clear
understanding of the sector, its strengths and weaknesses, its resource base, and its
prospects for the future. The initial PEA of Ghana tells us about some of the
achievements of civil society7, but less about the sector as a whole. Without a clearer
strategy based on a fuller analysis, there is a high risk of unintended consequences.
Furthermore, the strategy needs to be clearly reflected within the log frame.
This will ensure that STAR provides appropriate resources to maximise sustainable
impact, thereby maximising effectiveness and cost effectiveness.
»Without
progress towards exit strategy, long-term impact, sustainability and VfM
are at risk. There was uncertainty about what will happen after 2015. There was just
as much uncertainty about whether STAR itself will live on after 2015. Alternatives were
offered for the future of STAR itself, both in discussion and in the Operations Manual.
While everyone “hopes” for a sustained benefit, there is no clear strategy within the log
frame to achieve it, at either the programme or project level. An exit strategy, whether a
follow-on programme or otherwise, is important for STAR. Without it, there is a risk that
gains are dissipated, resulting in a serious loss of value for money. This is a
fundamental major risk to STAR’s sustained impact and therefore to its VfM.
6
Benchmarking of Fund Management Costs, STAR January 2012
Political Economic Analysis of Ghana and Thematic Strategy Development for STAR-Ghana, Inception Report,
Appendix 3.1, Section 6.3
7
Based on these findings, the Review Team recommends the following:
1. Focus on process and relationships with GPs to win real effectiveness and
VfM. See M&E recommendation no. 2.
2. Build a widely owned Civil Society PEA and strategy for sustainable impact.
This should be used as a basis for its exit or other strategy beyond 2015. This
process should be led by the steering committee, with the intention of
achieving broad ownership within civil society and other stakeholders.
3. Revise log frame to include a strategy for civil society and for long term
sustainability of its impact.
4. Consider extracting CB/TA from management cost, and make provision for it
in project grant budget. Where PMT and TA contribute to outputs beyond
such core functions as strategic management, secretariat and grant
administration, STAR should consider showing these as expenses against the
programme outputs.
7. Programme Implementation
The TOR for the Annual Review asked the Review team to: “assess the overall
effectiveness of programme implementation, drawing lessons and making
recommendations for future programming and implementation.” Programme
implementation has largely comprised of grant making, technical support from the PMT
and capacity building.
The Review Team found:
informed by in-depth understanding of sector – STAR has conducted political
economy analyses of Ghana generally and the education sector in particular. Both
analyses are very useful in that they enable STAR to understand the drivers of change
and disincentives and or incentives for government officials to behave in a more
transparent and accountable manner. In education, in particular, sharing the findings
and recommendations with the Grant Partners also contributed to a better
understanding within the CSOs of the key issues that needed to be addressed.
»Calls
»Well
managed and robust application process. From the inception period to date,
STAR has made five calls for proposals: Oil and Gas, Results Initiative, Education,
Elections and the Education Sustainability Grant. On the whole, these calls have been
well managed and robust. Furthermore, each time calls were made, the application
process was strengthened to respond to challenges identified in previous calls. Grant
Partners who have applied more than once for STAR funding gave examples of
changes that they really appreciated. For instance, the timing of the regional
information sessions were changed to make them early enough for CSOs to include
their learning in their applications. Grant Partners also noted that at first when the
deadlines were extended if you had already put in your application, you couldn’t retrieve
it to work on it during the extended period. In the last call, however, this changed and
now applicants can make the most of the extended deadline by retrieving their
applications. These changes reflect a learning culture within the PMT and flexibility in
designing processes to be more effective. As the PMT continues in this spirit, Grant
Partners have asked STAR to consider other areas in further strengthening the
application process. These include:
 Improving the IT platform that supports the application process – often it is
difficult for applicants to attach documents
 Ensuring regional consultations are held in all regional capitals; avoid
combining any of the northern regions; it puts the cost of travel on CSOs
rather than on STAR.
 Avoid extension of deadlines because it creates uncertainty and confusion.
Rather right from the beginning give a longer period for organisations to
submit EOIs and proposals.
»Gender
Equality and Social Inclusion (GESI) comprehensively integrated in key
STAR processes. STAR has gone a long way to integrate GESI into its systems and
procedures. The log frame includes specific indicators for GESI, the application form
asks for specific information on GESI, the website provides detailed guidance on GESI,
the information sessions also provide further information on GESI and the
organizational assessment tool includes a dimension on GESI. STAR reports that as a
result of these efforts, the GESI content of the education grantees was much higher
than that of the results initiative grantees. Capacity development of GPs in GESI has
been planned for early this year.
»STAR’s
approach to social change is very state-centred. STAR’s theory of
change is centred on the Government Business Cycle. The GBC provides a useful
framework against which to analyze Grant Partner action and ensures that CSO’s work
is effectively placed in the context of government’s work. However, as STAR itself
admits: “PEA shows that incentives often mean the de jure model of the government
business cycle is distorted. Change can happen by understanding the de facto situation
and working with it.”8 STAR aims ultimately to influence the behaviour of government.
As the PEA indicates many of the factors affecting the behaviour of government occur
well outside the GBC. Thus while STAR should continue to support grant partners to
situate their work within the de jure model of the GBC, they increasingly need to support
grant partners to work within the de facto model of government business in Ghana
which is largely driven by political patronage.
»Capacity building as currently implemented unlikely to achieve desired results.
By the time of this review, STAR had not fully embarked on its capacity building
activities. Up to this point, capacity building had primarily been provided by the Quality
Assurers and the Monitoring and Evaluation Service Providers and to a lesser degree
by the PMT through the information sessions and feedback on grant partner reports.
While the activities undertaken reflect a commitment to build the capacity of Grant
8
STAR Inception Report at pg. 7.
Partners, the manner in which they have been undertaken is not in alignment with
STAR’s own capacity building strategy and is unlikely to result in sustainable capacity
strengthening within grant partners. First, as STAR’s capacity building strategy quotes:
“Capacity-building efforts to be sustainable, interventions need to adopt a participatory
approach and develop into empowering partnerships for which those involved feel a
high degree of ownership.” The Strategy goes to explain that capacity building in STAR
should be principally demand-driven, with STAR-Ghana playing a connecting and
shaping role when needed in order to ‘join dots’, avoid duplication and put people in
touch. Grant Partners report that while they valued the sessions on M&E by the
service providers they were not consulted on the choice of the service provider nor on
the nature of support to be provided.The same could be said of the Quality Assurers.
Thus the use of external resource people was done in a manner that was neither
participatory nor owned by grant partners.
The second challenge with the STAR’s capacity building activities so far is that they
have been characterized by short training by the Service Providers or short inputs by
the Quality Assurers. A key lesson from other programmes is that achieving the goals
envisioned in STAR’s capacity building strategy (i.e. to be, to do and relate) is a labourintensive process often characterized by accompaniment. Given the increasing number
of grant partners it is not clear to what extent the PMT can play that role. It is not also
clear whether the Quality Assurers are expected to play that role as “critical friends” or
whether they are more “monitoring policemen.”9
This is an issue that STAR must study closely and decide on. Ultimately, with support
from the Technical Adviser on Capacity Building, STAR needs to revise its current
approach to capacity building to become one that is more demand-driven, participatory
and includes sufficient accompaniment to achieve the desired results.
»STAR still largely inaccessible to small remotely-located CSOs.
While STAR has gone some way in going beyond the usual Accra-based CSOs, the
programme is still largely inaccessible to small CSOs, particularly those outside of the
regional capitals. This point was emphasized by the grant partners outside of Accra.
International organisations such as Ibis talked about the extensive support they had to
provide to some of their partners in the application process because even though the
organizations were clear about what they wanted to do, they found it difficult to
articulate it in a manner required by STAR. Many of these small organizations are
those whose presence on the ground gives them the credibility and knowledge to
mobilize constituents. They, therefore, play an important role in enabling STAR to
achieve its ultimate objectives. STAR’s current application process is a one-size fits all.
There is a need to look at different processes for different types of organisations. One
option may be to create a window specifically for smaller emerging CSOs. Another
option may be to use intermediaries to support small organisations to access the
funding and implement the initiatives. Ultimately within the next year, STAR should
9
More details on the Review Team’s concerns with the capacity building activities are provided in Annex 2 to this
report on M&E.
examine various options and design something that is appropriate. Whatever option is
selected should be based on lessons from other CSOs funds in Ghana and elsewhere
and on consultations with civil society.
»Understanding
and ownership of the sustainability grant by shortlisted GPs.
One of the concerns that CSOs constantly raise about STAR is the absence of the type
of core funding that was available under G-RAP. The sustainability grant is STARGhana’s approach to develop greater sustainability among CSOs. The sustainability
grant has been offered on a competitive basis within a particular theme and to date 31
organisations have been shortlisted within the Education theme. Discussions with a
number of the shortlisted organisations revealed a lack of understanding about what
activities would ultimately be eligible for funding and how this grant is different from core
funding. The success of the sustainability grant and the extent to which civil society
buys into this idea will be largely influenced by the first group of grant partners. It will be
critical, therefore, that STAR ensures that those who are eventually awarded the grants
fully understand and own the process envisioned. The PEA of the civil society sector
should also inform further roll out of these types of grants.
Based on these findings, the Review Team recommends that STAR should:
1. Review implementation of capacity building strategy to ensure that it is more
demand driven, participatory and encompasses more long term
accompaniment support to grant partners.
2. Develop a strategy to enable smaller, remotely-based CSOs to access
funding.
3. Hold information session with shortlisted CSOs to better explain the
sustainability grant and include lessons from the PEA of the civil society
sector in subsequent sustainability grant awards.
8. Governance and Management
The TORs for the Annual Review asked the Review Team to: “assess the efficacy of the
programme’s governance and management arrangements (including financial
management).” As with the section on M&E and Value for Money, this section presents
primarily the key findings and recommendations. A more detailed discussion of STAR’s
performance on Governance and Management is provided in Annex 4.
The Review found:
»Broadly
effective working between contractor, management agent, consortium,
funders and PMT. The administrative working relationships for STAR between these
parties are broadly effective. The administrative axis between the PMT, Coffey and
DFID is a powerful driving force, but may be difficult to challenge.
Funders engage strongly in the programme, particularly DFID as lead contractor.
Currently it is also chair of both the Funders Committee and represents the funders on
the Steering Committee, roles which it will hand over this year. Other funders had varied
views on their own engagement, and maintain a constructively critical communication
with DFID. Broadly their views on this relationship are positive: they support STAR’s
contribution to donor harmonisation and lowering of transaction costs; they support
STAR’s breadth of coverage; they feel able to influence the programme or leave DFID
to take a leadership role, as is appropriate for them.
However, donors have concerns about the quality of STAR’s engagement with civil
society. There is concern whether STAR has sufficient insight into some specific
themes, such as the elections. Funders are interested in the future of STAR beyond
2015, but are generally open-minded, without a fixed vision of what this may be.
Consortium members have varying relationships with STAR. As Lead agent, Coffey
has a strong managerial relationship. Other consortium members were positive about
their relationship with each other and with Coffey. There are clear roles for the
Programme Director (Coffey) and Technical Director (ITAD). The other consortium
members have less defined roles. However, consortium relationships with the PMT
have not been easy for all partners, and this has reduced the contribution of members
of the consortium. The use of TA from consortium members is planned within the
annual TA work plan.
There was concern within the Consortium, echoed by service providers and others,
about the adequacy of technical assistance for the broad organisational development
needs of CSOs, partly as a result of budgetary constraint.
»Rights
and accountability re applicants, CSOs, wider Ghanaian society, staff and
consultants less clear. Accountability is about more than formal mechanisms. Trust
had been affected by the transition from former funding mechanisms, including IBIS, GRAP and RAVI, to STAR. Although donors and the emerging new organisation worked
hard to consult on proposed changes, there were many difficulties to overcome, and
some felt that the engagement of civil society was inadequate.
Some remain concerned that STAR’s relationship with civil society is as a “conduit” for
funds, rather than as a cooperative strategic agency, and that it is dominated by
pressure to “get money out the door”.
»The
Steering Committee does not yet steer strategy or hold management to
account. STAR has provided a structure for its committees, and appointed respected
members to its Steering Committee, but they have not yet been given authority to
exercise the full range of duties with which they are charged. Historically, many of the
key decisions in relation to STAR were made before the appointment of the Steering
Committee, notably the formulation of the project memorandum, the award of the
management contract and the establishment of the PMT. The SC was only appointed 4
months into the Inception Period.
The Steering Committee has therefore made a useful start in grant scrutiny, but has yet
to take on a wider oversight. There is no record of it commenting on or approving the
Year 1 Implementation Plan, Inception Report, Log frame updates, Operations Manual,
Quarterly Report or 6 Month Report, nor of receiving any report from the Funders
Committee. It has not yet made any major contribution to the strategic direction of
STAR. It was reported that at its meeting in December it was agreed that it would have
a regular role in policy development, but this has not been confirmed. The Steering
Committee might be expected to make a strong contribution to the capacity of STAR to
develop its strategic role in relation to civil society, but has not clearly had the
opportunity to do so.
In addition, the SC’s capacity to hold management to account is limited. The
Programme Director does not report to the Steering Committee but to DFID. Neither the
Programme Director nor the Technical Director regularly report to the SC, nor work
closely with members between meetings.
In assessing the performance of the governing structures, it is impossible to ignore the
strong administrative axis of power between PMT, Coffey and DFID and the Co-funding
DPs. These three have defined the main administrative components that drive STAR:
the Inception plan, Log frame, Implementation plan and Operations Manual. Major
reports are prepared by the PMT and Coffey, and presented to DFID. This
administrative axis remains the driving force for planning and implementation, and is
likely to be difficult to challenge. There is still some way to go before a governing body
can maintain an overview and intervene within this axis.
systems need development – document retrieval, flow of funds, and
audit. Completing office systems is important to reduce delays and to reduce fiduciary
risk. The review found it difficult and slow to obtain documents both on projects and for
the programme as a whole. We were not given complete project documents either in
physical, chronological files, or in complete electronic folders. Old emails were archived
and difficult to retrieve. This makes it difficult for either PMT staff or an external auditor
to construct the history of any project, and thereby monitor, control or correct it. It could
also affect the broader quality of relationship with each project. It was similarly difficult to
get the agenda, papers and approved minutes of each Steering Committee meeting. On
this basis, we were not able to reconcile practical document management with the
requirements of Coffey’s ISO Quality System as set out in the Operations Manual and
its appendix 5.3.
»Some
Audit responsibility remains to be fully established. Although responsibility for the
overview and execution of internal and external financial audit is specified in the
Operations Manual, no one appeared to be taking responsibility for internal audit. DFID
has proposed ToR for external audit, having agreed with STAR that the first audit
should be for 18 months which includes the inception phase and the full year
implementation. Early in a project’s life it is important to establish and confirm financial
controls at project and programme level, in order to mitigate fiduciary risk. After 15
months of operation, it could be expected that internal audit procedures would have
been put into operation. It is sometimes seen as good practice for the external and
internal auditor to cooperate in establishing and maintaining both internal controls,
reporting to management; and independent external audit, reporting to stakeholders10.
Based on these findings, the Review Team recommends that STAR should:
1. Steering Committee to evolve to lead direction of STAR and hold
management to account. STAR’s funders, management agent and PMT
need to ensure that the SC is empowered to exercise its duties of
governance, including the leadership of key strategic processes.
2. Deepen accountability to local stakeholders, including GPs. STAR should
develop formal and informal ways to engage applicants, GPs and wider
civil society. “Be the Change you wish to see in this world.” Gandhi
3. Develop PMT systems to provide easy access to all information about
projects, management and governance. STAR’s systems should meet the
management agent’s quality standards. Quality does not mean
complexity: STAR’s information requirements and their management
should be as simple as possible.
9. Conclusion
As noted before, the Review Team was highly impressed by how much STAR
achieved in such a short period of time. While the pressure will be on to get more
grants out the door in the coming year, it will be critical to balance this frantic pace
with reflection about some of the strategic issues emerging. The review team hopes
that we have provided a mirror by which STAR can look at itself and collectively with
its stakeholders determine how best to strengthen the programme to achieve more
accountable and transparent governance in Ghana.
*****
10
Cooperation between Internal and External Auditors, A Good Practice Guide, National Audit Office, UK