Common menu bar links
Institutional links
Proactive Disclosure
Mozambique Country Program Evaluation-Synthesis Report-2004-2005 to 2008-2009
Table of Contents
Introductory Note
The Country Program Evaluation (CPE) assessed the overall performance and results of CIDA''s interventions in Mozambique from 2004-2005 to 2008-2009, a five-year period corresponding to CIDA's latest Country Development Programming Framework (CDPF) for Mozambique.
The Program addressed relevant needs in the four sectors of focus: education, agriculture/rural development, health-HIV/AIDS, and governance. The CPE concluded that, while there is room for improvement, the Program was well managed and achieved very good results. The CPE also found that the Joint Review process in Mozambique, involving government/donor monitoring, is a good governance and oversight mechanism but needs to be strengthened further.
CIDA's contributions through projects and program-based approaches (PBAs) in Mozambique led to various development results. For example, the Education Sector Pool Funds contributed to net enrolment increases in lower primary schools from 69 percent to a national average of 100 percent. In rural development, 248 water points were constructed (52 percent of water committee members are women). The Training for Health Renewal Project captured the Health Ministry's attention as a model for other training centres throughout Mozambique.
Executive Summary
This Country Program Evaluation (CPE) for Mozambique has been undertaken as a response to the Canadian Federal Accountability Act, which requires all Departments to assess the performance of their programs every five years. It has also been undertaken in parallel with other CPEs in response to a Treasury Board requirement to review Program-Based Approaches (PBAs) including Budget Support.
Mozambique Context
Mozambique has been a major African success story in post-conflict resolution and rehabilitation. It is endowed with vast and untapped natural resources that can support its economic and human development. Since 1992, Mozambique has made considerable progress and the political and development terrain has rapidly changed for the better. The Government of Mozambique (GoM) has led the country's development with the Millennium Development Goals (MDGs), Agenda 2025, and the Program for the Reduction of Absolute Poverty (PARPA) providing the key elements of its development strategy.
While several of the PARPA targets will be achieved, several will not. Major human development deficits and tremendous challenges remain ahead. The most critical issue is how to translate high macroeconomic growth rates into effective action to reduce the acute poverty affecting the majority of the country's population. This challenge will require persistent commitment and hard work on the part of Mozambican citizens and their government (along with support from the international community) to reduce poverty, advance human development, and improve economic wellbeing in a sustainable manner. This will undoubtedly be made more difficult by the increasing frequency of natural disasters due to global warming. Mozambique's current situation can be summarized as follows:
- After a history of turmoil and civil war, since 1992 the country has experienced a period of relative peace.
- Over the past 10 years it has achieved a high level of economic growth (averaging 5 percent-8 percent per year).
- Human development remains very low. The prevalence of HIV/AIDS is increasing, affecting all social and economic activities.
- Strongly supported by the international community, the GoM has undertaken major administrative reforms at the central level that are slowly being expanded to the local and district levels.
- The country faces major environmental challenges with recurring cycles of drought and floods that are likely to be exacerbated by climate change.
Mozambique is a highly aid dependent country. In 2007 it received over US$1 billion from members of the Development Assistance Committee (DAC) of the Organization for Economic Cooperation and Development (OECD). The largest bilateral contributors are the United States and United Kingdom. Canada ranks ninth. There is a very dynamic donor community and coordination system in Mozambique through the Program Aid Partnership (PAP) in which Canada plays an active role.
Evaluation Coverage, Approach, Methodology
The Mozambique CPE, undertaken between October 2008 and June 2009, addressed eight evaluation criteria recognized by CIDA and the DAC: 1) relevance, 2) effectiveness/results, 3) sustainability, 4) coherence, 5) efficiency, 6) management principles, 7) cross-cutting issues, and 8) performance management/monitoring and evaluation. The CPE also looked at the relative performance of the delivery mechanisms/channels.
The CPE reviewed the CIDA interventions implemented over the CDPF period from 2004-2009, in the four sectors of concentration: 1) education, 2) agriculture and rural development, 3) HIV/AIDS, and 4) governance. It also examined the four cross-cutting themes: gender equality, HIV/AIDS, capacity development, and environment.
From FY 2004-2005 to FY 2008-2009, CIDA invested CAD$324 million of which CAD$227 million (70 percent) was contributed by bilateral funding, CAD$82 million (25 percent) came from Multilateral Branch and CAD$14 million (4 percent) came from the Canadian Partnership Branch (CPB).
A sample of 24 projects was reviewed on a rating scale of 0-5 (ranging from "highly unsatisfactory" to "highly satisfactory"). These projects represented approximately 70 percent of the value of CIDA's entire portfolio in Mozambique (for rating scores, see Annex E: Program Aggregate Averages).
The evaluation team has benefited from the collaborative spirit demonstrated by the Mozambique Program team at headquarters and in the field throughout the CPE process, which has resulted in a mutually beneficial learning experience.
General Conclusions
Over the course of the period reviewed by this CPE, the nature and scope of the Canadian cooperation program in Mozambique has shifted substantially. From a project approach within the context of a Canadian cooperation strategy managed under Canadian rules and regulations approved at CIDA headquarters, it became a much larger and broader program in the context of donor coordination where decisions are made in the field among cooperation partners led by the GoM. The adoption of PBAs is a significant element of such global dynamics. In that context, the Paris Declaration on Aid Effectiveness approved internationally and embodied in the Canadian Official Development Assistance Accountability Act and the more recent Accra Agenda for Action create new responsibilities and obligations for CIDA at headquarters and in the field.
The increased focus on donor coordination in the field has led CIDA to make the right decision to decentralize program management. The evaluation team sees this as a move in the right direction. However, such a decision will have a limited impact if it were not to be accompanied by a number of management and organizational decisions that support this move. The lack of predictability of Canadian cooperation that in the past reduced CIDA's credibility has been partially addressed by the Program through a long-term multi-sectoral Treasury Board Submission. The allocation of human resources to ensure the appropriate representation of CIDA in the various task forces and committees existing in Maputo remains an issue. The sectoral tables imply technical expertise and if CIDA wants to influence development outcomes including assuming a leadership role, the right person must be positioned at the right place. Finally, the issue of the levels of delegated authority to the field must be addressed.
Overall, CIDA's Program in Mozambique is satisfactory, contributing substantially to the well-being of the Mozambican people in all sectors of focus, and well aligned with PARPA (I and II). The evaluation team considers that, while there is room for improvement, the Mozambique Program-which has unfolded in a very favourable development context of local ownership and donor alignment-has achieved some very good results and is a sound, successful, and well-managed Program from which to learn and upon which to build.
CIDA is an active and respected member in the Program Aid Partnership (PAP) and Joint Review exercise. As a mid-sized actor, active and respected, CIDA is in a good position to recognize its limitations and pursue the discussion on Canadian cooperation with actors from the Canadian and international communities to find the "niches" where CIDA can be most effective within the whole, accepting to delegate responsibilities based on the principle of division of labour where appropriate.
CIDA's 2004-2009 Country Development Programming Framework (CDPF) for Mozambique was designed based on a "balanced approach", the appropriateness of which was confirmed by the 2007 CDPF Mid-Term Evaluation. The CPE team's viewpoint is that the balanced approach was appropriate and reasonable at the time at which the CDPF was started. CIDA is a mid-sized actor and should continue playing a positive role in Mozambique. With a new CDPF being considered, CIDA should weigh where events have moved since 2004. In doing so, it should position itself strategically, directing its funding and human resources to where they will have greater impact and deciding where CIDA can play a role and where it cannot. In the crowded donor environment, it is important that the Canadian cooperation find the niche where CIDA can be most effective to help the GoM achieve its poverty reduction goals.
Of concern, the Program has a low level of sustainability due to Mozambique's high level of aid dependency, the low capacity of its local institutions, and its limited taxation revenues. Support from the international community will continue to be needed in the foreseeable future. CIDA's decision to decentralize the administration of the Program in 2004 was appropriate but limited by the absence of real decision-making authority and appropriate technical resources in the field. The evaluation found that major efforts were dedicated to performance management on the part of CIDA, the GoM and the international community. Nevertheless, both the Canadian cooperation program and that of the GoM/PAP's management for results systems need to be strengthened.
Main Findings and Recommendations
The recommendations below apply to the bilateral program in Mozambique unless otherwise stated.
Findings on Relevance: The evaluation was asked to consider the extent to which the objectives of the development interventions were consistent with country needs. All sectors scored "highly satisfactory" (4.4 overall) on the issue of relevance, indicating a close alignment of the CDPF and PARPA. Given the human development deficit, most resources were dedicated to the social aspects of human poverty (education and health). The programs also kept strategic continuity in the programming despite the recurrent policy pendulum existing at CIDA during that period.
Recommendation #1 (Relevance): The Mozambique Program should keep the focus on poverty reduction especially in the area of education and find ways to reinforce its strategic role in that area, while opening a dialogue with Canadian and other actors of the international community and the Mozambican partners to determine where it can best contribute and make a difference in health, agriculture and rural development, while specific strategic interventions could be supported in the governance area.
Findings on Effectiveness/Results: The evaluation was asked to identify what has been achieved, related to the intervention's objectives, or is expected to be achieved. Overall, the effectiveness of the projects is "satisfactory" (3.1). General budget support, bilateral directive projects, and partnership responsive projects were rated "moderately satisfactory" (respectively 2.3, 2.4 and 2.6), whereas sector budget support, multilateral responsive projects, and bilateral responsive projects scored higher (respectively 3.1, 3.6 and 3.7, or "satisfactory"). This could be explained by some projects being over-ambitious, while others underachieved or had poor reporting on results. In general, results were most difficult to demonstrate in the area of governance, while health, education and agriculture had several success stories. Limited evidence found to substantiate the achievement of results is likely due to the fact that the results-based management (RBM) system is more geared to monitoring activities than to tracking results.
Recommendation (Effectiveness/Results): see Performance Management and Joint Review below.
Findings on Sustainability: The evaluation was asked to examine the possibility of continuation of benefits from a development intervention after major development assistance has been completed. Overall, this element was "moderately satisfactory" (2.4), given the country's high level of aid dependency and the low technical capacity of its national and local institutions, making it unlikely that the current development interventions can be sustainable in the short to medium term. The bilateral and multilateral responsive projects appear to have higher sustainability ratings (3.1 "satisfactory") based on financial and technical criteria, while bilateral directive and partnership responsive projects were "moderately satisfactory" (respectively, 2.1 and 2.3). PBAs appeared to be less sustainable (2.0 "unsatisfactory"). The issue of "aid dependency" could put the country at risk, especially in the case where many international actors were reconsidering their contribution to pool funds and budget support interventions.
Recommendation #2 (Sustainability): The Mozambique Program should work in tandem with the international community to assist the GoM to address the issue of aid dependency. It should collaborate with interested international partners and local authorities in a capacity development facility that would help the GoM access international expertise in key strategic areas.
Findings on Coherence: The evaluation was asked to examine the consistency of development interventions among development actors including the government, non-governmental organizations (NGOs), and international organizations. The issue of coherence scored "satisfactory" (3.7) with two different sides to the coin. Overall, interventions were very coherent when measured against criteria of external coherence and complementarity to other donor programs and the GoM. The General Budget Support (GBS) investment was rated the most coherent (5.0 "highly satisfactory") as it is the core structure of the PAP (which is formed by the nineteen international donors). In that context, the CIDA Maputo-based team appeared to be active and well respected by the donor community and the GoM. CIDA participates actively in several working groups, including as chair. Concerning internal coherence, there were experiences of collaboration among multilateral and bilateral actors at the project level, however more coherence among Canadian cooperation channels (bilateral, multilateral, and partnership) would be needed in terms of information sharing and strategic orientations. The Accra Agenda for Action if it were to be fully implemented would have major implications on how donor coordination is organized in Maputo and would create new requirements for the CIDA management team in the field regarding the increased participation of civil society organizations.
Recommendation #3 (Coherence): CIDA is well regarded and could envisage playing an even more active role in the context of the donor coordination dynamic in Mozambique. Program managers at headquarters and in the field should dedicate more effort to sharing information, strategic thinking and monitor progress among Canadian cooperation delivery channels (bilateral, multilateral, partnership programs). Multilateral and bilateral teams should collaborate more specifically in the areas of health-HIV/AIDS and humanitarian assistance, while the bilateral program and CPB should cooperate to help strengthen civil society in Mozambique.
Findings on Efficiency: The evaluation was asked to assess how economically resources/inputs (funds, expertise, time) were converted to results. Efficiency was assessed as being "satisfactory" (3.2) in all sectors however certain delivery channels were rated as less efficient than others. Bilateral directive and partnership responsive projects were "moderately satisfactory" (2.8), while all other delivery mechanisms were rated higher: bilateral responsive projects and PBAs were "satisfactory" (ranging from 3.3 to 4.0), and multilateral responsive projects were "highly satisfactory" (4.1). The level of human resources at headquarters and in the field remained stable during the CDPF period with a minimal increase in the last year. As PBA participation and leadership in sector working groups seems to be increasing, the human resource mix between generalists and specialists will need to be reconsidered to have the right person at the right place. As well, longer-term Canadian appointments are advised. The Program's efficiency was affected by the limited delegation of authority in the field, the very slow approval process at headquarters, and the absence of capacity to respond to legal issues in the context of the 2009 negotiations for the PAP Memorandum of Understanding (MoU), putting at risk the credibility of Canadian cooperation.
Recommendation #4 (Efficiency): The Mozambique Program should seek to have the appropriate level of sectoral or thematic expertise and skill sets in the field on a sustained basis in areas where it wants to play a strategic role. Corporate level consideration: In order to strengthen field presence, the Geographic Programs Branch, in consultation with the "providers" of Agency corporate services, should consider various scenarios to improve the provision of corporate services and response to the field including contractual, legal and financial services; time needed for approval; and increased delegation of approval authority to the field.
Findings on Management Principles: The evaluation was asked to examine the management principles applied in relation to ownership, alignment, and harmonization as defined in the Paris Declaration. Overall, the sectors reported "satisfactory" adherence (average of 3.2). The CPE's assessment was corroborated by two reviews on the same subject: (i) a survey sponsored by the DAC-OECD and (ii) a review sponsored by the PAP and undertaken by a private firm. All three gave Mozambique a high rating on the Paris Declaration principles, specifically mentioning the high quality of ownership and leadership, and strong application of the principle of alignment. Harmonization and coordination were acceptable, but could be improved, especially for common analytical work (e.g. in 2007, 337 donor missions took place but only 65 of them—19 percent—were jointly undertaken). However, since the 2008 Third High-Level Forum on Aid Effectiveness in Accra, it has been recognized that activities under the Paris Declaration principles need to include civil society organizations (CSOs) as well as government agencies. The issue of predictability of Canadian Official Development Assistance (ODA) that had negatively affected the credibility of the Canadian cooperation program in the field has been partially addressed by presenting a global program submission to the Treasury Board Secretariat in 2009-2010.
Recommendation #5 (Management Principles): In tandem with other actors of the international community, the Mozambique Program should enter into dialogue with the GoM to improve civil society organizations' participation in Mozambique in line with the Accra Agenda for Action including through strengthening the capacity and independence of the Poverty Observatory—now called Development Observatory—or other institutions that can play a similar role. The Mozambique Program should bring to the attention of the Program Aid Partnership the need to put in place a method or system for sharing information regarding donor missions to Mozambique in order to reduce their overall number and increase joint analytical work by international organizations in sectors and themes of common interest.
Findings on Cross-Cutting Issues: The evaluation was asked to consider the extent to which the CDPF's cross-cutting issues—gender equality, the environment, HIV/AIDS and capacity development—were integrated into the Program's design, implementation and results. The "satisfactory" score (3.4) for cross-cutting issues derives mostly from their having been considered in policy and planning, rather than from evidence of results. Special recognition should be given to CIDA's role regarding gender equality where it has played a leadership role recognized by the international community. The contribution made in policy dialogue and technical terms could be complemented by strategic investments.
Recommendation #6 (Cross-Cutting Issues): Given its already good credibility regarding gender equality, the Mozambique Program should consider developing a strategic approach including dedicating the necessary level of financial and human resources to be even more effective.
Findings on Performance Management: The evaluation was asked to assess whether the management strategy for assessing the performance of development interventions against stated results (outputs, outcomes and impacts) including Monitoring and Evaluation (M&E) was adequate and appropriate. Performance was considered "moderately satisfactory" (2.8) given again a variety of situations. The Joint Review (although requiring improvement, see below) and the PAP coordination process are considered by many as a good practice for the international community. While at the project level, most interventions have been subject to an evaluation, attention remains focused on activities rather than on results and impact.
Recommendation #7 (Performance Management): The Mozambique Program should strengthen the overall performance management of project interventions to make them more results-based and improve tracking and reporting on results especially regarding outcomes and impacts. This could include providing results-based management (RBM) training to staff and partners.
Findings on the Joint Review Process: The evaluation was asked whether the Joint Review process might constitute an appropriate tool to fulfil Canadian accountability requirements. In some aspects it was, and in others it was not. Given the high volume of aid and the large number of donors, a common assessment process is necessary to avoid GoM and donor duplication and to save transaction costs for all parties. However, the Joint Review process lacks independence and rigour. All participants in the process have an interest that the assessment be positive, and this influences how indicators are chosen and measured. The Joint Review process also includes a meeting between the GoM and CSOs. The evaluation team considered that this meeting did not meet commonly accepted standards of full participation. That said, even if it does not fulfil all of Canada's accountability requirements as far as evaluations are concerned, Mozambique's flagship Joint Review process is ostensibly one of the most sophisticated machineries that the donor community has put in place in Africa (the other one is in Tanzania) to ensure that donor coordination takes place. This system has entailed a proliferation of working groups, etc., that put much pressure and demands on the CIDA staff, managers, and advisers in the field. The CHC and PSU teams should be commended for their work at these venues which has earned Canada to be well-regarded and seen as a fair player in the donor community.
Recommendation #8 (Joint Review Process): The Mozambique Program should work in tandem with other actors of the international community to streamline the Joint Review process in place in Mozambique to make it less cumbersome, while complementing it by other evaluation activities in order to respect the principles of impartiality, independence and rigour. This should include strengthening the "accountability system" of the GoM and the capacity of other local actors.
Findings on Delivery Mechanisms/Channels: The evaluation was asked to examine the relative performance of the various delivery mechanisms used by the Canadian cooperation in Mozambique, namely directive projects, responsive projects, General Budget Support (GBS) and Sectoral Budget Support (SBS). The CIDA management team has adopted a mix of delivery mechanisms, which the CPE finds appropriate to country needs, i.e. to help the GoM to address poverty reduction. Among the sample of projects reviewed, the bilateral responsive and multilateral responsive projects had a higher overall performance rating (3.7 and 3.9, or "satisfactory"), compared to bilateral directive and partnership responsive projects (2.8 and 2.9, or "moderately satisfactory"). The Paris Declaration indicates that 66 percent of all development cooperation should be channelled through PBAs by 2010. CIDA is in good position to comply with this requirement. The delays and transaction costs created by the limited delegation of authority to the field have affected the credibility of the Canadian Program management in Mozambique and its capacity to play a leadership role in the field. The increasing size of projects, especially PBAs, including GBS and SBS, creates an obligation for CIDA and the Treasury Board Secretariat to revisit the issue.
Regarding channels (bilateral, multilateral, partnership), CIDA needs to decide if it wants to be seen and perceived as "one actor" and take advantage of the potential synergies of collaboration among channels of cooperation. Some very good experience of collaboration exists in the CIDA Mozambique Program among multilateral, bilateral and partnership interventions. The Program's capacity to identify and finance successful NGO projects initially funded by CPB, in order to increase the scope and reach of the good development results they were achieving in Mozambique in health, education, and ARD (e.g. TRHP, CRSP, SLAP, PLEM, IRWDP), is a success story that should be institutionalized beyond the project level, i.e. to the program and corporate levels at CIDA.
Recommendation #9 (Delivery Mechanisms/ Channels): The Mozambique Program should maintain the relative balance regarding the use and management of different delivery mechanisms, taking into account their strengths and weaknesses based on an assessment of risk as well as the results that the Mozambique Program aims to achieve in its next CDPF. The Program should work in tandem with other actors of the international community in order to improve General and Sector Budget Support regarding the effectiveness in attainment of results, as well as the sustainability and inclusion of cross-cutting issues.
Management Response
"The Mozambique team has seen this evaluation process as a positive experience. The evaluation team has been cooperative, open to suggestions and clear with their findings. The experience of conducting a Country Program Evaluation during the Joint Review in Mozambique was a learning experience for all parties involved. While it required an exceptional degree of flexibility on the part of the evaluation team, the Program feels it was a valuable experience which could impact the view of evaluation in highly harmonized donor environments. The recommendations are relevant to the Program and will affect future programming, but the findings, lessons, and conclusions of the evaluation will also have an impact on the Mozambique Program over the coming years."
You will find in Annex I the full Management Response including commitments for action, responsibility centres, and target completion dates.
Alternate Format
Note: If you cannot access the documents that are provided in an alternate format, refer to the Help page.
Mozambique Country Program Evaluation—Synthesis Report—2004-2005 to 2008-2009 (1.74 MB, 83 pages)
