Review of the Afghanistan Program - Executive Summary

1. Background
2. Rationale for the Review
3. Review Objectives, Approach and Limitations
4. Overall Findings and Results 5. Risks, Challenges, Lessons and Recommendations

1. Background

Afghanistan is one of the world's poorest nations. Most Afghans lack the basic services and necessities. Years of war and destruction left much of the country's infrastructure―roads, bridges, schools, health clinics, and sources of electrical power, and water management systems― in shambles. By the end of 2001, the state's legitimacy had been damaged by military coups, the capture of the state by armed commanders, and attempts to use a weak state to transform society by force and violence. In the aftermath of September 11, 2001, the world's attention turned to Afghanistan as a focal point for global security. International support galvanized around the need for Afghanistan to recover from its status as a failed state, so as to never again become a breeding ground for terrorism that would threaten the world.

In a series of 5 international conferences, beginning with the first in Bonn in December 2001, Tokyo in January 2002, Brussels in 2003, Berlin in 2004, and London in 2006 the international community, including Canada (see Annex 7 for list of 43 donors), made specific commitments and pledged substantial financial resources for the reconstruction and development of Afghanistan. To put the Canadian support in perspective with the broader effort of the international community, it was estimated in 2004 that "to make Afghanistan secure and enable its people to live in a typically underdeveloped country, US$ 27.6 billion dollars would need to be committed over seven years" The Afghanistan Compact, the London Conference on Afghanistan, 31 January―1 February 2006, p.13. This means an average of roughly US $4 billion per year. Canada's total allocation to Afghanistan over the 2001 to 2011 period stands at $1.12 billion.

The Afghanistan Program is CIDA's most important Program in terms of its magnitude, complexity, visibility, and challenges. These challenges derive from a formidable combination of risks in areas such as: the security situation, weak governance systems, widespread concerns of corruption, limited Afghan institutional capacity for financial administration and risk management, and significant absorptive capacity constraints within the Government of Afghanistan. Annex 1 of this report provides more information on the risks, constraints and challenges faced by the CIDA Program on a daily basis.

One illustration of how these risks impact on the development efforts of CIDA and the international community comes from the omnipresent and high level of security risk throughout the country, and especially in the southern provinces, such as Kandahar. Insurgent attacks have, in some cases, slowed and prevented, programming and monitoring thereby resulting in loss of efficiency and higher accountability risk. Some international partners have left the country due to the high level of insecurity and their inability to cope with the situation. The security situation has a tremendous impact on CIDA's (and its partners') ability to attract and maintain adequate level of experienced human resources in Afghanistan. The stress level in this insecure environment leads to short rotation assignments and frequent staff changes, which increases the difficulty for recruitment. In addition, their ability to implement and monitor CIDA's initiatives is constantly hampered by the fact that they have to comply with movement restrictions and military clearance due to the high level of insecurity.

The context and challenges of Afghanistan, a state in reconstruction after many decades of war, called for a different Agency strategy: simultaneously building the foundation of the country at the national, regional and grassroots levels while providing a secure environment. This is a unique situation, requiring Canadian engagement in terms of Development, Diplomacy and Defence / military at the same time (i.e. the "3D approach''). CIDA is working closely with the Departments of National Defence (DND) and Foreign Affairs and International Trade (DFAIT). This is a new way of operating for CIDA.

This state of affairs impacts the speed at which Afghanistan is re-built, CIDA's ability to control the implementation of Aid delivery and CIDA's image vis-à-vis the Afghans and the Canadian public. In order to be more effective and efficient in its aid delivery, the international community, including CIDA, committed to pool their resources and work together through the same international partners such as the World Bank and the United Nations Organizations, which have experience in fragile states and the necessary on-site 'infrastructure' to operate. This strategy has the advantages of reducing the risks and avoiding duplication of efforts amongst donors. This approach is also considered crucial to support the nascent Afghanistan government to have a degree of predictability of funding and to gain the confidence of the Afghan population in the context of this new democracy.

---

2. Rationale for the Review

This Review was conducted to respond to CIDA management needs to review and monitor this high profile Program in terms of the magnitude of its investment and associated risks as well as to satisfy the requirement of the Canadian Government. The Agency must submit a report on the results of the Afghanistan Program to the Treasury Board Secretariat of Canada. Given the high profile of and the intense public interest in the Afghanistan Program, the Review also provides information about the accountability and use of the funds provided to the Program.

---

3. Review Objectives, Approach and Limitations

The objectives of the Review are to:

1. assess performance in terms of progress made towards the achievement of the stated results of the Afghanistan Program;
2. determine if the results are contributing to the development of a legitimate and stable state, which is the longer-term aim of CIDA's support to Afghanistan;
3. assess the adequacy of the accountability framework to ensure that the funds are properly managed and spent for intended purposes; and
4. provide lessons and recommendations to improve the Program.

The Review covers a period of almost three fiscal years, from April 1, 2004 to the end of January 2007. Significant disbursements were still underway and planned for February and March 2007. It should be noted that the Review, initially planned for fiscal year 2005-06, was postponed to fiscal year 2006-07 for two reasons: firstly, most of the initiatives were too recent to assess development results at the outcome level; and secondly, the security constraints were a major issue at that time. The Agency Audit Committee also approved a multi-year audit plan to 2007 to 2010 in April 2007. This plan includes an examination of the Afghanistan Program, encompassing disbursements on the Kandahar component.

With regard to this Review, twelve of the largest among the 27 projects in CIDA's Program were examined in detail. These 12 projects account for 83 percent in terms of the Program budget and roughly 73 percent of the $285 million spent during the period. They covered the variety and complexity of challenges, different delivery mechanisms, and thematic areas of focus. The Review did not cover initiatives in Kandahar as these only started in 2006 and it was too early to formally assess results. CIDA has disbursed over $39 million in Kandahar for fiscal year 2006-2007. These recent initiatives will be covered in the up-coming audit mentioned above.

Information for the Review was derived from both primary sources (interviews with an extensive range of relevant and knowledgeable persons) and secondary sources (documents such as existing evaluation, audit and monitoring reports produced by CIDA, other donors, the World Bank, UN organizations, Government of Afghanistan, and other independent international research institutions). The Review team did not undertake any field visits. Instead, it draws indirectly on many field visits undertaken by others whose evaluations, audits, monitoring and other reports serve as a credible basis for this Review.

---

4. Overall Findings and Results

For most of the projects examined, CIDA is one of many donors, albeit an important one. As agreed in high level Consultative Group meetings between the Government of Afghanistan (GoA) and donor countries starting in 2001, CIDA has coordinated and "pooled" most of its funding with other donors to support the GoA's national programs and efforts. As the results below indicate, the joint, "pooled" approach is an effective way of achieving results in Afghanistan. One can assume that a commensurate proportion of the outcome is due to the Canadian contribution, particularly since CIDA is one of the leading financial contributors on several of the projects.

4.1 Progress on Building a Stable State

The overall goal of the Afghanistan Program is to support the efforts of the Afghan government, the Afghan people and the international community in stabilizing Afghanistan through the consolidation of the GoA's authority and legitimacy across the nation, and through improvements in the people's well being.

The Review Team recognizes that Afghanistan is still in the very early stages of the process of building a democratic, functional and stable state, where the population has confidence and trust in the government and where the government, in turn, can provide the services needed by its people to improve their well being needed to uphold and sustain this confidence and trust. However, there are encouraging early signs of progress in this direction:

• A 2006 national survey The Asia Foundation (assisted by AC Nielsen India and others), "Afghanistan in 2006: A survey of the Afghan People", 2006. of the views of Afghans about their country reveals a complex mix of both positive and pessimistic perceptions, some progress, some decline in progress, and overall a frankness that identifies challenges for both the government and donors.
  
  
• Among Afghans surveyed, twice as many (44 percent) felt their country was heading in the right direction compared with those who felt the opposite (21 percent). For those who said the country was heading in the right direction, the main factors were: good security; peace/end of war, and disarmament. More than half (54 %) of the respondents felt that their family was more "prosperous" under the current regime than under the previous Taliban regime.

A big part of the credit has to go to the people and government of Afghanistan, whose ownership, leadership, and commitment to work with the international community to implement initiatives in a financially accountable and responsible manner was key. Credit also has to go to the assistance provided by the international community, which has provided roughly 90 percent of the development resources needed, with Canada contributing a significant part of these resources.

The quality of this assistance and its contributions to the longer term objective of state building in Afghanistan, (the essential objective of CIDA, other donors and the GoA), can be illustrated by some of the major projects in terms of their relevance, form, reach, and results as indicated below:

• Relevance: most of the assistance has been consistent with the needs of Afghanistan, as articulated in the Afghan National Development Strategy (ANDS) and the Afghanistan Compact. Investments in areas such as elections support, micro finance, improved governance, community development and others are critical to help move Afghanistan towards a stable state.
  
  
• Form: most of the assistance has been provided in a form that allows the GoA to make good use of it in terms of strengthening the capacity of the national government machinery while providing the necessary services to its people. Operating Budget Support through the Afghanistan Reconstruction Trust Fund (ARTF), the National Solidarity Program (NSP), the Micro-finance Investment Support Facility (MISFA) and National Area-based Development Program (NABDP) are all projects that are or were implemented by GoA ministries. The Ministry of Finance is centrally involved in all of these projects. The MRRD currently implements NSP and NABDP, and originally implemented MISFA before it was set up as an independent company in mid 2006.
  
  
• Reach: most of the initiatives supported are national in scope, reaching many of the 34 provinces. They also reach the district and community levels. For example, Budgetary Support to the GoA through the ARTF reached the national and local levels; MISFA reaches the national and local level financial institutions; NSP operates at the community level; and the NABDP addresses the provincial and district levels.
  
  
• Results: examples of the results being achieved by the specific initiatives are outlined in the section 4.2 below.

4.2 Results at the Level of Initiatives

Although many of the investments of the CIDA Afghanistan Program are fairly recent and some are still ongoing, significant progress is being made towards the achievement of their objectives. Among the 8 largest projects, which accounted for $192 million, or 67 percent of the $285 million disbursed by CIDA during the period covered by the Review, 6 have significantly improved people's lives. One, the Afghanistan Stabilization Program, designed to improve the operations of provincial and district government, has been less successful and CIDA took action to reallocate the funds to more successful projects. The eighth, the Western Basin Water Resources Project, was slow getting off the ground and is just starting.

More detailed information about the results of these projects are included in section 4 of this report. Highlights of results include:

― The Afghanistan Reconstruction Trust Fund (ARTF), as a mechanism for providing Operating Budgetary Support to the GoA, has emerged as one of the key instruments for financing the country's recurrent budget deficit until the GoA develops the capacity to do so. It supports the state at the national level in establishing its ability to fund public servant salaries down to the district level to deliver services and gradually develop associated management capacity. Results to date indicate that the GoA is heading in the right direction with an increasing capacity to meet its recurrent budget―revenue collection for fiscal year 2005-06 was 55 percent above the previous year.

― The Microfinance Investment Support Facility for Afghanistan (MISFA), in which CIDA is the leading contributor among 5 donors with $37.0 million or 47.0 percent of the paid-in contributions (as of January, 2007), has established financially sustainable Micro Finance Institutions (MFIs) and provided credit and savings services to approximately 300,000 people, 72 percent of whom are women. This would appear to be on target to reach the GoA objective of 800,000 households by 2010. While the reach of the MFIs is limited beyond Kabul and other urban areas, this program has been very successful in establishing independent financial institutions that are essential to livelihoods and effective governance.

― The National Solidarity Program (NSP), in which CIDA is the second ranked contributor among donors, is the GoA's flagship program for strengthening community development, local governance, citizen participation and GoA legitimacy and credibility. It has created over 17,000 elected Community Development Councils (CDCs), which include women as members. It has reached more than 11.5 million people through community projects in 274 districts across most of the 34 provinces. The NSP is essentially grassroots democracy in action, fostering local empowerment and ownership. It demonstrates that the state is actively involved in addressing needs at the community.

― The National Area-Based Development Program (NABDP), which is a government of Afghanistan program, is beginning to develop the capacity at the provincial and district levels to serve broader regional needs, thus bridging the large gap between the central and community levels of governance. Some progress is reported at the district level; provincial level progress is much slower.

― The Elections Support Project helped to facilitate the participation of Afghan citizens and civil society in both Parliamentary and Provincial Council elections, witnessed by national and international observers. About 5,800 candidates (582 women) stood for election and 6.8 million persons voted (51.5 percent turnout, 43 percent women).

― The Mine Action Project succeeded in removing and destroying significant amounts of anti-personnel mines, reducing the numbers of mine-related deaths, and cleared significant areas of land which were previously unusable.

― Due to poor performance of the Afghanistan Stabilization Program (ASP), the World Bank decided to withdraw its support from it. After exercising its own due diligence, CIDA also decided to "terminate" its association with it and reallocate the $12 million that was intended for ASP to other better performing initiatives such as MISFA and NSP.

Among the remaining 19 smaller projects, some have had good results while others have had mixed results:

― Projects with good results include: support to hold presidential elections, de-mining, and collecting and containing heavy weapons in a safe location.

― Those with weak results include the establishment of radio stations to provide literacy and educational services, and technical assistance to government ministries.

― More complex and challenging projects such as judicial reform, reintegrating former combatants, and developing the capacity of parliament have been slow in showing results.

4.3 Accountability Framework and Use of Fund

A good Accountability and Management Framework, with appropriate oversight mechanisms, exists to enable the Afghanistan Program to exercise its due diligence up front and during implementation to ensure that funds are used for the intended purposes. These oversight mechanisms exist at three levels: within CIDA, at the level of our multilateral partner institutions; and at the Donor/Partner Country level.

Based on information received and documents examined during the Review, our analysis confirmed the use of these by the Afghanistan Program. Our analysis also confirmed that Program funds were used for intended purposes as outlined in the Results and Risk Management and Accountability Framework (RRMAF). Indeed, based on the March 2004 notional allocation, and analysis of financial and activity reports submitted by the World Bank, United Nations Organizations, CIDA monitoring reports and Joint donor evaluation reports, we are able to conclude that the Afghanistan funds were generally spent for intended purposes. Audit reports from the Auditor General of Afghanistan attested to the fairness of information presented in the financial statements produced by implementing partners.

Even so, the Review Team identified the need for improvements in monitoring and reporting by some partners. While most partners were able to provide the information required by CIDA to assess the use of funds and results achieved, the quality and timeliness of reporting by some UN partners is an area that requires attention. In addition, CIDA staff need to ensure that the reporting requirements in agreements with partners are respected.

With regard to risk management in an environment where potential corruption is a major concern, the management capacity of the partner government is limited and threats to personal security are always a factor, mitigation actions were taken by the Program to minimize these risks. For example, financial risks were minimized by working in conjunction with other donors to channel most of the funds through multilateral organizations that have good track record and infrastructure on the ground, and operational risks were mitigated by working closely with DFAIT and the Department of National Defence (DND), adhering to security measures and standard operating security procedures to protect CIDA's field staff and its partners.

With respect to other management aspects, the CIDA Program has demonstrated good flexibility in adjusting to the changing situation and needs of Afghanistan. More recently, the addition of the Kandahar component fills a gap at the provincial level that was not being addressed by the national programs. CIDA's efforts are intended to help support the roll out of the GoA's national programs to Kandahar. As the findings from previous evaluations show, project-based funding when used selectively can effectively complement and reach areas and groups of people that are "hard to reach" through national programs for reasons of remoteness or special needs.

CIDA staff has, for the most part, made good investment decisions on the choice of initiatives and implementing partners. In such a complex, high-risk environment, the Program has a good mix of national multi-donor initiatives through the World Bank / United Nations Development Program (WB/UNDP), complemented with some smaller bilaterally directed initiatives funded through Canadian and international non-state actors. Given the significant variation in the capabilities of the Afghan ministries to deliver development to the people, CIDA has invested most of its resources in projects involving the better performing ministries. In the cases where the investments were not performing, Program staff have taken the necessary corrective action, such as stopping disbursement and /or re-investing the funds in better performing initiatives.

Given the high profile of Canada's involvement in Afghanistan, there has been a very high level of interest among Canadians in the Program. With the growth of funding through CIDA, staff on the CIDA Afghanistan Program have been spending a significant portion of their time focusing on meeting the demand for information from various sources. As the Program was already under resourced, this was a real challenge: it further reduced the amount of time that staff had to devote to the substance of development programming, including monitoring progress.

---

5. Risks, Challenges, Lessons and Recommendations

This is not to say that all is well. Afghanistan is a nascent democracy, and there are significant risks and challenges ahead. In the survey responses, Afghans identified the biggest problems at the national level as security, unemployment, the poor economy, the presence of the Taliban, and Warlords. When asked about concerns of corruption, 77% saw it as a major problem at the national as well as the provincial and lower levels.

In terms of state building, the building of sustainable local capacity within the various government ministries will continue to be a major long-term challenge, as many of the ministries are dependent on substantial expatriate technical assistance, including the returnee Afghan diaspora. There are issues relating to inequities and "fairness' of the salaries paid to foreign and Afghan returnees, compared to the salaries of average public servants.

5.1 Risk Management Lessons

Given the challenges faced by the Program, several risk management lessons are worthy of note:

• In a rapidly evolving, unstable environment, it is important for CIDA to maintain flexibility in its program strategy to be able to react quickly to unforeseen events and/or take advantage of emerging opportunities;
  
  
• In such an environment, where not every initiative will succeed, it is also important to be realistic in our expectations and have a level of risk tolerance that allows for trial and error;
  
  
• The Afghanistan experience highlights the importance for the Program to continue to coordinate and harmonize efforts with other Federal Departments [Department of Foreign Affairs and International Trade (DFAIT), DND, Royal Canadian Mounted Police (RCMP) and Corrections Services Canada (CSC)] and international and multilateral partners in order to manage the very significant risks related to security, program operations and achievement of results;
  
  
• The fact that even experienced partners do not always succeed makes it important to limit the extent to which novice organizations without experience in such insecure environments are chosen as partners to implement initiatives;
  
  
• With a flexible strategy, due diligence takes on even more importance. In addition to clear selection criteria up front for the choice of implementing partners, it is important to monitor progress closely during implementation, even when well-known, experienced international partners such as Multilateral Organizations are involved; and
  
  
• On the one hand, challenging Programs such as Afghanistan need the best, most experienced staff. On the other hand, the high security risks, short staff rotation, stressful living conditions in field in Afghanistan, and overseas allowances that are the same as those for safer countries make it very difficult to recruit and retain staff. Therefore, an adequate incentive package is a critical success factor with regard to staffing field personnel in fragile states or states in crisis.

5.2 Recommendations

The Review offers the following Recommendations:

Recommendation 1: (the need to strengthen monitoring and oversight)
a) CIDA reporting requirements were noted in existing agreements with partners. In order to allow Program staff to effectively monitor the initiatives implemented by its partners, the Program should ensure that the Agency's reporting requirements, with regard to financial information and progress on development results, are more clearly reflected in the agreements with its partners, and respected. In the case of multi-donor arrangements, such as with the Multilateral Organizations, CIDA will need to work with the other donors to ensure this happens.

b) In order to strengthen accountability for results and support knowledge building for future programming and decision making, monitoring activities performed by CIDA field and Headquarters (HQ) staff should be documented on a more systematic basis and include clear indications on progress, variance analysis of expenditures and development results on a cumulative basis, identification of issues, lessons learned and follow-up actions taken. The Program may also want to consider providing staff with appropriate training in this area as needed.

Recommendation 2: (improve results tracking and reporting)
The recently created "Results and Accountability Unit" (RAU) at HQ needs to be fully operationalized to ensure the enhancement of the Results-Based Management (RBM) framework of the Program to allow management to improve its reporting of development results and knowledge sharing. This Unit would analyse the data and the information outlined in monitoring, evaluation and audit reports in order to extract results at the different levels (i.e. output, outcome and impact), lessons learned and actions to be taken for their integration into the programming cycle.

Recommendation 3 (strengthen field operations)
CIDA should strengthen its operations in the field in the following three areas:

a) Number of staff in the field
The Agency should give consideration to increasing the number of staff in Afghanistan in order to cope with the challenges and required level of effort needed to more effectively manage the Program (e.g. intelligence gathering, analysis, monitoring, various pro-active and follow-up actions and the numerous meetings with Donors representatives and GoA officials in the field).

b) A Better Incentive Package for Field Staff
CIDA should explore with DFAIT, TBS and other central agencies, as appropriate, options to enhance the field compensation package allowed by the Foreign Service Directives (FSD) in order to provide an enhanced incentive package (accommodation, overseas allowances, home leave, etc.), which better reflects the relative hardship level and security risks that staff face in countries of conflict/crisis such as Afghanistan. This would enable the Program to attract, hire and retain high quality experienced staff with the right profile.

c) Program Support Unit (PSU)
The PSU should be operationalized on a priority basis to provide the necessary assistance to the CIDA Program to implement its strategy in the most optimal manner. The roles and responsibilities of this unit (e.g. on site intelligence gathering, monitoring, sector analysis, data collection to establish base lines, logistical and administrative support, etc.) should be clearly defined.