CIDA's Policy for Performance Review

• Part I - Policy
• 1. Definition part 1
• 2. Context part 1
• 3. Scope of Application part 1
• 4. Purpose part 1
• 5. Elements of the Performance Review Framework part 1
  • 5.1 Performance Assessment part 1
  • 5.2 Evaluation part 1
  • 5.3 Internal Audit part 1
  • 5.4 External Reporting part 1
• 6. Organization of the Performance Review System part 1
• 7. Dissemination and Communication of Review Results part 1
• 8. Follow-up and Tracking Progress in Policy Implementation part 1
• Figure 1 : Framework for Performance Review
  
  
• Part II—Guide to the Policy
• 1. Definition part 2
• 2. Context part 2
• 3. Scope of Application part 2
• 4. Purpose part 2
• 5. Elements of the Performance Review Framework part 2
  • 5.1 Performance Assessment part 2
  • 5.2 Evaluation part 2
  • 5.3 Internal Audit part 2
  • 5.4 External Reporting part 2
• 6. Organization of the Performance Review System part 2
• 7. Dissemination and Communication of Review Results part 2
• 8. Follow-up and Tracking Progress in Policy Implementation part 2

Part I—Policy

1. Definition

The performance review system encompasses all functions and instruments used by managers and staff of the Canadian International Development Agency (CIDA) to assess Agency development and operational results. This includes, in particular, evaluation, internal audit, and monitoring of policies, programs, projects, institutions and operations of the Agency.

2. Context

CIDA's mission is to support sustainable development in developing countries. To achieve this, CIDA's programs support processes of social, economic and political betterment, which respect the environment.

The success of CIDA's efforts in helping developing countries achieve sustainable development depends on a set of circumstances, many of which are beyond its control. There is no definitive way of pursuing development. The risks associated with the experimental nature of some of CIDA's activities mean that a number of activities may fall short of meeting desired results, while others may exceed them. There is a need to ensure that the lessons learned from these activities become part of the Agency's knowledge base and lead to improved performance.

The wide scope of development cooperation undertaken by CIDA makes measurement of development results—the impact on economic, social and political development of developing countries—a challenging task. In some cases, such as projects of a physical nature, results are easily quantified. In other cases, such as human capacity building, results are more qualitative and long-term in nature. Nevertheless, it is essential to have a sound information base which allows the Agency to assess aid effectiveness: the degree to which its programs achieve the desired development results, and how efficiently programs are delivered.

The Performance Review system outlined below is designed to allow CIDA to assess aid effectiveness. It is a key element of the Agency's transformation into a more results-oriented, transparent and accountable organization. It is also an integral part of the management process that links a range of activities from corporate planning to reporting. It seeks to enable managers to better exercise their accountabilities by offering them tools to help them achieve expected results.

The system of review in place at CIDA at the time of our review was based on the Agency Internal Audit Policy (1982) and its Evaluation Policy (1987). Under these policies, audit and evaluation were used as surveillance and control functions, rather than for knowledge building and improving decision-making and overall performance. The consultations undertaken during preparation of this policy revealed a number of shortcomings in the current system which needed to be addressed, including: inadequate external reporting; lack of consistency in definition and purpose of review functions; and a defensive culture with respect to performance review.

A new approach to performance review was thus essential to translate into action the Agency management renewal which focuses on excellence, knowledge and learning, teamwork and innovation.

Performance review in CIDA must reflect the reality of development cooperation, including the needs of developing country beneficiaries, and the perspectives of Canadian partners and executing agencies which participate in program delivery.

3. Scope of Application

The Performance Review Policy applies to:

• all development and management policies, programs, projects, and other activities undertaken by the Agency;
• all management and staff.

4. Purpose

The goal of performance review in CIDA is improved aid effectiveness: achievement of development results and effective and efficient program delivery.

Performance review is based on the premise that managers are accountable for achieving results.

The purpose of the Performance Review Policy is to ensure that CIDA managers and staff at all levels have credible, timely and useful information on the performance of the Agency's policies, programs, projects, operations and other activities, including the development results they achieve.

A Performance Review framework seeks to enable CIDA to:

• report to Parliament and the public on CIDA's aid effectiveness;
• improve decision-making to achieve better performance;
• exercise accountabilities for development results, value for money, and relevance to mandate, policies and priorities;
• assess compliance with laws and regulations applicable to the Agency.

In order to meet these purposes, the Agency must define its intended development results and establish indicators of aid effectiveness.

5. Elements of the Performance Review Framework

The framework supporting this policy has three elements: performance assessment as a branch-level function; evaluation and internal audit as corporate functions; and a combination of these elements with other sources of information and analysis, which allows CIDA to provide an account of its activities through external reporting. Branch-level review activities are necessarily linked with corporate evaluation, audit and external reporting activities.

Accountabilities for each element are shown in Figure 1.

5.1 Performance Assessment

Performance assessment comprises program, project or institutional monitoring, operational reviews, end-of-project reporting, institutional assessments and special studies. It also comprises monitoring of Agency policies, management and service functions. In program branches, for example, performance assessment is carried out by managers and staff, with advice and support from designated program branch review staff. It provides managers with regular information on the operational and developmental performance of their programs, projects or institutional support activities.

Project or institutional monitoring enables managers and staff to:

• assess progress toward expected development results, using predetermined performance indicators;
• measure operational performance;
• receive information from executing agencies and partners on operational and development results;
• determine the desirability of undertaking subsequent phases or renewed funding;
• act on review findings.

Operational reviews enable managers to identify and resolve serious operational difficulties, and assess efficiency of delivery mechanisms.

End of Project Reports or Institutional Assessments provide managers with information on results achieved and lessons learned to take decisions on future funding. Institutional assessments from other sources can also serve as a basis for funding decisions. The rigour of such reports is ensured by common efforts of branch- and corporate-level staff to develop standards for conducting such reviews.

Branch-level research and special studies, drawing on end-of-project reports and institutional assessments, provide staff and managers with thematic or program syntheses of development results and lessons learned. Self-assessment and participatory approaches are encouraged.

Such syntheses provide branch managers with an aggregate picture of operational and developmental results and contribute to meeting corporate information needs on Agency performance. Branch and corporate performance review staff work together to ensure methodological consistency and objectivity of data which serve as input to the corporate information management system.

Review frameworks are incorporated into project, institutional support and program approval documents.

Branch heads can request the Performance Review Division to carry out audits or evaluations of their branch policies, programs, projects, or institutional support activities.

5.2 Evaluation

Evaluation comprises a set of applied research instruments that provides a systematic, objective assessment of elements of Agency performance.

It provides senior managers with the information they need to assess the performance of CIDA's policies and programs, as an input to improved corporate decision-making and external reporting. It also focuses on determining the developmental impact of priority project and institutional support activities. Evaluation is carried out in a manner which ensures objectivity of results, and the participation of staff, managers, CIDA partners and beneficiaries in developing countries.

Policies and programs to be evaluated, and corporate synthesis reports to be prepared, are determined by the Executive Committee in accordance with corporate priorities, as part of the corporate planning cycle.

Evaluation findings and synthesis reports provide the President and the Executive Committee with credible, timely, and objective information to enable them to:

• assess results, continued relevance, and cost-effectiveness of policies, programs and projects;
• determine consistency of branch-level programs with corporate policies and priorities;
• take informed decisions in allocating Agency resources;
• determine adequacy of follow-up action on key evaluation findings.

Evaluations also provide CIDA staff with analyses and findings which help build knowledge on development policies and practices.

Selected project or institutional evaluations that relate to corporate priorities are conducted to assess development impact. Sets of project impact studies and other analyses, including policy research and multi-donor studies, are analyzed in synthesis reports to allow the Agency and its partners to get a broader picture of development impact.

Evaluation is conducted in a way which is consistent with Treasury Board standards and guidelines.

5.3 Internal Audit

Internal audit is an independent appraisal function that assesses the effectiveness and efficiency of management frameworks at all levels. It provides CIDA managers and staff with information on the effectiveness and efficiency of systems, practices and built-in controls adopted to achieve stated objectives.

Internal audit assists CIDA managers and staff by identifying improvements to operational practices and systems that support the delivery of development assistance and contributes to the Agency's institutional learning capacity by disseminating best management practices.

Internal audits are conducted in areas of significance and risk to provide the President and the Executive Committee with credible, timely and objective information to enable them to determine:

• the effectiveness of program delivery activities and internal operations, including the economic and efficient use of resources;
• the reliability and integrity of information for decision-making and reporting on accountability;
• prudence and probity in the use of public funds and the safeguarding of assets;
• compliance with statutes and regulations;
• adequacy of follow-up action on key audit findings.

This is achieved through internal audits conducted on systems, functions, programs/responsibility centres and projects, as required, and through establishment of an Agency statutory compliance framework.

In all these areas, internal audit conforms to Internal Auditing Standards in the Government of Canada, and the Code of Ethics for Internal Auditors.

5.4 External Reporting

A major function of performance review is to provide credible, timely and relevant information needed to support external reporting on aid effectiveness. Indicators of performance, reflecting Agency development and management priorities, provide the basis for external reporting.

Corporate synthesis studies are important products for reporting purposes, and provide input to other external reports. In addition to reports to Parliament, as and when necessary, the major regular vehicles for external reporting include an enhanced Part III of the Main Estimates. Consideration will be given to developing other corporate products for publicly communicating aid effectiveness and relevance to Agency mandate. Special products are prepared to disseminate review results to CIDA partners and others. Policy and Communications Branches play a key role in this activity.

6. Organization of the Performance Review System

All parts of the Agency play important roles in ensuring the relevance and effectiveness of the Performance Review system:

• Executive Committee, with observers from the Office of the Auditor General and Treasury Board Secretariat, serves as the Agency Performance Review Committee. In addition to its role as set out in sections 5.2 and 5.3, it sets priorities for, and approves the annual plan for, corporate-level performance review.
• The Performance Review Division, situated in Corporate Management Branch, is responsible for the evaluation and internal audit functions, which are grouped together but have clear and distinct roles. The Division prepares the corporate-level performance review plan. It reports the results of evaluations, internal audits and synthesis studies to the Performance Review Committee. It is responsible for maintaining methodological rigour and providing support to branches.
• Designated Branch Performance Review Staff, located within branch policy and planning units, support managers and staff in their performance review activities. They provide methodological advice that enhances the quality of branch-level reviews, review branch inputs to the corporate information system, and ensure the wide dissemination of review results.
• A Performance Review Network, comprising the Director General of the Performance Review Division, the Directors General of Policy and Planning in the program branches, and the Director General of Policy Development in Policy Branch, fosters methodological consistency across the Agency and provides program and policy input into corporate evaluations and audits.
• Following a period of implementation of this policy, consideration will be given to establishing an Aid Effectiveness Advisory Committee, comprising representatives from inside and outside CIDA, to periodically advise the President on aid effectiveness and policy formulation.
• The Agency Information Management System supports the performance review system by providing access to relevant information on all levels of performance review.

7. Dissemination and Communication of Review Results

CIDA makes performance review reports publicly available and is committed to the widest dissemination of review products in a manner consistent with the Access to Information Act and respecting the Privacy Act.

8. Follow-up and Tracking Progress in Policy Implementation

Implementation of this policy will involve a transition period as old practices are phased out and new ones introduced. An Agency strategy will be finalized by the Director General, Performance Review Division, in collaboration with the Performance Review Network. The implementation of this policy will be reflected in the workplan of each branch. The Executive Committee will review progress in implementing this policy on an annual basis.

Figure 1

Framework for Performance Review

Level	External	Corporate	Branch
Primary Client	Parliament Public Developing Countries	Minister President Vice-Presidents	Vice-President Program managers Projet/ institution activity managers
Primary Purpose	Public reporting	Improved decision-making and accountability through learning Improved accountabilty Statutory compliance Improved decision-making	Managing program and project institution
Function	Aid effectiveness reporting	Evaluation Internal audit	Performance assessement
Information Produced	Policy relevance Aid effectiveness Cost effectiveness	Development knowledge Most effective delivery means Best technical practices Best management practices Compliance	Development results Operational performance
Accountability	Minister reporting to Parliament, public	President reports to Minister Vice-Presidents report to President Director General, Performance Review Division, reports review information to President/ Executive Committee	Program managers report to Vice-President PTL/ institutional managers report to program managers

Part II—Guide to CIDA's Policy for Performance Review

This Guide elaborates on sections of CIDA's Policy for Performance Review, in order to facilitate its interpretation.

1. Definition (see Part I—Policy)

2. Context

The Performance Review system emphasizes:

• more coherent linkage of review activities at all levels with the corporate decision-making, planning and management systems of the organization;
• the responsibility of managers at all levels for measuring and demonstrating development performance and acting on performance information;
• a productive alliance between managers and review professionals to link review activities more visibly to learning and management decision-making, as well as accountability and external reporting practices;
• the value of objective and professional internal audit and evaluation functions.

The Performance Review Policy is part of an overall Agency effort on management renewal, which is attempting to enhance performance and promote new management values and culture. This is being undertaken through such measures as introduction of results-based management practices, clarification of accountabilities, and better human resources management. The success of these measures will have a direct bearing on implementation of the new Performance Review Policy.

3. Scope of Application (see Part I—Policy)

4. Purpose (see Part I—Policy)

5. Elements of the Performance Review Framework

The previous approach to performance review involved too many review instruments, no corporate consistency in definition and purpose, in particular for evaluation, and a system which was directed inwards.

A simplified framework for performance review was developed to:

• provide a simple, integrated approach to performance review at all levels;
• clarify the three levels of performance review and the accountabilities for each;
• identify a limited number of instruments for use at each level;
• allow more review activities to be carried out directly by CIDA staff both at the branch and at corporate levels.

Review and reporting activities at the three levels are linked. A key element in making this framework work is developing the means to ensure systematic flow of information. Information gathered from performance reviews needs to be reported upwards for accountability purposes and to permit external reporting, and needs to flow across and down to encourage learning from past experience. Essential to the flow of useful information is appropriate analysis and synthesis at each level.

The framework puts emphasis on greater involvement of CIDA staff and lesser participation of outside consultants in review functions, both at the branch and at the corporate level, to enhance the knowledge base within the Agency. To the extent possible, and allowing for a transition phase, performance review activities will be carried out by Agency staff.

5.1 Performance Assessment

Performance assessment activities are carried out under the direction of managers and staff in the branches. Historically, expenditures on project-level and institutional evaluations in the program branches have been approximately $5 million annually, and have constituted a large portion of Agency spending on evaluation. While these met operational needs, they provided little useful information for corporate review and reporting, and few insights into aid effectiveness. In addition, somewhat higher levels of expenditures have been made on project/institutional monitoring.

In the program branches, performance assessment instruments—project/institutional monitoring, operational reviews, end-of-project reports and institutional assessments—are intended to provide the necessary review information which managers and staff need on operational and developmental performance. These instruments will be redesigned to be results-based and to emphasize developmental results, while maintaining their traditional operational relevance. Where other information needs are identified, special studies and research projects will be undertaken by branches using their own resources.

It is expected that over the next three years managers will increasingly rely on strengthened monitoring instruments, and will reduce their reliance on project evaluations.

Project, institutional and policy monitoring will be improved and will continue to help managers to determine whether inputs and outputs are proceeding according to plan, and to measure progress toward achieving expected developmental results—information which they need to take decisions at project milestones.

In the corporate branches, monitoring of Agency policies, management and service functions are intended to provide managers with information about progress in policy development and implementation, as well as information about effectiveness and efficiency of Agency management and service functions.

A significant proportion of CIDA activities is delivered by third parties such as executing agencies (through contracts) and partner organizations (through grants and contribution agreements). Managers will need to ensure that these third-party organizations report on operational and, in particular, on development results. This will require revisions to contractual arrangements.

End-of-project reports or institutional assessments will be prepared for all projects or institutional support arrangements. They will play an increasingly important role in providing managers at all levels with the information they need to take decisions on future funding. An analysis and information system will be implemented which allows determination of the degree to which projects and institutional support activities are meeting expected results.

End-of-project reports need to be rigorous and defensible, in order for the Agency to have confidence in the results it reports. Project managers are responsible for ensuring the collection and reporting of project results. To be credible, the quality of end-of-project reports will need to be improved, through the application of standard approaches and formats, and by validation through independent sampling at the corporate level.

Institutional assessments from other sources will serve as a basis for funding decisions, be they independent evaluations or self-assessments by the organizations themselves. CIDA managers will verify the adequacy of such third-party assessments.

Managers and staff, assisted by branch-level review staff, can contribute to the corporate-level evaluations and audits through the gathering of development and operational information. This is expected to reduce the cost and effort required for evaluations and audits at the corporate level. In order for this to happen, a coordinated effort will be undertaken at the corporate and branch levels to develop appropriate methodologies, formats and indicators to allow development results to be retained from end-of-project reports and institutional assessments.

Analyses of information from performance assessment activities in the branches will be combined with the results of corporate-level evaluations and audits and fed back to managers in a form which permits them to include lessons learned in their program, project and institutional support decision-making.

The preparation of concise review frameworks as part of project, institutional support and program approvals documents, including Regional/Country Development Policy Frameworks, program theme/sector strategies, and institutional support strategies, will enable the respective managers to identify the most appropriate review instruments, as well as the baseline information and performance indicators they need to monitor progress during implementation.

For program branches, indicators will help review of projects, institutional support, programs and branch-level priorities. For corporate branches, indicators will be developed to review development policies, management and administration policies, and communication strategies. These branch-level performance indicators will be developed jointly by the Performance Review Division, and program or corporate branches, where applicable.

5.2 Evaluation

The Performance Review Division is responsible for the evaluation function and evaluations are carried out under its direction with the active involvement of appropriate Agency managers and staff. Emphasis will be placed on improving methodological rigour of corporate evaluations, in order to ensure objectivity and utility of evaluation findings. However, evaluation will be participatory in nature, involving Agency managers and staff with relevant experience and expertise in their conduct, where appropriate.

The utility and success of the evaluation function depends on clear corporate policies and program priorities established by the Minister, and/or the Executive Committee. Thus, the Performance Review Committee, on the recommendation of the Performance Review Division, determines long-term priorities for evaluation and for internal audit, with annual updates to take account of changes to corporate priorities. These evaluation and audit priorities form the basis for the performance review workplan at the corporate level, as an element of the Corporate Plan. Coordination with plans for branch-level performance assessment activities may provide opportunities for more efficient use of resources.

The workplan for the Performance Review Division will maintain an element of flexibility to allow for response to unforeseen priority requests for evaluations or audits.

Full assessment of the immediate and wider impact of a project or institutional support activity on its intended beneficiaries can only be made after an interval following its completion, sometimes of many years. Projects which are in line with corporate priorities will be proposed by program branches, based on criteria established by the Performance Review Division, for inclusion in the annual list of projects to be the subject of impact evaluations at the corporate level.

The feasibility and methodologies for country program evaluations will be explored. Approaches to be considered include: evaluations of sectors of Canadian involvement in a country; evaluations of Canadian involvement in a sub-region or community; and joint-donor efforts, be they paper assessments using existing national and multilateral country data, or on-the-ground, multi-donor aid impact evaluations. In all cases, such efforts must be consistent with the review framework outlined in the Regional/Country Development Policy Framework.

5.3 Internal Audit

The Performance Review Division is also responsible for the internal audit function, and audits are carried out under its direction with the active involvement of appropriate Agency managers and staff. It focuses on the management framework and its encompassing elements: i.e. systems, procedures and management practices used by managers to achieve established objectives and results, as well as the controls that provide managers reasonable assurance that results expected will be achieved.

Emphasis will be put on functions, systems and operations of greatest significance, or at greatest risk, such as major systems under development. A significant component of internal audit will be ensuring cost-effectiveness of program, project, and institutional support delivery activities and internal operations, such that they are efficient and economical, as well as effective. The Office of the Auditor General defines an effective, efficient and economical operation as follows:

• an effective operation (or system, practice, and procedure) is one that achieves the results expected of a specific activity;
• an efficient operation is one that provides results, at lowest cost and at adequate or required levels of quality and service;
• an economical operation is one that gets the right amount of the right resources—financial, human, physical, and information—at the right level of quality, at the right time, in the right place, and at the right cost.

Emphasis will also be on the useability of audit products, provided through a variety of services:

• formal audits of various types outlined in the policy;
• special studies and reviews;
• expert advice on specific management and operational issues.

Even though internal audit is a corporate function, managers will participate in identifying key management issues to be addressed, assessing related risks, and establishing audit priorities.

The Performance Review Division will develop, in collaboration with Legal Services and Policy Branch, an Agency framework on statutory compliance. The Division will identify the managers responsible for assuring Agency compliance with individual statutory laws, policies and regulations. The Division will then track and report on Agency compliance to the Performance Review Committee.

Internal audit planning will be fully integrated into the corporate planning cycle, and will be flexible in order to respond to special requests from branches. Coordination with plans for other review activities, both activities in the branches and corporate-level evaluations, will provide opportunities for more efficient use of corporate review resources.

The Agency will pursue the option of integrating Contract Cost Audit and Special Investigations Units into the Performance Review Division and possibly transferring to it responsibility for other appropriate audit-related activities.

5.4 External Reporting

The 1987 Evaluation Policy was silent on the need for external reporting on aid effectiveness. As recent events demonstrate (e.g. need to demonstrate CIDA accountability for results, as expressed by the OAG in its 1994 Report; Government trend to improve Part III of the Main Estimates), this has become an urgent requirement. Currently neither the framework nor the necessary review information for such reporting is available in the Agency. In order to provide credible information to Parliament, our partners and the public, it is essential that the Agency be able to aggregate, analyze and synthesize project, institutional support, program and policy results along thematic and business lines. In addition, it will be useful for the Executive Committee to have an annual review reporting on aid effectiveness and operational relevance of CIDA's various activities.

The feasibility of developing indicators of performance at the program, policy, and corporate levels, reflecting Agency development and management priorities, will be explored jointly by the Performance Review Division, Policy Branch and program branch review staff. The Agency information management system will be adapted to allow results from the branch and corporate levels to be aggregated on the basis of these indicators.

All of this will take a major effort involving all parts of the Performance Review system outlined in the above sections. Central to this capacity are the redesigned Agency information management system and strengthened analytical capability at the corporate level. This will require synergy among the program branch Policy and Planning Divisions, the Performance Review Division, Policy Branch, and Communications Branch.

New or renewed vehicules for public reporting, such as a substantive annual review on aid and operational effectiveness, will be needed.

6. Organization of the Performance Review Systems

The Performance Review Committee, comprising all members of the Executive Committee, will also have observers from the Office of the Auditor General and the Treasury Board Secretariat. Such outside representation will help CIDA demonstrate its openness to renewing performance review. It will allow the central agencies to assess CIDA performance in an informal manner. CIDA will benefit in learning from the experience of other departments in performance review.

The Performance Review Division will remain in Corporate Management Branch, with the evaluation and internal audit functions grouped together but maintaining clear and distinct roles. There are advantages and disadvantages to keeping these two functions together, but the advantages outweigh the disadvantages. In particular, maintaining the groups in the same division will yield resource savings and will allow the functions to better complement each other.

The Performance Review Division will animate the Performance Review system, through regular interaction with branch performance review staff, and through the Performance Review Network.

The proposal for an Aid Effectiveness Advisory Committee holds potential benefits for the Agency. Such a committee would help build Agency credibility, and enable the Agency to obtain different points of view about its policies and performance. A number of other departments, including Statistics Canada and the Auditor General's Office, have used such advisory committees to considerable advantage. A broad membership, including representation from beyond the aid community, would enhance the Agency's ability to communicate with a wider public.

7. Dissemination and Communication of Review Results

The Agency needs to be transparent in reporting aid effectiveness, on a regular basis. Key evaluation reports, synthesis reports and appropriate audit reports will be made available and may be published in the form of performance review abstracts and a performance review series.

8. Follow-up and Tracking Progress in Policy Implementation

This implementation strategy will be further developed by the Performance Review Division, working in close collaboration with the Performance Review Network. A full implementation strategy for the policy will be prepared, addressing performance review at all levels. In addition, each branch will prepare a short, concise implementation strategy which translates the policy into a specific approach for addressing its particular needs for performance review.