Investing in organizations lies at the very heart of progressive and sustainable development. Donor agencies that reach out and cultivate partnerships can help to improve the performance levels of both established and emerging organizations. Organizations need to continuously learn, grow, and adapt. People within organizations need to learn to work together to reach common goals, and externally organizations have to function effectively within the context of their enabling environments. Organizations also need to exploit opportunities to work collaboratively to build knowledge bases, skill sets and capacities for delivery. A sound framework of government ministries, civil society bodies and other key organizations provides the underpinnings required for realizing economic, social and political aspirations.
CIDA brings a strong focus to the need for healthy, vibrant organizations in developing countries. Agency programming improves organizational performance directly through institutional capacity building interventions, and more indirectly by promoting the learning experience of partners during development cooperation collaborations. To this end, Organization Assessments (OAs) can play a valuable role in laying the foundations for institutional strengthening and in informing investment decision-making. OAs provide an assessment of an organization's performance, enabling environment, resident capacity and organizational motivation. This information can be then used to build on an organization's strengths—and address its weaknesses—to improve performance. Or OAs can be carried out to decide if an organization should be engaged as a partner and/or as a candidate for Agency funding.
The 'CIDA Organization Assessment Guide' presents a common framework for conducting OAs within the Agency and guidelines for shaping execution. A process is defined that provides a systematic approach to planning and design, implementation, reporting and taking action. The Guide provides the Agency's managers and staff with a framework for employing OAs to advantage, expectations for results and standards for the preparation of terms of reference, work plans and OA reports. Practitioners are provided with a generic approach to conducting OAs that can be adapted to the task-at-hand. The intent of the Guide is not to be prescriptive, but rather to suggest approaches, methods and practices to facilitate carrying out OAs.
Our thanks are extended to the many individuals who made key contributions to the preparation of this Guide. In particular, we would like to acknowledge the encouragement and support provided by Marie-France D'Auray-Boult (Director General, Performance and Knowledge Management Branch) and Goberdhan Singh,(Director, Evaluation Division). We also acknowledge the work of Dr. Charles Lusthaus from the Universalia Management Group and Dr. Fred Carden and the Evaluation Unit of the International Development Research Centre (IDRC) in developing the OA framework, and for their continuing research in this area.
This Guide should be viewed as a work-in-progress. Refinements and adjustments will no doubt be required to accommodate 'real life' parameters and limitations, and the realities of working in the development cooperation theatre.
We are asking more and more of our partner organizations. Donor agencies, in embracing country ownership, stronger partnerships, and comprehensive approaches, are becoming increasingly reliant on their partners in recipient countries for the delivery of development cooperation programming. Broad-based programming modalities are accentuating the value of partnership at a very fundamental level. In effect, donor participation is becoming more about enabling development through the provision of financial assistance, and the management of partner relationships in a government-led, and multi-donor environment.
With dramatic changes in the language of partnership, the Agency is still identifying and learning about the day-to-day implications. CIDA Program Managers are facing fresh challenges that demand new skill sets. A greater emphasis is being placed on in-country knowledge levels, and the ability to network and carry out policy dialogue in the field. The complexities attached to development cooperation programming initiatives can be unprecedented. An in-depth knowledge of partner organizations, both new and established, becomes a vital factor in effective decision-making.
Against this background, the need for departments and agencies to demonstrate accountability, responsible spending—and clear, concise results and directions—is more important than ever before. Treasury Board Secretariat (TBS) and government auditors expect the highest professional and ethical values in ensuring the integrity of federal programming.
CIDA is committed to a results-based doctrine that emphasizes management for results. Initiatives to improve transparency and accountability are helping to build confidence that the Agency, in fulfilling its mandate, uses public funds judiciously and responsibly to achieve vital, meaningful results.
Going forward, it is imperative that programming choices are adequately informed and that the Agency consistently demonstrates fiscal responsibility in determining how CIDA invests in development cooperation—and with whom. The dynamics of development cooperation today can imply higher degrees of risk and uncertainty.
It is often critical for CIDA management to know more about an organization and what it is capable of accomplishing. In particular, this can concern an organization's candidacy for funding or partnership, or for institutional strengthening. To this end, organization assessments (OAs) can play an instrumental role in informing the Agency's decision-making processes.
This Guide was 'purpose-built' to guide and facilitate carrying out OAs, providing a common framework for consistent application and guidelines for achieving meaningful, informative results. An emphasis is brought to identifying an organization's capacities, its 'track record' in demonstrating performance, its ability to function effectively within its external environment, congruence with CIDA's strategic interests, and the level of risk associated with partnership.
This guide is divided into seven chapters
An organization comprises a cluster of people working towards a shared goal. Generally, they are created when a group of individuals are brought together for a common purpose. Organizations can involve a wide spectrum of human activity. They can be categorized as private or public, for-profit or non-profit, governmental or non-governmental, and so forth.
Within all nations, there can be found a myriad of organizations dedicated to their individual agendas—whether they be economic, political or social. Some may be more formal than others. Some may be large and powerful. Others may be smaller and evolving. Agendas may also vary. Some may champion business development. Others may advocate for good governance and/or human rights. Some may perform well, others less well, and some fail altogether.
No organization is left to function on-its-own; the interdependency of organizations is a key common factor that determines how individual organizations contribute, grow and meet their objectives.
OAs are also a common feature to all organizations, for every organization in time becomes involved in some sort of assessment (pertaining either to itself or of others). For example, when engaging another organization as a partner, client or fortake-over target, it is crucial that going forward that you are confident about the organizations' capabilities, integrity and credibility. Such assessments typically involve due diligence to ensure acceptable levels of associated risk and likelihood for success. Internal OAs can help to ensure that organizations function optimally to stay competitive and meet targeted results. Here the emphasis may be on elevating performance and productivity, and aligning day-to-day activities with primary objectives. Generally, it is the need or desire to change performance that initiates internal OAs.
How such OAs are carried out can vary widely. OAs can amount to nothing more than obtaining referrals about the credibility of an organization ("What were your experiences with the ABC Foundation?" "How did they perform?"), or they can involve a more intensive exercise involving a structured process for workplans, information collection and analysis, and reporting. Typically the level of formality attached to OAs is a function of financial significance, complexity and the time constraints attached to decision-making.
In this Guide, we address how best to perform OAs for the purposes of advancing development cooperation, and more specifically to optimize CIDA applications. To this end, we have harnessed the extensive work in this area carried out by the Universalia Management Group (with IDRC's support) over the past decade. The Universalia/IDRC Framework is specifically designed for the international development theatre. Its measure is defined by acceptance on a global scale.Testing in the field—and with Southern partners—has been essential in bringing the refinements needed to respond to 'real-life' needs and priorities.
This chapter takes a broad look at OAs, explaining what they are, the complex environment they are carried out in, linkages to over arching disciplines, and the framework for conducting OAs put forward by the Universalia/IDRC Framework. In the next chapter, we look at the potential offered by OAs in the CIDA context, and set out a step-by-step process for conducting OAs, from conceptualization through to the implementation of results and/or the sharing of information.
OAs can be defined as:
"… the process for obtaining systematic information about the performance of an organization and the factors that affect performance in order to diagnose areas of possible investments for change and/orto demonstrate competence."
However, just as there is no one road to truth, in practice there are many combinations and permutations regarding the shape an OA can take. No two can ever be the same.
OAs by their very nature are compl0ex undertakings that require a great deal ofsensitivity, tact and understanding. When you are assessing an organization youare challenging in a very fundamental way how and why things are done the way they are. For many, change is a hostile act that threatens their security and well-being.
Think about instances during your career when your worked have been assessed.How did you respond? Likely you glowed with the complements and prickled with the criticisms. At the organizational level, when everyone is involved, the circumstance only becomes more complex. OAs have to deal with organizational insensitivities at a very broad level. Leaders of organizations tend to react badly when there directions are challenged. Ministers can get angry when you criticize their ministries. Managers may be totally incompetent and not know it. Too often, itis easier to shoot the messenger—an all-to-common occurrence. Managing this dimension successfully is never an easy proposition. It is key that your judgment is defensible and that your recommendations are well-supported and pragmatic.
This complexity associated with OAs is also accentuated by the following factors:
Graphic - refer to PDF.
When conducting an OA, we try to systematically collect data that will allow us tounderstand the success of an organization—its performance and the factors that drive that performance. To this end, the Universalia/IDRC approach provides a framework of analysis, a common language and some systematic tools.
The Universalia/IDRC framework implies that key contextual forces drive organizational performance. This approach sees performance as a function of an organization's external environment, its motivation (underlying traits that define its'personality'), and the ability to use its internal capacities to achieve results. The schematic representation provided on the facing page lists the many sub-components influencing each factor that may be considered when you carry out your OA. How the framework is applied will be determined by the overall design of your OA and which information needs are most relevant and critical.
A brief elaboration on the four factors that comprise the framework follows:
In saying that organizational performance is a function of three dominant variables, there remains a need to define what performance itself actually means. Most organizations view their performance in terms of 'effectiveness' in achieving their mission, purpose and/or goals. For example, NGOs would tend to link the larger notion of organizational performance to the results of their particular programs to improve the lives of a target group (e.g. the poor). At the same time, it is likely most organizations would also see performance in terms of their 'efficiency' in terms of deploying resources (optimal use to obtain the results desired). Finally, in order foran organization to remain sustainable, it must have: 1) an expansive sense of purpose that continues to be 'relevant' to its stakeholders (implying an ability to adapt to a changing context), and 2) 'financial viability' as measured by its ability to raise funds (and generate revenues) to met its functional requirements in the short, medium and long-term. The ultimate test of any organization over time is its sustainability.
Organizations exist within certain external contexts that facilitate or impede their performance. They need to get support from the environments that they function inif they are to survive and perform well.
The environment can be the key factor in determining the ease with which an organization can carry out its activities—or the level of available resources. It is unlikely that targeted results will be achieved unless the stakeholder environmentis supportive of what the organization is intent on accomplishing. Poor macroeconomic policies can lead to high interest rates, fluctuating currencies and a host of conditions that make it difficult for some organizations to operate and perform well. Also, it is difficult to operate if there are poor infrastructure services. Things such as road systems, electricity, phone lines and so forth also influence an organization's performance. It is clear that the characteristics and quality of an organization's external environment can be key determinants in affecting the performance of the organization. The framework, therefore, identifies a number of environmental sub-components that should be considered when carrying out your OA (e.g. administrative/legal contexts, stakeholder environment, economic conditions, technological context, political factors, socio-cultural conditions, geographical context).
Performance has a strong relationship to organizational capacity: performance can be conceived as the tip of the iceberg, with the organization's underlying capacity providing either support or impediments to performance. Organizational capacity exists in a number of basic areas: strategic leadership, human resources, core resources, programming/process management, and inter-institutional linkages.Each of these areas may be described in sub-components, for example strategic leadership capacity in terms of structure, governance, leadership skills, strategic planning and niche management. Human resources and core resources (financial and infrastructural capacity) are seen as resources, as well as the management of these resources. Organizations also have capacities that result from the relations, partnerships and alliances they have established with other institutions—referred to as inter-institutional linkages.
Organizations possess internal dimensions that collectively define its personality and play a key role in motivating members to perform. The culture operating withinan organization (and the incentive systems it offers) influences organizational motivation. The framework identifies history as one dimension that you may want to consider—how and why the organization got started, what the milestones are, and so forth. The organization's mission, values and vision may also be assessed to understand what the driving forces working within the organization are. Organizational motivation affects the quality of work, how the organization competes, and the degree of involvement of institutional stakeholders in decision-making processes.
CIDA values its progressive culture where critical analysis contributes to informed development. This chapter keys on how the Agency's investments in OAs can influence effective investment choices, strengthen infrastructure capacity and build stronger partnerships. We identify how OAs can function as a powerful tool well-suited to serving the Agency's commitments to the achievement of results and organizational learning.
The following sections elaborate on CIDA's approach to OAs:
Ostensibly OAs are implemented to assess candidacy for funding and/or partnership, or provide the underpinnings for strengthening infrastructure capacity. Program Managers may undertake OAs to inform programming investments. Or they may be called on to initiate an OA for the purposes of assessing an organization to identify its strengths and weaknesses. OAs help to build sound foundations for developing close collaborations with qualified partners. OAs can take on many forms:
OAs can be also categorized by: 1) the type of organization being assessed (for profit, not-for-profit, government organization, international agency), 2) location (North, South), or the 3) the intensity of the assessment given the significance/complexity of the exercise and the time frame allowed for completion (formal, informal, quick). When the objective is strengthening an organization, OAs may be defined by a stage in the organization's life cycle (base-line, mid-term, end-of-project). OAs may also be: 1) participatory, 2) joint, 3) external/internal, 4) fully independent, or 5) self-conducted.
OAs are often more intricate, political and complex than evaluations. By their very nature, they address a broader concept than program/project outputs, impacts and outcomes. Institutional and organizational sensitivities when conducting OAs will likely be more pronounced.
OAs are about effective investment choices, strengthening infrastructure capacity and building stronger partnerships. Achieving these results supports the Agency's key objectives, mandate and strategic directions. When CIDA invests in OAs, they are more likely to engage credible, capable partners…or they may set the stage for building more effective organizations that advance civil society and good governance.
Consider the following key developments and reflect on how OAs play into this mix:
For CIDA, partnerships are essential. The development of strong, dependable and enduring partnerships is fundamental to CIDA's operational framework for the delivery of results. The Agency's partners have increasingly become the front-linedelivery agents for development cooperation. Making informed choices about investing in partnerships is becoming more and more crucial. OAs help to ensure that the 'right' partners are engaged. Investing in OAs is tied to more effective and efficient development, and prospects for greater accountability.
Within the Government of Canada, departments and agencies are being held to increasing and evolving accountability requirements, and better management of the relationship between resource expenditures and results. CIDA is directly responsible for managing the development pool of the International Assistance Envelope and accountable to the Government of Canada and Canadians for the results it obtains from its investments in development cooperation programming.
Unless due diligence is attached to partner selection and investment choices, the Agency may become involved in unfavourable situations where practices are challenged and prudence and probity are brought into question. OAs can be an important factor in minimizing the Agency's exposure to unacceptable levels of risk, so important in meeting the challenges inherent in the Agency's new programming modalities.
Improving CIDA's development cooperation programming hinges on organizational learning. The effective management of knowledge gained from the Agency's OAs contributes to the development of new intellectual capital both in Canada and in recipient countries.
Why OAs Are Carried Out
What triggers an OA? OAs are planned and carried out in response to management's needs. They may be addressed in branch-level workplans, or initiated on an ad hoc basis.
Most often, consideration is prompted by one of two decision-points:
How is the decision made to proceed? Decisions are reached once management has a clear and precise understanding of the specifics involved, the parameters for implementation, and the value-added to be gained. The final decision likely rests with the Responsibility Centre Manager.
Senior branch management is responsible for bringing a strategic approach to the planning of all development cooperation programs/projects. More formal, complex OAs may be scheduled by the CIDA's Executive Committee in accordance with corporate priorities (as part of the Agency's corporate planning cycle).
Who Does What
For more formal and extensive OAs, the common practice is to contract out assessments to an individual, firm or organization. Contracting out eliminates the corporate costs associated with maintaining a permanent, in-house capacity within CIDA, and promotes creativity/innovation by bringing in external expertise.
Typically, the consultant is responsible for day-to-day management of activitiesand the preparation of deliverables. The CIDA Program Manager represents the Agency, oversees and monitors progress, and is responsible for the conduct and delivery of the OA. In some cases, the decision may be taken to build an OA team comprising several experts from different organizations, or an advisory committee may be formed to provide overall direction for larger, more complex initiatives (e.g. multi-donor assessments).
The following table provides a step-by-step process for carrying out a formal OA. Planning and design are initiated after CIDA management makes the decision to proceed. Terms of reference (TORs) are prepared to establish broad parameters defining what is to be done—and by whom. OAs are operationalized with the Agency's approval of the consultant's workplan. Assessments may or may not include missions to recipient countries.
When less formal OAs are conducted, all these steps may not be necessary. However, all assessments should respect the need to provide a level of credible information that is adequate to effectively inform decision-making and optimize learning.
As no organization is the same—and each functions within its own distinctive external environment—it follows that every OA is unique. The Universalia/IDRC OA framework provides a 'common' approach for a wide-range of applications that, in practice, has proven both reliable and successful.
|Preparation of Terms of Reference||TORs provide the first substantive overview for the OA, articulating management's initial requirements/expectations. CIDA Program Managers are responsible for preparing TORs, and senior branch management is responsible for approvals.|
|Consultant Selection||CIDA Program Managers are responsible for selecting the qualified candidate who demonstrates best value. Senior branch management oversees the selection process and is accountable for compliance with contracting requirements and authorities.|
|Preparation of Workplan||Workplans refine/elaborate on the information put forwardin TORs to provide more precise and detailed guidance toOAs. Consultants are responsible for preparing work plans.CIDA Program Managers oversee preparation, provide adviceand approve final plans.|
|Information Collection and Analysis||Consultants carry out assessments to identify findings, results and lessons learned. Sources of information can include file reviews, consultations, site observations and focus groups sessions. CIDA Program Managers monitor progress and inform senior branch management.|
|Preparation of Report||OA reports clearly distill and articulate findings, results and lessons. Consultants are responsible for report preparation. CIDA Program Managers oversee production, and advise senior management on developments. Senior branch management is responsible for approving the final report.|
|Next Steps||Senior branch management is responsible for ensuring findings, results and lessons are implemented and shared (as appropriate). In this way, OAs contribute to informed development.|
Note: In some cases, a CIDA manager (or staff member) may be made responsible forcarrying out the OA (without a Consultant being retained). That individual would then be responsible for preparing the workplan and the OA report in addition to the management and oversight activities identified above. In the following chapters, guidance speaks to the person assigned to carry out the task-at-hand.
You now have the 'green light' to carry out an OA. What's next? The first step is to bring some scope and focus to the task ahead. Subsequent steps in planning and design will bring elaboration, precision and refinement to why the OA is being done, how it is to be done, who is to do what, and when it is to be done.
This chapter stresses the importance of taking a strategic, results-based approach to framing OAs. Typically, the CIDA Program Manager is responsible for preparing the TOR, and overseeing consultant selection. The consultant develops the OA workplan for approval. Not everything set out in this chapter would have application for all OAs. Simpler assessments, of course, would entail a less involved process.
Think the 'big picture'. A good OA is measured not only by what is learned about the organization but also by how the findings, results and lessons were arrived at. In the end, success will be determined by the contribution made to informed decision-making and learning.
Initially, it may also be important to bring some perspective to several issues that may have to be addressed. How will expectations for the meaningful participation of stakeholders and the organization itself be covered off? Are there implications relating to gender equality and environmental sustainability? How will they be addressed?
TORs offer the first substantive overview of the OA. In effect, they frame the assessment, conceptualizing the exercise and identifying broad parameters for implementation. Management's initial expectations for the assessment are articulated, in turn laying the foundations for the next steps in the OA process -selecting the consultant and preparing the workplan. TORs guide assessments until workplans take over to guide execution and provide primary project control.
TORs are expected to: 1) profile the investment being assessed, 2) identify reasons for the OA, 3) establish scope and focus, 4) determine accountabilities and responsibilities, and 5) set out the process to be followed (with deliverables). They also identify consultant qualifications, set scheduling and time frames, and put forward an internal cost projection for conducting the OA.
TORs are usually prepared in close collaboration with keys stakeholders, including the organization being assessed. Typically, CIDA Project Managers prepare TORs, and senior branch management is responsible for approvals.
How do you prepare a TOR? Perhaps the best way to start is to go to Appendix A which sets out 'model text' for a fictitious assessment that meets the Agency's essential requirements. This example addresses whether or not an organization should be a candidate for continued core funding over a three or five-year period.You may also want to refer to Appendix B which provides a menu of sample questions for shaping your OA.
CIDA Program Managers can work from this standard and adapt the information to reflect their requirements. After completing a first draft, work through the table on the next page to ensure that your TOR meets expectations. This table suggests an outline for preparing TORs and elaborates on important reporting elements. The 'rating' checklist found later on in this section may be useful for determining if your TOR does the job.
|1 Title||Short, descriptive (good acronym)|
|2 Organization Profile||Mandate, history, operational framework, targeted beneficiaries, reach, funding, results achieved|
|3 Broad Considerations||Global, regional, national context, development impacting external environment|
|4 Reasons for OA||Primary rationale, risk management, learning opportunity, other value-added|
|5 Scope and Focus||Scope indicates broad issues, focus comprises questions central to these issues|
|6 Stakeholder Participation||Mapping of participation by beneficiaries, ministries in recipient countries, other donors, partners, other stakeholders|
|7 Accountabilities and Responsibilities||Delineating between roles of CIDA Program Manager and the Consultant|
|8 OA Process||Broad indication of how assessment to be carried out, workplan and OA report requirements, field mission if any (adequate detail to inform the workplan)|
|9 Deliverables||Timeframes for workplan and OA report delivery, on-going progress reporting|
|10 Consultant Qualifications||Experience, expertise, language capacities|
|11 Internal Cost Projection||Projected level of effort (number of days), anticipated 'consultant-related cost'(remains confidential to the Agency)|
Rating your TOR
Are you satisfied with your TOR? Does it meet expectations? Has managing for results, continuous learning and knowledge building been adequately addressed?
Reflect on the following questions to make sure your TOR measures up:
Does your TOR...
The selection of a competent, capable consultant is critical to success. No amount of direction and/or control from CIDA will salvage an OA if the consultant selected cannot perform at expected levels (for whatever reason). It is critical that sufficient time and effort is expended to ensure an appropriate choice.
Selecting the 'Right' Consultant
The challenge is to identify a fully qualified candidate who offers the best value to CIDA. The engagement of a consultant essentially involves four steps: 1) deciding on the sourcing option (e.g. open competition, standing offer, rosters of local professionals), 2) selecting best candidate from potential suppliers, 3) notifying the successful candidate; 4) negotiating and signing the contract. CIDA is committed to ensuing transparency, fairness and equality in its selection processes. The Agency benefits when opportunities are made available to a wide range of potential suppliers.
How should consultant qualifications be determined? Requirements for expertise, experience and abilities must respond to expectations for the work to be performed, and the deliverables to be produced. Consideration may be given to: 1) technical, analytical and sectoral expertise, 2) previous experience in conducting OAs (demonstrated capacity for delivery), 3) knowledge of thematic issues, 4) in-country, regional and 'like' experiences, and 5) language skills. Experience in conducting OAs embracing stakeholder participation should also be addressed.
There may be advantages in engaging a team of individuals. When selecting teams members, leadership and team dynamics should be at the forefront of decision-making. There is no understating the importance of team chemistry, particularly when operations are being carried in challenging working environments. The assessment team will be expected to function smoothly with authority and conviction, consistently demonstrating a singular outlook and purpose.
It is essential that the successful candidate does not enter into in a conflict of interest position with the awarding of contract. There should be no previous orintended involvement with the initiative being assessed, or any other connection that would be perceived as a conflict of interest.
Rating your Selection
Is the selected consultant or study team capable of doing a good job? Are there any issues that should be addressed before the contract is awarded?
Reflecting on the following questions may help to make you more comfortable and confident with your selection:
Does the selected consultant...
CIDA's Program Manager selects the successful candidate and makes a recommendation to the Responsibility Centre Manager for approval. Senior branch management oversees the selection process and is accountable for compliance with contracting requirements and authorities.
A contract is then negotiated to reach agreement on the value of contract, method of payment, and other terms and conditions. Once signed, the consultant initiates preparation of the OA workplan.
Once approved by CIDA, the workplan becomes the key management document for controlling the OA and guiding delivery in accordance with expectations. Inpreparing workplans, consultants are expected to build on and refine what was put forward in the TOR, adding elaboration/precision and ensuring practicability.
Consultants are expected to perform a thorough review of relevant information sources to bring a fully informed perspective to workplan preparation. Sources may include: 1) TORs, 2) country, program, project-level documents, 3) literature, 4) the Agency's Corporate Memory System, and 5) consultations with CIDA personnel, key stakeholders and others having relevant knowledge. Requirements to consult with the organization itself will also have to be addressed.
When workplans are being developed, consultants are expected to keep the CIDAP rogram Manager apprised of progress and developments. Moreover, CIDA's Program Manager and the consultant should strive to develop a good working relationship during OA planning, establishing a dialogue that leads to effective interpersonal communications throughout the life of the project. It is important that both parties surface from the planning process with a clear and single understanding of how the work is to be performed, who is to do what, what is to be produced, and when deliverables are expected. The value of involving the organization itself (and key stakeholders) during workplan development should not be understated.
How long should workplans be? Generally, the level of detail should be adequate to effectively inform and control the assessment. Consultants should endeavor to keep workplans clear, concise and precise in meeting this objective.
During planning, thinking strategically upfront may pay dividends downstream. To this end, an upfront session with organizational stakeholders might help to de-mystify the process.
It may also be helpful for the CIDA Program Manager and the consultant alike to reflect on the following guidelines:
At the core of each workplan is the methodology to be followed - and more particularly the OA framework matrix. The methodology provides a logical model for assessing the organization that responds to the key issues. The OA framework matrix systematizes what is to be assessed and how this assessment is to be carried out. The challenge is to develop an approach that best achieves this objective given the information available, and what is practical within the imposed time, resource and cost constraints.
The workplan should describe in some detail the methods selected for information collection. Your assessment may include: 1) project sampling, 2) consultations with the organization itself, ministry officials, beneficiaries, civil society, NGOs and partner organizations, 3) expert opinion from other donor agencies and the maticexperts, 4) interviews with CIDA managers/staff, Heads of Mission and DFAIT officials, 5) site visits, 6) case studies, and 7) surveys.
It is important to remain strategic and to select sources that will best inform the assessment. New sources of information may be identified during implementation, often in carrying out in-country missions. The workplan should remain receptive to this eventuality.
Information analysis techniques translate raw information into a meaningful and valid response to the assessment issues. Analytical techniques may include:1) statistical analysis, 2) non-statistical analysis, and 3) projecting longer-term outcomes and impacts. Cost and time constraints will limit what information can realistically be collected.
To facilitate workplan preparation, Appendix D provides 'model text' for a fictitious assessment that meets the Agency's requirements. Consultants can work through this example, making adaptations to reflect their requirements.
Reminder: Appendix B provides a menu of sample questions for developing key issues.
The following two figures may also be helpful. The first suggests an outline to guide workplan preparation, while the checklist ensures your workplan address esexpectations from a broader perspective. These tables are designed to cover OAs relating to both funding approvals and capacity building. Not all the requirements will be applicable in all cases— less formal OAs will likely entail a more abbreviated approach.
|1 Introduction||1) OA purpose,
2) Organization profile: mandate, mission, history, programming overview: strategies, operational framework, targeted beneficiaries, linkages,
3) Performance record: results achieved to date, reach of programming,
4) CIDA/donor funding,
5) Key audiences
|2 Objectives||Key issues (developed from TOR rationale, scope, focus), also address risk management, exploiting learning opportunities, other value-added (as appropriate)|
|3 Methodology||1) Approach (organization/stakeholder participation),
2) OA framework,
3) File review,
4) Project sampling,
6) Information analysis,
|4 Reporting Requirements||Foreword
Executive Summary (abstract)
1 Introduction: OA objectives, organization overview, methodology, study team members, report organization
2 External Environment: administrative, political, social/cultural, technological, economic, stakeholder
3 Organizational Performance: effectiveness, efficiency, relevance, financial viability
4 Organizational Motivation: history, mission, culture, incentives/rewards
5 Organizational Capacity: strategic leadership, humanresources, financial management, program management, process management, inter-organizational linkages
8 Lessons Learned
9 Future Directions
Appendices: TOR, consultations, documents reviewed, etc.
|5 OAManagement||1) OA team,
2) Responsibilities and accountabilities: CIDAProgram Manager, consultant, organization, governmentministries, other donors,
3) Work schedule,
4) Effort analysis
|Appendices||I Terms of Reference
II OA Framework
III Bios for OA Team
IV Proposed Field Mission Itinerary
Are you satisfied with your workplan? Does it meet expectations to implement and report on the OA as per management's expectations?
Reflect on the following questions to see how your workplan measures up:
Does your workplan...
The OA workplan should be submitted as a draft by the consultant to the CIDA Program Manager (and likely to the organization itself). As well, requirements to provide for the participation of key stakeholders in workplan approval are addressed.
CIDA's Responsibility Centre Manager has to approve the workplan before the OA is operationalized. Other approvals by the organization/stakeholders may also be required. If applicable, the post is notified of what is expected so that adequate preparations can be made for the field mission. The final workplan should also be shared with the organization and key stakeholders.
As the approved workplan governs the rest of the OA, any major, downstream deviations or alterations to the strategyfor carrying out the assessment are reflected in revisions that are subsequently approved by CIDAmanagement.
Now we are at the 'heart' of the exercise - executing the OA workplan. How do you actually go about getting sound, accurate information to develop useful findings, results and lessons that respond to stated objectives? Typically this will be a collaborative effort with the CIDA Program Manager overseeing, advising and supporting the consultant in carrying out information collection and analysis.
Undoubtedly, the greatest challenges are faced during implementation. Your approach will require strategic oversight, a reasoned approach, likely somepatience, and a healthy measure of good judgment. Any number of diversions and/or complications may occur, often more so in the field. They may be attributed to individual agendas, turf wars, misunderstanding and/or fear brought on by the prospect of change. How you manage the 'human dimension' will become a critical factor in determining the outcome of your efforts. It is very important that obstacles are not allowed to detract from the task-at-hand and the development of meaningful results.
Your OA may constitute a desk assessment only—or it may call for an information-gathering mission to the recipient country or countries. Field missions provide the opportunity for site observations and face-to-face, in-country interviews.
In this chapter, we provide a strategic outlook to information collection and analysis that: 1) stresses the importance of being prepared, 2) identifies challenges that may arise, and 3) offers strategies for overcoming obstacles and realizing expectations. This presentation assumes a field mission component.
Field missions need to be carefully planned and executed in order to achieve optimal benefit. They should be viewed as a valuable opportunity for enriching your OA. What you expect to achieve should be clearly articulated. While the OA workplan links key issues to information sources, fine-tuning information collection activities at this time may be advisable. Reflect on how stakeholders in recipient countries will respond to the mission—and formulate strategies for dealing with any issues that may surface. Preparing a field mission itinerary will help to translate your objectives into action. Field missions are costly—make sure CIDA's investment is well-spent.
Thinking ahead, the following guidelines may helpful:
During field missions, it is important to have access to key documentation. Limit what you bring along as you will have limited time for review in the field. To lessen the load, rely on electronic copies and/or executive summaries.
Going through the following checklist may help you to feel more confident before heading out on a field mission:
Interview protocols can be helpful in promoting a structured approach to information collection. In turn, data that is uniform facilitates the development of findings, results and lessons. To this end, interview guides and data recording worksheets may be assistance. Often when collecting information—particularly in the field—you only get that [']one' opportunity for access.
The following interviewing tips are suggested:
OAs are hugely invasive, particularly when they target how an organization can be improved. They bring into debate the direction of the organization, its performance and even how individuals contribute. They challenge everyone in the organization - from leadership on down. How the human dimension is managed will be critical to what is achieved. Being able to understanding the human face attached to OAs is a vital management responsibility, both for the leadership of the organization and the OA practitioner.
At a fundamental level, OAs imply change. Many see any form of change as threatening. How people respond and cope when organizations are being transformed will be determinants in how successful the outcome will be. Change must be managed and, to this end, communications are very important. Every effort needs to be to ensure that all members of the organizations understand that an OA is an opportunity for enhancing productivity and sustainability. Leadership needs to consistently demonstrate support for the OA and pro-change champions within the organization should be engaged and nurtured.
The following provides guidelines for functioning effectively in the field:
Guidelines for functioning effectively in the field
Document sources can be internal (annual reports, program-planning documents, strategic plans, etc.) or external (country policies, media, etc.). Or data can be obtained through people, either directly through conversation or indirectly using questionnaires. Observation can also serve as a valuable source of information. For example, consultants may visit a project site or spend time at organization headquarters to develop a greater understanding of day-to-day operations.
OAs are evidence-based exercises. While judgment and intuition should play a role in managing implementation, when formulating findings, results and lessons the consultant should limit tendencies for interpretation. Information collection is about determining the facts. Analyzing information is about determining their implications in responding to the OA issues. In the OA report, we have, in effect, an epiphany of sorts - that moment when recommendations and future courses of action are put forward made based on all that is learned. How best to respond to the OA key issues? What will be your main sources of information? The OA workplan links the issues to be addressed with 'generic' data sources. Fine-tuning your approach to be more specific will likely pay dividends, particularly for more complex assessments. It may be helpful to be more precise in identifying key sources that can provide you with the information you will require to respond to the key OA issues.
Typical Information Sources
Key documents Partner organization and/or project files, handbooks, mission statements, annual reports, resource documents, project reports, contextual studies, needs assessments, beneficiary impact studies, etc.
Observations Meetings, staff interactions and behaviors, procedures and processes in action, physical infrastructure, etc.
Start to develop your findings, results and lessons while the data is still fresh in your mind. This should be done during field missions, as undoubtedly you will face distractions when you return to your office.
The level of detail in your records should be adequate to inform and support your findings, results and lessons. Your primary record of consultations will likely be your notes. Key documents should also be copied and retained for easy reference.
Using work tools to organize what is learned in carrying out information collection activities may be of assistance. Information should be categorized by key issue (and where appropriate by sub-question). In this regard, the following template maybe helpful:
Data collection in practice will no doubt be a far cry from what you envisaged when you drew up your neatly laid-out OA workplan. However, it seldom actually takes place in an orderly, sequential manner. Likely complications should be counted on. For example, some interviewees may be traveling and unavailable to meet when you are in the country. Others may simply refuse to be interviewed. Important documentation may be missing or lost.
What to do in such circumstances? The answer lies in demonstrating leadership, flexibility and resourcefulness. Data collection is a human, organic process. It is important to recognize and understand that information collection can rarely be completed as planned and is never perfect. Imperfect yes, but like trial by jury the best approach devised so far.
Analyzing information is about focusing on the key issues, distilling what you have learned from your information collection activities and developing accurate, credible and useful findings, results and lessons that respond to the Agency's needs for informed decision-making and organizational learning. Think about how your work can best contribute to improved programming and appropriate partner choices. Ideally, the information that you have collected and analyzed will result in recommendations being implemented as appropriate, and key audiences sharing in and benefiting from your results.
It is important to take the broadest view of the value offered by the information collected, yet be strategic in determining what information will be presented in the OA report. Every effort should be made to reduce bias, error and misinterpretation in presenting the facts. Double check contradictory evidence, give more weight to reliable sources, and ensure significant information is not ignored. Credible results are derived from multiple information sources.
A group discussion involving all members of the OA study team (and others as appropriate) can be a very useful technique for data analysis. This provides participants with a forum to discuss/analyze the information collected and then formulate findings, results and lessons that they agree on. To assist, it may be helpful to prepare a list of key issues and indicators, and arrange the data according to this framework before the meeting.
In the following sections, we suggest goals for assessing organizational performance, external environment, organizational motivation and organizational capacity.
Organizations are located in countries and regions to which they are inextricably linked. Their external environment is shaped by a number of key contexts: 1) stakeholder, 2) political, 3) social, 4) cultural, 5) economic, 6) technological, and 7) administrative/legal. Although organizations have very few abilities to change their external environment, the better you understand the external context, the better you can adapt to it and develop appropriate strategies. The ability of an organization to function within its external context is directly linked to organizational performance.
The performance of organizations is made visible through the activities they conduct to achieve their mission. Organizations perform well when they successfully meet their purpose. Outputs and their effects are the most observable aspects of organizational performance. Two important questions are: 1) How does the organization define good performance?, and 2) Does good performance help the organization attain its mission? Key indicators of organizational performance are effectiveness, efficiency, relevance and financial viability.
Organizations, like people, have different rhythms and personalities. Each organization has a unique working ambience or climate that is an amalgam of a purpose, history and personality. It represents the complex array of beliefs, values and norms that guides organizational life now and in the future. The main four elements of organizational motivation are: history, mission, culture and incentivesor rewards. By gathering this type of information, your goal is to understand the underlying dynamics of the organization—the extent to which organization members are motivated to work towards organizational goals and aspirations.
The capacities of an organization are the existing and potential abilities to perform. The capacities of an organization can be defined by a series of interrelated areas that support organizational performance. Key foci include: 1) strategic leadership, 2) organizational structure, 3) human resources, 4) financial management, 5) infrastructure, 6) program management, 7) process management, and 8) inter-organizational linkages. Knowing the functional capacities of an organization will provide a larger indication of organizational performance.
CIDA's Framework of Results and Key Success Factors defines what constitutes achievement at the Agency and establishes the foundation for a consistent body of information on development cooperation activities. When conducting OAs, it is critical that information analysis and ultimately the OA report reflect a results-based orientation. The following 'helpful tips' are organized around the developmental results and key success factors set out in this Framework:
Down to the crunch. Your OA report will determine what action is taken as a result of your assessment. The decision may be taken to proceed with funding for an institution or its programming. Or it may lead to investments in organizational strengthening. Or recommendations may be considered and overturned. Dissemination to key audiences will contribute to knowledge building. In essence, the OA report represents the enduring value of your contribution—and the Agency's immediate and continuing return on its investment.
The primary objective of OA reports is to inform CIDA decision-making and organizational learning. The report should articulate a comprehensive response to the expectations set out in the TOR and refined in the OA workplan. The reader should be left with a thorough understanding of why the OA was carried out, what was done, what was found, what was learned for future application, and what is recommended.
Consultants are asked to fairly and objectively identify useful and credible findings, results and lessons. Presentation should follow a credible progression in logic, with a basis in fact that ensues from the information collected.
While CIDA does not prescribe a standard format for OA reports, this chapter provides guidelines for their preparation and identifies what should be addressed. Flexibility is encouraged to promote a final product that is most conducive to effective presentation.
CIDA management, having made an investment in the OA you have carried out, expects to learn what it needs to know. Remember your report is not an end in itself - it serves an ongoing purpose and process. Keeping the organization and key stakeholders informed during preparation may be advisable.
Your task is to you prepare an OA report that clearly and succinctly brings forward key information supported by the evidence that: 1) responds to the assessment issues (questions), 2) is geared towards informed decision-making by senior management (do we invest or not), and 3) extends the value of the assessment by contributing to progressive learning. What is brought forward should eliminate other explanations to determine causal inferences. Consultants are expected to rely on assumption, logical argument and/or empirical analysis in reaching this goal.
Good OA reports are accurate in distilling what is learned from OAs, and are skillful in being strategically informative. They communicate clearly, are factual and balanced, and avoid judgments of individuals. A well-written report is more likely to be read thoroughly, and therefore understood and acted on.
Keep in mind that clarity and succinctness can lead to higher readership as few of us have the time to read lengthy reports. The level of detail should effectively inform key audiences about what was learned, and what is recommended.
Before starting to write the report, the consultant should consult with CIDA's Program Manager (and the organization if appropriate) to discuss the structure, contents and timing for the report to develop a mutual understanding about expectations for this deliverable.
The following tips may help with report preparation:
The first priority of OA reports is to say what should be said. Contents should reflect the four dimensions of performance - external environment, organizational performance, organizational motivation and organizational capacity. Going forward, the report should provide recommendations and lessons.
Consultants are asked to look at all that has been learned during the course of the assessment in making a fair, objective and accurate assessment. It is crucial that what is presented provides adequate coverage, and that what is brought forward is fully supported by the evidence. A conceptual framework or logic model may be useful for providing systematic coverage and a balanced dissertation that supports the findings, results and lessons reached. The OA framework provides an excellent starting point for organizing what was learned, and aligning what is to be presented with the key issues.
Reports can go to audiences that are often diverse. The difficulty for the author is to write a report that can be used and understood by different audiences. A good report speaks directly to its primary audience, but should also respect the potential for a wider dissemination. There may be advantages to showing a preliminary draft to the organization, for it gives those directly involved a chance to correct incomplete or incorrect data before it becomes public. It also begins the process of softening the blow that a negative report will create. This strategy likely will hold more for assessments addressing organizational development than funding approvals.
Assessments can produce 'report shock', a highly-emotional reaction when first read by the leaders of an organization. Your critical analysis may depict the organization in a way that is perceptually different from what is imagined. 'Report shock' needs to be managed or it can destroy the utility of an assessment report. Too many negatives may call for rewording if the report is to receive a fair hearing.
Typically, the value and complexity of the OA will determine the length of the report. Strategic audiences may be hard-pressed to find the time to review extensive texts. There is little point to preparing a voluminous report that gathers dust on someone's shelf. Make your presentation significant and to the point.
The following sections suggest how your OA report could be organized and what could be addressed. Not all the reporting elements identified will, of course, be applicable to every OA. The information presented will largely be a function of the characteristics of the organization you are assessing and the key issues set out in the OA workplan. We suggest a results-based orientation to your report that emphasizes contribution to informed decision-making and organizational learning throughout.
Explain what the OA set out to do. Acknowledge contributions by key individuals.
The executive summary provides a concise synopsis of the OA report, addressing all substantive elements. A quick short read should impart a general understanding of what the organization is, what it does, how well it does it, and what it could do in the future to improve.
Only the most significant: 1) findings (affirmations based on the information collected), 2) conclusions, 3) recommendations, and 4) lessons should be highlighted. Executive summaries are used to inform both senior management and the Agency's corporate memory system.
In preparing executive summaries, often the best plan is work through the main body of the report compressing the information down to a succinct presentation of not more than eight pages (even for the most complex OAs). There may be value in organizing the executive summary as a shorter, mirror image of the longer main body. Excessive detail should be avoided and tone should remain consistent throughout the report.
Readership of the executive summary is typically higher and more influential than the rest of the report. Executive summaries are used to inform both senior Agency management and CIDA's Corporate Memory System.
Introduce the reader to both the OA and the organization. The introduction shoul dbriefly describe what your report is about and what was done.
A logical sequence for presentation would be:
For the more complex OAs, you may wish to provide separate sections in the report to address Agency programming interventions, organization profile and OA methodology.
Present your findings by responding to the key issues. The reader should be able to link the findings with the evidence gathered, with references being made to identifiable information sources. 'Real life' examples will add credibility and richness to your report (in turn promoting readership). This section is typically the longest of the report.
It may be advisable to organize your findings along the four factors that comprise the OA framework (as appropriate): 1) external environment: (e.g. administrative, political/social/cultural, technological, economic, stakeholder findings), 2) organizational performance (e.g. effectiveness, efficiency, relevance, financial viability findings), 3) organizational motivation (e.g. history, mission, culture, incentives/rewards findings), 4) organizational capacity (e.g. strategic leadership, human resources, financial management, program management, process management, inter-organizational linkage findings).
Organizations operate within an external environment that influences and shape show the organization functions. This section provides a brief review of the context for operations, identifies the resident issues faced and, where applicable, describes sectors of involvement. Moreover, it determines how the external environment either promotes or detracts from organizational performance - a key element is deciding if the Agency should invest in an organization and/or its programming.
How does the organization interface with its external environment? Are the forces at play supportive of what the organization is intent on accomplishing? Are there negatives that need to be addressed? Are any significant changes foreseeable that could significantly impact the organization's external environment?
At a more specific level, there may be value in identifying consistency with needs and priorities of targeted beneficiaries, local partners, government, and with the efforts of local organizations and donors addressing the same needs or problems.You may also want to determine the extent of local ownership of programming activities and the degree of 'buy-in' supporting these activities. The organization's congruence with domestic policies, and the institutional, national, international environments should be addressed.
It may helpful to explain what was learned about the organization's external environment by listing key findings that in turn are supported by the evidence. Findings may be categorized as relating either to: 1) the rules of the game (legal and regulatory frameworks, etc.), 2) institutional ethos (history, culture, etc.), or 3) capacity (resources, access to technology, etc.).
Ask yourself: What key elements were learned about organizational performance that could be a factor in deciding if CIDA should commit financially? How effective and efficient is the organization in moving toward the fulfillment of its mission? Are targeted objectives and results being achieved? To what extent has the organization adapted its operations and programming to meet the needs of its changing environment? Will it be able to sustain operations and activities over time?
Ideas about the concept of performance vary considerably. Each stakeholder may have an entirely different view about what counts. For instance, administrators might define performance in terms of the amount of money brought into the organization through grants. A donor, on the other hand, might see performance as being benefits realized by targeted groups. University leaders likely would point to educational contributions, research results and services to the community.
The following table suggests how findings on organizational performance could be organized. In this case, we are assessing the Azrae Institute (AI), a fictitious organization dedicated to good governance and building civil society in developing countries. The intent here is to give you 'an idea' as to what may be important.
|Effectiveness||The absence of a coherent performance measurement framework makes it difficult to assess organizational effectiveness.
There is little evidence, if any, that AI has been instrumental in influencing national policies (but a longer time frame may be required).
Efforts to build civil society indicate mixed results, but outputs and outcomes at local levels are encouraging.
|Efficiency||AI's contribution is not being adequately communicated to external clients and partners.
Cost-efficiency is not being achieved in AI's field operations.
The checks and balances between AI's Board and management do not ensure judicious investment decisions.
|Ongoing Relevance||How stakeholder support is managed going forward will determine if AI's strategic ambitions are realized.|
|Financial Viability||Historically, AI has demonstrated a considerable capacity for financial survival.
Implementing a formal revenue generation strategy will become critical with increased competition for donor funding in the future.
Organizations, like people, have different missions and agendas. A variety of factors may dictate internal motivation. Some organizations may be shaped by a widespread commitment 'to do good'. Others may be centrally driven by the personal ambitions of key leaders. Internal motivation - like an organization's capacity and its external environment - are key contextual factors that help to drive and determine performance.
The organizational concepts that drive your organization may include its history, its mission, its internal culture, its incentives or rewards and the widespread values and beliefs about the role your organization plays in society. Taken together, these factors help to give an organization it personality and direction - and affect its performance and quality of work.
Ask yourself: 1) How do motivational factors affect organizational performance?, and 2) Is the organization committed to ideals, strategies, values and practices that are congruent with those espoused by CIDA?
In responding to the first question, we suggest reflecting on the following:
CIDA's Framework of Results and Key Success Factors provides useful prompts for determining congruence with the Agency's mandate and programming objectives. To this end, assess consistency with: 1) CIDA's poverty reduction and sustainable development policies, and other policies, Branch priorities and programs, and 2) Canadian foreign policy.
Here you are to report on the organization's ability to use its resources to perform. Knowing the capacity of an organization is crucial to the making of informed investment decisions. Sustainability can hinge on anticipating and overcoming the greatest barriers to organizational growth. Your assessment should review both current organizational capacity and adequacy going forward.
For OA reports, it is suggested that focii be brought to determining the strengths and weaknesses of the inter-related factors described below. Again, this presentation is designed only to illustrate how information could be organized. Your report would provide substantiation for your findings. The weighting of factors should be relative to CIDA's information needs.
|Strategic Leadership||AI's new strategic plan is needs-driven and pragmatic.
Opportunities to reinforce the organization's visionamongst key audiences are not being exploited.
|Financial Management||Financial management systems are being upgraded to respond to the organization's anticipated growth cycle.
Plans are in place to improve the AI's annual and multi-year financial planning and reporting systems.
|Organizational Structure||AI's governance structure is not adequately articulated.
Responsibilities, accountabilities and performanceexpectations are not communicated to AI's managers and staff.
|Organizational Infrastructure||AI's technological resources will not be able to keep pace with the organization's evolution.
Our assessment of the strategic management of AI'scapital assets indicated major deficiencies.
|Human Resources||HR management were actively involved in thedevelopment of the strategic plan.
HR organizational planning is not being informed by local levels.
The implications of moving from project-driven to program-driven organization are not being addressed.
|Program Delivery||Project planning demonstrates a sound understanding of local context, needs and priorities. Risk management is not addressed.
Tasking is not always understood, supervised and/oraccomplished.
|Process Management||Decision-making by management is often not documented.
Change management is not being addressed as a vitalmanagement responsibility.
|Linkages/Networking||Management routinely exploits opportunities to develop and strengthen partner relationships.
Networking efforts by staff are paying dividends.
Again, CIDA's Framework of Results and Key Success Factors may provide useful prompts for determining operational capacity. Refer to the success factors (partnership, appropriateness of design, appropriateness of resource utilization, informed and timely actions).
The conclusion summarizes what was learned about the organization in responding to the key OA issues. The information presented should be significant and strategic to the needs of the Agency's decision-makers. It is also important to be conclusive (for example): 1) 'Our assessment indicates that this organization would be a viable candidate for core funding', or 2) 'Engaging this organization as a CIDA partner is not recommended due to concerns about its financial viability'.
The following sequencing is suggested:
Recommendations should be framed so they facilitate decision-making, are easily understood and limit any potential for misinterpretation. This is best accomplished when messaging is kept simple. Recommendations should be succinct and head-on. Targeting too much information can be unmanageable and counterproductive. Not every tidbit of information needs to be presented to know what is going on. Be practicable—respectful of what is doable—given resourcing constraints. Limiting the number of recommendations to reflect only key considerations may promote wider acceptance and value.
Recommendations should be listed individually and explained, referring to the information collected and supported by the evidence. What is put forward should be evidence-based and accurate, and not betrayed, in any way, by bias, sentiment or orthodoxy. They should be prescriptive, identifying who should be responsible for taking what action (i.e. The organization should, CIDAshould).
Lessons are general hypotheses based on the conclusions of a specific assessment that establishes or supports a general principle and is presumed to have the potential of being beneficial in other applications. In formulating lessons, consultants are expected to develop a perspective that goes beyond the subject assessment, and apply their expertise/experience to extrapolate what is learned for general application. The objective is to bring value-added to the Agency (and potentially to the international development community at large). Consultants are encouraged to limit the number of lessons put forward to those that have the greatest potential for useful, generic application.
Lessons generally are of two types:
In this section, the consultant responds to specific requirements, if any, set out in the OA workplan relating to the challenges, opportunities, prospects, etc. of the organization going forward. Or she/he may have to provide commentary on future actions suggested for CIDA for the purposes of strengthening infrastructure, investing in niche opportunities, etc.
Typically, appendices are used to amplify, illustrate or embellish your presentation of information, but are not essential to the reader's understanding of the main body. Appended information doesn't interrupt the flow of your presentation and/or the concentration of the reader. Moreover, appendices allow for the inclusion of detailed information without disrupting the 'balance' of your report.
Depending on the complexity of your assessment, you may decide to append the TOR and/or the OA workplan. Other appendices may include: 1) a list of interviewees, 2) documents reviewed, 3) bios for assessment team members, and 4) a bibliography of references (reports, publications). When appendices are particularly extensive or highly technical, they can be bound in separate volumes.
Now's time to put yourself in the place of the decision-makers who are responsible for using what you've learned about the organization to decide on a future course of action, or not. Reflect on the following questions to determine whether the information needs of senior management will be met and opportunities for organizational learning addressed. See your report as a reader would. When conducting your review, compress lengthy narratives and eliminate superfluous issues and detail.
Does Your Report...
CIDA Program Managers may be asked to assess organizations within extremely limiting time constraints. This may involve recommending an implementing agency to carry out development activities. Or assessing an organization as a prospective CIDA-funded partner. Or finding an organization to replace one that didn't work out.
In this chapter, we set out a framework for conducting a Quick OA. We address how to think strategically in planning and designing your assessment, key elements to focus on during implementation, and what your OA report could look like. What level of effort is required? Quick OAs can be carried out within five days- two days for set-up, a two-day site visit and a day for report preparation. Tighter time constraints may dictate an abbreviated approach.
Given that timing is of the essence, it is critical that a pragmatic and efficient approach is adopted for all aspects of the OA. Planning strategically will allow you to organize and 'visualize' the assessment process from start to finish.
Think the 'big picture'. Develop a strategy that addresses how best can you assess the organization as a viable, reliable performer (within the time constraints) to ensure that any downstream investment by the Agency is not exposed to unacceptable levels of risk.
It is suggested that you work through the following five-point, set-up plan prior to visiting the organization:
The interview guide will help to ensure that questioning during your on-site visit is aligned with and responds to your information requirements. Going in, you should also identify any key documents that you might need to pick up.
Leading up to your visit, make arrangements to meet with a suitable spectrum of people. Provide as much notice as possible to interviewees in respecting theirother commitments and obligations.
Here we set out a listing of key elements of an organization's external environment, performance, motivation and capacity for consideration. In framing your OA, select only those factors that respond best to the key issues. Remember you can't do everything. Keep your approach simple and limit your foci - or you may not be able to complete your assessment within the time constraints.
|Understanding the External Environment||Administrative and legal framework||Policy, legislation regulations, laws|
|Social/cultural environment||Norms, values, beliefs, attitudes in society, literacy|
|Economic environment||GDP, inflation, growth, debt, IMFconditionality, wage/price structure, community economics, hard currency access, government funding distribution|
|External political environment||Form of government, distribution of power, access to government resources, allocation decisions, political will|
|Technological and ecological environments||Infrastructure, utilities, geography, technological literacy, information technology, climate|
|Major stakeholders||Clients, donors, beneficiaries, government ministries, other institutions|
|Measuring Organizational Performance||Effectiveness||Organization performance: major achievements, general level of organizational productivity defined according to the organization's mission and values, utilization of results
Staff performance: clients served, quality of services/products)
Service performance: support to research community, transfer technology
Staff productivity: turnover, absenteeism, outputs
Administrative system efficiency
|Identifying Organizational Motivation||History||Date and process of founding, major awards/achievements, major struggles, changes in size, program and leadership, other projects and loans through IFI's or funding agencies|
|Mission||Evolution of mission statement, organizational goals, role of mission in shaping the organization, giving it purpose and direction, articulating research/research products that are valued|
|Culture||Attitudes about working, attitudes about colleagues, clients or stakeholders, values and beliefs, underlying organizational norms that guide the organization|
|Incentive and reward system||Key factors, values, motivations to promote productivity, intellectual freedom, stimulation, autonomy, remuneration, grant access, opportunity for advancement, peer recognition, prestige|
|Determining Organizational Capacity||Strategic leadership||Leadership: managing culture, setting direction, supporting resource development, ensuring tasks are done
Strategic planning: scanning environment, developing tactics to attain objectives, goals, mission
Niche management: area of expertise, uniqueness, recognition of uniqueness
|Financial management||Financial planning: operating expenses, forecast future monetary needs and requirements
Financial accountability: rules for member use of financial resources, transparent/verified system
|Organization structure||Governance: legal framework, decision-making process, methods for setting direction, external links
Operational: roles and responsibilities, coordination of labour, coordinating systems
|Organizational infrastructure||Facilities management: adequate lighting, clean water, electricity
Technology management: equipment, information systems, hardware/software, library
|Human resources||Planning: recruiting, selecting, staffing, orienting
Developing: performance management, monitoring, evaluation
Career management: career development, training
Maintenance: health/safety issues, gender issues, quality of working life
|Program and service management||Planning: identifying needs, setting objectives, costing alternatives/developing evaluation systems
Implementing: adherence to schedules, coordination of activities
Monitoring: projects/programs, systems for evaluating progress, communicating feedback to stakeholders
|Process management||Problem-solving: defining problems, gathering data
Decision-making: creating alternatives, deciding on solutions, monitoring decisions
Communications: exchanging accurate/vital information, achieving shared understanding among organizational members
|Linkages and networking||Networks: type, nature, appropriate membership, utility, coordination, cost-benefit
Partnerships: type, nature, sustainability
Electronic linkages: communication networks, information equipment, information resources, people of all skills/backgrounds
Your four-point plan for collecting information when visiting an organization:
With accountability comes transparency, and the need to document decision-making. It is important to explain in writing both your recommended course of action and the reasoning behind your recommendation.
To this end, Appendix E suggests a reporting format for Quick OAs. This example provides text for an assessment of Valyun Kind, a fictitious organization that specializes in the building of entrepreneurial capacities in developing countries. Here we concentrate on providing a succinct, evidence-based response to senior management's critical information needs. Key issues are addressed through the external environment, organizational performance, organizational motivation and organizational capacity foci.
Until now, the value realized from your OA has been largely defined by what participants have learned through their involvement in the process. Now we look at how the value of the OA report going forward is determined. Unless actionis taken to implement the OA report, it can be relegated to a dusty shelf. How the report is applied will govern its contribution.
Next steps for OA reports are determined case-by-case. Typically, the Responsibility Centre reviews the approved report before deciding a course of action. The nature of the OA assessment and confidentiality/sensitivity issues will determine the extent of information sharing. Internal audiences may include senior management, the country desk, post-managers and staff. Key external audiences can comprise the organization itself, government ministries, executing agencies, local stakeholders, other donors, etc. In deciding how information is to be communicated, it is important to ask some fundamental questions: What audiences would benefit from this information? What interests cannot be compromised? How are key audiences best reached?
OAs are about investing in informed development. Success is measured in terms of making informed investment decisions, laying the foundations for institutional strengthening, and/or organizational learning. In this chapter, we explore 'next steps' for achieving these objectives.
OAs carried out to inform investment decisions may be single-purposed (should we invest), or the workplan may identify other applications, including a learning component. A Quick OA may be carried out due to limiting time constraints. Or a more complex approach may be adopted in order to respond to CIDA's 'need to know' requirements. Regardless, the primary objective is to position Agency decision-makers so they have timely, credible information that contributes to informed decision-making.
When the CIDA Program Manager goes forward with the OA report, she/he must be prepared to make the case in recommending a course of action. Messaging should be kept simple and conclusive. It is often best to open with what was determined (for example): 1) the level of risk associated with a SWaps arrangement with this organization is not considered to be acceptable due to foreseeable political unrest, etc., or 2) our assessment indicates that a contribution agreement with this organization would help to advance its mission and broaden effective programming in marginalized areas of the country. Supporting arguments and explanations should be evidence-based and accurate.
CIDA's response to the OA report should be routinely documented. If a decision is reached that overturns the report, the rationale for rejecting the recommendation (e.g. not going ahead with core funding) should be placed on file. If the OA was managed by an independent, third party, a formal management response should be required.
Organizations are key to development discourse, yet historically their value has often being overlooked. Increasingly, we are recognizing the need to invest in organizations in the developing world to strengthen capacity and improve performance. Institutional strengthening is about facilitating change within organizations that will help them fulfill their missions and make a larger contribution. It is about developing a course of action working together with the organization that promotes reaching its potential.
OAs identify functional capacities, and strengths and weaknesses. Logically, the next step would be strategic capacity building. In some cases, the OA workplans identify the next step to be taken. For example, the OA might be one phase in a larger study that embraces organizational development. Or, the workplan may state that the consultant is to recommend what is to be done next.
Effective knowledge management can contribute to the development of new intellectual capital both in Canada and internationally. CIDA is committed to developing a culture where critical analysis and organizational learning are systematically employed to make a valued contribution to development cooperation wherever possible. When information is shared openly, credibility is enhanced and greater pressure is generated for recommendations to be implemented.
Every OA offers learning potential. Yet deciding on what can be shared requires tact and judgment. Not all information can or should be shared. It is critical that communication strategies function in the best interests of the Agency and demonstrate total respect for the organization itself.
How is information communicated? Oral briefings can represent the best option for communicating results. They typically bring together people with a shared interest in the assessment, and provide a fertile opportunity for discussion, feedback and generating 'buy-in' into the recommendations. Busy decision-makers are more inclined to attend a briefing session than to sit and read a lengthy report. Conducting workshops for the organization itself allows participants to benefit from what was learned.
Other options for organizational learning include:
The 'model text' below offers a quick and easy way to prepare TORs. Practitioners can work from a standard for a fictitious assessment that meets the Agency's essential requirements. This assessment addresses whether or not an organization should be a candidate for continued core funding over a three or five-year period.
Founded in 1992, the efforts of the GrahMar Women's Health Foundation are directed towards promoting the accessibility to health services that are responsive to women's needs and priorities at national/regional/community levels, and ensuring that gender equality is integrated into health and nutrition programming. The Foundation focuses on enhancing the capacities of key health providers, conducting awareness campaigns against tuberculosis, polio and malaria, and empowering women to advocate for 'health for all'. Although health services are considered above average for the region, delivery is hampered shortages of well-trained medical personnel, inadequate facilities in some rural communities and the need for systems and technical capability to improve sector management.
The strategy for allocations is largely iterative, to achieve maximum results and accommodate changing dynamics. Disbursements encourage complementary and cumulative actions to advance women's interests and rights across complex heal thissues. Women are the main participants and beneficiaries.
Project planning is carried out in conjunction with stakeholders both government ministries and civil society. Project funds provide a quick and flexible response to local requests and priorities. The average project cost is approximately $30,000.
Allocations for project funding are routinely discussed with CIDA's resident Head of Aid.
At present, CIDA is the sole international donor agency contributing financially to the GrahMar Women's Health Foundation, having provided $3.2 million over a five-year period extending from 2002 to 2006. Allocations were $0.8M for 2006; and $0.6M in each of the years 2002, 2003, 2004 and 2005. Over the five-year period some 100 projects were assisted through Agency funding.
Indications are that results achieved by the Foundation have been in-line with expected results. A mid-term evaluation of investments carried out in the Summer of 2003 conducted by Performance and Knowledge Management Branch found the Foundation had: 1) made significant interventions in health that were very responsive to compelling needs and clearly aligned with government and local priorities, and 2) contributed to cost-effective interventions to combat malaria and tuberculosis. Programming by Foundation-supported partners delivered access to reproductive health care services to approximately 25 percent of the rural female population. The Foundation also contributed to the construction of a number of clinics and health care centers. Targeting high rates of infant/maternal mortality has produced mixed results. The evaluation noted that achieving gender equality outcomes and impacts are a long-term process.
The Millennium Development Goals (MDGs) included a series of specific health-related targets that were agreed to by the world community. In support, CIDA's Action Plan on Health and Nutrition (APHN) issued in 2001 provided for investments totalling $1.2 billion over five years that doubled spending on basic health, nutrition and water/sanitation (from $152M in 2000 to $305M in 2005). Measuring progress to date, the World Health Organization report Health and the Millennium Development Goals (2005) indicated that without urgent investments in health systems, current rates of progress will not be sufficient to meet most MDGs.
The World Health Organization (WHO) stresses that locally-run services are often more efficient and more responsive to the needs of the poor. WHO attaches equal importance to country ownership and leadership, underscoring the need to strengthen government institutions and management structures.
A recent review carried out for CIDA to assess health programming for the years 1995-2005 determined that embracing local 'ownership' that is responsive to country needs/priorities and ensuring strong institutional capacity (led by skilled management) are key foundations for effective programming and long-term sustainability.
This assessment of the GrahMar Women's Health Foundation is being carried out to help determine if continued core funding for a further three or five-year period is advisable (current funding provisions terminate December 15, 2006). This was discussed at the Gender Fund Team meeting in January 2006, and the decision was taken shortly thereafter to proceed with an organizational assessment (OA).
To ensure that continued funding is justifiable and advisable, decision-making will be informed in a number of key areas. For example, the assessment will determine if the Foundation has the mandate and support, reputation, organizational capacity and resources needed to achieve targeted results going forward. Also, any possible risks associated with investing in this partner and areas for future collaboration with this partner will be identified. Value added from this assessment will result from the sharing of what is learned from this investment, leading to more efficient and effective allocation strategies for downstream investments.
The OA will be founded on the premise that performance going forward will be a function of the Foundation's enabling environment, functional capacity and organizational motivation.
Sub-components for each of these factors may include (for consideration):
Refer to the CIDA Organization Assessment Guide for elaboration on the Agency's approach to OAs.
Focus will be defined by the following key questions:
The OA will include an examination of a sampling of projects carried out within the last three years to identify key findings, results and lessons learned.
The Consultant will assess the level of risk associated with CIDA of providing continued funding to the Foundation. For example: What is the risk that funding will not be used for its intended purpose?, What is the risk that targeted programming will not be implemented as planned due to constraints (e.g. limited capacities, insufficient funding)?, What is the risk that achievements attained may not be sustainable?
The OA will exploit every opportunity to optimize the learning potential offered by this exercise - for the benefit of the Foundation and the Agency, and more widely for the international cooperation community at large.
Foundation representatives will be involved throughout the OA and consulted at important milestones during the process. It is intended that all decisions from the selection of the consultant through to finalization of the OA report will be made in full consultation with the Foundation. Early on, consultations will clarify the commitments, responsibilities and expectations of CIDA, the Foundation and the Consultant. Both CIDA and the Foundation will aporve the OA workplan. The final report will be provided to the Foundation as a draft for comments.
Stakeholder participation is fundamental to this assessment. The OA will provide for the active and meaningful involvement of key stakeholders as considered appropriate (e.g. beneficiaries, representatives of ministries of health at national, regional and local levels, NGOs, civil society.
The CIDA Program Manager will oversee the OA and be responsible for accountability and guidance throughout all phases of execution, and approval of all deliverables. The Canadian Consultant will be team leader and have overall responsibility for: 1) the day-to-day management of operations, 2) regular progress reporting to CIDA, 3) collecting credible, valid information, 4) the development of findings, results and lessons, and, 5) the production of deliverables in accordance with contractual requirements. The team leader will report to the CIDA Program Manager.
The OA will be carried out in conformity with the principles, standards and practices set out in the "CIDA Organization Assessment Guide".
7.1 Preparation of Workplan
The Consultant will prepare a workplan that, once approved by the CIDA's Program Manager, will serve as the agreement between parties on how the OA will be carried out. The workplan will refine and elaborate on the information presented in this TOR to bring greater precision to the planning and design of the assessment. The workplan will address the following reporting elements:
7.2 Field Mission
The Consultant will conduct a field mission to include a visit to Foundation headquarters and several project sites. Consultations will be carried out with CIDA field personnel and project stakeholders. Information will be collected as stipulated in the workplan. The mission is expected to be no longer than three weeks in duration. CIDA field personnel are to be briefed by the Consultant on arrival and before departure from the field.
7.3 Preparation of OA Report
The Consultant will prepare an OA report that describes the assessment and puts forward findings, results and lessons learned. The presentation of results is to be intrinsically linked to the key issues, establishing a flow of logic development derived from the information collected. Results are to be linked to CIDA's "Framework of Results and Key Success Factors".
The Consultant will prepare: 1) a workplan, and 2) an OA report in accordance with requirements identified in the CIDA Organization Assessment Guide. These deliverables are to be prepared in English only, and submitted in both hard copy and electronic (pdf.doc) formats.
The Consultant is to submit a draft workplan to the CIDA Program Manager and Foundation representative within four weeks of the signing of the contract. Within one week of receiving comments, the Consultant will produce a final workplan.
8.2 OA Report
The Consultant is to submit a draft OA report to the CIDA Program Manager and the Foundation representative for review within four weeks of returning from mission.Within two weeks of receiving comments, the Consultant will submit a final OA report (including an executive summary).
A Canadian Consultant will lead the OA. The OA will be carried out by a team of two senior consultants, with one individual being a resident national.
The Canadian consultant is expected to be:
The local consultant should have a good working knowledge of health issues locally, be fluent in English and local languages, and have experience with donor-funded health programming.
The basis for payment and payment scheduling will be determined during contract negotiations. Options for method of payment include: 1) fixed-price, or 2) cost plus on a fixed per diem basis.CIDA's projections for the 'level of effort' required for this OA and the anticipated'consultant-related costs' for carrying out this project are set out overleaf:
|Activity||Number of Days|
|Data collection/field work/travel time||19||15|
|Debriefing, analysis, report preparation||13||3|
|Type of Cost||Cost|
|Travel and other out-of-pocket expenses||$10,000||$2,500||$12,500|
1. Canadian professional per diem of $800. Local professional per diem of $400.
2. Costs are exclusive of GST.
Question: Does the stakeholder environment support the organization?
Question: How is the organization affected by political and governance issues in the country?
Question: How is the organization affected by the social/cultural environment?
Question: How is the organization affected by the economic environment?
Question: Are the technology and resources needed to carry out the organization's work available?
Question: How is the organization affected by existing rules, regulations and legal requirements?
Question: How is the organization affected by ecological and environmental challenges?
Question: How effective is the organization in achieving its objectives, commitments and targeted results?
Question: How efficient is the organization?
Question: Is the organization relevant and will its relevance be maintained over time?
Question: Is the organization financially viable?
Question: What are the memorable milestones, successes and/or crises in the organization's history?
Question: To what extent does a mission and vision drive the behaviour of the organization and its members?
Question: What aspects of the organization's culture contribute to the mission execution?
Question: Does the incentive/reward system encourage or discourage the performance of the organization's members?
Question: To what extent does strategic leadership affect the organization's performance?
Question: To what extent does strategic planning affect the organization's ability to achieve its goals?
Question: Is the organizational structure facilitating or hindering movement towards the mission and goals?
Question: To what extent does governance affect the organization's performance?
Question: To what extent does the organization's ability to plan for its humanresources needs affect its performance?
Question: To what extent does the organization have effective human resources relations?
Question: Is there adequate financial planning to support performance?
Question: Are financial systems appropriate to support performance?
Question: Is infrastructure adequate to support performance?
Question: To what extent do technological resources affect the organization's performance?
Question: Is program planning adequate?
Question: To what extent does the organization implement its programming appropriately?
Question: To what extent does the organization monitor its program and services appropriately?
Question: Are there problem-solving and decision-making processes supporting the organization's capacity to carry out its functions?
Question: Are communications effective in supporting performance?
Question: Are monitoring and evaluation linked to improved performance?
Question: Are external linkages adequately established or pursued to support performance?
What progress is being made toward achievement of results at the output, outcome and impact levels? Do these results contribute to the Agency's overall goals of poverty reduction and sustainable development, and/or to efforts to support democratic development and economic liberalization in Central and Eastern Europe?
Is the relationship between costs and results reasonable?
Does the initiative make sense in terms of the conditions, needs or problems to which it is intended to respond?
Will results/benefits continue after CIDA's involvement ends?
Is there shared responsibility and accountability for results?
Is the design appropriate and based on sound understanding of local context? Were risks identified and assessed and strategies developed for ongoing monitoring? How were innovative and creative ideas and approaches explored to achieve results?
Are suitable human, financial and physical resources involved and used well? Is financial information complete, accurate, and reliable? Are prudence and probity adequately exercised?
Did we anticipate and respond to change based on adequate information? Did we take appropriate action to manage risks?
In early 2006, the Institute's multi-donor support group agreed to invest institutional strengthening as a means to promote growth and meet the challenges ahead. It was decided that CIDA would lead this initiative, working jointly with the organization's leadership, the national government and bilateral donors.
To this end, a two-phase process is being implemented. First, an organizationassessment (OA) will identify strengths/weaknesses, and areas for potential improvements. The second phase will focus on institutional strengthening and the development of a strategic plan going forward. The OA is to be completed by August1, 2006, and the strategic plan is targeted for October 15, 2006.
The Grewnal Governance Institute was formed in 1995 by interested benefactors(and supported by fledgling civil society organizations) with the dismantling of government control after well over 45 years of socialist ideology. Democraticelections in 1997 saw the end of one-party rule. Since, development has been characterized by significant progress in achieving deep and fundamental structural reforms, the emergence of a vibrant and inclusive multi-party political system, and a continuing economic struggle that impacts most greatly on the country's most vulnerable people—women, and the poor and disadvantaged.
The mandate of the Grewnal Governance Institute is '...to develop an enablingenvironment for all citizens to participate freely and effectively in the establishment and delivery of good governance nationally, regionally and locally'. After 1997, the Institute contributed significantly to the country's democratization and political liberalization reforms. Recently, Institute programming has focused on promoting political rights, civil liberties, rule of law and ethics in government. Interventions have contributed to the strengthening of the functional capacities and improving the organizational performance of many NGOs and community-based organizations, establishing ethical standards within government ministries, and increasing media coverage of human rights and legal processes. Governance projects tended to focus on national level initiatives at the outset, but now training/education projects in particular are being implemented at regional and local levels.
Grewnal managers/staff work together with civil society organizations and a network of donor agencies to further the Institute's objectives. A respectful, cordial and productive relationship is maintained with the national government (albeit with some contentious issues).
Canada implemented bilateral development cooperation programming shortly after the first democratic elections were held. Opportunities were identified for CIDA to work with the recipient government and other donors in building the foundations to facilitate transition during this challenging period. The Agency functioned as a 'niche' donor within the overall development cooperation framework, supporting constitutional development, and the strengthening of government and civil society institutions at all levels.
From 2001 to 2005, CIDA contributed a total $12.5M in core funding to the GrewnalGovernance Institute, averaging $2.5M a year. Disbursements progressively increased from $0.9M in 2001, to $1.2M in 2002, $2.9M in 2003, $3.2M in 2004 and $4.3M in 2005. Assistance from other donors amounted to $11.7M in 2005.
Primary clients are: 1) Grewnal Governance Institute, 2) Executive Director, National Treasury (responsible for coordinating development cooperation), 3) CIDA senior management (desk, country), and 4) bilateral donor support group. Information will be shared with other external audiences as appropriate.
2 OA Objectives
The primary objective is to identify the Institute's strengths/weaknesses, and areas for potential improvements. An emphasis is to be brought to the development of clear, credible information that facilitates informed decision-making. The following key issues are to be addressed:
CIDA expects to optimize value-added from its investment in this assessment, and to this end expects learning opportunities to be fully exploited and key lessons learned brought forward for consideration.
Note:Terms of Reference can be attached.
The OA is to be carried out as a collaborative partnership between CIDA and the Grewnal Governance Institute. Both entities have worked together to develop theTerms of Reference and select the consultant responsible for carrying out the assessment. This collaboration will continue in the finalizing of the OA report and the sharing of information. Throughout, this assessment will emphasize the participation of all key stakeholders, and the mutual sharing of experiences at all levels.
The methodology adopted for this OA is designed to meet the requirements and expectations set out in CIDA's Terms of Reference. Information collection will focus on developing a better understanding of the performance of the Institute—and the factors that drive performance. Organizational performance is perceived as a function of the Institute's external environment, its motivation (underlying traits that define its 'personality'), and its ability to use internal capacities to achieve results.
The OA will comprise:
he Evaluation Team will carry out a three week, in-country mission in April 2006.
Note: The proposed field mission itinerary can be attached.
3.1 Framework Matrix
The OA framework attached as Appendix A systemizes the methodology, identifying the key issues to be addressed, sub-questions to provide elaboration, matters to be considered, sources of information and methods of information collection. The OA framework addresses historical performance, as well as forward-looking issues that relate to future directions.
The process of identifying and reviewing available documentation began with the awarding of the contract and facilitated preparation of this workplan. To date, an emphasis was brought to understanding and documenting the evolving political context from 1997 on. Research has also been carried out on the Institute's mission, policies, processes, and systems. More detailed information will be collected on the selected projects during the in-country mission. This will include reviewing project files made available by other donors.
The assessment of projects will be based on a representative sample germane to the 2001-to-2005 period. The Grewnal Governance Institute was actively involved in the selection process, and with CIDA jointly approved the final listing.
Project selection was based on: 1) strategic nature/importance (within the overall programming portfolio), 2) financial significance, and 3) the potential for learning and identifying lessons. The following seven projects will be assessed, representing a total donor investment of $22.4M in the period from 2001-to-2005 (27 percent of total):
All available project documentation will be reviewed for each of these projects.Then a series of interviews will be carried out with the organization staff and in-country project staff, beneficiaries and involved stakeholders (as appropriate). Interviews will focus on each project individually. In total, more than 30 interviews are planned for the mission.
In-country site visits will be carried out to provide opportunities to observe projects that are still ongoing, collect 'on the ground' information about results and carry outin-depth consultations with project implementers and beneficiaries. If possible, field level discussions may also comprise mini-workshops with implementers and beneficiaries.
Note: Interview guides can be attached.
3.2 Key Informant Interviews
Key informant interviews will be conducted to obtain qualitative information on the OA issues. These interviews will provide in-depth information that will allow the OA Team to assess the Institute's success in fulfilling its mandate, contributions to capacity building, support amongst stakeholders, strengths and weaknesses, sustainability over time, etc.
In total, some 30 interviews will be conducted with: 1) government officials involved in ODA and/or with thematic-specific experience, 2) experienced representatives from other donor agencies, 3) thematic experts, and 4) businesses and civil society organizations, and 5) CIDA managers from the program branches (both HQ, post).
Note: Interview guides can be attached.
3.3 Information Analysis
Information analysis will be results-oriented in responding to the key issues. CIDA's Framework of Results and Key Success Factors will help guide presentation. For example, the sustainability assessment will consider: 1) local ownership of program/project activities, 2) sufficiency of resources to maintain programming, 3) adequate institutional capacity, and 4) the degree of support in the external environment. Relevance will comment on congruency with Canada's and CIDA's development cooperation mandate, policies and strategies. Aninformation analysis tool will be prepared to record organizational strengths and weaknesses by key issue.
The OA Team will meet to refine the preliminary findings and develop conclusions, recommendations and lessons learned. These will be communicated to and discussed with CIDA's Program Manager. Preliminary indications will then be discussed with the organization's leadership, post-staff, other donors and keys takeholders at the end of the field mission.
Foreseeable limitations of the OA methodology are identified as being:
4 Reporting Requirements
The consultant will: 1) provide regular progress reporting to CIDA's Program Manager, and 2) keep her/him informed of any developments and/or issues that require immediate attention without delay. A draft OA report will be submitted to CIDA's Program Manager and the Grewnal Governance Institute within three weeks of returning from mission. Within two weeks of receiving comments on the draft report, the consultant will finalize the report and submit ten hard copies to both CIDA's Program Manager and the Institute.
The final report will be prepared in English only, with the executive summary being made available in both official languages. The executive summary in both French and English will be prepared as pdf.docs (for loading on CIDA's 'Entrenous' and publicly accessible Web sites).
A preliminary draft outline for the OA report follows:
5 OA Project Management
The following sections address the make-up of the OA Team, the accountabilities and responsibilities of key players, a projected effort analysis (person-days required), and the work schedule for taking the OA to completion.
5.1 OA Team
The OA Team reports to CIDA's Program Manager who is ultimately accountable for delivery. The team comprises two Canadian consultants (one leader, one senior), are presentative of the Grewnal Governance Institute, and an observer from the national government. The involvement of local beneficiaries, and other donors and stakeholders during the field trip will augment local 'content'.
Note: Bios for each OA team member can be attached.
5.2 Accountabilities and Responsibilities
|CIDA Program Manager||
|Representative, Grewnal Governance Institute||Representing the Institute||
|Organization Leader (consultant)||
5.3 Work Scheduling
|Activities and Deliverables||Time Frames
|Pre-Mission (May 1-15)|
|Strategic planning (preparation of workplan)||May 1-14|
|Finalization of field trip logistics||May 15|
|Phase I: Field Mission (May 23-June 9)|
|Briefing sessions||May 23-24|
|Site visits and project interviews||May 25-31|
|Key informant interviews||June 1-7|
|Debriefing sessions||June 8-9|
|Phase II: OA Report (June 12-July 7)|
|Preparation of first draft||June 12-23|
|Editing and delivery of final report||June 23-July 7|
5.4 Projected Level of Effort
|Tasks/Deliverables||Number of Person-Days|
|Pre-Mission (May 1-15)|
|Strategic planning (preparation of workplan)||7||5||12|
|Finalization of field trip logistics||1||1||2|
|Phase I: Field Mission (May 23-June 9)|
|Site visits and project interviews||7||5||12|
|Key informant interviews||7||5||12|
|Phase II: OA Report (June 12-July 7)|
|Preparation of first draft||9||7||16|
|Editing and delivery of final report||3||2||5|
|Key Issues||Sub-Questions||To Be Considered||Sources of nformation||Information Collection|
|To what extent has the Institute been successful in fulfilling its mandate?||Has the enabling environment for good governance been improved?||Democratic reforms
Degree of political liberalization
Extent of citizen participation
|Qualitative, statistical analysis|
|To what extent has the Institute contributed to improvements in the capacity of:1) government institutions to deliver good governance, and 2) civil society to effect real changes on governance issues? Identify unintended results, if any, attributable to the organization (both positive and negative).||What results are attributable to the Institute's capacity building interventions?||Changes in organizational capacities(government, civil society)
Ability of civil society organizations to influence government ideology, strategies, policies
|To what extent is the stakeholder environment supportive of what the organization is intent on accomplishing?||To what extent are the community and partners involved in the organization? Does the government value and support the organization's efforts? Are donors supportive?||Alignment of interests, needs, priorities
Degree of information sharing with communities and partners
Financial support from government, donors
|What major strengths contribute to the organization's ability to fulfill its mandate and achieve targeted results?||To what extent does strategic leadership affect performance? Are human resource levels needs adequate to meet performance expectations? Are the individuals in key jobs capable and well suited to the challenges ahead?||Leadership strengths and weaknesses
Human resource planning capacities Resident expertise and experience
|What key improvements should be introduced to improve Grewnal's structural organization, processes and systems going forward? What other deficiencies should be addressed?||Is the organizational structure conducive to growth and productivity? Are there adequate financial systems? To what extent do technology resources affect performance?||Alignment of mission/goals with structures Lines of accountability and responsibility
Adequacy of systems/processes
Adequacy of technological planning
|What opportunities will influence the Institute's growth over the next three-to-five years? Comment on sustainability and financial viability overthis period.||Has strategic planning identified needs and priorities? Is current programming expected to be sustainable in the near to medium term? Will Grewnal's revenue generation strategy be adequate going forward?||Policy commitments Targeted results at national, regional, local levels
Stakeholder/financial support for programming Government, donor funding levels
|Qualitative, statistical analysis|
|Under what conditions, if any, should CIDA and the donor group approve core funding at this time?||What foreseeable events could impact on Grewnal's capacity for delivery? What factors are key to ensure acceptable levels of risk management?||Political developments
Global and national economic trends
Civil stability (national,regional, local)
Donor support levels
Objective: To identify the strengths/weaknesses of this organization for the purposes of determining candidacyfor: 1) core funding for entrepreneurial-focussed programming, and/or 2) organization developmentthrough CIDA infrastructure strengthening programming.
|External environment||Adequate||On balance, the stakeholder environment is supportive of Valyun Kind (VK) and the work that it does. The government values VK programming and projects appear well-aligned with national and local priorities. Attempts to coordinate with similar efforts by donor agencies have generated mixed results. On occasion, government ministries have been critical of IFI's favouring VK.|
|Operational Performance||Exemplary||VK has achieved significant results that clearly responded to the needs of the local populations. Entrepreneurial training can be linked to close to 250 new business start-ups from 2002 to 2005. Interventions in microfinance improved accessibility and led to capacity building results. VK's role isexpected to grow appreciably with greater recognition of its contribution to private sector development. Revenue generation appears sound for the next three years. Investments in technological training are required at this time.|
|Operational Motivation||Exemplary||The human resources dynamic is inspired and energized. Managers and staff are supportive of VK's mission, and typically expend whatever effort is required to meet challenges and resolve issues.|
|Operational Capacity||Adequate||Leadership is focused on results, competent and organized. VK's organizational structure facilitates productivity. Strategic planning, human resources and financial management appear sound.|
This organization has earned its well-deserved reputation as an important contributor to private sector development.
Our assessment found no indications that VK operations were unsustainable. It is recommended that CIDA consider VK as a viable candidate for core funding over a three-to-five year time frame, with financial assistance being directed towards entrepreneurial-focused training within communities. In parallel, efforts to strengthen in-house technological capabilities should be initiated. Exposure to risk is considered to be at acceptable levels at this time (although unforeseen developments may have a significant, negative impact on this organization).