• New instruments. New instruments are being tried, including sector-wide approaches, the comprehensive development framework, poverty reduction strategy papers, poverty reduction and growth frameworks, medium-term expenditure frameworks, and the sustainable livelihoods approach. Some of these imply changes in the way development agencies work, requiring significant shifts in how evaluation is approached.
• New horizons. Aid delivery has been shifting from the project to the program to the country level. But there is also an array of complex issues that require a regional or global approach, including climate change, trade, migration, genetically modified organisms, HIV/AIDS, and avian flu.
Recently, the medium-term strategy of 2006–2008,7 the second in a series designed to flesh out the long-term strategic framework for shorter periods, adjusted operational priorities in response to evolving conditions. It identifies five priorities: (i) catalyzing investment, as a prerequisite for sustaining high growth, employment creation, and rising productivity; (ii) strengthening inclusiveness and the poverty-reducing impact of growth through rural infrastructure development and social development; (iii) promoting regional cooperation and integration as a means of reinforcing national-level poverty reduction strategies at the regional level; (iv) managing the environment to ensure that the pattern of economic growth is also environmentally sustainable; and (v) improving governance and preventing corruption, which is key to improving the development effectiveness and poverty-reducing impact of ADB’s operations. The second medium-term strategy also introduced certain adjustments in ADB’s business processes, including a more selective sector focus. Core sectors identified for future operations include road transport, energy, urban infrastructure, rural infrastructure, education, and the finance sector. The role of ADB in leveraging additional investment through financing partnerships and cofinancing was emphasized, as well as strengthening project implementation to achieve development results. Implementation is to be country focused, and it is operationalized through individual country partnership strategies. The rapidly changing development context requires continuing sharpening of methods, guidelines, and directions for operations evaluation, as well as new products and services.
The New Modus Operandi
Evaluation has changed with ADB. Early work focused on input–output relationships in projects, using economic analysis, but evolved to cover the entire results chain of inputs, outputs, outcomes, and impacts. The main unit of account shifted from the project to the country, informed by sector and thematic assessments, as well as by evaluations of ADB’s business processes. The full mix of lending and nonlending services that make up country assistance programs has now become the dominant preoccupation of evaluation, with priority attention to relevance, efficiency, efficacy, and sustainability. In parallel, feedback has grown from restricted circulation of evaluation reports to maximum transparency and active dissemination of findings and recommendations through the Internet. Institutionally, evaluation has gained recognition from the Board and ADB’s Management commitment to achieving development results. Indeed, more suggestions for evaluations are now received than OED can undertake with its current staff complement and budget.
Managing for development results is a prerequisite to improving the quality of assistance. It is one of the five key principles in the Paris Declaration on Aid Effectiveness of March 2005, which promotes development through country ownership, harmonization, alignment, results orientation, and mutual accountability. Multilateral and bilateral agencies are under pressure to demonstrate development effectiveness and to show impact on the ground. ADB is improving its effectiveness in reducing poverty and promoting development in the Asia and Pacific region by means of an action plan built on three pillars: (i) country capacity, (ii) ADB’s results orientation, and (iii) effective global partnerships. OED can contribute in several ways: It will review and evaluate progress toward achievement of managing for development results and ADB’s underlying culture and management style; it will assess how concepts of managing for development results feed into preparation of country partnership strategies and the project cycle with an eye to the needs of DMCs; and it will examine how ADB’s Management uses such concepts and relevant experience to guide decision making.
An important organizational change to increase the independence of OED took effect in January 2004.8 OED now reports directly to the Board of Directors through the Board’s Development Effectiveness Committee (DEC), instead of to the President. Other significant changes are that (i) the Board, rather than the President, appoints the director general of OED; and (ii) ADB Management’s role in evaluation changed from approving evaluation reports to responding to their conclusions.
Box 2: The Development Effectiveness Committee a
The DEC was established by ADB’s Board of Directors in December 2000 and initiated activities on behalf of the Board in 2001. It consists of not more than six members of the Board, who meet about 12 times a year. Its general mandate is to assist the Board to carry out its responsibility of ensuring that the programs and activities of ADB achieve development effectiveness. Such development effectiveness is assessed through ADB’s operations evaluation. For the purpose of the DEC’s work, “development effectiveness” is the measure of (i) whether ADB’s programs and activities in furtherance of its policy goals and objectives have resulted in the desired outcomes, and (ii) whether these programs and activities have made efficient use of ADB’s available resources. The DEC
• reviews the work program of OED;
• reviews selected evaluation reports and the actions taken by ADB on them;
• reports to the Board on high-priority operations evaluation issues, if any, that have a significant bearing on the relevance, efficiency, and effectiveness of ADB, and makes recommendations on such issues to the Board;
• monitors and reports to the Board on the implementation of decisions;
• reviews OED’s work program for preparation of project, program, and technical assistance completion reports; and
• reviews the Annual Evaluation Review and the Annual Report on Loan and Technical Assistance Portfolio Performance.
The DEC’s evolving key areas of concern are the following:
• How are the evaluation recommendations that ADB’s Management decides to accept turned into action so that targeted changes for improvement happen?
• How are actions and their impacts monitored?
• How can OED play a more strategic role? Specifically, how can the influence of independent evaluations on ADB’s long-term strategic thinking be increased?
• How can the DEC play a more strategic role and increase the value it adds to ADB’s learning process?
The specific steps that the DEC has taken to support OED include (i) suggesting topics of strategic interest for inclusion in OED’s work program, (ii) helping to get the timing right for policy studies, (iii) ensuring that country assistance program evaluations are discussed by the DEC before country partnership strategies are finalized, and (iv) initiating measures to strengthen ADB Management responses.
a Available: www.adb.org/bod/dec.asp
To ensure the independence and transparency of evaluation reports, a new approval, review, and disclosure process has been established. Since evaluation reports are final upon approval by the director general of OED, they are made publicly available immediately upon circulation to ADB’s Management and the Board. ADB’s Management is given a specific period in which to respond to lessons and recommendations. ADB Management responses and the DEC chair’s summaries of discussion are disclosed as they become available.9
The policy paper to enhance the independence and effectiveness of OED saw the need to formalize