Background Technical Document III: Note on Interview Evidence
- International Monetary Fund. Independent Evaluation Office
- Published Date:
- August 2010
1. This document provides an overview of interview evidence for the evaluation of IMF interactions with member countries. It summarizes the number and type of interviews conducted; sets out the methodological approach to the interviews; and discusses particular issues that arose in the context of the interviews with members, as well as staff working with them, in different country groups.
II. Interview Coverage
2. The evaluation team conducted interviews with country authorities, IMF staff and, for several countries, civil society organizations (CSOs) and representatives of partner organizations, such as the World Bank and donors. These interviews were focused primarily on 49 sample countries identified for closer study by the evaluation team (see paragraphs 7 and 8 below).
3. Interviews with authorities were conducted for almost all countries in the evaluation sample. These interviews were at the level of finance ministers, central bank governors, and/or senior officials. In some cases, interviews were conducted with former officials who had interacted with the IMF during the evaluation period. Supplementary material included written responses to interview questions received from several countries in the sample, for which there were also direct interviews in almost all cases. The evaluation also benefited from interviews conducted with authorities in the course of other recent evaluations—often in the context of country visits—in which issues related to interactions with the Fund had been discussed.
4. The interviews with staff covered all 49 countries1 in the sample and with current or former resident representatives assigned to 27 countries in the sample. In most cases, more than one member of staff per country was interviewed. For a number of PRGF-eligible countries, staff from the World Bank knowledgeable about Fund interactions with sample countries were also interviewed.
5. Also, as opportunities arose, interviews were conducted with officials or former officials from countries outside the sample, and with staff or former staff covering such countries. In addition, several interviewed staff who had worked on countries outside the sample shared their views on interactions with those countries. These additional sources of interview evidence are included in the evaluation’s interview database, and were taken into account by the evaluation team in preparing the main report and the companion country papers. However, no further research on those countries—such as follow-up examination of internal and/or published IMF documents and so on—was carried out.
6. Most interviews with civil society organizations took place in four advanced economies (Germany, Japan, Switzerland, and the United Kingdom) and three PRGF-eligible countries (Cameroon, Senegal, and Togo). These interviews provided an important perspective for the evaluation, especially with the respect to considering interactions outside government at different stages of development.2
7. As summarized in the main report, the evaluation considered three main country groups—advanced economies, emerging economies, and PRGF-eligible countries, subdividing the first two into “large” and “other” subgroups.3 The 49 countries in the sample are distributed across these groups as follows, and as shown in the table below: 4 are large advanced economies; 6 are other advanced economies; 9 are large emerging economies; 14 are other emerging economies; and 16 are PRGF-eligible countries.
|Large Advanced||Other Advanced||Large Emerging||Other Emerging||PRGF-Eligible|
|United Kingdom||Korea||Egypt||Botswana||Burkina Faso|
|United States||New Zealand||India||Bulgaria||Cameroon|
|Sweden||Indonesia||Costa Rica||Congo, Dem. Rep. of|
|St. Kitts and Nevis||Senegal|
|Trinidad and Tobago||Togo|
8. The countries in the evaluation sample were selected to facilitate the exploration of several themes. The evaluation team sought to capture the diversity of interactions with the Fund’s membership by achieving broad country representation across, inter alia, different regions, sizes, and stages of development. The evaluation team also aimed to reflect countries’ diversity with respect to (i) their past experience with economic and financial crises and their systemic importance; (ii) programs supported by use of Fund resources, technical assistance, Financial Sector Assessment Programs; and (iii) the hosting of resident representatives.
9. In several instances, countries of similar characteristics were included in the sample to provide an opportunity for exploring specific hypotheses. In one such example, four small states were included in the group of other emerging economies to enable a focused assessment of interactions with this category of countries, which have sometimes expressed concerns about being neglected by the Fund (and other international institutions). In another, two countries were included in the group of other advanced economies to enable an exploration of different members’ experiences with the same mission chief and country team.
III. Interview Methodology
10. Interviews were conducted on a confidential basis. Most took place face to face, but in a few cases, interviews were conducted by telephone or videoconference.
11. An initial round of interviews was guided by specific questions about interactions since 2001. The questions explored: what aspects of the interactions (with the IMF or with the country authorities, as appropriate) were the most successful and least successful; what the interviewee(s) had hoped to achieve through these interactions and what they thought their counterparts (the IMF or country authorities) were trying to achieve; whether there were ways in which the interactions (with the IMF or the country authorities, as appropriate) could be better managed; and to what extent authorities conveyed their views on the nature and efficacy of the interactions to the IMF, via what channel, and with what effect. The evidence from this initial round of interviews was instrumental in selecting and formulating the questions for the evaluation’s surveys of authorities, staff and civil society (see Background Technical Document I).
12. Additional questions were posed either in the same interview or subsequent interviews with the same or other country officials and/or staff members to follow up on issues that arose. Such follow-on questions and interviews typically pursued specific issues, beyond the basic line of questioning, about the effectiveness and management of IMF interactions, including diagnostic questions about key drivers of staff actions. In most cases, they involved the exploration of complaints by the authorities, but also examples of successful interactions highlighted by the authorities or of the evolution of interactions over time. They also involved the attempt by the evaluation team to look at the issues from different angles and take into account the views of different observers, including the authorities, staff, and others, such as donors and other partners. Other questions that were raised when appropriate included the role of Executive Directors and IMF management in particular relationships, the manner and extent of outreach, and the role of resident representatives.
13. The interview evidence provided more textured information about interactions and in many cases introduced nuances into the evidentiary base than would have been apparent from the survey and documentary evidence alone. It thus provided an important source for triangulation, so that the evaluation could juxtapose the views of interviewed authorities and staff with the relevant survey findings and internal documents, as well as dig deeper to understand information gleaned from survey evidence. This approach worked in both directions, with the interview evidence providing a vehicle for reality checking the survey and documentary evidence, and the survey and documentary evidence providing a vehicle for reality checking the interview evidence, thereby helping to guard against interview bias and possible misinterpretation of interview evidence.
IV. Country Groups
14. In carrying out the interviews across the different country groups, the evaluation team aimed to balance consistency of approach with customization to individual country conditions. In so doing, it utilized the basic set of questions described above as an initial framework for asking interviewees about interactions, allowing the ensuing discussion to follow the interests of the interviewees. In the event, and as discussed briefly below, the interviews on the advanced economies focused almost exclusively on surveillance issues, while those on the emerging economies covered technical assistance and programs as well as surveillance. In the interviews for the PRGF-eligible countries, the focus was on programs and technical assistance, with attention to surveillance typically surfacing as it interfaced with programs. A brief discussion of the modalities and content of the interviews follow.
15. Face-to-face interviews were conducted with authorities from all ten advanced economies in the evaluation sample. These interviews were mainly conducted during the IMF Spring and Annual Meetings in Washington in 2008–09, as well as in visits to some countries in late 2008 (after a visit to two countries at a preliminary stage of the evaluation in late 2007). The interviews with Fund staff who worked on the ten countries were carried out over the same time frame. Follow-up interviews were conducted when further information was needed. The evaluation team also conducted interviews with authorities from and staff working on advanced economies outside the sample countries as opportunities arose. Visits were made to Australia, Germany, Japan, Switzerland, and the United Kingdom, as well to U.S. officials. During these visits, the evaluation team met with a wider representative of authorities as well as other stakeholders, including, as mentioned earlier, with representatives from CSOs in Germany, Japan, Switzerland, and the United Kingdom.
16. The time frame of the interviews for the advanced economies affected their content, with the global financial crisis—both its unfolding and the Fund’s response—an important contextual factor underpinning many discussions. In interviews with authorities, recurring themes concerned questions about the international dimensions of Fund surveillance, especially with respect to timely projections and analysis of spillovers, including from the financial sector, and the provision of relevant cross-country analysis. For mission chiefs, a striking feature of the interviews was the pressure several said they felt from the authorities of some countries to give a positive assessment of their policies and prospects, and the lack of support they received—or thought they would receive—from management in case of disagreements with the authorities. Mixed messages emerged from the authorities and staff on outreach, with authorities suggesting that more might be done and staff indicating that they felt constrained by the authorities from so doing.
17. For the emerging economies, face-to-face interviews were conducted with current and/or former country officials of 21 of the 23 sample countries and with staff members working on all 23 countries. The authorities of the remaining sample countries (one large and one other emerging economy) sent written responses to the interview questions. Interviews were conducted with 27 mission chiefs and/or resident representatives working on the nine large emerging economies and 27 mission chiefs and/or resident representatives working on the 14 other emerging economies. The evaluation team also conducted interviews with authorities from and staff working on emerging economies outside the sample countries as opportunities arose. Interviews took place in Washington during the IMF Spring and Annual Meetings, during other visits by the authorities or staff to Washington, and by telephone, as well as by IEO staff during their visits to countries for other purposes. A member of the evaluation team visited Algeria, Kuwait, Lebanon, and Oman. Additional interviews with authorities of and staff were carried out by a consultant, an experienced former IMF staff member, contracted by the evaluation team to assess the role of resident representatives and management of them in interactions in five large emerging market economies. These additional interviews were conducted with current and former resident representatives, as well as mission chiefs and country officials who interacted with the resident representatives.4
18. As with the advanced economies, the time frame of the interviews with the emerging economies affected their content. Among interviewed authorities, a recurring theme was concern about surveillance, which many saw as adding little that was new, useful, or interesting. Officials also expressed much greater openness to the Fund’s technical assistance, conferences, seminars and other vehicles for acquiring knowledge. Fund “style” was also a recurring concern—including attitudinal issues, rigidities of analytic approaches, lack of appreciation of domestic political economy issues, and excessive turnover (especially problematic for other emerging economies, including small states). Interviewed mission chiefs and resident representatives reported challenges in engaging authorities in surveillance discussions. Although there was a negative legacy in some cases from earlier lending arrangements, staff noted that without a program, traction was much diminished. Several reported their concern that the particular attention to exchange rate policy issues, including in the wake of the 2007 Surveillance Decision, had been at the expense of other topics of interest and were seen as counterproductive by some authorities. More generally, staff were considering ways to be influential in a surveillance-only relationship. While they saw the potential benefits of greater public outreach, staff reported acute trade-offs between pressures from headquarters in this direction, and the adverse consequences for their relationship with the authorities, many of whom did not want the Fund taking anything but a low key approach to outreach.
19. For the PRGF-eligible countries, interviews were conducted with current and/or former officials of 15 out of 16 countries in the evaluation sample, in some cases by telephone and/or videoconference.5 As in the other country groups, interviews were also conducted with Fund staff—including resident representatives—who had worked on the evaluation sample countries during the evaluation period. Follow-up interviews were conducted when further information was needed, and according to the availability of country officials and staff. Such follow-up was extensive, given the large number of program countries in the sample and the evolving nature of the Fund’s engagement with them over the evaluation period. It also involved interviews with World Bank staff in many of these cases, especially those involving the HIPC Initiative and/or a Fund-supported program that was paralleled by a Bank-supported budget-support or other policy-based lending operation. Importantly, follow-up included country visits to Cameroon, Senegal, and Togo in March 2009. There the evaluation team met with a broad representation of current and former country officials (including retirees, who had had contact with Fund staff during the earlier part of the evaluation period), as well as with partners and stakeholders, including representatives of CSOs and market participants. These country visits enabled the IEO team to detail the evolution of IMF interactions with the authorities over the extended evaluation period from 2001–08, especially with respect to perceptions that there had been difficulties in the Fund’s style of interactions in the first half of the evaluation period and improvements in the second half and more recently. They also provided a basis for detailing the different country contexts in which staff conducted outreach activities. The team was able to hear first-hand the views of the authorities, civil society, partners, and the Fund’s resident representatives about the constraints to more and better outreach and the lessons learned for improving them.
20. Programs and related issues received the most attention in the interviews on PRGF-eligible countries, but technical assistance, surveillance, and outreach also were discussed. Among the authorities, there was a divide on the issue of program flexibility and its implications, but general agreement that the Fund had become more flexible over the course of the evaluation period. Most interviewees praised the Fund’s contribution to donor signaling (including support under the Policy Support Instrument, macro sign-off in the context of PRGFs, which donors used to inform their decisions about budget support, and/or assistance with the Paris Club) and technical assistance, although some expressed concerns about the strategy both for it and for taking into account country constraints more generally in the design of policies and programs. Interviewed staff also saw the need for a more country-focused approach, including for technical assistance, which some said had improved but still needed greater ownership by the authorities and the area departments. Many interviewed mission chiefs and resident representatives discussed outreach, and the concerns they felt about pressures to do more, but often without the necessary support from the authorities and headquarters (including resources and backing from senior staff) that would enable them to make progress.
DodsworthJohn2009“The Role of Fund Resident Representatives in the Large Emerging Economies” IEO Background Paper BP/09/07 (Washington: IEO).
Independent Evaluation Office of the International Monetary Fund (IEO)2004Report on the Evaluation of Poverty Reduction Strategy Papers (PRSPs) and the Poverty Reduction and Growth Facility (PRGF) (Washington).
Independent Evaluation Office of the International Monetary Fund (IEO)2007An Evaluation of the IMF and Aid to Sub-Saharan Africa (Washington).
Independent Evaluation Office of the International Monetary Fund (IEO)2008An IEO Evaluation of Structural Conditionality in IMF-Supported Programs (Washington).
ScholteJan Aart2009“IMF Interactions with Member Countries: The Civil Society Dimension” IEO Background Paper BP/09/08 (Washington: IEO).
In most cases, the staff interviewed were current or former mission chiefs. In two cases, however, only resident representatives were interviewed.
For purposes of this report, the “large advanced economies” are the 7 largest industrial countries (the “G-7”), while the “other advanced economies” are the 23 smaller advanced economies; the “large emerging economies” are the 19 largest such economies, while the “other emerging economies” are the 62 smaller emerging-market economies.
The evaluation also utilized the interview evidence base from three earlier IEO evaluations, which had involved case studies (including IEO country visits) in about half of the 16 countries in the evaluation sample, including the one for which there was not anew interview with country officials as part of the present evaluation. See IEO (2004, 2007, and 2008).