What Caused IMF’s Forecast Errors? Analysis from Political Economy Perspective
XiongAizong(熊爱宗)
Abstract: Providing high-quality economic forecasts is an important responsibility of the International Monetary Fund (IMF) in maintaining world financial and economic stability. However, errors are inevitable in IMF economic forecasts for its member countries. Based on forecast method and information, and political factor, this paper creates a political economics framework for analyzing the IMF’s forecast errors, and tests the effects of various factors on the IMF’s forecasts using the panel data analysis method. According to our findings, if a country receives IMF loans and shares a similar vote with the United States at the UN General Assembly, it will more likely receive an optimistic forecast by the IMF. Meanwhile, member countries’ data availability and IMF forecast errors for major economies may also affect forecast on a country. Therefore, this paper proposes recommendations on further improving the IMF’s forecast quality by creating more independent forecast procedures and enhancing forecast data quality and forecast accuracy.
Keywords: International Monetary Fund, surveillance, economic forecast, political
economy
JEL Classification Codes: F33; F53; F55
DOI:1 0.19602/j .chinaeconomist.2018.11.0619602/ j .chinaeconomist.2018.09.02
1. Introduction
Macroeconomic forecasting is an important aspect of the International Monetary Fund’s (IMF) surveillance activities. By forecasting global, regional, and member economies’ development trends, the IMF monitors world economic operations, identifies possible stability risks, proposes recommendations, and ensures world economic and financial stability. Accurate economic forecasting is of great importance both to the IMF and its member countries. For the IMF itself, precise economic forecasting is an important basis for providing policy support. This includes the provision of not only financial relief arrangements to member countries before or during an economic crisis, but policy recommendations during economic stability as well. For member countries, the IMF’s economic forecasting serves as an important reference for their planning and decision-making activities. In addition, the IMF’s forecast has an important influence on lenders and borrowers, rating agencies, the media, and the public at large. Therefore, the IMF’s economic forecast should be sound, evenhanded, and of high quality (IEO, 2014).
But, in reality, errors in the IMF’s economic forecasts on its member countries are inevitable. Figure 1 illustrates the errors of the IMF’s forecasts on real economic growth of 163 member countries from 2003 to 2015. As can be seen from the chart, such errors are manifested in overestimates (positive values) and underestimates (negative values). Overall, the IMF underestimated member countries’
growth in 2003-2007 and 2010, and overestimated their growth in 2008-2011 and 2015.
Errors derive from deviation of economic variables’ forecast values from real values. The following two methods may be applied in exploring what led to forecast errors, direct and indirect. First, the direct method seeks to identify the differences between the economic variable forecast behavioral equation and real variable operation equation, but this method is unable to precisely identify a variable’s real operation equation. Second, the indirect method seeks to identify factors that may cause the forecast behavioral equation to deviate from the real operation equation. This paper adopts the indirect method, and creates a political economy framework for analyzing the IMF’s forecast errors from three aspects: forecast method, forecast information, and political factor. Based on panel data analysis method, our conclusion is that the IMF’s forecast on a country is influenced by whether the country receives IMF loans, its political relationship with the IMF, its economic data availability, and the IMF’s forecast errors for major economies.
2. Literature Review
Providing high-quality economic forecasts is a comparative advantage of international economic organizations regarding information supply (Fratianni and Pattison, 1982). However, most studies find the existence of deviations in IMF economic forecasts. Judging by the literature, the IMF’s forecasts are believed to be influenced by IMF loans, data availability, initial conditions of forecast, political factors, and the IMF employees’ forecast capabilities (Genberg and Martinez, 2014).
First, most studies find that IMF forecast errors are correlated to whether a country receives IMF loans. Beach et al. (1999) finds that an increase of IMF loans by one billion Special Drawing Rightings ( SDRs) to countries in the Western Hemisphere increases forecast error by 0.17 percentage points. However, the IMF’s Independent Evaluation Office (IEO, 2014) notes that not all
loans influence IMF forecasts. Forecast errors are relatively significant only for countries that receive IMF loans through “exceptional access,” with most such errors being reduced or reversed at the first program review.
Second, the IMF forecast errors are correlated to data availability on whether a country adopts the IMF’s data publication standard. The IMF provides better forecasts on developed countries than on developing ones. The forecast data in loan programs for developed countries are more transparent and less volatile because the data are directly used as the basis of forecast (Artis, 1996). Tong (2004) analyzes whether and how data transparency standards affect macroeconomic forecasts. Based on macroeconomic growth quarterly forecast data of 16 countries from 1996 to 2003, this study finds that transparency standards are of great influence on the improvement of the IMF’s forecast accuracy. Mrkaic ( 2010) investigates how member countries’ participation in the IMF’s Data Standards Initiatives ( DSI) may influence the forecast quality of the World Economic Outlook. Results show that the World Economic Outlook’s forecasts on member countries that adopted the Special Data Dissemination Standards ( SDDS) were generally better than the forecasts on those that only adopted the General Data Dissemination Standards (GDDS) and those that did not participate in the DSI at all.
Third, IMF forecast errors are correlated to insufficient consideration of assumption of initial conditions and international variables. By analyzing the IMF’s macroeconomic forecasts on various regions, Takagi and Kucur (2006) find that unexpected changes in the monetary policies of major economies and oil prices significantly influence the IMF’s forecasts. Timmermann (2006) notes that errors in the forecast of U.S. GDP growth rates are significantly positively correlated with errors in the forecasts of GDP growth rates of most developed countries in the same years. Forecast accuracy is also related to the assumption of output gaps.
Fourth, IMF forecast errors are correlated to the impact of political factors. Aldenhoff (2007) notes that IMF economic forecasts on real economic growth, inflation, and unemployment rate are often distorted by political bias. Dreher et al. (2008) carry out a political economy analysis of the IMF’s forecast data on 157 countries during 1999-2005, and put forward a hypothesis on political strategy that points out that the governments of IMF member countries exert pressures on the IMF through public or concealed channels to cause the IMF to make optimistic economic forecasts on their countries. Empirical results reveal that countries with votes closer to those of the United States at the UN General Assembly receive lower forecasted inflation rates from the IMF.
Fifth, IMF forecast errors are correlated to IMF employees’ forecasting capabilities and experience. Genberg and Martinez (2014) note that IMF country representatives have considerable autonomy over the forecasts of the World Economic Outlook, and so it is particularly decisive what forecast methods they use and how capable and experienced they are.
Existing studies investigate various factors’ effects on the accuracy of IMF forecasts from different perspectives, either separately or collectively. Like the literature, this paper theoretically and empirically sheds light on what influences the IMF’s forecasts using a political economy framework. This paper identifies the sources of IMF forecast errors from three aspects ( forecast method and information, and political factors), and examines different factors’ effects on IMF forecast errors using panel data. Unlike the literature, this paper includes data availability and initial forecast conditions into the analytical framework. Regarding data availability, this paper examines both the IMF’s Special Data Dissemination Standards ( SDDS) and the World Bank’s Statistical Capability Indicator (SCI) to verify the effects of increased data availability on reducing forecast errors. Regarding initial forecast conditions, this paper investigates how IMF forecast errors for the United States, Germany and China are correlated with forecast errors for other countries, and thus verifies the expectation that the IMF’s forecasts on major economies will significantly influence forecasts on other countries.
3. Sources of IMF Forecast Errors: Analytical Framework
Referencing Atoian et al. (2004) and Sahin (2014), this paper assumes that the IMF’s behavioral equation of macroeconomic forecast is as follows:
According to equation (3), IMF forecast errors derive from the following sources:
First, forecast errors are derived from the difference between forecast model or method and real behavioral equation , which is related to deviations in the IMF’s forecast behaviors. In addition, the difference between forecast model and real behavioral equation caused by deviations in forecasting behaviors may be derived from various sources.
One source of errors is data availability problems. In some circumstances, data availability directly determines the forecast model or method employed by the IMF. IEO (2014) notes that data availability is the single most important factor that influences country representatives’ choice of forecast models. Due to poor data availability, statistical models such as structural ecomometric models, vector-autoregression, or reduced-form equations play a much smaller role in economic forecasting of low-income countries compared with adcanced countries. Given data restrictions and insufficient external forecast resources that can be referenced, the IMF has to rely on the assessments of country representatives in making forecasts on low-income countries (Genberg and Martinez, 2014).
Another source of errors is deviations in forecasting behaviors arising from the IMF’s responsibilities. Considering the IMF’s importance, its World Economic Outlook has an important influence on the world economy as a whole and various countries individually. The IMF’s forecast on the economic development conditions of various countries directly influences the market sentiments and economic performance of such countries. An important duty of the IMF is to promote global economic stability. For this reason, the IMF has a natural inclination to make optimistic forecasts to guide market expectations and promote world economic development. Such inclination is particularly obvious in an economic downturn, when boosting economic expectations can prevent downward risks and the selfrealization of a vicious cycle. IMF employees, for the most part, tend to play down risks and to avoid liability for contagious effect of financial crises by overemphasizing risks (Dreher et al., 2008). But such an optimistic tendency does not apply to all countries. Considering the world economy’s contagious effect, the IMF’s optimistic tendency is more likely to be manifested in the forecasts on the world economy as a whole and major countries and regions.
Moreover, deviations in the forecasting behaviors may also be induced by IMF loans. Most studies show that the IMF tends to overestimate the effectiveness of its lending policy and to preserve its reputation as an international financial stabilizer. The IMF’s country-office employees tend to make optimistic forecasts on member countries receiving IMF loans to justify their loans. The IMF’s optimistic
forecasts on recipient countries also help recipient governments to persuade other domestic stakeholders to accept IMF loans.
Second, forecast errors are derived from differences between available information and real information in the forecasting process. Available information employed in the forecasting process is often incomplete, which leads to forecast errors.
Differences between available information during forecasting and real information may derive from three sources: data availability, assumptions on the reliability of policies, and political factors. First, data availability is the foundation for macroeconomic forecasting. In addition to influencing the abovementioned forecasting methods, data availability will also directly influence forecast quality.
Second, forecasting a country’s growth requires assumptions on the reliability of its domestic policies, such as monetary and fiscal policy, and international factors like world economic growth, economic performance, international finance, trade, and bulk commodity prices. Errors in assumptions on the reliability of policies will affect the economic forecast on a country. If the IMF fails to correctly forecast policy changes of its member countries, deviations in its forecast on such countries will occur.
Third, political factor will also cause an impact on the forecast outcome. Forecasting economic variables involves both economic and political factors.
Such political influence may come from two sources: direct and indirect. First, member countries have a direct political influence on the IMF, including their quotas and voting rights. IMF employees work under the guidance of the Executive Board, and various business activities have to consider the preferences of major shareholder countries. Therefore, it is easier for the IMF’s major shareholder countries to acquire more favorable economic forecasts.
Second, member countries have an indirect political influence on the IMF. They may seek the IMF’s support in economic forecasting through close political ties with the IMF’s major shareholders (such as the United States), which is conducive to optimistic IMF forecasts on them. Most studies determine indirect political ties to the IMF by measuring the consistency of a country’s votes with U.S. votes at the UN General Assembly.
Table 1 identifies a few possible sources of IMF forecast errors, as well as possible directions of various factors’ impact on such forecasts. These factors may influence different aspects of forecasting. Data availability may influence the choice of forecast model and method, as well as forecast quality. Loan issuance may cause the IMF to overestimate a recipient country’s economic performance, but helps IMF employees gain a deeper understanding of such country and increase the forecast accuracy on the
1 country as well.
4. Empirical Analysis
4.1 Model Setting
We create the following model
(1) Forecast Error ( ). is the IMF’s forecast error for the real GDP growth rate of country i during period t , as defined in equation (3). Here, and are the IMF’s forecast value of real GDP growth rate of country i during period t and real GDP growth rate of country i during period t . Considering data availability, this paper includes a total of 160 sample countries, including 129 developing economies and 31 advanced economies, and sample period is from 2003 to 2015. Data are from the IMF’s World Economic Outlook Databases.
(2) Economic size of member countries ( ). is the share of country i in world GDP during period t by current-price GDP. Data are from the IMF’s World Economic Outlook Databases.
(3) IMF loans ( ). is the amount of IMF loans received by country i during period t as a share in its current-period GDP. Data are from the IMF Monitoring of Fund Arrangements Database.
(4) Member countries’ direct political influence on the IMF ( ). Here, we use two indicators to depict member countries’ direct political influence on the IMF ( and ), where is the quota of country i at the IMF during period t . is whether personnel from country i serve as executive director of the IMF Executive Board during period t . Where, data are from the IMF Financial Database, and data are from the IMF annual reports.
(5) Member countries’ indirect political influence on the IMF ( ). is the consistency of country i with the U.S. votes at the UN General Assembly during period t . If is 1, all the country’s votes are consistent with U.S. votes. If it is -1, the country’s votes are totally opposite to U.S. votes. It is from the United Nations General Assembly Voting Data of Voeten et al. (2009).
( 6) Data availability ( ). This paper employs two indicators to measure a country’s data availability. The first indicator is whether a country adopts the IMF’s Special Data Dissemination Standards (SDDS) ( ). If a country adopts the SDDS, is 1; otherwise it is 0. The second indicator is the statistical capacity indicator of the World Bank in evaluating the statistical capabilities
of countries (Statistical Capacity Indicator) ( ). This World Bank indicator evaluates countries’ statistical capabilities for a score range of 0-100. Information about the adoption of SDDS by countries is from the Dissemination Standards Bulletin Board, and SCI is from the World Bank Statistical Capability Indicator Database.
(7) Assumption of initial conditions ( , and ). , and respectively denote IMF forecast errors for the real economic growth rates of the U.S., Germany and China during period t . Considering the influence of these three countries on the world economy, the quality of the IMF’s forecasts on them will directly influence forecasts on other countries. Data are from the IMF World Economic Outlook Database.
4.2 Analysis of Empirical Result
4.2.1 Overall sample estimation result
We employ panel data analysis method to estimate equation (4), and decide whether to adopt fixed-effect model or random-effect model based on robust Hausman test result. We test two different estimation forms, i.e. variables’ effects on IMF forecast errors under and circumstances respectively. Aside from overall sample estimation, we also conduct sample-specific estimation for advanced and developing economies to evaluate the IMF’s different forecasting behaviors for them. In overall sample and sample-specific estimations, we first consider the impact of SDDS adoption on IMF forecast errors, and then introduce the World Bank SCI to verify data availability’s impact on forecasts. Overall sample estimation result is shown in Table 2.
Estimation coefficient of is significantly positive, which shows that the IMF tends to give optimistic forecasts on member countries that receive IMF loans. The larger amount of IMF loans received by a member country, the more likely it is to receive an optimistic forecast.
Estimation coefficient of is positive, and passes significance test on most occasions, which means that countries with greater GDP are more likely to receive optimistic forecasts from the IMF. These results are consistent with expectations. The IMF tends to issue optimistic forecasts on countries that represent a significant share of the world economy, so as to guide world economic expectations and promote world economic development.
For political relations, coefficients of both and are negative and insignificant, which implies that member countries’ direct political relations with the IMF have an insignificant effect on the IMF forecasts. However, the estimation coefficient of member countries similarity’ with the U.S. votes at the UN General Assembly ( ) is significantly positive, which means that the closer a country is to the political stance of the United States, the easier it becomes to receive an optimistic forecast from the IMF. Why do member countries’ direct political relations with the IMF have an insignificant effect on IMF forecast errors and their indirect political relations with the IMF have a rather significant effect? We believe that only the IMF’s major shareholders are able to influence the IMF’s forecast behaviors. For instance, Steinwand and Stone (2008) finds through literature survey that in various stages of the IMF’s lending activities, only the IMF’s major shareholders may exert a significant influence. That is to say, most member countries are not able to influence IMF decisions based on their own status. Instead, they fight for favorable decisions by coordinating political stances with the IMF’s major shareholders, especially the United States2.
Estimation coefficient of with respect to data availability is negative, which suggests that a country’s adoption of the IMF’s SDDS helps reduce our forecast errors and increase IMF forecast accuracy. Joining the IMF’s SDDS means that a country passes the “good statistical practice” test, and
undertakes to comply with good practices regarding data coverage, application frequency and timeliness,
3 channels for public access to data, data authenticity and data quality. This provides the IMF with timely and accurate data to make forecasts.
Errors of IMF forecasts for major economies will also affect forecasts on other countries. Estimation coefficients of IMF forecast errors for the U.S., Germany and China are all positive, which shows that in case of errors of IMF forecasts on these countries, forecast errors for other countries will increase. This indicates that forecasts on major economies play a pivotal role in IMF forecasts. Coefficients of forecast errors for China and Germany are significant, which means that the forecast accuracies of these two countries are particularly important.
4.2.2 Sample-specific estimation result
To uncover the IMF’s different inclinations in forecasting advanced and developing economies, we carry out separate estimations for such economies (Table 3). Advanced and developing economies are classified according to the IMF’s World Economic Outlook Database.
Overall, the sample-specific estimation result is similar to the overall result, but still offers some different discoveries.
First, it is easier for countries that receive IMF loans to be forecasted optimistically, which is consistent with the estimation results of the overall samples. However, the estimation coefficient of advanced economies is significantly higher and more significant than that of developing economies. This implies that, compared with developing economies, advanced economies that receive IMF loans are more likely to be forecasted optimistically by the IMF. The IMF’s forecast bias can be attributable to various reasons. Compared with developing economies, advanced economies generally have greater voice and voting rights in the IMF’s governance structure, and such rights are more concentrated and easier to influence the IMF’s forecast activities. Also, compared with developing economies, the IMF more urgently needs to justify loan programs to advanced economies through overestimate. The reason is that loan programs to advanced economies generally require larger amounts of capital.
Second, the coefficient of is positive, which is similar to the estimation result of total samples. The estimation coefficient for developing economies is relatively significant, but coefficient for advanced economies is insignificant.
Third, with respect to member countries’ direct political relations with the IMF, the group-specific estimation result is similar to the overall estimation result. For advanced and developing economies, the coefficients of both and are negative and insignificant, which shows that member countries’ direct political relations with the IMF do not have any significant influence on IMF forecasts.
Fourth, for member countries’ indirect political relations with the IMF, the estimation coefficient of similarity between developing economies’ votes and U.S. votes at the UN General Assembly ( ) is significantly positive, which is consistent with the estimation result of the overall samples. The implication is that the closer developing economies are to the political stances of the United States, the easier it is for them to be forecasted optimistically by the IMF. However, the estimation coefficient of advanced economies is significantly negative, that is, the similarity of political stances between other advanced economies and the United States will not cause the IMF to overestimate them. The difference in the estimation coefficients suggest that in the IMF’s forecast activities, the United States as a major shareholder attaches greater importance to whether developing economies back the U.S. political stances.
Fifth, with respect to data availability, differences also exist in the estimation results. The estimation coefficient of advanced economies’ remains negative, which is consistent with the estimation result of the overall samples, but both estimation coefficients are insignificant. However, the estimation coefficient for developing economies is positive but insignificant as well. This may have to do with the grouping of samples. For instance, among the 129 developing economies as samples, the number of countries adopting the IMF’s SDDS increased from 22 to 36 during the observation period, accounting for 17.1% to 27.9% respectively of the total samples. Among the 31 advanced economies as samples, the number of countries adopting the IMF’s SDDS increased from 27 to 30, accounting for 87.1% to 96.8% respectively. Such disequilibrium in data distribution led to differences in estimation results. To evaluate the data availability’s effects on forecast errors, this paper subsequently conducted estimates based on the World Bank’s Statistical Capability Indicator (SCI).
Sixth, the estimation coefficients of IMF forecast errors for the United States, Germany, and China are all positive, which is consistent with the estimation result of the overall samples. From
the perspective of estimation coefficient significance, the coefficients of forecast errors for Germany and China with respect to advanced economies are both relatively significant. The implication is that forecasts on advanced economies are more influenced by forecast errors for Germany and China. With respect to developing economies, only the coefficient of forecast errors for Germany is relatively significant, which means that forecast errors for the German economy have the greatest influence on forecasts on developing economies.
4.2.3 Retest of data availability’s effects on forecast errors
To further test data availability’s effects on forecast errors, we substitute whether a country adopts the IMF’s SDDS ( ) with the World Bank’s Statistical Capability Indicator for various countries (
) to re-estimate equation (4). The World Bank’s SCI is a comprehensive score of a country’s statistical system based on 25 indicators and the three dimensions of statistical methodology, data sources and periodicity
and timeliness. Each year, the statistical capabilities of over 140 developing countries are evaluated4. Compared with , has two statistical characteristics: First, the value range is 0-100, which is unlike the dummy variable of whose value is either 0 or 1. Second, a country’s statistical capability depicted by will increase or decrease with the passage of time. If we follow the standard of whether a country adopts SDDS, unless under special circumstances, a country normally will not exit the standard, i.e. will only depict the process from 0 to 1 and without any change from 1 to 0. Therefore, substituting with for estimation helps further verify the robustness of estimation. Based on data availability, there are 120 selected samples estimated by , all of which are developing countries during the period from 2005 to 2015.
Table 4 shows the estimation result after the substitution of by . Estimation coefficient of is significantly negative, which suggests that a country’s improvement of statistical capability helps reduce IMF forecast errors and increase IMF forecast accuracy. This confirms this paper’s prior hypothesis. But coefficient of IMF loan programs is negative and insignificant. The implication is that loan programs have an insignificant effect on the IMF’s economic forecasts on developing economies. This is consistent with sample-specific estimation result. Estimation results of other variables are also generally consistent with our prior estimates. If a country is closer to the U.S. political stance at the UN
General Assembly, it is more likely to be forecasted optimistically by the IMF. IMF estimation errors for major economies still have a positive influence on estimation errors for other economies. This verifies the robustness of this paper’s estimates.
5. Concluding Remarks
This paper creates a political economy framework for analyzing the IMF’s forecast errors from three aspects (forecast method and information, and political factor), and tests the effects of different factors on IMF forecasts through an empirical analysis of 160 IMF member countries during 2003-2015 with the example of real economic growth indicator. This paper’s key conclusions are as follows.
First, it is easier for IMF member countries receiving IMF loans to be forecasted optimistically by the IMF. Sample-specific estimation reveals that, compared with developing economies, advanced economies receiving IMF loans will be forecasted more optimistically by the IMF.
Second, political factor has a significant effect on IMF forecast errors. The closer a country is to the U.S. votes at the UN General Assembly, the more likely it is for the country to be forecasted optimistically by the IMF. However, IMF member countries’ direct political relations with the IMF (quotas at the IMF and seats at the Executive Board) has an insignificant effect on IMF forecast errors.
Third, increasing IMF member countries’ data availability helps improve IMF forecasts. Judging by overall sample estimation, a member country’s adoption of the IMF’s SDDS is conducive to reducing IMF forecast errors. In sample-specific estimates, however, such a relationship is insignificant. Thus, this paper conducts a retest based on the World Bank’s Statistical Capability Indicator (SCI). Result shows that the improvement of statistical capability helps reduce forecast errors, which further verifies the role of data availability in the IMF’s forecasts.
Fourth, assumptions of other conditions will also affect the IMF’s forecasts. IMF forecast errors for major countries (in this case, the United States, Germany and China) will magnify forecast errors for other countries. Nevertheless, the effect varies across different countries. For advanced economies, the coefficients of forecast errors for Germany and China are relatively significant. For developing economies, the coefficient is relatively significant only for economic forecasts on the German economy. The implication is that improving the forecast accuracy of these two countries is particularly important.
Based on this paper’s conclusions, we propose the following recommendations to improve the IMF’s forecast quality. First, the IMF should create a more independent forecast procedure to avoid interference of internal and external factors. To make its forecasts more reliable, the IMF should reduce the impacts of political factors and organizational preferences on its economic forecasts. Specifically, the IMF might introduce external mechanisms to enhance forecast supervision and research, and restrain its internal behaviors through such external mechanisms.
Second, the IMF should continue improving the forecast data quality. IMF member countries should be helped to standardize data reporting and improve statistical capabilities and data reporting quality through bilateral surveillance and technical support.
Third, the IMF should improve its forecast accuracy for major economies by staffing and training country researchers for major economies and enhancing country offices’ forecast capabilities. The IMF should also enhance communications with IMF member country governments and private sectors to reduce forecast errors for major economies.