Re-uniting development economics

Development economics is split between macro- and micro-development economists.  The former focus on economic growth and tend to analyze economy-wide policies such as trade, fiscal, and currency policies.  Think Anne Krueger, Jeff Sachs, Bill Easterly, Paul Collier.  The latter focus on individual-level outcomes and analyze microfinance, education, health and other social policies. Think Angus Deaton, Mark Rosenzweig, Michael Kremer, or Esther Duflo. The first group relies on cross-country econometrics or analytical country studies.  The second relies increasingly on randomized field experiments.  Ostensibly, both groups share the same objective: achieving sustainable improvement in standards of living of the poor.  But the way they go about it seems so different as to make them look like members of entirely different disciplines. 

My latest paper argues that there is both good and bad news on this front:

The good news is that there is substantial convergence in the policy mindset exhibited by micro evaluation enthusiasts, on the one hand, and growth diagnosticians, on the other. The emerging “consensus” revolves not around a specific list of policies, but around how one does development policy. In fact, practitioners of this “new” development economics—whether of the “macro” type or “micro” type—tend to be suspicious of claims to ex ante knowledge about what works and what does not work. The answer is neither the Washington Consensus nor any specific set of initiatives in health or education. What is required instead is recognition of the contextual nature of policy solutions. Relative ignorance calls for an approach that is explicitly experimental, and which is carried out using the tools of diagnostics and evaluation. Old dichotomies between states and markets play little role in this worldview and pragmatism reigns. The proof of the pudding is in the eating: if something works, it is worth doing.

This convergence has remained largely hidden from view, because the analytical and empirical tools used by economists at the macro and micro end of things—growth economists versus social policy economists—tend to be quite different. But I will make the case that there is indeed such a convergence, that it is a significant departure from the approaches that dominated thinking about development policy until a decade or so ago, and that it represents a significant advance over the previous generation of research.

The bad news is that there has been an accentuation of the methodological divergence between macro-development economists and micro-development economists, which threatens to overshadow the convergence on policy. In particular, the randomized field trials revolution led by researchers in and around the MIT Poverty Action Lab (Banerjee 2007, Duflo 2006, Duflo, Glennerster, and Kremer 2006) has greatly enriched the micro end of the field, while creating bigger barriers between the two camps. This is not just because randomization is rarely possible with the policies—such as trade, monetary, and fiscal—that macro-development economists study. More importantly, it is because of the raising of the stakes with regard to what counts as “admissible” evidence in development. The “randomistas” (as Deaton [2007] has called them) tend to think that credible evidence can be generated only with randomized field trials (or when nature cooperates by providing the opportunity of a “natural” experiment)....

In fact, the micro experimentalists tend to overstate how much we learn from randomized trials and underestimate how much we learn from other types of evidence.

The question we need to pose of any piece of research is the Bayesian one: does the finding change our priors on the question we are interested in? Randomized evaluations do pretty well when they are targeted closely at the policy change under consideration, but less so when they require considerable extrapolation. In the latter case, evidence from randomized field experiments need not be more informative than other types of evidence which may have less airtight causal identification but are stronger on external validity (because of broader geographical or temporal coverage). In practice internal validity—just like external validity—is not an either-or matter; some studies do better than others on this score than others, and deserve more of our attention on that account. But this preference has to be tempered with a consideration also of external validity. The bottom line is that randomized evaluations do not deserve monopoly rights—or even necessarily pride of place—in moving our priors on most of the important questions in development economics.

But the paper is not meant to be a critique of randomized evaluations, which have greatly enriched our empirical toolkit.  It is instead a plea for not letting prevailing methodological differences overshadow the larger convergence. My purpose is to get macro-development economists and micro-development economists to see that they have much more in common than they realize.

Case in point: the greatest development success of our time, China.  As I review and document in my paper, the experimentalist mind set was deeply ingrained in China’s approach to reform. Some of the experiments that proved extremely successful were: the household responsibility system, dual-track pricing, township-and-village enterprises, and special economic zones.  Of course, the informal evaluations to which these experiments got subjected would not satisfy the standards of the MIT Poverty Action Lab. “Seeing whether something works” is hardly as rigorous as randomized evaluations. But it would be silly to claim that Chinese policy makers did not learn something from their experiments. 

The China example is important because it illustrates, in a vastly significant real-world instance, how the experimental approach to policy reform need not remain limited in scope and can extend into the domain of national policies. China, of course, is a special case in many ways. The point is not that all countries can adopt the specific type of experimentation ... that China has used to such great effect. But the mindset exhibited in China’s reform process is general and transferable—and it differs greatly from the mindset behind the presumptive strategies outlined above. It is perfectly illustrative of the potential convergence between the ideas of micro-development economists and macro-development economists. One would hope that the response of micro experimentalists to China’s experimentalism is not to say “but this is worthless, none of the experiments were evaluated rigorously through randomization,” but to say instead: “great, here is how our ideas can make the world a better place not just one school or health district at a time.”

So my bottom line is that the practice of development economics is at the cusp of a significant opportunity. We have the prospect not only of a re-unification of the field, long divided between macro- and micro-development economists, but also of a progression from presumptive approaches with ready-made universal recipes to diagnostic, contextual approaches based on experimentation and policy innovation. If carried to fruition, this transformation would represent an important advance in how development policy is carried out.

But we need more work. Macro-development economists will have to recognize more explicitly the distinct advantages of the experimental approach and a greater number among them will have to adopt the policy mindset of the randomized evaluation enthusiasts. As the Chinese example illustrates, extending the experimental mindset to the domain of economy-wide reforms is not just possible, it has already been practiced with resounding success in the most important development experience of our generation. Micro-development economists, for their part, will have to recognize that one can learn from diverse types of evidence, and that while randomized evaluations are a tremendously useful addition to the empirical toolkit, the utility of the evidence they yield is restricted by the narrow and limited scope of their application.

In the end, macro-development economists have to be humbler about what they already know, and micro-development economists humbler about what they can learn.

Comments

[Anh Tran]

Both types of development economists you mentioned still won't be able to do justice to economic development because of the fact that much of their works are based on the Western-based economic ideology. I suggest spiritual economics is one way to go. Sounds weird?

To do justice to economic development, we have to realize these things:

1. Resouces are finite, so it does not matter how best or how hard you try, economies cannot grow sustainably for much longer if the current economic ideology dominates the discussion.

2. Economic conditions are resulted from a whole array of factors, including cultural, political, geographical, social, so that the focus squarely on economic analysis misses many points as we attempt to advance economic development.

3. Data can be distorted from both fronts: the object and the researcher. So, it does not matter how elaborate the econometric analysis is, the truth will never be reached. If the "truth" could be reached, how then the WB and the IMF failed in many fronts to help poor countries develop?

[Per Kurowski]

Dani Rodrik pleads “for not letting prevailing methodological differences overshadow the larger convergence.” And says, “development economics is at the cusp of a significant opportunity. We have the prospect not only of a re-unification of the field, long divided between macro- and micro-development economists, but also of a progression from presumptive approaches with ready-made universal recipes to diagnostic, contextual approaches based on experimentation and policy innovation. If carried to fruition, this transformation would represent an important advance in how development policy is carried out.”

And I have to ask what is so good about that and what is so bad with divergence. This is all about development and not about laying the foundations for a new development church that even while much freer spirit than the Washington Consensus, almost a hippy version of it, is still a church.

Dani Rodrik also urges “In the end, macro-development economists have to be humbler about what they already know, and micro-development economists humbler about what they can learn.” And I would say, humbler, in just a general way, would suffice.

By the way for the second comment in a row I subscribe completely to what Anh Tran says referring to the changing global realities. Just as an example I recently read a study by two prominent development economists who were scratching their heads in order to analyze why El Salvador had not grown as much as it should over the last decade, without even discovering the reality that most Salvadorians had just left their country to grow somewhere else, mainly in the US, to such an extent that what they produced gross in the US already exceeded by much El Salvador’s own local GDP.

[inthemachine]

"what is so bad with divergence"

The conversational manifestation of it can prompt non-economists in development to percieve economists as unable to ever agree on anything. If you need evidence of that, I have a long list of economist-jokes that have been trotted out at varoius meetings that I can share with you.

[Per Kurowski]

"The conversational manifestation of it can prompt non-economists in development to percieve economists as unable to ever agree on anything."

Yet it might be much worse if they agree!!!

I mean, in that case why would us economist be needed?

cheers

[Per Kurowski]

"The conversational manifestation of it can prompt non-economists in development to percieve economists as unable to ever agree on anything."

Yet it might be much worse if they agree!!!

I mean, in that case why would us economist be needed?

cheers

[rmx]

Dear Dani

Do you know anyone working on both micro and macro development? I would like to learn from someone doing both fronts. It must be a difficult task but a rewarding one as well, am I intending to do to much?

[Mao]

China has had very much success with experimentation. The great leap forward and the cultural revolution were really good examples of what not to do with people's lives.

[Frank]

Dear Prof. Rodrik,

I read your paper and I agree with you in principle but not in practice.

Yes, there are various research methodologies, and yes experimentation of various kinds is good.

I fundamentally disagree with your portrayal of Growth Diagnostics (GD), however.

First, it ignores history. Development economist have written whole treatises and careful case studies for a number of countries for decades. The idea that until GD came along economist where robots implementing blind laundry lists is a useful straw man, but not a fair description of reality. A little more humbleness is called for here. GD is a useful reminder of what good economic practice is about, perhaps stated more formally, but not a new paradigm.

Second, it is unfalsifiable. GD works when it works and it doesn't when the diagnostic is not performed right.

Third, it ignores practical reality. Essentially what you are arguing is that we need careful diagnosis on a case by case basis. Most well trained economists I know will fully agree. Yet the practical problem is scaling that up to institutions like the World Bank with some 5,000 economists, some brilliant, some terrible, most average. I call this "the we cannot all be Dani Rodrik problem" or DRP for short.

DRP is a problem of rules versus discretion. Can you trust 5,000 economists to exercise their discretion? My sense is that most IFIs don't. Indeed, there are already "best practices" circulating around for doing GD! Ultimately, to scale things up, this is inevitable. After all, it was Stieglitz, then Chief Economist of the WB, who famously quipped that IFIs have 3rd rate economists from 1st rate universities.

How to deal with the DRP? One could hope to recruit better personnel, but that is easier said than done: the best tend to go to academia, private sector, etc. Alternative, one may seek better training. Yet, even in medicine, where training is exhaustive, someone recently told me that 50% of all medical diagnoses are wrong. Forget the accuracy of the percentage - the point is the error rate is huge.

So if recruitment and training have limited impact, What do you do? You impose safeguards, controls, and guess what, rules and best practices.

To conclude, my sense is that your whole debate is one of rules versus discretion, and whereas we would all prefer discretion under the right people deciding, it is heavily conditional on who is in charge. If economists cannot be trained better, and if the best don't end up in policy, then we are going to need some rules, at a cost. But presumably that cost is less than the cost of giving discretions to semi-competent economists, or much worse, Mao or Pol Pot (as another commenter in this blog emphasized). Then again, the latter were not true "Growth Diagnosers" and experimentalists, or were they?

[jose carlos]

then, maybe it isn't rules-vs-discretion; but some basic rules-of-thumb and well-informed discretion. the quest for Aristotle's mean.

[Punditus Maximus]

I've often wondered why development is not taught as business is taught -- with case study after case study. Surely entire nations are at least as diverse and complicated as medium-sized businesses.