Is there useful work in economics?

David Leonhardt of the New York Times surveyed economists to find out, and he says the answer is definitely yes:

I received dozens of diverse responses, but there was still a runaway winner. The small group of economists who work at the Jameel Poverty Action Lab at M.I.T., led by Esther Duflo and Abhijit Banerjee, were mentioned far more often than anyone else.

Ms. Duflo, Mr. Banerjee and their colleagues have a simple, if radical, goal. They want to overhaul development aid so that more of it is spent on programs that actually make a difference. And they are trying to do so in a way that skirts the long-running ideological debate between aid groups and their critics.

“Surely the most important societal question economics can help answer is why so many people are crushingly poor and what can be done about it,” David Romer, a professor at the University of California, Berkeley, said. The macro issues (like how to build a democracy) remain maddeningly complex, Mr. Romer noted. But thanks in part to the poverty lab, we now know much more about how to improve daily life in the world’s poorest countries.

...

Mr. Banerjee and Ms. Duflo may not be a modern-day Keynes or Friedman. But they have still managed to do something rather profound. They have brought together the best of the new economics and the best of the old.

As has been the trend over the last decade, they have plunged into the world around them, refusing to accept the idea that economics is merely an extension of math. Yet no one can accuse them of working on some little problem that doesn’t matter.

It is wonderful to see my colleagues up the river getting the attention that they deserve.  The work that Leonhardt describes (and much other work in the same style) is extremely important, and has infused development economics with fresh air--most importantly in giving it a needed experimental focus.

At the same time, I worry that Esther, Abhijit and others are setting themselves up for the inevitable backlash by appearing to oversell what the randomized field experiments teach us.  I say "appear" because they are sufficiently careful in their writings. But what the outside world sometimes hears is different.

The backlash will have several lines of attack. One is that the experiments in question have little economics in them, and teach us little about economics.  So what if we are saving lives, Esther and Abhijit will rightly respond.  Second, most experiments have unclear external validity, critics will say. In other words, we don't know if we can generalize to other setting, and if so how exactly.  (See the comments thread here for how this has played out in one of my recent posts.)  So, repeat the experiment in other settings, the randomizers will say.

One way to silence the critics is to have the randomizers tell us what are some social policies they have learned are good things that should be emulated in other settings. Or since their approach is broadly within the "many recipes" camp, it would be useful to hear from them what we have learned about context-specific policies--i.e., what kind of policies should be implemented in what kind of settings.  

While on the subject of randomized field experiments, see also the very useful discussion by Chris Blattman, whose blog has turned into the most interesting development blog around.

Comments

[messels]

did you see the "philanthropic" activities of google and sorors in india?
http://www.marketingpilgrim.com/2008/02/googleorg-india-googles-master-plan.html
i definitely think investing in the right programs is more effective than "giving" away money but we shouldn't call it a philanthropic activity since everyone stands to make financial gains.

maybe we're coming up against a paradigm shift in the definition of philanthropy. what do you think?

[madmax]

I couldn't agree more.

Take for example a recent paper by Dean Karlan and company " Finding Missing Markets ... "

http://research.yale.edu/karlan/downloads/FindingMissingMarkets.pdfA)

They seem to be blaming eurepgap for the collapse of drumnet (which has been praised for its efforts to link smallholders with exporters) but eurepgap has been around since 1996 - see:

http://www.ifpri.org/pubs/dp/ifpridp00737.pdf

And Kenya has been exporting green beans since the 1960s -- most of the large exporters have outgrower schemes where they buy from smallholders. So, why weren't they going to the farmers targeted by Drumnet?

And, green bean exports have continued to increase every year since 1990, so how come eurepgap only affected drumnet?

Things seem fishy to me - I suspect that Drument was mismanaged - end of story but of course I have no way to prove that.

But wait, there are more problems with this paper -
what's the market failure here? I don't get it, these farmers are 1.5
hrs. from Nairobi - prime distance for selling to private firms. MAYBE its
too expensive to comply with eurepgap standards individually. Ifpri and
others report that the imposition of this type of standard in the 90s lead
to a big drop in purchases from smallholders. But this has been more than
made up for by larger more mechanized farms!

And anybody who has paid any attention to ag. In africa knows that export
crop farmers are richer than food crop farmers. Who gets to export is
determined primarily by geographical constraints, coffee only grows in
certain places etc. -- so why do we need this evaluation to tell us that
exporting could be good.

Though esther duflo would have us believe that these experiments are free
of the arrogance that goes along with lots of past development thinking, I
see it all over a project like this - stepping into a market you might have
visited for at most a month and thinking you can find a missing market.

MAYBE THE SAD EPILOGUE IS REALLY THAT A KNOW-IT-ALL, MISMANAGED NGO INTRODUCED A POORLY DESIGNED SCHEME (WITH NO REGARD TO THINGS LIKE EUREPGAP THAT HAVE BEEN AROUND SINCE 1996!!!) TO HELP SMALLHOLDERS AND INSTEAD OF HELPING THEM, IT PUT THEM THROUGH THE TRAUMA OF HAVING TO DEFAULT ON A LOAN....... THAT'S THE BACKLASH.

[Jon Stern]

I share the reservations about randomised experiments of the Bannerjee-Duflo type. They clearly have their uses - but their limitations are very real.

Since the 1980s, labour economic studies of the impact of training and programmes to help the unemployed, lone parents, etc back into jobs have tried to use randomised experiments. The results were very interesting but, as Heckman and others have since shown, raising almost as many design and interpretation issues as they have resolved. In particular, the problem of generalising from one-off small-scale experiments is crucial.

It is also not the case that conventional evaluation methods cannot be usefully extended to provide additional information on how and why policies do or do not achieve good outcomes.

I have worked for some years on infrastructure regulatory issues, particularly in developing countries. A key question is whether (and, if so, why) the electricity, telecom, etc regulatory agencies have improved outcomes in terms of connections, supply levels, efficiency, investment, etc - and if not, why not.

Econometric studies (a literature to which I have contributed) can help as to the "whether" not in the "how and why". But, conventional evaluation methods can be extended to provide 'analytic narratives' which provide useful information on this.
An attempt at providing a guide on how to do this is in the World Bank's 2006 'Handbook for Evaluating Infrastructure Regulatory Systems' by Brown, Stern and Tenenbaum.

I strongly agree with you and Chris Blattman that no single evaluation method is always right. I would strongly argue that the results of small-scale randomised experiments both have to be interpreted carefully and, more importantly, are VERY difficult indeed to use for generalised policy advice.

Jon Stern

[andthenyoufall]

Down the river, Dani. Down the river.