For the last two years, there has been a mystery about the evidence supporting the past favorable assessments of the scope for reducing poverty using microfinance instruments such as the famous Grameen Bank (GB). The chances for many poor people to benefit from access to this form of credit rest, in part, on solving that mystery.
To understand the mystery we need to go back to an influential paper by Mark Pitt and Shahidur Khandker (PK), published in the 1998 volume of the Journal of Political Economy. PK documented research supported by the World Bank—research that came to provide the most cited scholarly evidence yet to support the view that microcredit helps reduce poverty.
The PK paper was based on their analysis of a sample of about 1800 households in 87 villages in Bangladesh—using the GB eligibility criteria based on landholding to identify the program’s impact. Based on their analysis, PK claimed that providing small amounts of credit, especially to women, could help poor families get out of poverty. They found economically and statistically significant gains in household consumption for women borrowing from GB. It is this finding that led Muhammad Yunus, the founder of the GB, to claim often that 5% of GB borrowers escaped poverty per year.
The mystery appeared in a potentially damaging critique of the PK paper by David Roodman and Jonathan Morduch (RM) in a 2009 Working Paper of the Center for Global Development. Strikingly, RM claimed that PK’s main finding could not be replicated. (RM also raise some concerns about a previous paper by Morduch and a paper by Khandker on the same topic; here I focus solely on their critique of the original PK paper, which gets the bulk of RM’s attention and has clearly been the most influential paper in the literature.) Based on their analysis of the same data set used by PK, RM found that GB borrowing actually made women worse off! However, they distance themselves from this disturbing conclusion, preferring instead to question whether anything could be claimed one way or the other. Thus they conclude that “30 years into the microfinance movement we have little solid evidence that it improves the lives of clients in measurable ways.”
In a posting on his microfinance blog, David Roodman lays down the gauntlet in no uncertain terms: “…I think my paper with Jonathan is the academic equivalent not of a citation but an indictment... It is a long document packed with logic and evidence that the flaws are not merely possible but provable in academic court and important enough to generate wrong results.” Indeed, Roodman goes even further to conclude on the basis of his paper with Morduch that “a lot of research published in prestigious journals is wrong.” This is strong stuff indeed.
But we are still left with the mystery. Either PK or RM must be wrong; they simply cannot get different results with the same model and data. (RM made some changes to the PK dataset, but these matter little.) It would be unwise to make too much of the matter until that mystery is solved.
On March 26, Mark Pitt offered a solution. His new paper refutes RM’s main findings and confirms the main result reported by PK, namely that GB borrowing by women significantly increases household consumption.
Pitt makes two main points. First, he questions the appropriateness of the estimation method used by RM in the real-world situations in which there is a positive minimum to borrowing; his simulations using synthetic data suggest that RM’s method performs poorly in this case. It would seem that RM should have known this about their estimator.
Second, Pitt points to inconsistencies between the control variables used by RM and those used in the original PK study. The control variables are naturally key in any non-experimental study, but the changes made by RM—also not evident to their readers—make a big difference to the results.
Based on Pitt’s findings, it looks like RM did not actually test the PK specification, as they claim. They got different results simply because they estimated a different model, and Pitt questions their model on a number of counts.
It will be interesting to hear how Roodman and Morduch respond to Pitt’s rebuttal of their findings. The ball is now on their side of the academic court.
Join the Conversation