Debate: Who's Business is Development? Not a Popularity Contest: Bringing Rigor to Open Development

Participation, dialogue,openness. These are values we cherish and aspire to. Who would be in favor of unilateralism, monologue, or isolation as  giding principles of development? The call for open development as a multipolar and more democratic search for solutions is almost a platitude. The issue is not whether openness, a positive attribute in itself, can promote better policies, but rather what are the conditions required for
openness to succeed.

True, in many developing countries closed political systems have created far too many opportunities for bad policies to take hold. And there are certainly examples of misguided policies that have been foisted on poor countries by otherwise well-intentioned donors and development agencies. But it is equally true that when participation takes place in the context of unequal power relationships, whether in local decision making such as in school committees, or in national and international fora, it can also lead to solutions that hurt the weak. And deliberation alone, without the discipline provided by a strong empirical grounding, runs the risk of just creating noise and poor outcomes.

ARE YOU SURE ABOUT THAT?

The core concept here is contestability, without which the probability of mistakes and wrongheaded solutions increases. How to achieve it? At least a certain degree of openness, for example, access to information, the opportunity to debate, experimenting with alternative paths, is a necessary condition. But it is not sufficient. Without a means of measuring the consequences of alternative ideas and paths, there is no learning. And insufficient incentives to make use of what is learned from those measurements can perpetuate weak solutions.

SHOW ME

Two examples of applied research we have supported illustrate this point.

Educators and education policy makers want to create the best possible learning environment for students. How to do this, especially in primary school, where reading, writing, and mathematical skills are first acquired, is the subject of policy debates in many parts of the world. Should teachers be paid more? Should students be  rewarded for good test results? Do schools need more supplies and better infrastructure?

In Andhra Pradesh, India, researchers set out to study what works better at improving student test scores—paying teachers bonuses based on results or giving schools cash grants for supplies or cash grants to hire one extra teacher (Muralidharan and Sundararaman 2011). They ran an experiment where competing approaches were tested side by side. Students in schools where teachers received either individual based or group-based incentive pay did better than students in control group schools in all five grades, all districts, and at all levels of test question difficulty. Individual incentives and group incentives worked equally well in the fi rst year of the program. By the second year, students in the individual incentive schools signifi cantly outperformed the group incentive schools. This study has informed policy making in Andhra Pradesh, and has the potential to do so across the globe because the evidence it generated is robust and credible.

Health experts and policy makers want people to have access to affordable and high-quality medical care. But in some developing countries, making quality healthcare available may first require making essential medicines available, such as antimalaria pills and antibiotics. The challenge to guaranteeing a steady supply is not only related to the ability to pay for medicines; poor roads, limited communications, and storage problems can also get in the way of keeping medical facilities stocked with what they need to provide children and adults with regular and lifesaving care.

In Zambia, the World Bank supported a project to guarantee the availability of essential medicines in remote 55 health facilities. A pilot program tested two distribution models for overcoming bottlenecks at the district level (Vledder and others 2010). The project found that hiring a facilitator to work directly with health facilities, while minimizing the role of district stores for storing and delivering medicines, helped clinics stay better stocked. Those health facilities that had more control over ordering medical supplies were more successful in keeping essential drugs stocked than those that were subjected to more controls by district stores. This model was so successful that it has been extended to all districts in the pilot study.

EVIDENCE-INFORMED DEBATE

In both examples a conscious effort was made to measure rigorously the effects of alternative approaches so that debate could be informed by evidence rather than by preconceptions or prejudice.

From this perspective, the openness paradigm takes on a different smell and taste. It means subjecting development ideas not to a test of popularity but to the rigor of evidence. In a volume reviewing the lessons of the 2004 Shanghai Conference on Scaling Up Poverty Reduction (Fiszbein and Gevers 2005), we quoted the Italian author Primo Levi (1987): “Since it is difficult to distinguish the good from the bad prophet, we must be suspicious of all prophets: it is better to avoid revealed truths, even if we feel exalted by their simplicity and splendor, even if we find them comfortable because they come at no cost. It is better to be content with more modest and less inspiring truths that are laboriously conquered, step by step, with no shortcuts, by studying, discussion and reasoning, and that can be verified and demonstrated.”

Applying several of Levi’s ideas such as study, reasoning, and verification to the field of development should lead us away from the dangers of “bad prophecies,” and toward careful scientific consideration of alternative solutions based on evidence.

This implies that experts—those individuals and organizations that invest in the generation and systematic analysis of evidence—have a prominent role to play in the policy debate. In the volume on the Shanghai conference, we showed that only a small percentage of the cases that were heralded as “best practice” had any type of proper impact evaluation. This illustrates that the problem has often been not one of too much but rather too little expertise informing development. Experts come in different forms.

Openness here plays an important role in at least two ways.

First, Levi’s laborious, step-by-step process of inquiry requires that data and information be made available in a timely and affordable manner. It also implies that the evidence generated with that data be made easily accessible.

Second, diverse approaches and perspectives should inform the process of inquiry. As Scott Page (2007) convincingly argues: groups that display  range of perspectives (cognitive differences) outperform those that are homogeneous in the tools they utilize. This is a strong argument for multidisciplinary approaches.

Openness does not, however, displace the critical importance of careful use of systematic inquiry and evidence generation as the primary means to find development solutions. Let’s not throw the (knowledge) baby out with the (openness) bath water.

Ariel Fiszbein is Chief Economist in the World Bank’s Human Development Network. 

Download the PDF (3.49MB) to view all images, graphs and additional information.