Musing on the SDGs, R4D, and foresight work

I was asked to present to a class of bachelor students at the University of Utrecht on my experiences with the Sustainable Development Goals (SDGs) as a development practitioner and academic. I got to share the stage with 3 very impressive presenters, and all of us came to a similar point that the challenge of the SDGs is how to operationalize them.


How do we go from where we are to where we want to be?


And of course, there are no easy answers. However, I think I did the students a disservice, by focusing so much on the problems we are confronting, such as

  • lack of information
  • how to define the goals, and then measure and prove progress toward them
  • how do we prioritize different important outcomes
  • how to build consensus among various actors in society with very different agendas, objectives, and perspectives on what is desirable
  • how to foster and coordinate collective action across scales from the household to the globe on a myriad of goals that can be at cross-purposes

All these challenges are real, and the task at hand as presented by the SDGs is an impossible one. However, we must not lose hope. Without ambitious dreams we don’t reach beyond what we already know. I think I did a disservice to the class by focusing so much on what is holding us back, and not highlighting that we are still making progress, even as we make mistakes, and that we are applying and improving methodologies that should enable us to continuously improve the effectiveness of international development. What follows are a few thoughts that I wished I had shared more explicitly with the class.


Success is stumbling from failure to failure with no loss of enthusiasm
– Winston Churchill


While we must recognize that the development policies and interventions we have at our disposal are limited and imperfect, and that we don’t always know the full consequences of using them, we should also recognize that we are not without agency. We now have decades if not centuries of experience in trying to build and improve the world in which we live. Through these often ad hoc and uncoordinated actions we have learned a great deal, even as we may have created new problems while trying to resolve old ones. We have learned many things that generally work, and perhaps just as importantly we have learned a great many things that almost never work.

And while we still see incidences of truly catastrophic policies (e.g. Venezuela), even the most jaded analyst must recognize that humanity has made great leaps in tackling what was for most of our history our greatest threats: hunger, pestilence, and destitution. That we haven’t fully resolved these problems, or the fact that our solutions are proving to be unsustainable due to overuse of natural resources, should not detract from the fact that we have made incredible progress. This progress is evident not just by comparing changes over millennia, but also through changes observed after concentrated and coordinated efforts towards achieving the Millennium Development Goals. This should strengthen our resolve that progress is possible, even as it should humble us to recognize that it can be ephemeral and will almost certainly not be smooth. Progress is not self-sustaining, and it will require our continued attention and effort to see it continue.

The field of research for development is at its heart trying to document current and past efforts towards the greater goal of sustainable development. To collect, document, organize, and share our best understanding of the world, and how our actions impact and change it. It works to analyze past assumptions against evidence, to find misconceptions and correct them, and to share the lessons learned in the hope of inoculating us from repeating past mistakes.

My work has contributed to these efforts, by applying our current understanding of the macroeconomy to foresight. I have helped to build and improve computer simulation models that provide a framework to test our assumptions and biases in a transparent and systematic way. In conjunction with the the construction of scenarios of alternative futures, we can ask ourselves and others what would be the consequences of different events or actions, and test the interconnectivity of the complex socioeconomic, political, and environmental systems within which we act. These tools are illustrative and didactic as opposed to predictive. We cannot know the future, however, the doesn’t mean we can’t prepare for many different futures. These tools help to consider various ways in how the future could unfold, and whether our actions (policies, interventions, investments, etc.) are more, or less likely to be effective in achieving our desired outcomes. They suggest potential obstacles and challenges that we might not have originally considered, as well as suggesting potential positive and negative feedbacks, which could make problems more complicated to resolve.

We have applied these tools to help policymakers to better understand the potential consequences of their policies of interest, as well as highlight policies that look particularly beneficial or negative. This work has often been at the direct request of policymakers who themselves are more than aware of the complexity of the world and are looking for tools to help them better engage with complexity and uncertainty. The tools are helpful in reminding us that not all goods things go together, and that progress will not be panglossian. As we progress on some fronts, we may take steps back on others. Foresight methodologies and tools help give us a sense of the types and magnitude of tradeoffs we may have to make as we try to achieve the SDGs.


For example, efforts to prevent the conversion of land to agricultural purposes to prevent biodiversity loss and GHG emissions may make it more difficult to achieve the goal of zero hunger by reducing potential agricultural production, unless we layer additional interventions and investments to increase agricultural productivity. Of course, these subsequent investments may in fact be problematic for water quality and biodiversity if they involve heavy use of chemical inputs, and so on and so forth.


While these tools cannot fully account for all the connections and complexity in the world, and ultimately are themselves in constant need of questioning and improvement, they are still helpful. They help us to operationalize complexity in a transparent, systematic, and manageable way. In so doing, they improve our understanding of how our actions may have broad and unexpected implications. They push us away from purely linear thinking to gain a better appreciation for non-linear effects (i.e. network effects, vicious and virtuous cycles, etc.). They also encourage us to think of sustainable development as a process; to recognize we will need to constantly calibrate our efforts in response to unexpected consequences and events. All of which is valuable in the development of more holistic and robust policies that try to maximize positive outcomes while minimizing negative ones in an uncertain and constantly changing world.

It is also worth highlighting that the kind of system thinking encouraged by these tools does not require the use of advanced mathematical models. There are many ways to incorporate more holistic and systems thinking into policymaking, something that was quite evident from the very insightful questions that were raised by the class. In fact, I am heartened by the depth of some of the questions asked, as it suggests that we will continue plugging away at the great challenges confronting us.

Presenting in Bogota Colombia

cropped

Last week I presented work I’ve been involved with at IFPRI with the IMPACT model. I explained how the model works and how this tool can be used in conjunction with other tools developed at IFPRI can be applied to the Colombian context to analyze complex issues like the effects of climate change on agricultural prices, land-use, and GHG emmissions.

Take a look at the blogpost I posted about the event at the Global Futures website: http://globalfutures.cgiar.org/2014/12/03/leds-modelling-workshop-adapting-ifpri-tools-and-methodologies-to-the-colombian-policy-context/

 

Presenting in Buenos Aires

Just posted a blog on the Global Futures website about my trip to Buenos Aires, where I presented work done at IFPRI looking at the potential role of new agricultural technologies in sustainable intensification and mitigating the effects of climate change.

Agricultural Projections at the Cereals Exchange in Buenos Aires – INAI conference Argentina

Development Links – May-July

Wow, so I’ve been pretty busy and haven’t gotten around to writing up some posts, so let’s see if I can get back into the habit of blogging again by getting this draft of several articles I was looking at from the begining of the summer.

  • Violent Conflict and Gender Inequality – A World Bank working paper that takes a look at the varying gendered effects of conflict around the world. It feels a bit like a literature review. There are some very interesting papers cited in this review. I didn’t see a whole lot of new information, but it is a good summary of the many ways that conflict can effect both men and women.
  • Education Technology Success Stories – Excellent paper from Brookings looking at some of the successful and promising technologies that could be harnessed to improve our education system. I think that the spirit of the paper is especially important in that they are looking at technologies as aids, that can enable teachers and students to better use their skills and improve the efficiency of the education process.
  • A Welcome Half Loaf on Food Aid Reform (Center for Global Development) – An excellent post looking at the proposed reforms the the U.S. Food Aid program. I am generally in agreement with the author that any reform of the program would be welcome. Increasing local and regional sourcing would be more efficient, and allow for greater spill over effects into regional economies, instead of using food aid as an excuse to subsidize the transportation and agriculture sector. If a full reform isn’t possible at this time, incremental steps are better than nothing, and if implementing these reforms increases the push towards greater trade liberalization of the agriculture sector in Europe, than all the better.
  • RTC Study on School Uniforms in Ecuador (PDF)
  • Internet and Elections (World Bank Blog) – Not sure if the world is ready for voting on the internet. It is true electronic voting could encourage greater participation, improving access to voting, and speeding up the counting of votes and encouraging transparency, while reducing opportunities for voter intimidation. However, moving the election online could also open up elections to security attacks (hackings) and easier falsification of identity on the one hand, and if implemented by less than honest governments could facilitate ballot stuffing. Much like in most areas the promise of technology to improve our lives needs to be weighted with a good deal of skepticism, because the devil is always in the details.
  • Kenya’s laptops for schools dream fails to address reality (The Guardian)

Weekly Development Links – March 22

Some of these links are a little older, and I just hadn’t gotten around to reading them till this week. Any way on to the links:

  • White House Proposal Could Revolutionize Food Aid – From the Center of Global Development’s  Rethinking US Foreign Assistance blog, this blog post is a couple of weeks old. However, I think that it is an excellent piece especially considering that U.S. foreign aid and development budgets will almost certainly face cuts this year. This blog post points out that there are some low hanging fruit that could be picked to not only reduce spending but also to follow development best practices and potentially lead to improved outcomes. The article focus particularly on the extremely inefficient Food for Peace program, which is more of a subsidy to American Agriculture and Transportation sector than it is an effective development/assistance program. However, the preference towards U.S. producers and service providers are ingrained throughout the U.S. foreign assistance program. If we could move away from giving preferential treatment to U.S. producers and service providers, we could save money as well as encourage more local production. This could potentially build greater local resilience. Will there remain a role for U.S. produced food in creating a safety net in case of severe food emergencies? Probably, but we should try to source as much of the food aid as possible from the region, and thereby incur the double effect of boosting local economies, while reducing costs and trade distorting subsidies. Additionally, following through with this reform we could correct the farce of a foreign assistance program that largely benefits domestic sectors. If we want to assist domestic sectors it would be better to do so directly than covertly through U.S. foreign assistance.
  • Behind the Brands (PDF) – A briefing paper from OxFam looking at the role of the major food processors in the overall food production chain, and trying to assess how effectively these companies are towards promoting valuable social outcomes. On the whole their conclusion is that they are not doing enough to ensure that the global food chain is just. Overall the paper is interesting, but leaves a bit to be desired in terms of implementable policy solutions. Their main call is for increased transparency, which is probably a good starting point as there is terrifying ignorance about what goes on in the global food chain. This particular quote I found particularly insightful:
    This consolidation of the market-place has made it difficult for consumers to keep track of who produces which products and the “values‟ behind a brand… But perhaps more troubling is that since the global food system has become so complex, food and beverage companies themselves often know little about their own supply chains. Where a particular product is grown and processed, by whom, and in what conditions are questions few companies can answer accurately and rarely share with consumer

    As consumers and governments become more interested in the social aspects of production, and push for products that represent their own social values producers will feel monetary incentives to demonstrate to consumers the quality of their own supply chain, or at least this is the causal chain that OxFam is hoping for. OxFam makes a good case that it is in the best interest of companies to gain better control of their supply chains, if only to better protect their own brands from embarrassing negative publicity. On the whole I agree with this, but with some caveats. Demonstrating value in the way a good is processed in harder than showing value in the quality of the final good. Consumers and to a lesser extent producers cannot observe the whole production process (producers can’t observe everything their suppliers do), and must rely on others to confirm the value of the production process. To do this there needs to be better certifications that clearly explain to consumers (and producers) the intrinsic values of the way the good are made. Right now there are too many different standards (Green, Sustainable, Fair Trade, Organic, Shade-grown, Local, etc.), which imply certain social values in the production process. However, most consumers have trouble differentiating many of these standards, or knowing which is better. When consumers cannot effectively perceive the quality of the production process, then their actions are formed on poor information, which dilutes the market signals to producers to maintain socially beneficial production.  This paper is pretty timely considering the horse meat scandal in the United Kingdom, and some of the articles I’ve read recently about the mislabeling of seafood in the U.S.

  • The multilingual dividend – An interesting article from the Financial Times looking at the benefits of knowing more than one language. The benefits aren’t just limited to the language, but allow for more active brains, that allow people to look at the world from different perspectives. This improves problem solving, as well as making people more culturally aware and effective in teams. I thought it was particularly interesting that these benefits are not lost if you learn a language later in life, so for those of you interested in picking up another language go for it.
  • Quina: The little cereal that could (World Bank Blog) – Another post about my favorite pseudo-grain. This one has some good info about the global expansion of quinoa. It also speaks about the challenges producers in Bolivia are now facing in buying quinoa due to higher price.
  • ICTs and Literacy (World Bank Blog) – Interesting look at the role ICTs are currently playing in teaching literacy. I also really liked the point Michael Trucano made about the importance of 20th century skills (literacy and numeracy) and not to skip them in a rush to teach 21st century computer and technology skills. There are definitely opportunities to jump ahead to modern technologies, but without traditional skills like literacy people are not able to maximize the utility of these new technologies.

Weekly Development Links – March 15

Decided that the easiest way to summarize a bunch of the stuff that I read during the week is to start a weekly summary and review of some of the articles and papers I found interesting.

So on to the links:

  • Undernutrition: the invisible killer of 3 million children a year – A nice summary of many of the effects of malnutrition in children. There isn’t anything particularly new in the article. Nevertheless, I think it is important to keep highlighting the topic due to the enormous value to society of tackling and correcting child malnutrition.
  • 45-Year Trends in Women’s Use of Time and Household Management Energy Expenditure – Seemed to attract some controversy on the NY Times website, because people interpreted it as sexist and that the report was suggesting that women’s greater participation in the labor force was leading to their obesity. This is an incorrect interpretation. The report is the second part of an overall study looking at levels of activity today vs. 50 years ago. The first report looked primarily at office activity, which showed less physical activity today compared to 50 years. This second report was needed because women were underrepresented in the first report because of the lower level of labor participation by women 50 years ago. Therefore, this paper looked at the activity level of doing household chores 50 years ago as compared to doing the same chores today or activity in the office. The overall take away from both of these studies is that people are significantly less active today then they were 50 years ago, in large part due to greater automation and use of new technologies. Not sure what is controversial about this conclusion. It seems like it is a good first step in testing the anecdotal hypothesis of less physical activity contributing to increasing obesity in the U.S.A.
  • How USAID Could Score a Double-Win for Learning and Transparency – Greater sharing of data in development would be a huge plus. This article covers some excellent benefits that could be achieved by sharing the raw data of development interventions. Sarah Rose (author) points to improvements in transparency, and ensuring the quality of reported development results. However, Rose didn’t mention another major benefit, which is the power of cloud sourcing analysis. Development practitioners use their data for their own specific objectives. They collect all of their data usually with a very specific task in mind. However, all this information maybe very valuable for a wide array of other uses. By freely sharing it, who knows what independent data users and analysts might be able to make of data. There are undoubtedly thousands of interesting side-effects of development interventions that may be missed by the development agency. Or entirely new connections observable in the data that the data collectors may never have found.
  • Measuring impact: Keep it clear and simple – Article from Stanford Social Innovation Review. Makes the point of the importance of measurement in development. I do agree on the importance of measurement, I am not as much of a fan of  the KISS (keep it simple stupid) principle. Yes having clear objectives that we can measure progress toward is critical. However, I am less enthusiastic about focusing only on simple metrics. I am concerned only trying to influence things we can easily measure. Entire fields of development could be ignored, especially in areas like institution and political reform. Economic development isn’t easy. Communities are complex systems, looking for simple changes can encourage a sort of checklist mentality that implies development is just a long to-do list. We should instead be thinking about the complex interactions between people and institutions and the incentives that drive both. Certainly there can be simple changes that dramatically change systems, but we shouldn’t just get involved on easily measurable goals. This should instead encourage us to find ways to measure progress on more complex objectives. I have no doubts that simple goals make fund raising much easier, and that it is a lot easier to show that you are achieving your limited objectives. It is less clear if this would lead to overall better development outcomes if applied widely. While I am skeptical on simplicity in development, I wholeheartedly agree with one of the other points made in the article – the need to use longer term budgets in development. Far too many projects are designed around annual and biannual budgets. This isn’t realistic. Change takes time, we need to have development budgets that take this into account and allow for greater stability, such that development agencies can plan long term interventions with the necessary follow up with the knowledge that the funding will be there in the later stages of the project.

Migration is Development

Migration is Development

Very interesting article by Peter Sutherland at Project Syndicate illustrating the importance of considering migration in the overall context of economic development. I think that too often we think of economic development interventions with the idea that the people we are trying to help are fixed in location. I think Sutherland makes the important point that we shouldn’t do this. Migration be it international, or national (read urbanization) is neither a good or bad thing in and of itself. But it is a reality. People are often able and willing to move in search of better economic, political, and social opportunities. When we ignore this reality we limit our ability to create effective and sustainable interventions.

Admitting Failure, Greater Stakeholder Involvement, and Improving Development Outcomes

Since attending a FailFaire a couple years ago, I have been very interested in the challenges of identifying failure, admitting to the failure, and learning from failure. We all make mistakes in our own lives, and professionally. We make mistakes for hundred of reasons, but one of the most common reasons is lack of experience and information. Poor information and understanding of dynamics on the ground in places we try to do development greatly increases our chances of failing. International development is littered with failure, in fact, I think that the number of failed projects greatly outnumber successful projects. However, by not accepting we fail and owning up to these failures we are wasting an amazing body of experiences we could be learning from. Admitting to these failures not only give us opportunities to learn from what has gone wrong, but help us develop greater credibility with our stakeholders who see that we take responsibility for our actions and attempt to right things when they invariably go wrong.

Still, development institutions find it difficult to admit shortcomings in projects…there is a tendency to handle these shortcomings by defending the decisions made rather than showing empathy or directly addressing stakeholders’ concerns. Things have certainly improved, yet the immediate reaction is to be defensive. – Shamiela Mir (World Bank)

This defensiveness Mir points out is a major problem. Trying to pass off responsibility for a project’s failure is natural, but we need to start taking more responsibility for the failure of our projects. We also need to do a better job of involving stakeholders in all stages of the development process, and admitting to failure will make this easier. Increased stakeholder involvement would be beneficial in many ways

  1. Greater involvement from the beginning allows us to better design and target interventions that meet the needs of the beneficiaries we are trying to help
  2. When interventions target needs identified by the beneficiaries there is much higher likelihood of ownership of the project and the results, which should increase the possibility of sustainable development
  3. Keeping open communication during the implementation phase increases the opportunity to hear what is going right as well as what is going wrong. This allows for a more flexible just-in-time mentality where we can better adjust the implementation to local circumstances.
  4. When the beneficiaries are fully involved and own the project, they are more likely to participate in followup studies to assess the long-term effects of our interventions

Further Reading:
I highly recommend checking out the blog post by Mir that I quoted above. She looks at the value of giving apologies, and how owning mistakes could be beneficial to international development.

Not only is this blog post interesting there are some excellent links to other articles about making effective apologies and learning from failures. I particularly liked the following ones:

If you are interested in reading more about improving stakeholder involvement in development I highly recommend checking out Listening to Those Who Matter Most, the Beneficiaries, a recent article from the Stanford Social Innovation Review, by Fay Twersky, Phil Buchanan, and Valerie Threlfall

Urbanization, Poverty and Economic Development

Having spent much of my professional career in Latin America I have been greatly interested in the role that urbanization plays in economic and political development. Many of the trends that I have witnessed in Latin America are playing out all over the developing world. In fact by 2010, cities officially became the home of the majority of the human race, and the rate of urbanization is only increasing.

figure 1
Figure 1 Urban and Rural population (billions) over time
figure 2
Figure 2 Urban share (%)  of global population over time

The graphs above show rural population worldwide plateaued in the first decade of the century, is expected to decrease over the next 40 years (UNPop World Urbanization Prospects 2009). Although the rural population is decreasing globally, poverty is still disproportionately represented in the worlds’ rural periphery. The chart and table below show this urban-rural disparity on the global and regional scale. Globally, just over 301 million people living in extreme poverty (< $1.25/day), about 27% of population in extreme poverty, are living in cities, whereas just over 3 billion (almost half) of the total population live in cities. If we look at the break up of poverty regionally, we see the same disproportionate trend with rural poverty share of total poverty ranging from 27 percent in Latin America (around 20% of total population is rural) to nearly 80 percent in South Asia (70% of total population is rural).

figure 3
Figure 3 Global Population and Total Poverty
figure 4
Figure 4 Rural share of total Poverty

Poverty in general has been declining over the past 30 years, and rural poverty in most regions of the world has declined due to general economic development and rural-to-urban migration. I would like to highlight the importance of migration to reducing poverty. Historically, urbanization has been an important precursor to industrialization. Urbanization’s developmental effect is more than just a matter of population displacement, it is a dramatic and often economically revolutionary process that creates new economic opportunities for those who migrate and for those who to stay put. Growing cities provide industries with large and mobile work force, while developing larger markets that can encourage production at greater scales. Cities have powerful agglomeration effects allowing skills and knowledge to be pooled and employed in a wide range of applications spurring innovation and more rapid economic growth. It is not surprising then that cities are widely considered important engines of growth. Nevertheless, the urbanization story is not one of just improving lives in urban areas. Urbanization also has many potential benefits to rural areas. Some of these benefits were well illustrated in a World Bank blog post from last week (Urbanization is good for rural poverty (at least in India)) and include:

  • Large potential markets for agricultural goods produced in rural areas
  • Increasing demand for more diverse goods leads to more opportunities for diversification and specialization
  • Decreasing rural labor force leads to greater bargaining power for rural labor and higher wages
  • New pools of private rural investment through urban-rural remittances

Urbanization also has many negative effects, especially in cities without sufficient infrastructure to handle the large influxes of new people. Managing this transition is undoubtedly of great interest to many developing countries. In the World Bank post, Massimiliano Cali makes a good point that investments in urban areas should be considered in the overall portfolio of pro-poverty and pro-rural investment. Just investing in rural areas, may lead to missed opportunities to harness powerful economic effects from urbanization. Nevertheless, due to the disproportionate amount of poverty in rural areas it will continue to be critical to invest in pro-poor strategies in rural areas. This doesn’t preclude spending some time and money on facilitating and easing the transition to more urban societies, but it must be balanced by targeted interventions to improve rural productivity. Increasing rural productivity has the double benefit of improving rural livelihoods, and stemming urban migration, buying time for heavily stressed cities to build up necessary infrastructure and social services.

Improving rural productivity of course is no easy matter, but should start with a focus on increasing farm productivity and efficiency. Improving yields and decreasing post-harvest losses would contribute greatly to reducing food insecurity and increasing rural incomes. Income is one of the primary factors in determining a household’s vulnerability. Increased farm-level income can reduce vulnerability to all kinds of shocks, including climate change, while creating great economic opportunity through increased capacity to save, reinvest in the farm, or invest in other human capital or economic ventures. New technologies and management techniques must be developed and implemented for crops in much of the tropics, which have not been focused on in the past. Many current technologies (integrated pest management, no-till, etc.) if adopted could make substantial improvements in agriculture in the developing world and help reduce poverty. However, getting current and new technologies to farmers in the field is time intensive and expensive due to factors such as:

  • Bad infrastructure
  • Lack of access to markets
  • Lack of access to accurate and current data
  • Lack of access to financial services (savings, credit, insurance)
  • Poor access to quality education

The recent food crises have highlighted the need for greater investment in agriculture, and to relook at the agricultural systems in the developing world. I believe this is a good thing, and that agriculture has not received the attention that it may deserve. I would love to see greater investment in agricultural extension, and improving the mechanisms of transfers of knowledge and technology in the developing world. However, I am also concerned that development practitioners working on poverty reduction tend to silo themselves off in either urban or rural spheres, which may lead to development interventions with tunnel vision and miss the urban-rural connections that a more holistic approach might exploit.

The MCC Effect

Does the MCC Effect Exist?

This is an interesting post at the Center for Global Development’s Rethinking US Foreign Assistance Blog summarizing the current body of knowledge to the effects of the Millennium Challenge Corporation’s (MCC) Eligibility Indicators. The blog itself is an interesting quick read, and has a lot of good links to different studies looking at the influence of the MCC in policy application. I would definitely recommend checking out the new study by Parks and Rice that inspired this CGD blog post. While it isn’t  by any means a rigorous impact evaluation it is still an interesting survey looking at how MCC and other international indicators have influenced policy makers and development practitioners. I found two things found in the study particularly interesting.

  • The survey data suggest that the policy influence of external assessments may have more to do with signaling credibility to investors, creditors, and donor agencies than directly influencing specific aid allocation decisions. Parks and Rice 2013. p.3

I thought this was interesting. It means that for many policy makers the indicators created by external agencies like the MCC serve more as a cover of legitimacy to making certain reforms. Meaning, that instead of creating a push for previously unlikely reforms, it is more a facilitating tool that enables or eases the process of gaining legitimacy on public reforms that were already on the table. This doesn’t mean that the indicators don’t have value, it just weakens the effect of external indicators. It changes the counter-factual. No longer are we talking about indicators leading to reforms that wouldn’t have happened prior to the indicators. Instead it is talking about facilitating the implementation of reforms that were already being considered.

  • Politics matter, and politicians like all individuals respond to incentives. 
MCA Policy Influence. (Parks and Rice Feb 2013, p. 4)

MCA Policy Influence. (Parks and Rice Feb 2013, p. 4)

These incentives greatly impact how influential the specific MCC indicators are towards pushing reform. I copied and pasted the following graph from the Parks and Rice study because it illustrates the limitations of external indicators in influencing policy makers when they contradict local socio-political incentives structures. As we can see the more political the indicators get the less readily they are accepted by policy makers in their decision making progress. Does this mean that these political indicators should be dropped? Probably not, but it just suggests that we should be realistic as to how influential external indicators can be, and try to build up awareness of how these indicators are tied to beneficial results that have short-term as well as long term benefits for implementing countries, and perhaps more importantly potential reform implementer.