Susan Davis

By Susan Davis, Executive Director, Improve International

You can lead a horse to water but you cannot make it drink. You can do an evaluation but you cannot make us think.

The good news is there is a proliferation of evaluation databases. Donors like the US Agency for International Development (USAID) and the Norwegian Agency for Development Cooperation (NORAD) and implementing organizations like CARE and Catholic Relief Services (CRS) are publishing evaluation reports online.  This gets them off people’s desks and into the world. (Update: another handy link to multilateral development bank evaluation groups and their reports is here.)

But is anyone learning from them? Evaluations and other reports from many years ago show, for example:

• It has become overwhelmingly clear from both research and field observations (Warford and Saunders, 1976; Elmendorf, 1978; Burton, 1979) that the main obstacle in the use and  maintenance of improved water and sanitation systems is not the quality of technology, but the failure “in qualified human resources and in management and organization techniques, including a failure to capture community interest” (Nieves, 1980). An appalling 35 to 50% of systems in developing countries become inoperable after five years (Imboden, 1977;
  Warford and Saunders, 1976; White et al, 1972) (from USAID, 1981).
• National and central institutions are beginning to recognize that for community management to achieve its promise, long-term nurturing and support will be needed. Water supply and sanitation systems have costs and responsibilities that must be met, whether the systems are operated by local or central authorities (USAID 1992).

Yet, decades later, there are still many organizations that tell their donors that $25 or so can save a life (usually referring simply to the costs of building a water system). And many current budgets and implementation plans still focus on short-term programs for access to water with no plans for long-term monitoring or support when things break (whether by government or local institutions or the organization). And current evaluations show the same problems.

So what will it take to get the horse to drink? Donors typically like to fund tangible things, so time for learning often isn’t considered “paid work.” Furthermore, we are dealing with a big issue – millions or billions still without access.  Who has time to slog through one or more 30-page documents? Even those who do might not have the power to change the way their organizations do business or raise funds.

It seems that in addition to empowering practitioners to learn, educating donors, fundraisers, and the executives at implementing organizations must be part of the solution. One way is to make the evaluations more accessible and digestible.  Washfunders.org is beginning to assemble one-page summaries of key WASH reports and evaluations in its Knowledge Center.  Improve International is investigating ways to digest the information even further: at the organization level and by theme.

Another method to encourage learning is to get donors together. For example, the SustainableWASH folks are planning a March 12 donor gathering in DC where donors can share challenges and solutions.  WASHfunders.org also has a Funders’ Forum and a Funder Toolkit.

Or perhaps we should think more of “horses for courses,” as the Brits say.  Maybe formal, expensive evaluations by outsiders are not useful.  What if we engaged the customer communities (participatory or empowerment evaluation), practitioners peers and donors in evaluation? Imagine the learning! This is actually what the Accountability Forum is attempting to do. After the pilot in Honduras, COCEPRADIL (the local organization that was evaluated) is addressing the recommendations and asking for new types of funding. At least one horse is drinking!