Access to clean, reliable drinking water remains one of the biggest challenges in developing countries, and public water services in India are no exception. There, over 150 million people are served by intermittent piped water systems. In many Indian cities, water is available roughly four hours per day; while several cities report that water may flow through pipes to homes and businesses only once every five to ten days.
At the March Blum Center Faculty Salon, Professors Isha Ray of the Energy and Resources Group and Alison Post of the Political Science Department shared their analysis of the effects of the Development Impact Lab-supported social enterprise NextDrop, which designed a mobile phone intervention to alert Indian households via text when to expect water supply.
NextDrop was designed not so much to solve India’s water provision problems but to give Indian citizens back their lost time. Co-founders Thejo Kote, Emily Kumpel, Ari Olmos, Anu Sridharan, and Anish Jhinail piloted the project in 2010 in Hubli-Dharwad, a city of 1.1 million people in the state of Karnataka, where water can take up to eight days to arrive via pipe and faucet. They estimated that individual households lost seven days a year waiting for water and regularly needed to rely on unsafe sources.
With support from Big Ideas and the USAID-supported Development Impact Lab, the NextDrop team moved forward with piloting their service. In its first implementation phase, the team sourced real-time information about the distribution of water from valvemen, the individuals charged with turning water valves on and off in neighborhoods. NextDrop provided these valvemen with in-kind rewards for sending SMS notifications. In turn, NextDrop notified residents in the neighborhood, also via text, that their water would arrive within roughly 30 minutes. After piloting in Hubli-Dharwad, NextDrop began operations in Bangalore and Mysore.
That is where Professors Alison Post, co-director of the Global Metropolitan Studies Program, and Isha Ray, co-director of the Berkeley Water Center, became involved. Post and Ray worked with NextDrop in Bangalore so that the efficacy of its solution could be evaluated through a randomized control trial (RCT).
Post explained that the research design for the NextDrop RCT involved selecting a mixed-income study site that was representative of Bangalore’s demographics. The aim of the RCT was to capture the extent to which 3,000 households in the study area benefited from NextDrop’s system.
“We had several reasons to anticipate positive impacts on household welfare, particularly for household members charged with managing water supply,” said Post. “We hypothesized that improved predictability of water would decrease water wait times, and free up time for other tasks, earnings, community and family events. Additionally, we predicted that the intervention would have a positive psychological impact by decreasing stress related to water scarcity. We also expected receipt of NextDrop notifications to increase the frequency with which citizens contacted the utility directly with service problems, rather than going to informal intermediaries. These impacts were expected to be most notable among low-income households.”
However, the results of the RCT revealed a very different story. The study—two years in planning and execution—showed NextDrop’s SMS services had a null-to-modest impact on household welfare.
“The most evident program impact was a modest reduction in stress levels related to managing household water supply among low-income households,” said Post. “Other than that, there was very little impact.”
Ray and Post then set out to understand why the results were not as positive as expected. A major reason was that the Bangalore valvemen upon which NextDrop’s system depended were not reporting accurate water valve opening and closing times. During the Hubli-Dharwad pilot, NextDrop used in-kind and recognition-based incentives to encourage valvemen to send water release notifications. And since the community was more close knit in Hubli-Dharwad and the city much smaller (by more than 9 million people), NextDrop was able to develop one-on-one relationships with individual valvemen.
However, when NextDrop launched its system in Bangalore, the enterprise dropped its incentive program and asked the city’s water utility to require valvemen to send reports to NextDrop. This new hierarchical reporting system was arguably more sustainable at a larger scale, but proved to be less effective for keeping the valvemen on board. Analyzing their survey data and NextDrop’s internal data with Political Science Ph.D. student Tanu Kumar, Post and Ray found that valvemen reported only 70 percent of the time and 63 percent of the reports were inaccurate.
A parallel ethnographic study with graduate student Christopher Hyun shed light on how Bangalore’s valvemen operated and how they interacted with NextDrop’s information system. Hyun, a development engineering student pursuing a PhD from the Energy and Resources Group, discovered that valvemen in Bangalore generally had limited time to report when they turned on the water. The valvemen were constantly putting out fires—fixing broken pipes and rushing around the city trying to get water to residents with minimal resources and backup. Notifications, if sent at all, were often sent during tea breaks or during other downtime.
Analysis of survey data collected for the RCT revealed an additional reason the NextDrop system was not generating benefits in Bangalore: many women waiting for water at home didn’t own their own cell phones. The devices were often with their husbands at work or with their children at school or doing errands. NextDrop failed to understand a key aspect of its information value chain: the intended beneficiaries of their information didn’t have the means to receive it.
Kumar, Post, and Ray created a causal framework—an “information pipeline” with six nodes to mark where informational interventions can stop working.
“The framework is especially useful for helping practitioners consider all the necessary steps when scaling or replicating a development intervention in a new setting,” said Post. “It points out realistic challenges in a human information chain and shows the many ways in which informational interventions can break down.”
To further understand the RCT results, Ray and Post conducted a literature review, comparing their results across the broader landscape of the development intervention literature, specifically looking at the roles of last-mile human intermediaries. They found a surprising lack of discussion on the topic.
“Prominent studies in development literature seem to omit these key players,” said Ray. “There is little emphasis on the frontline actors and on what motivates them. It is absolutely essential to understand the role of human intermediaries and how drastically the conditions and results of an intervention can change from one setting to the next. Clearly, our RCT results demonstrate a need to place greater emphasis on considering the human element: these critical factors are usually not discussed unless the study failed, but should be taken seriously in all evaluation models of development work.”
—Lisa Bauer
