This study responds to the question of why most of the residents of urban informal settlements are still using unimproved water even after infrastructure for improved water distribution is installed. Huye town in Rwanda was used as a case study. Data was collected using household surveys, focus group discussions with community leaders, and observations of water source premises. Results indicate two main factors which compromised the use of improved water. Firstly, the processes of getting a water pipe connection from an immediate neighbour involved time-consuming negotiations and sometimes a significant cost while public water kiosks were closed most of the time. Secondly, the study observed that the lack of strong community organizations for consistent maintenance of the available public water sources (protected springs) could compromise the quality of water and nullify efforts to protect the sources. The study informs policy makers that extending piped water infrastructure is not enough for residents of urban informal settlements to use it. Further assistance and consistent maintenance of infrastructure is also important.