Rapid changes in land use and land cover pattern have exerted an irreversible change on different natural resources, and water resources in particular, throughout the world. Khambhat City, located in the Western coastal plain of India, is witnessing a rapid expansion of human settlements, as well as agricultural and industrial activities. This development has led to a massive increase in groundwater use (the only source of potable water in the area), brought about significant changes to land management practices (e.g., increased fertilizer use), and resulted in much greater amounts of household and industrial waste. To better understand the impacts of this development on the local groundwater, this study investigated the relationship between groundwater quality change and land use change over the 2001–2011 period; a time during which rapid development occurred. Water quality measurements from 66 groundwater sampling wells were analyzed for the years 2001 and 2011, and two water quality indicators (NO3− and Cl− concentration) were mapped and correlated against the changes in land use. Our results indicated that the groundwater quality has deteriorated, with both nitrate (NO3−) and chloride (Cl−) levels being elevated significantly. Contour maps of NO3− and Cl− were compared with the land use maps for 2001 and 2011, respectively, to identify the impact of land use changes on water quality. Zonal statistics suggested that conversion from barren land to agricultural land had the most significant negative impact on water quality, demonstrating a positive correlation with accelerated levels of both NO3− and Cl−. The amount of influence of the different land use categories on NO3− increase was, in order, agriculture > bare land > lake > marshland > built-up > river. Whereas, for higher concentration of Cl− in the groundwater, the order of influence of the different land use categories was marshland > built-up > agriculture > bare land > lake > river. This study will help policy planners and decision makers to understand the trend of groundwater development and hence to take timely mitigation measures for its sustainable management.