Many countries share an effort to understand the impact of growing urban areas on the environment. Spatial, spectral, and temporal resolutions of remote sensing images offer unique access to this information. Nevertheless, their use is limited because urban surface materials exhibit a great diversity of types and are not well spatially and spectrally distinguishable. This work aims to quantify the effect of these spatial and spectral characteristics of urban surface materials on their retrieval from images. To avoid other sources of error, synthetic images of the historical center of Venice were analyzed. A hyperspectral library, which characterizes the main materials of Venice city and knowledge of the city, allowed to create a starting image at a spatial resolution of 30 cm and spectral resolution of 3 nm and with a spectral range of 365–2500 nm, which was spatially and spectrally resampled to match the characteristics of most remote sensing sensors. Linear spectral mixture analysis was applied to every resampled image to evaluate and compare their capabilities to distinguish urban surface materials. In short, the capability depends mainly on spatial resolution, secondarily on spectral range and mixed pixel percentage, and lastly on spectral resolution; impervious surfaces are more distinguishable than pervious surfaces. This analysis of capability behavior is very important to select more suitable remote sensing images and/or to decide the complementarity use of different data.