Epidemiological studies have confirmed that long-term exposure to road traffic noise can cause cardiovascular diseases (CDs), and when noise exposure reaches a certain level, the risk of related CDs significantly increases. Currently, a large number of Chinese residents are exposed to high noise exposure, which could greatly increase the risk of cardiovascular disease. On the other hand, relevant studies have found that people with high noise sensitivity are more susceptible to noise. And it is necessary to pay more attention to the high noise-sensitive people. This study investigated the acute physiological effect of different noise-sensitive groups by indoor-level noise stimulus experiments under laboratory conditions, by observing heart rate variability (HRV) indicators, including standard deviation of NN intervals (SDNN), low frequency/high frequency (LF/HF), and heart rate (HR). The results showed that (a) there was no significant difference in HRV between the high-sensitive group and the low-sensitive group at the physiological baseline and the different stimulating noise levels. (b) Then, based on the theory of cumulative effect of noise proposed by WHO Regional Office for Europe, non-significant but observable differences between groups were further discussed. By analyzing differences of the variation trends and the within-group significant changes of SDNN and HR between the two groups, the results tended to show that the high-sensitive group is more affected by road traffic noise. In addition, the values of SDNN and HR showed observable between-group differences at 55 dB (A) and 65 dB (A) which corresponding to the SPL associated with a significantly increased risk of cardiovascular disease concerned by epidemiological studies. According to the cumulative effect theory (WHO), these differences in HRV caused by short-term noise stimulation may have the potential to produce physiological response and lead to between-groups differences in prevalence after long-term recurrent effect, and deserve attention and further research.