In order to determine the pollution sources and human health risks of metal elements in PM2.5, samples were collected by a large flow particulate matter sampler in the four seasons in 2013, 2015, and 2017 (January, April, July, and October). The mass concentrations of 10 metals (Ti, V, Cr, Mn, Ni, Cu, Zn, As, Cd, and Pb) were analyzed. The sources of heavy metals were identified by Unmix, and the potential non-carcinogenic/carcinogenic risk was evaluated. The influences of local and regional sources were also explored during the high-carcinogenic risk period (HCRP). The wind field and 72 h backward trajectories were performed to identify the potential local and regional sources in HCRP. The results showed that the average annual concentrations of PM2.5 in the urban area of Handan city were 105.14, 91.18, and 65.85 μg/m3 in 2013, 2015, and 2017, respectively. The average daily concentrations of the metals in PM2.5 in January were higher than that of April, July, and October. The average mass concentrations of the 10 heavy metal elements in PM2.5 were 698.26, 486.92, and 456.94 ng·m−3 in 2013, 2015, and 2017, respectively. The main sources of the metals in PM2.5 were soil dust sources, vehicular emissions, coal burning, and industrial activities. The carcinogenic risks of Cr and As were above 1 × 10−6 over the three years. Wind direction analysis showed that the potential local sources were heavy industry enterprises and the economic development zone. The backward trajectory analysis indicated that PM2.5 long transported from Shandong, Henan, and the surrounding cities of Handan had quite an impact on the heavy metals contained in the atmosphere of the studied area. The health risk assessment results demonstrated that the trend for non-carcinogenic risk declined, and there was no non-carcinogenic risk in 2017. However, the carcinogenic risk levels were high over the three years, particularly in January.