China is one of the countries most impacted by desertification, with Gansu Province in the northwest being one of the most affected areas. Efforts have been made in recent decades to restore the natural vegetation, while also producing food. This has implications for the soil carbon sequestration and, as a result, the country’s carbon budget. Studies of carbon (C) dynamics in this region would help to understand the effect of management practices on soil organic carbon (SOC) as well as aboveground biomass (ABVG), and to aid informed decision-making and policy implementation to alleviate the rate of global warming. It would also help to understand the region’s contribution to the national C inventory of China. The CENTURY model, a process-based model that is capable of simulating C dynamics over a long period, has not been calibrated to suit Gansu Province, despite being an effective model for soil C estimation. Using the soil and grassland maps of Gansu, together with weather, soil, and reliable historical data on management practices in the province, we calibrated the CENTURY model for the province’s grasslands. The calibrated model was then used to simulate the C dynamics between 1968 and 2018. The results show that the model is capable of simulating C with significant accuracy. Our measured and observed SOC density (SOCD) and ABVG had correlation coefficients of 0.76 and 0.50, respectively, at p < 0.01. Precipitation correlated with SOCD and ABVG with correlation coefficients of 0.57 and 0.89, respectively, at p < 0.01. The total SOC storage (SOCS) was 436.098 × 106 t C (approximately 0.4356% of the national average) and the average SOCD was 15.75 t C/ha. There was a high ABVG in the southeast and it decreased towards the northwest. The same phenomenon was observed in the spatial distribution of SOCD. Among the soils studied, Hostosols had the highest SOC sequestration rate (25.6 t C/ha) with Gypsisols having the least (7.8 t C/ha). Between 1968 and 2018, the soil carbon stock gradually increased, with the southeast experiencing the greatest increase.