The process of land urbanization may result in a great change in land use structure, land use intensity, and efficiency, which could further lead to an increase in carbon dioxide (CO2) emissions. Despite rich literature on the link between urbanization and CO2 emissions, the mechanism through which land urbanization promotes CO2 emissions reductions has not been fully investigated. To address this gap, this study theoretically and empirically explores the mechanism of land urbanization’s influence on CO2 emissions by integrating land use optimization and high-quality industrial development into a unified framework. Firstly, the theoretical mechanism analysis indicates that low-level industrial development and land use management promote the increase of CO2 emissions per unit of land at the extensive land use stage; however, high-quality industrial development and land use optimization lower CO2 emissions per unit of land at the intensive land use stage. Subsequently, a STIRPAT model and a spatial adaptive semi-parametric model are employed to verify the relationship between the land urbanization rate and total CO2 emissions. The results indicate that the land urbanization rate and total CO2 emissions present an inverted U-shaped relationship. In addition, the mediating mechanism of the advanced industrial structure, CO2 emissions per unit of GDP, and CO2 emissions per unit of land, are studied using the mediating effect model. Results indicate that CO2 emissions reduction can be achieved by promoting the advanced industrial structure, reducing CO2 emissions per unit of GDP or reducing CO2 emissions per unit of land. Ultimately, this study showed that the Chinese government may reduce CO2 emissions by promoting land use structure optimization, land use intensity regulation, land use efficiency improvement, and adjusting energy consumption structure, upgrading industrial structure, and promoting emission efficiency technologies.