Bone grafts are commonly used for the treatment of critical sized bone defects. Since the supply of autologous bone is insufficient, allogeneic bone grafts have been used most of the time. However, the poor osteogenic property of allogeneic bone grafts after pretreatment results in delayed union, non-union, or even occasional deformity. Calcium phosphate cement (CPC) is one of the most promising bone filling materials due to its good biocompatibility and similar chemical components as natural bone. However, clinical applications of CPC were hampered by limited osteogenic effects, undesired immune response which results in resorption, and poor mechanical stability in vivo. Magnesium (Mg) has been proven to trigger bone regeneration through modulating cell behaviors of mesenchymal stem cells and macrophages significantly. Unfortunately, the degradation raters of pure Mg and Mg oxide are extremely fast, resulting in early collapse of Mg contained CPC. In this study, we developed a novel magnesium contained calcium phosphate bone cement (Mg-CPC), possessing long-term mechanical stability and osteogenic effects through sustained release of Mg. Furthermore, in vitro studies showed that Mg-CPC had no cytotoxic effects on hBMMSCs and macrophage RAW 264.7, and could enhance the osteogenic differentiation as determined by alkaline phosphate (ALP) activity and calcium nodule staining, as well as suppress the inflammatory as determined by expression of anti-inflammatory cytokine IL-1RA. We also found that Mg-CPC promoted new bone formation and bone maturation in vivo. These results suggest that Mg-CPC should be a good substitute material for bone grafts in clinical use.