The high manufacturing cost of sulfoaluminate cement (SAC) has limited its application for a long time. The addition of solid waste in raw material can solve this problem and provide an effective method for green manufacturing of cement. In this study, ferrite-rich SAC (FSAC) clinker was calcined using red mud (RM) in combination with limestone, bauxite and anhydrite. The effects of anhydrite and desulfurization gypsum (DSG) on the hydration and mechanical properties of cement were evaluated. Results show that RM-based FSAC can be calcined at 1300 °C, and the hydration process is closely related to the varieties and quantities of the calcium sulfate set regulator. Anhydrite promotes the early compressive strength of cement minimally, but it can increase the strength to 94.61 MPa in 3 days. DSG can increase the strength to 30 MPa in 4 h, but the strength shrinkage occurs in the later stage. The reaction between different gypsum with water is the most significant effect on the hydration properties of cement. The cement prepared by anhydrite (12 wt%), DSG (3 wt%) and clinker (85 wt%) has the characteristics of fast mechanical strength development in the early hydration and continuous development after 1 day, which proves that the preparation of ferrite-rich sulfoaluminate cement using red mud and DSG is feasible.