In this study, refuse mudstone, ground granulated blast-furnace slag (GGBS) and red mud were used as the precursor materials for geopolymer. Water glass was used as the alkaline activator for polymer synthesis and standard sand was used as the aggregate in the polymer composites. A series of laboratory tests including unconfined compressive strength (UCS) tests, Brazil tensile strength (BTS) tests, scanning electron microscopy (SEM) tests were carried out for mechanical and microstructural analysis. The coupled influence of the content of refuse mudstone (P) and the content of alkaline activator (Q) was investigated. Results show that it is feasible to synthesize geopolymer using refuse mudstone, GGBS and red mud. The produced geopolymer composites could have the highest UCS of 23.48 MPa (P = 30% and Q = 25%) and the highest BTS of 2.98 MPa (P = 30% and Q = 10%). Due to the inactive crystal phases in refuse mudstone, the UCS of geopolymer composites was increased and the strain at the UCS (εUCS) was increased. The influence of Q on the UCS and εUCS was not evident and it was affected by P. Similarly, the influence of P and Q on the BTS of geopolymer composites was also found to be dependent on each other. Compared with cemented paste backfill composites, the relatively high strength and small permeability of geopolymer composites originated from its compact microstructure. The influence of P and Q on the geopolymer composites was also confirmed by failure mode analysis and SEM analysis at the micro-scale.