Arbuscular mycorrhizal fungi (AMF) could improve crop yield by nutrient uptake from soil. However, how AMF of topsoil and subsoil respond to long-term differential application of mineral and organic fertilizers and how they are influenced in presence of different crops and growth stages have never been assessed together. Next generation sequencing was applied to profile soil AMF communities in both topsoil and subsoil from a typical arable soil (Eutric Regosol) in the Sichuan Base, southwest China. Soils were collected at anthesis and harvest of winter wheat and maize after 15-years of differential fertilization of an annual maize-wheat rotation (same inputs of nitrogen, N; but different in carbon, C; phosphorus, P; and potassium, K): (i) no-fertilization control (CT), (ii) 100% mineral NPK fertilizer (NPK), (iii) 60% mineral N and 100% mineral PK plus 40% N as crop residues (NPKCR); and (iv) 60% mineral N and 100% mineral PK plus 40% N as pig (organic) manure (NPKOM). Compared to CT, AMF diversity significantly decreased in topsoil, but increased in subsoil for both wheat and maize under NPK, NPKCR and NPKOM. The AMF community composition differed between NPK and CT in both topsoil and subsoil of maize and wheat. Both distance-based redundancy analysis (db-RDA) and structural equation modeling (SEM) suggested that AMF communities in topsoil and subsoil of maize and wheat responded in differences in soil available P, soil organic carbon, fertilization and climate (temperature and/or rainfall). Fertilization and available N, not AMF community, had significantly positive effects on wheat and maize aboveground dry matter and grain yield. Results from the present study shed light on how AMF communities in topsoil and subsoil respond to different soil fertility management over the long-term. Insights of this study are valuable in managing an important part of biological soil fertility.