In temperate and boreal forests, competition for soil resources between free-living saprotrophs and ectomycorrhizal (EcM) fungi has been suggested to restrict saprotrophic fungal dominance to the most superficial organic soil horizons in forests dominated by EcM trees. By contrast, lower niche overlap with arbuscular mycorrhizal (AM) fungi could allow fungal saprotrophs to maintain this dominance into deeper soil horizons in AM-dominated forests. Here we used a natural gradient of adjacent forest patches that were dominated by either AM or EcM trees, or a mixture of both to determine how fungal communities characterized with high-throughput amplicon sequencing change across organic and mineral soil horizons. We found a general shift from saprotrophic to mycorrhizal fungal dominance with increasing soil depth in all forest mycorrhizal types, especially in organic horizons. Vertical changes in soil chemistry, including pH, organic matter, exchangeable cations, and extractable phosphorus, coincided with shifts in fungal community composition. Although fungal communities and soil chemistry differed among adjacent forest mycorrhizal types, variations were stronger within a given soil profile, pointing to the importance of considering horizons when characterizing soil fungal communities. Our results also suggest that in temperate forests, vertical shifts from saprotrophic to mycorrhizal fungi within organic and mineral horizons occur similarly in both ectomycorrhizal and arbuscular mycorrhizal forests.