Heavy metals with high average background values in soils from karst regions of China have been reported previously. However, the accumulation mechanism of heavy metals in karst soils derived from different parent materials related to the molecular soil chemistry is still unclear. In this study, six soil cores (0–100 cm) from three locations were collected from Hengxian County, a typical karst basin of Guangxi, among which soils of Maling (ML1 and ML2) were derived from the weathering and leaching residues of carbonate rocks, and soils of Yunbiao (YB1 and YB2) and Dengxu (DX1 and DX2) were mainly developed from deposits of Quaternary formations. Soil heavy metal quantification, X-ray diffraction (XRD) analysis, synchrotron radiation-based Fourier transform infrared (SR-FTIR) spectroscopy, and two-dimensional correlation spectroscopy (2DCOS) analysis were combined to study the contents of heavy metals, mineral compositions, and functional assignments in the karst soils. The results showed that the total concentrations of As, Cr, Cd, Cu, Ni, Pb, and Zn in the soils from ML exceeded the risk screening values from the agricultural soil quality standard in China (GB15618-2018), while the contents in the soils from YB and DX were below the risk screening values. The mineral components were significantly different in soils and their clay minerals. ML soils contained more Fe and Mn oxides than YB and DX soils. Furthermore, it was interesting to find extensive existence of chlorite in ML soils (36.73%-39.07%) and their clay minerals (90.49%-90.97%). While illite was the dominant mineral in YB and DX clay fractions (75.08%-100%) and its percentages decreased to 6.06%-35.23% in the soil samples. In addition, ML soils contained the hydroxyl groups with hydrogen bonds (vO-H: 3523, 3450, and 3378 cm−1) and the hydroxyl groups without hydrogen bonds (vO-H: 3698, 3623 cm−1), while YB and DX soils only had the hydroxyl groups without hydrogen bonds (vO-H: 3695, 3622 cm−1). Our study indicated that the accumulation of heavy metals in karst soils from different parent materials was significantly attributed to their molecular soil chemistry. For instance, the higher contents of Fe-Mn oxides and more reactive vO-H (3523, 3450, and 3378 cm−1) and vSi-O (1000 cm−1) assignments could contribute to the higher contents of heavy metals in ML soils compared to the soils from YB and DX.