We report the observation of g-factors of natural paramagnetic impurities in a pure synthetic quartz crystal at milli-Kelvin temperatures. Measurements are made by performing spectroscopy using multiple high-Q whispering gallery modes sustained in the crystal. Extreme sensitivity of the method at low temperatures allows the determination of natural residual impurities introduced during the crystal growth. We observe g-factors that significantly differ from integer multiples of the electron g-factor in vacuum, and with values of up to 7.6, which reveals much stronger coupling between impurities and the crystal lattice than in previous studies. Both substitutional and interstitial ions are proposed as candidates for the observed interactions. © 2013 AIP Publishing LLC.