Rayleigh scattering in test masses can introduce noise and reduce the sensitivity of laser interferometric gravitational wave detectors. In this paper, we present laser Rayleigh scattering imaging as a technique to investigate sapphire test masses. The system provides three-dimensional Rayleigh scattering mapping of entire test masses and quantitative evaluation of the Rayleigh scattering coefficient. Rayleigh scattering mapping of two sapphire samples reveals point defects as well as inhomogeneous structures in the samples. We present results showing significant non-uniforin scattering within two 4.5 kg sapphire test masses manufactured by the heat exchanger method.