Colour vision deficiency (CVD) is a genetic condition that has troubled people for a long time. This study proposes an improved colour-to-grey method for CVD using image segmentation and a colour difference model. In this method, the colour image is first segmented using a region growing method so that each region corresponds to one colour. Next, the colour difference is computed between arbitrary segmented region pairs. Finally, the greyscale image is obtained by minimising a target function. Experimental results show that compared with state-of-the-art colour-to-grey methods, the proposed algorithm can improve the E-score by about 10.99%.