Large volumes of images are collected by geoscientists using remote sensing platforms. Manual analysis of these images is a time consuming task and there is a need for fast and robust image interpretation tools. In particular the reliable mapping of lithological boundaries is a critical step for geological interpretation. In this contribution we developed an interactive image segmentation algorithm that harnesses the geologist's input and exploits automated image analysis to provide a practical tool for lithology boundary detection, using photographic images of rock surfaces. In the proposed method, the user is expected to draw rough markings to indicate the locations of different geological units in the image. Image segmentation is performed by segmenting regions based on their homogeneity in colour. This results in a high density of segmented regions which are then iteratively merged based on the colour of different geological units and the user input. Finally, a post-processing step allows the user to edit the boundaries. An experiment was conducted using photographic rock surface images collected by a UAV and a handheld digital camera. The proposed technique was applied to detect lithology boundaries. It was found that the proposed method reduced the interpretation time by a factor of four relative to manual segmentation, while achieving more than 96% similarity in boundary detection. As a result the proposed method has the potential to provide practical support for interpreting large volume of complex geological images.