Mammograms are X-ray images of the breast which are used to detect breast cancer. When mammograms are analyzed by computer, the pectoral muscle should preferably be excluded from processing intended for the breast tissue. For this and other reasons, it is important to identify and segment out the pectoral muscle. In this paper, a new, adaptive algorithm is proposed to automatically extract the pectoral muscle on digitized mammograms; it uses knowledge about the position and shape of the pectoral muscle on mediolateral oblique views. The pectoral edge is first estimated by a straight line which is validated for correctness of location and orientation. This estimate is then refined using iterative "cliff detection" to delineate the pectoral margin more accurately. Finally, an enclosed region, representing the pectoral muscle, is generated as a segmentation mask. The algorithm was found to be robust to the large variations in appearance of pectoral edges, to dense overlapping glandular tissue, and to artifacts like sticky tape. The algorithm has been applied to the entire Mammographic Image Analysis Society (MIAS) database of 322 images. The segmentation results were evaluated by two expert mammographic radiologists, who rated 83.9% of the curve segmentations to be adequate or better.