Age and growth data are central to management or conservation strategies for any species. Circumstantial evidence suggests that male whale sharks (Rhincodon typus) grow to asymptotic sizes much smaller than those predicted by age and growth studies and consequently, there may be sex-specific size and growth patterns in the species. We tested this hypothesis by using stereo-video and photo-identification studies to estimate the growth rates of 54 whale sharks that were resighted over a period of up to a decade at Ningaloo Reef. We found that male growth patterns were consistent with an average asymptotic total length (TL) of approximately 8–9 m, a size similar to direct observations of size at maturity at aggregation sites world-wide and much smaller than the sizes predicted by earlier modeling studies. Females were predicted to grow to an average asymptotic length of around 14.5 m. Males had growth coefficients of K = 0.088 year–1, whereas limited resighting data suggested a growth coefficient of K = 0.035 year–1 for females. Other data including re-sightings of an individual male over two decades, records of sex-specific maximum sizes of individuals captured in fisheries and data from juveniles growing in aquaria were also consistent with the suggestion of sex-specific growth profiles for the species. We argue that selection for sex-specific growth patterns could explain many of the otherwise enigmatic patterns in the ecology of this species including the tendency of the species to form aggregations of juvenile males in coastal waters.