Autistic traits vary across the general population, and are linked with face recognition ability. Here we investigated potential links between autistic traits and voice recognition ability for personally familiar voices in a group of 30 listeners (15 female, 16–19 years) from the same local school. Autistic traits (particularly those related to communication and social interaction) were negatively correlated with voice recognition, such that more autistic traits were associated with fewer familiar voices identified and less ability to discriminate familiar from unfamiliar voices. In addition, our results suggest enhanced accessibility of personal semantic information in women compared to men. Overall, this study establishes a detailed pattern of relationships between voice identification performance and autistic traits in the general population.