Independent studies have shown that candidate genes for dyslexia and specific language impairment (SLI) impact upon reading/language-specific traits in the general population. To further explore the effect of disorder-associated genes on cognitive functions, we investigated whether they play a role in broader cognitive traits. We tested a panel of dyslexia and SLI genetic risk factors for association with two measures of general cognitive abilities, or IQ, (verbal and non-verbal) in the Avon Longitudinal Study of Parents and Children (ALSPAC) cohort (N>5,000). Only the MRPL19/C2ORF3 locus showed statistically significant association (minimum P = 0.00009) which was further supported by independent replications following analysis in four other cohorts. In addition, a fifth independent sample showed association between the MRPL19/C2ORF3 locus and white matter structure in the posterior part of the corpus callosum and cingulum, connecting large parts of the cortex in the parietal, occipital and temporal lobes. These findings suggest that this locus, originally identified as being associated with dyslexia, is likely to harbour genetic variants associated with general cognitive abilities by influencing white matter structure in localised neuronal regions.