Aims: The aim of this study was to examine factors including family history, medical history and comorbidities associated with the risk of colorectal cancer (CRC) in young (18–49 years) and middle-age (50–69 years) individuals. Methods: State records were used to identify individuals born in Western Australia between 1945 and 1996, and their first-degree relatives. Individuals in the cohort and their relatives were linked to State cancer registry, hospital and mortality data to identify diagnoses of CRC and other risk factors. The associations between CRC and identified risk factors were examined using multivariable logistic regression. Results: For both young and middle-aged patients, family history of CRC, and a history of smoking, inflammatory bowel disease, liver disease and non-CRC cancer were associated with a significant increase in odds of CRC. In middle-aged patients, having a colonoscopy in the previous 10 years was associated with a reduced odds of CRC regardless of the detection of polyps. However, in young patients only the absence of polyps as confirmed by colonoscopy was associated with a decreased risk of CRC (OR: 0.38, 95%CI: 0.26 – 0.54, p < 0.001). Conclusions: Many of the risk factors associated with CRC were similar in young and middle-aged persons, and should be used to identify high risk young patients for screening. The association between colonoscopy and polyps with CRC was modified by age, likely as the result of routine screening in middle-aged patients.