Background: There is ongoing debate about whether a decline in body mass represents a true risk factor for dementia, whether it is a phenotypic marker of incipient dementia, or perhaps a marker of another process that increases dementia risk. This study was designed to determine if changes in body mass index (BMI) in later life are associated with hazard of incident dementia over a follow-up period of up to eight years. Methods: Method followed was a prospective cohort study of 4,181 men aged 65–84 years, resident in Perth, Australia. The exposure of interest was change in BMI measured between 1996–1998 and 2001–2004. The outcome was incident dementia, established using the Western Australia Data Linkage System until 2009. We used Cox regression models to establish crude and adjusted hazard of dementia for change in BMI. Results: Compared with men with a stable BMI, those with a decrease in BMI >1 kg/m2 had a higher adjusted hazard of dementia (hazard ratio (HR) = 1.89, 95% CI = 1.32–2.70). The cumulative hazard of dementia over follow-up for changes in BMI was greatest for men with a decrease in BMI >1 kg/m2; this trend was apparent for men in all BMI categories (underweight, normal, overweight, obese). A reverse “J-shaped” association between BMI change and incident dementia was observed, with the lowest dementia rate being for men whose BMI remained stable. Conclusions: Men who maintained a stable body mass had the lowest incidence of dementia. Further studies are needed to clarify causality and assess feasibility of interventional studies to preserve body mass in aging men.