Evidence from literature, including the BRIDGES study, indicates that germline protein truncating variants (PTVs) in FANCM confer moderately increased risk of ER-negative and triple-negative breast cancer (TNBC), especially for women with a family history of the disease. Association between FANCM missense variants (MVs) and breast cancer risk has been postulated. In this study, we further used the BRIDGES study to test 689 FANCM MVs for association with breast cancer risk, overall and in ER-negative and TNBC subtypes, in 39,885 cases (7566 selected for family history) and 35,271 controls of European ancestry. Sixteen common MVs were tested individually; the remaining rare 673 MVs were tested by burden analyses considering their position and pathogenicity score. We also conducted a meta-analysis of our results and those from published studies. We did not find evidence for association for any of the 16 variants individually tested. The rare MVs were significantly associated with increased risk of ER-negative breast cancer by burden analysis comparing familial cases to controls (OR = 1.48; 95% CI 1.07–2.04; P = 0.017). Higher ORs were found for the subgroup of MVs located in functional domains or predicted to be pathogenic. The meta-analysis indicated that FANCM MVs overall are associated with breast cancer risk (OR = 1.22; 95% CI 1.08–1.38; P = 0.002). Our results support the definition from previous analyses of FANCM as a moderate-risk breast cancer gene and provide evidence that FANCM MVs could be low/moderate risk factors for ER-negative and TNBC subtypes. Further genetic and functional analyses are necessary to clarify better the increased risks due to FANCM MVs.