Relationships between sex-specific floral traits and endogenous phytohormones associated with altitude are unknown particularly in dioecious trees. We thus examined the relationships between floral morphology or biomass and phytohormones in male and female flowers of dioecious Populus cathayana populations along an altitudinal gradient (1,500, 1,600, and 1,700 m above sea level) in the Xiaowutai Nature Reserve in northern China. The female and male flowers had the most stigma and pollen at 1,700 m, the largest ovaries and least pollen at 1,500 m, and the smallest ovaries and greater numbers of anthers at 1,600 m altitude. The single-flower biomass was significantly greater in males than in females at 1,600 or 1,700 m, but the opposite was true at 1,500 m altitude. The biomass percentages were significantly higher in anthers than in stigmas at each altitude, while significantly greater gibberellin A3 (GA3), zeatin riboside (ZR), indoleacetic acid (IAA), and abscisic acid (ABA) concentrations were found in female than in male flowers. Moreover, most flower morphological traits positively correlated with IAA in females but not in males. The biomass of a single flower was significantly positively correlated with ABA or IAA in males but negatively with ZR in females and was not correlated with GA3 in both females and males. Our results demonstrate a distinct sexual adaptation between male and female flowers and that phytohormones are closely related to the size, shape, and biomass allocation in the pollination or fertilization organs of dioecious plants, although with variations in altitude.