Alfalfa (Medicago sativa L.) is an important forage legume in crop–pasture and animal husbandry systems in the semiarid Loess Plateau of China, but comprehensive studies on forage yield, shoot N and P relationships, and soil water use as affected by nitrogen (N) and phosphorus (P) fertility levels are scarce. The objective of this study was to assess the yield response of alfalfa stands of different ages following N and P fertiliser application in relation to water use in the semiarid region of the Loess Plateau. Field experiments were conducted from 2011 to 2014 in newly-established (young) and well-established (old) rainfed alfalfa stands, planted in 2010 and 2005, respectively, on the semiarid Loess Plateau. Treatments were a factorial combination of two N rates (0 and 70 kg N ha−1) and three P rates (0, 17 and 34 kg P ha−1), applied annually, arranged in a randomized complete block design with three replicates. Forage biomass was measured in mid-July and mid-October each year. The forage yield of the two alfalfa stands increased with N and P fertilisation, but path analysis indicated that it was also affected by rainfall. Water use was greater in the young than the old stand. The water content in the 1–4 m soil layer of the young stand decreased each year from establishment; greater water depletion occurred with N and P fertilisation. In the five-year-old stand, the soil water content changed little over the four growing seasons, and N and P fertilisation did not affect the already depleted soil water content. Shoot P concentration was weakly, but positively, associated with shoot N concentration. The risk of low relative forage yield was greater when the shoot N:P ratio was more than 17. We conclude that N and P fertilisation, particularly P fertilisation, increases forage yield in both young and old rainfed alfalfa stands in a semiarid environment.