Alfalfa (Medicago sativa L.) plays an important role in crop–livestock mixed farming on marginal land in the semiarid Loess Plateau. However, the duration, yield performance and water use of long-term alfalfa stands and choice of appropriate subsequent crops are not clear. A 5-year field experiment was conducted at Zhonglianchuan, Gansu Province, China from 2001 to 2005. Productivity and water use were determined and compared between (1) three alfalfa stands that were 1–5 (A1–5), 6–10 (A6–10) and 11–15 (A11–15) years old during the trial; (2) alfalfa using conventional cultivation and a water-harvesting technique (RA1–5); and (3) conventional crop rotation (CK) and four 5-year crop sequence rotations sown after 10-year-old alfalfa had been ploughed, being millet–wheat–potato–pea–potato (MWLPL); millet–corn–corn–wheat–wheat (MCCWW); millet–potato–wheat–corn–corn (MLWCC) and millet–fallow–pea–potato–pea (MFPLP). Forage yield peaked in 7-year-old alfalfa (5740 kg ha−1), but 9-year-old alfalfa had the maximum forage yield profit (4477 kg ha−1 y−1) in terms of whole growing years. Soil water use efficiency (WUES in terms of forage yield and soil water use) of alfalfa increased dramatically up to the 11th year, and then leveled off from year 12 to 15. Forage yield and WUEB/ET (WUE in terms of aboveground biomass and evapotranspiration) of alfalfa were significantly higher using water harvesting compared with conventional cultivation, but were significantly lower than CK. Soil water content did not change in CK as stand age increased, but it decreased in conventional alfalfa stands. After 10 years of alfalfa, a fallow year was not necessary before planting annual crops as soil water was greatly restored after sowing subsequent annual crops. Yield of some crops in the four crop sequence rotations did not differ significantly from CK. MWLPL and MLWCC had more aboveground biomass than MCCWW and MFPLP but the choice of crop sequence needs to be further considered.