Drought is one of the main factors restricting the productivity of soybean (Glycine max L.), especially in arid and semi-arid areas where the crop may suffer from frequent and severe drought events unless irrigated. The aim of the present study was to determine the plant responses to water deficits in four landraces and four modern soybean cultivars that differed in yield in rainfed water-limited environments to determine the characteristics associated with yield under drought in the field and under a rainout shelter. During a slow progressive soil drying cycle, the stomata began to close at higher soil water contents (SWC) of 65–67% vs 61–63%, photosynthesis began to decrease at higher SWC of 60–62% vs 52–54% and leaf hydraulic conductance fell by 20% at higher SWC of 50–67% vs 36–48%, while the leaf relative water content began to decrease at lower SWC of 30–35% vs 38–39% in modern cultivars than landraces, respectively, indicating that the modern cultivars exhibited greater isohydric behaviour than the landraces. Water use after flowering was lower in the modern cultivars, consistent with the conservative leaf physiology, but yield was not correlated or negatively correlated with water use so the higher yield and water use efficiency for grain in the modern cultivars was associated with lower water use and higher distribution of biomass to grain, i.e., a higher harvest index. We conclude that breeders of modern high-yielding cultivars of soybean for rainfed semiarid environments have selected for lower biomass and leaf area, larger grain size and higher harvest index, while also selecting for stronger isohydric behaviour and water-conserving traits.