Conservation tillage is an important approach to prevent water loss and soil erosion and promote soil fertility that has been adopted widely throughout the world. However, despite promotion of the benefits of conservation tillage, obstacles are still encountered in some regions. A survey of 385 farmer households in the semi-arid Loess Plateau of China was conducted to assess the adoption of conservation tillage (ACT). This investigation was located in two counties that have run conservation tillage demonstrations with wheat for at least eight years. A binary logistic regression model was used to quantify the factors determining whether or not farmers adopt conservation tillage. Farmer's education level, the influence of training, and field demonstrations by agricultural departments had significant positive effects on ACT. Although the adoption rate of conservation tillage in this paper was very high (89%), farmers were reluctant to continue practicing conservation tillage based on their experiences, which is contrary to the expectations of the government. The area available for planting winter wheat and the number of arable plots per household also had significant positive effects on ACT. However, the total cultivated area of land per household had a significant negative impact on ACT. Farmer awareness of conservation tillage technology, the distance from a farmer's house to the nearest agricultural market, and the size of the active labor force in the family had significant negative impacts on ACT. These results will help in the development of more effective and targeted policies to improve the sustainability of farming systems on the semi-arid Loess Plateau.