Manure application to soil has declined globally due to increased availability of mineral fertilizers. However, mineral fertilizer overuse has caused serious consequences for soil quality and the environment. We analyzed the results of 20 long-term field trials (22–32 years, start year [ranged from 1980 to 1990] to 2012) and combined this with a climate change model and soil organic carbon (SOC) model to quantify the importance of manure and mineral fertilizers for grain yield and SOC sequestration across croplands in China. During the past three decades mineral fertilizers have increased grain yield for 91–184% but had minor impact on SOC sequestration (4–16%). In contrast, manure applied with mineral fertilizer increased grain yield by only 6–19% but strongly raised the SOC content (9–39%) compared with mineral fertilizer. Modeling (to the year 2099) indicated that manure used in combination with mineral fertilizers will increase future C sequestration in soils across China by 2,086 Tg C and by 2,482 Tg C based on current net primary productivity with no climate change and on increased net primary productivity with climate change scenarios, respectively. This corresponds to an additional 43–58% increase in C sequestration compared to mineral fertilizers only. The manure efficiency for C sequestration in soil was about 9.6% of C input and decreased with increasing SOC content. To maintain the current SOC content (i.e., 2010), 11 t·ha−1·year−1 fresh manure or 4.8 t·ha−1·year−1 dry maize straw would be required. We conclude that the regular use of manure with mineral fertilizers is essential for the long-term dual functions of soil for food production and SOC sequestration.