Predicting treatment response or survival of cancer patients remains challenging in immuno-oncology. Efforts to overcome these challenges focus, among others, on the discovery of new biomarkers. Despite advances in cellular and molecular approaches, only a limited number of candidate biomarkers eventually enter clinical practice. A computational modeling approach based on ordinary differential equations was used to simulate the fundamental mechanisms that dictate tumor-immune dynamics and to investigate its implications on responses to immune checkpoint inhibition (ICI) and patient survival. Using in silico biomarker discovery trials, we revealed fundamental principles that explain the diverging success rates of biomarker discovery programs. Our model shows that a tipping point—a sharp state transition between immune control and immune evasion—induces a strongly non-linear relationship between patient survival and both immunological and tumor-related parameters. In patients close to the tipping point, ICI therapy may lead to long-lasting survival benefits, whereas patients far from the tipping point may fail to benefit from these potent treatments. These findings have two important implications for clinical oncology. First, the apparent conundrum that ICI induces substantial benefits in some patients yet completely fails in others could be, to a large extent, explained by the presence of a tipping point. Second, predictive biomarkers for immunotherapy should ideally combine both immunological and tumor-related markers, as a patient’s distance from the tipping point can typically not be reliably determined from solely one of these. The notion of a tipping point in cancer-immune dynamics helps to devise more accurate strategies to select appropriate treatments for patients with cancer.