The evolution of groundwater quality in natural and contaminated aquifers is affected by complex interactions between physical transport and biogeochemical reactions. Identifying and quantifying the processes that control the overall system behavior is the key driver for experimentation and monitoring. However, we argue that, in contrast to other disciplines in earth sciences, process-based computer models are currently vastly underutilized in the quest for understanding subsurface biogeochemistry. Such models provide an essential avenue for quantitatively testing hypothetical combinations of interacting, complex physical and chemical processes. If a particular conceptual model, and its numerical counterpart, cannot adequately reproduce observed experimental data, its underlying hypothesis must be rejected. This quantitative process of hypothesis testing and falsification is central to scientific discovery. We provide a perspective on how closer interactions between experimentalists and numerical modelers would enhance this scientific process, and discuss the potential limitations that are currently holding us back. We also propose a data-model nexus involving a greater use of numerical process-based models for a more rigorous analysis of experimental observations while also generating the basis for a systematic improvement in the design of future experiments.