Context. Long gamma-ray bursts (GRBs) are produced during the dramatic deaths of massive stars with very short lifetimes, meaning that they explode close to the birth place of their progenitors. Over a short period they become the most luminous objects observable in the Universe, being perfect beacons to study high-redshift star-forming regions. Aims. We aim to use the afterglow of GRB 161023A at a redshift z = 2.710 as a background source to study the environment of the explosion and the intervening systems along its line of sight. Methods. For the first time, we complement ultraviolet (UV), optical and near-infrared (NIR) spectroscopy with millimetre spectroscopy using the Atacama Large Millimeter Array (ALMA), which allows us to probe the molecular content of the host galaxy. The X-shooter spectrum shows a plethora of absorption features including fine-structure and metastable transitions of Fe, Ni, Si, C, and O. We present photometry ranging from 43 s to over 500 days after the burst. Results. We infer a host-galaxy metallicity of [Zn/H] = -1.11 ± 0.07, which, corrected for dust depletion, results in [X/H] = -0.94 ± 0.08. We do not detect molecular features in the ALMA data, but we derive limits on the molecular content of log(NCO/cm-2) < 15.7 and log(NHCO+ /cm- 12, which are consistent with those that we obtain from the optical spectra, log(NH2 /cm-2)< 15.2 and log(NCO/cm-2) < 14.5. Within the host galaxy, we detect three velocity systems through UV, optical and NIR absorption spectroscopy, all with levels that were excited by the GRB afterglow. We determine the distance from these systems to the GRB to be in the range between 0.7 and 1.0 kpc. The sight line to GRB 161023A shows nine independent intervening systems, most of them with multiple components. Conclusions. Although no molecular absorption was detected for GRB 161023A, we show that GRB millimetre spectroscopy is now feasible and is opening a new window on the study of molecular gas within star-forming galaxies at all redshifts. The most favoured lines of sight for this purpose will be those with high metallicity and dust.