What is the quality of evidence informing vaccine clinical practice recommendations in Australia?

Background: Vaccine policy and guideline recommendations require high quality evidence. A review of the evidence quality used to inform vaccine clinical practice guidelines could help guide researchers on how to improve the design of their clinical studies to produce evidence of greater value to decision-makers. In Australia, the Australian Technical Advisory Group on Immunisation (ATAGI) develops evidence-based vaccine clinical practice recommendations using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology, a transparent, systematic and methodical framework for developing and presenting summaries of evidence and its certainty. Methods: We summarised the publicly available Australian GRADE assessments for the use of vaccines for prevention of cholera, diphtheria, tetanus and pertussis, human papillomavirus, influenza, meningococcal, pneumococcal, rabies and varicella zoster virus, including the certainty of evidence for each outcome (e.g., effectiveness, immunological or safety outcomes) and overall, in addition to the reasons for downgrade or upgrade of the certainty assessments. Results: Across 25 research questions, 189 separate outcomes were assessed; of these 43 (22.8 %), 38 (20.1 %), 68 (36.0 %) and 40 (21.2 %) were classified as informed by very low, low, moderate and high certainty of evidence, respectively. Overall, 4 (16 %), 10 (40 %), 9 (36 %) and 2 (8 %) research questions across the disease areas had their overall certainty of evidence classified as very low, low, moderate and high, respectively. Certainty of evidence was downgraded for confounding, uncertainty in the effect estimation, and differences between the research questions asked by ATAGI and those answered in the studies. Conclusion: There is an unmet need to improve the quality of evidence available to vaccine policy-makers and National Immunisation Technical Advisory Groups. This could be achieved by improving the design of vaccine trials, in particular improving the precision of statistical estimates, inclusion of relevant subpopulations and ensuring trial endpoints are better aligned with the needs of policy-makers.