Background Epidemiological analyses demonstrate that pneumonia survivors have a higher risk of myocardial infarction than people with similar load of risk factors for atherosclerotic cardiovascular disease (ASCVD) but without pneumonia. This may be due to a higher baseline burden of ASCVD in patients with pneumonia that is not captured by the accounting of known ASCVD risk factors in epidemiological analyses or to unfavorable accelerating effects of pneumonia on atherosclerosis. Methods We analyzed data from the Multi-Ethnic Study of Atherosclerosis. We identified 54 participants that were hospitalized for pneumonia during study follow-up and that also had assessment of coronary artery calcium (CAC, an objective marker of coronary atherosclerotic burden) before and after this hospitalization. We matched them to 54 participants who were not hospitalized for pneumonia but that had CAC assessments at the same study visits as the pneumonia cases. We compared baseline CAC scores and their progression between groups. Results Baseline CAC scores were similar in both groups (median [IQR]; 6.3 [0–356.8] in pneumonia participants vs. 10.8 [0–178.3] in controls; p = 0.25). After a median of 4.8 years, the direction and magnitude of CAC score change, and the slope of CAC score progression between groups was also similar (median change [IQR], 21.8 [0 to 287.29] in participants with pneumonia versus 15.8 [0 to 140.94] in controls, p = 0.28; difference in slope, 7.7, 95% CI -9.0 to 24.6, p = 0.18). However, among participants with high baseline ASCVD risk (i.e. ACC/AHA 10-year risk estimate 7.5%), participants with pneumonia showed a larger increase in CAC scores (median change [IQR]; 159.10 [38.55–407.34] versus 48.72 [0.97–246.99] in controls; p = 0.02) and a trend towards a steeper slope of CAC score progression (difference in slope, 19.7, 95% CI -6.6 to 45.6, p = 0.07). Conclusion Pneumonia may accelerate the progression of atherosclerosis in people with high baseline ASCVD risk.