Background: Workplace violence is a major concern for clinicians worldwide. There has been little data on the epidemiology of workplace violence against frontline clinicians during the COVID-19 pandemic. This study examined the pattern of workplace violence and its association with quality of life (QOL) against frontline clinicians during the outbreak of COVID-19 pandemic in China. Methods: A cross-sectional online study was conducted in China between March 15 and March 20, 2020. Frontline clinicians' experience with workplace violence was measured with six standardized questions derived from the Workplace Violence Scale, while anxiety, depressive, and insomnia symptoms, and QOL were measured using the General Anxiety Disorder Questionnaire, the Patient Health Questionnaire, the Insomnia Severity Index, and the World Health Organization Quality of Life Questionnaire, respectively. Univariate analyses, multivariable logistic regression analyses, and structural equation modeling (SEM) were conducted. Results: A total of 15,531 clinicians completed the assessment; 2,878 (18.5, 95% CI = 17.92–19.14%) reported workplace violence during the outbreak of the COVID-19 pandemic (verbal violence: 16.1%; physical violence: 6.9%). According to multivariable models, key correlates of workplace violence were male gender, longer work experience, higher education level, smoking, working in the psychiatry or emergency department, working in tertiary hospitals, being involved in direct care of infected patients, having infected family/ friends/ colleagues, and frequently using social communication programs. Clinicians working in inpatient departments were less likely to report workplace violence compared to those working in outpatient departments. SEM analysis revealed that both violence and emotional disturbances (anxiety, depression, and insomnia) directly affected QOL (standardized direct effect = −0.031, and −0.566, respectively, P < 0.05), while emotional disturbances partly mediated the association between work violence and QOL (standardized indirect effect = −0.184, P < 0.05). Conclusion: Frontline clinicians were vulnerable to workplace violence during the COVID-19 pandemic. Due to the negative impact of workplace violence on quality of care and clinicians' QOL, health authorities and policymakers should take effective measures to reduce workplace violence against clinicians.