The spatial distribution of population and economic activities has important impacts on both economic growth and the environment. This paper uses a slack-based measure to estimate the total factor environmental efficiency (TFEE) of 286 Chinese prefectural-and-above cities for the period 2002–2013. In particular, the relationship between city size and TFEE is investigated. The findings also show an inverted U-shaped relationship between TFEE and city size, which implies an optimal city size of 16.68 million residents in China. According to this estimate, most Chinese cities may be undersized due to the migration restrictions of the hukou registration system and, hence, suffer from great environmental efficiency losses. The estimated low average TFEE value of Chinese cities also suggests the large potential for efficiency improvement. Thus, government policies should focus on relaxing migration restrictions and encouraging the development of large cities.