© 2014. The American Astronomical Society. All rights reserved.. The nature of possible evolutionary pathways between various types of dwarf galaxies is still not fully understood. Blue compact dwarf galaxies (BCDs) provide a unique window into dwarf galaxy formation and evolution and are often thought of as an evolutionary stage between different classes of dwarf galaxies. In this study we use deep optical and near-infrared observations of the underlying hosts of BCDs in order to study the structural differences between different types of dwarf galaxies. When compared with dwarf irregular galaxies of similar luminosities, we find that the underlying hosts of BCDs have significantly more concentrated light distributions, with smaller scale lengths and brighter central surface brightnesses. We demonstrate here that the underlying hosts of BCDs are distinct from the broad continuum of typical dwarf irregular galaxies, and that it is unlikely that most dwarf irregular galaxies can transform into a BCD or vice versa. Furthermore, we find that the starburst in a BCD only brightens it on average by ∼0.8 mag (factor of two), in agreement with other studies. It appears that a BCD is a long-lived and distinct type of dwarf galaxy that exhibits an exceptionally concentrated matter distribution. We suggest that it is this compact mass distribution that enables the strong star formation events that characterize this class of dwarf galaxy, that the compactness of the underlying host can be used as a distinguishing parameter between BCDs and other dwarf galaxies, and that it can also be used to identify BCDs which are not currently experiencing an intense starburst event.