Galaxy interactions distort the distribution of baryonic matter and can affect star formation. The nearby Magellanic Clouds are a prime example of an ongoing galaxy interaction process. Here, we use the intermediate-age (similar to 1-10 Gyr) red clump (RC) stars to map the 3D structure of the Small Magellanic Cloud (SMC) and interpret it within the context of its history of interaction with the Large Magellanic Cloud (LMC) and the Milky Way. RC stars are selected from near-infrared colour-magnitude diagrams based on data from the Visible and Infrared Survey Telescope for Astronomy survey of the Magellanic Clouds. Interstellar reddening is measured and removed, and the corrected brightness is converted to a distance, on a star-by-star basis. A flat plane fitted to the spatial distribution of RC stars has an inclination i = 35 degrees-48 degrees and position angle, PA=170 degrees-186 degrees. However, significant deviations from this plane are seen, especially in the periphery and on the eastern side of the SMC. In the latter part, two distinct populations are present, separated in distance by as much as 10 kpc. Distant RC stars are seen in the north of the SMC, and possibly also in the far west; these might be associated with the predicted 'Counter-Bridge'. We also present a dust reddening map, which shows that dust generally traces stellar mass. The structure of the intermediate-age stellar component of the SMC bears the imprints of strong interaction with the LMC a few Gyr ago, which cannot be purely tidal but must have involved ram pressure stripping.