Driver distraction is a leading cause of accidents. While there has been significant research examining driver performance during a distraction, there has been less focus on how much time is required to recover performance following a distraction. To address this issue, participants in the current study completed a simulated 40-min drive while being presented with distractions. Distractions were followed by a visual Detection Response Task (DRT) to assess participants’ resource availability and potential capacity to respond to hazards, as well as continuous measures of driving performance including their ability to maintain a consistent speed and lane position. We examined recovery for a 40 s period following three types of distraction: cognitive only, cognitive + visual, and cognitive + visual + manual. Since safe driving requires cognitive, visual, and manual resources, we expected recovery to take longer when the distraction involved more of these resources. Consistent with this, each additional level of distraction further slowed DRT response times and increased speed variability during 0–10 s post-distraction. However, DRT accuracy was equally impaired for all conditions during 0–20 s post-distraction, while lane position maintenance from 0 to 10 s post-distraction was only impaired when the distraction included a manual component. In addition, while participants in all three conditions exhibited some degree of post-distraction impairment, only those in the cognitive + visual + manual condition reduced their speed during the time when distracted, suggesting drivers show limited awareness of the potential persistent consequences of distraction.