Evidence shows that participants performing a continuous visual categorization task respond slower following the presentation of a task-irrelevant sound deviating from an otherwise repetitive or predictable auditory context (deviant sound among standard sounds). Here, for the first time, we explored the role of the environmental context (instrumentalized as a task-irrelevant background picture) in this effect. In two experiments, participants categorized left/right arrows while ignoring irrelevant sounds and background pictures of forest and city scenes. While equiprobable across the task, sounds A and B were presented with probabilities of.882 and.118 in the forest context, respectively, and with the reversed probabilities in the city context. Hence, neither sound constituted a deviant sound at task-level, but each did within a specific context. In Experiment 1, where each environmental context (forest and city scene) consisted of a single picture each, participants were significantly slower in the visual task following the presentation of the sound that was unexpected within the current context (context-dependent distraction). Further analysis showed that the cognitive system reset its sensory predictions even for the first trial of a change in environmental context. In Experiment 2, the two contexts (forest and city) were implemented using sets of 32 pictures each, with the background picture changing on every trial. Here too, context-dependent deviance distraction was observed. However, participants took a trial to fully reset their sensory predictions upon a change in context. We conclude that irrelevant sounds are incidentally processed in association with the environmental context (even though these stimuli belong to different sensory modalities) and that sensory predictions are context-dependent.