Radio relics in galaxy clusters are extended synchrotron sources produced by cosmic-ray electrons in the microgauss magnetic field. Many relics are found in the cluster periphery and have a cluster-centric, narrow arc-like shape, which suggests that the electrons are accelerated or reaccelerated by merger shock fronts propagating outward in the intracluster plasma. In the X-ray, some relics do exhibit such shocks at the location of the relic, but many do not. We explore the possibility that radio relics trace not the shock fronts but the shape of the underlying distribution of seed relativistic electrons, lit up by a recent shock passage. We use magnetohydrodynamic simulations of cluster mergers and include bubbles of relativistic electrons injected by jets from the central active galactic nucleus or from an off-center radio galaxy. We show that the merger-driven gas motions (a) can advect the bubble cosmic rays to very large radii and (b) spread the relativistic seed electrons preferentially in the tangential direction - along the gravitational equipotential surfaces - producing extended, filamentary, or sheet-like regions of intracluster plasma enriched with aged cosmic rays, which resemble radio relics. Once a shock front passes across such a region, the sharp radio emission edges would trace the sharp boundaries of these enriched regions rather than the front. We also show that these elongated cosmic-ray features are naturally associated with magnetic fields stretched tangentially along their long axis, which could help explain the high polarization of relics.