Empirical Green's function (EGF) retrieval commonly relies on cross-correlating the long-term ambient seismic wavefield that is simultaneously recorded at multiple stations. Recent studies have demonstrated observationally that cross-correlating the coda of ambient noise cross-correlation functions enables reconstruction of the EGFs, regardless of the operating time of the stations. In this study, we examine the feasibility of using the nondiffuse energy (i.e., surface waves) of the ambient noise cross-correlation functions to retrieve EGFs between asynchronous stations. We show that source-receiver interferometry (SRI), which is conventionally applied to reconstruct virtual seismograms between earthquake-station pairs, provides an effective framework to retrieve EGFs between asynchronous stations. Our implementation of SRI exploits the nondiffuse wavefield rather than the scattered coda waves that may be contaminated by incoherent energy under nonideal (e.g., sparse, noisy, and short-duration) network configurations. We demonstrate the robustness of SRI by retrieving asynchronous EGFs and performing seismic tomography between (1) nearby stations and (2) distant temporary arrays from southern Australia. The additional raypaths from asynchronous EGFs provide better illumination of small-scale crustal structures beneath the regional network. In the larger-scale example, involving two asynchronous arrays, SRI offers new constraints to the sparsely sampled region along the continental margin of southern Australia. The resulting velocity model agrees well with the independent structural constraints from each seismic array study and sedimentary thickness measurements. This study demonstrates that SRI is a promising tool for integrating seismic arrays operated at different times and can greatly benefit the efforts of improving data coverage and resolution in seismic imaging.