Gravitational wave (GW) data can be used to test the parity symmetry of gravity by investigating the difference between left-hand and right-hand circular polarization modes. In this article, we develop a method to decompose the circular polarizations of GWs produced during the inspiralling stage of compact binaries, with the help of stationary phase approximation. The foremost advantage is that this method is simple, clean, independent of GW waveform, and is applicable to the existing detector network. Applying it to the mock data, we test the parity symmetry of gravity by constraining the velocity birefringence of GWs. If a nearly edge-on binary neutron-stars with observed electromagnetic counterparts at 40 Mpc is detected by the second-generation detector network, one could derive the model-independent test on the parity symmetry in gravity: the lower limit of the energy scale of parity violation can be constrained within O(10 4eV).