In order to determine the causes of kinematic asymmetry in the Hα gas in the SAMI (Sydney- AAO Multi-object IFS) Galaxy Survey sample, we investigate the comparative influences of environment and intrinsic properties of galaxies on perturbation. We use spatially resolved Hα velocity fields from the SAMI Galaxy Survey to quantify kinematic asymmetry (vasym) in nearby galaxies and environmental and stellar mass data from the Galaxy And Mass Assembly survey. We find that local environment, measured as distance to nearest neighbour, is inversely correlated with kinematic asymmetry for galaxies with log (M*/M⊙) > 10.0, but there is no significant correlation for galaxies with log (M*/M⊙) < 10.0. Moreover, lowmass galaxies [log (M*/M⊙) < 9.0] have greater kinematic asymmetry at all separations, suggesting a different physical source of asymmetry is important in low-mass galaxies. We propose that secular effects derived from gas fraction and gas mass may be the primary causes of asymmetry in low-mass galaxies. High gas fraction is linked to high σm/V (where σm is Hα velocity dispersion and V the rotation velocity), which is strongly correlated with vasym, and galaxies with log (M*/M⊙) < 9.0 have offset σm/V from the rest of the sample. Further, asymmetry as a fraction of dispersion decreases for galaxies with log (M*/M⊙) < 9.0. Gas mass and asymmetry are also inversely correlated in our sample. We propose that low gas masses in dwarf galaxies may lead to asymmetric distribution of gas clouds, leading to increased relative turbulence.