The establishment of agricultural matrices generally involves deforestation, which leads to fragmentation of the remaining forest. This fragmentation can affect forest dynamics both positively and negatively. Since most animal species are affected, certain groups can be used to measure the impact of such fragmentation. This study aimed to measure the impacts of agricultural crops (matrices) on ant communities of adjacent lower montane Atlantic rainforest fragments. We sampled nine forest fragments at locations surrounded by different agricultural matrices, namely: coffee (3 replicates); sugarcane (3); and pasture (3). At each site we installed pitfall traps along a 500 m transect from the interior of the matrix to the interior of the fragment (20 pitfall traps ~25 m apart). Each transect was partitioned into four categories: interior of the matrix; edge of the matrix; edge of the fragment; and interior of the fragment. For each sample site, we measured ant species richness and ant community composition within each transect category. Ant richness and composition differed between fragments and matrices. Each sample location had a specific composition of ants, probably because of the influence of the nature and management of the agricultural matrices. Species composition in the coffee matrix had the highest similarity to its corresponding fragment. The variability in species composition within forest fragments surrounded by pasture was greatest when compared with forest fragments surrounded by sugarcane or, to a lesser extent, coffee. Functional guild composition differed between locations, but the most representative guild was ‘generalist’ both in the agricultural matrices and forest fragments. Our results are important for understanding how agricultural matrices act on ant communities, and also, how these isolated forest fragments could act as an island of biodiversity in an ‘ocean of crops’.