Despite many studies explored the effect of livestock grazing on plant communities, the response of species composition and diversity to livestock grazing in arid rangelands remain ambiguous. This study examined the effects of livestock grazing on plant communities in arid steppe rangelands of North Africa. Plant diversity of annual species, perennial species and all species combined was measured and compared between grazed and grazing-excluded areas. We also examined the relative importance of species turnover and community nestedness. Moreover, the effects of livestock grazing on beta diversity at local among transects and landscape among sites scales were examined using the multiplicative diversity partitioning. Results revealed that livestock grazing significantly decreased the alpha diversity of all species combined and the diversity of annual plants. Livestock grazing induced a shift in plant community composition where most of species composition variation (~74%) was due to infrequent species replacement ‘turnover’ between the two management types rather than nestedness (~26%). Results revealed also that among transects, beta diversity was higher in grazed steppes than in grazing-excluded steppes. Whereas, among sites, beta diversity was lower in grazed steppes compared to grazing-excluded steppes. These findings suggest that livestock grazing in arid steppe rangelands increases the variation in plant species composition at a local spatial scale and engenders vegetation homogeneity at landscape spatial scale. Therefore, the implementation of appropriate management practices such as short-term grazing exclusion is mandatory to prevent these ecosystems from large scale biotic homogenization.