Livestock grazing is a major driver shaping the functioning and stability of grasslands. Although previous studies have documented the effect of grazing on grassland stability, whether this effect is scale-dependent remains unclear. Here, we conducted a sheep-grazing experiment in a temperate grassland to test grazing effects on biomass stability across scales and organizational levels. We found that an increase of grazing intensity increased species stability, but it substantially decreased local ecosystem stability due to reduced asynchronous dynamics among species. Moreover, grazing reduced ecosystem stability at larger spatial scales, but to a lesser extent. By decreasing biodiversity within and across communities, grazing impairs the insurance effects of biodiversity and hence the up-scaling of stability from species to ecosystem and further to larger scales. Our study provides the first evidence for the context-dependence of grazing effects on grassland stability via shaping biodiversity and contributes to bridging fine-scale experiments and broad-scale ecosystem management.