Passive restoration (without any intervention) has been proposed as an effective strategy for degraded cropland restoration. However, whether the vegetation in abandoned cropland can change towards the desired state and the time needed to reach a stable state are still uncertain. We investigated three abandoned croplands with different recovery times (5, 15 and 20 years) and one natural grassland in each of two different types of grassland (desert steppe and typical steppe) in the agro-pastoral ecotone of northern China to assess the restoration potential of abandoned cultivated grassland. Above- and below-ground productivity as well as species biodiversity increased gradually with increasing recovery time. After 20 years of restoration there was no significant difference between abandoned cropland and natural grassland in the typical grassland site, but above- and below-ground productivity and species biodiversity were still lower in abandoned cropland in the desert steppe site. At the beginning of restoration, the communities were dominated mainly by annual species, especially in the desert steppe. As recovery time increased, the biomass and richness of perennial grasses and forbs increased significantly and replaced annual species as the dominant species. In both desert steppe and typical grassland, species similarity between restored and natural grassland increased over time, suggesting that previously cultivated grassland recovered towards the desired state. Our results indicate that 20 years was long enough for the restoration of previously cultivated grassland in the typical steppe, but more time may be needed in the desert steppe.