We explore the links between elevation variability of the Antarctic Ice Sheet (AIS) and large-scale climate modes. Using multiple linear regression, we quantify the time-cumulative effects of El Niño Southern Oscillation (ENSO) and the Southern Annular Mode (SAM) on gridded AIS elevations. Cumulative ENSO and SAM explain a median of 29% of the partial variance and up to 85% in some coastal areas. After spatial smoothing, these signals have high spatial correlation with those from GRACE gravimetry (r~=0.65 each). Much of the signal is removed by a firn densification model but inter-model differences exist especially for ENSO. At the lower parts of the Thwaites and Pine Island glaciers, near their grounding line, we find the Amundsen Sea Low (ASL) explains ~90% of the observed elevation variability. There, modeled firn effects explain only a small fraction of the variability, suggesting significant height changes could be a response to climatological ice-dynamics.