The Indian Ocean has warmed rapidly over the last half of the 20th century, with widespread effects on regional weather, and global climate. Determining the causes of the observed warming is challenging due to the lack of a long instrumental record of interior ocean temperature, leaving uncertainty around the active physical mechanisms and the role of decadal variability. Here we utilize unique temperature observations from three historical German oceanographic expeditions of the late 19th and early 20th centuries: SMS Gazelle (1874–1876), Valdivia (1898–1899), and SMS Planet (1906–1907). These observations reveal a mean 20th century ocean warming that extends over the upper 750 m, and a spatial pattern of subsurface warming and cooling consistent with a 1°–2° southward shift of the ocean gyres. These interior changes occurred largely over the last half of the 20th century, providing observational evidence for the acceleration of a multidecadal trend in subsurface Indian Ocean temperature.