A data set of concurrent measurements of sea spray aerosol concentration, wind speed, sea state, and air and water temperature was acquired across all sectors of the Southern Ocean during the Antarctic Circumnavigation Expedition (Austral summer 2016/2017). In addition to the established dependence on wind speed, our observations demonstrate that sea spray aerosol concentrations depend on sea state and the stability of the marine boundary layer. Besides driving sea spray emissions, wind speed and sea state strongly influence the deposition onto the ocean surface and thus aerosol lifetime even for smaller particles. Stable atmospheric conditions allow a typical lifetime of up to 4 days, while tropospheric air entrainment in unstable conditions reduces the residence time of sea spray aerosol in the marine boundary layer to less than 2 days.