Images from the Mars Science Laboratory (MSL) mission of lacustrine sedimentary rocks of Vera Rubin ridge on Mt. Sharp in Gale crater, Mars, have shown stark color differences ranging from red to purple to gray. Because these color differences cross-cut stratigraphy, they are likely due to diagenetic alteration after deposition of the sediments. However, the chemistry and timing of the fluid interactions is unclear. Determining how diagenetic processes may have modified chemical and mineralogical signatures of ancient environments is critical for understanding the past habitability of Mars and the goals of the MSL mission. Here we use visible/near-infrared spectra from Mastcam multispectral images and ChemCam to determine the mineralogical origins of color changes in the ridge. We show that color variations are consistent with changes in spectral properties related to the crystallinity, grain size, and texture of hematite. Coarse-grained gray hematite dominates in the gray patches and is present in the purple areas, while nanophase and fine-grained red crystalline hematite are present in the red and purple areas. We hypothesize that these differences were caused by grain size coarsening of hematite by diagenetic fluids, as observed in terrestrial analogs with similar color variations. In this model, early primary reddening by oxidizing fluids near the surface was followed during or after burial by bleaching to form the gray patches, with limited secondary reddening after exhumation. Understanding the spectral signatures of these diagenetic processes will help in the future to identify less altered areas with a higher likelihood of organic preservation.