It has been standard practice for about two decades to compute GPS-based station velocity uncertainties using the apparent noise statistics of the non-linear position residuals rather than assume white noise (WN) behavior. The latter choice would yield unrealistic velocity uncertainties. The most common noise types used are power-law, usually close to flicker noise (FN), over most frequencies mixed with WN at the shortest periods. The complicating impact of offsets in the position time series, mostly caused by equipment changes or tectonic events, has not been fully appreciated. These are far less benign than recently suggested. In addition to contributing a pseudo-random walk noise (RW) component to the velocity errors, estimating offset parameters changes the apparent noise color towards whiter, to the point that FN can no longer be considered the model that objectively best describes the observed GPS position spectra. Spectral power is effectively drained by offsets at periods longer than roughly the mean span between them. This consequently promotes a Gauss-Markov process as the preferred noise model and, importantly, obscures the presence of RW and long-period Earth deformation in the series. Both effects can lead to potentially underestimated velocity uncertainties. The full value of decadal-long GPS time series for geodynamical applications is thereby greatly eroded by recurring offsets, especially when they occur quasi-regularly. In addition, contrary to common assumption, the noise color is generally not fixed with time, but clearly becomes whiter in more recent data. The origin of the colored noise and its whitening over time remain elusive.