The long-wavelength negative gravity anomaly over Hudson Bay coincides with the area depressed by the Laurentide ice sheet during the Last Glacial Maximum, suggesting that it is, at least partly, caused by Glacial Isostatic Adjustment (GIA). Additional contributions to the static gravity field stem from dynamic topography and density anomalies in the subsurface. Previous estimates of the contribution of GIA to the gravity anomaly range from 25 percent to more than 80 percent. However, these estimates did not include uncertainties in all components that contribute to the gravity field. In this study, we develop a forward model for the gravity anomaly. We combine density anomalies, isostatic balance, and non-isostatic contributions from GIA and dynamic topography. The largest uncertainty in the predicted gravity anomaly is due to the lower mantle viscosity; uncertainties in the ice history, the crustal model, the lithosphere-asthenosphere boundary and the conversion from seismic velocities to density are found to have a smaller effect. A preference for lower mantle viscosities >10^22 Pa s is found, in which case at least 60 percent of the observed long-wavelength gravity anomaly can be attributed to GIA. This lower bound on the lower mantle viscosity has implications for models employing a viscosity profile in the mantle, such as models for mantle convection and GIA, and inferences based on these models.