There is no consensus on the physical mechanisms controlling the scale at which convective activity organizes near the Equator, where the Coriolis parameter is small. High resolution cloud-permitting simulations of non-rotating convection show the emergence of a dominant length scale, which has been referred to as convective self-aggregation. Furthermore, simulations in an elongated domain of size 12228km x 192km with a 3km horizontal resolution equilibrate to a wave-like pattern in the elongated direction, where the cluster size becomes independent of the domain size. These recent findings suggest that the size of convective aggregation may be regulated by physical mechanisms, rather than artifacts of the model configuration, and thus within the reach of physical understanding. We introduce a diagnostic framework relating the evolution of the length scale of convective aggregation to the net radiative heating, the surface enthalpy flux, and horizontal energy transport. We evaluate these length scale tendencies of convective aggregation in twenty high-resolution cloud-permitting simulations of radiative-convective equilibrium. While both radiative fluxes contribute to convective aggregation, the net longwave radiative flux operates at large scales (1000-5000 km) and stretches the size of moist and dry regions, while the net shortwave flux operates at smaller scales (500-2000 km) and shrinks it. The surface flux length scale tendency is dominated by convective gustiness, which acts to aggregate convective activity at smaller scales (500-3000 km). We further investigate the scale-by-scale radiative tendencies in a suite of nine mechanism denial experiments, in which different aspects of cloud radiation are homogenized or removed across the horizontal domain, and find that liquid and ice cloud radiation can individually aggregate convection. However, only ice cloud radiation can drive the convective cluster to scales exceeding 5000 km, because of the high optical thickness of ice, and the increase in coherence between water vapor and deep convection with horizontal scale. The framework presented here focuses on the length scale tendencies rather than a static aggregated state, which is a step towards diagnosing clustering feedbacks in the real world. Overall, our work underscores the need to observe and simulate surface fluxes, radiative and advective fluxes across the 1km-1000km range of scales to better understand the characteristics of turbulent moist convection.