Fertilization is a common approach to increase or sustain soil fertility, but its impact on microbial biomass and community structure remains controversial, particularly in paddy soils. In this study, we investigated the effect of different long-term fertilization strategies, beginning in 1986, namely no fertilization, mineral fertilization, mineral fertilization combined with rice straw or chicken manure, on microbial biomass and community composition at four soil depths (0–10, 10–20, 20–30, and 30–40 cm). The extracted soil phospholipid fatty acids (PLFAs) were pooled into gram-positive (G+) bacteria, gram-negative (G−) bacteria, fungi, and actinomycetes groups. Results showed that irrespective of the fertilization type, the abundance of PLFAs decreased with soil depth in the following order due to nutrient decrease along soil profiles: fungi > G− bacteria > G+ bacteria > actinomycetes. Mineral fertilization induced G+ bacteria more than G− bacteria and actinomycetes, which suggested that the inorganic nutrients in mineral fertilizers are utilized more by G+ bacteria than by other microbial groups. Partial replacement of mineral fertilizer with manure further stimulates G+ bacteria at all depths. Redundancy analysis showed obvious microbial separation at the 0−20 and 20−40 cm soil depths due to the rhizodeposition effect and also revealed that the microbial communities were significantly correlated with nutrient content (soil organic carbon and available N) and pH. Overall, our findings highlight microbial community shifts due to different fertilizer types, which provides basic information for understanding how substrate availability controls the structure of soil microbial communities in paddy soil systems.