Since aluminum alloys have the low melting point and high thermal conductivity, friction stir weld (FSW)has been successfully used in the aerospace industry as an alternative to traditional welding methods. In the service of FSWed structures, the residual stress and external load would result in a secondary deformation and residual stress redistribution. Therefore, it is necessary to investigate the effect of residual stress on the FSWed fatigue responses. This paper studied the fatigue crack growth behavior of 2024-T3 and 7075-T6 homogeneous and dissimilar FSWed joints. The finite element model was built to calculate the fatigue crack growth rate of FSWed specimen, and the results were compared with the experiment results. It was demonstrated that residual stress significantly affected on the fatigue crack growth rate. Tensile residual stress promoted the crack growth, and it offset the decrease of fatigue crack growth rate by grain refinement. The numerical results also indicated that the longitudinal residual stress showed the greatest effect on the crack deflection. Under the same welding speed, the fatigue crack growth rate of 2024 increased with the increasing rotational speed. For the same rotational speed, the fatigue crack propagation rate of 7075 decreased with the increasing welding speed.