Pediatric posterior fossa tumors (PFTs) are successfully treated in approximately 70% of patients. However, most survivors experience long-term working memory impairment. The present study examined whether the parameters of diffusion MRI tractography could serve as working memory impairment biomarkers in 60 pediatric PFT survivors. Participants were at least 5 years post-treatment and had received treatment appropriate for their age and type of tumor. Groups included irradiated PFT had undergone radiotherapy, nonirradiated PFT had not, and age, sex, and handedness-matched healthy controls. All participants underwent a cognitive assessment and multimodal MRI including a diffusion MRI sequence. We combined fMRI data collected from the Human Connectome Project database with the acquired diffusion MRI data, to extract the working memory tract and determine tractography parameters for quantitative insights. Participants in the irradiated PFT group exhibited reduced tract volume, fiber density, fiber connectivity, mean length of streamlines, and number of streamlines, compared with both nonirradiated PFT and control groups. Participants in the nonirradiated PFT group also exhibited reduced fiber density, number of streamlines, and mean curvature of streamlines, compared with controls. Poorer working memory scores for the irradiated PFT group correlated with lower tract volume, fiber density, and number of streamlines for verbal working memory. Additionally, these lower scores correlated with reduced fiber density, mean length of streamlines, and number of streamlines for visual working memory. These tractography parameters could serve as biomarkers of working memory deficits and shed light on the detrimental impact of radiotherapy on the working memory tract.