Deep learning has the potential to revolutionize process analytical technology in the pharmaceutical industry. Here, we used Raman spectroscopy-based deep learning strategies to develop a tool for detecting microbial contamination. We built a Raman dataset for microorganisms that are common contaminants in the pharmaceutical industry for Chinese Hamster Ovary (CHO) cells, which are often used in the production of biologics. Using a convolution neural network (CNN), we classified the different samples comprising individual microbes and microbes mixed with CHO cells with an accuracy of 95-100%. The set of 12 microbes spans across Gram-positive and Gram-negative bacteria as well as fungi. We also created an attention map for different microbes and CHO cells to highlight which segments of the Raman spectra contribute the most to help discriminate between different species. This dataset and algorithm provide a route for implementing Raman spectroscopy for detecting microbial contamination in the pharmaceutical industry.