DIC: “(…) some of the included studies found
delayed vaginal opening and abnormal estrous cycling in rats given
methylphenidate. However, this data cannot be extrapolated to humans.”
ChatGPT 3.5 : “A study conducted on rats showed that
methylphenidate treatment reduced ovarian weight and impaired ovulation
(Dudley et al., 2009). However, extrapolating these findings to humans
is challenging and requires further investigation .”
ChatGPT 4.0: “One study conducted on rats reported that
high doses of methylphenidate led to a decrease in ovarian weight and
follicle number, suggesting potential adverse effects on fertility
(Motabar et al., 2016). However, it is important to note that
this study used doses that were much higher than those typically
prescribed for humans.”
A thorough check of the references provided by both versions of ChatGPT
showed that some of them are hallucinations, i.e. completely made-up;
while the journals and authors exist, there are no papers with the
specific titles in any database we could find. A couple of the
references are real though, and some even quite relevant. However, in
its current version, ChatGPT does not provide an estimate on the
legitimacy of the references, and therefore, you will need to
cross-check every reference to separate the wheat from the chaff, which
is tedious at best and unfeasible at worst.
To make the confusion complete, ChatGPT also reports plausible, but
made-up results: The chatbot replied to our question by presenting
specific but spurious odds ratios including a meaningful 95% confidence
interval, and referred to a cohort study as being a meta-analysis. Also,
the conclusion made by ChatGPT was not the conclusion made in the actual
study7.
The conclusions from the DIC and ChatGPT are overall comparable. To our
relief, admittedly, we found that as clinical pharmacologists, we did a
better job at translating the findings to clinical practice. We also
included information on breastfeeding, which often turns out relevant in
the context of a pregnancy-related question. The AI can answer the
question at hand, but we are able to anticipate other needs from the
inquiring healthcare professional.