PCORnet Q&A:
How does external validity come into play when planning and
conducting projects in the database? PCORnet research teams take a
variety of different approaches depending on the clinical question at
hand. They often specifically discuss the degree to which in-network
study populations are representative of the overall population of
interest, which can vary substantially based on the goals of the
research. This representativeness can include demographic factors,
comorbidities, health care delivery-level factors, and broader societal
factors. If new populations are being recruited into pragmatic trials or
observational cohorts within PCORnet, the study coordinators may either
target the overall PCORnet population or more specific recruiting
targets (such as specific underserved populations). Finally, just like
Sentinel and CNODES, differences between study sites and recruitment
groups are frequently explored when different associations or effects
are observed within nodes of the network.
What target populations, if any, underlie most analyses? Because
of the broad mission of PCORnet, specific target populations vary. That
said, most target populations are some subset of patients residing in
the United States with access to healthcare.
Are there ways to generalize the findings of the nodes to the
network? There are some existing ways to generalize site-specific
estimates, like meta-analyzing the estimates after checking for
heterogeneity. The focus is generally on estimating a network-wide
treatment effect. How these methods are applied varies depending on the
research team conducting the study and their importance for the clinical
question at hand.
How easily can node-specific estimates be transported between
nodes or to external populations? Currently, there are no out-of-the
box solutions for transporting estimates in this fashion. That said,
research groups can generate (if analyzing individual-level data) or
request (if the PCORnet Coordinating Center is generating aggregate
tables) cross-tabulation tables to stratify queries based on potential
effect measure modifiers.
Are choices ever made to maximize target validity, rather than
internal validity or precision? Target validity is frequently a major
consideration when PCORnet’s partner networks conduct large distributed
pragmatic trials. The diversity of the data and the direct relationships
with clinicians enable researchers to assess this target validity and
perform in-depth evaluation of study results within participating sites.