3 | DISCUSSION
For the first 14 CASP experiments, from 1994 through 2020, the issue of
alternative conformations was largely ignored, on the grounds that
methods for determining single structures were so imperfect that such
nuances were not worthwhile. But with the very high single structure
accuracy achieved in CASP14 (1) and the increasing availability of
experimental data on alternative conformations, it was obviously time to
reconsider. A similar conclusion has been reached across the structural
biology community, with greatly increased interest in this area (26). It
was also clear that the successful CASP14 deep learning methods might be
extendable to this problem. After extensive discussion with
experimentalists in the field and help from others, the systems
discussed above were identified as potentially suitable targets for CASP
experiments. It is a small set, and the CASP community had limited time
to prepare modeling pipelines, restricting participation and limiting
methodology. Thus, the results should be regarded as a pilot for
critical assessment in this area, rather than fully establishing the
state of the art. Nevertheless, the results do serve to illustrate three
things: some types of methods that are currently available, how these
perform, and the rich and varied nature of alternative conformations.
As described above, there are notable successes among the results.
Participants were able to reproduce a substantial domain swap caused by
a single mutation in a dimeric enzyme, to identify alternative
conformations of an ABC transporter caused by the state of ligand
binding, and to identify alternative conformations of a small dimer that
result from both environment and sequence differences. Possible
functional motifs in alternative conformations of kinases were also
identified. In all these cases, the AF2-based methods that were broadly
successful in calculating the structures of single proteins (12) and
multimers (13) were used. The methods vary in detail (see references in
(12, 13) for specifics), but all rely on much more extensive sampling of
possible conformations and/or alternative MSAs than the default AF2
protocols (for example those in (27)). A variety of methods for
conformational sampling with AF2 have now been developed (28), and it is
likely that the principles and best procedures will soon become clearer.
For one case, T1109, the mutation-induced domain swap, multiple groups
were able to provide accurate models. That is, not only to sample the
alternative conformation, but to rank them highly. For the different
ligand-induced conformations of the ABC transporter, although all three
distinct conformations were identified by multiple groups, different
groups were successful with different conformations, and generally the
appropriate conformations were not the highest ranked. This is also the
case for the crystal dependent alternative conformations of the
48-residue reduced amino acid set peptide. For alternative conformations
of the kinases, correct versions of functional motifs were present in
the submissions. In these cases, it appears that AF2 could sometimes
sample correct minor-state conformations, presumably with the help of
structures in the training set and/or possible template use. This is
consistent with the concept that these conformations are already
populated to some degree even in the absence of the appropriate
environment or ligand, with conformational selection depending on
conditions (29). In one sense, this is impressive, and promising for the
future. In another, it appears that to robustly achieve full sampling
more extensive computation than standard would be required, and in the
absence of the environmental factors (ligands, crystal environment)
these are (appropriately, given missing environmental factors) not the
highest scoring.
The RNA target in which multiple conformations were considered (R1156),
providing an experimental uncertainty ensemble, is a nice example of the
sort of data that will be needed now that calculated structures have
become so accurate. Even though RNA computed structures are not yet as
accurate as those of proteins, computed ensembles still allowed a model
consistent with the experimental information to be identified that could
otherwise have been missed. Other targets in CASP15, both RNA and
protein, likely have significant flexibility, and the relatively low
resolution of many of cryo-EM maps suggests that some inclusion of
experimental uncertainty is desirable.
A related conclusion from this CASP is that the long-time principle of
comparing computed structure coordinates with experimental ones is
sometimes inappropriate. That can be the case for single conformations,
but is more likely to be critical for ensembles. In these situations,
direct comparison of non-structural data computed from model
co-ordinates with experimental data is required, as illustrated by the
kinase target example, where comparison of computed models with the
experimental NMR data (e.g., nuclear Overhauser effect, NOE, data) might
avoid any biases introduced by the experimental modeling process, and be
more appropriate for assessment.
It is also possible to compare electron density implied by computed
structures directly with maps derived from experiment, and this has
already been explored in previous CASPs (30, 31) as well as the current
one (23). In this CASP, the highly flexible RNA target R1156 was
represented by 10 experimental atomic structures for each of four
election density maps, 40 co-ordinate sets in all. A metric of electron
density fit (SMOC (32)) shows a similar ranking of overall model quality
as the coordinate comparisons, but the map comparison provides useful
insight into the local experiment/calculation mismatches. Comparison
with electron density also provides a starting point for refining
models. For this target, it was possible to further refine some
submitted models into specific conformation density maps so that they
rival the reference structure (23).
Some types of ensemble were still beyond the state of the computational
art this CASP. For the protein/RNA complex, new methods such as
RosettaFold2 (24) are able to handle this sort of structure. At the
moment, very large, complex molecular machines such as the Holliday
junction may be too difficult. But new methods are appearing frequently,
and we will see in the next CASP whether this barrier has been breached.
A future goal is to include estimates of population level for each
member of an ensemble under specific conditions, where those data are
available.
All-in-all, this first inclusion of ensemble targets in CASP, although
limited in scope, has established that it is possible to apply CASP
principles to this type of structure problem and that some available
data can provide stringent tests of the methods. And further, that in
some cases the methods, especially those based on Alphafold2, can be
remarkably effective. We plan to include a ensembles category in CASP16
in 2024. We invite discussion of the most appropriate kinds of data and
suggestions on potential targets. Those interested may use the CASP15
Discord or write to
casp@predictioncenter.org.