Selecting the correct DefiningAllele for a Canonical Variant in Clinvar

[larrybabb]

While I've been working to 'canonicalize' the variants in clinvar over the past months I've run into an interesting issue that would be good to get community feedback on.

Background

When clinvar submissions come in, the general process for 'normalizing' variants is to align or lift over whichever singular form of variant is submitted to build 38 (GRCh38). Submitters can offer transcript or genomic variant representation for any refseq accession.

• If a transcript variant is submitted then an attempt is made to align it to build 38. if it can then the build 38 variant becomes the thing that uniquely identifies the variant in clinvar, is assigned a unique variation id and the submission(s) associated with it are aggregated within that variation id.
• If a genomic variant is submitted then it is lifted over to 38 (if it is not already in that form). and the same rules for identity apply.

If build 38 alignment or lift over is not possible (for whatever reason) then build 37 becomes the target for identity and if that is not possible then build 36, and so on. Its not too typical, but some variants that are unsuccessful can end up being identified by their submitted transcript representation. You may also be interested to know that clinvar originally allowed protein representations to be submitted but realized that some (most) protein variants did not resolve to a single transcript and genomic representation causing them to create "protein only" type variants for a period of time. It is my understanding that they do not allow variants that cannot be resolved to a single transcript/genomic representation anymore (even though some remain in the database).

Problem

Forgetting all the corner cases and focusing on the notion that transcript and genomic submitted representations are 'normalized' by clinvar in an attempt to arrive at a build 38 representation, we hit an interesting issue related to identity of the variant in terms of what Cat-VRS and Clinvar might call the 'canonical' form. If a build 37 variant is lifted over to build 38 and the 38 form represents the identity, then we end up with several different build 37 representations mapping to the same build 38 representations in some situations. This begs the question...Are they the same variant?

Argument 1: they are the same variant in build 38 even though they may lift over to multiple locations on another build
Argument 2: they are different variants in build 37 even though the lift over to the same location in another build

If a clinvar submitter submits a build 37 form of the variant they will most definitely want to have that retained in the variant identity of the clinvar record that their submission is associated to. So if several ISCA submitters do all their testing and calling on build 37 and come up with similar duplications and/or deletions that end up mapping to a single build 38 location how should it be handled?

This case exists. In the following table we have 9 different clinvar variation ids that all have the same 'preferred name' (green) and build 38 representation (red), but all the build 37 representations are different (purple). These submitters that where ISCA sites did all their calling and submissions using build 37 called variants. ClinVar lifted them over to build 38 and gave them the preferred name but clinvar did NOT give them the same variant identifiers and these submissions did not get aggregated and compared to each other even though they all mapped to the same exact build 38 regions.

I would argue that if Clinvar considers these different canonical variants then they should NOT provide the build 38 form, or they should at least clearly indicate that the build 38 form is derived and NOT representative of the identifying canonical variant.

If you argue that the build 38 form IS the identifying form then should ClinVar provide ALL the submitted build 37 forms and aggregate the submissions? If so, how would they identify which SCV was associated with which build 37 form?

I would really appreciate folks weighing in.

BTW there are other cases related to variant identity in clinvar (and other knowledge bases I'm sure) which we should discuss and determine how to help each other find a pragmatic way to handle to allow sharing and consistency in our datasets.

[rrfreimuth]

Interesting questions. They are not trivial. IMO there are nuances that must be teased apart, because while there is some technical element to the questions there are also other factors that must be considered.

In the case of a single site, an observed sequence (S1) can be mapped to either build (37 or 38). In the figure below, different pipelines might be used to do the calling so the algorithm (and parameters) used for build 37 (labeled CA1) may be different from that used for build 38 (CA2). Intuitively, we might assert that variations V1 and V2 are the same. (I am trying to avoid using the word "identical" here, because that imposes an additional level of semantics and mathematical equivalency that may not be desired.)

Declaring V1 and V2 are "the same", however, requires an implicit decision that calling algorithms CA1 and CA2 work in a similar manner (e.g., alignment parameters, shifting, etc). This is important because it may be possible to change the configuration of either algorithm such that it produces different output. Therefore, the answer to the question "Are they the same variant?" is dependent on both the specific configurations of CA1 and CA2, and a determination as to whether or not those configurations can be considered to operate "the same". A different combination of settings, in the context of a different use case, might yield a different answer to the question.

The challenge of liftover is similar, but a little different since it uses called variation as a starting point rather than an observed sequence (and therefore the configuration of the calling algorithm is embedded within it). The algorithms used to lift in one direction (e.g., LA1) or the other (e.g., LA2) each have their own parameters and require the same determination as to whether the two algorithms operate similarly.

As your example indicates, liftover is not necessarily a 1-to-1 process, so the determination of whether V3 and V4 are "the same" requires deciding (at least implicitly) whether or not that is acceptable in the context of a given use case. At a minimum, it should be possible to round-trip the lifts (V3 to V4 to V3, and V4 to V3 to V4), but this technical conversion may not be sufficient to determine whether V3 and V4 are the same for a given use case.

An easier question that avoids the 1-to-1 issue but that still gets at the heart of what I think you're asking, is to consider the scenario without the liftover:

In this example, two sites each observe a sequence and, using their own internal calling pipelines, call a variation relative to both build 37 and build 38. (We could think through cases where S1 and S2 are the same but calling algorithms are different, or vise versa. We could also consider whether "build x" is sufficiently unique to determine that two pipelines are using the exact same underlying reference sequences. I'll skip those complexities for now.)

In the example from the OP, nothing is known about the observed sequences or the calling algorithms. All that is known is that one site declares V1 is "the same" as V2, and that another site declares V5 is "the same" as V2. Without knowing the specifics of the builds, the algorithms, the parameters, or the observed sequences, do we have enough information to determine whether V1 is "the same" as V5?

I think the answer to that question is: maybe. It depends on the use case and the policies relevant to that use case. If all sites used the same calling algorithms, parameters, and builds, it might be easier to define the rules for "sameness" because the underlying non-reported data could be assumed or deterministically derived, but I do not think that is the case.

tl;dr
I would guess this response is wholly unsatisfying. In short, I think questions about whether a particular knowledge base should lump or split submissions are only partly answerable with technology and must include policy decisions. Similarly, while determinations about whether or not two variations are "the same" involve a technical component, they also require consideration of the use case. The technical piece will only get us so far, and the policy and use case dimensions are needed to get the rest of the way. Both of those are out of scope and not controllable by VRS or GA4GH. (All of this is IMO, of course.)

My approach to this has become: state what is known in an unambiguous and computable way, and report what is derived with provenance, then let the consumer determine how to use the data because only they will know the use case and policy elements of the equation.

[larrybabb]

I like where you ended up

let the consumer determine how to use the data because only they will know the use case and policy elements of the equation.

In the case of clinvar, they get one form (of several allowed) from the submitter and that is what they know. They retain it, but then they 'normalize' it to get it to b38 if possible and use that as the basis. They also may get the "testing method" but that is a different field which is tied to the submission (all good, but no part of the variant identity as far as the clinvar use case goes IMO).

So I think Clinvar should simply show the submitted variant form (which the currently don't), they already show the testing method if supplied, but they should stick with their normalized form for aggregating and comparing the classifications. As you suggest, the users of the data can decide for themselves if they want to re-aggregate or filter the data differently, based on their needs.

[rrfreimuth]

After re-reading my response I realize I left the last bit as an exercise for the reader. I'll finish the connection.

I think these are different types of statements:

0. V1, V2, V3, and V4 are defined variations [e.g., VRS]
1. V1 is "the same as" V2 [e.g., VA]
2. V3 lifts over to V4 [e.g., VA]
3. {the set of all variations that meet constraint C5}, where C5 is a DefiningAlleleConstraint [extrinsic; e.g., variations that normalize to the same representation on a given build]
4. {the set of all variations that after liftover meet constraint C5}, where C5 is a DefiningAlleleConstraint [extrinsic; e.g., all build 37 variations that lift over to a given build 38 representation]
5. {this defined set of variations that after liftover meet constraint C5}, where C5 is a DefiningAlleleConstraint [intrinsic; e.g., a specific list of build 37 submissions to clinvar that lift over to the same representation on build 38]

Note that the concept of liftover incorporates elements of algorithm, parameter, and build, so that defining a set that includes liftover as a criterion might require representation of those elements in either VA statements or CatVRS constraints (which would probably need VA statements underlying them anyway).

Back to what I think is your primary question:

So if several ISCA submitters do all their testing and calling on build 37 and come up with similar duplications and/or deletions that end up mapping to a single build 38 location how should it be handled?

This is a policy question. The data can be represented several ways, and which representation is the most appropriate will depend on the use case.

For example, submitters could use statements 0 and/or 2. Looking through a list of clinvar variations on build 38, statement 5 might be more natural. Statement 5 can be used as a seed to create the generalized statement 4. This approach supports the type of generalization that I think is desired but does not introduce assertions that go beyond the original submissions.

So in my mind, it is not a matter of choosing 1 thing - it's supporting all of the above and then using whichever is most appropriate to convey the desired semantics.

[cmprocknow]

Cat-VRS doesn't care about your ClinVar variant IDs or how you map and retain submissions to them! If you need or want, you can define canonical build 37 variants AND canonical build 38 variants. And you can map them to one another. Go crazy!

[larrybabb]

I agree 100%.
But, my task is to try to standardize clinvar without "redefining" it. So, I need to determine what they are actually doing so my canonical build 37 vs 38 variants map to their ids in a computationally consistent way. But I think this is possible as long as I document how I dealt with some of these "special" cases.

[cmprocknow]

I also disagree that we need or even should have consistent policies across datasets here. Cat-VRS doesn't do matching, it just shows you what matching exists.

[brendanreardon]

Thank you Larry for fostering such a wonderful discussion. Broadly I agree with the above comments from Bob and Catherine, that this is ultimately a policy decision for ClinVar to decide of what they/you want to define as the CanonicalAllele.

But, a couple of thoughts:

• Are you storing every variant in ClinVar as a categorical variant, with the goal of satisfying the CanonicalAllele Recipe? If you are not trying to deem each as a CanonicalAllele, you could instead specify that ClinVar only calls hg38 alleles as CanonicalAlleles, per ClinVar policy. For variants that do not map to hg38, could you specify in an extension that they are related to the hg38 CanonicalAllele version via one of the appropriate terms? e.g. liftover?
• Could you take advantage of the members property of the CategoricalVariant class to essentially perform mappings between related entries in ClinVar? You could use this relationship to provide hyperlinks between the variants. In your example, I imagine that you want to deem the GRCh38 version as the Canonical Allele because they are identical. You could thus list the GRCh37 and NCBI36 versions under members.

[brendanreardon]

As we talked about on Slack, I am going to close this Discussion per having the author of the canonical allele pick the defining allele, but please open it at any time! I'd also love to see how you implemented this.