Mismatch of variable mapping across projects

[danielhuppmann]

I looked further into the mapping of the EMISSIONS_VARIABLES database (see the docs) across projects, and there is an inconsistency: the column iamc maps only to the AR6 variable list, but not the currently used common-definitions emissions variables.

The specific mismatch is with regard to PFCs C2F6, C6F14 and CF4, which were defined as

Emissions|PFC|C2F6
...

in AR6, but are now defined without the "PFC" category, i.e.

Emissions|C2F6

as well as the aggregate

Emissions|PFC

in https://github.com/IAMconsortium/common-definitions/blob/main/definitions/variable/emissions/tag_level-1-species.yaml

This is also consistent with the pre-release scenarioMIP data.

The solutions that I see...

1. Rename the column iamc in the EMISSIONS_VARIABLES database to ar6.
2. Add an extra column common-definitions that will be kept consistent with common-definitions as the coverage of species expands (e.g. H2 emissions, see H2 emissions: should we add it? IAMconsortium/common-definitions#406)
3. Refactor the AR6PreProcessor to clearly distinguish wether it uses the AR6-climate-processing with the AR6 variables or the AR6-climate-processing with the new common-definitions.
4. Like 3 but implement it somewhere else in the workflow?

[danielhuppmann]

@znicholls @jkikstra @phackstock @mzecc

[znicholls]

Did you look at the cmip7_scenariomip column? That should have all the mappings needed and used in the CMIP7 ScenarioMIP process (which is the same or very close to latest common-definitions).

I think the renamings make sense, happy to add those at some point.

Like 3 but implement it somewhere else in the workflow?

Yep, can go in climate-processor. The AR6 pre-processor is for the AR6 workflow specifically. For a different naming convention, I would like users to just use the functions that are used in AR6PreProcessor but update for their naming of interest (copy the code out, then update as needed). I want to keep this sort of pre-processing and the number of paths through any pre-processing to an absolute minimum in gcages because it is a nightmare to maintain (as you know) and such decisions should be made by the application (i.e. somewhere else), not the library (gcages).

[danielhuppmann]

True, the column cmip7_scenariomip is consistent with the variables in the current common-definitions. So I guess there is no need for an extra column, but renaming iamc to ar6 and adding a cross-reference to common-definitions on the docs-page (and maybe the code) would be helpful.

I will discuss with @phackstock on how to implement this in climate-processor.

[danielhuppmann]

Coming back to this issue (because of IAMconsortium/pyam#999)... It would be very helpful to rename the iamc column of the "database" to ar6 to avoid confusion. There is no such thing as an "iamc" variable naming convention.

[znichollscr]

Happy to push through tomorrow

Can you help me understand something though: how does fixing this help IAMconsortium/pyam#999? You're using your own renaming there so what we do here doesn't matter, no?

[danielhuppmann]

It's not functionally related, but investigating why there was a discrepancy between gcages and pyam Koyto-GHG-aggregation led me back to that table and the column name "iamc" again left me confused and going down a rabbit hole - and then I remembered this issue. So changing the column name will avoid that I (and others) are confused again in the future (I hope).

[znichollscr]

Got it, will update tomorrow.

Are you happy with the name AR6_WG3 ?

[danielhuppmann]

Sure, thanks (maybe lowercase for consistency with the other columns?)