Enable IAU to be used within the DA outer loop - first stage of developments

[Samantha Pullen (SamanthaPullen)]

PR Summary

Sci/Tech Reviewer: Steven Sandbach (@ss421)
Code Reviewer: Harry Shepherd (@harry-shepherd)

An outer loop capability is in development for the JADA 4D-VAR data assimilation (DA) cycle. This will require lfric_atm to be run iteratively across the 6-hour DA window from the JEDI environment, with its initial state updated by a set of increments at each iteration. The IAU is well placed to carry out these incremental updates, however it will need some small adaptations to satisfy the requirements of the outer loop updates. Note that this is a completely separate application of lfric_atm (repeated short runs within the DA cycle as part of the process to calculate analysis increments) to its use for a full NWP forecast (new long forecast from an analysis state which has been updated with analysis increments from the completed DA cycle).

This PR represents the first stage of IAU developments for outer loop capability and enables the following IAU behaviour with the activation of a new namelist option iau_outerloop:

• IAU fields and field collection are created in the lfric_atm initialisation (subroutine initialise) as normal - it is anticipated that the main lfric_atm initialise routine will be run once at the start of the DA cycle, and then updates to modeldb fields made, as needed, at each iteration of the outer loop.
• IAU increments input from file is skipped, leaving the IAU fields uninitialised - this is because the increments will be passed in memory by JEDI later in the application, rather than from file.
• Application of the increments to update the prognostic state is skipped. This is because the IAU update routine will be called separately by JEDI at the end of each outer loop iteration. We do not want to activate it during the normal model initialisation process.
• The iau_fields field collection is not removed at the end of initialise for an outer loop application since the field collection will be needed, and filled, at the end of each outer loop iteration.

This change includes an upgrade macro, which has been tested on test branch: https://github.com/SamanthaPullen/lfric_apps/tree/test_iau_outerloop_stage1

Code Quality Checklist

• I have performed a self-review of my own code
• My code follows the project's style guidelines
• Comments have been included that aid understanding and enhance the readability of the code
• My changes generate no new warnings
• All automated checks in the CI pipeline have completed successfully - all pre-CR checks have completed

Testing

• I have tested this change locally, using the LFRic Apps rose-stem suite
• If any tests fail (rose-stem or CI) the reason is understood and acceptable (e.g. kgo changes)
• I have added tests to cover new functionality as appropriate (e.g. system tests, unit tests, etc.) - no new tests added at this stage (see below)
• Any new tests have been assigned an appropriate amount of compute resource and have been allocated to an appropriate testing group (i.e. the developer tests are for jobs which use a small amount of compute resource and complete in a matter of minutes) - NA

This change was tested by temporarily adding iau_outerloop=.true. to the rose-stem lfric_atm da optional config (at 508715c).
The da task ran successfully, and skipped reading IAU increments from file and the IAU updating of the prognostic state, as anticipated: output here
The only failures were the da task kgo checks, as expected (since we have not applied IAU increments)

Rose stem testing has been performed for the development branch SamanthaPullen/lfric_apps@iau_outerloop_stage1 and the test branch, with upgrade macro applied SamanthaPullen/lfric_apps@test_iau_outerloop_stage1

A new rose-stem test has not been added at this stage, to test a config with iau_outerloop enabled, but this will be added once outer loop developments have advanced a little more, most likely in combination with jedi_lfric testing.

trac.log for dev branch

Test Suite Results - lfric_apps - iau_outerloop_stage1/run8

Suite Information

Item	Value
Suite Name	iau_outerloop_stage1/run8
Suite User	samantha.pullen
Workflow Start	2026-05-15T14:13:41
Groups Run	lfric_atm', 'developer

Dependency	Reference	Main Like
casim	MetOffice/casim@2026.03.2	True
jules	MetOffice/jules@2026.03.2	True
lfric_apps	SamanthaPullen/lfric_apps@test_iau_outerloop_stage1	False
lfric_core	MetOffice/lfric_core@2026.03.2	True
moci	MetOffice/moci@2026.03.2	True
SimSys_Scripts	MetOffice/SimSys_Scripts@2026.03.2	True
socrates	MetOffice/socrates@2026.03.2	True
socrates-spectral	MetOffice/socrates-spectral@2026.03.2	True
ukca	MetOffice/ukca@2026.03.2	True

Task Information

✅ succeeded tasks - 1310

trac.log for test branch

Test Suite Results - lfric_apps - test_iau_outerloop_stage1/run1

Suite Information

Item	Value
Suite Name	test_iau_outerloop_stage1/run1
Suite User	samantha.pullen
Workflow Start	2026-05-15T14:45:55
Groups Run	all

Dependency	Reference	Main Like
casim	MetOffice/casim@2026.03.2	True
jules	MetOffice/jules@2026.03.2	True
lfric_apps	SamanthaPullen/lfric_apps@test_iau_outerloop_stage1	False
lfric_core	MetOffice/lfric_core@2026.03.2	True
moci	MetOffice/moci@2026.03.2	True
SimSys_Scripts	MetOffice/SimSys_Scripts@2026.03.2	True
socrates	MetOffice/socrates@2026.03.2	True
socrates-spectral	MetOffice/socrates-spectral@2026.03.2	True
ukca	MetOffice/ukca@2026.03.2	True

Task Information

✅ succeeded tasks - 1511

Security Considerations

• I have reviewed my changes for potential security issues
• Sensitive data is properly handled (if applicable)- NA
• Authentication and authorisation are properly implemented (if applicable)- NA

Performance Impact

• Performance of the code has been considered and, if applicable, suitable performance measurements have been conducted

AI Assistance and Attribution

• Some of the content of this change has been produced with the assistance of Generative AI tool name (e.g., Met Office Github Copilot Enterprise, Github Copilot Personal, ChatGPT GPT-4, etc) and I have followed the Simulation Systems AI policy (including attribution labels) - NA

Documentation

• Where appropriate I have updated documentation related to this change and confirmed that it builds correctly - NA

PSyclone Approval

• If you have edited any PSyclone-related code (e.g. PSyKAl-lite, Kernel interface, optimisation scripts, LFRic data structure code) then please contact the TCD Team - NA

Sci/Tech Review

• I understand this area of code and the changes being added
• The proposed changes correspond to the pull request description
• Documentation is sufficient (do documentation papers need updating)
• Sufficient testing has been completed

(Please alert the code reviewer via a tag when you have approved the SR)

Code Review

• All dependencies have been resolved
• Related Issues have been properly linked and addressed
• CLA compliance has been confirmed
• Code quality standards have been met
• Tests are adequate and have passed
• Documentation is complete and accurate
• Security considerations have been addressed
• Performance impact is acceptable

[Samantha Pullen (SamanthaPullen)]

Hi Steven Sandbach (@ss421) - I think everything is ready now. Passing to you for SR

[James Bruten (james-bruten-mo)]

Upgrade macro is good

[Steven Sandbach (ss421)]

Thanks for the update. Passing to: Harry Shepherd (@harry-shepherd) for code review.

[thomasmelvin]

All looks good to me

[Samantha Pullen (SamanthaPullen)]

Following discussion with Steven Sandbach (@ss421) iboutle and Harry Shepherd (@harry-shepherd) this PR has been extended to incorporate a small additional change.

During preparation for the next stage of JADA outer loop development, we have found that this additional change would make a substantial difference to the speed that the next stage can be progressed. A new flag trop_diag has been added as an argument to the locate_tropopause_alg subroutine and used to determine whether tropopause level diagnostic (processed__trop_level) is output as a diagnostic. The flag is enabled for the call to locate_tropopause_alg from the slow physics, but disabled for the call to locate_tropopause_alg from the IAU. This results in more appropriate behaviour in any NWP DA configuration, and avoids issues in an outer loop configuration when the IAU will be called after the completion of the driver initialisation.

I have checked that the values of variable trop_level in the lfric_diag.nc file output by the lfric_coupled rose-stem task:
lfric_coupled_nwp_gal9-C48_ex1a_cce_fast-debug-64bit
are identical before and after implementation of the new flag at 2b9d232

New trac.log for dev branch:

Test Suite Results - lfric_apps - iau_outerloop_stage1/run10

Suite Information

Item	Value
Suite Name	iau_outerloop_stage1/run10
Suite User	samantha.pullen
Workflow Start	2026-05-29T08:58:35
Groups Run	all

Dependency	Reference	Main Like
casim	MetOffice/casim@2026.03.2	True
jules	MetOffice/jules@2026.03.2	True
lfric_apps	SamanthaPullen/lfric_apps@iau_outerloop_stage1	False
lfric_core	MetOffice/lfric_core@2026.03.2	True
moci	MetOffice/moci@2026.03.2	True
SimSys_Scripts	MetOffice/SimSys_Scripts@2026.03.2	True
socrates	MetOffice/socrates@2026.03.2	True
socrates-spectral	MetOffice/socrates-spectral@2026.03.2	True
ukca	MetOffice/ukca@2026.03.2	True

Task Information

✅ succeeded tasks - 1511

I have not created a new test branch for this change as there are no changes to rose-meta.

Steven Sandbach (@ss421) and iboutle would you mind re-reviewing? Thank you all :)

[iboutle]

Looks good to me!

[Steven Sandbach (ss421)]

Thanks for making this update Samantha Pullen (@SamanthaPullen). Changes look good to me. I've applied the change you committed in: 2b9d232 via my branch and it runs as expected. Handing back to Harry Shepherd (@harry-shepherd).