Remove environment variables from `create_request_file.py` by NParsonsMO · Pull Request #529 · MetOffice/CMEW

Naomi Parsons (NParsonsMO) · 2026-07-14T10:11:44Z

Closes #523

PR creation checklist for the developer

The <issue_number> above ☝️ has been replaced with the issue number.
main has been selected as the base branch.
The feature branch name follows the format <issue_number>_<short_description_of_feature>.
The text of the PR title exactly matches with the text (not including the issue number) of the issue title.
Appropriate reviewers have been added to the PR (once it is ready for review).
The PR has been assigned to the developer(s).
The same labels as on the issue (except for the good first issue label) have been added to the PR.
The Climate Model Evaluation Workflow (CMEW) project has been added to the PR.
The appropriate milestone has been added to the PR.

Definition of Done for the developer

Copyright and IPR:

	GitHub handles
I confirm that all code is my own and that my contributions are not subject to copyright or license restrictions.	NParsonsMO
I confirm I have not knowingly violated intellectual property rights (IPR) and have taken sensible measures to prevent doing so, including appropriate attribution for usage of Generative AI. I confirm that this work is my own, and I understand that it is my responsibility to ensure I am not violating others’ IPR. This includes taking reasonable steps to ensure that all tools used while creating this contribution did not infringe IPR.	NParsonsMO

PR creation checklist for the reviewer

The <issue_number> above ☝️ has been replaced with the issue number.
main has been selected as the base branch.
The feature branch name follows the format <issue_number>_<short_description_of_feature>.
The text of the PR title exactly matches with the text (not including the issue number) of the issue title.
Appropriate reviewers have been added to the PR (once it is ready for review).
The PR has been assigned to the developer(s).
The same labels as on the issue (except for the good first issue label) have been added to the PR.
The Climate Model Evaluation Workflow (CMEW) project has been added to the PR.
The appropriate milestone has been added to the PR.

Definition of Done for the reviewer

Important

Remember to re-check the Definition of Done after making changes in response to a review.
The developer merges the PR.
Remember to use the format #<pull_request_number>: <pull_request_title> when writing the merge commit message for the pull request, so the pull request number is immediately visible on GitHub, regardless of the length of the pull request title.

…_request_filepy

Naomi Parsons (NParsonsMO) · 2026-08-07T09:46:42Z

This changes a lot of the same files as

Update to use CDDS v4.0.0 #542

so it will be much easier (fewer conflicts) to add in the conftest.py changes once CDDS is updated.

…_request_filepy # Conflicts: # CMEW/app/configure_standardise/bin/configure_standardise.sh # CMEW/app/configure_standardise/bin/create_request_file.py # CMEW/app/configure_standardise/bin/test_create_request_file.py

…_request_filepy

Naomi Parsons (NParsonsMO) · 2026-08-07T15:52:53Z

-        Path(__file__).parent.parent / "etc" / "request_defaults.yml"
-    )
-    stream_config_path = Path(__file__).parent.parent / "etc" / "streams.yml"
+    monkeypatch.setattr(create_request_file, "list_streams", fake_list_streams)


Can / should we avoid monkeypatch here?

Given what was done in #559, I expect to not have a test for the driver function (i.e. create_request). So happy to leave this for now 👍

Naomi Parsons (NParsonsMO) · 2026-08-07T16:17:38Z

The request file needs to be made after the variables file so I'm waiting until PR #530 is reviewed.

Naomi Parsons (NParsonsMO) · 2026-08-10T08:46:11Z


 [command]
-default=configure_standardise.sh
+default=cmew-esmvaltool-env create_variables_file.py


The variables file needs to have been created first, but this single line is probably already lots of lines by now (if PR #530 has been merged).

…_request_filepy

…_request_filepy # Conflicts: # CMEW/app/configure_standardise/bin/command_line.py # CMEW/app/configure_standardise/bin/configure_standardise.sh # CMEW/app/configure_standardise/bin/configure_standardise_conftest.py # CMEW/app/configure_standardise/rose-app.conf

Emma Hogan (ehogan)

Thanks Naomi Parsons (@NParsonsMO)! 🥳

Emma Hogan (ehogan) · 2026-08-10T15:09:07Z

+    with open(model_runs_yml_fp, "r") as f:
+        dataset_dict = yaml.safe_load(f)[dataset]


Suggested change

with open(model_runs_yml_fp, "r") as f:

dataset_dict = yaml.safe_load(f)[dataset]

with open(model_runs_yml_fp, "r") as file_handle:

dataset_dict = yaml.safe_load(file_handle)[dataset]

Emma Hogan (ehogan) · 2026-08-24T14:12:53Z

+        "--output_filepath",
+        help=(
+            "The full path to the file where the "
+            "variables from the ESMValTool recipe will be written."


Suggested change

"variables from the ESMValTool recipe will be written."

"request file will be written."

?

Emma Hogan (ehogan) · 2026-08-24T14:14:25Z

+    )
+    parser.add_argument(
+        "--mip_table_dir",
+        help="The MIP table to use from CDDS.",


Suggested change

help="The MIP table to use from CDDS.",

help=(

"The full path to the directory containing the "

"MIP tables for CDDS to use.",

Emma Hogan (ehogan) · 2026-08-24T14:15:28Z

+        The names and values of the command line arguments.
+    """
+    parser = argparse.ArgumentParser(
+        description="Create a request to standardise model data with CDDS.",


Suggested change

description="Create a request to standardise model data with CDDS.",

description="Create a request file to standardise model data with CDDS.",

Emma Hogan (ehogan) · 2026-08-24T14:16:07Z

+
+def main_for_create_request_file(arguments=None):
+    """
+    Generate and write the request file for the current task environment.


Suggested change

Generate and write the request file for the current task environment.

Create a request file to standardise model data with CDDS.

Emma Hogan (ehogan) · 2026-08-24T14:18:09Z

+    raw_data_dir_mode,
+):
    """
    Generate and write the request file for the current task environment.


Suggested change

Generate and write the request file for the current task environment.

Create a request file to standardise model data with CDDS.

Emma Hogan (ehogan) · 2026-08-24T14:23:10Z

-        Path(__file__).parent.parent / "etc" / "request_defaults.yml"
-    )
-    stream_config_path = Path(__file__).parent.parent / "etc" / "streams.yml"
+    monkeypatch.setattr(create_request_file, "list_streams", fake_list_streams)


Given what was done in #559, I expect to not have a test for the driver function (i.e. create_request). So happy to leave this for now 👍

Naomi Parsons (NParsonsMO) added 4 commits July 14, 2026 10:43

Reconfiguring create_request_file

2bafc3a

Missed environmental var

df511a9

changed test

83d64df

Missed feeding through argument

cbe4efc

Naomi Parsons (NParsonsMO) added this to the v0.2.0 (multiple model runs) milestone Jul 14, 2026

Naomi Parsons (NParsonsMO) self-assigned this Jul 14, 2026

Naomi Parsons (NParsonsMO) added enhancement New feature or request configure Anything related to configuration labels Jul 14, 2026

Naomi Parsons (NParsonsMO) linked an issue Jul 14, 2026 that may be closed by this pull request

Remove environment variables from create_request_file.py #523

Open

Naomi Parsons (NParsonsMO) added quality assurance Anything related to Quality Assurance (QA) technical debt Technical debt in CMEW labels Jul 14, 2026

Naomi Parsons (NParsonsMO) requested a review from Emma Hogan (ehogan) July 14, 2026 10:13

Naomi Parsons (NParsonsMO) marked this pull request as ready for review July 14, 2026 10:13

Merge branch 'main' into 523-remove-environment-variables-from-create…

8e3b30e

…_request_filepy

Naomi Parsons (NParsonsMO) added 6 commits August 7, 2026 12:43

Missed at merge

7d36736

Missed at merge

a141994

Merge branch 'main' into 523-remove-environment-variables-from-create…

264d77f

…_request_filepy

removing argument

f9fc36e

Using conftest.py

8e38656

Naomi Parsons (NParsonsMO) commented Aug 7, 2026

View reviewed changes

Listing from a variables file rather than all streams

d18aaeb

Naomi Parsons (NParsonsMO) commented Aug 10, 2026

View reviewed changes

Naomi Parsons (NParsonsMO) added 5 commits August 10, 2026 10:02

Missed removal

551d01b

Missed removal

46fd783

Merge branch 'main' into 523-remove-environment-variables-from-create…

77b0459

…_request_filepy

Copyright

01fc928

Emma Hogan (ehogan) requested changes Aug 24, 2026

View reviewed changes

Navigation Menu

Uh oh!

Remove environment variables from `create_request_file.py` - #529

Remove environment variables from `create_request_file.py`#529
Naomi Parsons (NParsonsMO) wants to merge 17 commits into
mainfrom
523-remove-environment-variables-from-create_request_filepy

Naomi Parsons (NParsonsMO) commented Jul 14, 2026 •

edited

Loading

Uh oh!

Naomi Parsons (NParsonsMO) commented Aug 7, 2026 •

edited

Loading

Uh oh!

Naomi Parsons (NParsonsMO) Aug 7, 2026

Uh oh!

Emma Hogan (ehogan) Aug 24, 2026

Uh oh!

Naomi Parsons (NParsonsMO) commented Aug 7, 2026 •

edited

Loading

Uh oh!

Naomi Parsons (NParsonsMO) Aug 10, 2026 •

edited

Loading

Uh oh!

Emma Hogan (ehogan) left a comment

Uh oh!

Emma Hogan (ehogan) Aug 10, 2026

Uh oh!

Emma Hogan (ehogan) Aug 24, 2026

Uh oh!

Emma Hogan (ehogan) Aug 24, 2026 •

edited

Loading

Uh oh!

Emma Hogan (ehogan) Aug 24, 2026

Uh oh!

Emma Hogan (ehogan) Aug 24, 2026

Uh oh!

Emma Hogan (ehogan) Aug 24, 2026

Uh oh!

Emma Hogan (ehogan) Aug 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		with open(model_runs_yml_fp, "r") as f:
		dataset_dict = yaml.safe_load(f)[dataset]

	"variables from the ESMValTool recipe will be written."
	"request file will be written."

-        help="The MIP table to use from CDDS.",
+        help=(
+            "The full path to the directory containing the "
+            "MIP tables for CDDS to use.",

	description="Create a request to standardise model data with CDDS.",
	description="Create a request file to standardise model data with CDDS.",

	Generate and write the request file for the current task environment.
	Create a request file to standardise model data with CDDS.

Uh oh!

Conversation

Naomi Parsons (NParsonsMO) commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR creation checklist for the developer

Definition of Done for the developer

PR creation checklist for the reviewer

Definition of Done for the reviewer

Uh oh!

Naomi Parsons (NParsonsMO) commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Naomi Parsons (NParsonsMO) commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Naomi Parsons (NParsonsMO) Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Emma Hogan (ehogan) left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Emma Hogan (ehogan) Aug 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Naomi Parsons (NParsonsMO) commented Jul 14, 2026 •

edited

Loading

Naomi Parsons (NParsonsMO) commented Aug 7, 2026 •

edited

Loading

Naomi Parsons (NParsonsMO) commented Aug 7, 2026 •

edited

Loading

Naomi Parsons (NParsonsMO) Aug 10, 2026 •

edited

Loading

Emma Hogan (ehogan) Aug 24, 2026 •

edited

Loading