Skip to content

Merge iterate - #314

Merged
romeokienzler merged 541 commits into
mainfrom
merge-iterate
Jun 26, 2026
Merged

romeokienzler merged 541 commits into
mainfrom
merge-iterate

Conversation

@romeokienzler

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

How was this pull request tested?

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.

Leonardo P. Tizzei and others added 30 commits April 11, 2025 14:45
Signed-off-by: Leonardo P. Tizzei <ltizzei@br.ibm.com>
Signed-off-by: Leonardo P. Tizzei <ltizzei@br.ibm.com>
Signed-off-by: Leonardo P. Tizzei <ltizzei@br.ibm.com>
Signed-off-by: Leonardo P. Tizzei <ltizzei@br.ibm.com>
Signed-off-by: Leonardo P. Tizzei <ltizzei@br.ibm.com>
Signed-off-by: Leonardo P. Tizzei <ltizzei@br.ibm.com>
pull updates


Signed-off-by: naomi-simumba <naomi.simumba@gmail.com>
Signed-off-by: Naomi Simumba <naomi.simumba@gmail.com>
option for test step


Signed-off-by: Leonardo P. Tizzei <leonardo.tizzei@gmail.com>
Signed-off-by: Leonardo P. Tizzei <ltizzei@br.ibm.com>
Signed-off-by: Leonardo P. Tizzei <ltizzei@br.ibm.com>
Signed-off-by: Leonardo P. Tizzei <ltizzei@br.ibm.com>
Signed-off-by: Leonardo P. Tizzei <ltizzei@br.ibm.com>
Signed-off-by: Naomi Simumba <naomi.simumba@gmail.com>
Signed-off-by: Naomi Simumba <naomi.simumba@gmail.com>
Signed-off-by: Paolo Fraccaro <paolo.fraccaro@ibm.com>
Signed-off-by: Paolo Fraccaro <paolo.fraccaro@ibm.com>
Signed-off-by: naomi-simumba <naomi.simumba@gmail.com>
Signed-off-by: Naomi Simumba <naomi.simumba@gmail.com>
Signed-off-by: Naomi Simumba <naomi.simumba@gmail.com>
Signed-off-by: Naomi Simumba <naomi.simumba@gmail.com>
Signed-off-by: Naomi Simumba <naomi.simumba@gmail.com>
…il.com>

Signed-off-by: Naomi Simumba <naomi.simumba@gmail.com>
Signed-off-by: Naomi Simumba <naomi.simumba@gmail.com>
Signed-off-by: Naomi Simumba <naomi.simumba@gmail.com>
Signed-off-by: Naomi Simumba <naomi.simumba@gmail.com>
Signed-off-by: Paolo Fraccaro <paolo.fraccaro@ibm.com>
Signed-off-by: Paolo Fraccaro <paolo.fraccaro@ibm.com>
Add custom modules and fixing various bugs


Signed-off-by: Paolo <36694818+paolofraccaro@users.noreply.github.com>
Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
romeokienzler and others added 21 commits April 20, 2026 15:29
Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
…ig/data_path

Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
Add PostgreSQL support and restore related configurations


Signed-off-by: Romeo Kienzler <5694071+romeokienzler@users.noreply.github.com>
… and Vela

- Removed hardcoded WLM options and parameters from the argument parser.
- Added support for a user-defined WLM plugin via `--wlm-plugin` argument.
- Implemented `load_wlm_config` function to read WLM settings from HPO YAML.
- Created reference implementations for LSF and Vela plugins in `examples/wlm_plugins/`.
- Updated `run_and_stream` to handle local execution and WLM plugin invocation.
- Enhanced logging to provide clearer feedback on trial execution and WLM interactions.
- Cleaned up unused functions and parameters related to previous WLM handling.

Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
- Updated `run_setter_example.sh` to use `bumpy_function.py` instead of `bumpy_setter.py`, simplifying the example for local trials.
- Modified `run_vela_example.sh` to clarify usage of the Vela/OpenShift job submission, ensuring better documentation and example clarity.
- Refined `lsf_plugin.sh` to streamline job submission for IBM Spectrum LSF, enhancing clarity on environment variable usage and command construction.
- Overhauled `_iterate2.py` to simplify the command-line interface, improve YAML loading, and enhance metric extraction logic.
- Removed deprecated features and improved logging for better traceability during execution.
- Enhanced the objective function to better handle parameter suggestions and metrics extraction.

Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
… clusters

Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
…parameter

Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
… directory, and performance reporting options

Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
…o require PostgreSQL URL

Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
… across examples

Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
Refactor WLM integration and iterate2 functionality with plugins


Signed-off-by: Romeo Kienzler <5694071+romeokienzler@users.noreply.github.com>
@socket-security

socket-security Bot commented Jun 26, 2026

Copy link
Copy Markdown

Warning

Review the following alerts detected in dependencies.

According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.

Action Severity Alert  (click "▶" to expand/collapse)
Warn High
Obfuscated code: pypi mlflow is 90.0% likely obfuscated

Confidence: 0.90

Location: Package overview

From: pyproject.tomlpypi/mlflow@3.14.0

ℹ Read more on: This package | This alert | What is obfuscated code?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/mlflow@3.14.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn High
Obfuscated code: pypi mlflow is 90.0% likely obfuscated

Confidence: 0.90

Location: Package overview

From: pyproject.tomlpypi/mlflow@3.14.0

ℹ Read more on: This package | This alert | What is obfuscated code?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/mlflow@3.14.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn High
Obfuscated code: pypi numpy is 90.0% likely obfuscated

Confidence: 0.90

Location: Package overview

From: pyproject.tomlpypi/numpy@2.5.0

ℹ Read more on: This package | This alert | What is obfuscated code?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/numpy@2.5.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

View full report

Fuses the terratorch-iterate repo (terrastackai/iterate main @ 9f68a9f)
into claimed as the umbrella project. Iterate's package, examples,
configs, tests, and docs are added at the top level; project identity,
license, and CI stay with claimed.

Conflict resolutions:
  * pyproject.toml — keeps name `claimed`; merges scripts (c3_*, claimed,
    iterate, iterate-classic), dependencies, optional-deps, and extends
    packages.find to include terratorch_iterate*.
  * setup.py — replaced legacy component-library stub with iterate's
    minimal stub that defers to pyproject.toml.
  * README.md, docs/index.md — claimed content kept, iterate appended
    under a divider.
  * mkdocs.yml — claimed theme/config kept; new `terratorch-iterate`
    nav section added for the imported docs/*.md.
  * LICENSE, CODE_OF_CONDUCT.md, CONTRIBUTING.md — claimed's kept.
  * .gitignore — union, deduplicated.
  * .github/workflows/python-package.yml — dropped (redundant with
    claimed's python-app.yml + pylint.yml).

Follow-ups to verify after merge:
  * src/c3/_version.py path referenced by setuptools_scm doesn't exist
    on claimed main (pre-existing).
  * .github/workflows/integration-test.yml targets nonexistent paths
    (component-library/, test_requirements.txt) — rewrite or drop.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
@romeokienzler
romeokienzler force-pushed the merge-iterate branch 3 times, most recently from 5260329 to b1a654f Compare June 26, 2026 16:49
@romeokienzler
romeokienzler merged commit 6b228c0 into main Jun 26, 2026
4 of 9 checks passed
romeokienzler added a commit that referenced this pull request Jun 26, 2026
All four Python CI workflows have been failing on every run since at least
April 2026; the iterate merge (#314) added more pylint warnings on top of
already-broken jobs. Root causes were stale paths and a missing
environment.yml, not new bugs.

- pylint: scope to src/ and terratorch_iterate/, run advisory (--exit-zero)
  to surface the large pre-existing duplicate-code backlog without gating
  every PR; bump python matrix to 3.11/3.12 to match pyproject's
  requires-python = ">=3.11".
- python-package-conda: drop the missing environment.yml step in favour of
  pip install -e . (this repo is pip-managed); scope flake8 to src/ and
  terratorch_iterate/; run pytest advisory because most tests need heavy
  optional deps (torch, terratorch).
- python-app: ./component-library no longer exists -- point crosshair at
  ./src; switch `watch` to `check` so the job actually terminates.
- integration-test: trigger on `main` (not the long-gone `master`) and
  invoke run_tests.py at the repo root.

Also refreshes pinned action versions (checkout@v4, setup-python@v5).

Signed-off-by: Romeo Kienzler <romeo.kienzler1@ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants