Skip to content

Concatenation of 3D cloud data sometimes fails #3150

Description

@schlunma

Running specific recipes sometimes leads to the following error:

ValueError: Expected a 1-dim C contiguous array  of dtype = 12
Full traceback
Traceback (most recent call last):
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
                    ^^^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/_task.py", line 983, in _run_task
    output_files = task.run()
                   ^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/_task.py", line 286, in run
    self.output_files = self._run(input_files)
                        ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/preprocessor/__init__.py", line 885, in _run
    product.apply(step, self.debug)
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/preprocessor/__init__.py", line 701, in apply
    self.cubes,
    ^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/preprocessor/__init__.py", line 713, in cubes
    self._cubes = [ds.load() for ds in self.datasets]  # type: ignore
                   ^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/dataset.py", line 780, in load
    cube = self._load()
           ^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/dataset.py", line 862, in _load
    result = preprocess(
             ^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/preprocessor/__init__.py", line 570, in preprocess
    _run_preproc_function(
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/preprocessor/__init__.py", line 510, in _run_preproc_function
    return function(items, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/preprocessor/_concatenate.py", line 286, in concatenate
    result = _concatenate_cubes(cubes, check_level=check_level)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/preprocessor/_concatenate.py", line 54, in _concatenate_cubes
    return CubeList(cubes).concatenate(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/iris/cube.py", line 659, in concatenate
    return concatenate(
           ^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/iris/_concatenate.py", line 618, in concatenate
    hashes = _compute_hashes(arrays)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/iris/_concatenate.py", line 535, in _compute_hashes
    (hashes,) = dask.compute(hashes)
                ^^^^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/dask/base.py", line 685, in compute
    results = schedule(expr, keys, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/scipy/interpolate/_polyint.py", line 88, in __call__
    y = self._evaluate(x)
    ^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/scipy/interpolate/_interpolate.py", line 566, in _evaluate
    y_new = self._call(self, x_new)
    ^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/scipy/interpolate/_interpolate.py", line 553, in _call_spline
    return self._spline(x_new)
    ^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/scipy/interpolate/_bsplines.py", line 887, in __call__
    self._delegate_to(
    ^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/scipy/interpolate/_bsplines.py", line 253, in __call__
    out = _dierckx.evaluate_spline(self.t, cc, self.k, x, nu, extrapolate)
    ^^^^^^^^^^^^^^^^^
ValueError: Expected a 1-dim C contiguous array  of dtype = 12

"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/_main.py", line 828, in run
    fire.Fire(ESMValTool())
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/fire/core.py", line 135, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/fire/core.py", line 468, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
                                ^^^^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/_main.py", line 664, in run
    self._run(recipe, session, cli_config_dir)
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/_main.py", line 717, in _run
    process_recipe(recipe_file=recipe, session=session)
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/_main.py", line 154, in process_recipe
    recipe.run()
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/_recipe/recipe.py", line 1342, in run
    self.tasks.run(max_parallel_tasks=self.session["max_parallel_tasks"])
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/_task.py", line 831, in run
    self._run_parallel(address, max_parallel_tasks)
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/_task.py", line 941, in _run_parallel
    _copy_results(task, running[task])
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/_task.py", line 971, in _copy_results
    task.output_files, task.products = future.get()
                                       ^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/multiprocessing/pool.py", line 774, in get
    raise self._value
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
    ^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/_task.py", line 983, in _run_task
    output_files = task.run()
    ^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/_task.py", line 286, in run
    self.output_files = self._run(input_files)
    ^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/preprocessor/__init__.py", line 885, in _run
    product.apply(step, self.debug)
    ^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/preprocessor/__init__.py", line 701, in apply
    self.cubes,
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/preprocessor/__init__.py", line 713, in cubes
    self._cubes = [ds.load() for ds in self.datasets]  # type: ignore
    ^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/dataset.py", line 780, in load
    cube = self._load()
    ^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/dataset.py", line 862, in _load
    result = preprocess(
    ^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/preprocessor/__init__.py", line 570, in preprocess
    _run_preproc_function(
    ^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/preprocessor/__init__.py", line 510, in _run_preproc_function
    return function(items, **kwargs)
    ^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/preprocessor/_concatenate.py", line 286, in concatenate
    result = _concatenate_cubes(cubes, check_level=check_level)
      ^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/esmvalcore/preprocessor/_concatenate.py", line 54, in _concatenate_cubes
    return CubeList(cubes).concatenate(**kwargs)
      ^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/iris/cube.py", line 659, in concatenate
    return concatenate(
    ^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/iris/_concatenate.py", line 618, in concatenate
    hashes = _compute_hashes(arrays)
      ^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/iris/_concatenate.py", line 535, in _compute_hashes
    (hashes,) = dask.compute(hashes)
      ^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/dask/base.py", line 685, in compute
    results = schedule(expr, keys, **kwargs)
    ^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/scipy/interpolate/_polyint.py", line 88, in __call__
    y = self._evaluate(x)
    ^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/scipy/interpolate/_interpolate.py", line 566, in _evaluate
    y_new = self._call(self, x_new)
    ^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/scipy/interpolate/_interpolate.py", line 553, in _call_spline
    return self._spline(x_new)
    ^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/scipy/interpolate/_bsplines.py", line 887, in __call__
    self._delegate_to(
    ^^^^^^^^^^^^^^^^^
  File "/home/b/b309141/repos/ESMValTool/.pixi/envs/esmvaltool/lib/python3.12/site-packages/scipy/interpolate/_bsplines.py", line 253, in __call__
    out = _dierckx.evaluate_spline(self.t, cc, self.k, x, nu, extrapolate)
    ^^^^^^^^^^^^^^^^^
ValueError: Expected a 1-dim C contiguous array  of dtype = 12
MWE recipe
# ESMValTool
---
documentation:
  title: >
    Emergent constraints for ECS
  description: >
    Multiple emergent constraints for ECS.
  authors:
    - schlund_manuel
  maintainer:
    - schlund_manuel

preprocessors:

  tropical_mask_40:
    regrid:
      target_grid: 2x2
      scheme: linear
    extract_region:
      start_latitude: -40
      end_latitude: 40
      start_longitude: 0
      end_longitude: 360

diagnostics:
  test:
    variables:
      cl:
        project: CMIP5
        exp: historical
        ensemble: r1i1p1
        preprocessor: tropical_mask_40
        mip: Amon
        start_year: 1980
        end_year: 2004
        reference_dataset: ''
    additional_datasets:
      - {dataset: ACCESS1-0}
    scripts:
      null

I cannot reliably reproduce this error. Running it on the same machine with the same configuration might result in a successful run or the above error.

It also seems to appear on multiple machines (see reports in ESMValGroup/ESMValTool#4514).

Known affected recipes (most likely more):

  • recipe_schlund20esd.yml
  • recipe_lauer22jclim_fig3-4_zonal.yml
  • recipe_ecs_constraints.yml

Known affected variables (most likely more):

  • cli
  • ci

Known affected models (most likely more):

  • ACCESS1-0
  • UKESM1-0-LL
  • HadGEM2-ES

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions