Skip to content

fix: plot_format leaks its settings when the wrapped block raises - #176

Open
WatchTree-19 wants to merge 1 commit into
man-group:masterfrom
WatchTree-19:fix/plot-format-restore
Open

WatchTree-19 wants to merge 1 commit into
man-group:masterfrom
WatchTree-19:fix/plot-format-restore

Conversation

@WatchTree-19

Copy link
Copy Markdown

What does this fix?

1. plot_format leaks its settings when the wrapped block raises.

old = get_plot_format()
set_plot_format(plot_format, dpi)
yield
set_plot_format(*old)

There is no try/finally, so if the wrapped code raises, the restore never runs and _PLOT_FORMAT, _PLOT_MIME_TYPE and _PLOT_DPI stay overridden for the rest of the process. Nothing warns, so every subsequent block silently renders at the wrong format and DPI. Against released 1.4.3:

baseline           : ('png', 100) mime = png
caught             : rendering blew up
after failed block : ('svg', 300) mime = svg+xml     <- leaked

2. set_plot_format half-applies an unsupported format.

_PLOT_FORMAT = plot_format
_PLOT_MIME_TYPE = _MIME_TYPES[plot_format]

The assignment happens before the lookup, so a format not in _MIME_TYPES raises KeyError with _PLOT_FORMAT already changed and _PLOT_MIME_TYPE still on the old value:

after set_plot_format("jpeg") : format='jpeg', mime='png'   <- inconsistent

The change

try/finally around the yield, and the mime lookup moved ahead of both assignments so a bad format changes nothing.

Tests

Two added next to the existing test_plot_format_ctx_manager, which only covers the success path. Both fail on master and pass with the change; the full test_image_unit.py module passes (9 passed).

One thing I left alone

set_plot_format with a bad format still raises a bare KeyError: 'jpeg', which is not a great message for what is really an invalid argument. I kept the exception type as-is to avoid changing behaviour anyone might be catching, but happy to turn it into a ValueError naming the supported formats if you would prefer that.

plot_format mutates module-level state and restores it after the yield with no
try/finally:

    old = get_plot_format()
    set_plot_format(plot_format, dpi)
    yield
    set_plot_format(*old)

If the wrapped code raises, which rendering readily does, the restore never
runs and _PLOT_FORMAT, _PLOT_MIME_TYPE and _PLOT_DPI stay overridden for the
rest of the process. Nothing warns; every later block silently renders at the
wrong format and DPI. Against released 1.4.3, a block that raises inside
plot_format("svg", 300) leaves the globals at ('svg', 300).

Wrap the yield in try/finally.

Separately, set_plot_format assigns _PLOT_FORMAT before looking up the mime
type, so an unsupported format raises KeyError with _PLOT_FORMAT already
changed and _PLOT_MIME_TYPE still holding the old value. On 1.4.3,
set_plot_format("jpeg") leaves format 'jpeg' paired with mime 'png'. Look the
mime type up first so a bad format changes nothing.

Adds two tests alongside the existing test_plot_format_ctx_manager, which only
covers the success path.

Signed-Off By Sandeep Singh Rai sandeeprai_dsp@hotmail.com
@coveralls

Copy link
Copy Markdown

Coverage Status

coverage: 88.607% (-0.2%) from 88.791% — WatchTree-19:fix/plot-format-restore into man-group:master

@WatchTree-19

Copy link
Copy Markdown
Author

CI is red here, but as far as I can tell none of it comes from this change. Flagging what I found in case it is useful.

build_3_8 fails in sudo apt-get update during image setup, before any tests run.

build_3_11 and build_3_12 have 8 failures, all in table formatter tests:

  • AttributeError: 'Series' object has no attribute '_append' (6 tests)
  • TypeError: Invalid value '' for dtype 'float64' (2 tests)

This PR only touches pybloqs/block/image.py and tests/unit/block/test_image_unit.py.

Both look like pandas 3 turning existing deprecations into errors. On clean master at pandas 2.3.3 those same code paths already emit the matching FutureWarnings, table_formatters.py:801 for the fillna downcasting and the data_rows.loc[:, ~data_rows.columns.isin(columns)] = "" setitem. build_3_9 and build_3_10 pass, which fits, since they resolve an older pandas.

I could not install pandas 3 in my environment to confirm master fails the same way, so treat that last part as inference rather than something I verified. The 9 tests in test_image_unit.py pass on this branch.

Glad to send the pandas 3 compatibility work as a separate PR if that would help.

@WatchTree-19

Copy link
Copy Markdown
Author

Following up on my last comment, I said the pandas 3 part was inference. I have now confirmed it.

I built a Python 3.12 environment with pandas 3.0.5 and ran clean master with no changes:

FAILED tests/unit/block/test_table_formatters.py::test_FmtAppendTotalsRow_modify_dataframe
FAILED tests/unit/block/test_table_formatters.py::test_FmtAppendTotalsRow_mixed_datatypes
FAILED tests/unit/block/test_table_formatters.py::test_FmtAppendTotalsColumn_modify_dataframe
FAILED tests/unit/block/test_table_formatters.py::test_FmtExpandMultiIndex_modify_dataframe
FAILED tests/unit/block/test_table_formatter_builder.py::test_smokey
5 failed, 60 passed, 1 xfailed

So it is pre-existing and unrelated to this PR. I have opened #177 with the fix, which is independent of this one.

build_3_8 is still a separate matter, it dies in sudo apt-get update before any tests run.

@WatchTree-19

Copy link
Copy Markdown
Author

The three red checks here are all pre-existing rather than caused by this change, and two of them are fixed by #177.

This branch touches pybloqs/block/image.py and tests/unit/block/test_image_unit.py only.

build_3_11 and build_3_12 — 8 failures, every one in table-formatter code:

8 failed, 282 passed, 1 skipped, 3 xfailed
tests/unit/block/test_table_formatters.py::test_FmtAppendTotalsRow_modify_dataframe
    AttributeError: 'Series' object has no attribute '_append'
tests/unit/block/test_table_formatters.py::test_FmtAppendTotalsColumn_modify_dataframe
    TypeError: Invalid value '' for dtype 'float64'
tests/unit/block/test_table_formatters.py::test_FmtExpandMultiIndex_modify_dataframe
tests/unit/block/test_table_formatter_builder.py::test_smokey
tests/integration/test_table.py::test_formatters_with_operator_on_df_with_nans_replaces
tests/integration/generate/test_table.py::test_df_to_jinja_table_add_extra_formatters
tests/integration/generate/test_data_tables.py::test_df_to_data_tables_jinja_table_add_extra_formatters

These are pandas 3 removals — Series._append is gone and setitem is strict about dtype. pandas 3 only resolves on 3.11+, which is why build_3_9 and build_3_10 pass and 3.11/3.12 don't.

#177 is the fix for exactly those, and it's green on 3.11 and 3.12. So merging #177 first should clear these; I'll rebase this onto master afterwards.

build_3_8 — the CircleCI image sudo apt-get update failure (Chrome repo, NO_PUBKEY FD533C07C264648F, exit 100). Same step, same error, on #177. Not code-related.

Nothing here needs changing on my side until #177 lands — happy to be told otherwise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants