fix(postprocessing): correct boolean parsing in MemoryFileIterator CSV reader#417
Open
awanawana wants to merge 1 commit into
Open
Conversation
…V reader
In Python, bool("False") evaluates to True because any non-empty string
is truthy. This caused MemoryFileIterator._format_csv() to incorrectly
parse "False" values as True when reading exported memory CSV files.
Fix: replace bool(item) with item == "True" so that only the string
"True" maps to True and "False" maps to False.
Also add tests/utils/test_postprocessing.py with 17 tests covering:
- MemoryFileIterator with NumPy (.npz), PyTorch (.pt), and CSV formats
- Boolean True/False parsing (including the bug regression tests)
- Empty glob, unsupported format, iterator protocol, and edge cases
- The postprocessing module previously had zero test coverage
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes a real bug in
MemoryFileIterator._format_csv()and adds the first test coverage for theskrl.utils.postprocessingmodule.Bug Fix
File:
skrl/utils/postprocessing.pyIn Python,
bool("False")evaluates toTruebecause any non-empty string is truthy. This caused the CSV memory reader to silently corrupt boolean data — every"False"value in an exported memory CSV was read back asTrue.This affects any user who exports a memory with boolean tensors (e.g.
terminated,truncated) to CSV format and then reads it back withMemoryFileIterator.Tests Added
File:
tests/utils/test_postprocessing.py(new — 17 tests)The
skrl.utils.postprocessingmodule previously had zero test coverage. This PR adds comprehensive tests forMemoryFileIteratorcovering:All 17 tests pass after the fix. The two boolean regression tests fail against the original code, confirming the bug.