fix: make tests more lenient with changing serialization of timestamps - #382
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #382 +/- ##
=======================================
Coverage ? 97.29%
=======================================
Files ? 54
Lines ? 5361
Branches ? 0
=======================================
Hits ? 5216
Misses ? 145
Partials ? 0 ☔ View full report in Codecov by Harness. |
2f59c5e to
f2e5599
Compare
| Traceback (most recent call last): | ||
| ... | ||
| ValueError: Values {'MEX'} for 'area (ISO3)' already exist and contain data. ... | ||
| ValueError: Values {...} for 'area (ISO3)' already exist and contain data. ... |
There was a problem hiding this comment.
Hmm, that is a case which makes the example significantly worse to read for a human, right? Knowing that the value which already exists is MEX and not confusingly ARG or so is actually quite helpful.
There was a problem hiding this comment.
True, but then we need to change either the generation of the ValueError (to print __str__ instead of __repr__) or the example to not rely on the exception. The shown value is 'MEX in older versions of numpy and np.str_('MEX') in newer versions
There was a problem hiding this comment.
A similar problem arises from the DataArray serialization. the expected <Quantity([0.4 0.9 1.7 4.8 3.2 9.1], 'hertz')> is actually Quantity(array([0.4, 0.9, 1.7, 4.8, 3.2, 9.1]), "hertz") in the development xarray versions... I don't think we can maintain this doctest across so many different library versions without reducing the output in the documentation to generics like ...Quantity...hertz... which are really hard to parse for humans. A couple of options I see:
- as we talked about: remove the examples from the documentation, add them as unit-tests (or similar) instead and reference those tests in the docs
don't check output in the doctests, only that the provided code actually runs and throws exceptions when it is supposed to (which is honestly what we would also do if they are unit tests instead?)- restrict the doctests to a single test environment and update the expected output from time to time to match the output in that single test environment
There was a problem hiding this comment.
I have ignored the exception detail for the mentioned examples and fixed the still failing tests by making them a bit more robust. I guess we have to make these examples not rely on the repr so much in the future.
For now this should pass all tests - though it probably is still not super robust
There was a problem hiding this comment.
Yeah, using str() instead of repr() in the ValueError is sensible, I think, in this specific instance. But more generally, I think we might drop doctests. A pity, but not feasible given how much work it creates.
…umented function; reverted to more helpful docstring examples
Pull request
Please confirm that this pull request has done the following:
Tests addedDocumentation added (where applicable){pr}.thing.mdfile in the directorychangelogadded - see changelog/README.md for detailsDescription
A few tests failed due to inconsistent precision of timestamps (6 vs. 9 positions after the seconds). Reduced the checks in the tests to the essential part that ensures that the correct failure branch was taken.