diff is not listed in the report

[sag1v]

For some reason (sometimes) when i run the visual tests (using storycap for capturing the images) reg-cli logs a certain diff (which is a valid change) but when i run the update script, it logs that 2 files were updated. The issue here is that the 2nd file was not even listed in the report.

When i push the changes to GitLab i can see a diff, it seems like the file size is the only thing that is different (i'm not sure why though).

It doesn't happen all the time so its hard to understand what is causing it, maybe storycap has a bug or a config i am missing that makes it generate a different image size.

Note that i am running the tests in a docker, so i doubt its an environment issue.

This is how i run it:

yarn reg-cli somePath somePath somePath -R somePath/report.html --extendedErrors --matchingThreshold 0.1

[sag1v]

@bokuweb Do you have an idea what can cause this?

[Jeiwan]

You have this parameter passed:
--matchingThreshold 0.1
Which increases the tolerance. Unset it to see the diff.

[sag1v]

@Jeiwan Yeah i use it because sometimes i get random false positive diffing.

The issue is that when we run the update command it doesn't ignore those diffs, for example when you run the test it shows 1 file changed, then when you run the update it shows it updated 3 files and this is why i get the diff in git.

reg-cli should ignore those files and provide same results on testing and updating

When running the reg-cli for testing:

When running reg-cli for update:

[sag1v]

@bokuweb Do you have any idea what can cause this behavior?

reg-cli run and show 1 file changed
reg-cli with update actually updates 3 files