chore: Add option to capture screenshots without cropping#237
Conversation
…vudual screenshots
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #237 +/- ##
==========================================
+ Coverage 92.19% 93.06% +0.86%
==========================================
Files 18 20 +2
Lines 423 476 +53
Branches 69 84 +15
==========================================
+ Hits 390 443 +53
Misses 29 29
Partials 4 4 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
| * Base class for screenshot page objects. Provides scroll helpers, | ||
| * full-page screenshot support, and window size management. | ||
| */ | ||
| export default class ScreenshotBasePageObject extends BasePageObject { |
There was a problem hiding this comment.
Base class which the old ScreenshotPageObject and the new RawScreenshotPageObject extend. Its methods are taken from ScreenshotPageObject.
| export async function cropAndCompare( | ||
| firstScreenshot: ScreenshotWithOffset, | ||
| secondScreenshot: ScreenshotWithOffset | ||
| firstScreenshot: Screenshot, |
There was a problem hiding this comment.
cropAndCompare now supports both approaches: comparing screenshots with raw PNG and with decoded image + cropping data.
| * Raw permutation screenshot captured via takeElementScreenshot. | ||
| * No decoded image, no offset — just the raw base64 PNG per element. | ||
| */ | ||
| export interface RawPermutationScreenshot extends RawScreenshot { |
There was a problem hiding this comment.
Initially I attempted to let the existing resources such as ScreenshotPageObject provide both options (raw screenshots vs decode), but it became way too complicated to do it in a backwards compatible way respecting the types that each consumer expects. So I added a sibling class RawScreenshotPageObject instead, which the visual regression action on Github will use.
This PR introduces a new RawScreenshotPageObject class (sibling of the existing ScreenshotPageObject) which allows to take screenshot objects in a way that is more performant in Github runners.
Details:
To compare screenshots, we first get a PNG (compressed) image from the browser, which we then decode (into a flat pixel-by-pixel format) so that we can compare it with
pixelmatch, and then encode it again to attach it to the test results. So:pixelmatchIn step 2, we actually have a shortcut (here): if the encoded PNG is identical for both screenshots, then we know that the decoded images are identical too (since PNG is lossless). This means that we don't always call
pixelmatch; however, in CI (Github runners), this optimization is not enough —even if we don't compare the images, we are still decoding and encoding them, and it turns out, these operations are a lot more expensive (around 10x) in that environment than in our internal pipeline.There is one pragmatic reason we decode and re-encode images even if their raw (compressed) PNG is identical: permutations. In order to get each permutations, we manually crop one single image, and for that we need the decoded pixel data. The new class avoids that by using Webdriver's
takeElementScreenshotAPI instead oftakeScreenshot, to take screenshots of particular elements instead of the entire page —this way, there is no need to crop so there is also no need to decode and encode at all unless the raw PNG comparison fails.Caveat: applying this approach to the internal pipeline turned out to not only not be beneficial but even detrimental. There is a tradeoff between the extra cost of taking more screenshots and (increases run time by about 2x in the internal pipeline) and the extra cost of decoding and encoding images (increases run time by about 10x on Github runners). So we need to provide both approaches.
Tested by implementing RawScreenshotPageObject in cloudscape-design/components#4509. Once this PR is merged and released, it will be possible to remove the duplicated implementation from the components package and use this one.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.