Skip to content

chore: Add option to capture screenshots without cropping#237

Open
jperals wants to merge 24 commits into
mainfrom
dev-v3-jotresse-optimized-screenshots
Open

chore: Add option to capture screenshots without cropping#237
jperals wants to merge 24 commits into
mainfrom
dev-v3-jotresse-optimized-screenshots

Conversation

@jperals

@jperals jperals commented Jun 24, 2026

Copy link
Copy Markdown
Member

This PR introduces a new RawScreenshotPageObject class (sibling of the existing ScreenshotPageObject) which allows to take screenshot objects in a way that is more performant in Github runners.

Details:

To compare screenshots, we first get a PNG (compressed) image from the browser, which we then decode (into a flat pixel-by-pixel format) so that we can compare it with pixelmatch, and then encode it again to attach it to the test results. So:

  1. PNG decode
  2. compare with pixelmatch
  3. PNG encode

In step 2, we actually have a shortcut (here): if the encoded PNG is identical for both screenshots, then we know that the decoded images are identical too (since PNG is lossless). This means that we don't always call pixelmatch; however, in CI (Github runners), this optimization is not enough —even if we don't compare the images, we are still decoding and encoding them, and it turns out, these operations are a lot more expensive (around 10x) in that environment than in our internal pipeline.

There is one pragmatic reason we decode and re-encode images even if their raw (compressed) PNG is identical: permutations. In order to get each permutations, we manually crop one single image, and for that we need the decoded pixel data. The new class avoids that by using Webdriver's takeElementScreenshot API instead of takeScreenshot, to take screenshots of particular elements instead of the entire page —this way, there is no need to crop so there is also no need to decode and encode at all unless the raw PNG comparison fails.

Caveat: applying this approach to the internal pipeline turned out to not only not be beneficial but even detrimental. There is a tradeoff between the extra cost of taking more screenshots and (increases run time by about 2x in the internal pipeline) and the extra cost of decoding and encoding images (increases run time by about 10x on Github runners). So we need to provide both approaches.

Tested by implementing RawScreenshotPageObject in cloudscape-design/components#4509. Once this PR is merged and released, it will be possible to remove the duplicated implementation from the components package and use this one.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@codecov

codecov Bot commented Jun 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.06%. Comparing base (bead6a8) to head (335790b).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #237      +/-   ##
==========================================
+ Coverage   92.19%   93.06%   +0.86%     
==========================================
  Files          18       20       +2     
  Lines         423      476      +53     
  Branches       69       84      +15     
==========================================
+ Hits          390      443      +53     
  Misses         29       29              
  Partials        4        4              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

* Base class for screenshot page objects. Provides scroll helpers,
* full-page screenshot support, and window size management.
*/
export default class ScreenshotBasePageObject extends BasePageObject {

@jperals jperals Jun 26, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Base class which the old ScreenshotPageObject and the new RawScreenshotPageObject extend. Its methods are taken from ScreenshotPageObject.

export async function cropAndCompare(
firstScreenshot: ScreenshotWithOffset,
secondScreenshot: ScreenshotWithOffset
firstScreenshot: Screenshot,

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cropAndCompare now supports both approaches: comparing screenshots with raw PNG and with decoded image + cropping data.

* Raw permutation screenshot captured via takeElementScreenshot.
* No decoded image, no offset — just the raw base64 PNG per element.
*/
export interface RawPermutationScreenshot extends RawScreenshot {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially I attempted to let the existing resources such as ScreenshotPageObject provide both options (raw screenshots vs decode), but it became way too complicated to do it in a backwards compatible way respecting the types that each consumer expects. So I added a sibling class RawScreenshotPageObject instead, which the visual regression action on Github will use.

@jperals jperals marked this pull request as ready for review June 26, 2026 17:25
@jperals jperals requested a review from a team as a code owner June 26, 2026 17:25
@jperals jperals requested review from avinashbot and removed request for a team June 26, 2026 17:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant