Debugging SDKs

👍

Covered in this doc

• How Percy SDKs work
• How Percy's infrastructure works
• SDK debugging tips
• Common issues

How Percy works

To effectively debug Percy builds you’ll have to have an understanding of how Percy works from the SDKs to how we capture screenshots. We’ll explain how both the SDKs and the infrastructure works in this document, but we’ll only cover how to debug/troubleshoot SDK level issues.

At a high level, Percy works by capturing a snapshot of the DOM in the test browser. From there the DOM is sent to Percy’s API to eventually be rendered concurrently across browsers/widths for screenshots. It’s the SDKs job to capture the DOM and assets. It’s the APIs job to proxy & render those snapshots in browsers for a screenshot. It’s important to call out screenshots are not captured in your test suite. They are captured in Percy’s infrastructure.

How SDKs work

There are two major steps in the Percy SDK to know about:

• Capturing the DOM state
• Capturing assets (Asset discovery)

Capturing DOM state

When percySnapshot is called, Percy’s SDKs will capture the DOMs exact state. The SDK will serialize the current page state into the DOM by applying form element values, capturing CSSOM, capturing accessible iframes, and capturing canvas elements to be images. This is handled by the @percy/dom package. We serialize those elements because their state is held in page memory and not encoded into the DOM. Without that serialization, that content would be missing from the snapshot.

Input elements

Input elements (input, textarea, select) are serialized by setting respective DOM attributes to their matching JavaScript property counterparts. For example, checked, selected, and value.

Frame elements

Frame elements are serialized when they are CORS accessible and if they haven’t been built by JavaScript when JavaScript is enabled. They are serialized by recursively serializing the iframe’s document element with the @percy/dom library.

CSSOM rules

When JavaScript is not enabled, CSSOM rules are serialized by iterating over and appending each rule to a new stylesheet inserted into the document’s head.

Canvas elements

Canvas elements’ drawing buffers are serialized as data URIs and the canvas elements are replaced with image elements. The image elements reference the serialized data URI and have the same HTML attributes as their respective canvas elements. The image elements also have a max-width of 100% to accommodate responsive layouts in situations where canvases may be expected to resize with JS.

Video elements

Videos without a poster attribute will have the current frame of the video serialized into an image and set as the poster attribute automatically. This is to ensure videos have a stable image to display when screenshots are captured.

Asset discovery

Once the DOM is captured & serialized, it is sent to @percy/core for asset discovery. Asset discovery will render the captured DOM in a Chromium browser where the SDK intercepts all network requests the DOM makes. This is to capture assets that are needed to render the page in Percy’s infrastructure for a screenshot. By default, all assets served on the same hostname as the tests will be captured. You can capture more hostnames with the allowed-hostnames config key. Asset discovery will also resize the viewport to the passed widths to ensure assets are captured for the right screen sizes.

Asset discovery by default will wait 100ms for no new network requests to be made by the captured DOM. Once that timeout has been reached asset discovery will close for the given snapshot. It’s not uncommon to have to increase the network-idle-timeout to allow for more network requests to be made.

Since Percy re-renders the DOM in a browser outside of your test suite, you may need to provide authentication to the requests this browser is making. The discovery configuration key in Percy’s SDKs provides a few ways to authenticate requests like request-headers, authorization, and cookies.

How the infrastructure works

With the DOM captured and the right assets gathered to render the page, it’s time to capture the screenshot. Percy will re-render the page concurrently across browsers/widths for screenshots.

By default, all snapshots are rendered with JavaScript disabled. This is because JavaScript will have already ran and modified the page before the DOM is captured. Enabling JavaScript in Percy’s infrastructure is possible but it typically results in unexpected issues. Web pages usually aren’t built to handle rendering with an already fully formed DOM. This could cause issues like redirects or serialized states being lost (clearing inputs, etc).

When the page is re-rendered, Percy modifies the captured DOM slightly to do things like remove <noscript> tags and freeze CSS animations.

If you think you’re experiencing an infrastructure issue, keep reading to make sure it’s not a common solvable issue. Otherwise, please reach out to support.

Debugging SDK’s

All Percy SDKs use @percy/cli, so the way to debug snapshot issues will be the same across all SDKs.

Debug vs verbose logging

If you’re sure you’re debugging an asset issue, it’s not worth using up your Percy screenshots while tracking down what’s going wrong. The --debug CLI flag will run do everything the SDK normally does except create build & upload the snapshots. The DOM will be captured and asset discovery will run over the captured DOM while also enabling verbose logs.

$ npx percy exec --debug -- [test command]

If you still would like to create Percy build, you can use the --verbose CLI flag. This will enable verbose logs and also upload the captured snapshots to Percy’s API.

$ npx percy exec --verbose -- [test command]

Most SDK debugging will require reading and understanding the SDKs verbose logs. Percy’s logs have labels that specify which package the log is coming from.

[percy:core] Percy has started! (3708ms)
[percy:cli:exec] Running "node index.js" (0ms)
[percy:core] --------- (4164ms)
[percy:core] Handling snapshot: (0ms)
[percy:core] -> name: example (0ms)
[percy:core] -> url: https://percy.io/ (0ms)
[percy:core] -> widths: 375px (0ms)
[percy:core] -> minHeight: 1024px (1ms)
[percy:core] -> clientInfo: @percy/puppeteer/2.0.0 (0ms)
[percy:core] -> environmentInfo: puppeteer/7.0.1 (0ms)
[percy:core:page] Initialize page (189ms)
[percy:core:page] Resize page to 375x1024 (29ms)
[percy:core:page] Navigate to: https://percy.io/ (2ms)
[percy:core:discovery] Handling request: https://percy.io/ (5ms)
[percy:core:discovery] -> Serving root resource (0ms)
[percy:core:discovery] Handling request: https://use.typekit.net/mzg8eqc.css (49ms)
[percy:core:discovery] Handling request: https://percy.io/static/assets/vendor-3dfb089bbaa3ebb01cb92c46e5ba8505.css (2ms)
[percy:core:discovery] Handling request: https://percy.io/static/assets/percy-web-cdf077c733a1a4f0148f7b50a9ea0132.css (1ms)
[percy:core:discovery] Processing resource: https://percy.io/static/assets/vendor-3dfb089bbaa3ebb01cb92c46e5ba8505.css (131ms)
[percy:core:discovery] -> sha: 6671e9721911f6e1343bb865b48848b4769874c6640e302ec06e5bbdc6c3b50a (1ms)
[percy:core:discovery] -> mimetype: text/css (0ms)

For example [percy:core] means the log is coming from the @percy/core package. This helps figure out if the log is from the client SDK or one of the packages that make up @percy/cli.

Display the asset discovery browser

Sometimes it’s easier to figure out what’s going on by actually watching the asset discovery browser render your captured page. This can be done by setting headless: false in the discovery.launch-options config.

version: 2
discovery:
  launch-options:
    headless: false

Common issues

Asset(s) never requested by asset discovery

Asset discovery by default will wait 100ms for no new network requests to be made by the captured DOM. Once that timeout has been reached asset discovery will close for the given snapshot. It’s not uncommon to have to increase the network-idle-timeout to allow for more network requests to be made. For example:

version: 2
discovery:
  network-idle-timeout: 250 # ms

This example config will now wait for 250ms for zero network requests to be made before closing asset discovery. The SDKs will accept a value up to 700ms, if you need anything more, it’s likely not related to the network-idle-timeout.

Assets not captured

The most common reason assets fail to capture is due to authentication issues. Since Percy re-renders the DOM in a browser outside of your test suite, you may need to provide authentication to the requests this browser is making. The discovery configuration key in Percy’s SDKs provides a few ways to authenticate requests like:

• request-headers: An object containing HTTP headers to be sent for each request made during asset discovery.
• authorization: A username/password combo to authenticate requests for Percy.
• cookies: Cookies to use for discovery’s browser session

Assets only accessible locally

Since Percy re-renders the DOM in a browser outside of your test suite you may need to tell the SDK to capture remote assets so they’re available locally to Percy when rendering for a screenshot. This is common if the assets are only accessible on your local VPN and not publicly accessible.

By default, the SDK will capture all assets served on the same hostname as where the tests were running. For example, the tests are run on percy.io, all assets with the percy.io hostname will be saved locally. Assets served from cdn-example.percy.io will not be captured without adding that hostname to the allowed-hostnames config.

version: 2
discovery:
  allowed-hostnames:
	  - cdn-example.percy.io

Asset discovery timeout after 30 seconds

Asset discovery will stay open for 30 seconds per-snapshot, if it takes longer than 30 seconds to discover & capture assets there's likely something wrong (a test shouldn't take longer than 30 seconds to execute).

Unfortunately, there isn't a single line config fix for this. You will have to read the debug logs from asset discovery to find which requests are hanging or resolving slowly. Double check your network-idle-timeout confg isn't set too high.. Look for slow requests and double check you're not capturing assets from hostnames that aren't necessary.

Disallowing requests

If you see a URL in the logs that is causing these timeouts, you can use the disallowedHostnames config option to have that hostname skipped by asset discovery. For example, ad network calls are almost never useful in Percy and commonly hang up asset discovery.

version: 2
discovery:
  disallowedHostnames:
    - ad.doubleclick.net

This will now no longer run any hostname matching ad.doubleclick.net through asset discovery.

Canvas elements not captured/bad state

Since canvas elements state only exist in the page memory, Percy will serialize the current canvas elements’ drawing buffers are serialized as data URIs. The canvas elements are then replaced with image elements in the cloned DOM.

The image elements reference the serialized data URI and have the same HTML attributes as their respective canvas elements. The image elements also have a max-width of 100% to accommodate responsive layouts in situations where canvases may be expected to resize with JS.

Incorrect state

If your canvas element has an incorrect state or is changing states between snapshots, you’ll want to make sure your page is settled before capturing a Percy snapshot. Once a snapshot is captured, the exact state of that canvas element will be serialized into an image.

WebGL canvas elements

The SDK captures the canvas element state by using the toDataURL API. This web API works by writing the state of the canvas drawing buffer and providing an image as a data URI.

WebGL canvas elements wipe the drawing buffer right after render, which is not how 2D canvas elements work. To serialize WebGL canvas elements as images, you will have to tell your canvas to preserve the drawing buffer: { preserveDrawingBuffer: true }

Videos & poster's

In order to display something for <video> elements, videos without a poster attribute will have the current frame serialized into an image and set as the posterattribute automatically. This is to ensure videos have an image to display when screenshots are captured.

It’s possible for this to cause flakes within your tests if the video is playing. Percy’s SDKs will capture the current frame of the video when percySnapshot is called. If this is causing flakes for you, the best path forward would be to provide your own poster image for the video. This will ensure the same image will be captured and used when screenshots are captured.

Iframe content not captured (CORS)

Frame elements are serialized when they are CORS accessible and if they haven’t been built by JavaScript when JavaScript is enabled. They are serialized by recursively serializing the iframe’s own document element with the @percy/dom library. Once the frame has been serialized, it’s set as a srcdoc on the iframe element.

If an iframe is not being correctly captured it could be either due to timing or the iframe not being accessible because of CORS restrictions. If you control the frame origin, you can add headers to allow cross-origin access from the root origin.

Responsive DOM changes

Percy’s SDKs currently capture a single DOM snapshot and then re-render that DOM across the snapshot widths. This approach assumes the DOM does not change at different widths.

If your application’s DOM changes at different widths, you will need to resize the test browser to the correct width and then capture a Percy snapshot. This is so the DOMs state is correct for the passed width.

How you achieve this is different between test frameworks but the principals are the same.

• Resize the test browser
• Capture a Percy snapshot for just that width

For example, in Cypress:

// A helper to make this easy, feel free to edit
Cypress.Commands.add('percyResponsiveSnapshot', (name, width, options = {}) => {
  delete options.widths // we never want to use those in this helper

  cy
    // https://docs.cypress.io/api/commands/viewport.html#Syntax
    .viewport(width)
    .percySnapshot(`${name} - ${width}`, { widths: [width], ...options })
    // Set back the orignal width if you'd like
    //.viewport()
});

// usage:
cy.percyResponsiveSnapshot('Homepage', 320) 

Invalid HTML

If you're seeing the page structure change in your snapshots it's likely due to invalid HTML being present in the DOM snapshot. For example, putting a <div> inside of a <button> element is not valid HTML. When a browser re-renders an HTML page with invalid DOM, the browser will correct it.

The only way to fix this is to ensure you're creating valid HTML that aligns with the spec. It's not Percy correcting the invalid HTML, it's the browser(s).

Bad character encoding

If you're seeing weird encoding on characters in your snapshot, it's likely due to a missing charset meta tag in the <head> of your web page. For more information, MDN provides good documentation about character encoding.

If you are absolutely sure your webpage includes a charset meta tag in the <head> of the document, make sure there isn't invalid HTML being placed into the <head>. We commonly see <div>s, <iframe>s, <img>s, and other types of content that cannot render in the <head> of an HTML document. Browsers will correct his bad HTML at render time, moving the invalid content along with everything after it out of the <head>. The HTML spec specifies only metadata content can be children of a <head> tag.

Chromium failed to launch browser

Asset discovery in the SDKs requires a Chromium browser in order to work. This error can have many flavors to it, but usually, when this error occurs it’s because the docker image in CI does not have the correct dependencies needed to launch the browser.

You can use a docker image that can launch a browser. Or googling the error message usually will return results for what dependencies you will need to install in your docker image.

If this issue persists, feel free to open a discussion in the CLI repo

Screenshots aren’t a full-page

By default, all browsers will capture a full-page screenshot, not just the visible viewport. If your snapshots are only of the visible viewport it likely means your page has CSS that is hiding the content overflow. Browsers will respect this CSS and only capture a screenshot of the visible portion of the page.

You can work around this in Percy by finding the element(s) that clip the overflown content and unset that CSS with Percy CSS. For example:

version: 2
snapshot:
  percy-css: |
    .container { overflow: unset !important; }

Broken lazy loading images

If you’re using a custom lazy image loading library, you most likely will have to scroll the page before taking a Percy snapshot. Libraries like lozad.js or lazyload wait to set the <img>’s src attribute until the page is scrolled near the image. Once the page scroll intersects with the image, the library will apply the src with the correct image.

For snapshots to render correctly, the page will need to be scrolled before a Percy snapshot is captured. Percy does not automatically scroll the page for you. If images are not rendering correctly in your snapshots, it’s likely the page did not scroll past the image (and did not trigger the intersection observer).

You can read more about lazy loading images, Percy, and examples here.

Proxying requests

To Percy's API

Percy's SDKs implement the industry convention for proxying requests via HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables.

# unix example, set the env variable the way your OS/System expects
$ export HTTP_PROXY=http://a-proxy.example.com:1234
$ export HTTPS_PROXY=http://another-proxy.example.com:1234
$ export NO_PROXY=http://example.com:1234

Asset discovery

If you need to proxy requests that occur in asset discovery, you may need to pass a --proxy-server CLI flag as a browser launch arg in the Percy discovery config.

version: 2
discovery:
  launch-options:
    args: ["--proxy-server=http://a-proxy.example.com:1234"]