Base build selection
Visual testing workflow and base build selection logic
Covered in this doc
- Introducing base builds
- The common cases
- Overriding defaults
- The full details
Introducing base builds
When you push changes to your codebase and have Percy generate screenshots, Percy groups the screenshots in snapshots, and snapshots into a build. New screenshots are compared to baseline screenshots from a previous build. This previous build is also known as the base build.
Percy takes a lot of care in automatically selecting the best base build to give you clean, accurate diffs. This page documents how Percy determines what build to use as the base build when generating comparisons.
The common cases
In almost all cases Percy's defaults "just work", but if you are looking for only latest approved build is selected as base build for comparisons then use "Approval Require Branches" feature from project settings, which is designed to work for the use cases where builds are getting created from a single branch.
This section explains the three common types of Percy builds, and the common base build selection for them. The descriptions below assume you're using Percy from a CI service we support and have our source code repository integration in place.
Pull request builds
The most common use case for Percy is visual reviews associated with pull requests. To select the base build, Percy will first determine the target branch of the pull request (often master). It will then determine the common ancestor commit (the merge base) between the source branch and target branch. Once it has that target branch and common ancestor commit, it will choose the Percy build associated with them as the base build.
This provides you with the best visual review, showing only the changes introduced by this pull request, avoiding any other changes that may have occurred on the target branch in the interim.
Feature branch builds
Feature branch builds are similar to pull request builds, with the exception that they don't have a pull request specifying the target branch. In this case, the 'master' branch is picked as the target branch. The target commit and the base build are subsequently determined in the same way as the pull request builds described above.
If you typically merge into a 'development' branch rather than the 'master' branch, you may want to set
PERCY_TARGET_BRANCH to 'development' in your environment variables, to specify development as the default base branch.
Master branch builds
Percy uses the most recent finished build on the master branch as the base builds for new master branch builds.
We encourage you to run Percy builds for every commit on the master branch, as these provide the baseline builds for the pull request and feature builds described above.
Overriding the defaults
Setting the projects default base branch
If git projects default branch is different from
master, you will want to update your Percy projects settings to mirror that. By default, Percy uses
master as the default branch, but it's not uncommon for teams to use a branch like
develop as the main development branch.
To update this setting, you will need to go to your projects settings page: percy.io///settings Once you're on the settings page, look for a "Branch settings" heading and update the "Default base branch" input:
Overriding the default base branch
Sometimes you will want to override the default baseline for certain builds. To do this, you will need to set a
PERCY_TARGET_BRANCH environment variable. The value of
PERCY_TARGET_BRANCH will be the branch that has a Percy build which you want to use as a baseline.
For example, if you want to compare your current checked out branch to the
staging branch, you would set to
PERCY_TARGET_BRANCH=staging. That will use the last Percy build on
staging as the baseline to generate comparisons.
Note that if you have an SCM integration,
PERCY_TARGET_BRANCH will only be used for builds NOT associated with a pull request. A pull request's target branch will take priority over the branch specified by
PERCY_TARGET_BRANCH. To override a base build set with a PR, you will also need to set
PERCY_TARGET_COMMIT to a specific commit SHA.
Overriding the default base commit
If you want to specify a specific commit to be used to selecting the base build, you can set the
PERCY_TARGET_COMMIT environment variable to the full commit SHA. This will only work if there is a finished Percy build for that commit.
The full details
Percy uses a variety of strategies to determine the optimal base build for comparison on every build you create. The particular strategy used depends on several factors, including any installed SCM integrations, the default branch of the project, and which commits have previously finished builds.
The chart below demonstrates a simplified view of Percy's base build selection logic. Click the thumbnail below to view the full-size diagram.
(a) Is the build attached to a pull request?
(c) Is there a merge base commit present, and a Percy build for the merge base commit?
If your project has an active source control management integration, like GitHub or GitLab, then Percy will always try to compare builds created in conjunction with pull (or merge) requests to the base branch for the pull request. This ensures that only the changes relevant to the pull request are visible in the Percy diff. If there have been changes on the base branch (e.g.,
master) since the pull request was opened, the visual diff will not reflect those changes.
(b) Has Percy attempted to find a build on the "master" branch?
(d) Is there any previous Percy build on this branch?
Percy's fallback behavior, when no merge base commit build is present, is to compare a build to the latest build on the default base branch. This is the "master" branch by default, but you can override it when creating new builds (see "Overriding the default base branch"). If there are no builds present on the default base branch, then Percy will fall back to using any previous build on the same branch as the build's commit.
(e) Has Percy tried every branch strategy?
If Percy has exhausted all the prior options and has not found a viable candidate for the base build, then no base build will be chosen, and all snapshots will be treated as new. This generally only happens on the first build for a new project, but can also occur when the default base branch is misconfigured (see "Overriding the default base branch" above).
Why does Percy show changes unrelated to my commits?
In most cases, this is because the base build that you would intuit does not have a Percy build, or in projects with high commit volume, because the Percy build for the expected commit did not finish before the head build was created. In these cases, you can wait until the base build is finished, and issue a rebuild for your head commit.
Why do all of my snapshots show "no previous snapshot"?
This can happen when Percy was unable to find any suitable base for comparison for your build. This generally means that no previous build has completed on either your branch or the default base branch. Percy will never compare the results of one branch to another branch, other than the default base branch or the base branch of a pull request.
Updated 3 months ago