Pixel-Diff Thresholds for Visual Regression Testing in Remotion
By RenderComp Team Editorial policy
Remotion turns React code into a video file, so your animations live in version control next to the rest of the project. A video that lives in git can also break in a pull request. Someone bumps a spring config, renames a CSS variable, or swaps a font weight, and frame 45 of a title card changes. Nobody notices until the render has shipped.
Remotion computes every frame as a function of useCurrentFrame() and the composition props, so you can check that frame the same way you check any other output. Render a still for a fixed frame number, store it as a baseline, and compare the next render against it. The hard part is the comparison. If it is too strict, every anti-aliasing wobble fails CI. If it is too loose, a 2-pixel layout shift gets through.
This guide builds the comparison from scratch. It uses two thresholds you can reason about, and it covers how renderStill output options such as format, JPEG quality, and scale change the numbers you should pick.
A composition with predictable test frames
Pick test frames on purpose, and make the animation tell you where they are. This title card drives its entrance with spring and a fixed durationInFrames, so the motion is finished by frame 30 at 30 fps. That gives three frames with different jobs. Frame 0 is the initial state, frame 15 is mid-motion, and frame 45 is the settled layout.
// src/TitleCard.tsx
import React from "react";
import {
AbsoluteFill,
interpolate,
spring,
useCurrentFrame,
useVideoConfig,
} from "remotion";
export const TitleCard: React.FC<{ title: string }> = ({ title }) => {
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
// durationInFrames: 30 stretches the spring so it finishes at frame 30,
// which makes frame 45 a stable "settled" snapshot target.
const progress = spring({ frame, fps, durationInFrames: 30 });
const translateY = interpolate(progress, [0, 1], [80, 0]);
const opacity = interpolate(frame, [0, 20], [0, 1], {
extrapolateRight: "clamp",
});
return (
// An explicit background keeps the alpha channel at 255 everywhere,
// so transparency never shows up in the diff by accident.
<AbsoluteFill style={{ backgroundColor: "#0f172a", justifyContent: "center", alignItems: "center" }}>
<h1
style={{
color: "white",
fontSize: 120,
opacity,
transform: `translateY(${translateY}px)`,
}}
>
{title}
</h1>
</AbsoluteFill>
);
};
The mid-motion frame catches the most regressions. A change to the spring curve can leave frames 0 and 45 identical while frame 15 moves several pixels.
Rendering baselines with renderStill
renderStill can write png, jpeg, webp, or pdf, and the default is png. Keep the default for snapshots. PNG is lossless, so two renders of the same frame should differ only where rendering itself differs, with no added encoder noise.
The script below bundles the project once, selects the composition, and writes one PNG per test frame. It takes a target directory, so the same script can produce both baseline and current.
// scripts/render-snapshots.ts
import path from "node:path";
import fs from "node:fs";
import { bundle } from "@remotion/bundler";
import { renderStill, selectComposition } from "@remotion/renderer";
const target = process.argv[2] ?? "current"; // "baseline" or "current"
const outDir = path.resolve("__snapshots__", target);
fs.mkdirSync(outDir, { recursive: true });
// Pin everything that affects pixels. Changing any of these
// invalidates the baselines, so treat them as part of the snapshot.
const SNAPSHOT_SCALE = 1;
const TEST_FRAMES = [0, 15, 45];
const serveUrl = await bundle({ entryPoint: path.resolve("src/index.ts") });
const inputProps = { title: "Release Notes" };
const composition = await selectComposition({
serveUrl,
id: "TitleCard",
inputProps,
});
for (const frame of TEST_FRAMES) {
await renderStill({
composition,
serveUrl,
inputProps,
frame,
imageFormat: "png", // lossless; also the default
scale: SNAPSHOT_SCALE,
output: path.join(outDir, `TitleCard-${frame}.png`),
});
}
Commit __snapshots__/baseline and ignore __snapshots__/current. The baseline is the reviewed truth. The current directory is scratch output for each run.
Two thresholds instead of one
A single “max differing pixels” number mixes two questions. The first is how different a pixel has to be before it counts as changed. The second is how many changed pixels it takes to fail the test. Splitting them lets you absorb tiny color shifts without hiding a large layout change.
channelTolerance is the largest per-channel delta, on a 0 to 255 scale, that still counts as unchanged. maxChangedRatio is the fraction of pixels allowed to exceed that tolerance. The diff below reads both PNGs with pngjs and returns enough detail to tune both numbers later.
// scripts/diff.ts
import fs from "node:fs";
import { PNG } from "pngjs";
export type Thresholds = {
channelTolerance: number; // 0-255, per R/G/B/A channel
maxChangedRatio: number; // 0-1, fraction of total pixels
};
export type DiffResult = {
changedPixels: number;
totalPixels: number;
changedRatio: number;
maxDelta: number; // largest channel delta seen anywhere, for calibration
passed: boolean;
};
export function diffPng(
baselinePath: string,
currentPath: string,
t: Thresholds,
diffOutPath?: string,
): DiffResult {
const a = PNG.sync.read(fs.readFileSync(baselinePath));
const b = PNG.sync.read(fs.readFileSync(currentPath));
// Different dimensions mean scale or composition size changed.
// Fail hard instead of resizing; resizing would invent pixels.
if (a.width !== b.width || a.height !== b.height) {
throw new Error(
`Size mismatch: baseline ${a.width}x${a.height}, current ${b.width}x${b.height}`,
);
}
const totalPixels = a.width * a.height;
const diff = new PNG({ width: a.width, height: a.height });
let changedPixels = 0;
let maxDelta = 0;
for (let i = 0; i < a.data.length; i += 4) {
let pixelDelta = 0;
for (let c = 0; c < 4; c++) {
pixelDelta = Math.max(pixelDelta, Math.abs(a.data[i + c] - b.data[i + c]));
}
maxDelta = Math.max(maxDelta, pixelDelta);
const changed = pixelDelta > t.channelTolerance;
if (changed) changedPixels++;
// Changed pixels in red, unchanged ones as a dimmed copy of the baseline.
diff.data[i] = changed ? 255 : a.data[i] >> 2;
diff.data[i + 1] = changed ? 0 : a.data[i + 1] >> 2;
diff.data[i + 2] = changed ? 0 : a.data[i + 2] >> 2;
diff.data[i + 3] = 255;
}
if (diffOutPath) fs.writeFileSync(diffOutPath, PNG.sync.write(diff));
const changedRatio = changedPixels / totalPixels;
return {
changedPixels,
totalPixels,
changedRatio,
maxDelta,
passed: changedRatio <= t.maxChangedRatio,
};
}
The per-pixel delta uses the maximum across channels, not the sum. A pure red-to-orange shift then counts as one channel moving, and alpha changes are caught on their own.
Why the ratio has to be relative
The scale option of npx remotion still (and the scale field of renderStill) must be greater than 0 and at most 16. The default is 1. Scale multiplies both dimensions, so the pixel count grows with the square. A 1920×1080 composition at scale 2 renders 3840×2160, which is four times as many pixels.
An absolute budget such as “500 pixels may differ” means something different at each scale. As a ratio, the same budget stays meaningful at scale 0.5 for fast CI and at scale 2 for a high-resolution check. The baselines still have to match the scale exactly, which is why the diff throws on a size mismatch and does not compare across sizes.
Calibrating the tolerance from real noise
Pick channelTolerance from a measurement, not a guess. Render the baseline on one machine and the current set on the machine that runs CI, with no code changes between them. Then diff with a tolerance of 0 and read maxDelta and changedRatio for every frame.
// scripts/calibrate.ts
import path from "node:path";
import { diffPng } from "./diff";
const frames = [0, 15, 45];
for (const frame of frames) {
const name = `TitleCard-${frame}.png`;
const r = diffPng(
path.resolve("__snapshots__/baseline", name),
path.resolve("__snapshots__/current", name),
{ channelTolerance: 0, maxChangedRatio: 1 }, // record everything, fail nothing
);
console.log(
`frame ${frame}: maxDelta=${r.maxDelta} changedRatio=${r.changedRatio.toExponential(2)}`,
);
}
If maxDelta is 0 on every frame, the environments match. You can run with channelTolerance: 0 and a maxChangedRatio of 0. If the deltas are nonzero but small, set the tolerance just above the observed maxDelta. Then confirm that a real change still fails: shift the translateY start value from 80 to 82 and check that frame 15 goes red.
Thresholds can differ per frame. The settled frame 45 should be strict, because nothing on it moves. The mid-motion frame 15 is where a looser ratio might be tempting. It is also where the spring regressions show up, so loosen it only as far as the calibration run requires.
// scripts/check.ts
import path from "node:path";
import { diffPng, type Thresholds } from "./diff";
const thresholds: Record<number, Thresholds> = {
0: { channelTolerance: 2, maxChangedRatio: 0 },
15: { channelTolerance: 2, maxChangedRatio: 0.0005 },
45: { channelTolerance: 2, maxChangedRatio: 0 },
};
let failed = false;
for (const [frameStr, t] of Object.entries(thresholds)) {
const name = `TitleCard-${frameStr}.png`;
const r = diffPng(
path.resolve("__snapshots__/baseline", name),
path.resolve("__snapshots__/current", name),
t,
path.resolve("__snapshots__/current", `diff-${name}`),
);
console.log(`${name}: ${r.changedPixels}/${r.totalPixels} changed, passed=${r.passed}`);
if (!r.passed) failed = true;
}
process.exit(failed ? 1 : 0);
The numbers in this map are placeholders for your own calibration output. Replace them with what calibrate.ts printed on your CI runner.
What JPEG does to the thresholds
jpegQuality must be an integer from 0 to 100, and you can only pass it when imageFormat is jpeg. If your pipeline ships JPEG thumbnails and you want to snapshot exactly what ships, both the format and the quality become part of the snapshot. A baseline encoded at quality 80 and a candidate at quality 90 will differ along every edge, even with identical React output.
Pin jpegQuality in the render script the same way SNAPSHOT_SCALE is pinned. Then rerun the calibration. A lossy encoder raises the noise floor, so the tolerance from a PNG calibration will not carry over to JPEG. It is cleaner to keep the regression snapshots in PNG and test the JPEG export separately with its own looser thresholds. That way a compression change cannot hide a layout change.
Your next step is to run render-snapshots.ts baseline on the CI runner itself, commit the output, and run calibrate.ts against a second render on that same runner. Whatever maxDelta it prints is your starting channelTolerance.
Now available
Get 1,000+ Remotion Templates
Pay once — no subscription. Lifetime updates. TypeScript-first.
View pricing →Free 50
Get the 50 templates as a ZIP
Enter your email and the ZIP link arrives right away.
Send me the 50-template ZIP. I agree to receive RenderComp template updates and product news, including the paid library (a few emails, one-click unsubscribe). Privacy policy
You do not have to use email. The GitHub repository stays public and needs no signup. Open the repository
On its way
Check your inbox for the ZIP link, and your spam folder if it is not there. You can also take it straight from GitHub right now.