zscale Conversion

Table of contents
  1. zscale Conversion
    1. Availability
    2. Recommended commands
    3. What zscale tags
    4. Options
    5. Sources without tags
    6. Things to watch
      1. Make sure zscale is doing the conversion
      2. Decoding untagged Y’CbCr
      3. Geometry
      4. Dithering

zscale is FFmpeg’s interface to the zimg library. For converting image sequences to Y’CbCr movies it is the converter we recommend, in preference to scale (libswscale), because in our tests it was the only one that was correct by default:

  zscale scale (libswscale)
White level, 10-bit or higher RGB to limited range 940 at 10-bit, as specified 943 at 10-bit, 3774 at 12-bit, on every version up to 9.x
Chroma siting, 4:2:2 and 4:2:0 Co-sited, as professional 4:2:2 and H.264/HEVC/AV1 expect Centred by default; co-sited only with two extra options
Scene-linear (float) EXR Handled, including values above 1.0 Clamped to 0.0 - 1.0
Accuracy against the ideal conversion Exact in all our tests (8, 10 and 12-bit; 4:4:4, 4:2:2, 4:2:0) Up to 4 code values off at 10-bit, 14 at 12-bit

See FFmpeg Version Changes, Chroma Siting and FFmpeg OCIO Filter for the measurements. libswscale is still useful when zscale is not available; see FFmpeg swscale Filter, and for resizing see Resizing.

Availability

zscale needs FFmpeg built with zimg (--enable-libzimg). Check with:

ffmpeg -filters | grep zscale

The Docker builds in this repository enable it, and the Conan build enables it by default. Many distribution and static builds include it, but not all.

RGB image sequence to BT.709 Y’CbCr, on every FFmpeg version we tested (6.0 to 9.0):

ffmpeg -i input.%06d.dpx \
  -vf "zscale=matrix=709:range=limited:filter=lanczos,setparams=color_primaries=bt709:color_trc=iec61966-2-1" \
  -c:v prores_ks -profile:v 4444xq -pix_fmt yuv444p10le \
  output.mov
  • matrix=709 and range=limited are the output. There are no input settings: zscale always treats an RGB source as full range, and takes the matrix and range of a Y’CbCr source (JPEG, video) from its tags, so the same command works for both. Only describe the input for a source that has no tags; see Sources without tags.
  • filter=lanczos sets the resampling filter used for the 4:2:2/4:2:0 chroma (and for any resize). zscale’s default, bilinear, gives softer chroma: on 4:2:0 test charts it left 99th percentile edge errors of 61 code values against 46 with lanczos. On 4:2:2 the two were within a code value or so on average, with bilinear very slightly cleaner on hard-edged graphics. The filter makes no difference to the color accuracy inside flat areas, or to the chroma siting.
  • zscale tags its output frames with the matrix and range, which the encoder copies into the file.
  • setparams adds the primaries and transfer, which zscale only sets if you ask it to convert them (see below).
  • -pix_fmt sets the output format; zscale converts straight to it. Use the codec’s own format (for example yuv422p10le for ProRes 422 or DNxHR HQX, yuv420p10le for H.264/HEVC).

The four -color_* flags (-color_range tv -colorspace bt709 -color_primaries bt709 -color_trc ...) are not needed with this filter chain, but are harmless if you prefer to keep them; see FFmpeg Version Changes.

For HDR (BT.2020, PQ), the same pattern with matrix=2020_ncl and setparams=color_primaries=bt2020:color_trc=smpte2084; see HDR and -x265-params.

What zscale tags

The encoder takes its color tags from the frames it receives, so what zscale sets on its output frames is what ends up in the file:

Property Set by zscale Otherwise
Matrix, range Always (matrix=, range=)  
Primaries, transfer Only when converted: give both the input and the output, e.g. primariesin=709:primaries=709:transferin=iec61966-2-1:transfer=iec61966-2-1 setparams
Chroma location Only when chromal= is set Left unset, which is what you want for 4:2:2 MXF; see Chroma Siting Tests

So this is equivalent to the recommended command, without setparams (tested on 6.0 and 9.0, ProRes):

-vf "zscale=matrix=709:range=limited:primariesin=709:primaries=709:transferin=iec61966-2-1:transfer=iec61966-2-1:filter=lanczos"

Options

The options that matter for color conversion (ffmpeg -h filter=zscale lists them all). Each has an input version (...in) describing the source, and an output version; the defaults (input) mean “the same as the input frame”.

Option Short Input option What it sets Default
matrix m matrixin / min Y’CbCr matrix: 709, 2020_ncl, 170m (BT.601), 470bg input
range r rangein / rin limited or full input
primaries p primariesin / pin Color primaries: 709, 2020, … input
transfer t transferin / tin Transfer: 709, iec61966-2-1 (sRGB), smpte2084 (PQ), linear, … input
chromal c chromalin / cin Chroma location: left, center, topleft, … input
filter f   Resampling filter, for resizing and for chroma subsampling: point, bilinear, bicubic, spline16, spline36, spline64, lanczos. We recommend lanczos bilinear
dither d   none, ordered, random, error_diffusion none
npl     Nominal peak luminance, for linear light conversions  

For encoding, the options you normally need are matrix and range. Leave the input options at their default unless the source has no tags (see Sources without tags). The chroma location defaults to co-sited for 4:2:2 and 4:2:0, which is correct for professional formats; leave chromal unset unless your delivery specification asks for something else.

Sources without tags

Both zscale and scale read the input matrix and range from the frames unless you override them (input in zscale, auto in scale; both are the default). What happens depends on the source (from the FFmpeg source, and measured on 6.0 and 9.0):

Source zscale scale (libswscale)
RGB: PNG, DPX, TIFF, EXR Always full range; matrixin and rangein are ignored The input matrix does not apply
JPEG Tagged full range BT.601 by FFmpeg’s decoder, as JFIF requires: correct Correct
Video with tags (ProRes, DNx, H.264, …) Uses the tags: correct Uses the tags: correct
Video with no matrix tag Stops with an error: no path between colorspaces Silently uses BT.601
Video with no range tag Assumes limited Assumes limited

So there is nothing to check for an RGB image sequence, and the only source that needs more on the command line is untagged video. Say what it is, with the input options, for example for HD:

-vf "zscale=matrixin=709:rangein=limited:matrix=709:range=limited:filter=lanczos,setparams=color_primaries=bt709:color_trc=bt709"

BT.709 is the usual assumption for HD and larger; standard definition is usually BT.601 (matrixin=470bg or 170m). For an automated pipeline, zscale’s error is the signal: check ffprobe’s color_space, or catch the error and run again with the input options.

Do not set the input options when the source is tagged. Earlier versions of these guidelines used in_color_matrix=bt709 (scale) and rangein=full (zscale) for every source. Neither changes anything for RGB, but they break everything else:

Source Forced input Result (10-bit code values)
JPEG sequence (4:2:0) scale=in_color_matrix=bt709:... BT.601 decoded as BT.709: a visible hue and saturation shift, mean error 19, 99th percentile 110
BT.709 limited range video zscale=rangein=full:... or scale=in_range=full:... Limited range read as full: white about 70 code values low

Primaries and transfer are a separate question. setparams sets what the output is labelled as, and should describe the signal your pipeline produced (for example, the OCIO display and view it rendered to), not whatever the source file says. Image files rarely say anything useful: FFmpeg 9.0 reads a PNG cICP chunk and reports EXR as linear, but DPX and TIFF come out unknown, and 6.0 reads none of them. Source tags are worth checking as a warning, for example a PNG tagged PQ in a pipeline that is labelling its output sRGB.

Things to watch

Make sure zscale is doing the conversion

If the format that follows zscale is one it cannot output, FFmpeg quietly lets zscale keep its input format and inserts libswscale afterwards to do the conversion, with all of libswscale’s problems. We hit this decoding Y’CbCr to rgb48be: zscale only changed the range, and libswscale did the Y’CbCr to RGB conversion.

Converting to the formats most encoders use (yuv422p10le, yuv444p10le, yuv420p10le, gbrp10le and so on) is fine. Hardware encoders that take semi-planar formats are not: prores_videotoolbox wants p210le, so add format=yuv422p10le after zscale and let FFmpeg repack it (see ProRes). For RGB output, put a planar RGB format straight after zscale (for example format=gbrp16le). To check, run with -v verbose and look for auto_scale lines: anything other than a plain repack (for example gbrp16le to rgb48be) means libswscale is converting.

Decoding untagged Y’CbCr

zscale on its own gives no output for Y’CbCr that carries no color tags (for example raw .yuv): it reports no path between colorspaces. Either give the input options, as in Sources without tags (zscale=matrixin=709:rangein=limited,format=gbrp16le converted raw 10-bit 4:2:2 on 6.1.6, 8.1 and 9.0.2), or tag the frames with setparams first and let zscale convert:

-vf "setparams=colorspace=bt709:range=tv:color_primaries=bt709:color_trc=bt709,zscale=range=full:chromalin=left,format=gbrp16le"

Geometry

zscale resizes (w, h, filter), but does not pad or fit to an aspect ratio, so command lines that combine scaling with pad, force_original_aspect_ratio or setsar need the color conversion separated from the geometry rather than swapping scale for zscale. Its default filter is bilinear; for resizing, as for chroma subsampling, use filter=lanczos. Resizing scene-linear EXR data must be done with zscale, not scale; see FFmpeg OCIO Filter.

Dithering

zscale does not dither by default (dither=none): each value is rounded to the nearest code, which is what our accuracy tests measure as exact. At 8-bit limited range that can band on smooth gradients, since 256 levels have to fit into 220 (Y′ 16 to 235). Dithering trades the bands for fine noise.

We measured each dither mode on H.264 4:2:0 at limited range, encoded as for web review (-crf 18), with CAMBI, Netflix’s banding index from libvmaf (libvmaf=feature=name=cambi): 0 is no banding, higher is more visible banding. The sources are a grey ramp and a radial gradient, each made at 16-bit (no banding of their own) and as an 8-bit image, and a photograph: a 640×360 part of the Kodak Digital LAD scan (Marcie), a genuine high bit depth image (about 30,000 distinct values in its 16-bit PNG). The results were the same on FFmpeg 6.1.6 and 9.0.2.

dither= Grey ramp, 16-bit Radial gradient, 16-bit Grey ramp, 8-bit Radial gradient, 8-bit Photograph, 16-bit
The source itself 0.0 5.5 10.9 15.2 0.0
8-bit none 13.5 22.4 13.0 22.0 0.0
8-bit ordered 3.3 7.8 5.3 8.2 0.0
8-bit random 6.6 5.6 5.3 3.4 0.0
8-bit error_diffusion 11.4 10.0 7.9 7.4 0.0
10-bit none 0.3 6.0 9.0 7.7 0.0
10-bit random 0.0 5.9 0.1 6.4 0.0

Dithering matters most for motion graphics, titles, CG and anything else with clean, smooth gradients. Live action already carries its own grain and noise, which does the job of a dither: the photograph shows no banding at 8-bit, with or without dither. A clean graphic has nothing to break up the steps, so without dither it bands badly (22.4 on the 16-bit radial gradient), even from a 16-bit render.

  • At 8-bit, ordered and random cut the banding by half or more on every gradient. random did best on the radial gradients, and looked best by eye: ordered leaves a regular pattern that can show close up, while random reads as grain. ordered did best on the plain ramps. error_diffusion helped least.
  • Dithering slightly raises the per-pixel error, since it adds fine noise. It survives -crf 18; a lower-quality encode may smooth it away.
  • From a 16-bit source, 10-bit needs no dither. From an 8-bit source, dithering to 10-bit also hides the source’s own 8-bit steps.

So for 8-bit deliveries of motion graphics, CG or anything with smooth gradients, add a dither to the conversion, for example zscale=matrix=709:range=limited:filter=lanczos:dither=random. For live action there is little to gain: on the photograph, dither changed neither the banding nor the error (mean 2.24 against 2.25 8-bit codes). The recommended commands leave it at none, which is what the accuracy tests check. To see the effect, flip between the two in the range and dither comparison.


Copyright © 2022 ORI Contributors. Distributed under a CC BY 4.0 license.

This site uses Just the Docs, a documentation theme for Jekyll.