Chroma Subsampling Before You Upscale: What 4:2:0 Actually Costs

Almost every video you upscale arrives as 4:2:0 — chroma stored at half resolution in both directions. Most upscaling advice treats that as a footnote. It is not a footnote: it is a pre-destroyed channel that no upscale can rebuild, and its damage is concentrated in exactly the places people notice — thin coloured lines, text on coloured backgrounds, saturated edges. Here are seven measurements on what it actually costs, including one result that contradicts the usual "4:2:0 saves bytes" claim and a half-pixel defect that every whole-frame metric passes.

Measured with numpy and ffmpeg 6.1.1 · source: one 1080×1620 photograph plus synthetic tests · every number below is reproducible from the snippet at the end

Why this is worth measuring at all

4:2:0 stores full-resolution luma (Y) and quarter-resolution Cb/Cr. Human vision is less sensitive to chroma resolution, which is why the format survives in delivery. The trap is conceptual: an upscaler invents high-frequency detail, so if chroma was band-limited to half resolution before it reached you, the upscaler is inventing detail in a channel that has none — and may be inventing detail that costs you file size.

Every measurement below stays in the YUV domain. An earlier version converted to RGB between steps and the results were wrong in an instructive way: luma PSNR appeared to drop 5.5 dB after a chroma-only operation — that was clipping in the RGB round trip, not chroma damage. Luma is never subsampled in 4:2:0, so measure it in YUV or you will attribute chroma error to luma.

Experiment 1 — luma is untouched, chroma loses 9.7 dB

Protocol: take a real 1080×1620 photograph as the reference. Downsample it by a factor k with lanczos to get the "source you actually have". Then reconstruct to full resolution two ways: (a) direct bicubic upscale of all three planes, and (b) a genuine 4:2:0 round trip on the source — 2×2 box filter on chroma, replication back — followed by the same upscale. Compare both against the true full-resolution image, luma and chroma separately.

Read the two Y columns: identical to three decimal places at every scale, because the luma plane never enters the 4:2:0 path. The chroma columns are where the cost lands — 9.7 dB is roughly the gap between a good encode and a visibly degraded one. Note the trend: the penalty shrinks from −9.7 dB at k=2 to −5.7 dB at k=4 while relative RMS error falls 3.30× → 1.88×, because at heavier downscaling the upscaler's own error swamps the chroma error. That is the opposite of the "smaller source is more fragile" intuition.

Experiment 2 — where in the pipeline the 4:2:0 step sits

Two places to put a 4:2:0 stage, and they are not equivalent. Same source, same final resolution (k=2), final RGB against the original:

Subsampling after the upscale costs 0.10 dB. Subsampling before costs 0.41 dB — four times as much. That is the whole argument for keeping intermediate results at 4:4:4 and only converting at final delivery. If you are handed a 4:2:0 source you have already paid the larger cost and cannot get it back; if you are producing one, convert last.

Experiment 3 — the best way to rebuild chroma is not lanczos

Given a 4:2:0 source, chroma has to be interpolated back to full resolution before the upscale. Conventional wisdom says lanczos, because it preserves the most detail. Measured on this photograph at k=2, with luma held at bicubic so only the chroma kernel varies:

Bilinear wins, monotonic from nearest upward then back down. The chroma being reconstructed contains no real high-frequency information — the encoder's box filter removed it — so a sharpening kernel applied to a band-limited signal recovers no detail; it amplifies the decimation's ringing. Lanczos is 0.447 dB worse than bilinear here for exactly the reason it is usually better on luma.

This is a 0.45 dB effect, not a 5 dB effect. It is worth knowing because it is free — you are choosing a kernel either way — but it is not a reason to restructure a pipeline.

Experiment 4 — a one-pixel colour line loses exactly half its contrast

Synthetic test: vertical coloured lines of increasing width on a white background, chroma amplitude measured as distance from neutral (128), at half-maximum width.

A clean step function, not a gradual falloff: 50.0% at one pixel, 100.0% at two. The 1px line and its neighbour share a single chroma sample in each 2×2 block, so the box filter averages the line against white and the value lands exactly halfway. Everything two pixels and wider has its own sample and survives untouched. Hence 4:2:0 is invisible on photographic content and obvious on UI screenshots, slide decks and line art: those are built from one-pixel features.

Experiment 5 — the negative result: 4:4:4 encoded smaller than 4:2:0

The standard justification for 4:2:0 is bandwidth. So encode the same source at both pixel formats with libx264, preset slow, identical CRF, single-threaded, and compare file sizes. The expectation is that 4:2:0 is smaller.

On photographic content, 4:4:4 is smaller — by 2.3% on the real photo and by 6.7% on the desaturated variant. It only becomes larger where chroma carries real structure: 10.7% larger on saturated colour bars, and roughly break-even on pure luma detail. The result reproduces across three CRF values and two resolutions:

The mechanism, measured

A plausible story would be that 4:4:4 just codes worse and the size win comes with a quality loss. It does not: at CRF 20 the 4:4:4 encode is both smaller (0.9771×) and better on chroma PSNR (43.98 dB vs 43.32 dB). So the size difference has to come from the signal itself.

Measure the high-frequency energy of the chroma planes, before and after a 4:2:0 round trip, using mean absolute Laplacian:

A 4:2:0 round trip increases chroma high-frequency energy by 35% on photographic content. Decimating and then replicating chroma turns smooth gradients into blocky 2×2 stair-steps, and those stair-steps are new high-frequency content that the encoder then has to spend bits describing. On content where chroma was already sharp, the round trip is a no-op and the energy ratio is exactly 1.000.

Limits, stated plainly. x264 at preset slow, one photograph and three synthetic controls — not a survey of encoders. The effect is a few percent, inside the range where encoder heuristics dominate. It is not "always use 4:4:4": most delivery targets reject 4:4:4 and hardware decoders often lack it. It is that the bandwidth argument for 4:2:0 is weaker than stated, and a high-quality intermediate kept at 4:4:4 costs nothing in bytes.

Experiment 6 — how much of a real photo is chroma detail?

Gradient magnitude percentiles on the same photograph, luma versus chroma:

Chroma's steepest gradients are 4.2× shallower and only 31% as many pixels are active — the quantitative version of "the eye is less sensitive to chroma". Roughly two-thirds of a photograph has chroma so flat it is effectively constant: the content 4:2:0 was designed for, where the format is near-free. The other third — saturated edges, coloured text, graphics over photos — pays the 9.7 dB.

Experiment 7 — the half-pixel that is metadata, not data

Everything above treats 4:2:0 as a resolution loss. There is a second cost rarely mentioned: where the surviving chroma samples sit relative to luma, which the format does not fix. Two conventions exist — centred on the 2×2 block (JPEG, x264, ffmpeg's default) and co-sited with the top-left luma pixel ("left", from MPEG‑2). The gap is exactly 0.5 luma pixels and lives entirely in the interpretation; the sample values are identical.

Rows one and three contradict each other on purpose, and that is the finding. On the photograph the wrong decode costs 0.017 dB — five hundred times less than Experiment 1's 9.7 dB, far below any automated gate. On a controlled edge of known position (a 1px ramp at x = 128.0, subsampled with the 2×2 box, which is centred siting by construction) the same error costs 3.0 dB and lands half a pixel left. Whole-frame PSNR averages the defect over the 53% of the frame that is chroma-flat and dilutes it into nothing.

Bucketing the difference by local chroma gradient shows where it lands: the steepest 0.83% of pixels carry mean |ΔCb| 5.97, the 1.86% band above gradient 1 carries 1.13, and the 44.15% just above zero carries 0.21. One result defied my prediction — I expected the exactly-flat 53.14% to be untouched, and it is not: mean 0.0992, p99 0.877. The wrong decode does not fail to shift a flat pixel, it samples a different neighbour, so error bleeds one pixel onto the flat side of every boundary. Frame-wide the error is 2.63% of chroma RMS; 1.29% of pixels exceed |ΔCb| = 2.

Regressing the difference between the two reconstructions against the horizontal gradient of the correct one gives r = 0.776: to first order the error is a pure 0.25-chroma-pixel shift. So it behaves geometrically, not photometrically — doubling with every doubling of output size, from half a pixel natively to a two-pixel fringe at 4×, where it looks like bad chroma upsampling and gets "fixed" with sharpening that cannot work, because the edge is displaced, not soft.

Two things I expected to work, and measured that they do not. Channel-difference metering is blind to it: phase-correlating R against B on a 2× upscale gives dy=0, dx=0 under both hypotheses, because a siting error moves Cb and Cr together where aberration moves them apart. The metadata flag does not protect you: -chroma_sample_location left|center|topleft gives three VUI values and one identical md5 after decoding to raw YUV, and again as rgb24. The flag propagates through stream copy, mkv remux and a rescale re-encode; no decoder tested applies it — the 1-byte size difference is the VUI bit pattern, not picture data. Scope: libx264, one frame, ffmpeg 6.1.1's swscale; a hardware decoder that honours the VUI could differ.

# luma pixel i spans [i, i+1); chroma sample j sits at luma coord 2j + c
#   c = 1.0 -> centred (x264/JPEG)      c = 0.5 -> left (MPEG-2)
SITING = {"centre": 1.0, "left": 0.5}

def up_chroma(d, H, W, siting):
    h, w = d.shape; c = SITING[siting]
    vy = np.clip((np.arange(H)+0.5-c)/2.0, 0, h-1)
    vx = np.clip((np.arange(W)+0.5-c)/2.0, 0, w-1)
    y0, x0 = np.floor(vy).astype(int), np.floor(vx).astype(int)
    y1, x1 = np.minimum(y0+1,h-1), np.minimum(x0+1,w-1)
    wy, wx = (vy-y0)[:,None], (vx-x0)[None,:]
    return (d[np.ix_(y0,x0)]*(1-wx)+d[np.ix_(y0,x1)]*wx)*(1-wy) \
         + (d[np.ix_(y1,x0)]*(1-wx)+d[np.ix_(y1,x1)]*wx)*wy

# the 2x2 box in sub420() IS centred siting, so only "centre" can win here
t   = np.clip(np.arange(256)+0.5-128.0+0.5, 0, 1)   # edge at exactly x=128.0
img = np.zeros((64,256,3))
for ch,(lo,hi) in enumerate([(40,220),(60,30),(200,40)]):
    img[...,ch] = (lo*(1-t)+hi*t)[None,:]
GY,GU,GV = ycbcr(img); du,dv = sub420(GU),sub420(GV)
for name in ("centre","left"):
    pc = (psnr(GU, up_chroma(du,64,256,name))
        + psnr(GV, up_chroma(dv,64,256,name))) / 2
    print(name, round(pc, 3))
# centre 40.597    left 37.586

What to actually do

1. Convert to 4:2:0 last, not first. The measured cost is 0.10 dB after the upscale versus 0.41 dB before it. Keep intermediates at 4:4:4.
2. Reconstruct chroma with bilinear, not lanczos. 0.45 dB better here, and free. Sharpening a band-limited channel amplifies decimation ringing.
3. Audit one-pixel features before blaming the upscaler. A 1px coloured stroke loses exactly 50% of its contrast to subsampling alone, so washed-out thin strokes predate the upscale and no sharpening restores them — the mechanism behind the ink-loss measurements on line art.
4. Do not scale a 4:2:0 source aggressively. The penalty is largest at small factors (−9.7 dB at k=2) and shrinks at k=4 (−5.7 dB) only because the upscaler's own error hides it. Absolute chroma error still climbs, 0.51 → 1.49 RMS.
5. Use centred chroma siting when you resample yourself. It is what the encoder assumed when it applied the 2×2 box. A co-sited assumption is 0.5 luma px off, costs 3.0 dB on an edge, and becomes a colour fringe at 2× and 4×. Do not expect chroma_location to save you — three values produced one identical decode.
6. Test your own content. All of the above is a two-minute script; the numbers that matter are the ones on your footage, not the ones on this page.

If you want to see what a whole frame looks like after these steps rather than a PSNR number, the browser upscaler at upgolab.com runs the full reconstruct-then-scale path on a single image, so you can compare a 4:2:0-sourced frame against a 4:4:4 one side by side instead of trusting a table.

Reproduce it

import numpy as np, math
from PIL import Image

def ycbcr(rgb):
    r,g,b = rgb[...,0],rgb[...,1],rgb[...,2]
    return (0.299*r+0.587*g+0.114*b,
            -0.168736*r-0.331264*g+0.5*b+128,
            0.5*r-0.418688*g-0.081312*b+128)

def sub420(c):                      # genuine 4:2:0 chroma step
    h,w = c.shape; h2,w2 = h-h%2, w-w%2
    d = c[:h2,:w2].reshape(h2//2,2,w2//2,2).mean(axis=(1,3))
    return np.repeat(np.repeat(d,2,0),2,1)      # replicate back

def psnr(a,b):
    m = np.mean((a-b)**2)
    return 99.0 if m <= 1e-12 else 10*math.log10(255**2/m)

rgb = np.asarray(Image.open("photo.jpg").convert("RGB")).astype(float)
H,W = rgb.shape[:2]
RY,RU,RV = ycbcr(rgb)               # full-res reference

k = 2
base = np.asarray(Image.fromarray(rgb.astype(np.uint8))
                  .resize((W//k,H//k), Image.LANCZOS)).astype(float)
by,bu,bv = ycbcr(base)

def rs(p,w,h,f=Image.BICUBIC):
    return np.asarray(Image.fromarray(np.clip(p,0,255).astype(np.uint8))
                      .resize((w,h),f)).astype(float)

# Y passes through untouched; only chroma takes the 2x2 box + replicate
print("PSNR-Y direct", psnr(RY, rs(by,W,H)))
print("PSNR-Y  4:2:0", psnr(RY, rs(by,W,H)))       # identical by construction
print("PSNR-C direct", psnr(RU, rs(bu,W,H)))
print("PSNR-C  4:2:0", psnr(RU, rs(sub420(bu),W,H)))

Both encodes used for the byte comparison are bit-identical across repeated runs — same command, same CRF, -threads 1, verified byte-for-byte — so the sizes in Experiment 5 are reproducible rather than one-off samples.