8-bit vs 10-bit: Why Banding Gets Worse When You Upscale, and What Dither Can't Fix

The standard advice is that banding in a gradient is a bit-depth problem, so work in 16-bit and dither at the end. I tested that on a synthetic sky ramp and on a real photograph, and three of the eight measurements disagreed with the advice. The one that matters most: band width scales exactly with your upscale factor, which is why a gradient that looks clean at 1080p shows hard stripes at 4K from the same master.

Measured with ffmpeg 6.1, numpy 1.26.4 and Pillow 12.2.0 · a 1024×256 synthetic sky ramp plus one 1080×1620 photograph · plateau widths counted along raw rows, not from a screenshot · every number reproducible from the snippet at the end

The claim, stated precisely enough to test

A smooth gradient in an 8-bit file cannot be smooth. There are only 256 code values, so a ramp that crosses the full tonal range over 1024 pixels must spend an average of four pixels on each value. Those four pixels are not a gradient — they are a flat plateau with a hard step at each end. Your visual system is very good at finding those steps, which is why banding shows up in skies, studio backdrops and skin falloff long before it shows up anywhere else.

The usual remedies, in the order people recommend them:

1. Work in 16-bit so the intermediate values exist.
2. Dither before quantising to 8-bit, so the steps become noise instead of edges.
3. Encode video in 10-bit, which is said to cost 20–50% more bitrate but preserves the gradient.

Points 1 and 2 are testable by counting. Point 3 turned out to be backwards in my test, and I will show the measurement and the control that says why.

Measurement 1 — plateau width, and how it scales

A 1024×256 sky ramp, generated analytically at 16-bit precision so the master has no banding of its own. The same ramp is written out at 8-bit and at 16-bit, then both are upscaled 4× with Lanczos to 4096×1024. I decode the result to raw grey and count, along each row, the average length of a run of identical values.

Sixteen pixels of identical grey, then a step. On a 4096-wide image that is a stripe you can see across a room. The 16-bit version never repeats a value twice in a row — plateau width 1.00 pixel is the arithmetic floor, meaning every pixel along the row is a different code.

Now the part I had not expected to be so tidy. Upscale the same 8-bit ramp by 1×, 2×, 4× and 8× and measure again:

Plateau width is exactly output_width / 255. Not approximately — four measurements land within half a percent of that formula. Interpolation cannot subdivide a plateau, because the plateau contains no information about how to subdivide it; every pixel in the run has the identical input value, so every kernel produces the identical output value.

Measurement 2 — the negative result: a 16-bit master does not survive an 8-bit delivery

Here is the trap. If 16-bit is clean and 8-bit bands, the obvious move is to keep the master at 16-bit through the whole pipeline. So I ran the 16-bit ramp through the same Lanczos upscale and then formatted the output as 8-bit in the same filter chain — the correct order, high precision through the resample, quantise once at the end.

Identical. To the pixel. A 4096-wide output has only 256 places to put a code if the file is 8-bit, and a monotonic ramp uses all of them exactly once — so 16 pixels per code is forced by arithmetic, not by the pipeline. Keeping 16-bit precision through the resample is still correct (the intermediate values matter for sharpening and compositing), but it cannot change the plateau width of the delivered file. Nothing can, at a fixed output bit depth and a fixed output width, except dithering.

Measurement 3 — dither works, and it loses on the metric

Dithering is the one technique that genuinely changes plateau width, because it deliberately breaks the monotonic ramp: instead of rounding every pixel in a plateau to the same code, it decides per pixel whether to round up or down, in a pattern fine enough that the eye averages it back into a smooth gradient.

I applied an 8×8 ordered Bayer matrix at the moment of quantising the 16-bit upscale to 8-bit, then measured both the plateau width and the RMSE against the exact 16-bit truth.

Two things to read here, and the second one is the negative result. First, dither does what it claims: 16.00 px of plateau becomes 2.00 px, an eightfold reduction. Second, it is 37% worse by RMSE (0.408 against 0.297). That is not a bug in my implementation — it is inherent. Rounding to nearest is the RMSE-optimal quantiser by definition. Any dither pattern trades per-pixel accuracy for local average accuracy, so it must lose on a per-pixel metric while winning on perception. If you pick a quantiser by RMSE you will always pick the one that bands.

Floyd–Steinberg lands in the same place as Bayer on this content (2.23 px, RMSE 0.393) which surprised me; error diffusion usually beats ordered dithering on photographs, and I suspect a smooth ramp is simply too easy a case to show the difference.

Measurement 4 — dither after the 8-bit step is not a fix, it is decoration

The advice people actually follow is "dither at the end", and sometimes they mean the very end — after the file has already been through an 8-bit pipeline. So I took the already-banded 8-bit upscale and applied the same Bayer pattern to it.

The stripes disappear — plateau 1.24 px, the smallest of the three, and the step count more than doubles. And the error is 2.6× the control. That is exactly what you would predict once you accept that the information is gone: the late dither has nothing to round toward, so it adds noise on top of the band instead of converting precision into pattern. It hides the artefact and makes every pixel further from the truth. Visually that is often a good trade. Measurably it is not a recovery, and it is worth being honest that the two statements are different.

Measurement 5 — grain is dither, and it is expensive

Which raises the practical question: if adding noise hides banding, how much noise, and what does it cost? I added gaussian grain at 0.25%, 0.5% and 1.0% of full scale to the 16-bit truth before quantising to 8-bit.

Grain buys the same plateau collapse for 1.7× to 6.3× the error of structured dither, at every strength. 0.25% grain already reaches plateau 1.67 px — better than Bayer on plateau width — while costing 1.7× the RMSE. If you are adding grain to hide banding you are using a 6× more expensive noise source than an 8×8 matrix that costs four lines of code.

Measurement 6 — the bit-depth ladder, and where the knee is

So how many bits is enough? Quantise the same continuous ramp to 8, 9, 10, 12 and 16 bits and count distinct levels and plateau width at 1280 px wide.

The 12-bit and 16-bit rows are identical, and that is a measurement floor rather than a result: the ramp is only 1280 pixels wide, so one distinct level per pixel is the most the test can resolve. Above 12 bits on a 1280-wide ramp there is nothing left to measure. What the table does establish is that 10-bit captures 76% of the total available improvement (969 of 1280 levels) and already sits at plateau 1.32 px, comfortably below the ~2 px where a band stops reading as a band at normal viewing distance. 8→10 bits is the step that matters; 10→12 is incremental; 12→16 buys nothing you can see on consumer content.

Measurement 7 — on a real photograph, banding lives in the smooth parts only

Everything above is a synthetic ramp, which is the best case for banding. On a real 1080×1620 photograph the mean plateau is 1.9 px, not the 4.2 px that 1080/255 predicts — texture breaks the runs up. 40.9% of pixels sit in runs longer than 4 pixels, which are the regions that will band: sky, backdrop, out-of-focus background, skin.

And this is where the third negative result lives. Take that 8-bit JPEG and upscale it 4× twice: once normally, and once by promoting to 16-bit internally first.

0.26 px of plateau for 54% more bytes on disk. The 16-bit path is doing real work — the resample genuinely runs at higher precision — but there is no extra information in an 8-bit JPEG to preserve, so the output collapses back to 8-bit onto very nearly the same rungs. Bit depth has to exist at capture or master time. Promoting an 8-bit source to 16-bit mid-pipeline buys you arithmetic precision, not detail, and the banding is unchanged.

Measurement 8 — 10-bit H.264 is smaller here, not bigger

The claim I most expected to confirm was the bitrate cost of 10-bit video: the usual figure is 20–50% more for the same quality. I encoded the identical 16-bit sky ramp to H.264 at 8-bit (yuv420p) and 10-bit (yuv420p10le), same preset, same CRF, 1 second, 1280×720, and measured the decoded frame.

10-bit is smaller at every CRF, by 14.6× at CRF 12. The mechanism is not mysterious: at 8-bit, the quantisation creates 255 hard edges per row, and a DCT has to spend coefficients representing every one of those steps. At 10-bit the ramp is smooth, so the transform has almost nothing to code. You are not paying for precision — on a gradient, precision is cheaper than the banding it removes.

What I would actually do

1. Decide bit depth at capture, not at export. Measurement 7 is the expensive lesson: promoting an 8-bit source later costs 54% more bytes and recovers 0.26 px of plateau. If the source is 8-bit, dither and move on; the information is not there.
2. Expect bands to grow, not multiply. Plateau width is width/255 for 8-bit. A gradient you shipped at 1080p will show bands 2 px wide at 4K and 4 px wide at 8K from the same master — same number of bands, each wider. Fix the master, do not re-tune the upscaler.
3. Dither with a matrix, at the last quantisation. 0.25% grain reaches plateau 1.67 px for 1.7× the RMSE of rounding; Bayer reaches 2.00 px for 1.37×. Grain at 1% reaches 1.13 px for 8.6×. The matrix is the better trade at every strength I tested.
4. Do not judge a dither by RMSE. Rounding wins RMSE by construction. If your acceptance test is a per-pixel metric you will always ship the banded version.
5. 10-bit is the knee. 969 of 1280 levels, plateau 1.32 px, ~2% smaller files on real footage. 12-bit and above bought nothing measurable on a 1280-wide ramp.

Reproducing all eight measurements

Needs ffmpeg, numpy. Every figure above comes from counting runs of identical values in raw decoded rows — no screenshots, no eyeballing.

import numpy as np, subprocess
W,H = 1024,256

def raw(p, pf, w, h):
    b = subprocess.run(['ffmpeg','-v','error','-i',p,'-frames:v','1',
                        '-f','rawvideo','-pix_fmt',pf,'-'],capture_output=True).stdout
    dt = np.uint8 if pf=='gray' else np.uint16
    return np.frombuffer(b,dtype=dt).reshape(h,w).astype(np.int64)

def plateau(img):                      # mean run of identical values per row
    return float(np.mean([img.shape[1]/(np.count_nonzero(np.diff(img[r]))+1)
                          for r in range(0,img.shape[0],8)]))

# continuous sky ramp, then the two bit depths
x = np.linspace(0,1,W)**0.75
for bits,pf,dst in ((8,'gray','s8.png'),(16,'gray16le','s16.png')):
    a = np.round(x*(2**bits-1)).astype(np.uint16 if bits>8 else np.uint8)
    a.astype('8 else np.uint8).tofile('/tmp/t.raw')
    subprocess.run(['ffmpeg','-v','error','-y','-f','rawvideo','-pix_fmt',pf,
                    '-s',f'{W}x{H}','-i','/tmp/t.raw',
                    '-pix_fmt','rgb48be' if bits>8 else 'rgb24',dst],check=True)

# M1: 4x upscale, count plateaus        M7: same at x1/x2/x4/x8
for f in (1,2,4,8):
    subprocess.run(['ffmpeg','-v','error','-y','-i','s8.png','-vf',
                    f'scale={W*f}:{H*f}:flags=lanczos','-pix_fmt','rgb24',
                    f'u{f}.png'],check=True)
    print(f, round(plateau(raw(f'u{f}.png','gray',W*f,H*f)),2),
             'predicted', round(W*f/255,2))

# M3: Bayer 8x8 at the 16->8 conversion
B = np.array([[0,32,8,40,2,34,10,42],[48,16,56,24,50,18,58,26],
              [12,44,4,36,14,46,6,38],[60,28,52,20,62,30,54,22],
              [3,35,11,43,1,33,9,41],[51,19,59,27,49,17,57,25],
              [15,47,7,39,13,45,5,37],[63,31,55,23,61,29,53,21]])/64.0 - 0.5
truth = raw('u4.png','gray16le',W*4,H*4)/257.0
tile  = np.tile(B,(H*4//8,W*4//8))
s8    = raw('u4.png','gray',W*4,H*4).astype(float)
for name,q in (('round', np.round(s8)),
               ('bayer', np.clip(np.round(s8+tile+0.5),0,255))):
    print(name, round(float(np.sqrt(np.mean((q-truth)**2))),3))

Output on this machine: 1 4.0 4.02 / 2 8.0 8.03 / 4 16.0 16.06 / 8 32.0 32.13 and round 0.297 / bayer 0.408. Measurement 8 needs only -pix_fmt yuv420p versus yuv420p10le swapped on the same libx264 command line, then ffprobe -show_entries stream=pix_fmt,width,height to confirm which one you actually got — a 10-bit encode silently falls back to 8-bit if the build lacks it, and every number in that table would become fiction.

If you are upscaling a gradient-heavy image or clip and the bands widened rather than disappeared, the bit depth was decided before the upscaler ever saw the file. The pipeline that produces the 4K output — and the resampling order inside it — is what the browser upscaler at upgolab.com handles, but it is working from the master you give it, and no amount of later precision puts rungs back into a ladder that was built with 256 of them.