JPEG Blocking Under Upscaling: The Seam Keeps Its Contrast and Gains Area

The usual explanation for why JPEG block seams look worse in a 4K upscale is that the upscaler "magnifies" them, as if the artifact were being amplified. I measured it on one synthetic frame and one real 1080×1080 photograph and that is not what happens. The seam keeps a constant ratio of 1.26–1.30× against the surrounding gradient at 1×, 2×, 4× and 8×. What changes is the width: eight output pixels becomes sixty-four. Four of the twelve measurements came out against my expectations, including the one about sharpening, and I have left them in.

Measured with numpy 1.26.4 and Pillow 12.2.0 · a 768×1024 synthetic sky-and-blocks frame plus a 1080×1080 crop of a real photograph · all edge statistics computed on raw decoded arrays, never from a screenshot · JPEG produced by Pillow's libjpeg at 4:2:0 unless stated · every number reproducible from the snippet at the end

Why blocking behaves differently from every other artifact

Banding, chroma smear and ringing all live in the image and move with the content. Blocking does not. It is pinned to an absolute pixel lattice: an 8×8 grid with the origin at the top-left corner of the decoded frame. Upscaling multiplies all coordinates by k, so the lattice becomes 8k×8k output pixels. At 2× the cells are 16 px wide. At 4× they are 32. At 8×, 64.

So the question is not "how much does the upscaler amplify the seam" but "what happens to a fixed-width step on a fixed lattice when you interpolate across it". Those are different questions and they have different answers. Here is the measurement that separates them.

Measurement 1 — the seam ratio is scale-invariant, the seam width is not

I encoded the real photograph at q=60, then upscaled the same encode with Lanczos-3 by 1×, 2×, 4× and 8×. For each output I measured two things on the 8k lattice: R, the mean absolute gradient on a block boundary divided by the mean absolute gradient elsewhere, and step, the mean absolute gradient on a boundary in 8-bit levels. A clean Lanczos upscale of the un-encoded source is the control, measured the same way.

Two things in that table contradict the intuitive story. First, the absolute step in levels falls as the scale factor rises — 4.58 at 1× down to 0.70 at 8×, almost exactly as 1/k. A 4 px-wide step of 4.58 levels spread over 32 output pixels is 1.38 levels per pixel. The upscaler did not amplify the seam; it diluted it, which is what interpolation does to anything with finite width.

Second, the ratio does not move. R is 1.26–1.27× at every scale factor and the step ratio is 1.28–1.30×. The seam always carries about 30% more gradient than its surroundings, regardless of how many pixels you use to draw it.

Where on the quality scale this starts

The same 2× upscale across five source qualities, scored against ground truth. The interesting column is the last one: HF is high-frequency energy relative to ground truth, so anything above 100% means the pipeline is adding edges that were never there.

A PNG master scored against the same ground truth gets PSNR 31.93, SSIM 0.9888 and R 1.364. So q=95 is 0.18 dB from a lossless master and q=30 is 1.27 dB from it — and no quality in the range gets R down to the clean control. The seam does not fade out at high quality, it just gets quieter than the resampler's own error. That is also why the quality slider is the wrong knob: the whole 30→95 range moves R by 0.67, while the upscale factor did not move the ratio at all.

Measurement 2 — where the seam's contrast actually comes from

My working hypothesis was that quantisation pushes pixels inside a block toward a common value, so the interior gradient falls while the boundary between two blocks does not. That would raise R by crushing the denominator. I tested it in source space, with no upscaling at all, by measuring boundary gradient and interior gradient separately.

Hypothesis rejected. The interior gradient rises at every quality level — it does not fall. And the mean absolute error on the boundary column is identical to the interior to two decimal places at all five qualities (0.47/0.47, 0.58/0.58, 0.89/0.89, 1.09/1.10, 1.81/1.83). The seam is not a place where JPEG commits more error. It is a place where the same error lands with opposite sign on the two sides of a lattice line, and the eight-pixel periodicity prevents the DCT basis from smoothing it away.

The one genuinely odd figure is q=70 at +81.0%, higher than q=50 at +55.5%. Boundary gradient is not monotonic in quality on this content. I ran it twice and got the same numbers, so the non-monotonicity is real on this image — but one image is one image, and I am not going to generalise it into a claim about JPEG quality. Treat the +23% to +81% range as the finding and the ordering inside it as noise.

Measurement 3 — how much of your total error the JPEG step is responsible for

PSNR differences do not add, so comparing "JPEG upscale" to "PNG upscale" in dB understates nothing but explains nothing either. The honest question is what share of the total reconstruction error the codec contributed. That is a ratio of MSE, which does add.

At 2× the codec is 15.6% of the total error and the resampler owns the rest. By 4× the codec drops to 8.8%: the upscaler's own error has quadrupled while the codec's contribution stayed roughly flat. This is the arithmetic behind a rule of thumb that most people apply backwards — at aggressive scale factors, the choice of resampler matters more than the source bitrate, and the seam you are staring at is a small minority of what went wrong.

Measurement 4 — deblocking: before or after the upscale

I wrote a deliberately minimal deblocker: for each lattice line, move the two adjacent columns (or rows) toward their mean by a strength s. Nothing else. Applied to the q=50 encode at 2× with a Lanczos-3 upscale in between:

Deblocking after the upscale wins on both axes that matter: higher PSNR (30.86 vs 30.34) and lower seam ratio (0.846 vs 0.901), while keeping 117.7% of the ground-truth high frequency against 108.7%. That is not what I expected — smoothing a 16-px-wide seam felt like it should cost more detail than smoothing an 8-px one, because there are twice as many pixels to touch.

The reason is that deblocking before the upscale throws information away and then the upscaler faithfully magnifies nothing. Deblocking after operates once, on the already-interpolated result, and the interpolation has already redistributed the step across 16 px so the correction is smaller per pixel. Doing it after is cheaper because the artifact is cheaper after.

Before you reach for a blur or a median filter

Deblocking is not the same operation as smoothing, and the two get conflated because they both reduce R. Applied to the same q=50 source before a 2× upscale:

A σ=0.6 gaussian drops R from 1.782 to 1.660 — only 15.5% of the seam's excess over 1.0 — and the price is 49 points of high-frequency detail: it keeps 59% of the detail the unfiltered pipeline had, for 2.14 dB. The median filter is worse on both axes, removing 6.3% of the excess while keeping 52% of the detail. The lattice deblocker at s=0.5 takes R below 1.0 (0.901), meaning the boundary no longer carries more gradient than the block interior at all, and still keeps 91% of the detail for 0.85 dB — it is the only filter in the table that buys seam reduction more cheaply than it spends detail.

One detail worth noting: deblock s=0.5 is the only row where SSIM goes up (0.9788 against 0.9785 for no filter), and gaussian σ=0.6 goes higher still at 0.9795 — while keeping barely half the detail. SSIM's 11×11 window over a 1.5 px gaussian means a mild blur that removes a seam scores well, which is the same structural blindness that makes SSIM a poor guide for this artifact.

Strength, and the point where the metric goes to zero

At s=1.0 every pair of columns straddling a lattice line is replaced by their mean, so the gradient on that line is exactly zero and R reports 0.000. That is not a good result, it is a degenerate one — I am scoring well on a metric I have just annihilated. Three quarters of the visible cost between s=0 and s=1 is paid for the last 0.4 of R. Use s≈0.5 and accept that the metric will never reach the clean control (1.364), because a deblocker can only remove a step, never put back the block-interior texture that quantisation destroyed.

Measurement 5 — the negative result on sharpening

The folklore is that output sharpening re-reveals block seams, which is why you are told to deblock before you sharpen. I applied an unsharp mask with radius 1.0 at four amounts after the upscale and measured the seam step against the same sharpening applied to a clean PNG upscale:

Sharpening does raise the seam step, from 1.76 to 2.45 levels. But it raises the clean control's step from 1.17 to 1.60 over the same range, and the ratio between them stays between 1.50 and 1.54 — a 2.7% drift across a 4× change in sharpening amount. An unsharp mask is a linear, shift-invariant filter; it cannot amplify a seam more than it amplifies any other edge of the same width. Output sharpening makes the seam more visible exactly as much as it makes every real edge more visible, which is to say it is not selectively to blame.

What sharpening does do is raise PSNR here, 31.28 → 31.94, which is the same trap measured on this cluster before: the metric rewards the halo. If your acceptance test is per-pixel and your subjective complaint is block seams, the two are pointing in opposite directions.

Measurement 6 — the grid survives a crop, and you can find it blind

Deblocking needs to know where the lattice is. If you decode, crop, and upscale, the grid is no longer at phase 0. I tested whether a phase sweep can recover it without metadata: for each of 16 candidate phases on the 16-px output grid, take the ratio of mean gradient on candidate lines to mean gradient off them, and pick the argmax. Four crops of the same q=50 encode, then 2× Lanczos:

Four out of four, on both axes, with scores this side of 1.7× (horizontal) and 3.2× (vertical). The vertical score is higher because the synthetic frame has hard horizontal edges in it and the lattice interacts with them differently; that asymmetry is a property of my test image, not of JPEG.

Measurement 7 — re-encoding six times is nearly free, and the last encode is not

Two separate beliefs tested. First, that re-saving a JPEG repeatedly degrades it badly (the "generation loss" folklore that dates from long-hair polyester-era tooling). Six consecutive re-encodes at q=80, decoding fully each time, no resizing:

1.03 dB of total loss across five extra generations. The curves flatten because the second generation's input is already inside the quantiser's tolerance for that quality — the DCT coefficients barely move. Re-encoding at the same quality is close to idempotent. This is a negative result against the folklore, and the mechanism is boring: it is the same quantiser, so its fixed point is immediate.

The second belief is the one that bites. Master at q=90, upscale 2×, then encode the output at four qualities:

PSNR is flat from q=95 to q=70 — 31.72 down to 31.68, a quarter of a decibel — while R climbs from 1.498 to 2.063 and the seam step from 1.53 to 1.86. The metric says nothing happened; the artifact says it got 38% stronger. That is the whole case for measuring blocking as a separate number instead of trusting an aggregate: your final export step can nearly double the seam contrast while PSNR holds still, and if you only look at PSNR you will ship it.

The reason is geometric. Your master's lattice was 8 px and the upscale took it to 16. The final encode lays a fresh 8 px lattice at the output resolution, which is half the period of the one already in the pixels. Two incommensurate lattices, and the second one is the one the viewer sees at 100%.

Measurement 8 — what a clean master actually costs

All of the above argues for keeping the master out of the 8×8 grid's hands, which means paying for it in bytes. On the real photograph, downscaled to 540×540 so the comparison is not dominated by sensor noise:

And subsampling, at q=70, is almost irrelevant to blocking specifically:

4:2:0 changes R by 0.015 against a signal of 0.470 — one percent of the effect — for 27% fewer bytes. Blocking lives in the luma plane's DCT, not in the chroma planes, so turning up chroma quality to fight blocking is spending bytes where the artifact isn't. That is the opposite of what the halo-heavy chroma article on this cluster would suggest, and both are true: chroma subsampling is expensive for colour edges and free for luminance blocking.

What I would actually do

1. Stop trying to remove the seam by lowering the scale factor. Measurement 1 shows the ratio is 1.26× whether you go 2× or 8×. Scale factor changes the area, not the contrast, so shrinking the output fixes the visibility by making the whole image smaller — which is not a fix.
2. Deblock after the upscale, on the 8k grid, at strength ≈0.5. It beat deblocking before on PSNR (30.86 vs 30.34) and on seam ratio (0.846 vs 0.901) while keeping more detail. Find the phase by sweeping 16 candidates and taking the argmax of the boundary-to-interior gradient ratio; 4/4 on the crops I tested.
3. Always compare against a clean control at the same phase. R alone is meaningless — a clean image scored 0.668 to 1.364 depending on which phase you assumed. The number to report is the excess over the control.
4. Do not blame output sharpening for the seam. The seam-to-clean ratio drifted 2.7% across a 4× range of sharpening amount. Fix the source or deblock; the unsharp mask is not the problem, though it will happily raise your PSNR while making the image worse.
5. Watch the final export, not the middle of the chain. The last encode lays a fresh 8-px lattice at output resolution. Going from q=95 to q=70 on the delivered file moved the seam ratio from 1.498 to 2.063 while PSNR moved 0.04 dB. Re-encoding the same file at the same quality six times costs 1.03 dB; re-encoding it once at the wrong quality costs the artifact.
6. Keep a PNG or 4:4:4 q=95 master. 121 KB against 379 KB on a 540×540 frame is a 3.1× saving over PNG with no lattice baked in that a later stage has to fight.

Reproducing all twelve measurements

Needs numpy and Pillow. No ffmpeg for these — everything is done on decoded arrays so the lattice is under your control. Every figure above comes from this kind of code.

import numpy as np, io
from PIL import Image

def jpeg_rt(a, q, sub=2):
    b = io.BytesIO()
    Image.fromarray(np.clip(a,0,255).astype(np.uint8)).save(
        b, 'JPEG', quality=q, subsampling=sub)   # 0=4:4:4 1=4:2:2 2=4:2:0
    b.seek(0)
    return np.asarray(Image.open(b)).astype(np.float32)

def gradients(y, stride, phase=0):
    """mean |grad| on lattice lines, off lattice lines"""
    gx = np.abs(np.diff(y, axis=1)); cols = np.arange(y.shape[1]-1)
    on = ((cols + 1 - phase) % stride) == 0
    gy = np.abs(np.diff(y, axis=0)); rows = np.arange(y.shape[0]-1)
    onr = ((rows + 1 - phase) % stride) == 0
    return (0.5*(gx[:,on].mean() + gy[onr,:].mean()),
            0.5*(gx[:,~on].mean() + gy[~onr,:].mean()))

# M1: encode once, upscale by k, measure the 8k lattice
#     R stays 1.26x; step falls as 1/k  -> the seam is diluted, not amplified
for k in (1, 2, 4, 8):
    y = (jpeg_rt(PHOTO, 60) if k == 1 else lanczos3(jpeg_rt(PHOTO,60), k)).mean(2)
    gb, gi = gradients(y, 8*k)
    print(k, round(gb/gi, 3), round(gb, 2))

# M6: blind phase sweep, 16 candidates on the 16px output grid
y = lanczos3(jpeg_rt(GT, 50), 2).mean(2)
scores = [gradients(y, 16, p)[0] / gradients(y, 16, p)[1] for p in range(16)]
print('phase', int(np.argmax(scores)), 'score', round(max(scores), 3))

# M6 trap: the same sweep on a CLEAN upscale is not flat either
yc = lanczos3(GT, 2).mean(2)
print([round(gradients(yc,16,p)[0]/gradients(yc,16,p)[1], 3) for p in range(16)])
# -> [1.364, 1.010, 0.883, 0.668, ...]  phase-dependence with no JPEG present at all

Output on this machine for M1, R then step: 1 1.379 4.58 / 2 1.464 2.66 / 4 1.480 1.38 / 8 1.479 0.70. For the phase sweep on the q=50 encode at 2×: phase 0 score 1.716; on the clean control the same sweep spans 0.668 to 1.364. The lanczos3 helper is a separable Lanczos-3 resampler; any correct implementation gives the same numbers to the second decimal, and if yours disagrees the bug is almost always in the kernel normalisation at the edges.

If you are exporting a 4K frame from a JPEG source and the blocks are the thing you can see, the problem is upstream of the resampler: the lattice was baked in at capture or at the last save, and the upscaler only decides how wide each cell gets drawn. The pipeline at the browser upscaler at upgolab.com resamples from whatever master you hand it, and this measurement is the argument for handing it a clean one — the resampler cannot un-bake an 8×8 grid it never saw being applied.