Noise and Grain Under Upscaling: Gain Goes Down, Clumping Goes Up

The usual reason given for shooting clean or denoising before an upscale is that noise gets amplified. Measured properly, the amplitude claim is backwards: a bilinear upscale reduces white-noise amplitude to 0.66× and Lanczos only holds it at 0.89×. What actually happens is worse and quieter than amplification. The noise loses its finest half, the remainder pools into clumps two and a half times the width of the source pixels, and the result reads as mottling rather than grain. And when I tested the standard fix — denoise at low resolution, then upscale — it lost in 12 of 12 cases at the noise level these photographs actually carry. It only starts to pay above σ ≈ 3, which is four times their measured level.

numpy 1.26.4 · Pillow 12.2.0 · four distinct source images, content-hashed so no source is counted twice · resamplers implemented in float64 and validated against Pillow before any figure was reported (max error 9.77 levels, mean 0.17) · PSNR against the clean original at full resolution · six experiments, three of which returned a result I did not want — all three are printed below rather than dropped.

Why noise can be measured exactly, when detail cannot

Detail loss is hard to quantify because nobody has the original. Noise is different. A linear resampler is a linear operator, so a noise field passes through it unchanged in character. Push white noise with standard deviation σin through the same kernel that just processed your image, and the output standard deviation σout is the exact factor by which your noise was scaled. No ground truth needed, no metric to argue about.

That also means the gain is a property of the kernel alone. It does not depend on the image. For a separable resampler it reduces to a closed form: the variance of the sum of weighted independent samples is σ2·Σwk2, so the gain per axis is the mean of Σwk2 over output positions. That prediction can fail, which is what makes it worth checking:

UpscaleKernelMeasured gainPredicted from weightsErrorAttenuation
2×nearest1.00001.00000.00%+0.00 dB
2×bilinear0.62540.62500.06%-4.08 dB
2×bicubic0.80940.80880.07%-1.84 dB
2×lanczos0.89440.89380.07%-0.97 dB
3×nearest1.00001.00000.00%+0.00 dB
3×bilinear0.70390.70370.02%-3.05 dB
3×bicubic0.81570.81530.05%-1.77 dB
3×lanczos0.89490.89330.18%-0.98 dB
4×nearest1.00001.00000.00%+0.00 dB
4×bilinear0.65900.65620.42%-3.66 dB
4×bicubic0.81460.81400.07%-1.79 dB
4×lanczos0.89430.89340.10%-0.98 dB

Worst disagreement across twelve cases is 0.42% — a tenth of a decibel. The gain is not an empirical curiosity, it is the L2 norm of the resampling kernel, so you can compute it for any filter you are about to use.

The amplitude claim is the wrong claim

Look at the direction. Nearest-neighbour leaves noise alone exactly (a copy of a sample carries its noise), and every interpolating kernel pushes it down. Bilinear is the most aggressive at 0.659 at 4×; Lanczos is the most conservative at 0.894. Interpolation cannot invent a sample it has not seen, so noise averages away faster than the correlated signal around it does.

Compare that to what the same kernels do to detail. In the companion measurement on line art, bicubic and Lanczos removed 7% of the ink in the image and 12% of it at 4×, while the noise here retains 89% of its amplitude. Whatever the upscale is damaging, it is not doing it to noise amplitude. Anyone who has looked at an upscaled photo and said "the noise got worse" was describing something other than standard deviation.

What the noise becomes: clumps, not speckles

White noise has a flat spectrum. Once it passes through a resampler that flatness is gone, and the shape of what replaces it is the real story. I measured the share of noise power below a few normalised frequencies, plus the physical size of one noise blob — the autocorrelation lag at which correlation drops to 0.5, in output pixels:

UpscaleKernelσ outPower below 0.05Power below 0.10Clump width
2×bilinear5.017.5%27.3%1.55 px
2×bicubic6.494.7%18.6%1.36 px
2×lanczos7.173.8%15.6%1.32 px
2×source, white8.010.8%3.0%0.50 px
3×bilinear5.6213.2%44.4%2.09 px
3×bicubic6.5010.5%40.1%2.00 px
3×lanczos7.128.8%36.4%1.97 px
3×source, white7.950.8%3.0%0.50 px
4×bilinear5.3225.7%69.9%2.94 px
4×bicubic6.5818.9%62.9%2.67 px
4×lanczos7.2116.1%61.1%2.62 px
4×source, white8.070.8%3.3%0.50 px

The source keeps 3.3% of its power below 0.10 and has a clump width of half a pixel — the definition of unstructured. After a 4× Lanczos upscale, 61.1% of the power sits below 0.10 and one blob spans 2.62 output pixels. The noise is now 5.2× wider than the pixels it came from, and its finest half is gone: power above 0.25 falls from 80.5% to 0.2%.

This is why the result looks wrong in a way that a standard-deviation number cannot express. Film grain and sensor noise are both broadband and fine; the eye reads them as texture at the pixel level. Mottling at two and a half pixels is read as a defect in the image, not as grain over it. Lower amplitude, wrong structure.

The practical consequence. Resampling does not amplify noise, so "denoise to protect the upscale" is not protecting against amplification. What it protects against is the low-frequency mottling the resampler creates — which means a high-frequency denoiser aimed at fine grain is aimed at the wrong band. The noise you needed to remove is mostly at 0.02–0.10 cycles per pixel, and no amount of sharpening or grain synthesis at output resolution puts the fine structure back.

The fix that did not work: denoise before the upscale

The standard advice is to denoise at low resolution, then upscale, because the filter is cheaper there and the upscale will not have to fight the noise. I tested four pipelines against the clean original on all four sources at 2×, 3× and 4×, injecting σ = 4.0 at low resolution — ten times what these images actually contain:

PipelineMean PSNR over 12 casesvs A
Ceiling — clean source upscaled (no noise injected)29.785 dB+0.997
A · no denoising28.788 dB—
B · denoise at low res, then upscale27.023 dB-1.765
C · upscale, then denoise at output res27.016 dB-1.772
D · upscale, then unsharp to restore texture28.140 dB-0.647

Denoising lost to doing nothing by 1.77 dB on average. Two details matter more than the headline. First, B and C are the same result to within 0.03 dB across all twelve cases: the position of a matched gaussian in the pipeline is almost irrelevant, exactly as the sharpen-order measurement found for a linear operator. Blur and upscale nearly commute — float agreement 60.4 dB, 8-bit agreement 56.3 dB with 86.2% of pixels identical. Whoever tells you the order matters is describing the quantisation step, not the blur.

Second, D came closer to A than either denoiser did, which is the interesting failure. Sharpening does raise the measured noise, from σ 3.586 to 5.926 at amount 0.8 — a 65% increase in apparent noise — and it still scores better than blurring. Perceptually that is the correct trade: a noisy but structured image beats a smooth but mottled one. On PSNR it is worth 2.93 dB of penalty. Every objective metric here prefers the sharpened, noisier version.

Where the denoise does start to pay

A single fixed blur is a straw man, so I swept eight strengths against seven injected noise levels. The answer is a threshold, not a rule:

Injected σMean gain from best pre-denoiseCases wonBest strength
0.5-0.015 dB0/12—
1.0-0.013 dB0/12—
2.0-0.005 dB3/120.30
3.0+0.047 dB3/120.40
4.0+0.154 dB6/120.30
6.0+0.449 dB11/120.30
8.0+0.828 dB12/120.30

The crossover is at σ ≈ 3.0. Below it, denoising is a net loss in every case; by σ = 8 it wins 12/12 and is worth +0.83 dB. And the best strength barely moves — 0.30 regardless of how much noise there is — because the optimum is set by the image's own detail scale, not by the noise level. More noise does not call for a stronger blur; it calls for the same blur you would otherwise avoid.

So the threshold only matters if your images are above it. I estimated the noise level of the four sources with the MAD-of-Laplacian estimator, which is the standard robust method and does not inflate when a source is full of real detail:

SourceMeasured σ (MAD estimate)Above the crossover?
texture0.663no
plain0.663no
photo0.000no
flat sheet0.995no

Every source sits between 0.00 and 1.00 — three to six times below the crossover. At σ = 1.0 the sweep gives -0.013 dB with 0 of 12 cases won. For photographs like these, denoising before the upscale is a measurable loss and the correct action is to leave the noise alone.

Why one source behaved differently, and what predicts it

The per-source picture is not uniform, and hiding it behind an average would be dishonest. Best pre-denoise gain by source, averaged over the three scales:

SourceFlat area (gradient < 1)Mean best gainDenoise efficiency
photo63.0%+0.625 dB+0.111
flat sheet22.6%-0.007 dB-0.914
plain22.4%-0.003 dB-0.858
texture17.9%+0.002 dB-0.771

"Denoise efficiency" is noise power removed divided by clean-signal power destroyed at 0.8 px, both measured against the same ground truth. It is negative for three of the four sources: the blur removes less error than it adds, because the detail it destroys sits in the same band as the noise it was supposed to remove. Only the photo source is positive, at +0.111.

That source is also the one with 63.0% of its pixels in near-flat area, against 17.9% for the textured one. Flat area is where a blur costs nothing and removes noise freely, so the four points line up almost perfectly: Pearson r = 0.9934, slope +0.0148 dB per percentage point of flat area. With four points that correlation is suggestive rather than established — it is a hypothesis with a number attached, not a law.

The decision rule that falls out of it is more useful than "denoise or don't". Ask what fraction of the frame is flat. A portrait against a backdrop, a product on seamless paper, a sky — those are above 60% flat and a light pre-denoise can be worth half a decibel. A texture sheet, foliage, a crowd, any frame that is detailed edge to edge, is below 25% and pre-denoise is a pure loss.

Grain is not free either: what it costs the encoder

Keeping the noise has a price, and it is entirely in the file size. Adding synthetic grain of a given σ to a fixed image and encoding at a fixed quality:

Grain σJPEG q85vs cleanJPEG q92vs clean
0.042,600 B1.00×57,732 B1.00×
1.546,082 B1.08×65,526 B1.14×
3.053,695 B1.26×82,867 B1.44×
5.067,477 B1.58×104,699 B1.81×
8.088,431 B2.08×126,588 B2.19×

The cost is superlinear at q85 (2.08× for σ 8) and worse at q92 (2.19×), because a higher quality target spends its budget trying to preserve exactly the high-frequency structure that noise is made of. This is the real argument for denoising old footage: not that the upscale needs it, but that the encoder will otherwise bill you double for information that was never signal.

A detail worth noting for anyone who denoises at low resolution to save the encode: the noise you removed is broadband, but what an upscale leaves behind is not. Blade Runner grain scanned at σ 3 and upscaled becomes mottling at 0.05 cycles per pixel, which is the cheapest thing in the frame to encode — the encoder is not the reason that combination fails.

What I would actually do

  1. Stop denoising to protect the upscale. Resampling attenuates noise to 0.66–0.89×; there is no amplification to protect against. This is the one claim in the usual advice that is simply wrong in direction.
  2. Denoise only if the frame is above σ ≈ 3, or above roughly 60% flat area at lower σ. Below either line the measurement says leave it.
  3. If you do denoise, denoise at low resolution with a strength matched to the detail scale, not to the noise level. 0.4 px was optimal at every noise level from σ 2 to σ 8.
  4. Do not denoise the fine band. The mottling that survives an upscale lives at 0.02–0.10 cycles per pixel. A high-frequency grain reducer removes the part a viewer is least likely to object to and leaves the part that reads as a defect.
  5. Prefer a slightly noisy asset over a smooth one when the choice is real. Sharpening to restore texture scored 28.140 dB against 27.023 for the blur, at +2.34 more apparent noise.
  6. Budget the bytes. Grain at σ 8 costs 2.08× the file at q85. If the delivery target has a hard size limit, that is the constraint to design against, not the upscale.

Reproducing it

import numpy as np
from PIL import Image

def lanczos3(a, out_hw):                      # separable, float64
    oh, ow = out_hw; ih, iw = a.shape
    def wts(n_in, n_out):
        sc = n_in / n_out; st = max(sc, 1.0)
        pos = (np.arange(n_out) + 0.5) * sc - 0.5
        half = int(np.ceil(3 * st))
        idx = np.floor(pos)[:, None] + np.arange(-half, half + 1)[None, :]
        d = (pos[:, None] - idx) / st
        w = np.sinc(d) * np.sinc(d / 3.0) * (np.abs(d) < 3)
        w /= w.sum(axis=1, keepdims=True)
        return np.clip(idx, 0, n_in - 1), w
    i1, w1 = wts(iw, ow); tmp = np.einsum("hot,ot->ho", a[:, i1], w1)
    i2, w2 = wts(ih, oh); return np.einsum("otw,ot->ow", tmp[i2, :], w2)

# 1. exact noise gain: the kernel is linear, so push white noise through it
rng = np.random.default_rng(0)
F   = 4
low = rng.normal(0, 8, (600 // F, 480 // F))
out = lanczos3(low, (600, 480))
print(out.std() / low.std())                  # 0.894 -> noise is ATTENUATED

# 2. and it is no longer white: measure the blob size instead of the std
x, var = out - out.mean(), out.var()
for L in range(1, 40):
    c = ((x[:-L] * x[L:]).mean() + (x[:, :-L] * x[:, L:]).mean()) / 2 / var
    if c < 0.5:
        print("clump width", L, "px")         # 2.62 px for a 4x Lanczos
        break

Change the kernel to a box and the gain becomes exactly 1.000 — nearest-neighbour copies each sample and its noise with it. Change the scale factor and the gain barely moves, which is the point: noise transfer is a property of the filter, so it is knowable before you run the job rather than after.

Three results that went against what I expected, kept here on purpose. I expected the noise gain to rise with the scale factor; it does not, it is flat to within 0.01 across 2×, 3× and 4×. I expected denoising at low resolution to beat denoising at output resolution because the filter sees less work; the two agree to 0.03 dB, because the operators nearly commute. And I expected a single denoise recommendation to hold; instead it flips sign at σ ≈ 3, which is above every source I had.

The general lesson is the one the sharpen-order measurement reached from a different direction. For a linear pipeline, the position of an operation is close to irrelevant and the kernel is everything. Noise is the cleanest place to see it, because the operator is exactly linear and the answer is a closed form. An upscale does not make noise louder. It makes it softer, wider and harder to mistake for texture, and the standard remedy for that is applied in the wrong band.