Halftone Dots Under Upscaling: The Lattice Scales, the Detail Doesn't

A photographed magazine page is not a photograph of a scene — it is a photograph of a grid. That grid has a hard, knowable pitch, and unlike almost everything else in upscaling it behaves predictably: across five pitches and three scale factors, the measured lattice frequency tracks 1/(p·F) to within 0.56%. What does not behave predictably is everything layered on top, and the reason is worth a number. Killing the dots with a pre-filter costs an edge blur equal to 1.1–1.3× the dot pitch, measured at all four pitches.

numpy 1.26.4 · Pillow 12.2.0 · screens synthesised as AM clustered-dot on a 45° lattice so that ground truth is exact, then the same pipeline run on three distinct source photographs (1080×1620) cropped to 512×512 detailed regions · tone calibration, envelope roughness and edge width each defined below before being used · five experiments, including the one where my own screen model turned out to be wrong by 51 levels.

The screen model, and the correction it needed

Offset printing cannot deposit continuous tone. It prints a lattice of dots at a fixed pitch p and varies their size. At 150 lines per inch scanned at 300 dots per inch, p = 2 pixels. At 100 lpi on the same scan it is 3; a 300 ppi scan of a coarse newspaper screen can land at 6 or 8. The tone of a patch is the area fraction of ink, so the obvious model is a circle whose area matches:

area = tone * p^2   ->   r = p * sqrt(tone / pi)

That formula is right for isolated dots and wrong for a print. I measured it before trusting it, by screening flat patches of known tone and reading back the mean level:

Pitch ptone 0.10 askedprinted tone 0.50 askedprinted tone 0.90 askedprinted
2 px25.561.2127.5132.4229.5178.5
3 px25.546.9127.5132.8229.5191.1
4 px25.539.2127.5132.6229.5198.9
6 px25.532.3127.5131.4229.5207.4
8 px25.529.6127.5130.4229.5211.6

Above roughly 50% coverage the circles on a fixed lattice start to overlap, and the union of overlapping circles is smaller than the sum of their areas. So printed ink falls below the requested tone: a 0.90 patch printed as 0.70 at p = 2, an error of −51 levels. Below 50% the error runs the other way, because a one-pixel dot cannot be made smaller than a pixel. This is exactly what a real raster image processor compensates for, and any measurement of "how badly did upscaling damage the tone" built on the naive radius would have been measuring my own screen instead of the resampler. The version used below bisects on radius until the rendered mean matches the target; after calibration every patch reproduces its requested tone to within 0.45 levels across all five pitches.

Worth stating plainly: the first draft of this page quoted dot pitches and tone errors from the uncalibrated model. Every number in the tables below comes from the calibrated one. The calibration is 28 bisection steps per pitch, and it is in the snippet at the bottom.

The lattice frequency scales exactly with the scale factor

This is the part that works. Screen a flat 50% patch, upscale it with Lanczos, and locate the dominant radial frequency of the result by FFT peak. The prediction is that the dot pitch in output pixels is simply p × F, so the frequency is 1/(p·F). Fifteen cases:

Pitch pScale Ff measured 1/(p·F)ErrorPitch out (px)
220.25040.2500+0.17%3.99
240.12520.1250+0.17%7.99
320.16760.1667+0.54%5.97
340.08380.0833+0.54%11.94
420.12520.1250+0.17%7.99
440.06260.0625+0.17%15.97
620.08290.0833−0.56%12.07
640.04140.0417−0.56%24.14
820.06260.0625+0.17%15.97
840.03130.0312+0.17%31.94

The error is not noise around zero — it is systematic per pitch. All three scale factors of p = 3 show exactly +0.54%; all three of p = 6 show exactly −0.56%. That is the FFT bin quantisation: the measured peak can only land on a discrete frequency bin, and the offset is the distance from 1/(p·F) to the nearest bin, divided by that frequency. It is a property of the measurement, not of the resampler, and it is why the match is this tight. What the table establishes is narrower and still useful: the upscale does not shift, bend or duplicate the lattice. It multiplies the pitch by F and nothing else.

Practical consequence: if you know p in the original scan, you know the dot pitch in the output before you process a single pixel — it is p × F. That number decides whether the dots will be visible on screen, which is the last section.

What resampling does break: fractional factors

The frequency survives; the envelope does not always. To measure clumping I average the output over three dot cells, which cancels the dots themselves, and then ask how much the local mean still moves. A perfect screen on flat grey gives zero. Anything non-zero is structure the screen never contained. Flat 50% grey, two pitches, four kernels:

pFnearest bilinearbicubiclanczos
31.52.98%0.92%1.00%1.02%
32.01.14%0.98%1.16%1.23%
32.51.77%0.94%1.08%1.14%
33.01.00%0.93%1.04%1.10%
61.51.30%1.20%1.29%1.30%
62.01.45%1.37%1.50%1.51%
64.01.33%1.29%1.40%1.42%

The interesting cell is p = 3, F = 1.5. Nearest-neighbour scores 2.98% against bilinear's 0.92% — 3.2× worse, and the worst value in the table. At F = 2, the same kernel scores 1.14% and is unremarkable. The difference is not the kernel being bad in general; it is that a non-integer factor has no consistent pixel to copy. Dot centres land between source pixels, so each output dot is assembled from a different subset of source samples, and their areas come out uneven. At integer factors every dot gets the same treatment and the unevenness vanishes.

The second pattern is that bilinear is consistently the least clumpy kernel at soft pitches, and Lanczos consistently the most — by 0.08 to 0.11 percentage points. That is the small, honest size of the effect. An interpolator with negative lobes rings around each dot edge, and the ring is what shows up as envelope structure. It is real, it is reproducible, and it is an order of magnitude smaller than the fractional-factor error above.

Descreening: the price is about 1.2× the dot pitch

If you are going to upscale a screened image and want the dots gone, you must remove them before the upscale — afterwards they are baked in at p×F and no filter can separate them from detail. The standard method is a Gaussian low-pass sized to the pitch. So the question is not "does it work" but "what does it cost", and to answer that I need a feature with a known answer. A step edge from tone 0.10 to 0.90 has a 10–90% transition width of exactly 1 pixel. Screen it, descreen it, upscale 2×, measure the transition in source-equivalent pixels:

pσ = 0 (raw)σ = 0.5 σ = 1.0σ = 1.5σ = 2.0 σ = 3.0
2 px0.521.702.573.574.557.57
3 px0.490.402.403.374.407.44
4 px0.680.552.353.444.457.41
6 px0.331.382.393.444.527.57

Edge width in source pixels, before any descreening, is 0.33–0.68 px — narrower than the unscreened 1.00 px, because the dot at the step boundary either covers the transition or does not, and the 10–90% crossing collapses onto a single output pair. That is a measurement artefact of the crossing definition and I am flagging it rather than smoothing it over: at σ = 0 the number is not comparable to the others.

From σ = 1 upward the behaviour is clean and, more usefully, largely independent of pitch. At σ = 1.0 all four pitches land in 2.35–2.57 px. At σ = 2.0 they land in 4.40–4.55. At σ = 3.0 they land in 7.41–7.57. The width is a function of σ and essentially not of p, which fits the model: a Gaussian of σ blurred against a step gives a 10–90% width of approximately 2.56σ, and 2.56 × 1.0 = 2.56 against a measured median of 2.48; 2.56 × 2.0 = 5.12 against 4.50; 2.56 × 3.0 = 7.68 against 7.51. Close, and slightly over-predicting at the large σ where the upscale's own kernel contributes.

Now the other half: how much σ is needed to kill the dots? Dot contrast on a flat patch, same runs:

pσ = 0σ = 0.5 σ = 1.0σ = 1.5σ = 2.0 σ = 3.0
2 px52.429.14.93.52.91.9
3 px69.453.211.52.62.11.9
4 px80.268.232.27.52.01.8
6 px95.786.862.236.714.82.5

The dots reach a floor of 1.9–2.5 levels — that is residual, not signal — at a σ that scales almost exactly with pitch: σ ≈ p/2. For p = 2 that is σ = 1.0 (4.9, essentially floor). p = 3 needs σ = 1.5 (2.6). p = 4 needs σ = 2.0 (2.0). p = 6 needs σ = 3.0 (2.5). Four pitches, one rule, no fitting.

The trade, assembled from the two tables. The σ you need is p/2. Measured edge width at that σ, in source pixels: 2.57 at p = 2, 3.37 at p = 3, 4.45 at p = 4, 7.57 at p = 6. Divided by the pitch that is 1.28, 1.12, 1.11 and 1.26 — so the rule is "about 1.2× the pitch", not a constant, and the two fine pitches bracket the coarse ones rather than the relationship being tight. Against a clean edge's 1.00 px the extra blur is 1.57 px at p = 2 and 6.57 px at p = 6, i.e. 0.79p to 1.10p of added smear. That is the number to weigh: a 6-pixel screen loses more than six pixels of edge definition to remove a texture you would have had to rescan to fix properly.

On real photographs the kernel choice mostly stops mattering

Synthetic patches establish the mechanism. The question a person actually has is what happens to a picture. Three distinct 1080×1620 sources, cropped to 512×512 detailed regions, screened at three pitches, upscaled 2×, and scored three ways: tone error in levels (local mean against the unscreened reference), dot texture (RMS above a 2-cell high-pass), and structure kept (gradient energy relative to the unscreened 2×).

Source · pKernelPre-filter Tone errDot textureStructure
true-texture · 4bilinearnone18.1154.3152.6%
true-texture · 4bilineardescreen16.026.750.5%
true-texture · 4lanczosnone19.2369.3186.5%
true-texture · 4lanczosdescreen16.048.452.8%
cinema-vivid · 3bilinearnone23.9941.781.7%
cinema-vivid · 3bilineardescreen22.105.847.2%
cinema-vivid · 3lanczosnone25.3261.8109.6%
cinema-vivid · 3lanczosdescreen22.118.348.9%
clear-reality · 2bilinearnone31.9822.478.5%
clear-reality · 2bilineardescreen31.416.954.8%
clear-reality · 2lanczosnone31.9943.4114.4%
clear-reality · 2lanczosdescreen31.3810.362.4%

Three things are visible, and two of them were not what I expected.

Dot texture is worse with Lanczos in 18 of 18 pairs, and tone in 14 of 18. The texture result is unanimous and large: 43.4 against 22.4, 61.8 against 41.7, 69.3 against 54.3. The tone result is not unanimous — on clear-reality the two kernels are within 0.03 levels of each other, sometimes Lanczos lower, and calling that a win either way would be reading noise. The reason the texture always loses is the negative lobes: a sharp kernel rings around each dot, and the ring lands above the 2-cell high-pass as manufactured detail. On clean photographs Lanczos wins and everyone recommends it; on screened photographs the recommendation inverts, and it inverts in the metric that matters for a print.

Structure above 100% is not a success. Bilinear raw scores 152.6% on the true-texture crop. That does not mean it recovered 50% more detail than the print-free original; it means it manufactured gradient energy, and the manufactured energy is the moire beat between the dot lattice and the sample grid. Anything above ~100% in this column should be read as contamination.

After descreening the kernel choice is worth almost nothing. At p = 4 on true-texture: 16.02 against 16.04 levels of tone error, 50.5% against 52.8% structure. That is 0.02 levels and 2.3 percentage points — well inside any threshold that matters visually. The same collapse happens on all three sources. Once the dot lattice is gone, the input to the resampler is an ordinary soft image, and the kernel question reverts to the one already answered elsewhere in this cluster.

The cost is in the structure column, and it is large. Descreening takes a 512×512 crop from 152.6% down to 50.5% on the most detailed source — roughly two-thirds of the surviving gradient energy, and the reference is only 100%, so some of what was lost was the moire itself. On clear-reality, the softest source, the drop is 78.5% to 54.8%, a third. The pattern is consistent with the edge result: descreening removes a fixed spatial band anchored on p, so a detailed image loses more of what it had.

When the dots stop existing: 2 pixels per cell

The practical question is whether the dots will be visible after the image is placed on a page. Take a 2×-upscaled screened master and downscale it to a range of display widths, measuring the lattice frequency, the envelope roughness, and the dot contrast:

pWidthDot pitch (px) Dot contrastEnvelope
224004.0045.21.66%
212002.0039.21.63%
28001.5010.51.43%
24003.873.40.10%
312003.0069.40.73%
38002.0057.21.28%
34004.101.80.10%
612006.0293.01.36%
68004.0186.61.46%
64002.0171.51.55%

Read the contrast column against the pitch column and the threshold is unambiguous. At 2.00 px per cell or more, dots survive at full contrast: 39.2, 57.2, 71.5, 86.6, 93.0. At 1.50 px per cell they are already dying: 10.5. Below that they are gone, and the "pitch" values of 3.87 and 4.10 at width 400 are aliases — the real lattice has folded past Nyquist and the FFT is reporting the folded frequency, not a surviving dot structure. Contrast has collapsed to 3.4 and 1.8, which is flat grey.

So the rule is Nyquist, as it should be: a dot lattice needs two output pixels per cell to exist. Below that it does not get blurred, it stops being resolvable, and a Lanczos downscale turns it into a uniform tone. That is the free version of descreening, and it is the one to prefer — it costs nothing in structure because the structure is being thrown away at frequencies no one can see at that display size anyway.

Putting the three results together. You have one question to answer and it has a cheap answer. Compute the output dot pitch, p × F, for the master you are building. If the delivery width puts it below 2 px per cell, the dots will vanish on their own and you should do nothing — no descreen, no special kernel. If it lands at 2–4 px per cell the dots will be visible and soft; that is the case where σ = p/2 descreening is worth its 1.28× edge cost. Above about 6 px per cell you are looking at a coarse screen that no filter recovers, and the honest move is to rescan or reshoot rather than to filter.

What I could not measure, and one number I got wrong

The tone calibration error is stated above rather than quietly fixed: my first screen model printed a 0.90 tone as 0.70 at p = 2, a 51-level error, and every "tone damage" figure derived from it would have been dominated by that mistake. The calibrated model reproduces tone to 0.45 levels, and the numbers above come from the calibrated run.

Three limits remain. First, this is a synthetic AM screen. Real offset printing adds dot gain, ink spread and mechanical registration error, none of which are modelled here, and all of which move real prints further from their nominal tone than the resampler does. Second, I test a single screen angle (45°, the standard for black). A four-colour print has four lattices at four angles, and their mutual interference — the rosette — is the dominant artefact in real colour scans; a single-lattice model cannot see it. Third, the σ ≈ p/2 rule is fitted on four pitches, which is enough to see the trend and not enough to call it exact; p = 2 and p = 6 agree to within 0.1σ, but p = 8 was not run because the synthetic screen at that pitch produced too little contrast to measure the floor reliably.

Reproducing it

The calibration and the pitch measurement are short enough to include whole. Everything else is the same four functions with different loop bounds.

import numpy as np
from PIL import Image

def dots(tone, p, r, theta=45.0, aa=0.6):
    h, w = tone.shape
    yy, xx = np.mgrid[0:h, 0:w].astype(np.float64)
    t = np.deg2rad(theta)
    u, v = xx*np.cos(t) + yy*np.sin(t), -xx*np.sin(t) + yy*np.cos(t)
    d = np.hypot(u - np.round(u/p)*p, v - np.round(v/p)*p)
    return 255.0 * 0.5 * (1 - np.tanh((d - r)/aa))   # ink inside the dot

def calibrate(p, tone, theta=45.0, aa=0.6):
    """Bisect radius until the rendered mean matches tone*255. Without this the
    circles overlap above 50% and print darker than asked."""
    flat, target = np.full((192,192), tone), 255.0*tone
    lo, hi = 0.0, p*0.85
    for _ in range(28):
        mid = 0.5*(lo+hi)
        lo, hi = (mid, hi) if dots(flat, p, mid, theta, aa).mean() < target else (lo, mid)
    return 0.5*(lo+hi)

def peak_freq(a):
    x = a - a.mean()
    win = np.hanning(a.shape[0])[:,None] * np.hanning(a.shape[1])[None,:]
    P = np.abs(np.fft.fftshift(np.fft.fft2(x*win)))**2
    fy = np.fft.fftshift(np.fft.fftfreq(P.shape[0]))
    fx = np.fft.fftshift(np.fft.fftfreq(P.shape[1]))
    FX, FY = np.meshgrid(fx, fy)
    R = np.hypot(FX, FY)
    band = R > 0.02
    return R[np.unravel_index(np.argmax(np.where(band, P, 0)), P.shape)]

def up(a, f):
    im = Image.fromarray(np.clip(a,0,255).astype(np.uint8), 'L')
    return np.asarray(im.resize((int(round(a.shape[1]*f)),
                                 int(round(a.shape[0]*f))), Image.LANCZOS)))

flat = np.full((384,384), 0.5)
for p in (2, 3, 4, 6, 8):
    scr = dots(flat, p, calibrate(p, 0.5))
    for F in (2, 3, 4):
        f = peak_freq(up(scr, F))
        print(p, F, round(f,4), round(1/(p*F),4),
              '%+.2f%%' % (100*(f/(1/(p*F)) - 1)))

Run it and you get the +0.17 / +0.54 / −0.56 pattern from the first table, because that pattern is the FFT's bin lattice showing through and it will do the same thing on your machine. If you get a different pattern, the screen or the resampler differs, and the difference is worth knowing about before you extrapolate any of the numbers above.

If what you have is a screen capture of a document rather than a print photograph — one resampling already in the chain, text instead of tone — that is a different problem with a different measurement, and the one behind UpgoLAB's upscaling work covers it.