Fourier Smoothing in Aeon: Symmetry, Cutoffs, and Better Tests

Aeon’s DiscreteFourierApproximation is a transformer for smoothing real-valued time series in the frequency domain. It splits a series into waves, keeps some of them, and adds the kept waves back together. The parameter r sets how many survive:

• With sort = false, it keeps a contiguous block of the lowest-index coefficients, which is a plain low-frequency cutoff.
• With sort = true, it ranks the coefficients by amplitude instead, so the kept set need not be one low-frequency block.

Before the change described here, it could return a wave it was told to keep at half its height, and its tests still passed. The cause is a property of real signals. In their Fourier transform, each wave except the mean (DC) and, for even lengths, the fastest wave (Nyquist) is stored as a conjugate pair of matching coefficients, and the old smoother could keep one and drop the other.

The checks in this post compare the output with answers known in advance. These checks expose concrete failure modes; they do not prove the transformer universally correct for every input and option.

Setup

The transform. The fast Fourier transform (FFT) rewrites a series of N samples as N coefficients. Each coefficient is a complex number. Its size says how strong one wave is, and its angle says where that wave starts. The wave at bin k completes k cycles over the series.

The smoother. DiscreteFourierApproximation takes the FFT, sets some coefficients to zero, and transforms back. The list of coefficients it keeps is called the mask.

Conjugate pairs. For a real-valued series, the coefficients come in pairs. The coefficient at a positive frequency has a partner at the matching negative frequency. The partner is its complex conjugate: same real part, opposite sign on the imaginary part. A real cosine is carried by both members of its pair, half each.

Two coefficients have no separate partner, because each is its own conjugate:

• DC, the zero-frequency term. It carries the mean.
• Nyquist, the highest frequency. It has its own coefficient only when N is even.

Keep one member of a pair and drop the other, and the output can still look like a normal waveform. It just carries half of the cosine it should.

What to measure. Looking at the shape of the output is not enough. A check needs a number to compare against. This post uses three:

• Passband gain: how much of a kept frequency comes back, as output amplitude over input amplitude. A unit cosine (amplitude one) at a kept bin should come back with gain 1.
• Stopband rejection: whether a removed frequency is really gone. A cosine just past the cutoff should come back at 0, and it locates the cutoff on the sort = false path.
• The round trip: a mixture kept in full should come back unchanged. It checks the whole transform, mask and inverse, and with them whether the conjugate pairs are respected end to end.

The gain and reconstruction numbers below come from a direct, deterministic model of the old and the fixed implementation, with these settings: N=64 samples, sort = false, unit cosines at bins 1 through 32, r ∈ {0.25, 0.50, 0.75, 1.0}, and an absolute tolerance of 1e-10. None of it measures classifier quality, throughput, or a production workload.

What went wrong

The old implementation worked in three steps:

1. Take an ordinary two-sided FFT. It holds all N coefficients, positive and negative frequencies alike.
2. Keep max(floor(r * N), 1) of them. With sort = false, these are the first indices.
3. Return the real part of the inverse transform.

Step 2 is the problem. The first indices hold positive frequencies. Their negative-frequency partners sit at the far end of the array, so a low cutoff zeroes them. Figure 1 shows the result for one cosine.

Figure 1. A cosine inside the kept band should come back unchanged. The old smoother returns it at half height; the fixed smoother returns it whole.

The existing value test did not catch this. It checked that the one-channel and multichannel paths agreed. That is a useful check, but both paths used the same masking rule. If the mask dropped the negative-frequency partner in both, the two outputs were equally wrong and still compared equal.

A consistency check shows that two paths give the same values. It does not show that the values are right. That needs an expected value from outside the code. Here that is cheap: feed in a unit cosine at a frequency that should be kept, and check that it comes back with amplitude one, within a tolerance.

A short reproduction

The smallest case is a unit cosine at bin 3, with N = 64 and r = 0.25. The code below compares the old two-sided mask with the merged real-spectrum mask. The real spectrum is the nonnegative half of the coefficients, which NumPy’s rfft returns.

The index arithmetic shows where the half comes from:

• The old path keeps floor(0.25 × 64) = 16 indices, 0 to 15. Coefficient 3 survives. Its partner, coefficient 61, is zeroed.
• The real path keeps floor(0.25 × 33) = 8 bins, 0 to 7. Bin 3 survives.

Bin 3 survives in both. Only the real representation carries what the inverse needs to rebuild its negative-frequency partner. The output depends on which coefficients survive, not on a scale factor.

This runs in a Python REPL:

import numpy as np

N = 64
r = 0.25
k = 3
x = np.cos(2 * np.pi * k * np.arange(N) / N)

old = np.fft.fft(x)
old[np.arange(N) >= max(int(np.floor(r * N)), 1)] = 0
old_y = np.fft.ifft(old).real

merged = np.fft.rfft(x)
merged[np.arange(N // 2 + 1) >= max(int(np.floor(r * (N // 2 + 1))), 1)] = 0
merged_y = np.fft.irfft(merged, n=N)

print(old_y.max(), merged_y.max())
assert np.isclose(old_y.max(), 0.5, atol=1e-10)
assert np.isclose(merged_y.max(), 1.0, atol=1e-10)

In the fixed-loop model behind this post’s numbers, the old peak is about 0.5000000000000002 and the merged peak about 0.9999999999999997. These are floating-point results, not exact decimals. The test passes when each is within 1e-10 of its expected value.

The max(..., 1) is deliberate. At r = 0, both models keep the single DC coordinate instead of returning an empty spectrum. At full retention, the even-length Nyquist coordinate is kept. Neither case splits a conjugate pair, so neither supports a blanket claim that every kept term is halved.

Why exactly one half

For a real-valued sequence, the DFT (discrete Fourier transform, which the FFT computes) is Hermitian. That means each coefficient’s mirror is its complex conjugate:

X[N-k] = conjugate(X[k]).

Take a bin-aligned unit cosine, one that completes a whole number of cycles over the series. Its positive and negative coefficients contribute equally to the inverse. Writing out that pair and ignoring the other frequencies:

x[n] = cos(2πkn/N)
     = (X[k] exp(2πikn/N) + X[N-k] exp(-2πikn/N)) / N.

If a two-sided mask keeps X[k] but zeroes X[N-k], the real part of the inverse has only one of the two equal contributions:

real{X[k] exp(2πikn/N) / N} = 1/2 cos(2πkn/N).

Figure 2 draws the same thing on the coefficient array. For the bin-3 cosine, X[3] and X[61] both have size 32, which is N/2.

Figure 2. A non-DC cosine needs both members of its conjugate pair for full amplitude. The old mask drops one partner, so the output has half gain. The real-spectrum inverse rebuilds the partner by construction.

That is the half-gain identity, and it is conditional. It applies to a non-DC component when exactly one member of its pair survives the old mask. It does not apply in three cases:

• DC, which is self-conjugate.
• Nyquist for even N, which is also self-conjugate.
• A pair where both members survive. At r = 0.75, for example, the old mask keeps enough coordinates that higher bins regain full amplitude, because both sides of their pair fall below the keep boundary.

The NumPy FFT documentation states the rule for real input: the negative-frequency terms are redundant conjugates of the positive ones. rfft returns only the nonnegative part, including DC and, for an even length, Nyquist. irfft takes the original length and fills in the conjugate partners when it inverts. The length matters because the half spectrum alone cannot tell an even length from the odd length one above it: 64 and 65 samples both give 33 coefficients.

Using rfft does not make every spectrum edit safe. It means the representation itself encodes the symmetry that the old first-index mask could break. Code that changes individual complex coefficients still has to keep the output real.

Results: gain by frequency

Figure 3 runs a unit cosine through every bin from 1 to 32 and plots the gain, for three values of r. The solid old line has a plateau at one half, from keeping only one partner. The dashed fixed line gives gain one for every kept non-DC cosine and zero for the removed range.

Figure 3. The fixed real-spectrum path restores gain one in its passband, but the same r now means a different cutoff, because it is a fraction of a different set of coordinates.

The r = 0.75 row is the exception. The old mask keeps both partners of bins 17 to 32, so they come back at gain one, while bins 1 to 16 still get one half.

Across all rows, the computed gains differ from these hand-derived values by at most about 2.220446049250313e-15, well within 1e-10.

The cutoff moved

The merged real-spectrum mask also changes what r means. At N=64 and r=0.5, the old two-sided mask keeps 32 of 64 coordinates while the merged real-spectrum mask keeps 16 of 33; the same r therefore selects a different cutoff.

The fix restores amplitude for the kept non-DC bins. It does not promise that an existing r keeps the same frequencies. Callers who depend on the cutoff should pick r from the boundary they want, check which bins are kept, and rerun their signal-level tests after this change.

A cutoff migration test makes this checkable:

1. Take the highest bin meant to pass and the first bin meant to be removed.
2. Build the mask with the representation’s actual width.
3. Assert gain one on the first and zero on the second.

That pair catches a moved boundary without confusing it with the half-amplitude defect. A passband check alone proves the gain of one kept bin. It does not prove that the kept set still matches the band the caller meant.

The model uses sort = false because it is an exact index mask, easy to reason about. That is not a claim that sort=True is safe. Ranking coefficients by amplitude can also split a conjugate pair if the selection rule is not built around real-signal symmetry.

The fix

The problem was raised in Aeon issue #3710 and repaired in PR #3711, which a maintainer reviewed and merged. This post uses that change as a worked example of how real-signal symmetry shapes a small repair and its tests.

The PR was merged as a squash merge, which turns a pull request into one new commit. So there are two revisions: the submitted PR revision a10dcceff403a86f01805655bd86f37c15d6ab8a and the separate squash merge 490f9fa61ce0f38d7899ab61170eff994dfafa7d.

The merged diff makes three changes:

1. It replaces the ordinary fft with rfft.
2. It replaces the real part of an ordinary inverse with irfft(..., n=original_length).
3. It redefines r, in the documentation, as a fraction of the N/2 + 1 real-spectrum coordinates instead of the old two-sided width.

The merged source and the full diff of squash merge 490f9fa, tests included, are pinned to that commit.

n=N tells the inverse which length to rebuild. An even length has a Nyquist coordinate; an odd length has a differently shaped half spectrum. The mask decides which coordinates survive, and n=N decides how they map back to samples. Test the length argument and the mask width together: testing only one leaves the other free to change the rebuilt endpoint or the kept band.

Test results. Before the change, the focused regression tests had 14 outcomes: 3 failed and 11 passed. The failures exposed the amplitude error that the cross-channel check could not see.

A wider capture covered the series test suite, still not the whole repository:

• Unchanged code: 209 passed and one skipped.
• After the submitted change: 217 passed and one skipped.

With one skip in both, the pass count rose by exactly eight. That shows the revision ran more passing cases in that suite. It says nothing about a published release or about systems that use the transformer.

Tests with a known answer

An oracle is a check against a value known in advance. Figure 4 shows three of them, run on the fixed model.

• Passband. Feed a unit cosine at a bin that should be kept, and expect gain one. For merged bin 3 at r = 0.25, the expected gain is exactly 1 and the observed gain is about 0.9999999999999998.
• Stopband. Feed a cosine at a bin that should be removed, and expect zero. For merged bin 8 at r = 0.25, the target is exactly 0 and the observed result is about 2.033782021272921e-31.
• Full round trip. A fixed mixture of DC, cosine, sine and Nyquist components at r = 1 should come back within the tolerance. No component is meant to be removed, so a failure points at the complete transform-mask-inverse pipeline and can come from selection or reconstruction. The mixture also exercises the two self-conjugate endpoints that the bin-3 case does not.

All three use a tolerance of 1e-10. For the round trip, the largest absolute reconstruction error is about 1.815214645262131e-14 in the old direct model and 1.4127587988355117e-14 in the merged one. Both are numerical round trips within 1e-10, not bit-for-bit equality.

Figure 4. Three oracles test amplitude, removal and reconstruction separately. Each has an exact target, a tolerance, and a failure that agreement between channels alone cannot reveal.

The merged test file adds amplitude cases, odd- and even-length full-retention checks, and a high-frequency rejection check alongside the existing consistency coverage.

Limits

• Half gain has a narrow condition. Exactly one member of a non-DC component’s conjugate pair must survive the old mask. DC, even-length Nyquist, and pairs kept on both sides fall outside it.
• The gain and round-trip numbers come from one model. They show agreement within 1e-10 for the direct model used here, not bit-for-bit identity or identical results from every FFT implementation.
• There is no general rule for converting an old r to a new one. The right cutoff depends on each caller’s band and signal, so callers have to derive and check their own.
• The regression captures show test behavior only. They do not measure classifier accuracy, speed, adoption, user outcomes, or any production incident.
• No release was verified to contain the fix. At the 2026-08-07 research cutoff, the latest recorded Aeon release was v1.5.0 from 2026-06-29, before the 2026-08-06 merge. Main-line documentation is not proof of a release.

Checklist

1. Before choosing a Fourier representation, write down the property the signal must keep (gain, phase, energy, cutoff or another quantity) and a check that can measure it.
2. For real input, check that every kept non-DC coefficient has the conjugate partner the inverse needs; prefer a real-spectrum API such as rfft/irfft when that is the requirement.
3. Add a passband test with a unit cosine, an explicit expected gain and a tolerance, then a stopband test for a bin that should be removed.
4. Test full retention with DC, cosine, sine and Nyquist content, and pass the original length to real inverse transforms so odd and even lengths are unambiguous.
5. For each r a caller uses, check the actual frequency cutoff after the representation change.
6. Keep cross-path consistency tests, but pair them with an independent check that would fail if every path shared the same wrong scale.

Sources

• Aeon, issue #3710: the reported symptom, a short reproducer, and the issue state.
• Aeon, PR #3711: authorship, review, merge time, and which files changed.
• Aeon, squash merge 490f9fa exact diff: the rfft/irfft repair and the added regression tests.
• Aeon, merge-pinned transformer source and merge-pinned tests.
• NumPy, Discrete Fourier Transform, rfft, and irfft: the Hermitian real-transform convention and why the inverse needs the length.