{"id":"5411af85-c350-4cec-bc26-bf4868d1d7b5","arxiv_id":"2505.04468","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"FFTKF applies an FFT low-pass mask and a Kalman filter to privatized gradients, reporting modest accuracy gains over DP-SGD and DiSK on several image benchmarks, but with incomplete privacy accounting.","lead":"This paper combines a frequency-domain filter with a Kalman filter to clean up the noise that differential privacy adds to machine learning gradients. The method reports small accuracy gains, but the paper does not account for the extra private information used by its filter.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Missing privacy accounting for the finite-difference prediction step (Eq. 22) invalidates the claim that FFTKF preserves DP-SGD/DiSK's (ε,δ) at the same budget.","rationale":"The reader's weakest_assumption is exactly the point that breaks the central claim. I verified the mechanism: in each iteration the released update direction ̃g_t is not merely a post-processing of the DP-SGD output; it also incorporates ̃g_{t|t-1} from Eq. (22), which is computed from two additional clipped gradient evaluations, one at x_t+γd_{t-1}. Because d_{t-1} depends on previous private gradients, this is an adaptive data-dependent query whose privacy cost must be composed with the DP-SGD step. The Sec. 3.5 'post-processing property' argument covers only the FFT map applied to g_t, not these extra queries. No moments accountant or RDP analysis is supplied for them, so the claimed equivalence with DP-SGD/DiSK at the same (ε,δ) is unsupported. This alone justifies the REJECT verdict. A secondary issue is that Theorem 2's bound contains the target Σ_t E||∇F(x_t)||² on both sides; without rearrangement and a check on the constants it does not establish the advertised convergence result. Empirically, gains over DiSK are often under 1 percentage point and no error bars, seeds, or exact hyperparameters are reported, but the privacy-accounting gap is the decisive flaw. If the authors complete the accounting and show that the composed ε is still at most the reported value at the stated noise scales, and if Theorem 2 is repaired, the underlying idea could merit a conditional acceptance; as written, the central privacy and utility claims are not established.","tokens_in":9711,"tokens_out":3881,"duration_ms":41387,"concrete_test":"Implement an RDP or moments accountant for Algorithm 1 that treats each iteration as containing three privatized queries: the standard clipped-gradient sum at x_t with noise σ_w, and the two finite-difference terms in Eq. (22) at x_t+γd_{t-1} and x_t with noise σ_fd. Derive the per-query ℓ2 sensitivity (accounting for the 1/B and 1/γ scalings) and compose over all T iterations at the hyperparameters used for the ε=4 results in Figure 3/Table 3. If the composed ε exceeds 4, or if σ_fd is not specified in the experiments, the claimed privacy equivalence is void and the reported accuracies do not correspond to a valid (4,δ)-DP guarantee.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's privacy argument in Sec. 3.5 relies on post-processing: the FFT map P(g_t)=F^{-1}ΦFg_t is applied after the clipped Gaussian mechanism, so it cannot increase privacy loss. That argument covers only the observation g_t. It does not cover the finite-difference prediction step in Eq. (22), which evaluates and clips gradients at x_t+γd_{t-1} and x_t, adds noise w_fd, and whose output is mixed into the final update ̃g_t via Eq. (C). These are additional data-dependent queries, and d_{t-1} itself depends on prior private gradients, so the queried points are adaptively chosen. No RDP or moments-accountant analysis for these extra queries is provided; the finite-difference term has sensitivity governed by C/(Bγ), not by the DP-SGD noise scale σ_w. The statement that FFTKF inherits 'exactly that of DPSGD and DISK' is therefore unsupported, and the reported ε=4 at the stated noise scales may be an undercount. A secondary issue is that Theorem 2's displayed bound has Σ_t E||∇F(x_t)||² on both sides, making the convergence claim circular as written, but the missing privacy accounting is the more direct threat to the paper's central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FFTKF, a differentially private optimizer that applies an FFT-based spectral mask to privatized gradients and combines this with a scalar-gain Kalman filter that includes a finite-difference prediction step. The authors claim that FFTKF preserves the (ε,δ)-DP guarantee of the base DP-SGD/DiSK optimizer while achieving higher test accuracy on MNIST, CIFAR-10, CIFAR-100, and Tiny-ImageNet across CNNs, Wide ResNets, and Vision Transformers, with per-iteration complexity O(d log d). A theoretical analysis is presented through Lemma 1 and Theorem 2, and experiments compare FFTKF against DPAdam and DiSK.","tokens_in":9997,"tokens_out":5123,"duration_ms":50650,"significance":"If the central claim were established, FFTKF would be a practically valuable contribution: it is a drop-in private optimizer with a plausible O(d log d) complexity, and the reported accuracy gains over DiSK are consistent across many architectures and datasets. The FFT post-processing step itself is correctly identified as privacy-preserving, and the broad experimental scope is a strength. However, the paper's privacy guarantee for the full algorithm is not established: the finite-difference prediction step makes additional data-dependent gradient queries that are not covered by the post-processing argument, and no moments-accountant or RDP analysis is given for them. The theoretical convergence bound is also circular as stated. These issues undermine the paper's main claims.","major_comments":[{"comment":"The finite-difference prediction step in Eq. (22) evaluates and clips gradients at x_t + γ d_{t-1} and x_t, with additional noise w^fd_t. These are fresh data-dependent queries; d_{t-1} itself depends on previous private gradients, so the queried points are adaptively chosen. The privacy argument in Sec. 3.5 and Proposition 1 covers only the post-processing map P(g_t) = F^{-1}Φ_ρ F g_t applied to the clipped Gaussian observation. It does not cover the two additional clipped gradient evaluations in Eq. (22). The claim that Algorithm 1 \"inherits exactly the same (ε,δ) guarantee as standard DP-SGD/DiSK\" is therefore unsupported. A valid analysis would need a moments accountant or RDP composition that includes, at each iteration, the DP-SGD query plus the finite-difference query with sensitivity proportional to C/γ and noise scale σ_fd. Without this, the reported ε=4 is an undercount and the privacy-utility comparisons are not valid.","section":null},{"comment":"The displayed bound in Theorem 2 has Σ_t E‖∇F(x_t)‖² on the left-hand side and a term ρ² G_T on the right-hand side, where G_T is defined as the same average (1/T) Σ_t E‖∇F(x_t)‖². As written, the inequality is circular and does not establish convergence. The statement needs to be reformulated, for example by moving the G_T term to the left-hand side under a condition such as C_1 > ρ², or by presenting a recursion that yields a non-circular final bound. This is a load-bearing issue because Theorem 2 is the paper's main theoretical privacy-utility result.","section":null},{"comment":"Theorem 2 refers to \"Assumptions A1–A3\" and to \"the same (η,κ,γ) schedule as in Algorithm 1,\" but the assumptions A1–A3 are never stated in the manuscript; they need to be given explicitly or precisely referenced to [32]. In addition, the proof of Lemma 1 says it follows from the post-processing theorem of differential privacy, but Lemma 1 is an algebraic statement about the mean and covariance of the filtered gradient; the post-processing theorem is irrelevant to that computation. The proof should instead directly use unitary invariance of the Fourier transform.","section":null},{"comment":"The experimental section reports accuracy at ε=4 without specifying the exact batch sizes, number of epochs, learning rates, noise scales σ_w and σ_fd, or the moments-accountant parameters used to compute ε. The text says these were \"experimentally adjusted\" to achieve optimal performance, which is not reproducible and, given the missing privacy accounting for Eq. (22), prevents the reader from verifying that the reported accuracies are actually attained under the claimed (ε,δ) budget. Full hyperparameter tables and a corrected privacy computation are needed before the empirical claims can be assessed.","section":null}],"minor_comments":[{"comment":"In Eq. (21) the first sum is divided by B (1/B Σ clip) but the second sum in Eq. (22) is not divided by B; this is inconsistent and should be fixed to make the batch-averaging convention explicit.","section":null},{"comment":"The text refers to \"Lemma 4\" at the end of Section 3.5 (\"Lemma 4 replaces the isotropic noise term...\") although only Lemma 1 is stated in Section 4; the numbering is inconsistent and the references should be harmonized.","section":null},{"comment":"There is a typo in the sentence \"the termFFTKF-is employed to denote...\" — the stray hyphen should be removed.","section":null},{"comment":"Reference [6], \"John Doe and Jane Smith,\" appears to be a placeholder rather than a real citation; it should be replaced with the actual source or removed.","section":null},{"comment":"The claim of \"consistent\" improvement over DiSK is not fully supported by the reported numbers; for example, on Tiny-ImageNet with CNN5 the accuracy drops from 23.45 (DISK) to 22.62 (FFTKF), and on CIFAR-100 with ViT-small the gain is only 0.02 percentage points. The text should qualify the consistency claim.","section":null}],"recommendation":"reject","confidential_remarks":"The missing privacy accounting for the finite-difference prediction step is not a local patch: it invalidates the central claim that FFTKF achieves the same (ε,δ) as DP-SGD/DiSK, and the reported accuracies at ε=4 would need to be recomputed under a correct composition analysis. The circular Theorem 2 and the unstated assumptions compound the problem. I see no way for the current manuscript to be accepted without a substantially revised privacy analysis and re-run experiments, which is beyond a minor or even a major revision in the usual sense. The placeholder reference [6] also suggests the manuscript is not yet in a polished state for journal review."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things before you read it. The actual new idea—an FFT mask over the DP noise followed by a DiSK-style scalar-gain Kalman update—is a reasonable thing to try, and the post-processing argument for the FFT step by itself is correct. But the central privacy claim as written fails: Eq. (22), the finite-difference prediction, evaluates and clips gradients at x_t + γ d_{t-1} and x_t with additional noise w_fd, and that part of the update is never included in the accounting. The post-processing argument in Sec. 3.5 covers only the FFT map on the observation. Those are extra data-dependent queries at adaptively chosen points because d_{t-1} comes from prior private gradients. Without an RDP or moments-accountant bound for them, the statement that FFTKF inherits \"exactly\" the DP-SGD/DiSK budget is unsupported, and the reported ε=4 may be an undercount.\n\nWhat the paper does well: it is straightforward about what it combines, the O(d log d) complexity is real, and the experiments are broad: five architectures, four datasets, against DPAdam and DiSK. The gains are small—mostly under two points and once negative (CNN5 on Tiny-ImageNet)—but the direction is consistent on CIFAR. The mask parameters are fixed a priori, so there is no per-dataset tuning of λ and ρ, which is honest.\n\nThe soft spots beyond the privacy gap: Theorem 2 as displayed has the quantity to be bounded, E||∇F(x_t)||², on both sides through the ρ² G_T term, so it is not a usable convergence bound as written. The empirical section reports no seeds, no error bars, and no code, so the headline numbers are hard to verify. The reference list contains a placeholder-looking entry [6] (\"John Doe and Jane Smith\") that should be checked. All of this is fixable in principle, but the privacy accounting is the necessary fix.\n\nWho this is for: anyone working on DP optimizers will find the combination plausible and the empirical comparison worth a look, but the paper needs major revision before its guarantees can be trusted. I would not desk-reject it: the idea is cheap to evaluate, the flaw is well-scoped, and a referee could extract a useful revised submission. But the current version's central guarantee is not established.","headline":"A plausible combination of FFT noise shaping and Kalman filtering, but the central privacy claim is unsupported because the finite-difference prediction step’s extra gradient queries are never accounted for.","tokens_in":10579,"tokens_out":3067,"would_cite":false,"duration_ms":30139,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FFTKF reshapes DP noise toward high frequencies and reports higher test accuracy than DP-SGD and DiSK at the same privacy budget.","keywords":["differential privacy","DP-SGD","Kalman filter","Fast Fourier Transform","noise shaping","private optimization","gradient denoising","frequency-domain filtering"],"falsifier":"Run a complete moments accountant that treats the two clipped finite-difference gradient evaluations in Eq. (22) as separate data-dependent queries per iteration, and check whether the reported $(\\varepsilon,\\delta)$ still holds; if the budget is exceeded, the same-budget privacy claim is void. A cheaper side check is to disable the finite-difference prediction term and measure how much of the reported accuracy gain disappears, revealing how much of the improvement rides on those extra queries.","tokens_in":9451,"feed_emoji":"📈","tokens_out":10617,"duration_ms":90201,"temperature":0.7,"pith_summary":"FFTKF is proposed as a drop-in differentially private optimizer that applies a frequency-domain mask to privatized gradients and then a scalar-gain Kalman filter. The paper's central claim is that this reshaping preserves the same $(\\varepsilon,\\delta)$ guarantee as DP-SGD and DiSK, because the FFT is unitary and the mask is a data-independent post-processing step, while it pushes privacy noise into high-frequency components that contribute least to learning. The payoff would be consistent test-accuracy gains on MNIST, CIFAR-10, CIFAR-100, and Tiny-ImageNet across CNNs, Wide ResNets, and Vision Transformers, at a per-iteration cost of $O(d\\log d)$. A theoretical bound asserts a tighter privacy-utility trade-off through reduced noise variance and controlled bias.","feed_headline":"FFT shifts DP noise to high frequencies to save accuracy","feed_subtitle":"A spectral mask plus Kalman filter keeps low-frequency gradients clean, beating DP-SGD and DiSK on four benchmarks.","key_machinery":"The load-bearing object is the linear spectral filter $G_\\rho=\\mathcal{F}^{-1}\\Phi_\\rho\\mathcal{F}$, which multiplies each Fourier coefficient of the privatized gradient by $1$ below a pivot index $k_0=\\lfloor\\lambda d\\rfloor$ and by $1-\\rho e^{-\\alpha(k-k_0)}$ above it. This filter does not increase $\\ell_2$ sensitivity and is data-independent, so it can be applied after noise addition without changing the DP guarantee; its eigenvalue structure (eigenvalues $1$ repeated $k_0$ times and $1-\\rho$ repeated $d-k_0$ times) is what produces the claimed variance reduction $\\rho^\\star$ and the bias term in Theorem 2. The second component is a scalar-gain Kalman update $\\tilde{g}_t=(1-\\kappa)\\tilde{g}_{t-1}+\\kappa\\hat{g}_t$, where $\\hat{g}_t$ is the filtered observation and the prediction step uses a privatized finite-difference formula to approximate the Hessian action $H_t d_{t-1}$ from two extra clipped gradient evaluations. Together the two pieces give per-iteration complexity $O(d\\log d)$ plus $O(d)$ vector operations.","core_discovery":"The paper argues that private gradient noise need not be fought only in the time domain: by transforming a clipped, noised gradient with an FFT, attenuating high-frequency Fourier coefficients with a soft mask $\\Phi_\\rho$, and transforming back, the noise is concentrated in spectral regions that matter least for optimization, while the low-frequency gradient signal is preserved. Because the filter is unitary-norm-preserving and data-independent, the paper claims the exact $(\\varepsilon,\\delta)$ budget of DP-SGD/DiSK is inherited through post-processing. Lemma 1 quantifies the effect: the filtered noise covariance becomes $\\sigma_w^2 A^2$ with trace $\\rho^\\star d\\sigma_w^2$, where $\\rho^\\star=[k_0+(1-\\rho)^2(d-k_0)]/d$, and the signal is scaled by a matrix within distance $\\rho$ of the identity; Theorem 2 then bounds the average squared gradient norm with the noise term reduced and a bias term $\\rho^2 G_T$ added. Empirically, the paper reports that FFTKF beats DP-Adam and DiSK at $\\varepsilon=4$ on all tested datasets, with gains up to 1.6 percentage points on CIFAR-10 with WRN-40.","pith_inferences":["A consequence the paper leaves implicit is that a full privacy audit must count the two clipped finite-difference gradient evaluations per iteration in Eq. (22) as data-dependent queries; the post-processing argument covers only the FFT filtering of the observation, so the true privacy cost of the prediction step is an open question.","A testable extension would be to measure a model's gradient power spectrum before applying FFTKF: on tasks where signal energy concentrates at low frequencies the mask should help, and on tasks with flat spectra it should remove genuine signal and hurt, predicting dataset-dependent gains.","The same spectral-shaping idea could be layered onto other private optimizers that inject per-step Gaussian noise, since the post-processing argument is mechanism-agnostic; the main design question would be whether a temporal model cheaper than the finite-difference Kalman step can deliver similar denoising."],"forward_implications":["The method can be inserted into existing DP-SGD or DP-Adam training loops by adding one forward pass, two FFTs, and scalar updates per step, so the accuracy gains come at modest extra cost.","With the paper's fixed choices $\\lambda=1/2$ and $\\rho=0.5$, the DP-noise term in the convergence bound is cut by 37.5% while the added bias penalty is at most 25%, which is the quantitative sense in which the trade-off improves on DiSK.","Reported gains at $\\varepsilon=4$ hold across four datasets and five architectures, so the benefit is not restricted to one model family.","If the guarantee holds, FFTKF offers a way to spend the same privacy budget more efficiently, which matters most under tight budgets where plain DP-SGD collapses in accuracy."],"supporting_citations":[{"why":"Defines DP-SGD, the Gaussian mechanism, and the moments accountant whose privacy budget FFTKF claims to inherit.","marker":"[1]"},{"why":"Supplies the scalar-gain Kalman simplification and the DiSK baseline whose convergence bound Theorem 2 extends.","marker":"[32]"},{"why":"Provides the post-processing theorem and Gaussian-mechanism guarantees used to argue that the spectral mask preserves DP.","marker":"[8]"},{"why":"Proposes low-pass filtering of privatized gradients, the spectral-denoising approach FFTKF builds on and compares with.","marker":"[33]"},{"why":"Gives the Kalman-filter-based DP optimization template whose analysis underpins the paper's convergence argument.","marker":"[20]"},{"why":"Establishes differentially private Kalman filtering, the theoretical basis for combining state estimation with DP noise.","marker":"[21]"},{"why":"Provides the DFT/FFT definitions and complexity bounds behind the claimed $O(d\\log d)$ per-iteration cost.","marker":"[4]"}],"fun_headline_variants":["FFT filters DP noise to high frequencies, boosting accuracy","Spectral DP: shift noise high, keep gradients low","FFTKF: Kalman-filtered gradients dodge DP noise","High-frequency DP noise, low-frequency signal: FFTKF wins","FFT shifts DP noise to high frequencies, saving signal"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported privacy guarantee assumes that the two extra clipped gradient evaluations in the finite-difference prediction step (Eq. 22) consume no additional privacy budget or are fully included in the moments accountant, yet the paper only proves post-processing for the FFT-filtered observation, not for these additional data-dependent queries.","fun_headline_variants_meta":{"raw":{"variants":["FFT filters DP noise to high frequencies, boosting accuracy","Spectral DP: shift noise high, keep gradients low","FFTKF: Kalman-filtered gradients dodge DP noise","High-frequency DP noise, low-frequency signal: FFTKF wins","FFT shifts DP noise to high frequencies, saving signal"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000208,"raw_usage":{"total_tokens":1427,"prompt_tokens":995,"completion_tokens":432,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":611,"completion_tokens_details":{"reasoning_tokens":347}},"tokens_in":611,"tokens_out":432,"duration_ms":4454,"temperature":1.0,"reasoning_tokens":347,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:27:49.516275+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a complete moments accountant that treats the two clipped finite-difference gradient evaluations in Eq. (22) as separate data-dependent queries per iteration, and check whether the reported $(\\varepsilon,\\delta)$ still holds; if the budget is exceeded, the same-budget privacy claim is void. A cheaper side check is to disable the finite-difference prediction term and measure how much of the reported accuracy gain disappears, revealing how much of the improvement rides on those extra queries.","supporting_citations":[{"cited_title":"Deep learning with differential privacy","cited_arxiv_id":null,"evidence_quote":"Defines DP-SGD, the Gaussian mechanism, and the moments accountant whose privacy budget FFTKF claims to inherit."},{"cited_title":"DiSK: Differentially private optimizer with simplified kalman filter for noise reduction","cited_arxiv_id":null,"evidence_quote":"Supplies the scalar-gain Kalman simplification and the DiSK baseline whose convergence bound Theorem 2 extends."},{"cited_title":"Now Publishers Inc., 2014","cited_arxiv_id":null,"evidence_quote":"Provides the post-processing theorem and Gaussian-mechanism guarantees used to argue that the spectral mask preserves DP."},{"cited_title":"DOPPLER: Differentially private optimizers with low-pass filter for privacy noise reduction","cited_arxiv_id":null,"evidence_quote":"Proposes low-pass filtering of privatized gradients, the spectral-denoising approach FFTKF builds on and compares with."},{"cited_title":"Kalman filter-based differential privacy federated learning method.Applied Sciences, 12(15):7787, 2022","cited_arxiv_id":null,"evidence_quote":"Gives the Kalman-filter-based DP optimization template whose analysis underpins the paper's convergence argument."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes differentially private Kalman filtering, the theoretical basis for combining state estimation with DP noise."},{"cited_title":"Briggs and Van Emden Henson.The DFT: An Owner’s Manual for the Discrete Fourier Transform","cited_arxiv_id":null,"evidence_quote":"Provides the DFT/FFT definitions and complexity bounds behind the claimed $O(d\\log d)$ per-iteration cost."}],"review_version":1}