{"id":"6fc31e60-ed03-4827-bd0d-a42b11bff770","arxiv_id":"2607.16279","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"RA-FR uses blind face restoration, DINOv1 ViT embeddings, and conformal risk control to return adaptive-size match sets with a user-specified miss-rate bound.","lead":"This paper builds a face-search system that returns a longer list of candidate matches when the query is uncertain, promising to keep the chance of missing the true match below a user-set limit. It combines image restoration, self-supervised transformer features, and conformal risk control to make surveillance face retrieval more auditable.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed (α,δ) guarantee is only shown for a matched calibration/deployment distribution; SCFace is presented as OOD without describing the calibration source, while §5 defers OOD handling.","rationale":"The reader's weakest assumption correctly identifies the OOD/domain-shift issue. My own reading confirms that the central statistical guarantee is conditional on exchangeability, and the paper's exposition of the SCFace experiments is ambiguous about the calibration source. This is the most load-bearing concern because the paper's headline value proposition—'a step forward towards trustworthy risk-aware facial retrieval' for surveillance—hinges on the system working under the kinds of distribution shift embodied by SCFace. If the SCFace results were obtained by calibrating on SCFace, then the paper has not demonstrated any domain-shift robustness beyond what standard RCPS would give on any new test set; if the results were obtained with an IMFDB-calibrated κ, the claimed probabilistic guarantee is not valid. Either way, the manuscript as written overclaims. That said, this is an addressable limitation: a clear statement of calibration protocol, or a cross-domain recalibration method, would resolve it. The other issues raised by the reader (Hoeffding-grid correction, multi-match risk) are real but less central to the core claim. Thus I would keep the verdict at CONDITIONAL—the concern does not invalidate the in-distribution contribution, but it does require the authors to revise the claims and experiments before the OOD generalization can be accepted.","tokens_in":9717,"tokens_out":3692,"duration_ms":36465,"concrete_test":"In the released repository, identify the configuration for the SCFace experiments and the exact command that produced Fig. 7/Fig. 9 for SCFace. Then run two calibrations: (i) fix κ_IMFDB from IMFDB calibration (same δ) and apply it unchanged to the SCFace query/test set; (ii) calibrate κ_SCFace on a SCFace split. Compute the empirical miscoverage rate on SCFace test queries for both cases at α=0.05 and δ=0.1. If ρ̂(κ_IMFDB) > 0.05, the cross-domain guarantee is contradicted; if the reported SCFace numbers used (ii), the OOD claim in the abstract is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The formal guarantee in §3.1 (Eqs. 5–7, Algorithm 1) holds only when D_cal is i.i.d. (or exchangeable) with the deployment queries. The paper does not restrict the abstract's 'guarantees ground truth inclusion' to the matched case; it presents SCFace as an OOD/held-out test of surveillance robustness (§4.1, §4.4). Yet the text never states whether the κ used for SCFace was calibrated on IMFDB, on a SCFace split, or on something else. If it was calibrated on SCFace, the experiments do not test cross-domain transfer and the OOD language is unsupported. If it was calibrated on IMFDB and applied to SCFace, the Hoeffding-Upper-Confidence-Bound guarantee is invalid for SCFace because the exchangeability assumption is violated; the fact that the empirical curve lies in the green region (Fig. 7) is then not a certificate but an anecdote. §5 concedes the point: 'Future research should extend this into a two-stage framework with a test-time OOD detector to dynamically adjust κ under distributional shift.' That is an admission that the current pipeline does not provide the claimed protection under shift. This is load-bearing because the central promise of RA-FR is a principled replacement for fixed-top-k in unconstrained surveillance, not just a matched-distribution IMFDB result.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"RA-FR combines blind face restoration (DiffBIR + InterLCM), DINOv1 ViT-B embeddings with GeM/GGeM pooling, and an RCPS-style calibration rule (Hoeffding upper confidence bound) to select a global scale κ that controls retrieval miscoverage at level α with confidence 1−δ. At inference, per-query set size is K(X,κ)=⌈κ·Φ[fu(X)]⌉, so retrieved sets adapt to estimated uncertainty. On IMFDB the authors report satisfying α=0.05 with average ~10 images; SCFace is used as a held-out surveillance/OOD test. The central claim is that RA-FR replaces fixed top-k by adaptively sized sets with a finite-sample risk guarantee.","tokens_in":10085,"tokens_out":6642,"duration_ms":65992,"significance":"The statistical tool is not new—it is the RCPS framework of Bates et al., adapted to retrieval as in RCIR. The paper's contribution is the engineering integration: restoration to reduce aleatoric uncertainty, DINOv1+GeM/GGeM features, and an empirical study of the resulting risk–efficiency trade-off. If the in-distribution calibration protocol is sound, the IMFDB result is a plausible and useful demonstration. However, the paper's headline deployment/OOD guarantee is not established by the experiments as reported; the SCFace protocol is underspecified and the formal guarantee is only for exchangeable calibration data. The paper is therefore promising but needs a major revision to make the claims match the evidence.","major_comments":[{"comment":"The SCFace evaluation is presented as a held-out/OOD test of surveillance robustness, but the manuscript never states where the κ used for SCFace was calibrated. If SCFace queries are not exchangeable with the calibration set, the guarantee of §3.1 does not apply; if κ was calibrated on an SCFace split, the results are not a domain-shift test. Please state the calibration protocol explicitly and either provide a shift-robust calibration method or restrict the guarantee claim to the matched-distribution case. As written, the abstract's 'guarantees ground truth inclusion' is not supported for SCFace, and §5's deferral of an OOD detector confirms this is a known limitation.","section":"§4.1, §4.4, Fig. 7/9"},{"comment":"The text derives a pointwise Hoeffding UCB and then selects κ̂ by searching a grid. A pointwise bound does not automatically control the data-dependent selected κ̂; the guarantee 'ρ(κ̂)≤α with probability 1−δ' requires a uniform or selection-adjusted argument. If the authors intend to rely on Bates et al.'s RCPS theorem, they should state it and verify its conditions (monotonicity, grid, δ definition); otherwise a union bound over the grid is needed. This is load-bearing because the central claim is the finite-sample guarantee.","section":"§3.1, Eqs. (5)–(7), Algorithm 1"},{"comment":"The paper says the empirical risk lies in the green region, but Algorithm 1 selects κ̂ with UCB(κ̂)≤α. If the risk in Fig. 7 is computed on the same calibration set, the green region is guaranteed by construction and carries no evidential weight. If it is a held-out test set, the protocol (number of calibration samples, grid size, δ, repeated splits) must be reported to assess the claim. Please clarify which curve is plotted and add standard errors over repeated calibration/test splits.","section":"§4.4, Fig. 7"}],"minor_comments":[{"comment":"The uncertainty score Φ[fu(X)] should be defined and bounded away from zero; if zero, K=0 gives an empty set and certain miscoverage.","section":"§3.1, Eq. (4)"},{"comment":"There is a typo 'thr risk control calibration process'; also specify what the shaded regions denote and whether the ECE is computed on the calibration or test split.","section":"§4.3, Fig. 6"},{"comment":"The IMFDB paragraph contains a duplicated phrase 'and significant temporal changes'; please remove.","section":"§4.1"},{"comment":"The grid K and its resolution are not defined. If a union bound is added, the relation between M (number of candidates), n, and δ must be given.","section":"Algorithm 1"},{"comment":"The text gives approximate values ('approximately 10 or fewer', 'roughly 70% larger') but no exact numbers or error bars; please report quantitative results in a table or in the text.","section":"§4.5, Fig. 9"},{"comment":"The RCIR reference [4] is cited as 'Proc. AAAI (2025)'; please provide full publication details or a DOI if available.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the paper's contribution is incremental but legitimate if the claims are scaled back. The main risk is that the OOD narrative is not supported; a reviewer should require the authors to either provide a proper domain-shift protocol or explicitly state that SCFace is an exploratory robustness check without formal guarantees. I also note the code link is provided but was not verified; the experiments lack split sizes and random-seed details."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: this is a solid application of RCIR to face retrieval, not a new method. The genuinely new parts are the specific pipeline (DiffBIR+InterLCM restoration, DINOv1 ViT-B with GGeM pooling) and the empirical demonstration that it meets α=0.05 on IMFDB with an average retrieval set of ~10 images. The statistical guarantee is inherited from Bates et al. and Cai et al., and it is applied correctly in the matched-distribution setting.\n\nThe paper is honest about building on RCIR and gives a clear account of the calibration/adaptation architecture. The ablations are useful: ViT+GGeM consistently beats the ResNet baseline, and the restoration experiments on SCFace show that aggressive cropping of low-res faces hurts performance, a nice practical nuance. Code is provided.\n\nThe main soft spot is the OOD story. The abstract and several passing claims say the system guarantees ground-truth inclusion in unconstrained surveillance settings, but the formal guarantee in §3.1 only holds when D_cal is i.i.d. with the deployment queries. SCFace is presented as a held-out OOD test, yet the paper never states whether the κ used for SCFace was calibrated on IMFDB, on a SCFace split, or on both. That matters. If it was calibrated on SCFace, the experiment doesn't test transfer; if it was calibrated on IMFDB, the empirical green region on SCFace is an anecdote, not a certificate. §5 defers OOD detection to future work, which is effectively an admission that the current pipeline doesn't control risk under shift. This is a real overclaim and should be fixed in revision.\n\nTwo smaller technical points. Algorithm 1 searches a grid of κ and takes the first value whose Hoeffding UCB is below α, but without a union bound or grid correction the bound doesn't hold uniformly over the grid; that's addressable but makes the stated (α,δ) guarantee slightly sloppy. Also, the risk definition uses a single Y; for faces with multiple true matches in the database the loss is ambiguous. Neither is fatal.\n\nWho this is for: people working on uncertainty-quantified retrieval or face surveillance who want a concrete demonstration that better features plus restoration shrink conformal sets at fixed risk. It deserves a serious referee — the empirical contribution is solid and the OOD gap is fixable — but I'd want the SCFace calibration protocol clarified and the grid correction acknowledged before publication. Recommend: send to peer review, with revision.","headline":"A competent applied extension of RCIR to face retrieval, with a real gap between the OOD claims and the actual calibration protocol on SCFace.","tokens_in":10563,"tokens_out":2723,"would_cite":false,"duration_ms":28854,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that facial retrieval in degraded, high-stakes settings can replace fixed top-k lists with per-query adaptive set sizes that come with a formal guarantee: the true match is included with probability at least 1−α, at confid","keywords":["facial retrieval","risk control","conformal prediction","uncertainty quantification","adaptive retrieval set","blind face restoration","vision transformer","surveillance"],"falsifier":"Calibrate κ on the IMFDB training/calibration split at α=0.05 and δ=0.1, then run the exact same adapter on a held-out SCFace probe set without recalibration. If the empirical fraction of queries whose true match is missing exceeds 0.05 (or breaches the stated confidence margin), the deployment-time guarantee as stated does not hold under shift. The paper does not report this cross-calibration number; reporting it would settle whether the guarantee transfers.","tokens_in":9592,"feed_emoji":"🔍","tokens_out":3335,"duration_ms":31300,"temperature":0.7,"pith_summary":"RA-FR claims that facial retrieval in degraded, high-stakes settings can replace fixed top-k lists with retrieval sets whose size adapts per query, and that this set carries a formal guarantee: with user-chosen risk α and confidence 1−δ, the true match is included. The paper argues the guarantee holds because risk is controlled by calibrating a single global scale against a held-out set using Hoeffding's inequality, then scaling each query's set size by its estimated uncertainty. On IMFDB it reports meeting a 5% risk target with an average retrieval set of about 10 images, and reports that stronger embeddings plus blind restoration shrink sets further under domain shift.","feed_headline":"Facial retrieval gets provable risk bounds, not fixed top-k","feed_subtitle":"Adaptive set sizes keep the true match at 5% risk, averaging about 10 images on IMFDB.","key_machinery":"The risk-controlling threshold κ: a single positive scalar calibrated offline. For a query, the retrieval set size is K(X,κ)=⌈κ·Φ[f_u(X)]⌉, with Φ a normalizing map of the uncertainty head f_u. The empirical miscoverage risk ρ̂(κ) plus the Hoeffding upper-confidence term sqrt(ln(1/δ)/(2n)) is kept below α, and monotonicity of risk in κ lets the controller pick the tightest valid κ by grid search. This is what converts uncertainty scores into a certified set size.","core_discovery":"The central claim is that a risk-aware controller can certify retrieval coverage at deployment time: after calibrating one parameter κ on an i.i.d. calibration set, the online adapter returns for each query X the top ⌈κ·Φ[f_u(X)]⌉ matches, where Φ maps an uncertainty score to a scale. Because the loss—missing the true match—is monotone decreasing in set size, the paper uses Hoeffding's inequality to bound finite-sample risk and picks the smallest κ whose upper confidence bound stays below α. Deploying this κ guarantees ρ(κ) ≤ α with probability 1−δ, turning retrieval into an auditable decision rule. The authors further claim that combining blind restoration (DiffBIR plus an LCM refinement) w","pith_inferences":["Implicit in the paper: the guarantee is marginal over the calibration distribution, not conditional on each query; per-query reliability would need a stronger notion such as conditional coverage.","The reported SCFace results are presented as out-of-distribution evidence, but the calibration procedure described uses a held-out set; if SCFace is used only as test data without recalibration, the empirical risk on SCFace is informative but is not covered by the Hoeffding guarantee. A direct test would be to calibrate on IMFDB and measure miscoverage on SCFace.","A natural extension is to feed the estimated uncertainty back into the restoration step or to gate retrieval with a lightweight OOD detector that enlarges κ when shift is detected; the paper names this as future work.","The same risk-control mechanism could be tested on other high-stakes retrieval domains, such as medical image search, where a miss is costly and set size can be interpreted as a second-opinion list."],"forward_implications":["Fixed top-k heuristics can be replaced by per-query set sizes with a stated worst-case risk of missing the true match.","A user can set α and δ up front; the system will report the implied average set size, making the reliability-efficiency trade-off explicit and auditable.","The guarantee is finite-sample and distribution-free with respect to the calibration set, so it does not rely on model assumptions about the feature space.","Improvements in embedding quality or input restoration translate directly into smaller certified sets at the same risk level.","The framework applies beyond faces to any retrieval task with a monotone miscoverage loss and an uncertainty signal."],"fun_headline_variants":["Risk-aware facial retrieval: adaptive sets, provable bounds","Guarantee the right face: adaptive retrieval with risk control","Facial retrieval with certified 5% risk, not fixed top-k","Dynamic retrieval sizes that keep miss risk under control","RA-FR: retrieval that sizes itself to your risk limit"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The guarantee depends on the calibration set being drawn from the same distribution as the queries seen at deployment; if the deployment distribution shifts (e.g., new cameras, resolutions, or demographics), the calibrated κ may no longer keep the true-match risk below α.","fun_headline_variants_meta":{"raw":{"variants":["Risk-aware facial retrieval: adaptive sets, provable bounds","Guarantee the right face: adaptive retrieval with risk control","Facial retrieval with certified 5% risk, not fixed top-k","Dynamic retrieval sizes that keep miss risk under control","RA-FR: retrieval that sizes itself to your risk limit"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000345,"raw_usage":{"total_tokens":1770,"prompt_tokens":824,"completion_tokens":946,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":568,"completion_tokens_details":{"reasoning_tokens":876}},"tokens_in":568,"tokens_out":946,"duration_ms":8641,"temperature":1.0,"reasoning_tokens":876,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T07:50:02.157786+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Calibrate κ on the IMFDB training/calibration split at α=0.05 and δ=0.1, then run the exact same adapter on a held-out SCFace probe set without recalibration. If the empirical fraction of queries whose true match is missing exceeds 0.05 (or breaches the stated confidence margin), the deployment-time guarantee as stated does not hold under shift. The paper does not report this cross-calibration number; reporting it would settle whether the guarantee transfers.","supporting_citations":[],"review_version":1}