{"id":"165b79c8-d743-4757-add7-81df7f82f755","arxiv_id":"2605.13214","paper_version":2,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"Backdoors can be embedded in ResNet and ViT models as statistically indistinguishable latent directions, reducing cryptographic undetectability to an intractable hypothesis test over parameter distributions.","lead":"This paper constructs backdoor attacks on modern neural networks like ResNet and Vision Transformers by repurposing existing latent directions in the model rather than adding new structure. A smart generalist might read it to understand emerging risks where AI models could contain hidden behaviors that blend into normal training and resist standard detection.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"Undetectability reduces to an unproven conjecture that no efficient test distinguishes the two parameter distributions","rationale":"The reader's weakest assumption correctly isolates the conjecture as the load-bearing step. The empirical results on ResNet and ViT supply practical evidence that the backdoor survives the listed defenses, but the cryptographic claim requires computational hardness that is only conjectured. Because the reduction itself is shown and the construction is concrete, the appropriate adjustment is CONDITIONAL rather than outright rejection.","tokens_in":1762,"tokens_out":363,"duration_ms":12626,"concrete_test":"Using the released model weights, train a simple meta-classifier (logistic regression or small MLP) on a feature vector consisting of the top-100 singular values of each convolutional or attention weight matrix plus the first four moments of the activation histograms on a held-out clean validation set; evaluate its accuracy at distinguishing the backdoored ResNet/ViT checkpoints from an equal number of independently trained clean checkpoints. If accuracy exceeds 70 % on a 50/50 test split, the conjecture is falsified for these architectures.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim reframes cryptographic undetectability as a hypothesis test between the distribution of clean-model parameters and backdoored-model parameters, then asserts that this test is intractable in practice. No reduction to a known hard problem (e.g., distinguishing pseudorandom from random or solving a lattice problem) is supplied, nor is a formal argument given that the latent-direction construction forces the two distributions to be computationally indistinguishable. The reported experiments demonstrate resistance only to a finite list of post-training defenses; they do not rule out the existence of an efficient statistical or learning-based distinguisher that was not among the tested suite.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper claims that backdoors can be embedded into modern neural networks (ResNet, Vision Transformer) trained on standard image datasets by treating them as latent directions already present in the learned parameter geometry. It reframes cryptographic undetectability as the intractability of distinguishing the parameter distribution of a clean model from that of a backdoored model via hypothesis testing, conjecturing this test to be intractable in practice. The construction is shown to achieve high attack success rates with negligible clean accuracy loss and to resist a suite of post-training defenses.","tokens_in":1927,"tokens_out":485,"duration_ms":18437,"significance":"If the intractability conjecture is substantiated, the result would be significant for showing that cryptographic-style undetectability need not rely on stylized architectures but can arise from the geometry of representations learned by practical end-to-end models. The empirical demonstration on ResNet and ViT provides concrete evidence of attack viability against existing defenses.","major_comments":[{"comment":"Abstract: the central claim that undetectability 'reduces to a hypothesis test between two unknown distributions over model parameters, which we conjecture to be intractable in practice' supplies the intractability needed for the cryptographic guarantee but provides neither a reduction to a known hard problem nor a formal argument that the latent-direction construction forces computational indistinguishability; this is load-bearing for the main result.","section":"Abstract"},{"comment":"Section 4 (construction) and Section 5 (experiments): resistance is demonstrated only against a finite list of post-training defenses; the manuscript does not argue why this list rules out the existence of an efficient statistical or learning-based distinguisher outside the tested suite, leaving the conjecture without direct empirical grounding.","section":"Section 4, Section 5"}],"minor_comments":[{"comment":"Notation for the two parameter distributions (clean vs. backdoored) is introduced without an explicit definition or reference to how they are sampled in the hypothesis test.","section":"Section 3"},{"comment":"Table 2 (attack success rates) lacks error bars or details on the number of random seeds used for the reported means.","section":"Table 2"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the thoughtful and constructive report. The comments correctly identify that our central claims rest on a conjecture of intractability rather than a formal reduction, and that the empirical evaluation covers a finite set of defenses. We address each point below and indicate the revisions we will make.","responses":[{"response":"We agree that the intractability is presented as a conjecture rather than derived from a reduction to a known hard problem. The manuscript explicitly uses the word 'conjecture' and motivates the claim by showing that backdoor directions align with the geometry of naturally learned latent directions in ResNet and ViT parameter spaces. No formal proof of computational indistinguishability is provided. We will revise the abstract and the opening of Section 1 to state more explicitly that the cryptographic-style guarantee rests on this unproven but empirically supported conjecture, and we will add a short paragraph in the discussion section outlining why a reduction appears difficult given the data-dependent nature of the learned representations.","revision_made":"partial","referee_comment":"[Abstract] Abstract: the central claim that undetectability 'reduces to a hypothesis test between two unknown distributions over model parameters, which we conjecture to be intractable in practice' supplies the intractability needed for the cryptographic guarantee but provides neither a reduction to a known hard problem nor a formal argument that the latent-direction construction forces computational indistinguishability; this is load-bearing for the main result."},{"response":"The observation is accurate: the experiments evaluate a concrete but finite collection of post-training defenses and statistical tests. We do not claim that passing these tests constitutes a proof that no efficient distinguisher exists. The conjecture draws its support from the construction itself—embedding the backdoor as a direction already present in the clean model's latent geometry—together with the consistent failure of the tested methods to isolate it without destroying utility. We will expand Section 5 with an explicit discussion acknowledging the limitation of any finite test suite and clarifying that the results are consistent with, but do not prove, the intractability conjecture. If space allows, we will also report results from two additional learning-based probes (e.g., membership-inference-style classifiers on parameter subsets) to further illustrate the difficulty.","revision_made":"yes","referee_comment":"[Section 4, Section 5] Section 4 (construction) and Section 5 (experiments): resistance is demonstrated only against a finite list of post-training defenses; the manuscript does not argue why this list rules out the existence of an efficient statistical or learning-based distinguisher outside the tested suite, leaving the conjecture without direct empirical grounding."}],"tokens_in":1394,"tokens_out":560,"duration_ms":18560,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main takeaway is that this work moves cryptographic-style backdoor undetectability from stylized networks to practical ResNet and Vision Transformer models by treating the backdoor as a learned latent direction rather than an added artifact. That framing and the empirical demonstration are the actual new elements.\n\nThey show the attack reaches high success rates on standard image datasets while keeping clean accuracy nearly intact, and it survives a range of post-training defenses without making the model unusable. The experiments appear to be the solid part of the paper; the construction is concrete enough to test on real architectures.\n\nThe soft spot is the cryptographic claim. Undetectability is reduced to a hypothesis test between the parameter distributions of clean and backdoored models, then declared intractable by conjecture. No reduction to a known hard problem is given, and the experiments only rule out the specific defenses they tried. That leaves open whether some other efficient distinguisher exists. The circularity the stress-test note flags is real: the main result depends on the conjecture supplying the intractability.\n\nThis is for readers working on model verification and AI security who need to know whether backdoors can hide in the geometry of ordinary trained networks. The empirical results give them something concrete to consider even if the formal guarantee stays conjectural.\n\nI would send it to peer review. The extension to modern models is worth referee scrutiny, and the authors should be asked to either strengthen the intractability argument or clarify exactly what the experiments establish versus what remains open.","headline":"The paper extends backdoor attacks to ResNet and ViT via latent directions but rests undetectability on an unproven conjecture about an intractable hypothesis test.","tokens_in":2413,"tokens_out":378,"would_cite":false,"duration_ms":19526,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Modern neural networks can contain backdoors hidden as learned latent directions that are cryptographically undetectable.","keywords":["backdoor attacks","neural network security","latent space","cryptographic undetectability","hypothesis testing","machine learning security","ResNet","Vision Transformer"],"falsifier":"Discovery of an efficient algorithm that can reliably distinguish a backdoored model from a clean model by inspecting its parameters or behavior would falsify the claim of cryptographic undetectability.","tokens_in":2669,"feed_emoji":"🔐","tokens_out":554,"duration_ms":33501,"temperature":0.7,"pith_summary":"The paper constructs a backdoor attack for modern architectures like ResNet and Vision Transformers by framing backdoor channels as learned latent directions rather than artificial additions. This reframing reduces the problem of undetectability to a hypothesis test between two unknown distributions over model parameters, which the authors conjecture is intractable in practice. If true, this means attackers can exploit the geometry the network already possesses instead of introducing foreign structure that might be detectable. The attack is shown to achieve high success rates with little impact on clean accuracy and to resist various post-training defenses.","feed_headline":"Backdoors hide as latent directions in modern neural nets","feed_subtitle":"Attack reduces undetectability to an intractable test between parameter distributions, succeeding on ResNet and ViT while evading defenses.","key_machinery":"Backdoor channels identified as learned latent directions in the latent space of the neural network, which allows the backdoored model to be statistically close to a clean model.","core_discovery":"By identifying backdoor channels as learned latent directions, the question of undetectability reduces to a hypothesis test between two unknown distributions over model parameters, conjectured to be intractable, allowing the attack to succeed on state-of-the-art architectures trained on standard datasets while resisting defenses.","pith_inferences":["If the hypothesis test is indeed intractable, then existing detection methods based on statistical anomalies may not suffice for these backdoors.","This suggests that the latent space geometry learned during standard training can be repurposed for malicious use without additional modifications.","Future work could explore whether training regularizations can prevent such latent directions from being exploitable.","Similar approaches might apply to other domains like language models where latent directions are studied."],"forward_implications":["The attack works on ResNet and Vision Transformer architectures with high success rates and negligible clean accuracy degradation.","The backdoor resists a comprehensive suite of post-training defences without making the model unusable.","Cryptographic backdoors can be inherent properties of the geometry of learned representations rather than requiring exotic architectures.","Undetectability holds if no efficient algorithm can distinguish the backdoored model from a clean one based on parameter distributions."],"fun_headline_variants":["Latent directions hide backdoors in ResNet and ViT","Backdoors evade detection as natural latent directions","Intractable tests allow backdoors in modern neural nets","Neural net geometry conceals cryptographic backdoors"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The hypothesis test distinguishing the parameter distributions of backdoored and clean models is intractable in practice.","fun_headline_variants_meta":{"raw":{"variants":["Latent directions hide backdoors in ResNet and ViT","Backdoors evade detection as natural latent directions","Intractable tests allow backdoors in modern neural nets","Neural net geometry conceals cryptographic backdoors"]},"model":"grok-4.3","cost_usd":0.008134,"raw_usage":{"total_tokens":3693,"prompt_tokens":664,"num_sources_used":0,"completion_tokens":59,"cost_in_usd_ticks":81337000,"prompt_tokens_details":{"text_tokens":664,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2970,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":664,"tokens_out":59,"duration_ms":23419,"temperature":1.0,"reasoning_tokens":2970,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-30T21:53:01.958438+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Discovery of an efficient algorithm that can reliably distinguish a backdoored model from a clean model by inspecting its parameters or behavior would falsify the claim of cryptographic undetectability.","supporting_citations":[],"review_version":2}