{"id":"f6a3e630-6b10-4950-bb30-8d114f33af39","arxiv_id":"2412.00120","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"RAMLN combines a relation-aware quadruplet loss with a memory-based meta-learned margin to achieve state-of-the-art zero-shot sketch-based image retrieval.","lead":"A new training framework for sketch-based image retrieval uses a quadruplet loss with two negatives and a learned margin to help models recognize object categories they have never seen. If the gains hold, it would make sketch search practical for new categories without retraining.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The RAMLN margin is the paper's key proposed mechanism, but no meta-objective or update rule for it appears in the submission, leaving the claimed generalization from seen to unseen classes unestablished; this is grounds for rejecting the manuscript as submitted.","rationale":"I agree with the reader that the weakest assumption is the training signal for RAMLN's margin. The contribution statement makes the adaptive margin central; however, all of the margin's properties—its loss, its episodic/task split if any, and its update—are deferred. Algorithm 1 does not show a meta-gradient: the parameters of R(x) would be updated by whatever autograd flows through Eq. (10), unless a separate step is hidden, which is not in the text. A fixed-margin ablation directly isolates whether R(x) is responsible for the improvement. If a fixed margin does equally well, the headline SOTA gain cannot be attributed to the meta-learning. This does not disprove the empirical numbers, but it changes their interpretation from 'meta-learned adaptive margins generalize' to 'a quadruplet loss plus a large ViT backbone achieves these scores,' weakening the paper's novelty claim. Because the reader already framed the missing meta-objective as the core weakness, the verdict should remain REJECT (UNCHANGED) unless the supplementary is made available and the test is passed. I would not call the authors fraudulent; the issue is that the submitted document is incomplete at its central methodological juncture.","tokens_in":12627,"tokens_out":6896,"duration_ms":66910,"concrete_test":"Re-run the full RAMLN pipeline on the TU-Berlin unseen split, replacing the GRU/memory margin with fixed margin values 0.1, 0.2, 0.3, 0.4, and 0.5, holding the quadruplet loss, classification loss, PK sampling, backbone, and 20-epoch schedule fixed. If the best fixed margin reaches or exceeds the reported 0.649 mAP@all, then the learned margin is not necessary for the headline result and the meta-learning claim fails. Separately, if the supplementary's meta-objective cannot be located in the code or is not an actual meta-gradient update, the method should be treated as joint optimization, not meta-learning.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Equations (5) and (6) use R(x), the adaptive margin produced by a GRU plus external memory, but nothing in the main text specifies how R(x) is optimized. Algorithm 1 lists hyperparameters beta and phi but never defines or uses them; step 11 updates theta with L_cls + lambda*L_qua, and lambda is simultaneously the balancing weight inside Eq. (5). The paper defers all margin optimization to a supplementary document. This is load-bearing because RAMLN's novelty and the claimed SOTA gains depend on the learned margin generalizing to unseen classes. If the margin network is trained only jointly with the embedding on seen classes, no mechanism for zero-shot generalization has been demonstrated; and without a meta-objective, the method is indistinguishable from a jointly trained network that could memorize seen-class margins. Supporting defects (Eq. (3) has an incorrect negative-sample gradient; the text's 8%/17% gains are computed against different ZSE rows of Table 1) show that the technical details are not yet reliable enough to verify the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes RAMLN for zero-shot sketch-based image retrieval (ZS-SBIR). It introduces a relation-aware quadruplet loss that uses two negative samples from different modalities to reduce the sketch–photo modality gap, a hard-mining variant based on Eq. (4), and a GRU-plus-external-memory module that predicts the quadruplet margin R(x), which the paper calls meta-learning. The training objective combines the quadruplet loss with a softmax classification loss. Experiments on TU-Berlin Extended and Sketchy (NO and Extended) report strong gains over prior work, e.g., 0.649 mAP@all and 0.719 Prec@100 on TU-Berlin Extended and 0.695 mAP@200 on Sketchy-NO, and the paper includes ablations and qualitative visualizations.","tokens_in":12748,"tokens_out":12190,"duration_ms":97188,"significance":"If fully supported, the paper would make a useful contribution to ZS-SBIR: an adaptive-margin quadruplet loss is a plausible way to handle the large intra-class sketch/photo spread, and the reported numbers are competitive with recent ViT-based methods. The paper is also reasonably thorough in its ablations and includes qualitative t-SNE and retrieval examples. However, the central meta-learning mechanism is not actually specified in the main text, the gradient derivation in Eq. (3) is incorrect, and the reported SOTA comparisons are internally inconsistent. These issues are load-bearing because the paper's novelty and the claimed transfer to unseen classes rest on the meta-learned margin. Without a meta-objective, the method is indistinguishable from a jointly trained margin predictor, and the empirical claims cannot be verified with the current text. The absence of code and error bars further limits reproducibility.","major_comments":[{"comment":"The meta-learning objective for the margin network is not defined. Equations (5), (6), and (8) use R(x) and m_t, but no loss, meta-train/meta-validation split, or update rule for the margin predictor appears in the main text; Algorithm 1 only updates θ with ∇θ(L_cls + λL_qua). The paper defers to the supplementary materials for \"a detailed explanation of the margin optimization in meta-learning,\" but the core method must be self-contained. As written, the margin predictor is trained jointly on the seen classes, so the claim that it predicts optimal margins for unseen classes is unsupported; the method could be memorizing seen-class margins.","section":"Sec. 3.3, Algorithm 1"},{"comment":"The gradient with respect to the negative sample is algebraically wrong. For L_tri = ||xa − xp||^2 − ||xa − xn||^2, the correct expression is −∂L_tri/∂xn = −2(xa − xn), not −2(xn − xp). Because the paper uses this gradient discussion to motivate the second negative in the quadruplet loss, the mathematical motivation is invalid as written.","section":"Eq. (3)"},{"comment":"The claimed \"8.0% and 17% improvement in mAP over ZSE on the two datasets\" cannot be reproduced from Table 1. Against ZSE-Ret the TU-Berlin gain is 8.0 points but the Sketchy-NO gain is 19.1 points; against ZSE-RN the Sketchy-NO gain is 17.0 points but the TU-Berlin gain is 10.7 points. The sentence therefore mixes two different ZSE baselines. Similarly, the \"9.2% and 8.0%\" improvement over IVT matches only the TU-Berlin mAP@all and Sketchy-NO mAP@200 columns; on Sketchy Extended the gap is 0.7 points. State the comparison per baseline and per evaluation column.","section":"Table 1 and Sec. 4.2"},{"comment":"Hyperparameters β and φ are listed as inputs but never defined or used anywhere in the paper. The use of λ is also ambiguous: it balances the two negative terms inside Eqs. (5) and (6), but Algorithm 1 uses λ as the global weight on the entire quadruplet loss in the update rule, while Eq. (10) omits it entirely. These inconsistencies make the optimization objective non-reproducible.","section":"Algorithm 1, Eqs. (5), (6), (10)"},{"comment":"The memory-augmented margin predictor is underspecified. The read weights w_t^r(i), write weights w_t^w(i), keys k_t, and the memory update M_t(i) ← M_{t−1}(i) + w_t^w(i)k_t are not defined or normalized, and no formula connects m_t to R(x) beyond a verbal description. Since the memory is claimed to help unseen classes by recording uncommon seen-class information, these missing definitions block verification of the generalization mechanism.","section":"Sec. 3.3, Eq. (8)"}],"minor_comments":[{"comment":"The text states that in TU-Berlin Extended \"the number of photos is one-tenth that of sketches,\" but the set contains 204,070 photos and 20,000 sketches; the ratio is reversed.","section":"Sec. 4.1"},{"comment":"The phrase \"hard positive pairs (the closest negative sample and anchor)\" should read \"hard negative pairs\"; hard positives are the farthest positive pairs, as encoded by max in Eq. (4).","section":"Sec. 3.2"},{"comment":"The text says \"quadratic loss\" where \"quadruplet loss\" is meant, and \"for simplify\" should be \"for simplicity.\"","section":"Sec. 3.2"},{"comment":"The max/min in Eq. (4) do not specify the set of samples over which they are taken; define the batch-level search range.","section":"Eq. (4)"},{"comment":"The learning rate is denoted μ, which conflicts with the triplet margin μ in Eq. (1); use separate symbols.","section":"Algorithm 1 / Eq. (1)"},{"comment":"All results are single runs with no standard deviations; given that several differences are small (e.g., 0.008 in mAP@all between rows 3 and 5), it is hard to judge whether the margin component helps significantly.","section":"Table 2"}],"recommendation":"reject","confidential_remarks":"For the editor: the main technical contribution is not formalized in the text, and the experimental claims are internally inconsistent. If the authors have a supplementary containing the meta-objective, they should be asked to provide it in any future submission; unless the missing objective and the algebraic error are fixed, I cannot recommend publication. The paper is also not reproducible without code or seeds."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The new thing here is the specific combination: a relation-aware quadruplet loss with two negatives from different modalities, plus an adaptive margin produced by a GRU with external memory. That is a fresh configuration for ZS-SBIR, and the reported numbers on TU-Berlin Extended and Sketchy are strong if they hold. The ablation suggests each piece contributes, and the comparison table covers a wide range of prior methods. The idea of learning the margin instead of fixing it is sensible and worth building on.\n\nThe soft spots are not minor. The paper's key contribution is the meta-learned margin R(x), but Section 3.3 defers the entire training objective for it to a supplementary document. That is load-bearing: without knowing how R(x) is optimized, \"meta-learning\" is just a label. If the margin network is trained jointly with the embedding on the same seen classes using the same quadruplet loss, then it is an adaptive margin predictor, not meta-learning in any standard sense, and no mechanism for generalizing to unseen classes has been demonstrated. The main text must at least state the meta-objective and the update rule; pointing to supplementary is not enough for the central novelty.\n\nThere are also concrete technical errors. Equation (3) gives the gradient for the negative sample as -2(x_n - x_p); it should be -2(x_a - x_n). Algorithm 1 lists hyperparameters beta and phi that never appear again. And the main text claims 8% and 17% improvements over ZSE, but Table 1 does not support those numbers against either ZSE-RN or ZSE-Ret consistently; they look like selective comparisons. No code, seeds, or error bars are provided, so the empirical claim is not independently checkable.\n\nNone of this means the idea is dead. The loss formulation is reasonable, and a properly specified meta-training procedure could make the paper worthwhile. But as submitted, the central mechanism is opaque, and the supporting details are not reliable enough to verify the reported gains. I would send it to review because the topic is timely and the results are impressive on their face, but I would expect the reviewers to require the missing meta-objective, a corrected equation, and code or at least clear hyperparameter definitions before anything is accepted. My own verdict on the current version is reject, with a path to resubmission if the authors can show the margin is actually meta-learned and not just jointly optimized.\n\nFor a reader working on cross-modal retrieval or adaptive metric learning, this is a maybe for a reading group, and I would not cite it in its current form. But it deserves a serious referee rather than a desk reject.","headline":"A plausible new loss formulation with a genuinely unexplained meta-learning core: the adaptive margin has no training objective in the main text, so the central claim is unverifiable as submitted.","tokens_in":13348,"tokens_out":2973,"would_cite":false,"duration_ms":46909,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A quadruplet loss with a meta-learned margin lifts zero-shot sketch search","keywords":["zero-shot sketch-based image retrieval","relation-aware quadruplet loss","meta-learning","adaptive margin","metric learning","cross-modal retrieval","external memory","GRU relation network"],"falsifier":"Train RAMLN on the standard seen split, then freeze the backbone and margin network and, on the unseen classes, replace each learned $R(x)$ with a grid-searched fixed margin; if the fixed margin matches or beats the learned one, the generalization claim fails. Additionally, run the same model with the external memory read replaced by a random vector; if accuracy does not drop materially, the memory is not doing the work assigned to it.","tokens_in":12296,"feed_emoji":"🖼️","tokens_out":8343,"duration_ms":67162,"temperature":0.7,"pith_summary":"Sketch-based image retrieval lets a user find photos by drawing a rough sketch, but standard systems fail when the query's category was never seen in training. The paper argues that the root cause is twofold: triplet losses leave the embedding space unbalanced between sketch and photo modalities, and the metric-learning margin is a fixed number that cannot adapt to new categories. To address this, it introduces RAMLN, which uses a relation-aware quadruplet loss with two negative samples from different modalities and learns the loss margin through a memory-augmented meta-learning network. On the two standard benchmarks, TU-Berlin Extended and Sketchy, the method reports large improvements over existing methods without using semantic side information, including 0.649 mAP@all on TU-Berlin Extended.","feed_headline":"Quadruplet loss with meta-learned margin tops zero-shot sketch search","feed_subtitle":"Without semantic labels, RAMLN beats ViT baselines and hits 0.649 mAP@all on TU-Berlin.","key_machinery":"The engine of the method is the relation-aware quadruplet loss $$L_{qua}=D(x_a,x_p)-(1-\\$\\lambda$)D(x_a,x_{n1})-\\$\\lambda$ D(x_a,x_{n2})+R(x),$$ where $D$ is the squared Euclidean distance, $x_a$ is the anchor (a sketch or a photo), $x_p$ is the hardest positive, and $x_{n1},x_{n2}$ are negative samples from the other and from the anchor's own modality. A global inter-modal quadruplet and a local intra-modal quadruplet are combined, and hard mining selects the farthest positive and the closest negatives. The margin $R(x)$ is computed by a bidirectional GRU whose hidden states are pooled, then combined with a read from an external memory matrix through weighted keys, then passed through a linear layer with ReLU to produce two sigmoid-normalized scalars. A softmax cross-entropy classification head is added to the embedding to stabilize training and avoid local optima.","core_discovery":"The central claim is that replacing the standard triplet loss with a relation-aware quadruplet loss, and making the loss margin a learned function of the input rather than a fixed constant, yields an embedding in which unseen classes are well separated and sketches and photos of the same class are well merged. The quadruplet pairs an anchor with one positive and two negatives from different modalities, so the anchor is pushed away from the other modality and from its own modality at the same time; both sketches and photos serve as anchors. The margin $R(x)$ is produced by a GRU-based relation network that reads an external memory matrix and outputs two values, one for the inter-modal term and one for the intra-modal term. In the paper's experiments, RAMLN reaches 0.649 mAP@all and 0.719 Prec@100 on TU-Berlin Extended, and 0.695 mAP@200 on Sketchy-NO, surpassing all compared ViT-based methods that do not use semantic labels.","pith_inferences":["The main text does not specify a separate meta-objective for the margin network; if the margin is learned only by joint gradient descent, the claimed 'meta-learning' could reduce to ordinary training, and the gain might come mostly from the quadruplet geometry.","The external memory's contribution is not isolated in the ablations; replacing the memory read with a random vector would show whether stored features are actually load-bearing.","A direct transfer test, training on half of the seen classes and generating margins for the other half, would show whether margins generalize to categories the network has never seen.","If the margin network does generalize, the same two-negative, memory-guided loss could transfer to other cross-modal retrieval settings such as text-to-image or sketch-to-3D."],"forward_implications":["On the two standard benchmarks, RAMLN outperforms every compared method that does not use semantic side information, including the strong ViT-based baselines.","The learned margin removes the need to hand-tune the margin hyper-parameter, adapting per category and per modality instead.","Using both sketches and photos as anchors closes the sketch-photo gap for unseen classes and improves inter-class separation.","Ablations show the classification loss and the quadruplet loss reinforce each other, and the meta-learned margin beats a fixed margin of 0.3.","Because no semantic text vectors are needed, the method is a cheaper alternative to semantic-augmented ZS-SBIR pipelines."],"supporting_citations":[{"why":"introduces the zero-shot SBIR task and the training/test class split used on TU-Berlin.","marker":"[22]"},{"why":"provides the Sketchy dataset of sketches and photos used for training and evaluation.","marker":"[20]"},{"why":"provides the TU-Berlin sketch dataset and its 250 object categories.","marker":"[6]"},{"why":"expands the photo galleries for both datasets, making the extended evaluation protocols possible.","marker":"[16]"},{"why":"defines the Sketchy-NO protocol with 21 ImageNet-free unseen classes.","marker":"[38]"},{"why":"introduces the triplet-based doodle2search baseline that the paper's quadruplet loss extends.","marker":"[3]"},{"why":"proposes the two-negative, two-modality loss design that the relation-aware quadruplet refines.","marker":"[30]"},{"why":"the strongest ViT-based baseline without semantic labels that RAMLN is compared against.","marker":"[15]"},{"why":"supplies the memory-augmented meta-learning mechanism used for the margin network.","marker":"[21]"},{"why":"provides the PK sampling and hard triplet mining strategy used to form quadruplet batches.","marker":"[8]"}],"fun_headline_variants":["Meta-learned margin quadruplet loss improves zero-shot sketch search","RAMLN: Relation-aware meta-learning for zero-shot sketch retrieval","Zero-shot sketch search improved by relation-aware quadruplet loss","Quadruplet loss with meta-learned margins excels in zero-shot SBIR"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that margins learned from the seen classes will also be right for unseen classes, even though the margin network is trained only on seen data and the main text defers its meta-objective to the supplementary.","fun_headline_variants_meta":{"raw":{"variants":["Meta-learned margin quadruplet loss improves zero-shot sketch search","RAMLN: Relation-aware meta-learning for zero-shot sketch retrieval","Zero-shot sketch search improved by relation-aware quadruplet loss","Quadruplet loss with meta-learned margins excels in zero-shot SBIR"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000645,"raw_usage":{"total_tokens":2991,"prompt_tokens":1001,"completion_tokens":1990,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":617,"completion_tokens_details":{"reasoning_tokens":1916}},"tokens_in":617,"tokens_out":1990,"duration_ms":12976,"temperature":1.0,"reasoning_tokens":1916,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T10:37:24.738734+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train RAMLN on the standard seen split, then freeze the backbone and margin network and, on the unseen classes, replace each learned $R(x)$ with a grid-searched fixed margin; if the fixed margin matches or beats the learned one, the generalization claim fails. Additionally, run the same model with the external memory read replaced by a random vector; if accuracy does not drop materially, the memory is not doing the work assigned to it.","supporting_citations":[{"cited_title":"Zero- shot sketch-image hashing","cited_arxiv_id":null,"evidence_quote":"introduces the zero-shot SBIR task and the training/test class split used on TU-Berlin."},{"cited_title":"Deep sketch hashing: Fast free-hand sketch-based image retrieval","cited_arxiv_id":null,"evidence_quote":"expands the photo galleries for both datasets, making the extended evaluation protocols possible."},{"cited_title":"A zero-shot framework for sketch based image retrieval","cited_arxiv_id":null,"evidence_quote":"defines the Sketchy-NO protocol with 21 ImageNet-free unseen classes."},{"cited_title":"Doodle to search: Practical zero-shot sketch- based image retrieval","cited_arxiv_id":null,"evidence_quote":"introduces the triplet-based doodle2search baseline that the paper's quadruplet loss extends."},{"cited_title":"An efficient framework for zero- shot sketch-based image retrieval","cited_arxiv_id":null,"evidence_quote":"proposes the two-negative, two-modality loss design that the relation-aware quadruplet refines."}],"review_version":1}