{"id":"45d74410-c6d3-4b2f-ab5c-c03621d13dc5","arxiv_id":"2508.19464","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A contrastive alignment objective applied during few-shot fine-tuning improves cross-lingual transfer for NLI and relation extraction in many settings, though the reported advantage over prompting baselines is inconsistent across models and datasets.","lead":"CoLAP adds contrastive training objectives to prompt-based few-shot adaptation so models transfer English task knowledge to low-resource languages with very few labeled examples. The paper evaluates the method on three multilingual benchmarks and three model families, reporting gains in many settings but also several settings where a prompting baseline wins.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The abstract's unqualified claim that CoLAP outperforms few-shot cross-lingual baselines is contradicted by the paper's own Table 1, including at K=5.","rationale":"The Reader's REJECT verdict is correct, but the stated weakest_assumption — the need for parallel corpora in XRCL — is not the most load-bearing concern. The paper's own Table 1 directly contradicts the abstract's universal superiority claim, so the rejection follows from internal evidence without relying on any external assumption about translation quality or availability. The K=5 counterexamples are particularly damaging because the text explicitly singles out K=5 as the setting where CoLAP 'outperforms all baseline approaches' for XLM-R and Gemma and is 'on par' for Mistral; Table 1 shows PCT beating CoLAP on MultiTACRED for both decoder models at K=5. The Section 5 narrative also contradicts itself within two sentences. The loss-equation mis-specification and test-set layer selection are additional correctness problems, but they are secondary to the fact that the headline empirical claim fails on the reported numbers. The paper does contain useful components — XCCL removes the parallel-corpus requirement, and CoLAP does win in several settings — so the appropriate remedy is a revised, qualified claim, not necessarily a wholesale dismissal of the method.","tokens_in":33226,"tokens_out":5236,"duration_ms":50485,"concrete_test":"Produce a pairwise win/loss tally from Table 1: for each dataset (XNLI, AmNLI, MultiTACRED) × model (XLM-R, Gemma, Mistral) × K (5,10,50,100,250), compare CoLAP (both XRCL and XCCL rows) against PCT and CA using the reported means and reported standard deviations. Count how many cells PCT wins and by how much; separately report the K=5 count. If PCT wins in even a single-digit number of cells, the unqualified abstract claim is false and the manuscript must be revised to state settings where CoLAP helps rather than a universal superiority claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim — 'CoLAP outperforms few-shot cross-lingual transfer baselines and in-context learning, even with limited available data' — is not supported by the submitted results. In Table 1, the K=5 setting already falsifies the text's assertion that CoLAP outperforms all baselines for Gemma and is on par for Mistral: on MultiTACRED, PCT beats CoLAP w/ XRCL for Gemma (44.00 vs 43.47) and for Mistral (43.35 vs 38.73). This is not an isolated cell. PCT also beats CoLAP at K=50 on MultiTACRED-XLM-R (75.36 vs 73.98) and MultiTACRED-Gemma (62.14 vs 59.76); at K=100 on MultiTACRED-XLM-R (78.26 vs 77.29) and MultiTACRED-Mistral (74.67 vs 73.97); at K=250 on AmNLI-Mistral (45.21 vs 42.45) and MultiTACRED-XLM-R (82.19 vs 82.16). Section 5 contains internally contradictory sentences: 'CoLAP surpasses the performance of its benchmarks, except in the AmNLI dataset for Gemma and Mistral' is immediately followed by 'Even in the high-resource setting of K=250 exemplars, CoLAP exceeds the performance of its benchmarks.' The parallel-corpus assumption flagged by the Reader is a real limitation of XRCL, but the universal claim fails even when parallel data are available; the internal contradiction with Table 1 is the load-bearing issue.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CoLAP, a few-shot cross-lingual transfer method that combines prompt-based fine-tuning with two contrastive objectives: XRCL, which aligns parallel source–target sentence representations, and XCCL, which aligns representations of same-class instances without parallel data. The method is evaluated on XNLI, AmericasNLI, and MultiTACRED using XLM-R, Gemma 2, and Mistral, and the authors claim it outperforms few-shot cross-lingual baselines and in-context learning even with very few target-language examples.","tokens_in":33509,"tokens_out":4426,"duration_ms":47142,"significance":"If the claims held, CoLAP would be a practically useful, data-efficient method for adapting multilingual PLMs to low-resource languages, and the introduction of a non-parallel contrastive objective (XCCL) is a sensible direction. The paper ships code, covers three model families and three tasks, and evaluates very-low-resource languages. However, the central empirical claim is not supported by the paper's own tables, and the loss equations are formally incorrect as written, so the contribution is not currently established.","major_comments":[{"comment":"Both L_XRCL and L_XCCL are written as -log[exp(phi(ri,T, r+_i)/tau) / exp(phi(ri,T, r−_i)/tau)], with r−_i defined as a set. This is not the InfoNCE/contrastive loss described in the text: there is no summation over negative pairs, and the denominator does not include the positive pair or a partition function. As written, the expression is not even a well-defined probability. This is a load-bearing specification error because the method's core objective is not reproducible from the text.","section":"Section 3.2, Eqs. (1) and (2)"},{"comment":"The abstract states that CoLAP 'outperforms few-shot cross-lingual transfer baselines and in-context learning, even with limited available data.' Table 1 directly contradicts this. For example, at K=5 on MultiTACRED, PCT beats CoLAP w/ XRCL for Gemma (44.00 vs 43.47) and for Mistral (43.35 vs 38.73). PCT also wins at K=50 on MultiTACRED-XLM-R (75.36 vs 73.98), at K=100 on MultiTACRED-XLM-R (78.26 vs 77.29), and at K=250 on AmNLI-Mistral (45.21 vs 42.45), among other cells. The text in Section 5 is internally contradictory: 'CoLAP surpasses the performance of its benchmarks, except in the AmNLI dataset for Gemma and Mistral' is immediately followed by 'Even in the high-resource setting of K=250 exemplars, CoLAP exceeds the performance of its benchmarks.' The headline claim is overstated and not supported by the reported results.","section":"Section 5, Table 1, and Abstract"},{"comment":"The paper states that no validation set is used during few-shot fine-tuning, yet Section 5.2 selects the contrastive layer by evaluating on the test sets of XNLI and AmNLI (Figure 5). This is test-set-based hyperparameter selection, which can inflate reported performance and invalidate the strength of the comparisons. Additionally, Appendix A observes that models with K>100 'gain performance when trained up to 50 epochs,' but the main training protocol fixes 10 epochs; it is unclear whether the K=100 and K=250 results in Table 1 used 10 or 50 epochs, and whether this decision was also based on test performance. The evaluation protocol needs to be clarified and, ideally, a validation split introduced for any hyperparameter choice.","section":"Sections 4.1 and 5.2"},{"comment":"The XRCL variant assumes that the target-language few-shot examples and the source-language examples are parallel translations. The Limitations section acknowledges this, but the paper does not analyze how sensitive XRCL is to noisy or unavailable translations. More importantly, the XCCL variant, which is supposed to remove this dependency, also fails to beat PCT in many configurations (e.g., MultiTACRED at K=50 for all models, and AmNLI at K=250 for Mistral). Thus the claimed practical advantage of CoLAP over a strong prompt-based baseline is not demonstrated even under the favorable parallel-data assumption for XRCL, nor under the more realistic non-parallel setting for XCCL.","section":"Sections 4.2 and 7"}],"minor_comments":[{"comment":"The text says the hidden state of the <EOS> token is used for label prediction, but footnote 2 correctly notes that masked language models use the <mask> token. This should be stated in the main text for clarity.","section":"Section 3.1"},{"comment":"The row labeled 'CoLAP' at K=0 is not described anywhere; it appears to be the prompted model without contrastive adaptation. This should be defined explicitly, or the row should be relabeled, since 'CoLAP' normally implies the contrastive objectives.","section":"Table 1, K=0 row"},{"comment":"The claim that similarity-based exemplar selection 'improves data efficiency by at least 50%' is not directly supported by the table. The reported improvements are small absolute accuracy differences, and the '50%' figure is never defined. Please clarify the metric and provide a significance test or variance estimate.","section":"Section 5.3 and Table 2"},{"comment":"There is a typo: 'the number relation types' should be 'the number of relation types.' Also, the relation mapping from 41 to 31 labels is only mentioned in the appendix; this is an important experimental detail that should appear in the main text, since it changes the task definition for all models.","section":"Appendix A"},{"comment":"Equations (1) and (2) are identical in form; only the definitions of r+ and r− differ. This is fine, but the text could explicitly state that the same loss functional is used with different positive/negative sets, rather than presenting the same equation twice.","section":"Section 3.2"}],"recommendation":"reject","confidential_remarks":"The paper has a public code release and addresses an important problem, but the central claim is contradicted by the paper's own Table 1, the loss equations are not mathematically well-formed, and the evaluation protocol includes test-set-based hyperparameter selection. These are load-bearing issues that cannot be resolved by minor text edits; a re-analysis and substantial re-writing would be required."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I read this carefully. The core idea is solid and worth engaging with. CoLAP adds two task-specific contrastive losses to prompt-based few-shot cross-lingual adaptation: XRCL aligns parallel source-target sentence representations, and XCCL aligns same-class representations across languages without parallel data. XCCL in particular is a genuinely new and useful variant, and the experiments are broad: three model families, three datasets, 27 languages, five shot values. The code is released. The method does help in several settings, especially on XNLI and with larger K on MultiTACRED. The ablation study is honest, and the Limitations section openly notes that XRCL requires translated pairs. That is real, reproducible engineering work, and I would not want to dismiss it.\n\nThe problems are real too, and they are load-bearing. The loss equations in Section 3.2 are mis-specified as written: they look like pairwise margin losses, not InfoNCE. There is no sum over negatives and no positive term in the denominator. You cannot reproduce the method from the paper alone. That alone should force a major revision. More important, the abstract's central claim that \"CoLAP outperforms few-shot cross-lingual transfer baselines and in-context learning\" is contradicted by the paper's own Table 1 in multiple settings, including at K=5: on MultiTACRED with Gemma, PCT gets 44.00 vs CoLAP's 43.47; with Mistral, PCT gets 43.35 vs CoLAP's 38.73. The text in Section 5 even says CoLAP outperforms all baselines at K=5 for Gemma and is on par for Mistral, and then immediately contradicts itself two sentences later about the K=250 case. The claims need to be rewritten to match the tables.\n\nThere is also a methodological soft spot: the contrastive layer is selected based on test-set performance (Figure 5), with no validation set anywhere. Choosing a hyperparameter on the test set inflates the reported gains, especially in few-shot settings. The paper's choice to avoid a validation set is defensible as a practical stance, but then the layer index cannot be tuned on the test set. The parallel-corpus assumption for XRCL is a real limitation, but it is acknowledged, and XCCL partially addresses it.\n\nWho is this for? Researchers working on low-resource multilingual classification. The XCCL objective alone may be worth taking seriously. But the paper needs a corrected formulation, a validation-based or pre-registered layer choice, and claims that match the actual tables. I would send it to peer review, but I would expect substantial revision before acceptance. The underlying method is not a mirage; the presentation currently oversells it.","headline":"A genuinely useful few-shot cross-lingual adaptation recipe, but the paper overclaims its own results and has reproducibility blockers in the loss definitions and layer selection.","tokens_in":34147,"tokens_out":1413,"would_cite":false,"duration_ms":18589,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CoLAP claims contrastive alignment of target-language examples with English makes few-shot cross-lingual transfer outperform baselines.","keywords":["CoLAP","few-shot cross-lingual transfer","contrastive learning","prompt-based fine-tuning","low-resource languages","natural language inference","relation extraction","multilingual language models"],"falsifier":"Train CoLAP XCCL and the PCT baseline on the same K=5 and K=10 episodes for a language with no parallel English data and check whether XCCL's average accuracy actually exceeds PCT's across a held-out set of languages; if the advantage does not reproduce, the data-efficiency claim is settled. The paper's own ablation, which removes the contrastive loss and shows a performance drop, is a within-paper version of this test.","tokens_in":32989,"feed_emoji":"🌐","tokens_out":6139,"duration_ms":59228,"temperature":0.7,"pith_summary":"The paper proposes CoLAP, a method for adapting pretrained multilingual language models to low-resource languages using only a handful of labeled target-language examples. The idea is that downstream classification representations are simpler than language-modeling representations, so they can be aligned across languages with very little data. CoLAP adds a contrastive loss during few-shot adaptation that pulls target-language representations toward English: either toward the exact translation of each example (XRCL) or toward all English examples sharing the same class label (XCCL, no parallel data needed). The paper reports that CoLAP beats strong few-shot cross-lingual baselines and in-context learning on natural language inference and relation extraction across 27 languages, including languages not seen in pretraining, and that similarity-based exemplar selection further cuts the number of examples needed. If correct, this makes low-resource language adaptation substantially cheaper in labeled data and annotation effort.","feed_headline":"CoLAP beats few-shot cross-lingual baselines with 5 examples","feed_subtitle":"Pulling target-language examples toward English alignments lets models adapt with far fewer labels.","key_machinery":"The mechanism is a pair of contrastive objectives added to the cross-entropy loss during the few-shot target-language adaptation phase. Prompted inputs are passed through the PLM and the hidden state at <EOS> (or the <mask> token for masked models) is used as the representation. XRCL pulls each target-language representation toward its English translation (positive pair) and away from all other English instances; XCCL instead uses all same-label English instances as positives and different-label instances as negatives, removing the need for parallel translations. The paper also identifies layer 10 of XLM-R as the best representation layer for the contrastive loss, and proposes selecting few-","core_discovery":"The central claim is that discriminative task-specific information can be transferred from English to lower-resource languages with very few labeled examples by aligning representations at few-shot fine-tuning time rather than at pretraining time. CoLAP does this by prompting a multilingual model for the task, extracting an <EOS>/<mask> hidden representation, and training with L = L_CE + L_contrastive on an episode of K target-language examples plus K English examples. The XRCL variant treats each target example's English translation as its positive pair; the XCCL variant treats all English examples with the same class label as positives. Across XNLI, AmericasNLI, and MultiTACRED, with encod","pith_inferences":["The failure of English-based exemplar selection on unseen languages suggests a testable fix: build class prototypes from target-language representations rather than English, which may extend the data-efficiency gain to truly unseen languages.","If task-specific representations are as simple as the paper argues, the same contrastive recipe should transfer to other classification tasks and to token-level tasks with minimal adaptation, though the XCCL objective in its current form is limited to classification.","The layer-10 finding is model- and task-specific; a practical extension would search layer choice per task family or make the contrastive layer learnable rather than fixing it at one layer.","The ablation shows combining XRCL and XCCL helps but requires translations; a translator-free hybrid could use back-translation or synthetic parallel data to get part of the XRCL signal, a direction the paper does not explore."],"forward_implications":["CoLAP with K=5 target examples outperforms in-context learning on XNLI, AmNLI, and MultiTACRED for Gemma 2 and Mistral, with average gains of 6.41 and 6.93 accuracy points.","The XCCL variant needs no parallel translations and stays within about one point of XRCL once K>=10, so labeled target examples alone can drive adaptation.","Prompt-based contrastive alignment works across encoder-only and decoder-only models and needs no new model parameters.","Similarity-based exemplar selection improves data efficiency for languages represented in pretraining; with K=5 selected examples CoLAP exceeds randomly selected K=250 examples on XNLI."],"supporting_citations":[{"why":"Supplies the prompt-based few-shot classification method and label-word mapping that CoLAP builds on.","marker":"(Schick and Schütze, 2021)"},{"why":"InfoXLM contrastive alignment during pretraining is adapted by CoLAP to task fine-tuning without a memory queue.","marker":"(Chi et al., 2021)"},{"why":"Motivates projecting lower-resource representations into the English space, the premise behind CoLAP's alignment objectives.","marker":"(Yang et al., 2022)"},{"why":"Provides the InfoNCE contrastive loss form used by both XRCL and XCCL.","marker":"(van den Oord et al., 2019)"},{"why":"XNLI is the natural language inference benchmark used for evaluation.","marker":"(Conneau et al., 2018)"},{"why":"AmericasNLI supplies the very low-resource indigenous languages, including languages unseen during pretraining.","marker":"(Ebrahimi et al., 2022)"},{"why":"MultiTACRED supplies the multilingual relation extraction benchmark.","marker":"(Hennig et al., 2023)"},{"why":"Checkpoint averaging is the strong few-shot cross-lingual transfer baseline that CoLAP compares against and ablates.","marker":"(Schmidt et al., 2023)"},{"why":"PCT is the strongest prompting baseline, providing multilingual prompt templates and a consistency loss that CoLAP is measured against.","marker":"(Qi et al., 2022)"}],"fun_headline_variants":["CoLAP adapts to new languages with just 5 examples","5 examples per language: CoLAP beats cross-lingual baselines","Few-shot cross-lingual: CoLAP's contrastive trick wins with 5 labels","CoLAP: contrastive alignment for cross-lingual transfer in 5 shots","CoLAP uses English pairs to master new languages in 5 shots"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The load-bearing premise is that the few-shot target-language examples and the source-language examples are parallel corpora whose direct translations are meaning-preserving; if translations are noisy, misaligned, or unavailable in a real deployment, the XRCL contrastive signal is corrupted and the claimed gains would not transfer.","fun_headline_variants_meta":{"raw":{"variants":["CoLAP adapts to new languages with just 5 examples","5 examples per language: CoLAP beats cross-lingual baselines","Few-shot cross-lingual: CoLAP's contrastive trick wins with 5 labels","CoLAP: contrastive alignment for cross-lingual transfer in 5 shots","CoLAP uses English pairs to master new languages in 5 shots"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001161,"raw_usage":{"total_tokens":4621,"prompt_tokens":698,"completion_tokens":3923,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":442,"completion_tokens_details":{"reasoning_tokens":3829}},"tokens_in":442,"tokens_out":3923,"duration_ms":27203,"temperature":1.0,"reasoning_tokens":3829,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T15:47:45.104900+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train CoLAP XCCL and the PCT baseline on the same K=5 and K=10 episodes for a language with no parallel English data and check whether XCCL's average accuracy actually exceeds PCT's across a held-out set of languages; if the advantage does not reproduce, the data-efficiency claim is settled. The paper's own ablation, which removes the contrastive loss and shows a performance drop, is a within-paper version of this test.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Motivates projecting lower-resource representations into the English space, the premise behind CoLAP's alignment objectives."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MultiTACRED supplies the multilingual relation extraction benchmark."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Checkpoint averaging is the strong few-shot cross-lingual transfer baseline that CoLAP compares against and ablates."}],"review_version":1}