{"id":"5c329bd7-f4ec-4cbe-a9c7-6dd438edfe74","arxiv_id":"2411.18472","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A contrastive fine-tuning method that adds frozen semantic-embedding negatives reduces topic leakage in authorship embeddings and improves accuracy on topic-heavy attribution benchmarks.","lead":"The paper adds content embeddings produced by a separate semantic model as extra negative examples in the contrastive loss used to train authorship embeddings, pushing author style away from writing topic. This improves authorship attribution on blog and fanfiction benchmarks, especially when many authors write about the same topics, though the reported gains lack error bars and the code is not actually provided.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The detangled loss is compared against a simple InfoNCE baseline that has a different negative-pool size (3N vs N), so the reported gains may be due to more negatives rather than to content disentanglement. This confound is the most load-bearing issue for the central claim.","rationale":"The reader's weakest assumption concerns the frozen content model's style-freeness, which is a legitimate threat to the theoretical mechanism. However, the paper itself acknowledges this assumption is 'obviously untrue,' and the method may still work as long as UAE is a sufficiently good content surrogate. The more directly load-bearing flaw is experimental: the detangled loss differs from the simple InfoNCE baseline in two ways at once—the semantic source of the extra negatives and the total number of negatives. Because InfoNCE performance typically improves with more negatives, the central claim that the improvement is due to disentangling style from content is not established by the reported comparisons. This is an internal control issue rather than a disagreement with external consensus, so it is appropriately addressed by a concrete control experiment. If the proposed 3N-negative style baseline matches the detangled results, the paper's central claim would collapse to a negative-count effect; if not, the disentanglement mechanism gains real support. The verdict remains CONDITIONAL: acceptance should require this control, along with the statistical rigor issues already noted by the reader.","tokens_in":13348,"tokens_out":7569,"duration_ms":73982,"concrete_test":"Retrain the 'Fine-tune (simple)' baseline with the same 3N negatives as the detangled objective, but with the two extra blocks drawn from style embeddings of an additional author-disjoint document set, keeping batch size, epochs, learning rate, and temperature identical. Then rerun Tables 2, 4, and 5, especially the top-K=10 row of Table 5. If this 3N-negative style baseline reaches or exceeds detangled accuracy (0.4469 on blogs, 0.7924 on fanfiction, 0.8158 at top-K=10), the central claim is not supported; if it stays at the original simple InfoNCE level, the content-negative mechanism is supported and the confound is rejected.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim (Sec. 5.4) is that the detangled model 'outperforms a contrastive fine-tuning under the same conditions and training time.' But the comparison is not controlled for the number of negatives. In Algorithm 1 and Eq. 2, the reference set is R(DA, DB) = s(DB) ∥ c(DA) ∥ c(DB), giving a denominator of 3N terms per anchor. The 'Fine-tune (simple)' baseline is standard InfoNCE with only the N style embeddings in the batch, i.e., a denominator of N terms. InfoNCE is known to improve as the negative set grows, so the gains in Tables 2, 4, and 5 (up to ~10 points in Table 5) could come from a larger negative pool or stronger contrastive regularization, not from semantically informed disentanglement. The paper's own metric for disentanglement, 'same topic when miss' (Table 3), also decreases monotonically from RoBERTa to STAR to both fine-tuned models, so it does not isolate the proposed mechanism. The acknowledged imperfection of the frozen content model (Sec. 3) is real but secondary: even if UAE were a perfect style-free content reference, the current experiments would not identify its contribution because the loss objective and negative count are changed simultaneously.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a modification of the InfoNCE contrastive objective for authorship attribution: in addition to the usual in-batch style negatives, the loss adds frozen semantic-embedding negatives produced by a content model (UAE-Large-V1), so that the style embedding space is pushed away from the content embedding space. The method is evaluated on a blog dataset, a large fanfiction corpus, and the PAN 11-19 authorship challenges, comparing against RoBERTa, a zero-shot style model (STAR), and a simple InfoNCE fine-tuning of STAR. The paper reports consistent accuracy gains for the proposed 'detangled' model, with the largest improvements (up to about 10 points) on a subset of highly topic-diverse authors, and a reduction in the fraction of attribution errors that confuse authors writing about the same topic.","tokens_in":13622,"tokens_out":5938,"duration_ms":53629,"significance":"If the central claim were fully supported, the proposal would be a simple and inexpensive regularizer that reduces content leakage in authorship embeddings, with practical value for authorship attribution in topic-diverse settings. The paper evaluates on multiple datasets and standard challenges, which is a strength. However, the current empirical support is weakened by a confounded baseline: the 'detangled' loss uses three times as many negatives as the simple InfoNCE baseline, and the paper's own disentanglement metric (same-topic-when-miss) does not isolate the proposed mechanism because it improves for the simple fine-tuning baseline as well. The assumption that the frozen content model is free of style is acknowledged as false but is not tested through an alternative content-model ablation.","major_comments":[{"comment":"The comparison between 'Fine-tune (simple)' and 'Fine-tune (detangle)' is not controlled for the number of negative examples. In Algorithm 1 and Eq. (2), the reference set R(DA, DB) = s(DB) || c(DA) || c(DB) gives a denominator of 3N terms per anchor, whereas the simple InfoNCE baseline has only the N style embeddings in the batch as negatives. InfoNCE is known to improve as the negative set grows, so the accuracy gains in Tables 2, 4, and 5 could be due to a larger negative pool or stronger contrastive regularization rather than to semantically informed disentanglement. The claim in Section 5.4 that the detangled model 'outperforms a contrastive fine-tuning under the same conditions and training time' requires a controlled comparison, for example a simple InfoNCE baseline with additional style negatives (or sampled negatives) to match the 3N pool size, or a detangled variant that subsamples the combined reference set to N negatives.","section":"Section 5.4, Eq. (2), Algorithm 1"},{"comment":"The loss definition in Eq. (3) is mathematically incorrect as written: LCE sums log P over all 3N reference items, but the standard InfoNCE / cross-entropy loss for a single positive is -log P(positive). This is inconsistent with Algorithm 1, which calls cross_entropy(simf, labels) and thus correctly implements the standard loss. The equation needs to be corrected, otherwise the method's derivation in Section 3 does not match the implemented objective.","section":"Section 3, Eq. (3)"},{"comment":"The reported results do not include standard deviations, confidence intervals, or significance tests. The paper states that evaluation is repeated 10 times for stability, but this appears to be 10 random selections of validation documents rather than 10 independent training runs. With only one training run per condition, the observed improvements of 2-4 points (and even the larger gains in Table 5) could be within run-to-run variance. The authors should report variance across multiple random seeds, or at least provide bootstrap confidence intervals for the evaluation metric.","section":"Tables 2, 4, and 5"},{"comment":"The 'Same topic when miss' metric decreases monotonically from RoBERTa (0.1382) to STAR (0.1258) to Fine-tune simple (0.1193) to Fine-tune detangle (0.1031). This monotonic trend means the metric does not isolate the proposed disentanglement mechanism: simple fine-tuning already reduces topic-related errors, and the detangled model's lower value could simply be a continuation of that trend. To support the claim that the semantic-embedding negatives specifically reduce topic-driven confusions, a comparison that matches training conditions and controls for the number of negatives is needed.","section":"Table 3, Section 5.1"}],"minor_comments":[{"comment":"In Algorithm 1, lines for sim2 and sim3 use 'DA' and 'DB' as inputs to cosine_distance, but they should be the content embeddings CA and CB; as written, these lines are dimensionally inconsistent with the text of Section 3.","section":"Algorithm 1"},{"comment":"The term Scos(DA,i) in Eq. (2) should be Scos(SA,i), where SA,i = s(DA,i), since the similarity is computed between style embeddings.","section":"Eq. (2)"},{"comment":"The sentence 'Our code is available at lightning.ai/docs/pytorch/stable/' points to the PyTorch Lightning documentation rather than to a code repository for this paper; the authors should provide an actual repository link or remove this claim.","section":"Section 4.3"},{"comment":"The caption of Table 4 says 'Attribution results across the entire blog author set', but the table presents fanfiction results; the caption should be corrected.","section":"Table 4 caption"},{"comment":"The paper consistently uses 'detangle' and 'detangling' where 'disentangle' and 'disentangling' are standard; this is a wording issue, not a technical one, but it should be normalized for clarity.","section":"Abstract and throughout"}],"recommendation":"major_revision","confidential_remarks":"The negative-pool confound is the principal barrier to publication: without a controlled baseline that matches the number of negatives, the central empirical claim is not supported. I believe this is fixable with additional experiments, and the paper's method is simple and potentially useful, so I recommend major revision rather than rejection. Also, the code availability statement is misleading and should be corrected."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: this paper extends InfoNCE by adding frozen semantic-embedding negatives to push style embeddings away from content embeddings, and it reports consistent accuracy gains on two datasets. The idea is simple and worth testing. The paper is also honest about its key assumption—that the frozen content model (UAE) is not style-free—and it does not oversell that part. It ships no code, but the ablations and PAN challenge results are real work.\n\nWhat's genuinely new: the specific mechanism of using a frozen semantic model's embeddings as extra negatives in the contrastive loss. That is a concrete modification to the authors' own STAR model, and it is not in the cited disentanglement literature (ADNet, ContrastDistAA, ARR, Wegmann et al.). The empirical pattern—detangled beats simple fine-tuning on both in-domain sets and retains zero-shot performance reasonably—is consistent across tables.\n\nNow the soft spots, in proportion. The stress-test note is correct and it is load-bearing: the \"simple\" InfoNCE baseline has N negatives per anchor, while the detangled loss has 3N (style batch plus two content batches). InfoNCE is known to improve with more negatives, so the reported gains in Tables 2, 4, and 5 cannot be attributed to semantic disentanglement. This is not a minor detail; it is the central claim of Section 5.4. The paper's own disentanglement metric (\"same topic when miss\") decreases monotonically even for simple fine-tuning, so it does not isolate the proposed mechanism. There is also a typo in Algorithm 1 (cosine_distance(SA, DA) should be SA vs CA), no standard deviations or significance tests, and only one training run per condition. The absence of comparisons with prior disentanglement methods is a missed opportunity, though not fatal.\n\nThe paper's admission that the content model's subspace retains style is real, but that is secondary to the confound. Even a perfect content model would not save the current experiments, because the loss change and negative-count change are conflated.\n\nWho this is for: researchers working on authorship attribution with contrastive learning. It would be a useful contribution if the confound is resolved. As it stands, it deserves a serious referee because the idea is novel and the writing is transparent, but it needs major revision: match the negative-pool size across conditions, run multiple seeds, report significance, and add the missing baseline comparisons. I would not cite it yet, but I would bring it to a reading group to discuss the experimental-design issue.","headline":"The core idea is plausible and honestly reported, but the main comparison is confounded by the baseline using fewer InfoNCE negatives, so the headline gains may not come from disentanglement.","tokens_in":14149,"tokens_out":1398,"would_cite":false,"duration_ms":14765,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Frozen content negatives sharpen authorship attribution by pushing style embeddings away from topic.","keywords":["authorship attribution","style-content disentanglement","contrastive learning","InfoNCE","semantic embeddings","hard negatives","zero-shot transfer","transformer fine-tuning"],"falsifier":"Train the same loss with a content model that is deliberately style-aware (for example, a model fine-tuned on authorship) and check whether accuracy gains shrink or reverse; or measure how much authorship signal a classifier can recover from the frozen content embeddings alone — if that signal is high, the repulsion is pushing against a space that already contains style.","tokens_in":13163,"feed_emoji":"✍️","tokens_out":6361,"duration_ms":51534,"temperature":0.7,"pith_summary":"This paper argues that authorship embeddings trained with contrastive learning are contaminated by topic: because authors tend to write about the same subjects, the model learns spurious content correlations. The authors propose a modified InfoNCE loss that, alongside the usual same-author positives and different-author negatives, adds negatives produced by a frozen semantic-similarity model, pushing the trainable style space away from a fixed content space. On two datasets the procedure improves attribution accuracy, with the largest gains (up to 10 percentage points) in the hard case where prolific authors write across many topics, and it reduces the share of errors caused by confusion within one topic. The method also preserves more zero-shot ability on PAN challenges than plain fine-tuning does.","feed_headline":"Frozen content negatives sharpen authorship attribution","feed_subtitle":"Adding frozen semantic negatives cuts topic-driven errors and lifts accuracy on hard author sets.","key_machinery":"The central object is the modified InfoNCE objective of Eq. (2) and Algorithm 1. For two document sets $D_A$ and $D_B$, the reference set is $R(D_A, D_B) = s(D_B) \\parallel c(D_A) \\parallel c(D_B)$, so each row of the cosine-similarity matrix contains one same-author positive and $3N-1$ negatives, including the content embeddings of the anchor text itself. The frozen content model (UAE-Large-V1) fixes the content subspace, and the trainable style model (warm-started from STAR) is pushed away from it by the cross-entropy loss. This geometric separation is the entire mechanism: the only learnable parameters are in the style encoder.","core_discovery":"The paper's central claim is that style-content entanglement in authorship representations can be reduced by extending InfoNCE with hard negatives drawn from a frozen semantic embedding space. Concretely, for each document the loss treats same-author embeddings as positives and, as negatives, different-author style embeddings, content embeddings of the same text, and content embeddings of other texts; minimizing the cross-entropy then separates the style subspace from the content subspace. The paper reports that this 'detangled' fine-tuning beats contrastive fine-tuning under identical conditions and training time, and that on the most heterogeneous-author subsets the accuracy gain reaches about 10 percentage points while topic-related miss rates fall. It also reports better retention of zero-shot attribution performance than plain fine-tuning, especially when the disentangling dataset is large.","pith_inferences":["Beyond the paper, the same loss could be applied to any contrastively trained style encoder, so the disentanglement recipe is not specific to authorship attribution; tasks like author profiling or style-transfer evaluation could use the identical frozen-content negative trick.","Beyond the paper, swapping the frozen content model — for example to a different semantic-similarity encoder or to an MLM embedding — would test how much of the gain depends on the specific blind spots of UAE rather than on content removal generally.","Beyond the paper, because the PAN19 challenge overlaps the fanfiction training corpus, the strong zero-shot result on PAN19 may partly reflect domain overlap; a held-out-domain evaluation with no overlap would give a cleaner estimate of preserved zero-shot ability."],"forward_implications":["On the blog dataset, detangled fine-tuning raises attribution accuracy from 0.3928 (simple fine-tune) to 0.4469 and reduces the same-topic miss proportion from 0.1193 to 0.1031.","On the fan-fiction dataset, detangled fine-tuning reaches 0.7924 accuracy versus 0.7741 for simple fine-tuning, and on the most topic-diverse author subsets it outperforms zero-shot STAR by roughly 8 to 11 percentage points.","Across PAN11-19 challenges, the detangled fan-fiction model achieves the highest average accuracy (0.6346) among the fine-tuned variants, while plain fan-fiction fine-tuning drops to 0.5748, suggesting the method preserves zero-shot ability better than ordinary fine-tuning.","Because the method is a regularizer rather than a new architecture, it applies to any contrastively trained style encoder paired with any frozen semantic encoder of matching dimension.","The detangled model consistently lowers topic-related attribution failures, which is direct evidence that the learned style space depends less on content."],"supporting_citations":[{"why":"Supplies the InfoNCE objective that the paper extends by adding content-space negatives.","marker":"[2]"},{"why":"Provides the pre-trained style encoder STAR that is warm-started and then fine-tuned; also serves as the zero-shot baseline.","marker":"[25]"},{"why":"Provides the frozen semantic-similarity model (UAE-Large-V1) whose embeddings define the content space used as negatives.","marker":"[26]"},{"why":"Motivates the target of content-independent style representations with a contrastive framework that controls for same-topic pairs.","marker":"[23]"},{"why":"Presents the closest prior contrastive disentanglement method for authorship attribution, which the paper's approach is measured against conceptually.","marker":"[15]"},{"why":"Supplies the blog dataset with topic labels, used for attribution accuracy and same-topic error analysis.","marker":"[27]"},{"why":"Supplies the large fan-fiction corpus used to train and to build the hard multi-topic author evaluation.","marker":"[28]"}],"fun_headline_variants":["Semantic negatives untangle style from content","Contrastive learning separates author style from topic","Hard negatives improve authorship attribution on tricky cases","Frozen embeddings sharpen style embeddings for author ID","Contrastive learning with semantic negatives isolates author style"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the frozen semantic model's embedding space contains essentially no stylistic information, so pushing style embeddings away from it removes content rather than distorting style; the paper itself admits this is not strictly true.","fun_headline_variants_meta":{"raw":{"variants":["Semantic negatives untangle style from content","Contrastive learning separates author style from topic","Hard negatives improve authorship attribution on tricky cases","Frozen embeddings sharpen style embeddings for author ID","Contrastive learning with semantic negatives isolates author style"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000484,"raw_usage":{"total_tokens":2349,"prompt_tokens":862,"completion_tokens":1487,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":478,"completion_tokens_details":{"reasoning_tokens":1417}},"tokens_in":478,"tokens_out":1487,"duration_ms":10429,"temperature":1.0,"reasoning_tokens":1417,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:09:11.263943+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same loss with a content model that is deliberately style-aware (for example, a model fine-tuned on authorship) and check whether accuracy gains shrink or reverse; or measure how much authorship signal a classifier can recover from the frozen content embeddings alone — if that signal is high, the repulsion is pushing against a space that already contains style.","supporting_citations":[{"cited_title":"Understanding writing style in social media with a supervised contrastively pre-trained transformer","cited_arxiv_id":null,"evidence_quote":"Provides the pre-trained style encoder STAR that is warm-started and then fine-tuned; also serves as the zero-shot baseline."},{"cited_title":"Contrastive disentanglement for authorship attribution","cited_arxiv_id":null,"evidence_quote":"Presents the closest prior contrastive disentanglement method for authorship attribution, which the paper's approach is measured against conceptually."},{"cited_title":"Effects of Age and Gender on Blogging","cited_arxiv_id":null,"evidence_quote":"Supplies the blog dataset with topic labels, used for attribution accuracy and same-topic error analysis."},{"cited_title":"[Online; accessed 16","cited_arxiv_id":null,"evidence_quote":"Supplies the large fan-fiction corpus used to train and to build the hard multi-topic author evaluation."}],"review_version":1}