{"id":"b4cb92a6-8d55-4516-8bc9-4038ea6bf49f","arxiv_id":"2412.12954","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Fine-tuned language models predict the gender of a message recipient better than chance across three conversation datasets, establishing a new 'recipient profiling' task.","lead":"Researchers show that machine learning models can often predict the gender of the person receiving a message, based only on the message text. This finding extends familiar author profiling to recipients, raising new privacy questions about who is exposed in everyday communication.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SWDA preprocessing may mix both speakers' turns, letting models classify the recipient's own words; this could explain the 0.77 peak and must be checked before the feasibility claim is accepted.","rationale":"The reader's weakest_assumption correctly identifies the direct-identifier confound and the authors' admission in Section 7 that they did not investigate it. My stress-test agrees that this is a genuine limitation, but I see a more concrete and potentially more damaging issue that the reader did not explicitly name: the SWDA preprocessing may not be speaker-restricted. The paper's description in Section 4 is ambiguous, and the gap between SWDA accuracy (0.75-0.77) and the other datasets (0.59-0.68) is exactly what one would expect if the input chunks contained both sides of each conversation, allowing the model to perform ordinary author profiling on the recipient's own words. This is not a matter of direct identifiers like names or titles; it is a question of whether the task definition was followed at all in the highest-performing dataset. I am not claiming the leak is definitely present; the ambiguity is resolvable by inspecting the released code and dataset. If the check shows that chunks are speaker-restricted, the concern is void and the feasibility claim retains its support. If it shows mixing, the paper's central quantitative evidence weakens substantially, though MDC and TIC would still provide some positive evidence. Because the paper is explicitly a proof-of-concept and the released artifacts make this test feasible, the existing CONDITIONAL verdict remains appropriate, with the added condition that the SWDA construction be validated. The reader and I partially agree: we both worry about shortcut signals, but I prioritize the construction-level leak over the direct-identifier confound, which is secondary once the SWDA inputs are verified.","tokens_in":9105,"tokens_out":5629,"duration_ms":59082,"concrete_test":"Inspect the released preprocessing code and dataset for SWDA to determine whether the 9,030 concatenated chunks are speaker-restricted. If they are not, rerun BERT, MPNet, and DeBERTa on SWDA using only the author's turns as input, with the same recipient-based split. If balanced accuracy drops from roughly 0.77 toward the 0.6 range seen on MDC/TIC, or to chance, the strongest evidence for recipient profiling is an artifact. As a second check, mask proper names and gendered titles (e.g., 'sir', 'ma'am') in MDC and TIC and re-measure; this separates direct-identifier shortcuts from generalizable accommodation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing assumption is that each classifier input is a message, or a concatenation of messages, produced by one author and addressed to the recipient. Section 4 describes SWDA preprocessing as 'concatenated segments from the same conversations until we reached a limit of 1,000 characters.' In a two-party Switchboard conversation, consecutive dialogue acts alternate speakers; unless the implementation groups by speaker, each 1,000-character chunk contains both the author's and the recipient's own turns. The classifier could then infer recipient gender from the recipient's own words, reducing the SWDA experiment to author profiling and invalidating it as evidence for recipient profiling. This is consistent with the numbers: SWDA balanced accuracy is 0.75-0.77, far above MDC (0.59-0.60) and TIC (0.67-0.68). The paper does not report a speaker-side filter for SWDA, and the released dataset/code should be checked. The authors' stated limitation—that direct vs. indirect identifiers were not investigated—is a related but separate issue; the SWDA turn-mixing leak would be a construction artifact rather than a linguistic signal. If this leak is real, the strongest quantitative support for the central claim disappears, leaving only MDC and TIC results, which are themselves not yet separated from direct-identifier shortcuts.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a new NLP task, Recipient Profiling: predicting a sensitive attribute (here, gender) of the intended recipient of a message, as opposed to the author. The authors formalize the task, then provide proof-of-concept experiments on three conversation datasets (SWDA, MDC, TIC) using three pretrained encoders (BERT, MPNet, DeBERTa) fine-tuned with a linear classification head. They report above-chance balanced accuracy on all same-domain settings, with the largest effects on SWDA (0.75–0.77) and TIC (0.67–0.68), and weaker but above-chance cross-dataset transfer (0.51–0.58). The paper also reports a female-vs-male accuracy gap and a between-model agreement analysis. The authors frame the result as evidence that messages leak information about the recipient, raising new privacy concerns.","tokens_in":9392,"tokens_out":3636,"duration_ms":34360,"significance":"If the results hold, the paper introduces a genuinely new task that is a natural counterpart to author profiling and has plausible privacy implications. The authors provide a clean task formalization, use recipient-based held-out splits, report balanced accuracy with standard deviations over three seeds, and release the preprocessed datasets on HuggingFace, which supports reproducibility. The central claim, however, is stronger than the evidence: the above-chance accuracy could be driven by trivial cues (recipient names, titles) or by a preprocessing artifact in SWDA, and the paper explicitly acknowledges that direct vs. indirect identifiers were not separated. The significance therefore hinges on resolving these confounds; as it stands, the paper is a useful proof-of-concept of the task's feasibility, but the privacy-leak interpretation is not yet established.","major_comments":[{"comment":"The preprocessing description states that 'we concatenated segments from the same conversations until we reached a limit of 1,000 characters' without specifying that the concatenation is restricted to a single speaker's turns. In the two-party Switchboard conversations, consecutive dialogue acts normally alternate between the two speakers; if each 1,000-character chunk contains both the author's and the recipient's utterances, then the classifier can infer the recipient's gender from the recipient's own words, turning the SWDA experiment into author profiling rather than recipient profiling. This would also explain why SWDA accuracy (0.75–0.77) is far above MDC (0.59–0.60) and TIC (0.67–0.68). Please clarify whether the concatenation grouped by speaker, and ideally add an experiment that removes the recipient's own turns from the input (or reports results on author-only segments) to confirm that the signal resides in the author's messages.","section":"Section 4 (SWDA preprocessing)"},{"comment":"The conclusion that recipient profiling 'constitutes a new privacy risk' assumes that the predictions are based on indirect linguistic accommodation rather than direct identifiers such as recipient names, pronouns, or titles like 'sir'/'ma'am'. Section 7 explicitly acknowledges that the paper 'did not investigate ... whether their predictions were due to direct or indirect identifiers.' This is a load-bearing gap because direct identifiers would make the task trivial and would not support the stated accommodation-based interpretation. I recommend adding a simple ablation (e.g., masking names and gendered address terms, or comparing against a model trained only on such features) or explicitly rephrasing the privacy claims to reflect that the source of the signal is unknown.","section":"Sections 5.1, 6, and 7"},{"comment":"The cross-dataset transfer claim rests on balanced accuracies in the narrow interval [0.51, 0.58] being 'better than chance,' but no significance tests are reported and the text does not list the standard deviations for the transfer matrix (they appear only in the figure). With three seeds, a 0.51 mean may not be reliably above 0.5. The authors should report confidence intervals or a per-cell permutation/binomial test against chance to support the transferability claim.","section":"Section 5.2"},{"comment":"Throughout the results, the evidence for 'above-chance' performance is limited to means and standard deviations over three seeds. No statistical significance tests are provided for any comparison to chance or between models. Given that the central claims are 'better-than-chance' outcomes, adding a minimal significance test (e.g., a bootstrap or permutation test on the balanced accuracy) would materially strengthen the conclusions.","section":"Section 5 (overall)"}],"minor_comments":[{"comment":"The sentence 'There is two main aspects to our work' should be 'There are two main aspects'.","section":"Section 7"},{"comment":"The split description says '80% of recipients in the train set and 20% distributed in the test and validation sets,' but Table 1 gives 352/18/70 recipients out of 440, which is 80.0% train, 4.1% validation, and 15.9% test; please clarify the exact proportions.","section":"Section 4"},{"comment":"The data link appears as '[data:HF-datasets ]' with an incomplete bracket; ensure the URL and its formatting are correct in the final version.","section":"Abstract and Section 1"},{"comment":"The inline expression for kappa is ambiguous: 'κi,j = Pi,j − Ri,j / 1 − Ri,j' should be properly typeset as (Pi,j − Ri,j)/(1 − Ri,j).","section":"Equation (1)"},{"comment":"The caption contains a typo: 'The models were train and test with in the same domain' should be 'trained and tested in the same domain.'","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The SWDA turn-mixing concern is the most serious risk; if the released preprocessing code confirms that chunks contain both speakers' turns, the paper's strongest quantitative evidence collapses. I would ask the authors to verify the construction and, if necessary, rerun SWDA with speaker-separated inputs before the claim of recipient profiling feasibility is accepted. The direct-identifier confound is acknowledged but remains a major interpretive gap; an ablation removing names and titles should be feasible and would greatly improve the paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper formalizes recipient profiling—predicting a recipient's attributes from messages addressed to them—as a task distinct from author profiling. That framing is genuinely new, and the proof-of-concept idea is reasonable: if writers accommodate to their audience, the recipient's gender should leave traces in the text. The authors also release the datasets, and the same-domain results are consistent across three encoders with small standard deviations. The limitation section is unusually honest, especially the admission that they did not investigate direct versus indirect identifiers.\n\nThe soft spots, though, are real and one is potentially load-bearing. The stress-test note about SWDA is correct on the face of it: the preprocessing concatenates consecutive segments from the same two-party conversation until 1,000 characters, which, without a speaker-side filter, mixes both speakers' turns. The classifier then sees the recipient's own words, turning the SWDA experiment into author profiling. This would explain why SWDA accuracy (0.75–0.77) is far above MDC (0.59–0.60) and TIC (0.68). The paper does not report grouping by speaker, so this must be checked in the released code before the central claim is accepted. If the leak is real, the strongest quantitative support disappears.\n\nEven without the SWDA issue, the MDC and TIC numbers are above chance but modest, and the authors acknowledge they did not control for direct identifiers like names or gendered titles. If the signal is mostly those, the privacy-leak interpretation loses much of its force. The transfer results (0.51–0.58) are close to chance, which the authors honestly describe as a loss in accuracy, but it is still weak evidence of transferable accommodation.\n\nThe paper is worth serious peer review because the task formulation is new and the evaluation, once the SWDA preprocessing is fixed, would be straightforward to redo. The authors' own limitations section shows they are aware of the main threats to validity, which is a good sign. My recommendation: send it to review, but require the authors to verify whether SWDA chunks are speaker-separated, rerun if not, and add a direct-identifier ablation or at least an explicit analysis of what the models rely on. With those changes, the paper could make a solid contribution to privacy and communication-accommodation research.","headline":"New task formulation worth a look; the SWDA evidence may be an artifact, so the feasibility claim rests on MDC and TIC.","tokens_in":9893,"tokens_out":1526,"would_cite":false,"duration_ms":16841,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The gender of a message's intended recipient can be predicted from text alone.","keywords":["recipient profiling","gender prediction","privacy leakage","text classification","language accommodation","transfer learning","conversation datasets","author profiling"],"falsifier":"Mask all direct identifiers in the test messages—recipient names, gendered honorifics and titles, and third-person references to the recipient—then retrain and evaluate the same models. If balanced accuracy drops to chance on all three datasets, the claimed recipient-specific signal beyond direct identifiers is not supported; if it stays above chance, the accommodation signal is real.","tokens_in":8900,"feed_emoji":"💬","tokens_out":6744,"duration_ms":57956,"temperature":0.7,"pith_summary":"The paper introduces 'recipient profiling,' a counterpart to author profiling: instead of inferring what a text reveals about its writer, it asks what a message reveals about the person it was written to. The authors formalize this as a classification task and test it by predicting the recipient's gender from individual messages using fine-tuned pretrained language models. On three very different corpora—telephone conversations, movie dialogues, and post-match tennis interviews—all models beat chance, and models trained on one corpus still beat chance on the others. The authors argue that if this holds, message content is a genuine privacy channel for recipients, not just authors, and that the task can also expose how speakers accommodate their audience.","feed_headline":"Messages predict the recipient's gender, not just the author's","feed_subtitle":"Fine-tuned language models beat chance on three conversation corpora and transfer across domains, opening a new privacy concern.","key_machinery":"The machine that carries the argument is the recipient-profiling classifier $f: \\mathcal{U}_{A\\to R} \\to S$, a map from messages addressed by authors to recipients into a sensitive-attribute space such as gender. Because individual messages are often too short to be informative, the paper concatenates utterances from the same conversation to a minimum length before classification. The empirical machinery consists of fine-tuning three pretrained text encoders—BERT, MPNet, and DeBERTa—with a single linear classification layer, training and testing on recipient-disjoint splits so that no recipient appears in more than one split, and scoring with balanced accuracy. The transfer experiments then ask whether a classifier trained on one corpus still classifies recipients in another, testing whether the learned signal is domain-general.","core_discovery":"The paper claims that messages carry a recoverable signal about the intended recipient, not just the author, and that this signal is strong enough to classify the recipient's gender above chance from text alone. Fine-tuned BERT, MPNet, and DeBERTa models reach balanced accuracies of about 0.77 on Switchboard, 0.60 on the Movie Dialog Corpus, and 0.68 on the Tennis Interviews Corpus, and models trained on one corpus exceed chance (0.51–0.58) on the other two. The authors take this as evidence that recipient profiling is feasible and that it constitutes a new privacy risk inherent in textual messages, one that runs parallel to the better-studied risk of author profiling.","pith_inferences":["The paper leaves untested whether the signal comes from direct identifiers (names, 'sir'/'ma'am') or from indirect stylistic accommodation; masking those direct identifiers and retraining would show which channel drives the result.","The higher accuracy for female recipients is left unexplained; comparing messages whose topics are held fixed across recipient genders would indicate whether the asymmetry is about the recipient or about topic conventions.","Transfer accuracies sit only slightly above chance, so the practical privacy risk may be domain-dependent; a realistic attacker evaluation would need messages from unseen communication channels, not just held-out recipients from the same corpus."],"forward_implications":["Recipient gender classification is feasible on at least three distinct conversation genres, so recipient profiling can be treated as a measurable task rather than a hypothetical one.","Because models transfer across datasets at above-chance accuracy, at least part of the signal generalizes beyond a single corpus's vocabulary or topic distribution.","Message collections that include recipient metadata now count as privacy-relevant in the same way author-labeled text does, and recipient-side anonymization becomes a concrete design target.","Recipient-profiling models offer a quantitative probe for communication accommodation theory, turning a documented stylistic phenomenon into a testable predictor."],"supporting_citations":[{"why":"Provides the Switchboard phone-conversation corpus, the highest-accuracy in-domain test set.","marker":"(Stolcke et al., 2000)"},{"why":"Provides the Movie Dialog Corpus, the largest and lowest-accuracy test set.","marker":"(Danescu-Niculescu-Mizil and Lee, 2011)"},{"why":"Provides the Tennis Interviews Corpus and the precedent that journalists' questions differ by athlete gender.","marker":"(Fu et al., 2016)"},{"why":"Supplies BERT, one of the three fine-tuned encoders.","marker":"(Devlin et al., 2019)"},{"why":"Supplies MPNet, one of the three fine-tuned encoders.","marker":"(Song et al., 2020)"},{"why":"Supplies DeBERTa, one of the three fine-tuned encoders.","marker":"(He et al., 2021)"},{"why":"Defines balanced accuracy, the evaluation metric used for all comparisons.","marker":"(Brodersen et al., 2010)"},{"why":"Supplies communication accommodation theory, the mechanism the paper invokes for indirect recipient signals.","marker":"(Giles and Ogay, 2007)"}],"fun_headline_variants":["Messages leak the recipient's gender, not just the author's","Recipient profiling: your texts reveal who you're writing to","Chats betray the recipient's traits, study shows","Texts unwittingly expose the intended recipient's gender","Predicting the recipient from the message: a new privacy risk"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The above-chance accuracy is evidence about recipients only if it comes from how people write to different recipients, not from direct identifiers such as recipient names, 'sir'/'ma'am', or other labels already present in the message; the paper states that it did not check this distinction.","fun_headline_variants_meta":{"raw":{"variants":["Messages leak the recipient's gender, not just the author's","Recipient profiling: your texts reveal who you're writing to","Chats betray the recipient's traits, study shows","Texts unwittingly expose the intended recipient's gender","Predicting the recipient from the message: a new privacy risk"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000499,"raw_usage":{"total_tokens":2390,"prompt_tokens":841,"completion_tokens":1549,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":457,"completion_tokens_details":{"reasoning_tokens":1466}},"tokens_in":457,"tokens_out":1549,"duration_ms":9980,"temperature":1.0,"reasoning_tokens":1466,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:32:55.847296+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Mask all direct identifiers in the test messages—recipient names, gendered honorifics and titles, and third-person references to the recipient—then retrain and evaluate the same models. If balanced accuracy drops to chance on all three datasets, the claimed recipient-specific signal beyond direct identifiers is not supported; if it stays above chance, the accommodation signal is real.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies communication accommodation theory, the mechanism the paper invokes for indirect recipient signals."}],"review_version":1}