{"id":"caa5b297-b052-42b0-8b09-18c4e0904d6b","arxiv_id":"2507.16860","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"GPT-assisted adversarial retraining restores LinkedIn fake-profile detectors from a 42-52 percent false accept rate on AI-written profiles to 1-7 percent.","lead":"The paper evaluates how well existing LinkedIn fake-profile detectors handle profiles written by AI, and finds they miss 42-52 percent of AI-written fakes. It then shows that retraining detectors on AI-written examples cuts the miss rate to 1-7 percent, while still accepting real profiles.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Adversarial training is tested only on profiles from the same OpenAI model family used to generate training data; the headline FAR 1-7% likely overstates robustness to other LLMs.","rationale":"The reader's weakest_assumption identifies the same load-bearing issue: all training and testing of the adversarial countermeasure uses GPT-family profiles. This is not a minor technicality; it directly affects the abstract's headline claim. The paper itself flags the limitation in Section 5, so the authors are aware of it, but the abstract and conclusion still present the 1-7% FAR as a general robustness result. The internal evidence strengthens the concern: GPT3.5-assisted training fails to generalize to GPT4Ps and vice versa, showing that even within the OpenAI family, adversarial training does not automatically transfer across model versions. Therefore, cross-family transfer is genuinely uncertain. My proposed concrete test—evaluating on non-OpenAI LLM profiles—would settle whether the concern actually lands. If it passes, the paper's central claim holds under a broader attack distribution; if it fails, the correct scientific claim is that adversarial training works for same-family attacks. I agree with the reader's conditional verdict: the paper is valuable and internally consistent, but the overgeneralization in the abstract needs to be fixed or empirically supported. No ad hominem is intended; this is a standard distribution-shift concern. The paper's strengths—reproducible code, clear experimental setup, and explicit limitation—are real, but they do not remove the need for an out-of-family test.","tokens_in":529,"tokens_out":2407,"duration_ms":67956,"concrete_test":"Generate a held-out set of 600 fake profiles using non-OpenAI LLMs (e.g., Claude, Gemini, Llama) with the same prompt templates used for GPT3.5Ps/GPT4Ps, then evaluate the best combined-trained Flair+XGBoost model on these profiles. If the FAR remains below 7%, the concern is resolved; if FAR rises above 30%, the central claim should be qualified to 'same-family' OpenAI models only.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that GPT-assisted adversarial training restores the False Accept Rate to 1-7% with negligible impact on FRR. This claim is load-bearing and depends on the assumption that the detector's robustness transfers beyond the specific LLM family used for training. The paper's own Section 5 acknowledges this: 'both the creation of fake profiles for generating attack vectors and adversarial training used models from the same LLM family (OpenAI GPT).' The empirical evidence inside the paper already shows that robustness is narrow even within that family: GPT3.5-assisted training reduces FAR on GPT3.5Ps to ~2%, but leaves FAR on GPT4Ps at 16.9-19.3%; GPT4-assisted training does the reverse. Thus the reported 1-7% FAR is an in-distribution result, not evidence of general robustness. If deployed attackers use different LLMs (e.g., Claude, Gemini, Llama), the detector's decision boundary may not transfer because profile style, structure, and artifact patterns can differ across model families. The abstract's unqualified phrasing, 'restoring the False Accept Rate to between 1-7%,' therefore overstates the practical guarantee. This is the weakest assumption in the argument.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper investigates the robustness of text-based fake profile detectors on LinkedIn against fake profiles generated by LLMs. The authors augment a public dataset with 1,200 GPT-3.5 and 600 GPT-4 generated profiles, extract Section Tag Embeddings (STE) from four transformer encoders plus 17 numerical features, and evaluate six classifiers, focusing on XGBoost and CatBoost. Baseline models perform well on manual fakes (F1~97%, FAR 6-7%) but degrade sharply on GPT-generated profiles (FAR up to 52%). The proposed GPT-assisted adversarial training, especially with combined GPT-3.5+GPT-4 generated profiles, restores FAR to 1.3-2.6% and F1 above 97.5% on held-out profiles from the same model family, while keeping FRR below 2.5%. The paper also benchmarks human and GPT-4 evaluators, showing both underperform the trained detectors, and reports ablation results on text-only, numerical-only, and fused features.","tokens_in":8819,"tokens_out":5665,"duration_ms":55461,"significance":"If the reported results hold beyond the evaluated model family, the paper makes a practical contribution: a simple retraining strategy that uses LLM-generated profiles to restore detector accuracy, along with a publicly available dataset extension and an analysis of feature modality robustness. The paper's strengths include the release of code and data, a careful hyperparameter tuning procedure, calibration analysis, and the explicit recognition in Section 5 that the attack and defense are from the same LLM family. The within-family asymmetry (GPT3.5-trained detectors leave GPT4P FAR at 16.9-19.3%; GPT4-trained detectors leave GPT3.5P FAR at 36-38%) is honestly reported in Figure 2 and is important evidence that the headline 1-7% FAR is an in-distribution result, not a demonstrated property across LLM families.","major_comments":[{"comment":"The central claim 'restoring the False Accept Rate to between 1-7%' is supported only for test profiles generated by the same OpenAI GPT model family used for adversarial training. The paper's own results in Figure 2(b) show that GPT3.5-assisted training leaves GPT4P FAR at 16.9-19.3%, and GPT4-assisted training leaves GPT3.5P FAR at 36.1-38.5%; only the combined GPT3.5+4 training closes the gap, and only on OpenAI-generated profiles. Since Section 5 acknowledges that the system has not been evaluated against other LLM families, the abstract and conclusion should qualify the 1-7% FAR as an in-distribution result, and ideally the authors should add a transfer experiment with profiles from a different LLM family (e.g., Gemini, Claude, Llama) before claiming practical robustness.","section":"Section 5 (third limitation); Section 4.2 and Figure 2(b)"},{"comment":"All key results are reported as point estimates without confidence intervals, standard deviations, or repeated runs. The selection of Flair+XGBoost as the best setup rests on differences of a few tenths of a percent (e.g., F1 98.2 vs 98.0, FAR 1.34 vs 1.5), which may be within experimental noise. Please provide variance estimates (e.g., multiple seeds or bootstrap) for the headline FAR/FRR comparisons in Figure 2, or explicitly state that these differences are not statistically evaluated.","section":"Section 4.2; Figure 2"},{"comment":"The claim of outperforming Ayoobi et al. [4] compares the authors' STE models with numbers reported in that paper, rather than with a controlled re-run using the same train/test splits and preprocessing. The paper does not specify whether the original dataset splits were preserved after augmenting the data. Please clarify the comparison protocol or provide a matched re-implementation, since the reported improvement could be an artifact of different splits or feature preprocessing.","section":"Section 4.2 (baseline comparison)"}],"minor_comments":[{"comment":"The composition of the 360 profiles given to GPT-4 (how many FLPs vs LLM-generated fakes) and the selection of the 15 profiles used for human evaluation are not specified; please state the sampling procedure and class mix.","section":"Section 3.5"},{"comment":"The sentence 'PCA was applied to reduce embeddings to 150 dimensions from 786' does not match the varying dimensionalities of the encoders (e.g., Flair is typically 1024-dimensional); please report the input dimension for each encoder.","section":"Section 3.2"},{"comment":"The matrix-style display is dense; please add axis labels to the heatmaps and consider moving the full numeric table for all encoder/classifier combinations to an appendix.","section":"Figure 2"},{"comment":"The phrase 'LLM-assited' should be 'LLM-assisted'.","section":"Section 2"},{"comment":"Reference [32] points to an anonymous p5js editor URL, which is not a stable archival source; please provide a persistent link or institutional page.","section":"References"},{"comment":"The abstract reports FAR '1-7%' while Section 4.2 reports 1.34-2.6% and Section 5 reports 1.34-2.28%; please harmonize these ranges and ensure the chosen range matches the presented results.","section":"Abstract and Section 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the journal's scope and the same-LLM-family limitation is disclosed, which I appreciate. However, the abstract's unqualified FAR claim is likely to mislead readers. I recommend requesting a cross-family evaluation or, failing that, a substantially qualified abstract. The lack of uncertainty quantification is also worth addressing before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is a straightforward, honest empirical paper. The authors show that existing LinkedIn fake-profile detectors that work well on manually crafted fakes (FAR ~6–7%) fall apart on GPT-generated profiles (FAR 42–52%), and that GPT-assisted adversarial training can restore FAR to ~1–3% on the tested attacks. That is a real result, and the dataset augmentation plus the code release make it reproducible.\n\nWhat's actually new: the vulnerability was noted before, but here they quantify it across GPT-3.5 and GPT-4 and evaluate a countermeasure. The ablation across textual, numerical, and fused features is useful: numerical features are sturdier under attack, and fusion is best. The benchmarking of human annotators and GPT-4 as detectors is a nice complement, if anecdotal.\n\nThe soft spots are not fatal, but they are real. The headline FAR 1–7% is measured on test profiles generated by the same OpenAI model family used for adversarial training. The paper's own Section 5 says this, but the abstract doesn't qualify it. Their internal data actually shows that single-model training does not generalize across GPT-3.5 to GPT-4 (FAR ~20% or ~36% on the untrained model). Combined training handles both, but that is still OpenAI-only. If attackers use Claude or Llama, we have no evidence the detector holds up. This should be either fixed with a new experiment on a non-OpenAI LLM, or the claims should be reworded to 'GPT-generated profiles.'\n\nOther issues: no error bars or repeated runs, so we can't tell whether the 1.3% vs 2.6% differences are noise. The comparison to Ayoobi et al. uses their reported numbers rather than re-running their model, which is a bit soft. The human evaluation is small (30 annotators, 15 profiles each) but it's clearly a secondary benchmark.\n\nThe central claim, for the distribution tested, holds up. This paper deserves a serious referee. I'd recommend peer review, and the main revision is to calibrate the generality claims and ideally add cross-family LLM experiments.","headline":"Useful, honest empirical study of LLM-generated LinkedIn fake profiles; the countermeasure works in-distribution, but the abstract overstates cross-LLM generality.","tokens_in":9371,"tokens_out":3167,"would_cite":true,"duration_ms":31626,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Existing LinkedIn detectors accept GPT-generated fake profiles 42-52% of the time, and adversarial training on GPT-generated profiles restores the false accept rate to 1-7%.","keywords":["fake profile detection","large language models","adversarial training","LinkedIn","GPT-4","Section Tag Embeddings","false accept rate","profile authenticity"],"falsifier":"Generate a fresh test set of fake LinkedIn profiles with non-GPT LLMs such as Gemini, Claude, or Llama, or with humans editing GPT output, and run the Flair+XGBoost detector trained on the combined GPT3.5+4 data; if its false accept rate rises well above the reported 1-7%, the robustness claim holds only within the GPT model family rather than for LLM-generated fakes in general.","tokens_in":8380,"feed_emoji":"🤖","tokens_out":9409,"duration_ms":91008,"temperature":0.7,"pith_summary":"This paper tries to establish that text-based LinkedIn fake-profile detectors have a specific weak point: they catch manually written fakes (false accept rate 6-7%) but let GPT-3.5- and GPT-4-written profiles through as legitimate 42-52% of the time. It argues that the fix is GPT-assisted adversarial training, adding LLM-generated profiles to the training data, which restores the false accept rate to 1-7% while keeping false reject rates around 1.5-2.4%. The study also claims that fusing numerical profile features with text embeddings gives the most robust detector, that numerical features alone are sturdier against attack than text alone, and that neither human evaluators (F1 58.9%) nor prompted GPT-4 (F1 up to 85.7%) match the retrained automated detectors.","feed_headline":"GPT-written fake profiles fool LinkedIn detectors 42-52% of the time","feed_subtitle":"Adversarial training with GPT-generated profiles cuts the false accept rate back to 1-7% without hurting genuine users.","key_machinery":"The mechanism is GPT-assisted adversarial training: synthetic fake profiles generated with GPT-3.5 and GPT-4-Turbo are mixed into the training set alongside genuine profiles and manually created fakes, forcing the boosted tree classifiers to learn the statistical footprint of LLM-written profiles. The input representation is the Section Tag Embedding (STE), computed as the mean over a profile's sections of the difference between the section-text embedding and the section-tag embedding, then reduced with PCA to 150 dimensions and concatenated with 17 normalized numerical features into a 167-dimensional vector. The classifiers that carry the result are gradient-boosted trees, specifically XGBoost and CatBoost. The combined GPT3.5+4 training mixture is the part that yields cross-variant generalisation; single-family training leaves the other GPT variant's false accept rate elevated in double digits.","core_discovery":"The central claim is that the vulnerability created by LLM-generated fake profiles is largely a training-data gap: detectors trained only on genuine profiles and manual fakes fail on GPT profiles because those profiles closely mimic real ones, with a mean textual similarity of 88.9%. Adding GPT-3.5- and GPT-4-generated profiles to training closes the gap. Training on either family alone reduces that family's false accept rate but leaves the other family elevated; training on the combined GPT3.5+4 set yields false accept rates between 1.34% and 2.6% on all attacks, F1 scores above 97.5%, and false reject rates between 1.48% and 2.41%, without sacrificing performance on genuine profiles. The best performing configuration pairs Flair text embeddings with XGBoost. Feature-removal experiments show that numerical profile features resist attack better than text features alone, while combining the two types is the strongest configuration.","pith_inferences":["Because the attack profiles and the adversarial training data were generated by the same GPT-3.5/GPT-4 family, the restored 1-7% false accept rate is best read as a same-family result; an attacker using a different LLM family could still evade, and this transfer is the paper's own stated limitation.","The reported correlation between calibration and false accept rate suggests a practical drift alarm: platforms could track the Brier score of their detector and trigger retraining when calibration worsens, without waiting for confirmed fake-profile labels.","As legitimate users increasingly use LLMs to write or polish their own profiles, the assumption that 'looks LLM-written' equals 'fake' will erode; detectors will likely need to shift from stylistic signals toward behavioural or relational signals, a consequence the paper leaves implicit."],"forward_implications":["Platforms can restore robustness by periodically retraining detectors on LLM-generated profiles rather than relying only on datasets of manually created fakes.","Keeping structural numerical features such as counts of jobs, education entries, skills, and connections matters, because text-only models degrade most sharply under LLM attack.","Task-specifically trained automated detectors currently outperform both human moderators and general-purpose LLM prompting on this detection task.","Robustness does not fully generalise across GPT versions, so training data should combine multiple LLM generations; training on only one variant leaves the other with double-digit false accept rates.","Calibration quality tracks robustness: lower Brier scores co-occur with lower false accept rates, so monitoring calibration can flag when a detector is losing its edge against new synthetic profiles."],"supporting_citations":[{"why":"supplies the original LinkedIn genuine and manually created fake profiles, plus the STE/SSTE baseline that the study attacks and improves.","marker":"[4]"},{"why":"documents the near-doubling of LinkedIn fake profiles that motivates the threat model.","marker":"[5]"},{"why":"is the companion code and data release containing the full feature, embedding, and training details.","marker":"[6]"},{"why":"hosts the GPT-4-Turbo prompts and similarity-based quality checks used to create the 600 added synthetic profiles.","marker":"[16]"},{"why":"provides BERT, one of the encoders tested for building Section Tag Embeddings.","marker":"[17]"},{"why":"provides Flair, the encoder whose embeddings produced the best-performing detector configuration.","marker":"[21]"},{"why":"provides XGBoost, the classifier used in the best-performing setup.","marker":"[25]"},{"why":"provides the Brier score used to measure calibration and to quantify the calibration-robustness correlation.","marker":"[31]"}],"fun_headline_variants":["GPT fakes pass LinkedIn's detector 42-52% of the time","LinkedIn's fake-profile check fails on GPT-written bios—retraining fixes it","Train on GPT lies: LinkedIn fake profile acceptance drops to as low as 1.3%","Blocking LLM fake profiles: combining profile stats and text works best","LLM-generated profiles expose LinkedIn's text-detector blind spot"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the fake profiles used to train the countermeasure are representative of the fake profiles an attacker will actually deploy, since both the attack vectors and the adversarial training data came from the same GPT-3.5/GPT-4 model family; if real-world fakes are written by other LLMs or edited by humans, the restored detection rates may not transfer.","fun_headline_variants_meta":{"raw":{"variants":["GPT fakes pass LinkedIn's detector 42-52% of the time","LinkedIn's fake-profile check fails on GPT-written bios—retraining fixes it","Train on GPT lies: LinkedIn fake profile acceptance drops to as low as 1.3%","Blocking LLM fake profiles: combining profile stats and text works best","LLM-generated profiles expose LinkedIn's text-detector blind spot"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000425,"raw_usage":{"total_tokens":2160,"prompt_tokens":909,"completion_tokens":1251,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":525,"completion_tokens_details":{"reasoning_tokens":1149}},"tokens_in":525,"tokens_out":1251,"duration_ms":13037,"temperature":1.0,"reasoning_tokens":1149,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T15:23:37.184724+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate a fresh test set of fake LinkedIn profiles with non-GPT LLMs such as Gemini, Claude, or Llama, or with humans editing GPT output, and run the Flair+XGBoost detector trained on the combined GPT3.5+4 data; if its false accept rate rises well above the reported 1-7%, the robustness claim holds only within the GPT model family rather than for LLM-generated fakes in general.","supporting_citations":[{"cited_title":"The looming threat of fake and llm- generated linkedin profiles: Challenges and opportunities for detection and preven- tion,","cited_arxiv_id":null,"evidence_quote":"supplies the original LinkedIn genuine and manually created fake profiles, plus the STE/SSTE baseline that the study attacks and improves."},{"cited_title":"Fake accounts are getting way more common on linkedin (data),","cited_arxiv_id":null,"evidence_quote":"documents the near-doubling of LinkedIn fake profiles that motivates the threat model."},{"cited_title":"Linkedin fake profile detection,","cited_arxiv_id":null,"evidence_quote":"is the companion code and data release containing the full feature, embedding, and training details."},{"cited_title":"Available: https://sites.google.com/view/weaklinksinlinkedin","cited_arxiv_id":null,"evidence_quote":"hosts the GPT-4-Turbo prompts and similarity-based quality checks used to create the 600 added synthetic profiles."},{"cited_title":"Unmasking fake social network accounts with explainable intelligence,","cited_arxiv_id":null,"evidence_quote":"provides BERT, one of the encoders tested for building Section Tag Embeddings."},{"cited_title":"Deberta: Decoding-enhanced bert with disentangled attention,","cited_arxiv_id":null,"evidence_quote":"provides Flair, the encoder whose embeddings produced the best-performing detector configuration."},{"cited_title":"Optimizing deep learning hyperparameters through an evolutionary algorithm,","cited_arxiv_id":null,"evidence_quote":"provides the Brier score used to measure calibration and to quantify the calibration-robustness correlation."}],"review_version":1}