{"id":"fb53d34f-bda9-48f4-8021-26c1c9d88468","arxiv_id":"2411.10683","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"Seven of 27 tested LLMs (25.93%) exhibited identity confusion, which the authors link to hallucination and show reduces user trust, especially in critical tasks.","lead":"A study of 27 large language models found that seven, about 26 percent, misstated their own identity, for example claiming to be another company's model. The authors attribute this to hallucination rather than copying, and a 208-person survey shows such mistakes erode user trust more than ordinary errors.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The causal claim that identity confusion stems from hallucination rather than reuse rests on an unvalidated output-similarity test that cannot distinguish independent models from fine-tuned or distilled derivatives; the paper's own data show cross-model pairs as close as same-model versions.","rationale":"The reader's weakest_assumption correctly identifies the same load-bearing point. I agree: §5.3's inference from POS-distribution divergence to 'distinct models' and hence hallucination is invalid. The paper compares different versions of the same model (GPT-2 variants) and different companies' models, but it never tests a known derivative pair. A fine-tuned/distilled model can have a different POS profile on HC3 while sharing weights, training data, or generation behavior with another model. Figure 4 even contains a counterexample to the claimed separation: DeepSeek vs. Yi-34b has the same Euclidean distance (0.9) as GPT2-xl vs. GPT2-large. Figure 5's radar chart compares only cross-model pairs, not same-model baselines, so it cannot show that divergence indicates independence. The prevalence result also has internal inconsistencies and no validated classifier, but those affect RQ1; the causal claim (RQ2) is the more fundamental failure because it is stated as 'confirmed' in the abstract and conclusion. A positive-control experiment with known derived pairs would settle whether the method has any power to distinguish derivation from independence. If it does not, the paper should be revised to report identity confusion as an observed phenomenon without attributing a cause. For these reasons, the current verdict should remain REJECT.","tokens_in":25218,"tokens_out":5966,"duration_ms":63688,"concrete_test":"Run the same P-III POS-distance analysis on known derived pairs (e.g., LLaMA-2-chat vs. LLaMA-2 base, or a fine-tuned model and its teacher) across the HC3 subsets used in Figure 5. If any known-derived pair shows cross-subset divergence comparable to the identity-confusion pairs (e.g., DeepSeek vs. Yi-34b), then the §5.3 inference is invalid; if all known-derived pairs remain tightly clustered while confused-model pairs diverge, the hallucination conclusion is supported.","verdict_should_be":"REJECT","load_bearing_attack":"The central causal assertion—that identity confusion stems from hallucination rather than reuse (abstract, §5.3, Finding (III))—rests entirely on the claim that confused models have 'significantly divergent output distributions' and are therefore distinct models. The evidence for this is Euclidean distance between part-of-speech frequency vectors on HC3 subsets (§4.1, Eq. 1). POS frequencies are a coarse aggregate: a fine-tuned or distilled model can easily diverge from its teacher or base model on a single dataset while still being derived from it. The method has no positive control: no known fine-tuned/distilled pair is shown to be detectable as similar. The paper's own numbers undermine the separation: DeepSeek vs. Yi-34b has distance 0.9 in Figure 4, the same as GPT2-xl vs. GPT2-large; the radar-chart analysis (Figure 5) uses only cross-model pairs and no same-model controls, so it cannot establish that divergence implies independence. A second, compounding issue is that the prevalence measurement uses an unvalidated LLM classifier with internally inconsistent counts (§5.2 says 2/12 proprietary models affected, Table 4 says 3/12; dataset counts sum to 30 of 27), so RQ1 is also not robust. The load-bearing flaw is the RQ2 inference: divergence of POS distributions on one benchmark cannot rule out reuse, plagiarism, or derivation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces the concept of \"identity confusion\" in LLMs, proposes a six-category taxonomy (C1–C6), and reports a measurement study of 27 models. The authors use an LLM-assisted pipeline consisting of documentation analysis, self-identity recognition questions, and output-distribution comparison based on part-of-speech (POS) frequencies on HC3 subsets. They claim that 25.93% of evaluated models exhibit identity confusion and that output-similarity analysis shows the cause is hallucination rather than model reuse or replication. A Credamo survey with 208 valid respondents is used to argue that identity confusion erodes user trust more than logical errors or response inconsistencies, especially for critical tasks.","tokens_in":25490,"tokens_out":5321,"duration_ms":53707,"significance":"The phenomenon is timely, and the taxonomy separating self-identification, reference, capabilities, profile, relationship, and creation confusion is a useful organizing device. The paper also deserves credit for assembling a diverse set of 27 models and for attempting a multi-phase measurement rather than relying only on anecdotal examples. However, the two headline quantitative claims are not supported by the evidence as presented. The prevalence figure depends on an unvalidated LLM-based classifier, and the counts inside §5.2 and Table 4 are inconsistent. The causal claim that identity confusion stems from hallucination rather than reuse rests on a POS-distance test that cannot distinguish independent models from fine-tuned or distilled derivatives, and the paper's own distance matrix shows cross-model pairs as close as same-model versions. As a measurement paper, the current experimental design does not establish the central findings.","major_comments":[{"comment":"The 25.93% prevalence figure rests on an LLM-based response classifier that is never validated: no precision/recall on a gold-standard set, no human agreement study, and no error analysis are reported. This section also contains internally inconsistent counts: the text reports 7 affected models and states that 2 out of 12 proprietary models are affected, while Table 4 reports 3 out of 12 proprietary; the dataset rows sum to 30 (16 public plus 14 private) although only 27 models were evaluated; and §5.3 later refers to \"six LLMs\" exhibiting identity confusion. The paper also excludes Hailuo AI in §6 after listing it in the 27-model set, without reconciling how an excluded affected model affects the denominator. These inconsistencies make the RQ1 result unreliable as a quantitative claim.","section":"§5.2 / §4.1 (P-II) / Table 4"},{"comment":"The causal conclusion that identity confusion is due to hallucination rather than replication or reuse rests on the claim that confused models are distinct because their POS frequency vectors diverge on HC3 subsets. This inference is invalid: a fine-tuned or distilled model can diverge from its base or teacher on a single dataset subset while still being derived from it. There is no positive control showing that any known derived pair is detected as similar by the POS-distance test. Moreover, Figure 4 shows the DeepSeek–Yi-34b distance (0.9) is identical to the GPT2-xl–GPT2-large distance (0.9), so the data do not separate cross-model pairs from same-series pairs. The radar chart in Figure 5 uses only cross-model pairs and contains no same-model controls. The evidence therefore cannot rule out reuse, plagiarism, or derivation.","section":"§5.3 / §4.1 (P-III) / Eq. (1)"},{"comment":"The architecture and dataset categorization, which feeds the breakdowns in Table 4 and the discussion in §6, is produced by an LLM reading technical documentation, with no reported human validation of the extracted categories. The paper says manual reviews confirmed document provenance but does not say that the architecture/dataset labels were manually verified. Without validation or error analysis, the 30%-vs-0% difference between architecture classes and the public/private dataset comparison are not supported.","section":"§4.1 (P-I) / Table 3 / Table 4"},{"comment":"The survey results are presented as percentage changes without any statistical testing, confidence intervals, or effect sizes. The relationship between the \"Initial\" column and the per-scenario counts is not fully reconciled (e.g., Personal Entertainment shows +30.8% for C3 while the TOTAL row declines), and the claim that identity confusion erodes trust more than Fault I and Fault II needs paired significance tests. As presented, the reported differences could be within sampling variability, especially for the smaller task subgroups such as Professional Q&A.","section":"§5.4 / Table 5"}],"minor_comments":[{"comment":"The text refers to \"Figure 5.2\" and \"results are presented in Figure 5.2\", but the referenced figure is numbered Figure 5; the cross-reference should be corrected.","section":"§5.3 / Figure 5"},{"comment":"The paper says Hailuo AI was excluded from evaluation due to lack of an API, but Table 3 still lists it among the 27 models and Figure 3 includes it in the identity-confusion breakdown; clarify exactly which analyses include Hailuo AI and how this affects the reported prevalence.","section":"§6 / Table 3"},{"comment":"The survey appendix lists only the questions and answer options; the manuscript does not report the full scenario texts or the order in which Fault I, Fault II, and C1–C6 were presented, which would be needed to assess possible ordering effects.","section":"§4.2 / Appendix"},{"comment":"Some motivating examples are cited to informal sources such as Hacker News threads and community forum posts; for a measurement paper, primary sources or archived versions would strengthen the reproducibility of the motivating claims.","section":"§3.1 / References"},{"comment":"The text says \"2 out of 12 (17%)\" but 2/12 is 16.7%; this is a minor rounding issue, but it contributes to the impression that the numerical reporting in this section is not carefully checked.","section":"§5.2"}],"recommendation":"reject","confidential_remarks":"The paper's fit to cs.CR is somewhat loose, as much of the content is an NLP measurement and a trust survey. More importantly, the primary result rests on an unvalidated classifier and on an invalid inference from POS-output divergence to model independence. Even a substantial revision would require new data collection, human validation of the classifier, and positive controls for the fingerprinting method, rather than local fixes to the current experiments. I therefore recommend rejection rather than major revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You asked about the identity confusion paper. Short version: it's a real topic, the taxonomy is handy, and the survey is a useful first cut, but the paper's signature quantitative claims don't survive contact with the methods.\n\nWhat's new and worth keeping: they ran a systematic sweep of 27 models with a consistent set of identity prompts, and they organized the failures into six categories. That alone is more than the anecdote pile we had before. The user survey (208 valid responses) gives a plausible picture: identity confusion hurts trust more than a logical slip, and people read it as a design/training failure. The finding that some previously confused models (Spark) got fixed is interesting.\n\nNow the soft spots. First, the prevalence number is built on an LLM classifier that is never validated against human labels, and the paper's own counts disagree with each other: 7 in §5.2, 6 in §5.3, and Table 4 foots to 8 (5 open-source + 3 proprietary). That's a factual inconsistency, not a style quibble.\n\nSecond, the causal story in RQ2 is not supported. Showing that two models have different POS distributions on one HC3 subset does not rule out derivation; a fine-tune or distilled model can easily diverge on a single benchmark. The paper's own heatmap undercuts the claim: DeepSeek vs Yi-34b sits at distance 0.9, the same as GPT2-xl vs GPT2-large. The radar-chart follow-up has no same-model or known-derived-pair controls, so it can't rescue the inference. \"Hallucination rather than replication\" is a reasonable conjecture, but the paper presents it as a confirmed result.\n\nThird, the survey analysis is purely descriptive. No confidence intervals, no significance tests, so the \"outperforms logical errors\" claim is a claim, not a demonstrated difference.\n\nI agree with the reader's reject call, but this isn't a desk-reject paper. The phenomenon is measurable, the taxonomy is reusable, and the survey questions are a decent starting point. With a validated classifier, a cleaned-up count, and an honest re-framing of RQ2 as \"we cannot distinguish reuse from hallucination using this method,\" it could be a solid measurement paper. I'd send it to review with a request for major revision, mainly to force the authors to fix the count and temper the causal language.","headline":"Useful first measurement of LLM identity confusion, but the prevalence count is internally inconsistent and the hallucination-vs-reuse conclusion isn't supported by the output-similarity analysis.","tokens_in":26006,"tokens_out":3464,"would_cite":false,"duration_ms":35819,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that about 26 percent of tested large language models misstate their own identity, that this is a hallucination rather than evidence of model copying, and that the error erodes user trust more than logical mistakes do.","keywords":["identity confusion","LLM fingerprinting","hallucination","user trust","output distribution analysis","LLM security","self-identification"],"falsifier":"Show a single identity-confused model pair (for example, a model that claims to be ChatGPT) that, when tested on several different prompt subsets rather than one, produces output distributions close to the claimed model on at least one subset; or find memorized training text shared between the two models. Either observation would break the inference that divergence on one dataset rules out reuse.","tokens_in":25012,"feed_emoji":"🎭","tokens_out":4499,"duration_ms":41044,"temperature":0.7,"pith_summary":"This paper defines and measures 'identity confusion' in large language models: a model that misstates its own name, creator, affiliated company, or capabilities. Testing 27 LLMs with an automated pipeline of document analysis, identity-recognition questions, and output-similarity checks, the authors find that 7 models (25.93%) show the behavior. They argue that the cause is hallucination, not reuse of another model's code or weights, because identity-confused models still produce output distributions that are clearly distinct from the models they claim to be. A 208-person survey shows the failure erodes user trust more than logical errors or inconsistent answers, especially for education and professional work. The paper's point is that identity confusion is a measurable, common, and trust-relevant failure mode that developers can fix with targeted fine-tuning.","feed_headline":"1 in 4 LLMs misidentifies itself in tests","feed_subtitle":"The errors look like hallucinations, not copying, and they hurt user trust more than wrong math does.","key_machinery":"The argument is carried by a three-phase measurement pipeline. Phase one collects and classifies each model's official documentation into architecture and dataset categories. Phase two tests self-identity recognition with 77 reformulated questions targeting six confusion types (self-identification, reference, capabilities, profile, relationship, and creation confusion). Phase three compares part-of-speech frequency distributions of outputs using Euclidean distance, cosine similarity, and Jaccard similarity to fingerprint models. The fingerprinting logic is the load-bearing step: if two different models are actually distinct, their output distributions on the same prompts should diverge, and versions of the same model should converge; the authors use the HC3 dataset as the common prompt set and show exactly this pattern.","core_discovery":"The paper's central claim is that identity confusion is a real and measurable failure mode of LLMs, that it is primarily a hallucination phenomenon rather than evidence of one model being a copy of another, and that it damages user trust out of proportion to its severity as a technical error. The authors report that 25.93% of 27 tested LLMs exhibited at least one of six types of identity confusion, with creation confusion (claiming the wrong creator) being the most common at 63.13% of incidents. They support the hallucination conclusion by showing that models which admit to being a different product still have part-of-speech output distributions that are far apart from the distribution of the model they claim to be, whereas different versions of the same model are close. Their survey data indicate that trust drops by more than 35% across all six confusion types, with the sharpest declines in educational and professional tasks—declines larger than those caused by logical or consistency errors.","pith_inferences":["If hallucination is the root cause, then general hallucination-reduction techniques—retrieval grounding, reinforcement learning, or constrained decoding—should also reduce identity confusion, a prediction the paper does not test directly.","The survey's attribution result (29.81% of users suspect plagiarism) implies that even innocent developers face reputational damage similar to that suffered by actual copycats, so identity confusion is a brand-risk problem as much as a technical bug.","A natural extension would be to measure identity confusion in multimodal or agentic LLMs, where identity is expressed through tool use and actions rather than only text.","The output-distribution method could be sharpened by replacing part-of-speech counts with semantic embeddings, which would better separate stylistic similarity from content reuse."],"forward_implications":["Identity confusion can be treated as a distinct defect category, separate from factual errors and inconsistency, with its own remediation path in fine-tuning.","Fine-tuned models in the sample showed zero identity confusion, suggesting that explicit identity training during fine-tuning is an effective mitigation.","Because creation confusion dominates (63.13%), developers should prioritize identity statements in the model's own voice during post-training.","Reference confusion (C2) caused the largest trust decline, so providing wrong API links or docs is the most reputationally damaging subtype.","The iFLYTEK Spark case shows the issue can be fixed quickly; a model that previously confused itself with ChatGPT passed all identity questions in this evaluation."],"supporting_citations":[{"why":"Supplies the HC3 dataset used to elicit model outputs for the distribution-similarity analysis that distinguishes same-model consistency from cross-model divergence.","marker":"[6]"},{"why":"Provides the fingerprinting observation that versions of the same LLM have similar output distributions, used as the control baseline in the cause analysis.","marker":"[15]"},{"why":"Introduces a chain-and-hash LLM fingerprinting technique that supports the paper's choice of output similarity as a fingerprinting method.","marker":"[29]"},{"why":"Proposes a fingerprint for LLMs using unique output patterns, cited as the basis for the output-distribution comparison approach.","marker":"[42]"},{"why":"Describes REEF, a representation-encoding fingerprint method, cited alongside other fingerprinting work to justify the output-similarity analysis.","marker":"[44]"}],"fun_headline_variants":["1 in 4 LLMs misidentifies itself, not copying","Identity confusion in LLMs is hallucination, not plagiarism","LLMs' identity errors erode trust more than logical mistakes","25% of LLMs hallucinate their own identity","Identity confusion hurts trust more than logic errors"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The conclusion that identity confusion is hallucination rather than reuse assumes that two models with divergent part-of-speech distributions on one test set cannot be derived from a common source, but a model fine-tuned or distilled from another can produce quite different word distributions on some inputs.","fun_headline_variants_meta":{"raw":{"variants":["1 in 4 LLMs misidentifies itself, not copying","Identity confusion in LLMs is hallucination, not plagiarism","LLMs' identity errors erode trust more than logical mistakes","25% of LLMs hallucinate their own identity","Identity confusion hurts trust more than logic errors"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0005,"raw_usage":{"total_tokens":2477,"prompt_tokens":1008,"completion_tokens":1469,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":624,"completion_tokens_details":{"reasoning_tokens":1389}},"tokens_in":624,"tokens_out":1469,"duration_ms":9865,"temperature":1.0,"reasoning_tokens":1389,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T19:25:05.293492+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Show a single identity-confused model pair (for example, a model that claims to be ChatGPT) that, when tested on several different prompt subsets rather than one, produces output distributions close to the claimed model on at least one subset; or find memorized training text shared between the two models. Either observation would break the inference that divergence on one dataset rules out reuse.","supporting_citations":[{"cited_title":"Human chatgpt comparison corpus (hc3)","cited_arxiv_id":null,"evidence_quote":"Supplies the HC3 dataset used to elicit model outputs for the distribution-similarity analysis that distinguishes same-model consistency from cross-model divergence."}],"review_version":1}