{"id":"efcf3d6d-d2af-42f6-b4d0-84cece36c721","arxiv_id":"2605.26891","paper_version":2,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Presents a publicly available multilingual corpus of 1,122 customer service self-help documents in four Nordic languages totaling 274,599 words.","lead":"This paper releases a new multilingual corpus of 1,122 customer service self-help documents in Finnish, Danish, Norwegian, and Swedish. Researchers building NLP tools or retrieval systems for Nordic languages in customer service domains may use it for training and evaluation.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"Filtering pipeline for PII removal and relevance lacks reported error rates, agreement metrics, or sample audits","rationale":"The reader's weakest_assumption directly identifies the same curation-quality dependency that determines whether the released counts and license claims are usable. Full text availability does not change this, as the paper remains a data release whose value rests on unverifiable filtering fidelity.","tokens_in":1682,"tokens_out":287,"duration_ms":16042,"concrete_test":"Release the exact LLM prompts, annotation guidelines, and a random sample of 100 pre- and post-filter documents together with their per-document decisions and any disagreement logs; recompute the final corpus size after an independent re-annotation of that sample.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim asserts a corpus of 1,122 manually validated documents after LLM+human filtering for person-identifiable information and relevance. For this to hold, the pipeline must have produced low residual error. The description provides no inter-annotator agreement, no LLM prompt details, no false-positive/false-negative rates on held-out checks, and no post-filter audit statistics. Without these, the headline counts (1,122 docs, 274,599 words) cannot be treated as reliably cleaned; residual PII or off-topic documents would directly falsify the \"manually validated\" and \"filtered\" qualifiers.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper presents a multilingual customer service self-help corpus comprising 1,122 manually validated documents in Finnish, Danish, Norwegian, and Swedish (totaling 274,599 words and 1,884,833 characters). Documents were sourced from public self-help pages of four Nordic telecom operators and filtered for person-identifiable information and relevance via a combined LLM and human annotation pipeline. The dataset is released publicly under CC-BY-NC-SA-4.0 at a Zenodo URL to support Nordic NLP and IR research, with an accompanying analysis of length variation and topical coverage.","tokens_in":1779,"tokens_out":323,"duration_ms":31092,"significance":"If the validation claims hold, the release fills a documented gap in domain-specific Nordic-language resources for customer service, a domain relevant to RAG, cross-lingual transfer, and agent architectures. The public availability under a clear license is a concrete strength for reproducibility.","major_comments":[{"comment":"The description of the LLM+human filtering pipeline (for PII removal and relevance) provides no inter-annotator agreement scores, no LLM prompt text or parameters, no false-positive/false-negative rates from held-out checks, and no post-filter audit statistics. This directly affects the load-bearing claim that the 1,122 documents are 'manually validated' and reliably cleaned; residual PII or off-topic items would falsify the headline counts and qualifiers.","section":"filtering pipeline description (Methods)"}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback on the filtering pipeline. We address the major comment below and commit to revisions where possible to strengthen the manuscript.","responses":[{"response":"We agree the current Methods description is insufficiently detailed. In revision we will add the exact LLM prompts, model names, and parameters used for the initial filtering steps. We will also expand the description of the human validation stage, including the criteria applied for PII and relevance, and report any available post-filter audit numbers (e.g., documents reviewed or removed after the LLM stage).","revision_made":"yes","referee_comment":"[filtering pipeline description (Methods)] The description of the LLM+human filtering pipeline (for PII removal and relevance) provides no inter-annotator agreement scores, no LLM prompt text or parameters, no false-positive/false-negative rates from held-out checks, and no post-filter audit statistics. This directly affects the load-bearing claim that the 1,122 documents are 'manually validated' and reliably cleaned; residual PII or off-topic items would falsify the headline counts and qualifiers."}],"tokens_in":1275,"tokens_out":289,"duration_ms":38274,"standing_objections":["Inter-annotator agreement scores and false-positive/false-negative rates from held-out checks, because these metrics were not computed during the original single-pass LLM-assisted human validation process."]},"desk_editor":{"model":"grok-4.3","letter":"Colleague,\n\nThis is a data release paper for a new multilingual corpus of customer service self-help documents from four Nordic telecom operators. It contains 1,122 documents in Finnish, Danish, Norwegian, and Swedish, totaling roughly 275k words, collected from public pages and filtered for PII and relevance via an LLM plus human pipeline. The set is available on Zenodo under CC-BY-NC-SA.\n\nWhat the work does is supply domain-specific text in languages where such material has been scarce, particularly for customer service topics like billing, hardware, and streaming. The abstract notes variation in document length and structure across operators and lists broad topical coverage. Making the data public with a clear license is the concrete contribution here.\n\nThe soft spot is the filtering process. The claim of manual validation after the combined pipeline is central to the headline numbers, yet the description provides no inter-annotator agreement, no prompt details, no held-out error rates for PII removal or relevance, and no post-filter audit stats. That leaves the reliability of the cleaned counts open to question, exactly as the stress-test note flags. The rest of the paper is descriptive rather than analytical.\n\nThis is for researchers who need real Nordic customer-service text for RAG, retrieval, or agent experiments and who are willing to do their own quality checks. It is not a methods advance.\n\nI would send it to peer review. Reviewers can ask for the missing pipeline metrics, and the field benefits from the released resource even if the paper stays short.","headline":"New public Nordic telecom customer service corpus of 1,122 docs that fills a data gap but reports no error rates on its LLM+human filtering.","tokens_in":2221,"tokens_out":389,"would_cite":false,"duration_ms":28286,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A new public corpus supplies 1,122 validated self-help documents across Finnish, Danish, Norwegian, and Swedish for customer-service NLP.","keywords":["customer service corpus","multilingual dataset","Nordic languages","self-help documents","information retrieval","natural language processing","telecommunications data"],"falsifier":"Inspection of the released files reveals either persistent personal names, phone numbers, or account details, or a large fraction of documents that do not address customer service topics.","tokens_in":2560,"feed_emoji":"📄","tokens_out":637,"duration_ms":8788,"temperature":0.7,"pith_summary":"The paper releases a collection of customer service texts drawn from the public self-help pages of four Nordic telecommunications operators. The documents total 274,599 words and were cleaned of personal information through a combined automated and manual review process. The authors position the resource as a remedy for the scarcity of domain-specific data in these languages, where it can support work on retrieval, cross-language model adaptation, and automated service systems. Variation in length and topical coverage across operators is documented as part of the release.","feed_headline":"New corpus supplies 1,122 Nordic customer service documents","feed_subtitle":"Validated texts in Finnish, Danish, Norwegian and Swedish total 274k words and are released for retrieval and automated support research.","key_machinery":"The multilingual customer service self-help corpus, which aggregates and validates operator documents to serve as training and evaluation data for Nordic-language NLP tasks.","core_discovery":"The paper establishes a multilingual customer service self-help corpus of 1,122 manually validated documents in Finnish, Danish, Norwegian, and Swedish. The texts were gathered from public operator pages, filtered for relevance and absence of personal data via an LLM-plus-human pipeline, and released under CC-BY-NC-SA-4.0. Analysis shows differences in document length and structure by operator together with coverage of network hardware, mobile services, TV, billing, and account management.","pith_inferences":["The corpus could be used to test whether language models trained on general web text underperform on the specific vocabulary and phrasing found in operator self-help pages.","Release under a non-commercial license may limit certain industry applications while still permitting academic and nonprofit reuse.","The observed variation in document length suggests that any downstream model would need to handle both short procedural answers and longer explanatory articles."],"forward_implications":["Enables reproducible experiments on retrieval-augmented generation in customer-service settings for Nordic languages.","Supports cross-lingual transfer learning studies that include Finnish, Danish, Norwegian, and Swedish.","Provides material for training or evaluating agent-based service architectures in the telecom domain.","Allows quantitative comparison of editorial practices across operators through measurable differences in document length and structure."],"fun_headline_variants":["1122 validated Nordic customer service documents released in four languages","Self-help corpus includes 1122 documents from Nordic telecom operators","1122 documents in Finnish Danish Norwegian and Swedish for customer service","Nordic customer service self-help corpus totals 1122 validated documents"],"cache_read_input_tokens":64,"weakest_assumption_plain":"The combined LLM and human filtering step has removed personal information and retained only relevant documents without introducing substantial errors or omissions.","fun_headline_variants_meta":{"raw":{"variants":["1122 validated Nordic customer service documents released in four languages","Self-help corpus includes 1122 documents from Nordic telecom operators","1122 documents in Finnish Danish Norwegian and Swedish for customer service","Nordic customer service self-help corpus totals 1122 validated documents"]},"model":"grok-4.3","cost_usd":0.009074,"raw_usage":{"total_tokens":3978,"prompt_tokens":643,"num_sources_used":0,"completion_tokens":69,"cost_in_usd_ticks":90740500,"prompt_tokens_details":{"text_tokens":643,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3266,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":643,"tokens_out":69,"duration_ms":27944,"temperature":1.0,"reasoning_tokens":3266,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-29T18:23:30.965196+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Inspection of the released files reveals either persistent personal names, phone numbers, or account details, or a large fraction of documents that do not address customer service topics.","supporting_citations":[],"review_version":1}