{"id":"d2ab0f0b-12cc-4cac-a675-371303db9ba1","arxiv_id":"2509.07889","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A LoRA fine-tuned Qwen2.5 system with majority voting and multi-temperature sampling detects, classifies, and mitigates Chinese gender bias, ranking fourth in NLPCC-2025 Task 7 with an average score of 47.90%.","lead":"This paper describes a system for detecting, classifying, and rewriting gender-biased Chinese sentences, built by fine-tuning an LLM with LoRA and combining six expert models by vote. It placed fourth in a 2025 shared task, so the question is whether its components genuinely improve on simpler approaches.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Missing ablation leaves the claimed contribution unsupported; the paper's own Table 2 shows the Prompt baseline (0.502) outperforms the proposed system (0.479), so 'demonstrating effectiveness' is not established.","rationale":"The reader's verdict is CONDITIONAL, and the primary underlying gap is the absence of controlled comparisons isolating the proposed contributions. I agree that this is the most load-bearing weakness: the paper's 'effectiveness' claim is centrally supported only by the leaderboard rank, but Table 2 itself shows the Prompt baseline scoring higher on average. I focus on missing ablations rather than on the lexical-overlap metric concern, because the latter is an official shared-task evaluation design and is somewhat outside the authors' control; the ablation gap is internal to the paper and directly testable. If the proposed voting and multi-temperature mechanisms do not improve over a single LoRA expert and a fixed temperature, the method's contribution reduces to standard LoRA fine-tuning of an already-capable instruct model. The verdict remains CONDITIONAL, since the reported leaderboard numbers are plausibly accurate, but the effectiveness claim is not established on the paper's own evidence.","tokens_in":8648,"tokens_out":6673,"duration_ms":59478,"concrete_test":"Run the official dev set through four configurations: (1) zero-shot/few-shot prompting of Qwen2.5-7B-Instruct without fine-tuning; (2) a single LoRA expert trained on the full training set; (3) the full six-expert majority-voting system; (4) the full data-recombination pipeline with fixed temperature 0.1 (no multi-temperature sampling). Report Task1/Task2 F1 and Task3 BLEU/METEOR/ROUGE-L. If (3) does not beat (2) on Task2, the voting contribution is unsupported; if (4) does not beat fixed-temperature runs shown in Table 4, the multi-temperature contribution is unsupported. This directly tests whether any component beyond LoRA fine-tuning is necessary.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the proposed method is effective, but the paper gives no controlled comparison isolating any of its components. Table 2 shows the organizer's Prompt baseline has average score 0.502 vs. 0.479 for the proposed system, and Prompt outperforms it on Task2 (0.505 vs 0.453); therefore the leaderboard result, while accurate as a rank, does not by itself demonstrate effectiveness over a no-fine-tuning baseline. Section 3.4 asserts that six-expert majority voting improves robustness, but no ablation reports a single-expert or non-voting result. Table 4 shows temperature variation changes Task3 score from 0.394 to 0.392, so the multi-temperature contribution is empirically negligible as reported. Without ablations on the same dev set, the 'effectiveness' conclusion cannot be attributed to the proposed data recombination, voting, or temperature mechanism rather than to LoRA fine-tuning of Qwen2.5-7B-Instruct.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper is a system description for NLPCC-2025 Shared Task 7 on sentence-level gender bias detection, classification, and mitigation in Chinese. The authors fine-tune Qwen2.5-7B-Instruct with LoRA, rebalance the training data by randomly combining biased and unbiased instances into subsets, train six expert models on these subsets, and apply majority voting for subtasks 1 and 2. For subtask 3, they vary the decoding temperature across {0.01, 0.1, 0.3}. On the official leaderboard the system achieves an average score of 0.479 and ranks fourth, behind the organizer's 'Prompt' baseline (0.502). The paper claims that the results 'demonstrate the effectiveness' of the proposed approach.","tokens_in":8832,"tokens_out":3356,"duration_ms":30801,"significance":"If the effectiveness claim were adequately supported, the paper would provide a practical recipe for adapting a 7B LLM to Chinese gender-bias tasks with modest compute. Its strengths are that the leaderboard scores are externally measured, the work uses a real annotated corpus (CORGI-PM), and the LoRA-based design is parameter-efficient. However, the central claim is not yet established: the proposed system underperforms the Prompt baseline on average and on subtask 2, and no ablation isolates the contribution of data recombination, majority voting, or multi-temperature sampling. The paper's own reported temperature sweep shows near-identical scores. The paper is a reasonable system-description draft, but the load-bearing evidence for 'effectiveness' is missing.","major_comments":[{"comment":"The abstract and conclusion claim that the results 'demonstrate the effectiveness of the proposed approach,' but Table 2 shows that the organizer's Prompt baseline achieves a higher average score (0.502) than the proposed system (0.479), and on subtask 2 the gap is 0.505 vs. 0.453. A leaderboard rank alone does not establish effectiveness relative to a no-fine-tuning baseline. The authors should either temper the claim to 'competitive performance' or provide a controlled comparison on the dev set, e.g., the same Qwen2.5-7B-Instruct model with zero-shot prompting, with LoRA fine-tuning without recombination/voting, and with the full proposed pipeline.","section":"§4.4, Table 2"},{"comment":"The majority voting component is asserted to 'enhance the generalization capability and stability of the model,' but no ablation is reported. There is no comparison of a single expert, a small ensemble, or a non-voting aggregation on the same dev set. Without such an ablation, the benefit of six-expert majority voting is unsubstantiated. Additionally, the tie-breaking procedure is underspecified: the sentence 'retains the original confidence scores for downstream processing' does not explain how a final prediction is produced when no majority of more than three experts is reached.","section":"§3.4"},{"comment":"The multi-temperature sampling mechanism is claimed to improve bias mitigation, yet Table 4 shows that varying the temperature changes the subtask 3 score only from 0.394 to 0.392 over {0.01, 0.1, 0.3}. This is within noise and does not demonstrate any benefit. Moreover, the paper does not describe how multiple temperatures are combined: Section 3.3 says texts are generated with different styles by varying temperature, but Table 4 reports each temperature independently, not an ensemble. If multi-temperature sampling is intended as an ensemble, the integration procedure must be specified and its effect measured; otherwise the claim should be removed.","section":"§3.3, Table 4"},{"comment":"The evaluation of subtask 3 uses BLEU, METEOR, and ROUGE-L against a single reference rewrite (reported in Table 6 as BLEU 0.009, METEOR 0.391, ROUGE-L 0.367). These lexical-overlap metrics measure how closely the output resembles the reference text, not whether gender bias has actually been removed. A system that produces fluent but still-biased rewrites could score well if the reference is lexically close, while a genuinely debiased paraphrase could score poorly. The subtask 3 scores therefore do not support the 'mitigation' part of the effectiveness claim. At minimum, the authors should include qualitative examples or a bias-specific evaluation (e.g., classification of the rewritten text as biased or unbiased by a separate model) to validate that the rewrites remove bias.","section":"§4.3, Table 6"},{"comment":"The paper explicitly states, 'we do not submit our best-performing configuration, indicating that our model still has room for further improvement.' This is an important limitation: the reported leaderboard score is not the system's best result, which further weakens the conclusion that the approach is effective as described. The authors should report their dev-set results for the configurations they actually compare, including the best configuration, rather than relying on the submitted run.","section":"§4.4"}],"minor_comments":[{"comment":"The phrase 'demonstrating the effectiveness of the proposed approach' appears in both the abstract and conclusion, but the evidence presented does not support such a strong claim. Please rephrase to describe the results as 'competitive' or 'among the top-performing systems' until controlled comparisons are provided.","section":"Abstract and §5"},{"comment":"References [33] and [34] are the authors' own previous work, as indicated by the author list. If these are self-citations, this should be disclosed in the text; otherwise the citation practice is unclear. Also, references [36] and [37] appear to be the same CORGI-PM paper duplicated; merge them.","section":"§3.4, References [22,33,34,35]"},{"comment":"The use of 'multi-temperature sampling' is misleading: the experiments in Table 4 evaluate three individual temperature values, not a sampling ensemble. If the intended method is to sample multiple outputs at each temperature and then select or aggregate, that procedure should be described; otherwise use 'temperature sweep' instead of 'sampling'.","section":"§4.2"},{"comment":"Figure 1 labels the data subsets as 'Data A, B, C, N' but the caption does not explain that N corresponds to the six experts; the figure would benefit from a clearer mapping between the subsets and the expert models. Also, 'T able 1' and 'T able 2' in the text contain stray spaces.","section":"Figure 1 and Table 1"},{"comment":"The discussion of hyperparameter tuning says 'our optimization in LoRA design and learning rate scheduling confirms the practical value of our efficient parameter search strategy,' but no learning-rate or LoRA-rank ablation is reported. Either provide the supporting results or soften this sentence.","section":"§4.4"}],"recommendation":"major_revision","confidential_remarks":"This is a shared-task system description, and the acceptance bar is typically lower than for a full research paper. The central issue is that the paper's own table shows the proposed system below the Prompt baseline, and none of the claimed components (data recombination, majority voting, multi-temperature sampling) is isolated. These are fixable in principle with dev-set ablations, and the claims can be reframed. I therefore recommend major revision rather than rejection, provided the authors either supply the missing controlled experiments or substantially soften the effectiveness claim. I would also flag the self-citations (refs 33 and 34) to the editor, as the ensemble discussion cites the authors' own work without marking it as such."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: treat this as a shared-task report, not a methods paper. The fourth-place score on NLPCC-2025 Task 7 is externally measured and probably accurate, but the paper's own conclusion overstates what the numbers show.\n\nWhat's new: applying LoRA fine-tuning plus balanced data recombination, six-expert majority voting, and multi-temperature decoding to CORGI-PM is a new empirical datapoint. The prompts are given in full, the writing is clear, and the limitations section is surprisingly honest about skipping task-specific tuning on subtasks 2 and 3 and about the three-submission cap.\n\nWhere it falls short: the central effectiveness claim is not supported by the paper's own Table 2. The organizer's Prompt baseline averages 0.502; the proposed system averages 0.479. So the system underperforms a no-fine-tuning baseline on the aggregate metric. There are no ablations isolating the contributions of voting, data recombination, or temperature sampling; Section 3.4 asserts voting improves robustness without reporting a single-expert comparison. Table 4 shows temperature changes move the subtask-3 score from 0.394 to 0.392, effectively a rounding error. And the mitigation evaluation uses BLEU/METEOR/ROUGE-L, which measure surface similarity to a reference rewrite, not whether bias was actually removed. The paper acknowledges this limitation in passing but still frames the result as demonstrating mitigation effectiveness.\n\nThese are problems of framing and evidence, not fabrication. The scores are external and truthful, and the paper is transparent about its own constraints. For a shared-task report that is roughly the expected baseline, but the abstract should say 'we ranked fourth' and stop there, not claim the approach is effective.\n\nWho it's for: people working on Chinese gender-bias detection or participating in NLPCC shared tasks will get a useful data point and a reasonable description of a standard LoRA+voting pipeline. No new principle, dataset, or theory.\n\nRecommendation: I would send it to peer review in a workshop or shared-task venue, with the expectation of heavy revision. The referee should ask for at least a single-expert baseline and a softened effectiveness claim. As it stands, the empirical result is credible but the interpretation is not.","headline":"Read this as a shared-task system description, not a methods paper: the ranked results are real, but the effectiveness claim is undercut by the Prompt baseline and missing ablations.","tokens_in":9377,"tokens_out":3420,"would_cite":false,"duration_ms":27550,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A LoRA-tuned Qwen2.5 model with class-balanced expert voting detects, classifies, and mitigates Chinese gender bias, scoring 47.90% on the shared task and finishing fourth.","keywords":["gender bias","Chinese NLP","LoRA","large language models","text classification","text rewriting","class imbalance","majority voting"],"falsifier":"Run a human-annotation or independent-classifier study on the system's rewritten outputs: if sentences with high BLEU/METEOR/ROUGE-L are still judged to contain gender bias, the metric-based mitigation claim is refuted. A second decisive check is an ablation comparing the six-expert voting pipeline with a single expert trained on all data, using the development set's F1 scores.","tokens_in":8434,"feed_emoji":"⚖️","tokens_out":11560,"duration_ms":90883,"temperature":0.7,"pith_summary":"The paper tries to establish that a complete gender-bias handling pipeline for Chinese can be built by efficiently adapting a general instruction-tuned LLM instead of training from scratch. Starting from the CORGI-PM corpus, the authors fine-tune Qwen2.5-7B-Instruct with LoRA, rebalance the strongly skewed training data by recombining biased and non-biased sentences into roughly 1:1 subsets, and train six expert models. For detection and classification, the experts' predictions are combined by majority vote; for rewriting, the model samples de-biased outputs at several temperatures. On the official evaluation the method reaches 47.90% average score and fourth place, with detection F1 of 0.720. A sympathetic reader would take this as evidence that lightweight LoRA plus data recombination is a workable recipe for practical bias-detection tools.","feed_headline":"LoRA tuning plus six-expert voting ranks 4th on Chinese gender bias","feed_subtitle":"A frozen 7 billion-parameter LLM with LoRA adapters scores 47.90 on Chinese gender-bias detection, classification, and rewriting.","key_machinery":"The mechanism is a pipeline with three coordinated pieces. LoRA keeps the Qwen2.5-7B-Instruct backbone frozen and inserts small trainable low-rank matrices, so each subtask-specific expert adapts with few parameters. For classification, the 21,418 non-biased training sentences are split into five subsets and randomly combined with the 4,172 biased sentences at about 1:1 ratio, producing six experts; the final prediction is the majority answer when more than three of the six agree. For rewriting, generation temperatures of 0.01, 0.1, and 0.3 are sampled to produce stylistic variation, and the official overlap metrics are used to assess rewrites. A unified prompt with role, task description, and examples is shared across subtasks, and the bias-type prediction from subtask 2 is inserted into the rewriting prompt.","core_discovery":"The central claim is that LoRA-based fine-tuning of a general-purpose instruction-tuned LLM, together with class-balanced data recombination and six-expert majority voting, is sufficient for competitive sentence-level gender-bias detection, classification, and mitigation in Chinese. On the official test set the model detects bias with F1 0.720, ranking second on that subtask; classifies bias type with F1 0.453; and scores 0.265 on rewriting as measured by the average of BLEU, METEOR, and ROUGE-L against reference rewrites. The authors attribute the detection result to their LoRA and learning-rate configuration, explain the classification gap by missing within-category imbalance and reuse of detection-oriented hyperparameters, and describe the temperature sweep on rewriting as exploratory rather than exhaustive. The overall result, an average of 47.90% and fourth place, is offered as confirmation that the proposed pipeline works in practice.","pith_inferences":["A human-annotation comparison would probably reveal that lexical-overlap metrics reward surface similarity to a reference rewrite rather than actual de-biasing, so the subtask-3 score should be read as a proxy, not as proof of bias removal.","The paper never ablates the voting mechanism, so a single-expert baseline on the same full training set would separate the contribution of the ensemble from the contribution of simply using more data.","The flat temperature results suggest that a broader sweep or an adaptive sampling strategy might find gains the current design misses, or alternatively that the pipeline is already robust to this hyperparameter.","Since none of the mechanism components are gender-specific, the same recombination-and-voting recipe could be transferred to other bias axes such as occupation, race, or age by swapping the annotated corpus."],"forward_implications":["If these results hold, a frozen 7-billion-parameter LLM with LoRA adapters is enough for usable Chinese gender-bias detection, lowering the hardware and data barrier for fairness tooling.","The rebalancing-and-voting design implies that class imbalance can be handled by training several moderately balanced experts rather than by carefully reweighting a single model.","Since the three temperatures tested gave nearly identical mitigation scores, the paper's own results suggest temperature variation is not the main driver of rewrite quality in that range.","The authors' statement that they reused subtask-1 hyperparameters for subtask-2 implies the classification result underestimates what the same pipeline could achieve with task-specific tuning.","Because the rewriting prompt consumes subtask-2 bias-type predictions, classification errors propagate into mitigation, so improving the classifier should directly improve rewriting relevance."],"supporting_citations":[{"why":"Supplies the annotated CORGI-PM Chinese corpus whose bias labels define all three subtasks.","marker":"[36]"},{"why":"Provides the LoRA ensemble fine-tuning method used to adapt the backbone efficiently.","marker":"[30]"},{"why":"Supports the weighted/majority voting ensemble strategy for classification.","marker":"[22]"},{"why":"Demonstrates majority voting for LLM multi-label classification, the basis for the six-expert vote.","marker":"[35]"},{"why":"Defines BLEU, one of the three metrics scored in the mitigation subtask.","marker":"[23]"},{"why":"Defines METEOR, the second overlap metric used for rewrite evaluation.","marker":"[2]"},{"why":"Defines ROUGE-L, the third overlap metric used for rewrite evaluation.","marker":"[15]"}],"fun_headline_variants":["LoRA and six-expert voting rank 4th on Chinese gender bias","Chinese gender bias: LoRA + voting scores 47.90, places 4th","2nd in detection, 4th overall: LoRA and voting on Chinese bias","Efficient tuning and voting: a 4th-place fix for Chinese bias","LoRA, rebalanced data, and voting: 4th in Chinese bias task"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that measuring a rewrite's lexical overlap with one reference de-biased version through BLEU, METEOR, and ROUGE-L actually tells whether gender bias was removed; if that metric is not measuring bias removal, the subtask-3 scores do not support the mitigation claim.","fun_headline_variants_meta":{"raw":{"variants":["LoRA and six-expert voting rank 4th on Chinese gender bias","Chinese gender bias: LoRA + voting scores 47.90, places 4th","2nd in detection, 4th overall: LoRA and voting on Chinese bias","Efficient tuning and voting: a 4th-place fix for Chinese bias","LoRA, rebalanced data, and voting: 4th in Chinese bias task"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00065,"raw_usage":{"total_tokens":2978,"prompt_tokens":936,"completion_tokens":2042,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":552,"completion_tokens_details":{"reasoning_tokens":1934}},"tokens_in":552,"tokens_out":2042,"duration_ms":13471,"temperature":1.0,"reasoning_tokens":1934,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T16:08:58.133618+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a human-annotation or independent-classifier study on the system's rewritten outputs: if sentences with high BLEU/METEOR/ROUGE-L are still judged to contain gender bias, the metric-based mitigation claim is refuted. A second decisive check is an ablation comparing the six-expert voting pipeline with a single expert trained on all data, using the development set's F1 scores.","supporting_citations":[{"cited_title":"Expert Systems with Applications62, 1–16 (2016)","cited_arxiv_id":null,"evidence_quote":"Supports the weighted/majority voting ensemble strategy for classification."}],"review_version":2}