{"total":40,"items":[{"citing_arxiv_id":"2607.08268","ref_index":31,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Different Teachers, Different Capabilities: Sub-1B On-Device Distillation for Structured Text Enrichment","primary_cat":"cs.AI","submitted_at":"2026-07-09T09:10:49+00:00","verdict":"CONDITIONAL","verdict_confidence":"HIGH","novelty_score":6.0,"formal_verification":"none","one_line_summary":"Distilling an 8B reasoning teacher into a 0.6B student recovers most summary quality at ~50× speed, but teacher type—not scale alone—determines which capabilities transfer.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.25757","ref_index":9,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"OPERA: Aligning Open-Ended Reasoning via Objective Perplexity-based Reinforcement Learning","primary_cat":"cs.CL","submitted_at":"2026-06-24T12:31:14+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"OPERA uses perplexity dynamics as intrinsic rewards for RL alignment on open-ended tasks, synthesizes a 20k trajectory dataset via guiding words and log-prob rollouts, and sets new SOTA on Qwen3-8B matching some proprietary models.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.24004","ref_index":117,"ref_count":2,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Towards Spec Learning: Inference-Time Alignment from Preference Pairs","primary_cat":"cs.CL","submitted_at":"2026-06-22T23:21:55+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"Proposes compiling preference pairs into readable natural-language specifications for inference-time LLM alignment, claiming outperformance over DPO on dense-preference domains.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.19640","ref_index":72,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Creating Multilingual Mental Health Dialogue Datasets: Limits of Persona-Based Localization via Nationality and Language","primary_cat":"cs.CL","submitted_at":"2026-06-17T22:36:06+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":4.0,"formal_verification":"none","one_line_summary":"Modifying nationality and language parameters in English-centric personas for mental health dialogues introduces clinical inconsistencies across languages and causes LLM judges to perform inaccurately on non-English depression severity assessments.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.19057","ref_index":17,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Quantifying and Auditing LLM Evaluation via Positive--Unlabeled Learning","primary_cat":"stat.ML","submitted_at":"2026-06-17T13:26:04+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"A positive-unlabeled learning approach using partial optimal transport is introduced to audit and correct biases in LLM-as-a-judge systems by aligning limited human positives with unlabeled outputs in embedding space.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.11424","ref_index":42,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"SOMA-SQL: Resolving Multi-Source Ambiguity in NL-to-SQL via Synthetic Log and Execution Probing","primary_cat":"cs.CL","submitted_at":"2026-06-09T20:18:22+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"SOMA-SQL resolves multi-source ambiguity in NL-to-SQL using synthetic query logs and ambiguity-driven execution probing, reporting 13% average execution accuracy gains over baselines on six benchmarks.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.10315","ref_index":14,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Catching One in Five: LLM-as-Judge Blind Spots in Production Multi-Turn Transaction Agents","primary_cat":"cs.CL","submitted_at":"2026-06-09T02:11:01+00:00","verdict":"ACCEPT","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"Empirical study of a production multi-turn ordering agent finds LLM-as-judge recall below 25% for human-confirmed defects, missing cross-turn state issues due to limited rubric and routing.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.03650","ref_index":17,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"CoEval: Ranking Language Models for Custom Tasks Without Labeled Data or Trustworthy Benchmarks","primary_cat":"cs.CL","submitted_at":"2026-06-02T13:41:43+00:00","verdict":"CONDITIONAL","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"CoEval generates task-specific benchmarks by rotating models through teacher, student, and judge roles, then weights questions by discriminative power and judges by panel consensus to recover accurate model rankings without labels.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.02258","ref_index":67,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Matter to Mechanism: A Benchmark for AI Co-Scientists in Materials and Battery Research","primary_cat":"cs.CE","submitted_at":"2026-06-01T13:45:18+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"Introduces the Matter to Mechanism benchmark of 2,645 structured instances and a composite metric suite for evaluating AI co-scientists on problem-to-hypothesis reasoning in battery materials research.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.27914","ref_index":129,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Does Capability Transfer to Subjective Behavior -- and Would Our Instruments Tell Us? A Self-Evolving, Trust-by-Construction Evaluation Paradigm","primary_cat":"cs.CL","submitted_at":"2026-05-27T03:41:11+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"Self-evolving rubric with anti-gaming fitness reveals that objective capability scaling fails to transfer to subjective LLM behaviors, with advice-restraint as the universal lowest dimension that can regress.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.26769","ref_index":9,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Generative artificial intelligence and the marginalization of minoritized knowledges in higher education: the case of disability","primary_cat":"cs.CY","submitted_at":"2026-05-26T09:39:09+00:00","verdict":"UNVERDICTED","verdict_confidence":"UNKNOWN","novelty_score":3.0,"formal_verification":"none","one_line_summary":"Generative AI reinforces epistemic coloniality and double marginalization of disabled knowledges in higher education via Anglophone/Western datasets and reductive stereotypes or design exclusion.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.25030","ref_index":54,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"MimirRAG: A Multi-Agent RAG Framework for Financial Data Retrieval with Metadata Integration","primary_cat":"cs.LG","submitted_at":"2026-05-24T12:15:27+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":4.0,"formal_verification":"none","one_line_summary":"MimirRAG, a multi-agent RAG framework with metadata integration and table-aware chunking, reaches 89.3% accuracy on FinanceBench and outperforms prior baselines for financial document retrieval.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.22714","ref_index":32,"ref_count":2,"confidence":0.98,"is_internal_anchor":true,"paper_title":"AMEL: Accumulated Message Effects on LLM Judgments","primary_cat":"cs.AI","submitted_at":"2026-05-21T16:51:04+00:00","verdict":"UNVERDICTED","verdict_confidence":"MODERATE","novelty_score":6.0,"formal_verification":"none","one_line_summary":"LLMs exhibit an accumulated message effect where conversation history polarity biases subsequent judgments, stronger for high-entropy items, independent of context length, and with a negativity bias.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.19141","ref_index":67,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"GRASP: Deterministic argument ranking in interaction graphs","primary_cat":"cs.LG","submitted_at":"2026-05-18T21:49:02+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"GRASP aggregates stable local LLM interaction judgments into global argument rankings via a convergent attack-defense propagation operator on interaction graphs, yielding higher reproducibility than holistic judging and no correlation with human convincingness.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.18661","ref_index":241,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"AI for Auto-Research: Roadmap & User Guide","primary_cat":"cs.AI","submitted_at":"2026-05-18T17:08:26+00:00","verdict":"CONDITIONAL","verdict_confidence":"MODERATE","novelty_score":4.0,"formal_verification":"none","one_line_summary":"AI can generate research artifacts faster than it can verify them, so across all eight lifecycle stages the credible deployment mode is human-governed collaboration rather than full autonomy.","context_count":1,"top_context_role":"background","top_context_polarity":"background","context_text":"trends indicate that AI involvement in peer review is already widespread, while enforceable governance remains immature. 24 Adversarial manipulation further complicates deployment. Breaking the Reviewer [113] studies adversarial robustness of LLM-based review assessments, while Keuper [88] shows that simple prompt injections, such as white text on a white background, can manipulate LLM reviews. Yeet al. [241] show that covert content injection can substantially raise review scores and that manipulating a small fraction of reviews can alter rankings. Zhouet al. [265] further demonstrate that in-paper prompt injection can raise LLM scores under static and iterative attacks. At the lexical level, Rainaet al. [157] show that benign adjectives can function as"},{"citing_arxiv_id":"2605.16712","ref_index":11,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Recall Isn't Enough: Bounding Commitments in Personalized Language Systems","primary_cat":"cs.AI","submitted_at":"2026-05-15T23:50:15+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"CBEA with LCV bounds evidence sets and validates commitments before response generation, achieving zero failures in scoped tests at 0.49-0.60 availability versus near-zero for baselines.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.16386","ref_index":29,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Auditing Multimodal LLM Raters: Central Tendency Bias in Clinical Ordinal Scoring","primary_cat":"cs.CV","submitted_at":"2026-05-11T15:37:24+00:00","verdict":"CONDITIONAL","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"Multimodal LLMs exhibit central tendency bias when scoring ordinal clinical images, over-predicting low scores and under-predicting high scores even after prompt ablations.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.08522","ref_index":25,"ref_count":1,"confidence":0.9,"is_internal_anchor":true,"paper_title":"Coordinates of Capability: A Unified MTMM-Geometric Framework for LLM Evaluation","primary_cat":"cs.CL","submitted_at":"2026-05-08T22:05:19+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"A new MTMM-geometric framework unifies LLM evaluation metrics into three latent dimensions to separate method variance from true capabilities.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.06161","ref_index":49,"ref_count":1,"confidence":0.9,"is_internal_anchor":true,"paper_title":"Beyond Accuracy: Policy Invariance as a Reliability Test for LLM Safety Judges","primary_cat":"cs.AI","submitted_at":"2026-05-07T12:49:09+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"LLM safety judges flip verdicts on equivalent policy rewrites up to 9.1% of the time and cannot distinguish meaningful from meaningless changes, requiring new invariance-based reliability metrics.","context_count":1,"top_context_role":"background","top_context_polarity":"background","context_text":"Section 6 concludes. 2 Related Works LLM-as-a-Judge reliability.The LLM-as-a-Judge paradigm, popularized by MT-Bench and Chatbot Arena [54], has become the default evaluation pipeline for open-ended generation tasks. Subsequent studies documented systematic biases: position bias in pairwise comparisons [ 44], the CALM framework identifying 12 distinct bias types [49], and adversarial vulnerabilities in judge prompts [34, 39, 53]. A comprehensive survey by Gu et al. [14] categorizes these failure modes and proposes design guidelines. Li et al. [25] found that even frontier models fail to maintain consistent preferences in ∼25% of difficult cases. The Trust-or-Escalate framework [ 20] introduces confidence-based"},{"citing_arxiv_id":"2605.06036","ref_index":210,"ref_count":1,"confidence":0.9,"is_internal_anchor":true,"paper_title":"Optimal Transport for LLM Reward Modeling from Noisy Preference","primary_cat":"cs.LG","submitted_at":"2026-05-07T11:26:04+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"SelectiveRM applies optimal transport with a joint consistency discrepancy and partial mass relaxation to produce reward models that optimize a tighter upper bound on clean risk while autonomously dropping noisy preference samples.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.02010","ref_index":20,"ref_count":1,"confidence":0.9,"is_internal_anchor":true,"paper_title":"Reliable AI Needs to Externalize Implicit Knowledge: A Human-AI Collaboration Perspective","primary_cat":"cs.AI","submitted_at":"2026-05-03T18:31:45+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":3.0,"formal_verification":"none","one_line_summary":"Proposes Knowledge Objects to externalize implicit AI knowledge for human verification, addressing a gap in current reliability methods.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2607.01240","ref_index":22,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Prompt Framing Distorts Count Based Evaluation of LLM Error Detection: Evidence from Numeric Anchoring","primary_cat":"cs.CL","submitted_at":"2026-05-03T12:21:42+00:00","verdict":"CONDITIONAL","verdict_confidence":"HIGH","novelty_score":5.0,"formal_verification":"none","one_line_summary":"Supplying the true error count in a prompt raises LLM Count-F1 by ~0.22 but edit-level F0.5 by only ~0.04, so count agreement mostly measures prompt compliance.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2604.27132","ref_index":43,"ref_count":1,"confidence":0.9,"is_internal_anchor":true,"paper_title":"TRUST: A Framework for Decentralized AI Service v.0.1","primary_cat":"cs.AI","submitted_at":"2026-04-29T19:32:58+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"TRUST is a decentralized AI auditing framework that decomposes reasoning into HDAGs, maps agent interactions via the DAAN protocol to CIGs, and uses stake-weighted multi-tier consensus to achieve 72.4% accuracy while proving a Safety-Profitability Theorem that rewards honest auditors.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2604.24700","ref_index":55,"ref_count":1,"confidence":0.9,"is_internal_anchor":true,"paper_title":"Green Shielding: A User-Centric Approach Towards Trustworthy AI","primary_cat":"cs.CL","submitted_at":"2026-04-27T17:04:17+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"Green Shielding introduces CUE criteria and the HCM-Dx benchmark to demonstrate that routine prompt variations systematically alter LLM diagnostic behavior along clinically relevant dimensions, producing Pareto-like tradeoffs in plausibility versus coverage.","context_count":1,"top_context_role":"background","top_context_polarity":"background","context_text":"[53] Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, et al. A survey on llm-as-a-judge.The Innovation, 2024. [54] Jiayi Ye, Yanbo Wang, Yue Huang, Dongping Chen, Qihui Zhang, Nuno Moniz, Tian Gao, Werner Geyer, Chao Huang, Pin-Yu Chen, et al. Justice or prejudice? quantifying biases in llm-as-a-judge.arXiv preprint arXiv:2410.02736, 2024. [55] Koki Wataoka, Tsubasa Takahashi, and Ryokan Ri. Self-preference bias in llm-as-a-judge.arXiv preprint arXiv:2410.21819, 2024. 21 [56] Lin Shi, Chiyu Ma, Wenhua Liang, Xingjian Diao, Weicheng Ma, and Soroush Vosoughi. Judging the judges: A systematic study of position bias in llm-as-a-judge. InProceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the"},{"citing_arxiv_id":"2604.24621","ref_index":42,"ref_count":1,"confidence":0.9,"is_internal_anchor":true,"paper_title":"Evaluation of LLM-Based Software Engineering Tools: Practices, Challenges, and Future Directions","primary_cat":"cs.SE","submitted_at":"2026-04-27T15:51:22+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":4.0,"formal_verification":"none","one_line_summary":"LLM-based SE tools lack stable ground truth and deterministic outputs, making standard evaluation assumptions invalid and requiring new approaches for reliable assessment.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2604.23593","ref_index":13,"ref_count":1,"confidence":0.9,"is_internal_anchor":true,"paper_title":"When AI reviews science: Can we trust the referee?","primary_cat":"cs.AI","submitted_at":"2026-04-26T08:03:32+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"AI peer review systems are vulnerable to prompt injections, prestige biases, assertion strength effects, and contextual poisoning, as demonstrated by a new attack taxonomy and causal experiments on real conference submissions.","context_count":1,"top_context_role":"background","top_context_polarity":"unclear","context_text":"Nature 643:887-888. DOI:10.1038/d41586-025-02172-y [11] Ji Z., Lee N., Frieske R., et al. (2023). Survey of hallucination in natural language generation. ACM Comput. Surv. 55:1-38. DOI:10.1145/3571730 [12] Jin Y ., Zhao Q., Wang Y ., et al. (2024). Agentreview: Ex- ploring peer review dynamics with llm agents. arXiv preprint. DOI:10.48550/arXiv.2406.12708 [13] Y e J., Wang Y ., Huang Y ., et al. (2024). Justice or preju- dice? quantifying biases in llmas-a-judge. arXiv preprint. DOI:10.48550/arXiv.2410.02736 [14] Lin T.-L., Chen W.-C., Hsiao T.-F ., et al. (2025). Breaking the reviewer: Assessing the vulnerability of large language models in automated peer review under textual adversarial attacks. arXiv preprint."},{"citing_arxiv_id":"2604.22597","ref_index":33,"ref_count":1,"confidence":0.9,"is_internal_anchor":true,"paper_title":"Rethinking Math Reasoning Evaluation: A Robust LLM-as-a-Judge Framework Beyond Symbolic Rigidity","primary_cat":"cs.AI","submitted_at":"2026-04-24T14:25:01+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"An LLM-as-a-judge evaluation framework for math reasoning outperforms symbolic methods by accurately assessing diverse answer representations and formats.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2604.16790","ref_index":31,"ref_count":1,"confidence":0.9,"is_internal_anchor":true,"paper_title":"Bias in the Loop: Auditing LLM-as-a-Judge for Software Engineering","primary_cat":"cs.SE","submitted_at":"2026-04-18T02:35:05+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"LLM judges for code tasks show high sensitivity to prompt biases that systematically favor certain options, changing accuracy and model rankings even when code is unchanged.","context_count":1,"top_context_role":"background","top_context_polarity":"background","context_text":"and several recent surveys now position LLMs as a central enabling technology for modern SE prac- tice. Fan et al. [6] summaris how code-oriented LLMs are being adopted for activities ranging from requirements and design assistance to coding, refactoring, performance tuning, and maintenance, and highlight both their promise and the new reliability and safety challenges they introduce [6]. More recently, Zhang et al. [31] provide systematic overviews of LLMs for SE, cataloguing dozens of code LLMs, pre-training objectives, and over one hundred downstream SE tasks, and emphasise that LLMs are increasingly embedded into developer workflows rather than treated as standalone models [31]. Parallel work on LLM-based agents for SE further argues that orchestrating multiple"},{"citing_arxiv_id":"2604.07709","ref_index":39,"ref_count":1,"confidence":0.9,"is_internal_anchor":true,"paper_title":"IatroBench: Pre-Registered Evidence of Iatrogenic Harm from AI Safety Measures","primary_cat":"cs.AI","submitted_at":"2026-04-09T01:54:33+00:00","verdict":"CONDITIONAL","verdict_confidence":"HIGH","novelty_score":7.0,"formal_verification":"none","one_line_summary":"Frontier models systematically withhold clinically necessary guidance from laypeople that they provide to physicians on identical facts, and LLM judges fail to detect that omission harm.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2604.07650","ref_index":24,"ref_count":1,"confidence":0.9,"is_internal_anchor":true,"paper_title":"A Statistical Framework for Auditing Behavioral Dependence and Induced Bias in LLM Judges","primary_cat":"cs.AI","submitted_at":"2026-04-08T23:32:06+00:00","verdict":null,"verdict_confidence":null,"novelty_score":null,"formal_verification":null,"one_line_summary":null,"context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2604.05955","ref_index":42,"ref_count":1,"confidence":0.9,"is_internal_anchor":true,"paper_title":"Does Pass Rate Tell the Whole Story? Evaluating Design Constraint Compliance in LLM-based Issue Resolution","primary_cat":"cs.SE","submitted_at":"2026-04-07T14:47:27+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"LLM agents resolve fewer than half of issues while satisfying design constraints despite passing tests, as shown by a benchmark of 495 issues and 1787 constraints from six repositories.","context_count":1,"top_context_role":"background","top_context_polarity":"background","context_text":"proposed Kantara [9], which aims to automatically construct ratio- nale and decision graphs from commits and further instantiates the framework with LLM-based extraction [10]. DRMiner [40, 41] simi- larly combines LLMs with heuristic signals and decomposes issue discussions into multiple classification tasks to mine latent design information. Other recent studies, such as Zhouet al.[ 42], leverage LLMs to generate design rationales for architectural decisions from textual artifacts. Collectively, these approaches primarily aim to recover decisions and therationalesunderlying them. In contrast,DesignHunterfocuses on extractingdesign con- straintsthat operationalize decision knowledge. Beyond capturing \"why\",DesignHunterexplicitly models \"when it holds\", which is"},{"citing_arxiv_id":"2604.05593","ref_index":51,"ref_count":1,"confidence":0.9,"is_internal_anchor":true,"paper_title":"Label Effects: Shared Heuristic Reliance in Trust Assessment by Humans and LLM-as-a-Judge","primary_cat":"cs.AI","submitted_at":"2026-04-07T08:43:30+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"Both humans and LLMs trust content more when labeled human-authored than AI-generated, with LLMs showing denser attention to labels and higher uncertainty under AI labels, mirroring human heuristic patterns.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2604.05371","ref_index":5,"ref_count":1,"confidence":0.9,"is_internal_anchor":true,"paper_title":"LLM-as-Judge for Semantic Judging of Powerline Segmentation in UAV Inspection","primary_cat":"cs.AI","submitted_at":"2026-04-07T03:16:44+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":4.0,"formal_verification":"none","one_line_summary":"An LLM produces consistent categorical judgments and appropriate confidence declines when evaluating powerline segmentation quality under controlled visual degradations, suggesting it can serve as a reliable watchdog.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2604.02406","ref_index":127,"ref_count":4,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Evaluating AI-Generated Images of Cultural Artifacts with Community-Informed Rubrics","primary_cat":"cs.CY","submitted_at":"2026-04-02T17:17:12+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"Case studies with blind UK residents and people from Kerala and Tamil Nadu demonstrate that community input at the systematization stage produces culturally grounded definitions of appropriateness for text-to-image model outputs.","context_count":2,"top_context_role":"background","top_context_polarity":"background","context_text":"participants' preferences for cultural representation, rather than engaging the community as full collaborators who were given the power to shape and own research outputs [31, 107]. While there are several potential benefits to pursuing lower-touch approaches to scaffolding participation, such as respecting and attempting to minimize the labor required from participants [127], we acknowledge that such approaches run the risk of being exploitative if outputs are misused by researchers, or if community members are not adequately compensated for their labor [ 27, 31]. There is more work needed to develop methods that meaningfully shift power to community members,e.g.,as part of a grassroots, community-led project where community members have full ownership over critical measurement decisions [87]."},{"citing_arxiv_id":"2604.02359","ref_index":71,"ref_count":1,"confidence":0.9,"is_internal_anchor":true,"paper_title":"Using LLM-as-a-Judge/Jury to Advance Scalable, Clinically-Validated Safety Evaluations of Model Responses to Users Demonstrating Psychosis","primary_cat":"cs.CL","submitted_at":"2026-03-20T04:31:03+00:00","verdict":"CONDITIONAL","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"Seven clinician-informed safety criteria enable LLM-as-a-Judge to reach substantial agreement with human consensus (Cohen's κ up to 0.75) on evaluating LLM responses to users demonstrating psychosis.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2512.19728","ref_index":8,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Hard Negative Sample-Augmented DPO Post-Training for Small Language Models","primary_cat":"cs.LG","submitted_at":"2025-12-17T06:15:52+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"A six-dimensional MathVerifier supplies hard negatives and per-sample weights that improve DPO performance on math reasoning for a 1.5B Qwen2.5 model over standard SFT and unweighted DPO.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2510.18196","ref_index":3,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Contrastive Decoding Mitigates Score Range Bias in LLM-as-a-Judge","primary_cat":"cs.CL","submitted_at":"2025-10-21T00:47:11+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"Contrastive decoding reduces score-range sensitivity in LLM judges for summarization and raises average Spearman correlation with human ratings by up to 11.7 percent across different score ranges.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2412.05579","ref_index":277,"ref_count":1,"confidence":0.9,"is_internal_anchor":true,"paper_title":"LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods","primary_cat":"cs.CL","submitted_at":"2024-12-07T08:07:24+00:00","verdict":"ACCEPT","verdict_confidence":"LOW","novelty_score":3.0,"formal_verification":"none","one_line_summary":"A survey that organizes LLMs-as-judges research into functionality, methodology, applications, meta-evaluation, and limitations.","context_count":1,"top_context_role":"method","top_context_polarity":"use_method","context_text":"3)SELF-REFINE [155], SELF-DEBUGGING [32], REFINER [176], Xu et al. [261], Self-Correct [93, 223], Valmeekam et al. [224] DataConstruction (§3.3) DataAnnotation (§3.3.1)He et al. [85], Gilardi et al. [71], Törnberg et al. [217], FullAnno [79], Latif et al. [117], AnnoLLM [83], LLMAAA [282] DataSynthesize (§3.3.2)SELFEE [268], SynPO [52], Arif et al. [2], SELF-INSTRUCT [235], Evol-Instruct [254, 278], STaR [277], Mendoncca et al. [157],ReSTEM [199], Kim et al. [108] METHODOLOGY (§4) Single-LLM (§4.1) Prompt-based (§4.1.1) In-Context LearningGPTScore [68], LLM-EVAL [141], TALEC [280], Jain et al. [94], ALLURE [81], Song et al. [204] Step-by-step Chain-of-Thought (CoT) [113, 242], G-EVAL [145], ICE-Score [304], ProtocoLLM [271], Chiang et al. [36],FineSurE [203]"},{"citing_arxiv_id":"2411.15594","ref_index":192,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"A Survey on LLM-as-a-Judge","primary_cat":"cs.CL","submitted_at":"2024-11-23T16:03:35+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":4.0,"formal_verification":"none","one_line_summary":"A survey on LLM-as-a-Judge that reviews reliability strategies, proposes evaluation methods, and introduces a novel benchmark for assessing such systems.","context_count":1,"top_context_role":"background","top_context_polarity":"background","context_text":"In the training phase, LLM-as-a-Judge frequently operates within reinforcement learning paradigms, where it functions as a reward model or evaluator for data or processes. This enables the creation of high-quality reasoning datasets through mechanisms such as step-by-step verification [84], Direct Preference Optimization(DPO) [115], and self-refinement [192]. Recently, several LLMs trained with reinforce- ment learning to exhibit advanced reasoning and thinking abilities have gained attention, such as o11, DeepSeek-R12,gemini-thinking3, and QVQ4. In the test-time framework, LLM-as-a-Judge is crucial for evaluating and selecting the best reasoning paths. For example, in \"Best-of-N\" gener- ation scenarios, where multiple reasoning outputs are produced, the judge determines the most"},{"citing_arxiv_id":"2408.09049","ref_index":53,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Inertia in Moral and Value Judgments of Large Language Models","primary_cat":"cs.CL","submitted_at":"2024-08-16T23:24:10+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":4.0,"formal_verification":"none","one_line_summary":"LLMs exhibit persistent inertia in value orientations, with harm avoidance and fairness remaining skewed across persona prompts.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null}],"limit":50,"offset":0}