Challenges and Applications of Large Language Models

1. CXXCrafter: An LLM-Based Agent for Automated C/C++ Open Source Software Building

   cs.SE 2025-05 conditional novelty 7.0 of 10

   An LLM-driven agent, CXXCrafter, automatically builds 587 of 752 C/C++ open-source projects (78%), beating default build commands (39%) and bare LLMs (32 to 38%).

2. RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Models

   cs.CL 2025-04 conditional novelty 7.0 of 10

   RAG can make language models less safe than their non-RAG equivalents, even with safe documents, and current jailbreak methods transfer poorly to RAG.

3. Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Models

   cs.CL 2024-12 conditional novelty 7.0 of 10

   Jailbreak attacks push LLM activations outside a safety boundary, mostly in low and middle layers, and a tanh-based penalty that pulls activations back inside this boundary blocks most tested attacks with under 2% uti...

4. Hidden Language Consistency Phenomena in Reasoning LLMs

   cs.CL 2026-08 conditional novelty 6.0 of 10

   Reasoning models often stop using the requested language as problems get harder, and this language breakdown can make accuracy look better than it is.

5. FastTPS: An Optimized Method for LLM Token Phase for AI accelerators

   cs.LG 2026-07 conditional novelty 6.0 of 10

   FastTPS accelerates LLM token-phase inference via reloading-free static KV-cache management, tiled fused RoPE attention, and interlaced-weight MLP fusion, yielding up to 6× speedup at 93% bandwidth on AMD NPUs.

6. Interpreting learning dynamics of autoencoders: Transient scaling and emerging concepts of the Ising model

   cs.LG 2026-07 conditional novelty 6.0 of 10

   Unsupervised autoencoders on Ising configurations form magnetization then energy representations in two dynamical regimes, with recursive error flow fields sharing topology across layers.

7. Agentic and Generative AI for Open-Source Intelligence and Cyber Investigations: Taxonomy, Evaluation, Challenges, and Future Directions

   cs.CR 2026-07 conditional novelty 6.0 of 10

   Across 74 OSINT/CTI AI studies, hallucination is widely named but end-to-end measured in only one non-reproducible system, so a human–AI co-pilot is the most defensible near-term architecture.

8. Real Faults in Model Context Protocol (MCP) Software: a Comprehensive Taxonomy

   cs.SE 2026-03 conditional novelty 6.0 of 10

   MCP server faults form five empirical categories—server setting, server/tool configuration, server/host configuration, documentation, and general programming—confirmed by a 41-practitioner survey.

9. Enhancing Robustness of Autoregressive Language Models against Orthographic Attacks via Pixel-based Approach

   cs.CL 2025-08 conditional novelty 6.0 of 10

   A word-as-image pixel language model trained with next-token prediction reports lower perplexity than a token-embedding LLaMA on noisy and non-Latin-script text, though its noise evaluation holds tokenization fixed.

10. PhantomHunter: Detecting Unseen Privately-Tuned LLM-Generated Text via Family-Aware Learning

    cs.CL 2025-06 conditional novelty 6.0 of 10

    PhantomHunter detects text from privately fine-tuned LLMs by learning shared token-probability traits within LLaMA, Gemma and Mistral families, reporting F1 above 96% on held-out derivatives.

11. InFact: Informativeness Alignment for Improved LLM Factuality

    cs.CL 2025-05 conditional novelty 6.0 of 10

    InFACT trains LLMs with hierarchical informativeness rewards plus abstention, improving factual precision on QA benchmarks while largely preserving recall.

12. Real-Time Verification of Embodied Reasoning for Generative Skill Acquisition

    cs.RO 2025-05 conditional novelty 6.0 of 10

    VERGSA trains a process reward model on MCTS-labeled subtask outcomes and uses it to select scene configurations and subtask supervisions, improving simulated task success rates.

13. MedArabiQ: Benchmarking Large Language Models on Arabic Medical Tasks

    cs.CL 2025-05 conditional novelty 6.0 of 10

    MedArabiQ is a seven-task Arabic medical benchmark showing that closed models generally beat open ones on structured questions, while BERTScore misses serious hallucinations that an LLM judge later reveals.

14. OET: Optimization-based prompt injection Evaluation Toolkit

    cs.CR 2025-05 conditional novelty 6.0 of 10

    OET is an optimization-based evaluation toolkit that benchmarks prompt injection attacks and defenses across eight datasets and shows current defenses remain vulnerable in several domains.

15. Evaluating Evaluation Metrics -- The Mirage of Hallucination Detection

    cs.CL 2025-04 conditional novelty 6.0 of 10

    A large-scale comparison of six hallucination detection metric families across 37 models and five decoding methods finds most metrics align poorly with human judgments, with GPT-4-based evaluation performing best.

16. Helping Blind People Grasp: Enhancing a Tactile Bracelet with an Automated Hand Navigation System

    cs.HC 2025-04 conditional novelty 6.0 of 10

    An automated vision-to-vibration hand navigation system on a tactile bracelet lets blindfolded and blind users grasp target objects, track one instance among distractors, and avoid obstacles.

17. Are Vision LLMs Road-Ready? A Comprehensive Benchmark for Safety-Critical Driving Video Understanding

    cs.CV 2025-04 conditional novelty 6.0 of 10

    DVBench introduces 10,000 expert-annotated questions on crash and near-crash driving videos and reports that no tested vision LLM exceeds 40 percent accuracy under its strict GroupEval scoring.

18. Mind the Gap! Choice Independence in Using Multilingual LLMs for Persuasive Co-Writing Tasks in Different Languages

    cs.CL 2025-02 conditional novelty 6.0 of 10

    Users who first used a Spanish AI writing assistant subsequently used the English AI writing assistant less, suggesting a spillover that violates choice independence.

19. SelfElicit: Your Language Model Secretly Knows Where is the Relevant Evidence

    cs.CL 2025-02 conditional novelty 6.0 of 10

    SelfElicit uses deep-layer attention to automatically highlight relevant evidence sentences in the input context, yielding consistent QA accuracy gains across six instruction-tuned LLMs.

20. Coarse-to-Fine Process Reward Modeling for Mathematical Reasoning

    cs.AI 2025-01 conditional novelty 6.0 of 10

    Merging adjacent reasoning steps into coarser training steps for process reward models improves best-of-n accuracy on GSM-Plus and MATH500 by about 0.5 to 3.4 percentage points.

21. Consolidating TinyML Lifecycle with Large Language Models: Reality, Illusion, or Opportunity?

    cs.SE 2025-01 conditional novelty 6.0 of 10

    An LLM-based framework automates TinyML data processing and model conversion reliably, but automated Arduino sketch generation fails in 63.3% of runs, making full automation an open challenge.

22. Context-DPO: Aligning Language Models for Context-Faithfulness

    cs.CL 2024-12 conditional novelty 6.0 of 10

    Context-DPO fine-tunes LLMs with direct preference optimization on counterfactual passages, yielding 35-280% context-faithfulness gains on its new ConFiQA benchmark.

23. Feature Coding in the Era of Large Models: Dataset, Test Conditions, and Benchmark

    cs.MM 2024-12 conditional novelty 6.0 of 10

    A public benchmark and unified test conditions for compressing intermediate features of large models, with two image-codec baselines evaluated.

24. SMoLoRA: Exploring and Defying Dual Catastrophic Forgetting in Continual Visual Instruction Tuning

    cs.CV 2024-11 conditional novelty 6.0 of 10

    SMoLoRA uses two separately routed LoRA expert groups, one for visual understanding and one for instruction following, to reduce dual catastrophic forgetting in continual visual instruction tuning.

25. Neon: News Entity-Interaction Extraction for Enhanced Question Answering

    cs.CL 2024-11 conditional novelty 6.0 of 10

    Neon builds a timestamped knowledge graph of entity-event tuples extracted from news, and augmenting LLM prompts with these tuples improves temporal entity-centric question answering.

26. The VLLM Safety Paradox: Dual Ease in Jailbreak Attack and Defense

    cs.CR 2024-11 conditional novelty 6.0 of 10

    Near-perfect jailbreak defenses for vision-language models are mostly over-refusal, and the two standard ways of scoring jailbreaks agree only at chance level.

27. Semantic Drift and the Stability of Operator Control in Reasoning-Class Decision Support Systems

    cs.AI 2026-07 conditional novelty 5.5 of 10

    Reasoning LLMs in ultra-long sessions exhibit latent semantic drift that inverts operator control; a fitted stability coefficient Ks detects the bifurcation and a latent-steering arbitrator is proposed to restore it.

28. Is Inter-Seed Cross-Play Enough? Evaluating the Robustness of Zero-Shot Coordination Algorithms to Implementation Details

    cs.AI 2026-08 conditional novelty 5.0 of 10

    For Other-Play in Yokai, agents trained with different implementation details coordinate across implementations about as well as across seeds, supporting inter-seed cross-play as a proxy for cross-implementation evaluation.

29. Backdoor Samples Detection Based on Perturbation Discrepancy Consistency in Pre-trained Language Models

    cs.CR 2025-08 conditional novelty 5.0 of 10

    Backdoor text samples show smaller log-probability changes under mask-filling perturbations than clean samples, which enables zero-shot backdoor detection without the poisoned model.

30. What Language(s) Does Aya-23 Think In? How Multilinguality Affects Internal Language Representations

    cs.CL 2025-07 reject novelty 5.0 of 10

    Aya-23-8B appears to activate multiple related languages internally and concentrate code-mixing neurons in final layers, but the paper's own limitations undercut the claim that these are language-specific neurons.

31. The Impact of Fine-tuning Large Language Models on Automated Program Repair

    cs.SE 2025-07 conditional novelty 5.0 of 10

    On three Java APR benchmarks, LoRA and IA3 adapters match or beat full-model fine-tuning for most tested code LLMs while training less than one percent of parameters.

32. Hallucination Detection with Small Language Models

    cs.CL 2025-06 reject novelty 5.0 of 10

    A multi-small-model ensemble with sentence splitting, z-score normalization, and harmonic mean detects hallucinations in RAG answers with a reported 10% F1 gain over single-model baselines.

33. DLM-One: Diffusion Language Models for One-Step Sequence Generation

    cs.CL 2025-05 conditional novelty 5.0 of 10

    DLM-One distills a continuous diffusion language model into a one-step student, achieving roughly 500x inference speedup while staying within a few percent of the teacher on BLEU, ROUGE, and BERTScore, with substantia...

34. AnchorAttention: Difference-Aware Sparse Attention with Stripe Granularity

    cs.LG 2025-05 conditional novelty 5.0 of 10

    AnchorAttention uses the maximum attention score from initial and local tokens as an anchor to threshold-select important key-value positions at stripe granularity, achieving faster prefill with comparable accuracy.

35. Masking in Multi-hop QA: An Analysis of How Language Models Perform with Context Permutation

    cs.CL 2025-05 conditional novelty 5.0 of 10

    Ordering retrieved documents along the reasoning chain and replacing the causal mask with a prefix mask during LoRA fine-tuning improves multi-hop QA accuracy; peak attention scores can select the best context order.

36. Enigme: Generative Text Puzzles for Evaluating Reasoning in Language Models

    cs.AI 2025-05 conditional novelty 5.0 of 10

    The paper introduces enigme, a procedurally generated text-puzzle library for benchmarking reasoning in transformer-decoder language models.

37. Understanding and Mitigating Risks of Generative AI in Financial Services

    cs.CL 2025-04 conditional novelty 5.0 of 10

    Open-source AI guardrails miss most financial-services content risks that a new domain-specific taxonomy identifies, even when their prompts are expanded to cover the new categories.

38. A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment

    cs.CR 2025-04 conditional novelty 5.0 of 10

    A large collaborative survey organizes LLM and LLM-agent safety issues into a full-stack lifecycle framework from data preparation to deployment.

39. To Code or not to Code? Adaptive Tool Integration for Math Language Models via Expectation-Maximization

    cs.AI 2025-02 conditional novelty 5.0 of 10

    An EM-style training loop lets 7B math LLMs learn when to invoke code, improving MATH500 by 11 points and AIME by 9.4 points.

40. Generative AI Uses and Risks for Knowledge Workers in a Science Organization

    cs.HC 2025-01 accept novelty 5.0 of 10

    At Argonne National Lab, early adopters of generative AI reported copilot and workflow agent use cases, small but growing usage, and concerns about reliability, privacy, academic publishing, and jobs.

41. PromptShield: Deployable Detection for Prompt Injection Attacks

    cs.CR 2025-01 conditional novelty 5.0 of 10

    PromptShield reports a 65.3% true positive rate at 0.1% false positive rate for prompt injection detection, more than six times the best prior model, on its own out-of-distribution evaluation split.

42. LLM Augmentations to support Analytical Reasoning over Multiple Documents

    cs.CL 2024-11 conditional novelty 5.0 of 10

    LLMs alone and with dynamic evidence tree augmentation still fail to produce the implicit, speculative reasoning that intelligence analysis requires.

43. Developer Challenges on Large Language Models: A Study of Stack Overflow and OpenAI Developer Forum Posts

    cs.SE 2024-11 conditional novelty 5.0 of 10

    A BERTopic analysis of 8,593 Stack Overflow posts and 26,474 OpenAI Developer Forum posts yields 9 and 17 LLM developer challenge topics, with API usage dominant and high unresolved rates.

44. CoCoP: Enhancing Text Classification with LLM through Code Completion Prompt

    cs.CL 2024-11 conditional novelty 5.0 of 10

    CoCoP, which formats text classification as code completion, improves LLM accuracy over few-shot prompting and lets small code models approach large general models.

45. Psychologically Enhanced AI Agents

    cs.AI 2025-09 conditional novelty 4.0 of 10

    MBTI personality prompts measurably change how LLM agents write stories and play strategic games, with self-reflection before communication supporting cooperative behavior.

46. Insights into User Interface Innovations from a Design Thinking Workshop at deRSE25

    cs.HC 2025-08 conditional novelty 4.0 of 10

    A workshop at deRSE25 produced seven user-interface sketches for LLMs that emphasize branching, context management, and user weighting, which the authors map onto their whiteboard-based interface concept.

47. LOCOFY Large Design Models -- Design to code conversion solution

    cs.SE 2025-07 reject novelty 4.0 of 10

    A proprietary design-to-code pipeline is described with claimed high fidelity and LLM outperformance, but the evaluation is self-referential, unquantified, and unreproducible.

48. Large Language Models in Cybersecurity: Applications, Vulnerabilities, and Defense Techniques

    cs.CR 2025-07 conditional novelty 4.0 of 10

    A survey that maps LLM applications, vulnerabilities, and defenses across eight cybersecurity domains, but with significant citation and rigor problems.

49. Exploring the Limits of Model Compression in LLMs: A Knowledge Distillation Study on QA Tasks

    cs.CL 2025-07 conditional novelty 4.0 of 10

    Distilled students at 43% to 50% of teacher size keep over 90% of teacher Exact Match on SQuAD and MLQA, though one-shot gains reverse on SQuAD test for Pythia.

50. Large Language Model Powered Intelligent Urban Agents: Concepts, Capabilities, and Applications

    cs.MA 2025-07 conditional novelty 4.0 of 10

    The paper defines urban LLM agents, surveys their sensing, memory, reasoning, execution, and learning workflows, and organizes their applications across planning, transportation, environment, safety, and society.

51. Software Engineering for Large Language Models: Research Status, Challenges and the Road Ahead

    cs.SE 2025-06 conditional novelty 4.0 of 10

    A literature review organizes LLM development into a six-phase software engineering lifecycle and identifies challenges and research directions for each phase.

52. LFTF: Locating First and Then Fine-Tuning for Mitigating Gender Bias in Large Language Models

    cs.CL 2025-05 reject novelty 4.0 of 10

    A block-localizing fine-tuning method for gender debiasing is presented, but its stated loss is inconsistent with its reported behavior and the evaluation tables contain duplicate rows.

53. Validating the Effectiveness of a Large Language Model-based Approach for Identifying Children's Development across Various Free Play Settings in Kindergarten

    cs.AI 2025-05 reject novelty 4.0 of 10

    An LLM-based pipeline labeled kindergarten play narratives and achieved high rater agreement, but the claimed validity as a measure of child development is not supported by the evidence.

54. Boosting Self-Efficacy and Performance of Large Language Models via Verbal Efficacy Stimulations

    cs.CL 2025-02 conditional novelty 4.0 of 10

    Emotionally styled verbal prompts (encouraging, provocative, critical) modestly improve zero-shot LLM accuracy on many tasks, with the best style varying by model and task zone.

55. Dynamic benchmarking framework for LLM-based conversational data capture

    cs.CL 2025-02 conditional novelty 4.0 of 10

    An LLM-based framework that benchmarks conversational data capture using synthetic users, applied to loan applications, shows adaptive follow-up questions improve extraction accuracy.

56. AI Governance through Markets

    econ.GN 2025-01 conditional novelty 4.0 of 10

    Market governance mechanisms, supported by standardized AI disclosures, can create financial incentives for responsible AI development, according to this policy paper.

57. Towards Advancing Code Generation with Large Language Models: A Research Roadmap

    cs.SE 2025-01 conditional novelty 4.0 of 10

    A roadmap paper that organizes LLM code generation into a six-layer architecture and a four-phase human-in-the-loop workflow, and lists open challenges and recommendations.

58. Visual RAG: Expanding MLLM visual knowledge without fine-tuning

    cs.CV 2025-01 conditional novelty 4.0 of 10

    Retrieval-selected demonstration examples let a multimodal LLM classify images as accurately as random many-shot prompting with far fewer examples.

59. Adaptive Parameter-Efficient Federated Fine-Tuning on Heterogeneous Devices

    cs.DC 2024-12 conditional novelty 4.0 of 10

    Assigning federated fine-tuning devices different numbers of LoRA layers near the output, with ranks increasing toward the output, reaches target accuracy 1.5-2.8x faster and with up to 42.3% less communication than e...

60. RAG Playground: A Framework for Systematic Evaluation of Retrieval Strategies and Prompt Engineering in RAG Systems

    cs.LG 2024-12 conditional novelty 4.0 of 10

    A new RAG evaluation framework reports that hybrid vector-keyword retrieval and structured self-evaluation prompting improve answer quality, reaching a 72.7% pass rate on its own unvalidated metrics.

See all 74 Pith citations