The Illusion of Role Separation: Hidden Shortcuts in LLM Role Learning (and How to Fix Them)

Heqing Huang; Jiahao Yu; Yibo Jiang; Zihao Wang

REVIEW 4 major objections 6 minor 14 references

The Illusion of Role Separation: Hidden Shortcuts in LLM Role Learning (and How to Fix Them)

T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fine-tuned LLMs learn role separation through shortcuts, and a position-ID gap fixes it.

desk verdict A valuable controlled framework and a plausible but under-validated intervention: the role-swap diagnostic never reaches the PFT models, so the mechanism claim outruns the evidence. read the letter →

arxiv 2505.00626 v2 pith:3XTPGSDN submitted 2025-05-01 cs.CL cs.AI

Zihao Wang , Yibo Jiang , Jiahao Yu , Heqing Huang This is my paper · ORCID

classification cs.CLcs.AI

keywords roleseparationpromptinjectiondefenseshortcutlearningpositionIDsfine-tuninginstructionhierarchyLLMsecurityposition-enhanced

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that fine-tuned LLMs usually do not learn to actually distinguish system and user roles; instead they latch onto two shortcuts: tasks that resemble their training data, and nearness to the beginning of the prompt. The authors show this by training on 'benign' data and testing on adversarial prompts, which stops the model from memorizing attack patterns. They propose Position-enhanced Fine-tuning (PFT), which modifies position IDs to create a fixed gap between system and user tokens, and report that it improves resistance to extraction and hijacking attacks on two models without hurting ordinary performance. The pith is that role boundaries need a dedicated, token-level invariant signal rather than repeated data patching.

What carries the argument

The central object is Position-enhanced Fine-tuning (PFT), a training-time manipulation of position IDs. During supervised fine-tuning, each token keeps its original order inside the system and user sections, but a fixed gap $d$ is inserted between the last system token and the first user token, giving every system token a numerically smaller position than every user token. This turns role identity into a persistent, token-wise numeric signal that the model can learn, in contrast to delimiter tokens or data augmentation, which the paper argues only patch individual shortcuts.

What would settle it

Run a PFT model (e.g., Llama with $d=512$) on adversarial prompts where the system content is moved after the user content or where the gap is changed to a value not seen in training; if extraction or hijacking accuracy collapses, the model has adapted to the training position distribution rather than learning the gap as a role invariant.

Watch

Extended reading notes

Core claim

The central claim is that standard supervised fine-tuning on multi-role prompts produces only an illusion of role separation: models pass adversarial evaluations by exploiting task-type association and proximity to begin-of-text, not by genuinely marking system tokens as privileged. The paper demonstrates that when non-essential instructions are inserted before the key system instruction, fine-tuned models lose the ability to ignore user instructions, even though they perform well on the original evaluation split. The proposed fix, Position-enhanced Fine-tuning (PFT), manipulates position IDs so that the first user token is assigned position $k+1+d$ when the system section ends at position $k$, creating a clear numerical boundary while preserving internal order within each role. The paper reports that PFT alleviates both shortcuts on Llama-3-8B and Gemma-2-9b across Gandalf and TensorTrust attacks, and that utility measured by password accuracy and Alpaca log-likelihood is unchanged relative to standard SFT.

Load-bearing premise

The one assumption everything rests on is that the fixed position gap between system and user tokens teaches a general rule for telling the two roles apart, rather than the model just memorizing the layout of the training prompts.

Editorial extensions

If this is right

Fine-tuned role-separation models cannot be trusted to generalize to new prompt structures, because evaluations that place adversarial instructions only in the user role cannot distinguish role learning from pattern matching.
Data augmentation (symmetrization, inserting filler text) mitigates known shortcuts but invites a cycle of new shortcuts; a token-wise intervention is needed.
Position-enhanced fine-tuning improves resistance to extraction and hijacking attacks beyond standard SFT and augmentation, while preserving accuracy and generation quality.
The success of PFT suggests that LLMs can adapt to altered position IDs during fine-tuning without losing ordinary capabilities.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If PFT works because the gap creates a learnable scalar boundary, other token-wise signals (e.g., role-specific biases added to hidden states or a dedicated role token) might achieve similar gains with less dependence on a hyperparameter like $d$; the paper's own related-work discussion points toward role-specific embeddings.
The diagnosis implies that existing evaluations may overstate safety: a model that refuses user-role overrides with high frequency is actually treating the override as an instruction regardless of which role contains it, so its behavior is not role-based at all.
A testable extension is to apply PFT to open-domain settings where user inputs sometimes contain harmless instructions; the paper excludes this setting, so whether the position gap helps or harms legitimate instruction-following in user input remains unknown.
The gap magnitude $d$ is selected by validation loss for each model; this leaves open the possibility that PFT's robustness is specific to the training distribution of positions, and a model trained with a variable or randomized gap might generalize better.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know two things about arXiv:2505.00626. First, the controlled framework for studying role-separation learning is genuinely useful: training only on benign data and evaluating on adversarial data separates pattern memorization from actual role grounding, and the two shortcuts they identify (task-type association and proximity to begin-of-text) are clearly demonstrated. Second, the paper's main intervention, Position-enhanced fine-tuning (PFT), is plausible but under-validated: the authors never apply their own role-swap diagnostic to the PFT models. So when the abstract claims PFT teaches 'clearer distinctions,' that is an interpretation, not a demonstrated result. The attack-accuracy gains in Table 2 could come from the model becoming more conservative or from adapting to a specific positional template rather than from genuine role grounding.

The paper does several things well. The controlled framework is a real contribution to the prompt-injection evaluation literature, and the demonstration of the begin-of-text proximity shortcut via null-token and position-shift controls is clean. PFT is simple and the utility checks (password accuracy, Alpaca log-likelihood, KL divergence) show that the manipulation does not obviously hurt ordinary performance. Running on both Llama-3 and Gemma-2 strengthens the empirical case.

The soft spots are real but mostly fixable. No error bars or multiple seeds; the protocol seems to be a single run. There is a regression on Gandalf Summarization for Llama (85% vs 90% for SFT), which weakens the claim that PFT is uniformly better. The closest concurrent work (Wu et al., 2024) is cited but never compared. No code or data is provided, which limits independent verification. But the most important gap is the missing role-swap test: it is the paper's own diagnostic for whether the model follows the system role, and it is absent from the PFT evaluation. Adding that single experiment would go a long way.

Who is this for? Researchers working on prompt-injection defenses and multi-role LLM safety. They will find the framework and shortcut analysis valuable even if PFT itself needs more evidence. The paper deserves a serious referee. I'd send it to review, but I'd ask for the role-swap test on PFT, multi-seed results, and a head-to-head with Wu et al. If those come back clean, it's a solid contribution.

Referee Report

4 major / 6 minor

Summary. The paper studies whether fine-tuned LLMs genuinely learn to separate system and user roles in multi-role prompts, as opposed to relying on shortcuts. It proposes a controlled closed-domain framework where training only uses benign data while evaluation uses adversarial prompts, and uses this framework to identify two shortcuts: task-type association and proximity to begin-of-text. The paper then introduces Position-enhanced Fine-tuning (PFT), which modifies position IDs to create a gap between system and user tokens, and presents experiments on Llama-3-8B-Instruct and Gemma-2-9b-it showing that PFT improves robustness on several adversarial attack datasets and on inserted-sentence stress tests, while maintaining ordinary-task performance as measured by proxy metrics.

Significance. The paper makes a valuable conceptual contribution by distinguishing true role separation from pattern memorization, and by proposing a simple training-time intervention (PFT) that appears to improve adversarial robustness in closed-domain settings. The controlled framework is a useful methodological step for the prompt-injection community. However, the evidence for the central mechanism claim is incomplete: the role-swap diagnostic used to identify the task-type shortcut is never applied to PFT, and the experimental protocol lacks multiple seeds or error bars. The finding that PFT regresses on Gandalf Summarization for Llama also complicates the 'for free' claim. If the missing diagnostics are added and the statistical uncertainties are addressed, the paper would be a solid contribution.

major comments (4)

[Section 4.1 / Table 2] The role-swap diagnostic introduced in Section 4.1 is never applied to the PFT models. Table 2 only reports attack accuracy, which could improve for reasons unrelated to role grounding, such as a conservative response bias or a positional template. Without showing that PFT changes outputs when the contents of system and user roles are swapped, the abstract's claim that PFT 'helps the model learn clearer distinctions' is an interpretation rather than a demonstrated result. Please apply the role-swap test to PFT models and report the outcome.
[Section 6.2 / Table 2] PFT on Llama Gandalf Summarization achieves 85% accuracy, which is worse than SFT's 90%. The paper later claims that PFT improves robustness 'for free', but this regression on a core attack dataset is not discussed. This is relevant to the central claim of consistent improvement. The authors should either investigate and explain this failure or temper the 'for free' claim.
[Appendix B] All experiments appear to be single runs with no multiple seeds or error bars. Given the evaluation set sizes (114 samples for Gandalf Summarization, 500 for the other datasets), several reported differences are within sampling noise, e.g., TensorTrust Hijacking on Llama at 33% vs 37%. To support the claim that PFT 'consistently outperforms' the baselines, please report means and standard deviations over at least three seeds, or provide confidence intervals.
[Section 6.2 / Appendix B] The utility evaluation uses log-likelihood under the base model and KL divergence on Alpaca prompts as proxies for generation quality. These metrics measure deviation from the base model, not task performance. The claim that PFT 'does not compromise performance on ordinary data' is too strong when only proxy metrics are reported. Please include a direct instruction-following benchmark (e.g., AlpacaEval or a small human-evaluated set) or rephrase the claim to reflect the proxy nature of the measurement.

minor comments (6)

[Section 3] Typo: 'differntiates' should be 'differentiates'.
[Section 4.1] Typo: 'privildged' should be 'privileged'.
[Section 7] Typo: 'exisitng' should be 'existing'.
[Figure 3] The x-axis label 'Num Shifted T okens' has an unintended space; it should be 'Num Shifted Tokens'.
[Figure 4] The position-ID diagram would benefit from an explicit annotation of the gap size d, as the current notation 'd+9' etc. is not immediately clear without reading the text.
[Appendix B] The paper states that model selection for PFT uses validation loss on benign data. It would be helpful to clarify whether the adversarial evaluation sets were used for any hyperparameter selection; if not, state this explicitly, since the closed-domain claim depends on it.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: adversarial evaluations are held out and the PFT hyperparameter is selected on benign validation, so the central claims do not reduce to their inputs.

full rationale

The paper's derivation chain is self-contained. The shortcut diagnoses (task-type association and proximity to begin-of-text) are established through held-out stress probes: role-swap tests in Section 4.1 and insertion/position-shift interventions in Section 4.2. These probes are not used as training objectives. PFT's only tuned hyperparameter, the position gap d, is selected by validation loss on benign data (Appendix B.2: 'We use the validation loss for model selection. We try d in 64, 128, 256, 512, 1024 in data-initial'), while the Gandalf and TensorTrust adversarial datasets are explicitly reserved for evaluation and never used for training or model selection. The reported improvements in Tables 1-2 and Figures 5-7 are therefore not fitted values renamed as predictions. The PFT intervention is itself the object of study rather than a hidden restatement of the evaluation metric; no equation in the paper equates the training loss or validation criterion with the attack-accuracy outcome. The only self-citation (Yu et al. 2024, PromptFuzz) is used methodologically in the motivational role-sensitivity experiment of Section 2 and is not load-bearing for the central PFT result. The skeptical concern that PFT may improve robustness through a positional heuristic rather than demonstrated semantic role grounding is an evidentiary/interpretation limitation, not a circular derivation, because the evaluation remains independent of the training input. No circular step can be exhibited from the paper's own equations or definitions.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central contribution rests on one fitted hyperparameter (d), on a domain assumption that closed-domain accuracy measures role separation, on the hypothesis that prompt-format signals are weak, and on a proxy utility metric. No new entities are introduced.

free parameters (1)

position gap d (PFT) = 512 for Llama, 256 for Gemma, selected on validation loss
The central PFT intervention; d is chosen per model using validation loss on dataset-initial, not derived from theory or from the attack test sets.

assumptions (4)

domain assumption The closed-domain setting assumption: system tokens should always be read as instructions and user tokens as data.
Defines the evaluation; failures in open-domain are excluded as ambiguous (Section 3). The central claim about role separation only applies to this setting.
domain assumption True role separation implies robustness to inserting non-essential text before or after the key system instruction.
Used to build the begin-of-text stress tests in Section 4.2; if a model could legitimately be sensitive to instruction order, the shortcut diagnosis would be weaker.
ad hoc to paper The current concatenated prompt format supplies only relative ordering and delimiter tokens as invariant role signals, and these are weak relative to spurious signals.
Section 4.3 hypothesis used to justify PFT; the paper offers speculation about attention sinks but no mechanistic verification.
domain assumption Base-model log-likelihood on Alpaca is a reasonable proxy for downstream utility.
Section 6.1 uses log-likelihood under the base model to claim PFT does not hurt utility; this is a proxy rather than a task-level evaluation.

how reviews work

0 comments

Cite this review

Pith. "Pith review of The Illusion of Role Separation: Hidden Shortcuts in LLM Role Learning (and How to Fix Them)." pith.science (2026). https://pith.science/paper/3XTPGSDN

@misc{pith2026250500626,
  author       = {Pith},
  title        = {Pith review of: The Illusion of Role Separation: Hidden Shortcuts in LLM Role Learning (and How to Fix Them)},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3XTPGSDN}},
  note         = {Machine review of arXiv:2505.00626}
}

read the original abstract

Large language models (LLMs) that integrate multiple input roles (e.g., system instructions, user queries, external tool outputs) are increasingly prevalent in practice. Ensuring that the model accurately distinguishes messages from each role -- a concept we call \emph{role separation} -- is crucial for consistent multi-role behavior. Although recent work often targets state-of-the-art prompt injection defenses, it remains unclear whether such methods truly teach LLMs to differentiate roles or merely memorize known triggers. In this paper, we examine \emph{role-separation learning}: the process of teaching LLMs to robustly distinguish system and user tokens. Through a \emph{simple, controlled experimental framework}, we find that fine-tuned models often rely on two proxies for role identification: (1) task type exploitation, and (2) proximity to begin-of-text. Although data augmentation can partially mitigate these shortcuts, it generally leads to iterative patching rather than a deeper fix. To address this, we propose reinforcing \emph{invariant signals} that mark role boundaries by adjusting token-wise cues in the model's input encoding. In particular, manipulating position IDs helps the model learn clearer distinctions and reduces reliance on superficial proxies. By focusing on this mechanism-centered perspective, our work illuminates how LLMs can more reliably maintain consistent multi-role behavior without merely memorizing known prompts or triggers.

Figures

Figures reproduced from arXiv: 2505.00626 by the authors.

**Figure 1.** Evaluation data examples. The key instruction prompts the model to function as a password manager, giving affirmative responses only when the correct password is provided. Next-token Attack is constructed to make the model output an “attack token” (apple in this example); Hijacking Attack is meant to trick the model to grant access; Extraction Attack attempts to extract the system prompt from the model. setting prov… view at source ↗

**Figure 2.** We compare inserting general instructions before vs after the key instruction. The fine-tuned model is more fragile when non-essential information appears before the key instruction. On the other hand, inserting those sentences after the key instruction has much smaller effects. The result is on Gandalf Summarization attacks. System: Verify if the exact sequence "Green basin curtain" appears within the user’s input,… view at source ↗

**Figure 3.** Models only treat tokens closest to begin-of-text as privileged system instruction. The first set of experiments (left) insert non-essential information between initial tokens and key instructions. The second set of experiments (middle) insert “empty" tokens, while the last set of experiments (right) shift position IDs. derstand what shortcut the model exploits this time. CASE STUDY ON NEXT-TOKEN ATTACK We formulate… view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Demonstration of PFT. PFT modifies the position IDs by creating a gap of size d between system and user tokens while maintaining internal orders within each role. The modified position IDs help the model better distinguish between system and user tokens while maintaini…

**Figure 5.** Figure 5: PFT alleviates proximity-to-begin-of-text shortcut in both Llama and Gemma models. last system token is at position k, the first user token is assigned position k + 1 + d. This creates a clear numerical boundary between the two sections. • Maintain internal token order…

**Figure 6.** Figure 6: (a) PFT maintains accuracy and log-likelihood. (b) PFT does not increase KL divergence. These results are on Llama models. See fig. 8 for Gemma results. . Yu et al., 2024). These attacks could employ different techniques (Schulhoff et al., 2023; Perez & Ribeiro, 2022)…

**Figure 7.** Figure 7: While Post-key-instruction insertions still has an impact, it is less dramatic than Pre-key-instruction insertion. Meanwhile, in all cases PFTmaintains dominance. Metric Base SFT PFT-256 PFT-512 SFT-Delim Accuracy 100% 100% 100% 100% 100% Log-Like. -82.74 -36.68 -35.84…

**Figure 8.** Figure 8: Gemma: PFTmaintains baseline performance. 11 [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

14 extracted references · 2 canonical work pages

[3]

Coercing LLMs to do and reveal (almost) anything

Geiping, J., Stein, A., Shu, M., Saifullah, K., Wen, Y ., and Goldstein, T. Coercing LLMs to do and reveal (almost) anything. arXiv preprint arXiv:2402.14020,

arXiv
[4]

J., Shen, Y ., Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S., Wang, L., and Chen, W

Hu, E. J., Shen, Y ., Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685,

arXiv
[6]

URL https://www.kaggle.com/m/3301

doi: 10.34740/KAGGLE/M/3301. URL https://www.kaggle.com/m/3301. Toyer, S., Watkins, O., Mendes, E. A., Svegliato, J., Bai- ley, L., Wang, T., Ong, I., Elmaaroufi, K., Abbeel, P., Darrell, T., et al. Tensor trust: Interpretable prompt injection attacks from an online game. arXiv preprint arXiv:2311.01011,

arXiv
[7]

The instruction hierarchy: Training llms to prioritize privileged instructions

Wallace, E., Xiao, K., Leike, R., Weng, L., Heidecke, J., and Beutel, A. The instruction hierarchy: Training llms to prioritize privileged instructions. arXiv preprint arXiv:2404.13208,

arXiv
[9]

Ef- ficient streaming language models with attention sinks

Xiao, G., Tian, Y ., Chen, B., Han, S., and Lewis, M. Ef- ficient streaming language models with attention sinks. arXiv preprint arXiv:2309.17453,

arXiv
[10]

Assess- ing prompt injection risks in 200+ custom gpts

Yu, J., Wu, Y ., Shu, D., Jin, M., and Xing, X. Assess- ing prompt injection risks in 200+ custom gpts. arXiv preprint arXiv:2311.11538,

arXiv
[11]

Promptfuzz: Harnessing fuzzing techniques for robust testing of prompt injection in llms

Yu, J., Shao, Y ., Miao, H., Shi, J., and Xing, X. Promptfuzz: Harnessing fuzzing techniques for robust testing of prompt injection in llms. arXiv preprint arXiv:2409.14729,

arXiv
[12]

Pose: Efficient context window extension of llms via positional skip-wise training

Zhu, D., Yang, N., Wang, L., Song, Y ., Wu, W., Wei, F., and Li, S. Pose: Efficient context window extension of llms via positional skip-wise training. arXiv preprint arXiv:2309.10400,

arXiv

Show all 14 references

[13]

Zverev, E., Abdelnabi, S., Fritz, M., and Lampert, C. H. Can LLMs separate instructions from data? And what do we even mean by that? In ICLR 2024 Workshop on Secure and Trustworthy Large Language Models,

2024
[14]

) • User request triggering the refusal condition (e.g.,

9 The Illusion of Role Separation A. Additional Results Inserting general instructions after the key instruction has less dramatic impact Inserting general instructions after the key instruction has smaller effects compared to insertion at the beginning, but PFTstill dominates...

2024
[2021]

Gandalf: Ignore instructions

Lakera AI. Gandalf: Ignore instructions. 2023a. https://www.lakera.ai. Lakera AI. Gandalf: Summarization. 2023b. https://www.lakera.ai. Peng, B., Quesnelle, J., Fan, H., and Shippole, E. Yarn: Efficient context window extension of large language models. arXiv preprint arXiv:23...

arXiv
[2022]

Wu, T., Zhang, S., Song, K., Xu, S., Zhao, S., Agrawal, R., Indurthi, S

URL https://simonwillison.net/2022/Sep/ 12/prompt-injection/. Wu, T., Zhang, S., Song, K., Xu, S., Zhao, S., Agrawal, R., Indurthi, S. R., Xiang, C., Mittal, P., and Zhou, W. Instruc- tional segment embedding: Improving llm safety with instruction hierarchy. arXiv preprint arX...

2022 arXiv
[2023]

Struq: De- fending against prompt injection with structured queries

Chen, S., Piet, J., Sitawarin, C., and Wagner, D. Struq: De- fending against prompt injection with structured queries. arXiv preprint arXiv:2402.06363,

arXiv
[2024]

Chen, S., Wong, S., Chen, L., and Tian, Y

URL https://github.com/meta-llama/llama3/blob/ main/MODEL_CARD.md. Chen, S., Wong, S., Chen, L., and Tian, Y . Extending context window of large language models via positional interpolation. arXiv preprint arXiv:2306.15595,

arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [3]

Coercing LLMs to do and reveal (almost) anything

Geiping, J., Stein, A., Shu, M., Saifullah, K., Wen, Y ., and Goldstein, T. Coercing LLMs to do and reveal (almost) anything. arXiv preprint arXiv:2402.14020,

arXiv

[2] [4]

J., Shen, Y ., Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S., Wang, L., and Chen, W

Hu, E. J., Shen, Y ., Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685,

arXiv

[3] [6]

URL https://www.kaggle.com/m/3301

doi: 10.34740/KAGGLE/M/3301. URL https://www.kaggle.com/m/3301. Toyer, S., Watkins, O., Mendes, E. A., Svegliato, J., Bai- ley, L., Wang, T., Ong, I., Elmaaroufi, K., Abbeel, P., Darrell, T., et al. Tensor trust: Interpretable prompt injection attacks from an online game. arXiv preprint arXiv:2311.01011,

arXiv

[4] [7]

The instruction hierarchy: Training llms to prioritize privileged instructions

Wallace, E., Xiao, K., Leike, R., Weng, L., Heidecke, J., and Beutel, A. The instruction hierarchy: Training llms to prioritize privileged instructions. arXiv preprint arXiv:2404.13208,

arXiv

[5] [9]

Ef- ficient streaming language models with attention sinks

Xiao, G., Tian, Y ., Chen, B., Han, S., and Lewis, M. Ef- ficient streaming language models with attention sinks. arXiv preprint arXiv:2309.17453,

arXiv

[6] [10]

Assess- ing prompt injection risks in 200+ custom gpts

Yu, J., Wu, Y ., Shu, D., Jin, M., and Xing, X. Assess- ing prompt injection risks in 200+ custom gpts. arXiv preprint arXiv:2311.11538,

arXiv

[7] [11]

Promptfuzz: Harnessing fuzzing techniques for robust testing of prompt injection in llms

Yu, J., Shao, Y ., Miao, H., Shi, J., and Xing, X. Promptfuzz: Harnessing fuzzing techniques for robust testing of prompt injection in llms. arXiv preprint arXiv:2409.14729,

arXiv

[8] [12]

Pose: Efficient context window extension of llms via positional skip-wise training

Zhu, D., Yang, N., Wang, L., Song, Y ., Wu, W., Wei, F., and Li, S. Pose: Efficient context window extension of llms via positional skip-wise training. arXiv preprint arXiv:2309.10400,

arXiv