Pith. sign in

REVIEW 3 major objections 5 minor 82 references

Do Influence Tactics Matter? Investigating Prompt Framing Effects in LLM Code Generation

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Coercive prompt wording reduces the correctness and security of LLM-generated code.

desk verdict A large, transparent empirical study of influence-tactic framing in LLM code generation; the headline pressure-framing result is plausible but the binary-outcome model is misspecified, so the correctness claim needs a re-analysis before I would repeat it. read the letter →

arxiv 2608.11513 v1 pith:GG5BQZCQ submitted 2026-08-11 cs.SE cs.AIcs.CL

Alex Deaconu , Anubhav Gupta , Manaal Basha , Nicholas Haydu , Gema Rodríguez-Pérez This is my paper · ORCID
classification cs.SEcs.AIcs.CL
keywords influencetacticspromptframinglargelanguagemodelscodegenerationsoftwarequalitysecurityengineeringempirical
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tests whether psychologically inspired ways of phrasing a request, borrowed from a management-research taxonomy of influence tactics, change the code that LLMs produce when the task content is held fixed. Across roughly 123,000 LiveCodeBench generations and 57,000 SWE-bench Verified generations from five open-weight LLMs, prompts framed with pressure or coercive urgency produced functionally correct code less often and code with more static-analysis security warnings than a plain neutral prompt. The effect was strongest on structured algorithmic problems; on real-world GitHub maintenance tasks, the only consistent framing effect was that pressure made output more verbose. The paper's practical point is that prompt wording is a real but secondary factor in code generation, and that coercive phrasing is the one to avoid when correctness and security matter.

What carries the argument

The argument is carried by reproducible prompt templates that translate eight influence tactics from Yukl and Falbe's taxonomy, a classification of persuasion behaviours from organizational psychology, into concrete wording, with each template built from the item requirements of the validated Influence Behavior Questionnaire-General. The key contrast is Neutral, a straightforward request, versus Pressure, which adds demands, surveillance, and threatened consequences. The quantitative engine is a mixed-effects model treating ProblemID as a random intercept, with tactic, LLM, and difficulty as fixed effects; binary correctness is modelled with a generalized linear mixed model described as negative-binomial with log link, and continuous quality metrics with a linear mixed model. The evaluation stack, using LiveCodeBench and SWE-bench Verified harnesses for correctness, Radon for complexity and maintainability, PyLint for style, and Bandit for security, turns a linguistic manipulation into measurable software-quality signals.

What would settle it

Re-run the LiveCodeBench comparison of Neutral versus Pressure using a model appropriate for binary outcomes, such as a logistic or binomial GLMM with the same fixed and random effects, and check whether the $p=0.002$ and $p=0.03$ contrasts survive; if they do not, the paper's headline claim fails. A complementary check is to paraphrase the Pressure prompt into different coercive wordings and see whether the correctness drop is reproducible.

Watch

Extended reading notes

Core claim

The central claim is that influence-tactic prompt framings, particularly pressure, shift LLM code-generation outcomes even though the underlying task is identical. On LiveCodeBench, neutral prompts significantly outperformed both Pressure ($p=0.002$) and Pressure Alternative ($p=0.03$) on functional correctness, and both pressure variants were associated with significantly more Bandit-flagged security warnings than Neutral ($p<0.001$ and $p=0.0004$). By contrast, no significant tactic effects appeared for maintainability, complexity, PyLint scores, or lines of code on LiveCodeBench, and on SWE-bench Verified the only significant pairwise framing effect was that Pressure produced more code than Neutral ($p=0.0025$); model identity and task difficulty accounted for most of the variance. The authors interpret the effects as distributional cues learned from human communication rather than as evidence that LLMs experience pressure, and they frame the result as the first large-scale demonstration that psychologically grounded prompt framing is a measurable, if modest, factor in code generation.

Load-bearing premise

The load-bearing premise is that the statistical model in Eq. (2), described as a negative-binomial GLMM with a log link, is a valid way to analyze the binary pass/fail correctness outcome, and the paper gives no diagnostic check showing that this count-style model fits binary data better than a standard logistic GLMM.

Editorial extensions

If this is right

  • Developers should avoid coercive or urgency-laden phrasing in code-generation prompts when correctness and security are priorities, because neutral wording performed best in this study.
  • Prompt framing is a minor but non-negligible factor: model choice and task difficulty explain far more variance, so selecting a capable model matters more than polishing prompt tone.
  • Framing effects are task-dependent: they showed up in structured algorithmic problems on LiveCodeBench but mostly disappeared in realistic maintenance tasks on SWE-bench Verified, where only verbosity shifted.
  • Influence-tactic framings can alter surface features of generated code, such as comment density, explanation length, error handling, and hallucination patterns, but these changes are too modest to be a reliable style-control technique.
  • The released prompt templates give other researchers a reusable, taxonomy-aligned instrument for studying pragmatic framing in LLM code generation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the distributional-cue explanation is right, the effect size should track how strongly a model's training corpus associates coercive phrasing with truncated or non-deliberative responses; this could be tested by holding the task constant and varying only the frequency of words like 'must', 'urgent', and 'watching'.
  • Replacing the negative-binomial GLMM with a logistic GLMM for the binary correctness outcome would be the first robustness check a reader should run, because the paper's reported p-values for the main correctness claim depend on that distributional choice.
  • The authors' contrast with adversarial persuasion results, where psychologically framed prompts can achieve very high success rates in jailbreak-style settings, suggests that pressure framing may matter far more in open-ended or security-sensitive interactions than in benchmarked code tasks; a direct test would combine the Pressure template with a prompt-injection payload and measure success rates
  • Because the reasoning model had only a single run and SWE-bench cross-run absolute change was 26.56 percent, the null results on maintenance tasks should be treated as provisional until multi-seed replicates are available.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper investigates whether prompt framings based on Yukl and Falbe's influence-tactic taxonomy affect the code generated by five open-weight LLMs. Eight influence tactics plus a neutral baseline are operationalized as prompt templates and evaluated on LiveCodeBench (1,055 problems) and SWE-bench Verified (485 problems), with approximately 123,000 and 57,000 generations respectively. Outcomes are functional correctness, code-quality metrics (cyclomatic complexity, maintainability index, PyLint, SLOC, comment percentage), and Bandit security warnings, complemented by a qualitative codebook analysis. The headline findings are that, on LiveCodeBench, Pressure and Pressure Alternative prompts reduce functional correctness and increase security warnings relative to Neutral prompts, while effects on SWE-bench Verified are mostly absent. The paper concludes that model choice matters more than prompt framing, but that coercive wording is a small, non-negligible risk factor.

Significance. If the results hold, this is a useful empirical contribution to the prompt-engineering and LLM4SE literature. The study is carefully scoped, uses two established external benchmarks, includes five models and multiple runs for non-reasoning models, applies Bonferroni corrections, and ships a replication package. The qualitative phase is conducted with blinded coders and inter-rater reliability reporting, which is a strength. The main limitation is the statistical model used for the binary correctness outcome; because the headline claim depends on that model, the current manuscript does not provide verified statistical support for its central contribution. The security warning finding, based on count outcomes, is more defensible, and the qualitative evidence is suggestive rather than conclusive.

major comments (3)
  1. [Section 3.6, Eq. (2)] The functional correctness outcome is binary (Section 3.4 defines C_i in {0,1}), but Eq. (2) specifies a negative binomial GLMM with a log link. A negative binomial distribution has support on nonnegative integers and variance mu + mu^2/theta, which cannot represent a Bernoulli response whose variance is mu(1-mu); the resulting standard errors and p-values are therefore not trustworthy. All LiveCodeBench correctness p-values in Table 5 and Section 4.1, including the central claim that Neutral outperforms Pressure (p=0.002) and Pressure Alternative (p=0.03), come from this model. The authors should refit the correctness analyses with a Bernoulli/binomial GLMM using a logit link (or an equivalent appropriate model) and report the resulting estimates. The Bandit security warning analyses concern count outcomes and are not affected by this particular objection.
  2. [Section 3.6] The paper states that 'model fit was validated using residual diagnostics' and that 'all models were fit using Restricted Maximum Likelihood (REML),' but no residual plots, dispersion checks, or model-comparison diagnostics are presented. REML is not a standard estimation criterion for GLMMs, so the reader cannot verify the distributional adequacy of the fitted models. The authors should either provide the diagnostics in an appendix or explicitly describe the estimation method (e.g., Laplace approximation or adaptive Gauss-Hermite quadrature) and justify its use.
  3. [Section 3.6 and Section 4] All statistical inferences are based on the first trial per condition for non-reasoning models, rather than on all three runs, and run-level variability is only reported descriptively in Table 3. For SWE-bench Verified the between-run variability is substantial (mean 10.31% difference, absolute change 26.56%), so inferences based on a single run may not be stable. The authors should either include run as a random effect or perform a sensitivity analysis across all three runs to confirm that the reported conclusions, especially the null results for most tactics on SWE-bench, are not artifacts of selecting the first trial.
minor comments (5)
  1. [Section 2.1] The phrase 'which established the distinctiveness and utility of the expanded autonomy' appears to be a typo for 'expanded taxonomy'.
  2. [Section 3.1.2] The sentence 'responses to SWE-bench must be written in diffformat' should read 'in diff format'.
  3. [Section 4.3] The qualitative analysis says 1,600 samples were selected but only 350 prompt completions were ultimately coded; the relationship between the sampled 1,600 and the coded 350 should be clarified, including whether the reported frequencies are based on the 350 or the full 1,600.
  4. [Table 5] The table reports eta-squared values (such as eta^2_p(Tactic)=0.015) alongside results from GLMMs; it should be clarified how partial eta-squared is computed for generalized mixed models, or the effect-size measure should be changed to one that is standard for the fitted model family.
  5. [Section 4.1, representative example] The two-response example is illustrative but anecdotal; it should be labeled as such and not presented as independent evidence for the quantitative conclusions.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the study is an empirical benchmark evaluation using external datasets and external evaluation metrics, and the load-bearing claims rest on observed output differences rather than on fitted inputs or self-cited premises.

full rationale

The paper's central claims are empirical comparisons of LLM-generated code under different prompt framings. Correctness is measured by the official LiveCodeBench and SWE-bench Verified test harnesses, and security, maintainability, and quality are measured by external tools (Bandit, PyLint, Radon). The influence-tactic prompt templates are constructed from the IBQ-G behavioral descriptions before any outcome is observed; the Neutral-versus-Pressure contrast is then estimated from the resulting model outputs. No equation or definition reduces a predicted quantity to an input: Eq. (1) and Eq. (2) are statistical models fit to observed outcome data, not derivations of the outcome from the prompt design. The only self-reference is the authors' own replication package [16], which is cited for data and materials availability and is not load-bearing for any scientific claim. The paper does contain a clear statistical validity concern in Section 3.6: Eq. (2) describes a negative-binomial GLMM for binary correctness outcomes, which is a count-distribution model and therefore a questionable choice for pass/fail data; this is a correctness risk in the inference, not a circular dependency. Because the outcomes are externally benchmarked and the estimates are not constructed to equal the inputs, the circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claims rest on the distributional-cue interpretation of LLM prompting, the fidelity of the prompt operationalization, the validity of static metrics as proxies, and the correctness of the statistical models. No free parameters were fitted to produce the result, and no new entities are introduced.

assumptions (4)
  • domain assumption Surface-level lexical and pragmatic framing can shift LLM outputs through statistical associations in training data.
    Section 2.3.2 grounds all hypotheses in this distributional-cue assumption and explicitly disclaims that LLMs understand persuasion.
  • domain assumption The operationalized prompt templates faithfully capture the intended IBQ-G influence tactics while controlling for tone.
    Section 3.2 describes an iterative three-round author review; construct validity depends on this mapping.
  • domain assumption Static analysis metrics (CC, MI, PyLint, Bandit, SLOC, comments) provide meaningful relative signals of code quality and security across prompt conditions.
    Section 3.4 presents these as proxies; Section 3.8 acknowledges differences from professional judgment.
  • domain assumption The statistical inference models in Section 3.6 are correctly specified for the outcomes they analyze.
    This is the load-bearing premise for all reported p-values; the negative binomial specification for binary correctness is questionable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Do Influence Tactics Matter? Investigating Prompt Framing Effects in LLM Code Generation." pith.science (2026). https://pith.science/paper/GG5BQZCQ

@misc{pith2026260811513,
  author       = {Pith},
  title        = {Pith review of: Do Influence Tactics Matter? Investigating Prompt Framing Effects in LLM Code Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GG5BQZCQ}},
  note         = {Machine review of arXiv:2608.11513}
}
read the original abstract

Large Language Models (LLMs) are increasingly integrated into software engineering workflows, helping developers write, debug, test, and maintain code. While prompt wording and structure are known to influence model performance, the impact of psychologically inspired prompt framings remains unexplored. This study investigates whether different psychology-based communication strategies that humans use to persuade or motivate others can lead to more effective prompt framing, which may, in turn, affect LLM behaviour in coding tasks. Drawing on Yukl & Falbe's well-known taxonomy, we operationalized eight influence tactics (like rational persuasion, ingratiation, and exchange) into reproducible prompt templates. These prompt templates were evaluated across five leading open-weight LLMs using two widely adopted benchmarks: LiveCodeBench and SWE-bench Verified. We assessed the resulting code output on four key software quality dimensions: functional correctness, quality, maintainability, and security. Our results show that certain influence-induced prompt framings, particularly those emphasizing urgency, were associated with reduced correctness and security. This work presents the first large-scale empirical study of influence-induced prompt framing in software engineering tasks, offering insights into how linguistic cues may shape LLM outputs. We conclude with practical insights for designing transparent and interpretable human-AI interactions in code generation.

Figures

Figures reproduced from arXiv: 2608.11513 by the authors.

Figure 1
Figure 1. Overview of the Influence Tactic Study Design [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Structure of prompt used for evaluating the influence of psychological tactics [PITH_FULL_IMAGE:figures/full_fig_p017_2.png] view at source ↗
Figure 3
Figure 3. Distribution of selected qualitative features across psychologically influenced [PITH_FULL_IMAGE:figures/full_fig_p028_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

82 extracted references · 41 canonical work pages

  1. [1]

    https://ai.meta.com/blog/llama-4-multimodal-intelligence/(2025)

    AI, M.: The llama 4 herd: The beginning of a new era of natively multimodal ai innovation. https://ai.meta.com/blog/llama-4-multimodal-intelligence/(2025). Accessed: 2025-07- 17

    2025

  2. [2]

    Empirical Software Engineering22(6), 3057–3087 (2017)

    Antinyan, V., Staron, M., Sandberg, A.: Evaluating code complexity triggers, use of com- plexity measures and the influence of code complexity on maintenance time. Empirical Software Engineering22(6), 3057–3087 (2017)

    2017

  3. [3]

    Information systems research12(2), 195–207 (2001)

    Austin, R.D.: The effects of time pressure on quality in software development: An agency model. Information systems research12(2), 195–207 (2001)

    work page 2001

  4. [4]

    arXiv preprint arXiv:2508.15503 (2025)

    Baltes, S., Angermeir, F., Arora, C., Barón, M.M., Chen, C., Böhme, L., Calefato, F., Ernst, N., Falessi, D., Fitzgerald, B., et al.: Guidelines for empirical studies in software engineering involving large language models. arXiv preprint arXiv:2508.15503 (2025)

    arXiv 2025

  5. [5]

    arXiv preprint arXiv:2408.16601 (2024)

    Beer, R., Feix, A., Guttzeit, T., Muras, T., Müller, V., Rauscher, M., Schäffler, F., Löwe, W.: Examination of code generated by large language models. arXiv preprint arXiv:2408.16601 (2024)

    arXiv 2024

  6. [6]

    Empirical Software Engineering28(6), 128 (2023)

    Börstler, J., Bennin, K.E., Hooshangi, S., Jeuring, J., Keuning, H., Kleiner, C., MacKellar, B., Duran, R., Störrle, H., Toll, D., et al.: Developers talking about code quality. Empirical Software Engineering28(6), 128 (2023)

    work page 2023

  7. [7]

    Advances in neural information processing systems33, 1877–1901 (2020)

    Brown,T.,Mann,B.,Ryder,N.,Subbiah,M.,Kaplan,J.D.,Dhariwal,P.,Neelakantan,A., Shyam, P., Sastry, G., Askell, A., et al.: Language models are few-shot learners. Advances in neural information processing systems33, 1877–1901 (2020)

    work page 2020

  8. [8]

    arXiv preprint arXiv:2312.16171 (2023)

    Bsharat, S.M., Myrzakhan, A., Shen, Z.: Principled instructions are all you need for ques- tioning llama-1/2, gpt-3.5/4. arXiv preprint arXiv:2312.16171 (2023)

    arXiv 2023

Show all 82 references
  1. [9]

    arXiv preprint arXiv:2107.03374 (2021)

    Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H.P.D.O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al.: Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021)

    2021 arXiv

  2. [10]

    In: 2025 IEEE International Confer- ence on Software Analysis, Evolution and Reengineering (SANER), pp

    Chen, Z., Jiang, L.: Evaluating software development agents: Patch patterns, code quality, and issue complexity in real-world github scenarios. In: 2025 IEEE International Confer- ence on Software Analysis, Evolution and Reengineering (SANER), pp. 657–668. IEEE (2025)

    2025

  3. [11]

    URLhttps://openai.com/index/ introducing-swe-bench-verified/

    Chowdhury, N., Aung, J., Shern, C.J., Jaffe, O., Sherburn, D., Starace, G., Mays, E., Dias, R., Aljubeh, M., Glaese, M., Jimenez, C.E., Yang, J., Ho, L., Patwardhan, T., Liu, K., Madry, A.: Introducing SWE-bench verified (2024). URLhttps://openai.com/index/ introducing-swe-ben...

    2024

  4. [12]

    Chowdhury, S., Holmes, R., Zaidman, A., Kazman, R.: Revisiting the debate: Are code metrics useful for measuring maintenance effort? Empirical Software Engineering27(6), 158 (2022)

    2022

  5. [13]

    In: European Conference on Cyber Warfare and Security, pp

    Cohen, E., Heverin, T.: Evaluating the effectiveness of psychological prompt injection at- tacks on large language models for social engineering artifact generation. In: European Conference on Cyber Warfare and Security, pp. 879–883. Academic Conferences Interna- tional Limited (2025)

    2025

  6. [14]

    URLhttps://github.com/pylint-dev/pylint

    contributors, P.: Pylint (2025). URLhttps://github.com/pylint-dev/pylint. Online; ac- cessed 10 Jul 2025 1 https://osf.io/uxhde/overview?view_only=d507800dd6a6434a8c18f8f4607713ea 34 Alex Deaconu * et al

    2025

  7. [15]

    Sage publications (2014)

    Corbin, J., Strauss, A.: Basics of qualitative research: Techniques and procedures for de- veloping grounded theory. Sage publications (2014)

    2014

  8. [16]

    Replica- tion Package

    Deaconu, A., Gupta, A., Basha, M., Haydu, N., Rodríguez-Pérez, G.: Do influence tactics matter? investigating prompt framing effects in llm code generation.https: //osf.io/uxhde/overview?view_only=d507800dd6a6434a8c18f8f4607713ea(2025). Replica- tion Package

    2025

  9. [17]

    arXiv preprint arXiv:2504.16833 (2025)

    Deng, S., Huang, R., Zhang, M., Cui, C., Towey, D., Wang, R.: Lrasgen: Llm-based restful api specification generation. arXiv preprint arXiv:2504.16833 (2025)

    2025 arXiv

  10. [18]

    arXiv e-prints pp

    Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al.: The llama 3 herd of models. arXiv e-prints pp. arXiv–2407 (2024)

    2024

  11. [19]

    In: 2024 34th International Conference on Collaborative Advances in Software and COmputiNg (CASCON), pp

    Esfahani, A.M., Kahani, N., Ajila, S.A.: Understanding defects in generated codes by language models. In: 2024 34th International Conference on Collaborative Advances in Software and COmputiNg (CASCON), pp. 1–10. IEEE (2024)

    2024

  12. [20]

    In: Proceedings of the 32nd IEEE/ACM International Conference on Program Comprehension, pp

    Fagadau, I.D., Mariani, L., Micucci, D., Riganelli, O.: Analyzing prompt influence on automated method generation: An empirical study with copilot. In: Proceedings of the 32nd IEEE/ACM International Conference on Program Comprehension, pp. 24–34 (2024)

    2024

  13. [21]

    Psychological Science in the Public Interest25(3), 105–161 (2024)

    Flusberg, S.J., Holmes, K.J., Thibodeau, P.H., Nabi, R.L., Matlock, T.: The psychology of framing: How everyday language shapes the way we think, feel, and act. Psychological Science in the Public Interest25(3), 105–161 (2024)

    2024

  14. [22]

    arXiv preprint arXiv:2503.13510 (2025)

    Gandhi, V., Gandhi, S.: Prompt sentiment: The catalyst for llm change. arXiv preprint arXiv:2503.13510 (2025)

    2025 arXiv

  15. [23]

    arXiv preprint arXiv:2407.21783 (2024)

    Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al.: The llama 3 herd of models. arXiv preprint arXiv:2407.21783 (2024)

    2024 arXiv

  16. [24]

    Ac- cessed: 2025-07-18

    Groq: Groq console documentation.https://console.groq.com/docs/overview(2025). Ac- cessed: 2025-07-18

    2025

  17. [25]

    arXiv preprint arXiv:2501.12948 (2025)

    Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., et al.: Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 (2025)

    2025 arXiv

  18. [26]

    arXiv preprint arXiv:2312.13010 (2023)

    Huang, D., Zhang, J.M., Luck, M., Bu, Q., Qing, Y., Cui, H.: Agentcoder: Multi- agent-based code generation with iterative testing and optimisation. arXiv preprint arXiv:2312.13010 (2023)

    2023 arXiv

  19. [27]

    In: Proceedings of the 2018 International Conference on technical debt, pp

    Izurieta, C., Rice, D., Kimball, K., Valentien, T.: A position study to investigate techni- cal debt associated with security weaknesses. In: Proceedings of the 2018 International Conference on technical debt, pp. 138–142 (2018)

    2018

  20. [28]

    arXiv preprint arXiv:2403.07974 (2024)

    Jain, N., Han, K., Gu, A., Li, W.D., Yan, F., Zhang, T., Wang, S., Solar-Lezama, A., Sen, K., Stoica, I.: Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974 (2024)

    2024 arXiv

  21. [29]

    Jimenez,C.E.,Yang,J.,Wettig,A.,Yao,S.,Pei,K.,Press,O.,Narasimhan,K.:Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:2310.06770 (2023)

    2023 arXiv

  22. [30]

    In: Proceedings of the 2014 ACM Workshop on Security Information Workers, pp

    Jordan, T.B., Johnson, B., Witschey, J., Murphy-Hill, E.: Designing interventions to per- suade software developers to adopt security tools. In: Proceedings of the 2014 ACM Workshop on Security Information Workers, pp. 35–38 (2014)

    2014

  23. [31]

    arXiv preprint arXiv:2210.02406 (2022)

    Khot, T., Trivedi, H., Finlayson, M., Fu, Y., Richardson, K., Clark, P., Sabharwal, A.: Decomposed prompting: A modular approach for solving complex tasks. arXiv preprint arXiv:2210.02406 (2022)

    2022 arXiv

  24. [32]

    Journal of Applied Psychology65(4), 440–452 (1980)

    Kipnis, D., Schmidt, S.M., Wilkinson, I.: Intraorganizational influence tactics: Explo- rations in getting one’s way. Journal of Applied Psychology65(4), 440–452 (1980). DOI 10.1037/0021-9010.65.4.440. URLhttps://search.ebscohost.com/login.aspx? direct=true&AuthType=shib&db=pd...

    1980 doi

  25. [33]

    Information and Software Technology121, 106257 (2020)

    Kuutila, M., Mäntylä, M., Farooq, U., Claes, M.: Time pressure in software engineering: A systematic review. Information and Software Technology121, 106257 (2020)

    2020

  26. [34]

    URLhttps://github.com/rubik/radon

    Lacchia, M.: Radon (2025). URLhttps://github.com/rubik/radon. Online; accessed 10 Jul 2025

    2025

  27. [35]

    The Leadership Quarterly28(1), 210–228 (2017)

    Lee, S., Han, S., Cheong, M., Kim, S.L., Yun, S.: How do i get my way? a meta-analytic review of research on influence tactics. The Leadership Quarterly28(1), 210–228 (2017)

    2017

  28. [36]

    arXiv preprint arXiv:2307.11760 (2023) Do Influence Tactics Matter? 35

    Li, C., Wang, J., Zhang, Y., Zhu, K., Hou, W., Lian, J., Luo, F., Yang, Q., Xie, X.: Large language models understand and can be enhanced by emotional stimuli. arXiv preprint arXiv:2307.11760 (2023) Do Influence Tactics Matter? 35

    2023 arXiv

  29. [37]

    arXiv preprint arXiv:2305.08360 (2023)

    Liu, C., Bao, X., Zhang, H., Zhang, N., Hu, H., Zhang, X., Yan, M.: Improving chatgpt prompt for code generation. arXiv preprint arXiv:2305.08360 (2023)

    2023 arXiv

  30. [38]

    arXiv preprint arXiv:2302.09185 (2023)

    Lu, A., Zhang, H., Zhang, Y., Wang, X., Yang, D.: Bounding the capabilities of large language models in open text generation with prompt constraints. arXiv preprint arXiv:2302.09185 (2023)

    2023 arXiv

  31. [39]

    IEEE Transactions on software EngineeringSE- 2(4), 308–320 (1976)

    McCabe, T.J.: A complexity measure. IEEE Transactions on software EngineeringSE- 2(4), 308–320 (1976)

    1976

  32. [40]

    Organization Management Journal17(1), 43–58 (2020)

    McCarthy, K., Pearce, J.L., Morton, J., Lyon, S.: Do you pass it on? an examination of the consequences of perceived cyber incivility. Organization Management Journal17(1), 43–58 (2020)

    2020

  33. [41]

    arXiv preprint arXiv:2406.12513 (2024)

    Mohsin, A., Janicke, H., Wood, A., Sarker, I.H., Maglaras, L., Janjua, N.: Can we trust large language models generated code? a framework for in-context learning, security pat- terns, and code evaluations across diverse llms. arXiv preprint arXiv:2406.12513 (2024)

    2024 arXiv

  34. [42]

    arXiv preprint arXiv:2505.23836 (2025)

    Needham, J., Edkins, G., Pimpale, G., Bartsch, H., Hobbhahn, M.: Large language models often know when they are being evaluated. arXiv preprint arXiv:2505.23836 (2025)

    2025 arXiv

  35. [43]

    In: Proceedings Conference on Software Maintenance 1992, pp

    Oman, P., Hagemeister, J.: Metrics for assessing a software system’s maintainability. In: Proceedings Conference on Software Maintenance 1992, pp. 337–338. IEEE Computer Society (1992)

    1992

  36. [44]

    In: 2018 IEEE International conference on software maintenance and evolution (ICSME), pp

    Pantiuchina, J., Lanza, M., Bavota, G.: Improving code: The (mis) perception of quality metrics. In: 2018 IEEE International conference on software maintenance and evolution (ICSME), pp. 80–91. IEEE (2018)

    2018

  37. [45]

    In:Proceedingsofthe2025CHIConference on Human Factors in Computing Systems, pp

    Pickering, M., Williams, H., Gan, A., He, W., Park, H., Piedrahita Velez, F., Littman, M.L., Ur, B.: How humans communicate programming tasks in natural language and im- plicationsforend-userprogrammingwithllms. In:Proceedingsofthe2025CHIConference on Human Factors in Computin...

    2025

  38. [46]

    gradient descent

    Pryzant, R., Iter, D., Li, J., Lee, Y.T., Zhu, C., Zeng, M.: Automatic prompt optimization with" gradient descent" and beam search. arXiv preprint arXiv:2305.03495 (2023)

    2023 arXiv

  39. [47]

    URLhttps://github.com/PyCQA/bandit

    PyCQA: Bandit (2025). URLhttps://github.com/PyCQA/bandit. Online; accessed 10 Jul 2025

    2025

  40. [48]

    In: Proceedings of the 2023 15th International Conference on Machine Learning and Computing, ICMLC ’23, p

    Qian, J., Chen, Q., Yue, Y., Atkinson, K., Li, G.: Injecting commonsense knowledge into prompt learning for zero-shot text classification. In: Proceedings of the 2023 15th International Conference on Machine Learning and Computing, ICMLC ’23, p. 427–432. Association for Comput...

    2023

  41. [49]

    Interactive Learning Environments33(2), 1020–1039 (2025)

    Quan, Z., Chen, Z.: Human–computer pragmatics trialled: some (im) polite interactions with chatgpt 4.0 and the ensuing implications. Interactive Learning Environments33(2), 1020–1039 (2025)

    2025

  42. [50]

    arXiv preprint arXiv:2501.16998 (2025)

    Rasheed, Z., Waseem, M., Kemell, K.K., Ahmad, A., Sami, M.A., Rasku, J., Systä, K., Abrahamsson, P.: Large language models for code generation: The practitioners perspec- tive. arXiv preprint arXiv:2501.16998 (2025)

    2025 arXiv

  43. [51]

    International journal of environmental research and public health18(5), 2294 (2021)

    Rasool, S.F., Wang, M., Tang, M., Saeed, A., Iqbal, J.: How toxic workplace environment effects the employee engagement: The mediating role of organizational support and em- ployee wellbeing. International journal of environmental research and public health18(5), 2294 (2021)

    2021

  44. [52]

    arXiv preprint arXiv:2403.14380 (2024)

    Salvi, F., Ribeiro, M.H., Gallotti, R., West, R.: On the conversational persuasiveness of large language models: A randomized controlled trial. arXiv preprint arXiv:2403.14380 (2024)

    2024 arXiv

  45. [53]

    URL: https://www

    Schmidt, D.C., Spencer-Smith, J., Fu, Q., White, J.: Cataloging prompt patterns to enhance the discipline of prompt engineering. URL: https://www. dre. vanderbilt. edu/schmidt/PDF/ADA_Europe_Position_Paper. pdf [accessed 2023-09-25] (2023)

    2023

  46. [54]

    arXiv preprint arXiv:2109.05093 (2021)

    Scholak, T., Schucher, N., Bahdanau, D.: Picard: Parsing incrementally for constrained auto-regressive decoding from language models. arXiv preprint arXiv:2109.05093 (2021)

    2021 arXiv

  47. [55]

    arXiv preprint arXiv:2406.066085(2024)

    Schulhoff, S., Ilie, M., Balepur, N., Kahadze, K., Liu, A., Si, C., Li, Y., Gupta, A., Han, H., Schulhoff, S., et al.: The prompt report: A systematic survey of prompting techniques. arXiv preprint arXiv:2406.066085(2024)

    2024 arXiv

  48. [56]

    arXiv preprint arXiv:2310.10508 (2023)

    Shin, J., Tang, C., Mohati, T., Nayebi, M., Wang, S., Hemmati, H.: Prompt engineering or fine tuning: An empirical assessment of large language models in automated software engineering tasks. arXiv preprint arXiv:2310.10508 (2023)

    2023 arXiv

  49. [57]

    Advances in Neural Information Processing Systems 36, 8634–8652 (2023) 36 Alex Deaconu * et al

    Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K., Yao, S.: Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems 36, 8634–8652 (2023) 36 Alex Deaconu * et al

    2023

  50. [58]

    10th, vol

    Sommerville, I.: Software engineering. 10th, vol. 10. Addison-Wesley (2015)

    2015

  51. [59]

    Sage Publications, Inc., Thousand Oaks, CA (1998)

    Strauss, A., Corbin, J.: Basics of qualitative research techniques. Sage Publications, Inc., Thousand Oaks, CA (1998). Print ISBN 9780803959392, eBook ISBN 9780585383323

    1998

  52. [60]

    URL https://arxiv.org/abs/2408.11198

    Taherkhani, H., Sepindband, M., Pham, H.V., Wang, S., Hemmati, H.: Automated prompt engineering for cost-effective code generation using evolutionary algorithm (2025). URL https://arxiv.org/abs/2408.11198

    2025 arXiv

  53. [61]

    Journal of Systems and Software86(6), 1498–1516 (2013)

    Tom, E., Aurum, A., Vidgen, R.: An exploration of technical debt. Journal of Systems and Software86(6), 1498–1516 (2013)

    2013

  54. [62]

    arXiv preprint arXiv:2407.07064 (2024)

    Tony, C., Ferreyra, N.E.D., Mutas, M., Dhiff, S., Scandariato, R.: Prompting techniques for secure code generation: A systematic investigation. arXiv preprint arXiv:2407.07064 (2024)

    2024 arXiv

  55. [63]

    arXiv preprint arXiv:2203.11171 (2022)

    Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., Zhou, D.: Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171 (2022)

    2022 arXiv

  56. [64]

    URLhttps://arxiv.org/abs/2603.19282

    Wang, Z., Zhang, Z.: Framing effects in independent-agent large language models: A cross- family behavioral analysis (2026). URLhttps://arxiv.org/abs/2603.19282

    2026 arXiv

  57. [65]

    Advances in neural information processing systems35, 24824–24837 (2022)

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q.V., Zhou, D., et al.: Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems35, 24824–24837 (2022)

    2022

  58. [66]

    arXiv preprint arXiv:2302.11382 (2023)

    White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer- Smith, J., Schmidt, D.C.: A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv preprint arXiv:2302.11382 (2023)

    2023 arXiv

  59. [67]

    arXiv preprint arXiv:2505.09388 (2025)

    Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al.: Qwen3 technical report. arXiv preprint arXiv:2505.09388 (2025)

    2025 arXiv

  60. [68]

    In: Proceedings of the Second Workshop on Social Influence in Conversations (SICon 2024), pp

    Yin, Z., Wang, H., Horio, K., Kawahara, D., Sekine, S.: Should we respect llms? a cross- lingual study on the influence of prompt politeness on llm performance. In: Proceedings of the Second Workshop on Social Influence in Conversations (SICon 2024), pp. 9–35 (2024)

    2024

  61. [69]

    Journal of Organizational Behavior: The International Journal of Industrial, Occupational and Organizational Psychology and Behavior26(6), 705–725 (2005)

    Yukl, G., Chavez, C., Seifert, C.F.: Assessing the construct validity and utility of two new influence tactics. Journal of Organizational Behavior: The International Journal of Industrial, Occupational and Organizational Psychology and Behavior26(6), 705–725 (2005)

    2005

  62. [70]

    Journal of applied psychology75(2), 132 (1990)

    Yukl, G., Falbe, C.M.: Influence tactics and objectives in upward, downward, and lateral influence attempts. Journal of applied psychology75(2), 132 (1990)

    1990

  63. [71]

    In: Toronto, paper presented at the Society for Industrial and Organizational Psychology annual conference (2002)

    Yukl, G., Seifert, C.F.: Preliminary validation research on the extended version of the influence behavior questionnaire. In: Toronto, paper presented at the Society for Industrial and Organizational Psychology annual conference (2002)

    2002

  64. [72]

    The Leadership Quarterly19(5), 609–621 (2008)

    Yukl, G., Seifert, C.F., Chavez, C.: Validation of the extended influence behavior ques- tionnaire. The Leadership Quarterly19(5), 609–621 (2008)

    2008

  65. [73]

    Computer58(07), 122–128 (2025)

    Zhang, L., Singhal, A., Zou, Q., Sun, X., Liu, P., Lin, H.Y.: Can ai fix buggy code? exploringtheuseoflargelanguagemodelsinautomatedprogramrepair. Computer58(07), 122–128 (2025)

    2025

  66. [74]

    arXiv preprint arXiv:2506.15227 (2025)

    Zhang, Q., Fang, C., Gu, S., Shang, Y., Chen, Z., Xiao, L.: Large language models for unit testing: A systematic literature review. arXiv preprint arXiv:2506.15227 (2025)

    2025 arXiv

  67. [75]

    arXiv preprint arXiv:2210.03493 (2022)

    Zhang, Z., Zhang, A., Li, M., Smola, A.: Automatic chain of thought prompting in large language models. arXiv preprint arXiv:2210.03493 (2022)

    2022 arXiv

  68. [76]

    a helpful assistant

    Zheng, M., Pei, J., Jurgens, D.: Is" a helpful assistant" the best role for large lan- guage models? a systematic evaluation of social roles in system prompts. arXiv preprint arXiv:2311.100548(2023)

    2023 arXiv

  69. [77]

    Proceedings of the AAAI Conference on Artificial Intelligence38, 21841–21849 (2024)

    Zhong, L., Wang, Z.: Can llm replace stack overflow? a study on robustness and reliability of large language model code generation. Proceedings of the AAAI Conference on Artificial Intelligence38, 21841–21849 (2024). DOI 10.1609/aaai.v38i19.30185. URLhttps://ojs. aaai.org/inde...

    2024 doi

  70. [78]

    arXiv preprint arXiv:2402.16906 (2024)

    Zhong, L., Wang, Z., Shang, J.: Debug like a human: A large language model debugger via verifying runtime execution step-by-step. arXiv preprint arXiv:2402.16906 (2024)

    2024 arXiv

  71. [79]

    URLhttps://arxiv.org/abs/2509.10402

    Zhong, S., Zou, Y., Adams, B.: Developer-llm conversations: An empirical study of inter- actions and generated code quality (2025). URLhttps://arxiv.org/abs/2509.10402

    2025 arXiv

  72. [80]

    arXiv preprint arXiv:2310.04406 (2023)

    Zhou, A., Yan, K., Shlapentokh-Rothman, M., Wang, H., Wang, Y.X.: Language agent tree search unifies reasoning acting and planning in language models. arXiv preprint arXiv:2310.04406 (2023)

    2023 arXiv

  73. [81]

    URLhttps://arxiv.org/abs/2211.01910

    Zhou, Y., Muresanu, A.I., Han, Z., Paster, K., Pitis, S., Chan, H., Ba, J.: Large language models are human-level prompt engineers (2023). URLhttps://arxiv.org/abs/2211.01910

    2023 arXiv

  74. [82]

    Zhuo, J., Zhang, S., Fang, X., Duan, H., Lin, D., Chen, K.: Prosa: Assessing and under- standing the prompt sensitivity of llms (2024). URLhttps://arxiv.org/abs/2410.12405 Do Influence Tactics Matter? 37 Authors and Affiliations AlexDeaconu.AnubhavGupta.ManaalBasha.NicholasHay...

    2024 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.