PACT: A Contract-Theoretic Framework for Pricing Agentic AI Services Powered by Large Language Models

Quanyan Zhu; Ya-Ting Yang

REVIEW 3 major objections 6 minor 17 references

PACT: A Contract-Theoretic Framework for Pricing Agentic AI Services Powered by Large Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A menu of QoS-and-price contracts can price LLM agentic services so users self-select the tier that fits their task.

desk verdict Competent application of textbook contract theory to LLM agent pricing, with a useful QoS framing but a numerical section that is internally inconsistent and unsupported by the stated model. read the letter →

arxiv 2505.21286 v1 pith:UJHJY5W4 submitted 2025-05-27 cs.GT

Ya-Ting Yang , Quanyan Zhu This is my paper

classification cs.GT MSC 91B0391B26

keywords contracttheoryagenticAIpricingLLMservicequalityofincentivecompatibilityindividualrationalityliabilitycostinformationasymmetry

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes PACT, a pricing scheme for cloud-based agentic AI services—LLM-powered agents that reason, decide, and act on a user's behalf. Its central claim is that a service provider can publish a menu of quality-and-price contracts, one per service tier, and set prices so that each user voluntarily selects the tier that matches how much their task depends on quality, even though the provider cannot observe the user's private willingness to pay. QoS is modeled as a weighted blend of response time and the estimated probability that the agent's output satisfies the user, and the provider's cost includes computation, infrastructure, and liability. The paper proves the menu satisfies incentive compatibility and individual rationality, and its numerical experiments on a cybersecurity log-analysis task show users choosing their intended tiers while liability costs shift prices and quality upward. It also warns that setting liability costs too high can turn the provider's expected profit negative.

What carries the argument

The load-bearing object is the contract menu $\{(q_k,p_k)\}$, with QoS index $q_k = \delta A_k + (1-\delta)(1-T_k)$; this collapses multi-dimensional latency and satisfaction into a single ordered quality level, which is what makes contract theory applicable. The user's private type $\theta_k$ enters through the utility $\theta_k v(q_k)-p_k$, and the feasibility proof reduces the $K$ individual-rationality constraints and $K(K-1)$ incentive-compatibility constraints to the lowest-type IR condition plus local downward IC constraints, using monotonicity of $q_k$ in $\theta_k$. The cost side $C(q_k)=c_{\rm tok}(q_k)+c_h(q_k)+c_m(q_k)+c_l(q_k)$ injects LLM computation, hardware, and liability into the same one-dimensional scale, so prices reflect both resource usage and risk.

What would settle it

Collect user-satisfaction data from an actual agentic log-analysis service and compute $q_k=\delta A_k+(1-\delta)(1-T_k)$ for the eight configurations in the paper's Table I; if the resulting quality levels are not strictly increasing in the assumed type order, or if any $T_k$ cannot be normalized into $[0,1]$, the monotonicity condition behind Proposition 1 fails and the computed menu would not be incentive compatible for real users.

Watch

Extended reading notes

Core claim

The paper's central claim is that the pricing problem for LLM-agent services reduces to a contract-design problem over a one-dimensional QoS index $q_k=\delta A_k+(1-\delta)(1-T_k)$, where $A_k$ is the probability that the output satisfies the user's expected response and $T_k$ is total latency normalized to $[0,1]$. With user types $\theta_1<\cdots<\theta_K$ and utility $\theta_k v(q_k)-p_k$ for a concave valuation $v$, a feasible menu $\{(q_k,p_k)\}$ exists exactly when QoS levels are monotone in type, the lowest type's utility is nonnegative, and adjacent local downward incentive constraints hold; the provider then maximizes expected profit $\sum_k P(\theta_k)(p_k-C(q_k))$ over such menus. The paper demonstrates numerically, for a log-analysis task with eight service configurations and fifteen user types, that the optimal menu makes each type prefer its own contract, and that inserting liability costs into $C(q)$ raises both optimal QoS and prices. Under asymmetric information with liability costs, the provider's expected utility flips negative, which the paper reads as a caution for regulators setting liability levels.

Load-bearing premise

The contract menu is only as grounded as the number $q_k$: the provider must be able to estimate a trustworthy satisfaction probability $A_k$ and normalize latency $T_k$ into $[0,1]$, because in the numerical study $A_k$ is assumed to rise with model and data size in the absence of real user data.

Editorial extensions

If this is right

A provider can post a finite menu and let users self-select, because the incentive-compatibility constraints guarantee each user type finds its own tier optimal.
Adjusting the weight $\delta$ in the QoS index tilts pricing toward satisfaction-heavy or latency-sensitive tasks, so the same contract skeleton works across different domains.
Including liability costs in the provider's cost shifts optimal QoS and prices upward, giving providers a financial reason to offer higher-assurance service tiers.
If liability costs are set too high, the provider's expected profit under information asymmetry becomes negative, so a regulator must balance liability against service viability.
The constraint reduction keeps the optimization problem small (one IR constraint and $K-1$ IC constraints), so the pricing menu can be recomputed as QoS measurements change.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An extension the paper does not pursue is to estimate $A_k$ from production logs, such as query reformulation and downstream task completion, which would make the menu adaptive to real satisfaction data rather than assumed monotonic values.
Because $q_k$ is a single index, the framework could absorb extra dimensions such as explainability or auditability as additional normalized terms, but the paper does not model how those dimensions interact with liability.
The type distribution $P(\theta_k)$ is taken as known; coupling the menu with an online estimator of user types from behavioral signals would let the provider update prices as the user base evolves.
A testable refinement is to make the liability cost $c_l(q_k)$ depend on task risk and output confidence rather than model family, which would change the optimal tier structure in regulated domains.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this paper applies standard contract theory to pricing cloud-based agentic LLM services, modeling QoS as a weighted sum of user satisfaction and latency, and adding a liability cost term. The theoretical machinery is textbook and correct; the numerical validation is not, and the headline empirical claim that PACT "aligns QoS with user preferences" is not supported by the simulation as it stands.

What's genuinely new is the specific QoS index (δ·A + (1−δ)(1−T)) and the inclusion of liability costs in the provider's cost function. The latency decomposition into transmission, tokenization, and inference is a sensible first-order model, and the authors are transparent that satisfaction data are unavailable and liability costs are speculative. That honesty counts for something.

The soft spot is Section IV. The QoS values in Table I do not follow from the model. For row 8, using the published parameters (Gemma 7B, γ=31200 GFLOPS, r=20 Mbps, δ=0.5, A=0.9), Eq. (3) gives T_inf ≈ 0.359 s, so T_total ≈ 0.539 s and q = 0.5·0.9 + 0.5·(1−0.539) ≈ 0.68. The table lists q = 0.848, which is close to the value obtained by dropping the inference time entirely (T ≈ 0.18 s, q ≈ 0.86). Similar mismatches appear in other rows. The authors then fit C(q) to these q_k values and feed that fitted curve into the optimization, so every price and utility figure in Figs. 2–4 inherits the inconsistency. On top of that, A_k is assumed, the fitted C(q) is never reported, and no code or data are provided. The "verification" that users choose their assigned contract is circular — it is just checking the IR/IC constraints that the optimization already imposes.

A minor but real issue: Eq. (6) mixes A_k on [0,1] with raw latency in seconds without normalization, making the QoS index dimensionally inconsistent.

That said, the core theory is fine. Lemmas 1–4 are standard contract-theory results, correctly stated and proved. The application to LLM agent pricing is new and could be useful once the empirical side is redone with real or clearly labeled synthetic data, a reported cost fit, and model-consistent latency accounting.

This paper is for readers working on pricing mechanisms for AI services or contract theory applications. It is a building block, not a breakthrough.

Recommendation: send to peer review, but as a major-revision candidate. The theory deserves refereeing; the simulation needs to be made reproducible and consistent with the stated model.

Referee Report

3 major / 6 minor

Summary. The paper proposes PACT, a contract-theoretic framework for pricing cloud-based agentic AI services. QoS is modeled as a convex combination of user satisfaction and latency (Eq. 6), the provider's cost includes computational, infrastructure, and liability components, and the provider designs a menu of QoS-payment contracts under information asymmetry subject to individual rationality and incentive compatibility constraints. The theoretical part follows the standard contract-theory reduction of IR/IC constraints, and the numerical section applies the framework to a cybersecurity log-analysis task with a small set of LLM service configurations. The paper claims that the numerical results demonstrate QoS alignment with user preferences and show how liability costs affect prices, provider utility, and social welfare.

Significance. If the numerical evaluation were sound, PACT would be a useful application of standard contract theory to the timely problem of pricing LLM-based agentic services; the composite QoS model and the liability-cost extension are reasonable modeling choices, and the theoretical development is clearly presented. The paper identifies a real gap in the pricing literature and the contract-design machinery is appropriate. However, the numerical demonstration, which is the main evidence for the headline claim, is not reproducible and is internally inconsistent: key QoS values do not follow from the stated equations, the fitted cost curve is not reported, and several input quantities are assumed without data. The paper ships no code or data, so the current contribution is a modeling proposal plus a standard theoretical derivation rather than a validated pricing framework.

major comments (3)

[Section IV, Table I and Eq. (6)] The QoS values in Table I are inconsistent with the stated model. Using the parameters given in Section IV for row 8 (D_in=D_out=100 KB, β=7, γ=31200 GFLOPS, r=20 Mbps, 0.5 ms/KB tokenization, g_t=g_dt=4x, n_layer=28, n_ctx=8192, n_attn=16), Eqs. (1)-(4) give T_tran=0.08 s, T_tok=0.1 s, and T_inf≈0.359 s, so T≈0.539 s. With δ=0.5 and A_8=0.9, Eq. (6) yields q_8≈0.68, not the tabulated 0.848; the tabulated value is close to the result obtained when the inference time is dropped. Similar discrepancies affect other rows. Because the cost curve C(q) is fit from these q_k values and then used as an input to optimization (16), all prices and utilities in Figs. 2-4 inherit this inconsistency. In addition, Eq. (6) subtracts T_k in seconds from 1 while A_k∈[0,1]; no normalization of T_k to [0,1] is specified, so the QoS expression is dimensionally inconsistent as written.
[Section IV, cost fitting and K=15 setup] The numerical study is not reproducible: after computing q_k, the authors state that they 'fit the relationship curve between q and C(q)' from the eight configurations, but they do not report the fitted functional form, the cost values C(q_k), or the monetary parameter c in c_tok(q_k). Since C(q) is an input to the optimization in (16), every contract price depends on an unspecified fit. Furthermore, Table I lists only 8 service configurations while the simulations use K=15 user types; the paper does not explain how the 8 QoS levels are expanded or interpolated to the 15 types needed for the contract menu. The authors should provide the complete numerical specification, including the fitted cost function, or replace this part of the evaluation with a clearly stated synthetic procedure.
[Section III, Lemma 1 and Proposition 1] The proof of strict monotonicity of QoS in user type is not valid as written. Adding the two IC constraints for types θ_k and θ_j yields (θ_k−θ_j)(v(q_k)−v(q_j)) ≥ 0, which implies v(q_k) ≥ v(q_j) and hence q_k ≥ q_j, not the strict q_k > q_j claimed in Lemma 1. Equality of q and p is not ruled out, so duplicate contracts (bunching) are consistent with IC and IR. The strict monotonicity in Proposition 1 is then used in Lemma 4 to reduce the IC constraints, so the reduction may exclude valid pooling solutions. Please add an explicit separation assumption or revise the constraint-reduction proof to handle non-strict monotonicity.

minor comments (6)

[Section III, Eq. (14) and Lemma 3] Equation (14) writes the reduced IR constraint as θ_1 u_1(q_1,p_1) ≥ 0, but the IR constraint is u_1(q_1,p_1) ≥ 0; the factor θ_1 should be removed.
[Section III, Lemma 4 proof] In the proof of Lemma 4, the index in 'i = 2, · · ·, K' is never used, and the later text switches to k; the notation should be made consistent.
[Section III, Lemma 4] The sentence 'with monotonicity in Proposition 1, LDICs also imply that LUICs can be satisfied' is asserted without proof or reference; please provide the argument or a citation.
[Section IV, satisfaction values] The paper acknowledges that the satisfaction values A_k are assumed because no public user-satisfaction data exist for this task domain; this is acceptable for a modeling paper, but the abstract and conclusion should not describe the numerical study as a validation of QoS alignment given that A_k is a free input.
[Introduction] The phrase 'on behave of users' in the first paragraph should be 'on behalf of users'.
[Section II-C, Eq. (6)] If latency is intended to be normalized, the normalization should be stated explicitly; otherwise 1−T_k can become negative for realistic latencies, which makes q_k fall outside the [0,1] interval implied by the later monotonicity condition.

Circularity Check

1 steps flagged · score 6.0 of 10

Numerical 'validation' of self-selection restates the IC constraints that were imposed as hard constraints in Eq. (16), so the evaluation is circular by construction; the theoretical contract derivation itself is standard and not circular.

self definitional [Section IV, Figure 2 caption and text; cf. Definitions 2–3 and Eq. (16)]
"The results show that each user achieves the highest utility by selecting the contract designed for their type. This behavior confirms that the designed contracts satisfy the individual rationality and incentive compatibility constraints, as defined in Definitions 2 and 3, respectively."

The optimization problem (16) is solved subject to the reduced IR constraint u_1(q_1,p_1) >= 0 and the reduced IC constraints theta_k v(q_k) - p_k >= theta_k v(q_{k-1}) - p_{k-1} for k = 2..K. Thus any feasible solution of (16) is, by construction, a menu in which each type theta_k weakly prefers its own contract. The numerical 'result' that each user obtains highest utility from their own contract is therefore a restatement of Definition 3 enforced as a constraint, not an independent prediction or empirical validation. Moreover, the satisfaction values A_k are assumed in Section IV ('we assume satisfaction increases with larger input/output sizes and more powerful models'), so the experiment contains no external data that could refute the self-selection property.

full rationale

The mathematical contract-design part (Lemmas 1–4 and problem (16)) is a standard application of contract theory and is not circular: the lemmas are proved from the stated utility and cost assumptions, and the IR/IC constraints are the intended design requirements rather than outputs claimed as predictions. No load-bearing self-citation chain exists: reference [14] is cited for the contract-mechanism idea, but the present paper re-derives the needed IC/IR reduction, and no uniqueness theorem is imported from the authors' prior work. The one genuine circularity is the numerical 'confirmation' of self-selection, which directly restates the IC constraints imposed in (16); this is a partial circularity in the evaluation section, not in the framework derivation. Separately, the numerical values in Table I are not reproducible from Eqs. (1)–(4) and (6) with the stated parameters: for row 8, including inference time gives q approximately 0.68 rather than the tabulated 0.848, and several rows appear to omit T_inf. That internal inconsistency is a correctness risk, not a circularity, and is not counted in the circularity score. The paper also openly notes that A_k is assumed and liability costs are ad hoc, so the synthetic experiment is illustrative rather than externally validated. The score of 6 reflects that one 'prediction' (QoS alignment via self-selection) reduces by construction, while the underlying contract-optimization framework retains independent content.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The framework relies on standard contract theory assumptions (quasi-linear utility, known type distribution) and on several ad hoc modeling choices: the QoS formula, arbitrary satisfaction and liability values, and a fitted cost curve. No new physical or conceptual entity is introduced.

free parameters (6)

δ (QoS weighting) = 0.5
Task-specific weight balancing satisfaction vs latency in Eq. (6), set to 0.5 in Section IV without sensitivity analysis.
A_k (user satisfaction) = 0.1 to 0.9 (Table I)
Assumed satisfaction probabilities; no user data. The paper states 'we assume satisfaction increases with larger input/output sizes and more powerful models' (Section IV).
c_l (liability cost) = 0.7, 0.5, 0.2 for GPT-2, Phi-2, Gemma 7B
Arbitrary liability values chosen in Section IV to illustrate the effect; no legal or regulatory basis.
Fitted C(q) curve = unspecified functional form and parameters
The paper fits the cost as a function of q from 8 configurations in Section IV, but does not report the fitted function or parameters.
Transmission/tokenization constants (α_t, α_dt, g_t, g_dt, r) = α = 0.5 ms/KB, g(D) = 4D, r = 20 Mbps
Numerical example parameters chosen by hand in Section IV.
User type distribution and valuation function v(q) = θ_k = k; v(·) unspecified
Types chosen as integers in Section IV; v is assumed concave but never specified, making the simulation incomplete.

assumptions (6)

standard math User utility is quasi-linear: u = θ v(q) - p (Eq. 9).
Standard in contract theory.
standard math Valuation v(q) is strictly increasing and concave with v(0) = 0.
Ensures the single-crossing condition required for the IC reduction.
ad hoc to paper QoS is measured by q = δ A + (1-δ)(1 - T) (Eq. 6).
This additive normalization is introduced without derivation; T must be scaled to [0,1] but is not.
domain assumption The SP knows the distribution P(θ_k) but not the exact type.
Standard assumption in contract theory (Section III-B).
domain assumption Inference time follows the FLOP scaling law from Kaplan et al. (Eq. 3).
Heuristic borrowed from prior literature.
domain assumption Liability cost c_l is a fixed additive cost per service level.
Not derived from any legal model; the paper itself notes assessing liability requires further investigation.

how reviews work

0 comments

Cite this review

Pith. "Pith review of PACT: A Contract-Theoretic Framework for Pricing Agentic AI Services Powered by Large Language Models." pith.science (2026). https://pith.science/paper/UJHJY5W4

@misc{pith2026250521286,
  author       = {Pith},
  title        = {Pith review of: PACT: A Contract-Theoretic Framework for Pricing Agentic AI Services Powered by Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UJHJY5W4}},
  note         = {Machine review of arXiv:2505.21286}
}

read the original abstract

Agentic AI, often powered by large language models (LLMs), is becoming increasingly popular and adopted to support autonomous reasoning, decision-making, and task execution across various domains. While agentic AI holds great promise, its deployment as services for easy access raises critical challenges in pricing, due to high infrastructure and computation costs, multi-dimensional and task-dependent Quality of Service (QoS), and growing concerns around liability in high-stakes applications. In this work, we propose PACT, a Pricing framework for cloud-based Agentic AI services through a Contract-Theoretic approach, which models QoS along both objective (e.g., response time) and subjective (e.g., user satisfaction) dimensions. PACT accounts for computational, infrastructure, and potential liability costs for the service provider, while ensuring incentive compatibility and individual rationality for the user under information asymmetry. Through contract-based selection, users receive tailored service offerings aligned with their needs. Numerical evaluations demonstrate that PACT improves QoS alignment between users and providers and offers a scalable, liable approach to pricing agentic AI services in the future.

Figures

Figures reproduced from arXiv: 2505.21286 by the authors.

**Figure 2.** User utilities for types 1,5,10,15 under different [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** The QoS levels and corresponding prices of contracts [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: The user’s and provider’s utilities as well as the social [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

17 extracted references · 6 canonical work pages

[1]

Agentic ai: Autonomous intelligence for complex goals–a comprehensive survey,

D. B. Acharya, K. Kuppan, and B. Divya, “Agentic ai: Autonomous intelligence for complex goals–a comprehensive survey,”IEEE Access, 2025

2025
[2]

Generative ai as a service in 6g edge-cloud: Generation task offloading by in-context learning,

H. Zhou, C. Hu, D. Yuan, Y . Yuan, D. Wu, X. Liu, Z. Han, and J. Zhang, “Generative ai as a service in 6g edge-cloud: Generation task offloading by in-context learning,”IEEE Wireless Communications Letters, vol. 14, no. 3, pp. 711–715, 2025

2025
[3]

Optimus: Optimization modeling using mip solvers and large language models,

A. AhmadiTeshnizi, W. Gao, and M. Udell, “Optimus: Optimization modeling using mip solvers and large language models,”arXiv preprint arXiv:2310.06116, 2023

arXiv 2023
[4]

The cost perspective of adopting large language model-as-a-service,

V . Liagkou, E. Filiopoulou, G. Fragiadakis, M. Nikolaidou, and C. Michalakelis, “The cost perspective of adopting large language model-as-a-service,” in2024 IEEE International Conference on Joint Cloud Computing (JCC). IEEE, 2024, pp. 80–83

work page 2024
[5]

Llm as a system service on mobile devices,

W. Yin, M. Xu, Y . Li, and X. Liu, “Llm as a system service on mobile devices,”arXiv preprint arXiv:2403.11805, 2024

arXiv 2024
[6]

The economics of large language models: Token allocation, fine-tuning, and optimal pricing,

D. Bergemann, A. Bonatti, and A. Smolin, “The economics of large language models: Token allocation, fine-tuning, and optimal pricing,” arXiv preprint arXiv:2502.07736, 2025

arXiv 2025
[7]

Assessing llms for high stakes applications,

S. K. Gallagher, J. Ratchford, T. Brooks, B. P. Brown, E. Heim, W. R. Nichols, S. Mcmillan, S. Rallapalli, C. J. Smith, N. VanHoudnos et al., “Assessing llms for high stakes applications,” inProceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice, 2024, pp. 103–105

work page 2024
[8]

Chain-of-thought prompting elicits reasoning in large language models,

J. Wei, X. Wang, D. Schuurmans, M. Bosma, b. ichter, F. Xia, E. Chi, Q. V . Le, and D. Zhou, “Chain-of-thought prompting elicits reasoning in large language models,” inAdvances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35. Curran Associates, Inc., 2022, pp. 24 824–24 837

work page 2022

Show all 17 references

[9]

Understanding user experience in large language model interactions,

J. Wang, W. Ma, P. Sun, M. Zhang, and J.-Y . Nie, “Understanding user experience in large language model interactions,”arXiv preprint arXiv:2401.08329, 2024

2024 arXiv
[10]

Agentbench: Evaluating llms as agents,

X. Liu, H. Yu, H. Zhang, Y . Xu, X. Lei, H. Lai, Y . Gu, H. Ding, K. Men, K. Yanget al., “Agentbench: Evaluating llms as agents,”arXiv preprint arXiv:2308.03688, 2023

2023 arXiv
[11]

Interpretable user satisfaction estimation for conversational systems with large language models,

Y .-C. Lin, J. Neville, J. W. Stokes, L. Yang, T. Safavi, M. Wan, S. Counts, S. Suri, R. Andersen, X. Xuet al., “Interpretable user satisfaction estimation for conversational systems with large language models,”arXiv preprint arXiv:2403.12388, 2024

2024 arXiv
[12]

Exploring prosocial irrationality for llm agents: A social cognition view,

X. Liu, J. Zhang, H. Shang, S. Guo, C. Yang, and Q. Zhu, “Exploring prosocial irrationality for llm agents: A social cognition view,”arXiv preprint arXiv:2405.14744, 2024

2024 arXiv
[13]

Contract-based incentive mechanisms for device-to-device communications in cellular networks,

Y . Zhang, L. Song, W. Saad, Z. Dawy, and Z. Han, “Contract-based incentive mechanisms for device-to-device communications in cellular networks,”IEEE Journal on Selected Areas in Communications, vol. 33, no. 10, pp. 2144–2155, 2015

2015
[14]

Qos-based contract design for profit maximization in iot-enabled data markets,

J. Chen, J. Farooq, and Q. Zhu, “Qos-based contract design for profit maximization in iot-enabled data markets,”IEEE Internet of Things Journal, vol. 10, no. 11, pp. 10 080–10 094, 2023

2023
[15]

How to evaluate llms: A com- plete metric framework,

M. Research, “How to evaluate llms: A com- plete metric framework,” September 2023. [Online]. Available: https://www.microsoft.com/en-us/research/articles/how-to- evaluate-llms-a-complete-metric-framework/

2023
[16]

Scaling laws for neural language models,

J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei, “Scaling laws for neural language models,”arXiv preprint arXiv:2001.08361, 2020

2001 arXiv
[17]

Inherent and emergent liability issues in llm-based agentic systems: a principal-agent perspective,

G. A. Gabison and R. P. Xian, “Inherent and emergent liability issues in llm-based agentic systems: a principal-agent perspective,”arXiv preprint arXiv:2504.03255, 2025

2025 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Agentic ai: Autonomous intelligence for complex goals–a comprehensive survey,

D. B. Acharya, K. Kuppan, and B. Divya, “Agentic ai: Autonomous intelligence for complex goals–a comprehensive survey,”IEEE Access, 2025

2025

[2] [2]

Generative ai as a service in 6g edge-cloud: Generation task offloading by in-context learning,

H. Zhou, C. Hu, D. Yuan, Y . Yuan, D. Wu, X. Liu, Z. Han, and J. Zhang, “Generative ai as a service in 6g edge-cloud: Generation task offloading by in-context learning,”IEEE Wireless Communications Letters, vol. 14, no. 3, pp. 711–715, 2025

2025

[3] [3]

Optimus: Optimization modeling using mip solvers and large language models,

A. AhmadiTeshnizi, W. Gao, and M. Udell, “Optimus: Optimization modeling using mip solvers and large language models,”arXiv preprint arXiv:2310.06116, 2023

arXiv 2023

[4] [4]

The cost perspective of adopting large language model-as-a-service,

V . Liagkou, E. Filiopoulou, G. Fragiadakis, M. Nikolaidou, and C. Michalakelis, “The cost perspective of adopting large language model-as-a-service,” in2024 IEEE International Conference on Joint Cloud Computing (JCC). IEEE, 2024, pp. 80–83

work page 2024

[5] [5]

Llm as a system service on mobile devices,

W. Yin, M. Xu, Y . Li, and X. Liu, “Llm as a system service on mobile devices,”arXiv preprint arXiv:2403.11805, 2024

arXiv 2024

[6] [6]

The economics of large language models: Token allocation, fine-tuning, and optimal pricing,

D. Bergemann, A. Bonatti, and A. Smolin, “The economics of large language models: Token allocation, fine-tuning, and optimal pricing,” arXiv preprint arXiv:2502.07736, 2025

arXiv 2025

[7] [7]

Assessing llms for high stakes applications,

S. K. Gallagher, J. Ratchford, T. Brooks, B. P. Brown, E. Heim, W. R. Nichols, S. Mcmillan, S. Rallapalli, C. J. Smith, N. VanHoudnos et al., “Assessing llms for high stakes applications,” inProceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice, 2024, pp. 103–105

work page 2024

[8] [8]

Chain-of-thought prompting elicits reasoning in large language models,

J. Wei, X. Wang, D. Schuurmans, M. Bosma, b. ichter, F. Xia, E. Chi, Q. V . Le, and D. Zhou, “Chain-of-thought prompting elicits reasoning in large language models,” inAdvances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35. Curran Associates, Inc., 2022, pp. 24 824–24 837

work page 2022