Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

Performance Prediction for Large Systems via Text-to-Text Regression

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 60M-parameter encoder-decoder trained from random initialization predicts Google's Borg cluster efficiency with near-perfect rank correlation (0.99) and 100x lower MSE than tabular approaches, adapting to new cells from only 512 examples.

desk verdict The 0.99 fleet-wide result is real but belongs to a 267M model fine-tuned per task; the 60M default's best is 0.86—the paper's core method is credible, but the abstract overstates it. read the letter →

arxiv 2506.21718 v1 pith:KOIXCZ3R submitted 2025-06-26 cs.LG cs.AIcs.PFcs.SEcs.SYeess.SY

Yash Akhauri , Bryan Lewandowski , Cheng-Hsi Lin , Adrian N. Reyes , Grant C. Forbes , Arissa Wongpanich , Bangding Yang , Mohamed S. Abdelfattah
show 2 more authors
Sagi Perel Xingyou Song
This is my paper
classification cs.LGcs.AIcs.PFcs.SEcs.SYeess.SY
keywords text-to-textregressionperformancepredictionencoder-decodertransformernext-tokenBorgclusterfew-shotadaptationuncertaintyquantificationtabular
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that text-to-text regression—training a small encoder-decoder language model to predict numbers from raw string representations of system state—can replace hand-engineered feature pipelines for performance prediction in large industrial systems. On Google's Borg compute cluster, a 60M-parameter model trained from random initialization with standard next-token cross-entropy predicts the MIPS-per-GCU efficiency metric with Spearman rank correlation up to 0.99 (0.9 average across the evaluated cells) and achieves 100x lower MSE than what is possible with tabular-only features. The model also adapts to unseen cells with as few as 512 fine-tuning examples and produces predictive distributions whose variance tracks actual prediction error. If correct, this makes cheap, accurate 'universal simulators' of real-world outcomes feasible without manual feature engineering or expensive simulations.

What carries the argument

The machinery is a text-to-text regression language model (RLM): inputs x are serialized as YAML strings containing nested features—cell name, time window, scheduler hyperparameters, machine distributions, and per-job platform profiles—and the numeric target y is tokenized with the normalization-free P10 scheme, e.g. <+><7><2><5><E-1> for 72.5. A 2-encoder/2-decoder transformer (58-60M parameters) is trained from random initialization using cross-entropy next-token prediction, so the model learns a conditional density p_theta(y|x) from which 128 samples are drawn and aggregated (mean for MSE, median for rank). The encoder is load-bearing for processing complex x, and skipping language pretraining works because regression needs token correlations, not semantics. Pretraining over multiple cells and months provides transferable knowledge that few-shot fine-tuning with 512 examples exploits for new tasks.

What would settle it

Run the same pretraining and fine-tuning protocol on a random sample of cells drawn from the full fleet, including low-spread cells, and check whether the average Spearman rank correlation stays at or above 0.9 and the MSE gap against the tabular-only bound remains at least 10x; if either fails, the 'entire fleet' generalization is unsupported.

Watch

Extended reading notes

Core claim

The paper's central claim is that a 60M-parameter encoder-decoder, trained from random initialization solely with next-token cross-entropy on YAML-serialized system states, becomes an accurate regressor for a metric—MIPS per GCU—that is the product of bin-packing algorithms, hardware mixes, workload patterns, and scheduler hyperparameters. For the highest-spread cell the rank correlation reaches 0.99, and on average across evaluated cells it is about 0.9. Measured in MSE, the model's residuals sit 100x below the theoretical lower bound for any regressor that sees only tabular features, and its sampled predictive distribution tracks the true conditional distribution well enough that the variance of samples correlates with squared error (Spearman 0.685). The same checkpoint fine-tunes to a new cell on 512 examples, with more pretraining tasks improving out-of-distribution transfer.

Load-bearing premise

The load-bearing premise is that the 40 cells with the largest y-value spreads, used for benchmarking, are representative of the entire fleet, so that the reported rank correlations and MSE gains transfer to cells with smaller spreads.

Editorial extensions

If this is right

  • Any large system with logged (x, y) data could be simulated directly from raw text, avoiding the per-dataset cost of designing tabular features, normalizing values, and re-engineering pipelines whenever new hardware or workloads appear.
  • Scheduler and cluster optimizers that currently rely on tabular-only Gaussian-process regressors could instead query the RLM's fast, accurate predictions, cutting hours-long digital-twin simulations to near-instant inference.
  • A pretrained RLM can be repurposed to a new cell or scenario with roughly 500 examples, so new tasks need tiny labeled sets rather than millions of points.
  • Because p_theta(y|x) is a full density, the model reports calibrated uncertainty (variance correlates with residual error), which is directly usable for Bayesian optimization and risk-aware decisions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural stress test the paper leaves open: the reported gains are measured against the theoretical variance floor for tabular features, not against a strong trained tabular baseline, so head-to-head comparisons with gradient-boosted trees or MLPs on the best-effort features could shrink the practical gap.
  • The evaluation pool is restricted to the 40 highest-variance cells; extending the identical protocol to the full fleet, including low-spread cells, would test whether the 0.9 average rank correlation transfers beyond the selected subset.
  • The calibrated predictive variance suggests an active-learning loop—query states where the model is most uncertain—which the paper does not explore but its own uncertainty-versus-error evidence would support.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper proposes text-to-text regression as a general approach for predicting performance metrics of large industrial systems, demonstrated on Google's Borg cluster. The method uses an encoder-decoder language model trained with next-token cross-entropy on string representations of system state, predicting MIPS/GCU. The authors report high Spearman rank correlations, a 100x lower MSE than tabular-feature lower bounds, few-shot adaptation with 512 examples, density estimation, and ablations on loss, architecture, feature observability, and uncertainty. The appendix contains detailed data organization, experimental settings, and per-figure configuration notes.

Significance. If the headline results were fully supported, this would be a significant demonstration that a small text-to-text model can replace manual feature engineering and expensive simulations for a complex industrial prediction task. The paper's strengths include a clearly described method, an open-source code repository, a thoughtful use of total-variance lower bounds to motivate feature observability, and useful ablations (encoder-decoder vs decoder-only, sequence length, feature subsets) that are informative for practitioners. However, the principal quantitative claims are not tied to the configuration described in the abstract, and the sampling protocol limits fleet-level generalization; these issues need to be resolved before the contribution can be assessed at face value.

major comments (4)
  1. [Abstract; Appendix C.2-C.3; Figures 10-12] The abstract's headline claim that a 60M encoder-decoder trained from random initialization achieves 0.99 rank correlation across the entire fleet is not supported by the reported experiments. Appendix C.2 defines the 'Limit Testing' run as a 267M-parameter model with 4096 sequence length and batch size 256, and Appendix C.3 states that Figures 10 and 11, as well as Figure 12, use this Limit Testing setting, with per-task fine-tuning on 512 examples. The only default-size (58M/60M) result in the main text is Figure 4, where the best in-distribution Spearman correlation shown is 0.86. No figure or table reports a default-size model reaching 0.99. The abstract should either attribute the fleet-wide numbers to the 267M fine-tuned model or the authors should run the multi-task benchmark with the default configuration; as written, the central quantitative claim is disconnected from the experimental configuration.
  2. [Section 4.1; Appendix B.2; Abstract] The 'across the entire fleet' claim is not supported by the evaluation protocol. Section 4.1 states that the paper uses a total pool of 40 cells with the largest y-value spreads, and Appendix B.2 confirms that the cell index is sorted by spread. This is a selected subsample, not a random or representative sample of the fleet, and the selection is on the outcome variable, which can inflate rank correlations and make MSE gains easier to achieve. The paper should either evaluate on a representative sample of cells or explicitly restrict all fleet-level claims to the high-spread cells used in the study.
  3. [Section 4.2, Figure 6; Section 2.3, Eq. (2)] The '100x lower MSE than tabular approaches' claim compares the RLM's MSE against the TotalVariance_phi lower bound for tabular features, not against a trained tabular baseline. This comparison is a legitimate way to demonstrate the value of feature observability only if the bound is estimated reliably. Equation (2) computes empirical variances within equivalence classes; classes with very few samples yield unreliable variance estimates (for singletons, zero), which can make the tabular lower bound artificially small and the RLM's relative gain look larger. The paper does not report class-size statistics or a debiased estimator. Please report the distribution of equivalence-class sizes, use a debiased variance estimate, and/or add a trained tabular baseline (e.g., gradient boosting on the same tabular features) to make the comparison concrete.
  4. [Appendix C.3, Figure 12; Appendix A.2] The density-estimation metric R2_NLL relies on NLLnull estimated by a randomly initialized model fine-tuned on empty strings with 1024 examples. This is not a neutral estimate of the unconditional density p(y): a randomly initialized model fine-tuned on empty strings may produce a degenerate or poorly calibrated distribution, which would inflate R2_NLL and make the density-estimation gains look larger than they are. The authors should validate NLLnull against a simple empirical baseline (e.g., a histogram or Gaussian fit to the pooled y-values) or report both, and should state whether the qualitative density results in Figures 5, 9, and 11 are affected by this choice.
minor comments (6)
  1. [Abstract; Appendix C.1] The abstract says 60M parameters, but Appendix C.1 specifies the default model as 58M parameters; please align these numbers.
  2. [Abstract; Section 3.3] The abstract says 'only 500 few-shot examples,' while the experiments and Appendix C.3 use 512 examples; please unify the terminology or explain the rounding.
  3. [Abstract; Section 4.2] The '0.9 average' rank correlation is undefined; please specify whether it is the mean over the ten tasks in Figure 10, over all 40 selected cells, or over some other set, and report the corresponding standard deviation or range.
  4. [Figures 10-12; Appendix C.2-C.3] The figure captions should state that these results use the 267M Limit-Testing model with 4096 sequence length, per-task fine-tuning on 512 examples, and tasks that are in the Limit Testing pretraining list; the main text should point readers to Appendix C.2-C.3 when these figures are introduced.
  5. [Figure 8] The y-axis label contains a typo ('Uncertainity' should be 'Uncertainty'), and the figure would benefit from a statement of the number of inputs and seeds used.
  6. [Section 5.1] The claim that validation loss correlates with MSE and Spearman would be stronger with the numeric correlation coefficients, which are currently only shown visually in Figures 13 and 14.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: the Borg prediction results are held-out empirical evaluations, and the only self-referential elements (OmniPred basis, RLM-as-null for NLL) are not load-bearing for the central claims.

full rationale

The claimed derivation chain is empirical rather than definitional. The RLM predictions are evaluated on held-out test splits after standard 8/1/1 random splits (Section 4.1), using scale-invariant Spearman correlation and MSE. The TotalVariance lower bound (Eq. 2, Appendix A.1) is a mathematical consequence of the law of total variance, not a fitted quantity, and comparing RLM MSE against it is a legitimate benchmark. The 100x MSE and rank-correlation numbers are measurements on test data, not quantities constructed from the baselines or from the model's own outputs. The strongest headline claim (0.99 rank correlation) is attributed in the abstract to a 60M model, while Figures 10-11 use the 267M Limit Testing checkpoint (Appendices C.2-C.3). This is an internal-consistency and reporting defect, not a circularity: the reported metric is still an evaluation of a trained model on held-out data, not an identity or a fitted quantity. Similarly, selecting the 40 highest-spread cells is a sampling and external-validity concern, not a circular derivation. The only self-referential elements are (i) the method is described as based on OmniPred (Song et al. 2024a), a prior paper with overlapping authorship, and (ii) Appendix A.2 chooses the RLM itself, citing Song and Bahri (2025), as the null density estimator for R^2_NLL. Neither is load-bearing for the central Borg prediction claim: OmniPred motivates the architecture, but the Borg results are independently reported; and the NLL_null baseline is an empirical comparison that could in principle be computed with any density estimator, while the self-cited universal-density-approximation remark is not needed to define the metric and does not force the outcome. Appendix A.2 itself notes that NLL_null is an empirical approximation of the true entropy, which is a limitation but not circularity. No equation in the paper reduces to its own input, and no fitted parameter is renamed as a prediction. The score of 1 reflects the presence of minor self-citations without load-bearing circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard statistical identities, the fidelity of the Borg digital twin, the sufficiency of the text representation, and the representativeness of the selected cells. The model hyperparameters are standard training choices, not fitted constants for a theoretical derivation, but they do influence the reported numbers.

free parameters (4)
  • Default sequence length (2048) = 2048
    Chosen as default; ablation (Figure 17) shows larger lengths up to 4096 give modest gains, so the choice affects performance.
  • Fine-tuning learning rate (5e-5) = 5e-5
    Chosen for fine-tuning; ablation (Figure 19) shows large sensitivity to this value.
  • P10 mantissa digit count (4) = 4
    Tokenization detail from Charton (2022); the number of mantissa digits limits numeric precision.
  • Outlier removal range [500, 3000] = [500, 3000]
    Used when sampling predictions; it clips extreme samples and could bias density estimates.
assumptions (5)
  • standard math Law of total variance and bias-variance decomposition
    Used in Section 2.3 to define lower bounds on regressor MSE and motivate feature observability.
  • domain assumption The Borg digital twin's MIPS per GCU output is a meaningful, accurate measure of cluster efficiency
    The paper treats the logged outcomes from the backtesting framework (Section 2.1) as ground truth for training and evaluation.
  • domain assumption The string representation of x contains all relevant features for predicting y
    The method relies on the assumption that converting system state to YAML-like text preserves the information needed to predict efficiency (Section 2.4).
  • domain assumption The 40 cells with largest y-spread represent the fleet
    Dataset selection in Section 4.1 and Appendix B.2; external validity of the 'entire fleet' claim depends on this.
  • domain assumption A model trained with one y per x can learn the conditional distribution p(y|x)
    The paper claims density estimation from training data where each x is unique (Figures 5, 9, 11); this is a strong statistical assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Performance Prediction for Large Systems via Text-to-Text Regression." pith.science (2026). https://pith.science/paper/KOIXCZ3R

@misc{pith2026250621718,
  author       = {Pith},
  title        = {Pith review of: Performance Prediction for Large Systems via Text-to-Text Regression},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KOIXCZ3R}},
  note         = {Machine review of arXiv:2506.21718}
}
read the original abstract

In many industries, predicting metric outcomes of large systems is a fundamental problem, driven largely by traditional tabular regression. However, such methods struggle on complex systems data in the wild such as configuration files or system logs, where feature engineering is often infeasible. We propose text-to-text regression as a general, scalable alternative. For predicting resource efficiency on Borg, Google's massive compute cluster scheduling system, a 60M parameter encoder-decoder, trained from random initialization, achieves up to a near perfect 0.99 (0.9 average) rank correlation across the entire fleet, and 100x lower MSE than tabular approaches. The model also easily adapts to new tasks in only 500 few-shot examples and captures the densities of complex outcome distributions. Ablation studies highlight the importance of using encoders, increasing sequence length, and the model's inherent uncertainty quantification. These findings pave the way for universal simulators of real-world outcomes.

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ArchEval: Measuring AI Agents as Computer Architects

    cs.AR 2026-07 conditional novelty 7.0 of 10

    LLM agents beat architecture baselines with full simulator harnesses, but only one configuration stays above baseline without feedback, and performance modeling remains weak.

  2. Towards Reliable C-to-Rust Translation with Rule-Guided Reasoning and Reinforcement Learning

    cs.SE 2026-07 conditional novelty 6.0 of 10

    A rule-guided MCTS plus dual-reward reinforcement learning pipeline improves LLM-based C-to-Rust translation accuracy and cuts unsafe Rust output across three benchmarks.

Reference graph

Works this paper leans on

29 extracted references · 17 canonical work pages · cited by 2 Pith papers

  1. [1]

    F. Charton. Linear algebra with transformers. Trans. Mach. Learn. Res., 2022, 2022

    work page 2022

  2. [2]

    Didona, F

    D. Didona, F. Quaglia, P. Romano, and E. Torre. Enhancing performance prediction robustness by combining analytical modeling and machine learning. In L. K. John, C. U. Smith, K. Sachs, and C. M. Llad \' o , editors, Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering, Austin, TX, USA, January 31 - February 4, 2015 , pages 1...

    work page arXiv 2015

  3. [3]

    C. Finn, P. Abbeel, and S. Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017 , volume 70 of Proceedings of Machine Learning Research, pages 1126--1135. PMLR , 2017

    work page 2017

  4. [4]

    S. Fu, S. Gupta, R. Mittal, and S. Ratnasamy. On the use of ML for blackbox system performance prediction. In J. Mickens and R. Teixeira, editors, 18th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2021, April 12-14, 2021 , pages 763--784. USENIX Association, 2021

    work page 2021

  5. [5]

    Golovin, B

    D. Golovin, B. Solnik, S. Moitra, G. Kochanski, J. Karro, and D. Sculley. Google vizier: A service for black-box optimization. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada, August 13 - 17, 2017 , pages 1487--1495. ACM , 2017. doi:10.1145/3097983.3098043

    arXiv 2017

  6. [6]

    H \" u llermeier and W

    E. H \" u llermeier and W. Waegeman. Aleatoric and epistemic uncertainty in machine learning: A tutorial introduction. CoRR, abs/1910.09457, 2019

    arXiv 1910

  7. [7]

    Kudo and J

    T. Kudo and J. Richardson. Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 66--71, 2018

    work page 2018

  8. [8]

    H. V. Madhyastha, T. E. Anderson, A. Krishnamurthy, N. Spring, and A. Venkataramani. A structural approach to latency prediction. In J. M. Almeida, V. A. F. Almeida, and P. Barford, editors, Proceedings of the 6th ACM SIGCOMM Internet Measurement Conference, IMC 2006, Rio de Janeriro, Brazil, October 25-27, 2006 , pages 99--104. ACM , 2006. doi:10.1145/11...

    work page arXiv 2006

Show all 29 references
  1. [9]

    Mahan, D

    D. Mahan, D. V. Phung, R. Rafailov, C. Blagden, N. Lile, L. Castricato, J.-P. Fränken, C. Finn, and A. Albalak. Generative reward models, 2024

    2024

  2. [10]

    Manousis, R

    A. Manousis, R. A. Sharma, V. Sekar, and J. Sherry. Contention-aware performance prediction for virtualized network functions. In H. Schulzrinne and V. Misra, editors, SIGCOMM '20: Proceedings of the 2020 Annual conference of the ACM Special Interest Group on Data Communicatio...

    2020

  3. [11]

    McFadden

    D. McFadden. Conditional logit analysis of qualitative choice behavior. In P. Zarembka, editor, Fontiers in Econometrics , pages 105--142. Academic press, New York, 1974

    1974

  4. [12]

    Mesnard, C

    T. Mesnard, C. Hardin, R. Dadashi, S. Bhupatiraju, S. Pathak, L. Sifre, M. Rivi \` e re, M. S. Kale, J. Love, P. Tafti, L. Hussenot, A. Chowdhery, A. Roberts, A. Barua, A. Botev, A. Castro - Ros, A. Slone, A. H \' e liou, A. Tacchetti, A. Bulanova, A. Paterson, B. Tsai, B. Sha...

    2024 doi arXiv

  5. [13]

    Schneider and T

    I. Schneider and T. Mattia. Carbon accounting in the cloud: a methodology for allocating emissions across data center users, 2024

    2024

  6. [14]

    Shazeer and M

    N. Shazeer and M. Stern. Adafactor: Adaptive learning rates with sublinear memory cost. In J. Dy and A. Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 4596--4604. PMLR, 10--15 Jul 2018

    2018

  7. [15]

    Silver and R

    D. Silver and R. S. Sutton. Welcome to the era of experience, 2025

    2025

  8. [16]

    Song and D

    X. Song and D. Bahri. Decoding-based regression, 2025

    2025

  9. [17]

    X. Song, O. Li, C. Lee, B. Yang, D. Peng, S. Perel, and Y. Chen. Omnipred: Language models as universal regressors. CoRR, abs/2402.14547, 2024 a

    2024 arXiv

  10. [18]

    X. Song, Q. Zhang, C. Lee, E. Fertig, T. Huang, L. Belenki, G. Kochanski, S. Ariafar, S. Vasudevan, S. Perel, and D. Golovin. The vizier gaussian process bandit algorithm. CoRR, abs/2408.11527, 2024 b . doi:10.48550/ARXIV.2408.11527

    2024 doi arXiv

  11. [19]

    Stewart, T

    C. Stewart, T. Kelly, and A. Zhang. Exploiting nonstationarity for performance prediction. In Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007, EuroSys '07, page 31–44, New York, NY, USA, 2007. Association for Computing Machinery. ISBN 978...

    2007

  12. [20]

    R.-X. Tan, M. Chen, K. Xue, Y. Wang, Y. Wang, F. Sheng, and C. Qian. Towards universal offline black-box optimization via learning string embedding space. In International Conference in Machine Learning, 2025

    2025

  13. [21]

    Touvron, T

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M. Lachaux, T. Lacroix, B. Rozi \` e re, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample. Llama: Open and efficient foundation language models. CoRR, abs/2302.13971, 2023. doi:10.48550/ARXIV.2302.13971

    2023 doi arXiv

  14. [22]

    Vacareanu, V

    R. Vacareanu, V. Negru, V. Suciu, and M. Surdeanu. From words to numbers: Your large language model is secretly A capable regressor when given in-context examples. CoRR, abs/2404.07544, 2024. doi:10.48550/ARXIV.2404.07544

    2024 doi arXiv

  15. [23]

    Venkataraman, Z

    S. Venkataraman, Z. Yang, M. J. Franklin, B. Recht, and I. Stoica. Ernest: Efficient performance prediction for large-scale advanced analytics. In K. J. Argyraki and R. Isaacs, editors, 13th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2016, Santa Clar...

    2016

  16. [24]

    Verma, L

    A. Verma, L. Pedrosa, M. R. Korupolu, D. Oppenheimer, E. Tune, and J. Wilkes. Large-scale cluster management at Google with Borg . In Proceedings of the European Conference on Computer Systems (EuroSys), Bordeaux, France, 2015

    2015

  17. [25]

    Zhang, A

    L. Zhang, A. Hosseini, H. Bansal, M. Kazemi, A. Kumar, and R. Agarwal. Generative verifiers: Reward modeling as next-token prediction. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025 , 2025

    2025

  18. [26]

    Zhang, L

    S. Zhang, L. Dong, X. Li, S. Zhang, X. Sun, S. Wang, J. Li, R. Hu, T. Zhang, F. Wu, et al. Instruction tuning for large language models: A survey. arXiv preprint arXiv:2308.10792, 2023

    2023

  19. [27]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  20. [28]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  21. [29]

    )tf th-t:'v0!¦ Vd&Cvmt&Fl وEO X= uL+]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.