Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Arctic Long Sequence Training: Scalable And Efficient Training For Multi-Million Token Sequences

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims ALST trains the 8-billion-parameter Llama model on sequences up to 15 million tokens with 32 H100 GPUs — a 469x increase over the 32K baseline — while keeping the same training loss.

desk verdict Genuine and useful memory engineering for long-sequence HF training, but the multi-million-token 'training' claims are backed only by a 32K loss match and a NaN check; deserves peer review with a request for a longer-scale correctness check. read the letter →

arxiv 2506.13996 v1 pith:GDDAEGGY submitted 2025-06-16 cs.LG

Stas Bekman , Samyam Rajbhandari , Michael Wyatt , Jeff Rasley , Tunji Ruwase , Zhewei Yao , Aurick Qiao , Yuxiong He This is my paper
classification cs.LG
keywords longsequencetrainingparallelismUlyssestilingactivationcheckpointoffloadingmemoryoptimizationfine-tuningmulti-milliontokencontexts
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that long-sequence training — the ability to fine-tune models on contexts of millions of tokens — can be made practical on ordinary GPU clusters without rewriting model code. It introduces a stack of memory optimizations built around sequence parallelism: the sequence is split across GPUs for most computations, reassembled only inside the attention block, and memory-hungry per-token operations (logits, loss, MLP layers) are computed in small sequence tiles. The paper reports that this lets an 8-billion-parameter Llama model train at 500 thousand tokens on a single H100 GPU, 3.7 million tokens on one 8-GPU node, and more than 15 million tokens on 32 GPUs — a 469-fold increase over the 32-thousand-token baseline. It also reports that the training loss matches the baseline when both run at 32K sequence length, indicating the optimizations preserve training quality.

What carries the argument

The load-bearing mechanism is Ulysses sequence parallelism adapted for the Transformers library: each GPU processes a shard of the sequence up to the attention block, an all-to-all exchange reshuffles the hidden states so each GPU owns a subset of attention heads for the full sequence, and a second all-to-all restores the sequence-sharded layout. Two tiling mechanisms supplement it: TiledCompute/TiledMLP break the logits, loss, and MLP computations along the sequence dimension so intermediate tensors stay small, and activation-checkpoint CPU offload moves checkpointed hidden states to host memory, flattening the per-layer memory 'hill.' Together they convert activation memory from a quantity that scales with sequence length and layer count into one that is bounded per tile or offloaded, while the attention block remains the only place where the full sequence is materialized.

What would settle it

Try to train a model with nine query heads and three key/value heads on an eight-GPU node at parallelism degree eight: the paper states this cannot be deployed without the head counts dividing evenly, so any successful out-of-box launch would refute the claimed generality, while the headline sequence-length numbers could be checked by re-running the published recipes and observing whether peak GPU memory stays within the 80GB limit.

Watch

Extended reading notes

Core claim

The central claim is that combining three mechanisms — attention-agnostic Ulysses-style sequence parallelism, sequence tiling for per-token operators, and activation-checkpoint offloading to CPU memory — makes multi-million-token training feasible out of the box for open-source Transformer models on standard hardware. With all mechanisms enabled, the paper reports maximum sequence lengths of 500K on one GPU, 3.7M on one 8xH100 node, and over 15M on four nodes for Llama-8B, corresponding to 16x, 116x, and 469x improvements over its 32K baseline. The ablations trace the gains: tiled logits and loss, Ulysses sequence parallelism, tiled MLP, and activation-checkpoint CPU offload each add headroom, with tiling becoming dominant at very long lengths. The correctness experiment shows an almost exact overlap of training loss curves with and without ALST at 32K sequence length, which the paper takes as evidence that the optimizations do not distort training.

Load-bearing premise

The claim that ALST works out of the box for a wide variety of Transformer models rests on the assumption that each target model's attention heads can be split across GPUs under Ulysses sequence parallelism — the number of query heads must be divisible by the parallelism degree, or the key/value heads must be replicable — and that the attention backend is one of the two supported implementations.

Editorial extensions

If this is right

  • Post-training and fine-tuning at multi-million-token contexts become feasible on a single node for open-source models, not just enterprise clusters.
  • Sequence length scales roughly linearly with GPU count, so adding nodes extends context further, up to the limits of CPU memory for activation offload.
  • The matching 32K training-loss curves imply that the memory optimizations can be enabled without sacrificing training quality at comparable sequence lengths.
  • The attention-agnostic design means future sparse or block-local attention mechanisms can be plugged into the same sequence-parallel machinery without rewriting it.
  • Out-of-box compatibility with the widely used Transformers library lowers the barrier for data scientists who want long-context post-training without custom model code.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The headline numbers come from a single iteration with batch size one and substantial CPU offload; sustained multi-step training at 15M tokens will be dominated by wall-clock time and host-memory capacity, which the paper itself notes limits Llama-70B and Qwen3-32B.
  • The head-count divisibility constraint — for example, a model with nine query heads can only use parallelism degree 1, 3, or 9 — means the abstract's 'wide variety of models' claim is currently narrower than it sounds, and removing that constraint is the natural next step for generality.
  • The same tile-along-sequence idea used for logits, loss, and MLP could extend to other per-token operations such as embedding lookups or normalization, yielding further memory savings without additional sequence parallelism.
  • Correctness was validated only at 32K sequence length; if the mechanism is as loss-preserving as claimed, the loss should also match at 3.7M and 15M, but that remains untested in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper presents Arctic Long Sequence Training (ALST), a combination of (1) Ulysses sequence parallelism adapted for Hugging Face Transformers, (2) sequence tiling for logits, loss, and MLP computations, and (3) PyTorch-specific memory optimizations including activation-checkpoint offloading to CPU. The authors report maximum sequence lengths for Llama-8B of 500K tokens on one H100, 3.7M on eight H100s, and 15M on thirty-two H100s, with analogous results for Llama-70B and Qwen3-32B, and they report a feature ablation (Table 1) and a 32K training-loss comparison with a baseline (Figure 13).

Significance. If the reported results hold, ALST is a practically valuable contribution: it brings multi-million-token fine-tuning within reach of users of standard Hugging Face models on commodity GPU clusters, and it is open-sourced. The paper's strengths include the clean feature-ablation study in Table 1, the roughly linear sequence-length scaling across GPU counts, the explicit software versions and reproduction pointers, and the 32K loss-overlap experiment in Figure 13. The main gap is evidence that the multi-million-token runs are actually training correctly rather than merely fitting in memory with a finite loss; this is the load-bearing issue for the headline claims.

major comments (3)
  1. [§5.3, §5.6] The headline multi-million-token 'training' results are supported only by memory-stability checks and the statement in §5.3 that 'several iterations were completed' and the loss was 'valid' (not NaN). The only direct training-quality comparison is Figure 13, which is at 32K sequence length. Because the correctness of the longer runs depends on the pre-shifted-label handling across SP shards (§4.3), the monkey-patched causal-mask suppression (§3.4), and bf16 sequence-parallel communication (§5.2), a run can fit in memory and produce a finite loss while computing a subtly wrong objective (e.g., a dropped or duplicated boundary label, or disabled masking in packed data). Please add a gradient-norm or loss comparison against an unsharded reference at a sequence length where a baseline can run (for example, 500K with gradient accumulation on 8 GPUs), or add a small-scale unit test that verifies the loss and gradients equal an unsharded reference for SP=2/4/8.
  2. [Abstract, §7.1] The abstract and introduction claim out-of-box support for 'a wide variety of HF models,' but the method requires q_heads to be divisible by the SP degree, or kv_heads to be replicable, as stated in §3.2.1 and §7.1. Section 7.1 gives the example of a model with 9 q_heads, for which SP=8 is impossible, and the three evaluated models (Llama-8B, Llama-70B, Qwen3-32B) all have q_heads=32 or 64 and kv_heads=8, which is the favorable case. The generality claim therefore exceeds the demonstrated evidence. Please either narrow the claim in the abstract to models with compatible head counts or provide a fallback mechanism for non-divisible head counts.
  3. [§3.4, §7.2] The paper's reliance on position_ids for correctly attending to sub-samples within a packed sequence depends on the attention backend honoring those position_ids. Section 7.2 states that SDPA in Hugging Face Transformers ignores position_ids and attends to the whole packed sequence, which is both slower and incorrect. Since SDPA is a common default, the 'out-of-box' claim is conditional on the user switching to Flash Attention 2 for packed data. Please make this a first-class limitation near the top of the paper, or provide an SDPA-compatible masking solution.
minor comments (4)
  1. [§5.4] The text says 'Once we hit sequence lengths larger than 5M tiled MLP starts to massively contribute,' but Table 1 and Figure 11 only show sequence lengths up to 3.7M. Please either add the >5M ablation data or rephrase the claim to match the shown results.
  2. [§1] There is a duplicated word in 'supporting 1M and and a whopping 10M sequence length'; please fix this typo.
  3. [§5.6] The sentence 'Thus we know ALST provides the same training quality as the baseline' is stronger than the evidence supports; the comparison is at a single sequence length (32K) and a single model. Please qualify it to 'at 32K for Llama-8B'.
  4. [§3.4] The monkey-patch of _update_causal_mask with 'lambda *args: None' is described in code-like form; consider presenting it as a numbered code listing or pseudocode for clarity, since this is a key correctness-relevant modification.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the sequence-length and loss-match claims are measured outcomes, not outputs of a fitted model or a self-citation chain.

full rationale

This is an empirical systems paper. The headline sequence lengths (500K, 3.7M, 15M) are directly measured maximum lengths that fit in memory and produce finite loss, not quantities derived from fitted parameters or from the paper's own definitions. The training-quality claim rests on a controlled 32K loss comparison against a baseline HF+ZeRO setup with matched data via gradient accumulation, which is an external benchmark rather than a tautology. The memory optimizations (tiling, Ulysses SP, activation offload) are implementation mechanisms supported by memory profiling and ablations; they are not parameters fitted to make the headline numbers come out. Self-citations to DeepSpeed, Ulysses, and ArcticTraining function as provenance and code pointers, not as load-bearing uniqueness theorems or smuggled ansatze. The main weakness is evidentiary: multi-million-token runs were accepted based on finite/non-NaN loss rather than gradient checks, but that is a validation gap, not a circularity. No step in the paper reduces, by construction or by self-citation, to its own inputs.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No fitted parameters in the mathematical sense; the hand-chosen tiling shard size, shard-count rule, and bf16 SP communication are implementation knobs that influence reported sequence lengths. The axioms are the integration and correctness assumptions the system relies on. No invented physical or mathematical entities are introduced; the software components are implemented and open-sourced.

free parameters (3)
  • logits and loss tiling shard budget = 1 GiB
    Hand-picked in Section 3.1 to cap peak logits and loss memory at about 1GiB, dividing the 16K-token, 128K-vocab logits into about 8 chunks. The reported maximum sequence lengths depend on this memory and performance tradeoff.
  • TiledMLP shard count = ceil(seqlen / hidden_size), e.g., 63 shards for seqlen=256K and hidden=4096
    Auto-derived in Section 3.1.1. It directly sets peak MLP activation memory and is a design choice rather than a fitted constant.
  • sequence parallel communication dtype = bf16
    Chosen in Section 5.2 instead of reducing the communication buffer size. It halves all-to-all communication memory and is one of the knobs that affects the reachable sequence length.
assumptions (4)
  • domain assumption Hugging Face attention implementations can be overridden by injecting a wrapper via transformers.modeling_utils.ALL_ATTENTION_FUNCTIONS without changing model correctness.
    Used in Section 4.2 to integrate Ulysses SP into HF models. Correctness is validated only at 32K sequence length.
  • domain assumption FlashAttention2 and SDPA accept a fully recomposed sequence plus position_ids and preserve causal semantics; SDPA ignores position_ids, making packed-sample training incorrect under SDPA.
    Relied on in Sections 3.2 and 7.2. The paper assumes FA2 with position_ids is correct for packed samples but SDPA is not.
  • domain assumption CPU memory and CPU-to-GPU copy bandwidth are sufficient for activation-checkpoint offload without degrading training quality.
    Introduced in Section 3.3. The paper itself notes CPU memory is the limiting factor for Llama-70B and Qwen3-32B at large sequence lengths.
  • domain assumption Dense attention is the only non-tileable quadratic-cost component, so tiling MLP and logits and offloading checkpoints leaves training correctness intact.
    Underlies Section 3.1; the attention block cannot be tiled because it needs the whole sequence to attend over.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Arctic Long Sequence Training: Scalable And Efficient Training For Multi-Million Token Sequences." pith.science (2026). https://pith.science/paper/GDDAEGGY

@misc{pith2026250613996,
  author       = {Pith},
  title        = {Pith review of: Arctic Long Sequence Training: Scalable And Efficient Training For Multi-Million Token Sequences},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GDDAEGGY}},
  note         = {Machine review of arXiv:2506.13996}
}
read the original abstract

Long sequences are critical for applications like RAG, long document summarization, multi-modality, etc., and modern LLMs, like Llama 4 Scout, support max sequence length of up to 10 million tokens. However, outside of enterprise labs, long sequence training is challenging for the AI community with limited system support in the open-source space. Out-of-box, even on a modern NVIDIA H100 80GB GPU cluster, training Llama 8B model with sequence over 32K runs out of memory on a basic Hugging Face (HF) model due to two reasons: i) LLM training workloads are not optimized to fully leverage a single GPU memory, ii) existing solutions for leveraging multiple GPU memory are not easily available to HF models, making long sequence training inaccessible. We address this with Arctic Long Sequence Training (ALST). It offers a combination of attention-agnostic single GPU and multi-GPU memory optimizations, that enables it to support out-of-box training of multi-million sequence length for a wide variety of HF models. ALST supports training Meta's Llama 8B model with 500K sequence length on a single H100 GPU, 3.7M on a single 8xH100 GPU node, and over 15M on a 4 node cluster, an increase of over 400x compared to the 32K baseline for the latter. ALST is fully compatible with HF models and open-sourced via Deepspeed https://www.deepspeed.ai/tutorials/ulysses-alst-sequence-pallellism/ and Arctic Training https://github.com/snowflakedb/ArcticTraining/blob/main/projects/sequence-parallelism/README.md.

Figures

Figures reproduced from arXiv: 2506.13996 by the authors.

Figure 1
Figure 1. A dramatic improvement in sequence length with ALST enabled on 1, 8 and 32 H100 GPUs with Llama-8B. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Estimated memory usage for Llama-8B activation memory with different sequence lengths. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. PyTorch memory usage plots before (left) and after (right) using Sequence Tiling to reduce loss calculation [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: 1x forward-backward cycle on a single Llama-8B LlamaMLP layer (Left) without tiling (Right) with tiling. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Ulysses Sequence Parallelism diagram with 4 attention heads per attention block model. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: MHA, GQA and MQA types of model attention heads (source [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: PyTorch memory profiler 1 iteration forward-backward CUDA memory usage: Left: normal setup. Right: [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Maximum achieved sequence length with meta-llama/Llama-3.1-8B-Instruct [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Maximum achieved sequence length with meta-llama/Llama-3.1-70B-Instruct [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Maximum achieved sequence length with Qwen/Qwen3-32B [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: Feature ablation results visualized [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]
Figure 12
Figure 12. Figure 12: The impact of enabling ALST for LLama-8B on 1, 8 and 32 GPUs [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]
Figure 13
Figure 13. Figure 13: Training loss comparison with and without ALST [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Untied Ulysses: Memory-Efficient Context Parallelism via Headwise Chunking

    cs.LG 2026-02 conditional novelty 6.0 of 10

    UPipe chunks attention by head so QKV and all-to-all buffers scale with a small tunable chunk size rather than head count, enabling 5M-token Llama3-8B training on one 8×H100 node with throughput close to Ulysses.

Reference graph

Works this paper leans on

17 extracted references · 3 canonical work pages · cited by 1 Pith paper

  1. [1]

    Ring attention with blockwise transformers for near-infinite context,

    H. Liu, M. Zaharia, and P. Abbeel, “Ring attention with blockwise transformers for near-infinite context,” 2023. [Online]. Available: https://arxiv.org/abs/2310.01889

    arXiv 2023

  2. [2]

    Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models,

    S. A. Jacobs, M. Tanaka, C. Zhang, M. Zhang, S. L. Song, S. Rajbhandari, and Y . He, “Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models,”arXiv preprint arXiv:2309.14509, 2023. [Online]. Available: https://arxiv.org/abs/2309.14509

    arXiv 2023

  3. [3]

    Reducing activation recomputation in large transformer models,

    V . Korthikanti, J. Casper, S. Lym, L. McAfee, M. Andersch, M. Shoeybi, and B. Catanzaro, “Reducing activation recomputation in large transformer models,” 2022. [Online]. Available: https://arxiv.org/abs/2205.05198

    arXiv 2022

  4. [4]

    Sequence parallelism: Long sequence training from system perspective,

    S. Li, F. Xue, C. Baranwal, Y . Li, and Y . You, “Sequence parallelism: Long sequence training from system perspective,” 2022. [Online]. Available: https://arxiv.org/abs/2105.13120

    arXiv 2022

  5. [5]

    Distflashattn: Distributed memory-efficient attention for long-context llms training,

    D. Li, R. Shao, A. Xie, E. P. Xing, X. Ma, I. Stoica, J. E. Gonzalez, and H. Zhang, “Distflashattn: Distributed memory-efficient attention for long-context llms training,” 2024. [Online]. Available: https://arxiv.org/abs/2310.03294

    arXiv 2024

  6. [6]

    Striped attention: Faster ring attention for causal transformers,

    W. Brandon, A. Nrusimha, K. Qian, Z. Ankner, T. Jin, Z. Song, and J. Ragan-Kelley, “Striped attention: Faster ring attention for causal transformers,” 2023. [Online]. Available: https://arxiv.org/abs/2311.09431

    arXiv 2023

  7. [7]

    Flashattention-2: Faster attention with better parallelism and work partitioning,

    T. Dao, “Flashattention-2: Faster attention with better parallelism and work partitioning,” 2023. [Online]. Available: https://arxiv.org/abs/2307.08691

    arXiv 2023

  8. [8]

    Usp: A unified sequence parallelism approach for long context generative ai,

    J. Fang and S. Zhao, “Usp: A unified sequence parallelism approach for long context generative ai,” 2024. [Online]. Available: https://arxiv.org/abs/2405.07719

    arXiv 2024

Show all 17 references
  1. [9]

    Loongtrain: Efficient training of long-sequence llms with head-context parallelism,

    D. Gu, P. Sun, Q. Hu, T. Huang, X. Chen, Y . Xiong, G. Wang, Q. Chen, S. Zhao, J. Fang, Y . Wen, T. Zhang, X. Jin, and X. Liu, “Loongtrain: Efficient training of long-sequence llms with head-context parallelism,” 2024. [Online]. Available: https://arxiv.org/abs/2406.18485

    2024 arXiv

  2. [10]

    Liger kernel: Efficient triton kernels for llm training,

    P.-L. Hsu, Y . Dai, V . Kothapalli, Q. Song, S. Tang, S. Zhu, S. Shimizu, S. Sahni, H. Ning, and Y . Chen, “Liger kernel: Efficient triton kernels for llm training,” 2025. [Online]. Available: https://arxiv.org/abs/2410.10989

    2025 arXiv

  3. [11]

    Datasets: A community library for natural language processing,

    Q. Lhoest, A. Villanova del Moral, Y . Jernite, A. Thakur, P. von Platen, S. Patil, J. Chaumond, M. Drame, J. Plu, L. Tunstall, J. Davison, M. Šaško, G. Chhablani, B. Malik, S. Brandeis, T. Le Scao, V . Sanh, C. Xu, N. Patry, A. McMillan-Major, P. Schmid, S. Gugger, C. Delangu...

    2021

  4. [12]

    ArcticTraining: Simplifying and accelerating post-training for large language models,

    Snowflake AI Research, “ArcticTraining: Simplifying and accelerating post-training for large language models,” https://github.com/snowflakedb/ArcticTraining, 2025, versionv0.0.4 (released June 3, 2025); Apache-2.0 license

    2025

  5. [13]

    Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters,

    J. Rasley, S. Rajbhandari, O. Ruwase, and Y . He, “Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters,” inProceedings of the 26th ACM SIGKDD International Conference 18 Arctic Long Sequence Training (ALST) on Knowledge Discove...

    2020

  6. [14]

    Transformers: State-of-the-art natural language processing,

    T. Wolf, L. Debut, V . Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y . Jernite, J. Plu, C. Xu, T. L. Scao, S. Gugger, M. Drame, Q. Lhoest, and A. M. Rush, “Transformers: State-of-the-art nat...

    2020

  7. [15]

    Zero: Memory optimizations toward training trillion parameter models,

    S. Rajbhandari, J. Rasley, O. Ruwase, and Y . He, “Zero: Memory optimizations toward training trillion parameter models,” 2020. [Online]. Available: https://arxiv.org/abs/1910.02054

    2020 arXiv

  8. [16]

    Bloom: A 176b-parameter open-access multilingual language model,

    B. Workshop, :, T. L. Scao, A. Fan, C. Akiki, E. Pavlick, S. Ili ´c, D. Hesslow, R. Castagné, A. S. Luccioni, F. Yvon, M. Gallé, J. Tow, A. M. Rush, S. Biderman, A. Webson, P. S. Ammanamanchi, T. Wang, B. Sagot, N. Muennighoff, A. V . del Moral, O. Ruwase, R. Bawden, S. Bekman...

    2023 arXiv

  9. [17]

    The case for co-designing model architectures with hardware,

    Q. Anthony, J. Hatef, D. Narayanan, S. Biderman, S. Bekman, J. Yin, A. Shafi, H. Subramoni, and D. Panda, “The case for co-designing model architectures with hardware,” 2024. [Online]. Available: https://arxiv.org/abs/2401.14489 19

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.