Pith. sign in

REVIEW 4 major objections 4 minor 128 references

Full-parameter post-training of trillion-parameter Mixture-of-Experts models is feasible on Ascend NPU clusters, and a solver-grounded CPT-SFT workflow lifts operations-research accuracy above stronger models.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 10:37 UTC pith:IAWCD73N

load-bearing objection Solid Ascend systems work; the OR benchmark claims need a held-out re-evaluation before they can be trusted. the 4 major comments →

arxiv 2607.20145 v2 pith:IAWCD73N submitted 2026-07-22 cs.CL cs.AI

SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD

Dongfang Li , Xiaodong Luo , Ruoyu Sun , Xuhui Chen , Linyuan Qiu , Jian Meng , Zhengxuan Lu , Yiting Wang
show 57 more authors
Yucheng Xie Tao Guo Tianxiang Fang Jing Li Sihang Chen Shihao Hong Chang Liu Weihua Dai Zirong Zeng Ziwei Zhu Zhuohan Wang Zhengjun Yue Igor Vasilyev Min Liu Weijian Sun Xin Chen Yingmeng Gao Jinhua Zhou Taolue Chen Chenwei Wu Dong Zhang Wenlong Jin Jinmin Xiang Barkova Maria Ushakov Anton Xianfei Jin Tian Ding Zhihang Lin Qian Chen Linxin Yang Mingzhe Yang Bingwei Zhang Hongzhang Yang Fangxue Zhang Shijun Qin Jie Yu Cuihua Hu Tolstykh Vasiliy Nosov Ivan Abdullin Amir Zhicheng Zhou Xin Zhang Zhixiong Ning Xutong Zhao Junjie Huang Jiajun Liu Weiyan Kong Zheng Zhang Wenhan Luo Lin Hu Yangbo Guo Li Zeng Shihao Zhang Baotian Hu Min Zhang Haizhou Li Zhiquan Luo
This is my paper
classification cs.CL cs.AI
keywords mixture-of-expertsascend npumodel flops utilizationkernel optimizationcontinued pre-trainingsupervised fine-tuningoperations researchsolver-verified data
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This report argues that two interventions work together: (1) a full-stack Ascend system optimization that raises MFU from 11.67% to 34.22% on DeepSeek-V4-Pro (2.93x over the baseline), and (2) a solver-verified CPT-SFT data pipeline that, on DeepSeek-V4-Flash, beats GPT-5.4-Mini by 3.98 average points on four OR benchmarks. Applying the same workflow to the trillion-parameter Pro model raises its average OR score from 70.16% to 77.33%. The key claim is that domain post-training benefits most from structural priors injected via CPT and behavior alignment via SFT, and that both are practical on non-GPU hardware.

Core claim

The central discovery is that system optimizations and solver-grounded data quality reinforce each other: the optimized Ascend training stack runs full-parameter MoE training at 34.22% MFU, while the CPT-SFT workflow—where CPT is initialized on a solver-verified OR corpus and SFT uses cleaned, CoT-enhanced self-distilled samples—produces strong OR specialization. The largest CPT gains appear on structural equivalence, suggesting that continued pretraining supplies modeling priors that instruction tuning alone cannot.

What carries the argument

The load-bearing mechanisms are: (a) a hierarchical Ascend training framework (expert–tensor parallelism, overlapped MoE dispatch, double-buffered swap optimizer, VPP pipeline) plus an OR-guided kernel autotuning agent; and (b) a solver-verified data engine that generates optimization instances, verifies them with Gurobi, renders them into natural-language problems, and re-verifies forward-modeled answers, feeding CPT and SFT with 10K verified samples.

Load-bearing premise

The load-bearing premise is that the OR benchmarks used to select the CPT checkpoint and guide data cleaning are effectively held out; the paper does not demonstrate a disjoint validation split because the final checkpoint was selected to produce positive transfer on the primary OR benchmarks and error analysis on the same benchmarks guided the cleaning.

What would settle it

Re-run the pipeline using a strictly held-out validation set of solver-generated OR problems (never used for checkpoint selection or cleaning), then compare zero-shot Pass@1 on the public benchmarks; if the margin over GPT-5.4-Mini shrinks below a substantial threshold, the headline result is partly benchmark-adaptive rather than capability-level.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Trillion-parameter MoE post-training can run on NPU clusters at an MFU (34.22%) that makes full-parameter training practical.
  • Domain-adaptive CPT provides the largest measured gain on structural-equivalence benchmarks, indicating that modeling priors matter beyond instruction following.
  • Data verification and repair outperform raw scaling: 10K cleaned samples beat 50K uncleaned samples on natural-language modeling benchmarks.
  • The workflow transfers across scales: DeepSeek-V4-Pro's OR score rises from 70.16% to 77.33% without broad general-capability collapse.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the CPT checkpoint and cleaning design were guided by the same OR benchmarks used in final evaluation, the reported scores are likely optimistic under a strict held-out protocol; a pre-registered validation split would clarify true generalization.
  • The OR-guided tiling optimization inside the kernel autotuner could apply to other memory-bound accelerators, since it decouples tiling from heuristics and relies on cycle-level cost models.
  • The non-monotonic scaling of self-distilled data suggests that synthetic-data pipelines need verifier-in-the-loop as volume grows; reinforcement learning from solver feedback might push the recipe further.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper presents SLAI T-Rex, a full-stack framework for post-training DeepSeek-V4-family models on Ascend CloudMatrix384 SuperPOD with Ascend 910C NPUs. It combines system-level optimizations (load-balanced pipeline scheduling, expert–tensor parallelism, overlapped MoE dispatch, double-buffered swap optimizer, and two kernel-optimization paths: AuraKernel and AscendC fusion/rewrite) with an OR-oriented CPT+SFT workflow built from solver-verified synthetic data and self-distilled SFT samples. The main claims are: (i) DeepSeek-V4-Pro training MFU improves from 11.67% to 34.22% (2.93×); (ii) the specialized DeepSeek-V4-Flash model achieves an average OR score of 71.81% across NL4OPT, OptiBench, B4O-Feasible, and B4O-ORGEval, exceeding GPT-5.4-Mini by 3.98 points; (iii) applying the workflow to DeepSeek-V4-Pro raises its average OR score from 70.16% to 77.33%. The paper releases prompts, contracts, curriculum schedules, and monitoring callbacks, and plans to release the optimized AscendC kernels.

Significance. If the reported results hold, the system-level contribution is significant: it is one of the few detailed demonstrations of full-parameter post-training of a trillion-parameter MoE model on non-GPU SIMD hardware, with extensive profiling, a concrete bottleneck taxonomy, and a verifiable kernel-optimization loop. The AuraKernel design — OR-guided tiling, correctness-gated candidate acceptance, and skill distillation — is a credible advance over purely heuristic kernel tuning, and the paper is unusually transparent about measurement conditions and reproducibility artifacts. The OR-specialization results are interesting and the matched CPT-to-SFT comparison (Section 4.5) is a well-designed experiment in isolation. However, as detailed in the major comments, the headline OR numbers are not demonstrated to be independent of the training and selection loop, which currently undermines the paper's central empirical claim.

major comments (4)
  1. [§3.2.1 and §4.6] OptiBench appears both as a collected CPT resource and as an evaluation benchmark. §3.2.1 states the collected corpus includes 'ReSocratic and OptiBench (Yang et al., 2025c)', while §4.6 and Table 15 report OptiBench as a held-out test benchmark. No split, exclusion, or deduplication of OptiBench instances from the CPT corpus is described. Consequently, the reported OptiBench scores, which contribute to the overall 71.81% average and the +3.98 vs. GPT-5.4-Mini claim, are potentially inflated by train/test overlap. The paper must either remove OptiBench from the training mixture and re-evaluate, or demonstrate instance-level exclusion with a concrete audit trail.
  2. [§3.2.3 and Tables 12/13/15] The final CPT checkpoint is explicitly selected to 'produce positive CPT-to-SFT transfer on the primary OR benchmarks' — the same NL4OPT/OptiBench/B4O numbers later reported as results. This is selection on the test set. The paper does not report a separate validation split that was used for checkpoint selection and then held out from the final evaluation. Without such a split, the reported CPT+SFT gains (e.g., +10.66 pp on B4O-ORGEval in Table 12) are not unbiased estimates of the pipeline's effect. A pre-registered or clearly separated validation protocol is required before the OR claims can be accepted.
  3. [§4.4 and §3.3.3] The cleaning and CoT design decisions are guided by error analysis on the same benchmarks that are later reported as final results. For example, §4.4 states that Clean-CoT is chosen because it improves NL4OPT and OptiBench, and §3.3.3 describes cleaning targets driven by failures on these benchmarks. This is an adaptive data-cleaning loop fitted to the test set. Unless the paper can show that the final reported scores were obtained from a locked pipeline on data never used for any development decision, the gains from cleaning (Table 11) and the headline end-to-end numbers are subject to selection bias. The paper should report results on a truly held-out OR benchmark set or apply a rigorous nested evaluation protocol.
  4. [§4.6, Table 15] The comparison against GPT-5.4-Mini, Gemini-3-Flash, Kimi-K2.6, and other baselines is central to the 'highest average score' claim. The text says 'All evaluations share the same prompt template and decoding budget for fair comparison,' but it is not stated whether the external model numbers were obtained by the authors under this shared protocol or taken from published reports with potentially different prompts, decoding settings, or answer extraction. If they were not run under identical conditions, the comparison is not controlled. Please specify the evaluation provenance for each external model and, if available, provide the pass rates under the same protocol.
minor comments (4)
  1. [§2.3.2] The sentence beginning 'Admittedly the' at the end of the AuraKernel description is incomplete and appears to cut off before the next paragraph. Either complete the thought or remove the fragment.
  2. [§2 and §4.1–4.3] The MFU, kernel speedup, and training-stability numbers are self-reported with no measurement uncertainty or run-to-run variation. Given the reproducibility artifacts promised, adding standard deviations or at least stating the number of repeated measurements would strengthen the system claims.
  3. [§4.7, Table 17] HumanEval is reported as 100.00% for both DeepSeek-V4-Pro and DeepSeek-V4-Pro+SFT, which is an unusually perfect score. A brief note on the evaluation setup (e.g., number of problems, pass@1 vs. pass@k, any known benchmark contamination) would help the reader calibrate this result.
  4. [§3.2.1] The 'separation between the data-generation assets and the ORGEval evaluation assets' is described only for ORGEval. The same level of separation should be documented for NL4OPT, OptiBench, and B4O-Feasible, since those benchmarks also appear in development decisions.

Circularity Check

3 steps flagged

OR benchmark gains are fitted to the same benchmarks used for evaluation: final CPT checkpoint selection, cleaning/CoT choices, and general-anchoring decisions all use NL4OPT/OptiBench/B4O scores that are later reported as held-out results, and OptiBench appears in the CPT corpus.

specific steps
  1. fitted input called prediction [§3.2.3 (Checkpoint Selection and CPT-to-SFT Transfer Validation), used in §4.6/§4.7]
    "The final checkpoint is selected with a multi-objective criterion. A preferred checkpoint should reduce OR validation loss, maintain stable optimization signals during training, preserve general capabilities within the expected range, and produce positive CPT-to-SFT transfer on the primary OR benchmarks."

    The 'primary OR benchmarks' named in §3.2.3 and evaluated in §4.6 are NL4OPT, OptiBench, B4O-Feasible, and B4O-ORGEval — exactly the four benchmarks whose averages (71.81% Flash; 77.33% Pro) are reported as the paper's headline OR results. Thus the reported scores are not independent predictions of CPT/SFT benefit: the checkpoint was chosen to maximize a criterion that includes those same scores. The +10.66 pp on B4O-ORGEval and the other Table 13/15 gains are partly the selection objective itself. The paper does not demonstrate that these downstream benchmarks are a held-out split distinct from the selection set.

  2. other [§3.2.1 (OR-CPT data mixture) vs §4.6 (Evaluation Setup)]
    "The complete CPT corpus combines these solver-verified synthetic documents with collected OR-domain resources. The collected portion covers optimization modeling, mathematical reasoning, and solver-oriented problem solving, including OptMATH (Lu et al., 2025), ReSocratic and OptiBench (Yang et al., 2025c), Text2Opt-Bench (Gao et al., 2026), OR-Instruct (Huang et al., 2025), NLP4LP (Ahmaditeshnizi et al., 2024)..."

    In §4.6, OptiBench is one of the four test benchmarks whose scores are reported (e.g., 63.33→67.12 in matched transfer). The paper never describes splitting or excluding OptiBench instances from the CPT training pool; the only explicit separation asserted is from ORGEval reference formulations and evaluator templates, not from OptiBench test items. Consequently the OptiBench improvement attributed to CPT is not clean evidence of domain generalization — it may reflect train/test overlap, making the 'CPT supplies OR-domain priors' claim partly circular with the composition of the CPT corpus.

  3. fitted input called prediction [§3.3.3 (Contract-Aware Cleaning and CoT Enhancement) and §4.4 (SFT Scaling and Cleaning Analysis)]
    "The observations from preliminary SFT runs motivate a shift from merely enlarging the self-distilled dataset to improving the learnability of already distilled samples. In the comparison among 3K, 10K, and 50K reported in Section 4.4, the 10K dataset already significantly improves Bench4Opt protocol-oriented tasks, but it remains below the baseline on NL4OPT and OptiBench. Further scaling to 50K does not recover this ability. Therefore, ... we perform fine-grained cleaning of the existing 10K data."

    The cleaning/CoT branch and the general-capability anchoring in §3.3.4 were chosen by examining performance and error types on NL4OPT and OptiBench (e.g., 'On NL4OPT, Clean-CoT reduces wrong_objective_or_model errors from 52 cases before cleaning to 31 cases'). The same benchmarks are then used in Tables 11–13 to validate Clean-CoT and the overall pipeline. This is a selection loop: the design was fitted to the evaluation set, and the reported NL4OPT/OptiBench gains are the return on that test-set information rather than an independent test of the cleaning hypothesis.

full rationale

The system-level contribution is not circular: the 34.22% MFU and 2.93× improvement are backed by measured end-to-end training on Ascend SuperPOD, and while the AuraKernel cost-model constants are fitted to hardware, the final operator speedups are measured on real NPUs rather than derived from the fitted model. The circularity is confined to the OR-specialization half. The final CPT checkpoint is explicitly selected to 'produce positive CPT-to-SFT transfer on the primary OR benchmarks', which are the same NL4OPT/OptiBench/B4O-Feasible/B4O-ORGEval numbers later reported as the result. Cleaning/CoT and general-capability anchoring decisions are likewise made from error analysis on NL4OPT and OptiBench, and OptiBench is listed inside the CPT corpus while also serving as a test benchmark, with no split or exclusion described. The paper therefore does not demonstrate that the reported OR gains are independent held-out evaluations: they are partly fitted to the evaluation set and possibly contaminated by train/test overlap. This is a partial circularity of the 'fitted input called prediction' kind, not a definitional identity, so the system MFU results and the general-capability checks keep the paper from being fully circular.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The central OR claims depend on several unverifiable assumptions: benchmark validity, data-leakage control, and fair comparison with external models. The most consequential free parameters are the benchmark-guided checkpoint selection and data-cleaning decisions, which are not independent of the evaluation targets.

free parameters (3)
  • AuraKernel cycle-cost constants (Eq. 1) = not reported
    Per-storage-level MTE1/MTE2/MAC/FixPipe cycle counts and L2 cache-hit model are fitted to Ascend 910C measurements and used to choose tiling strategies; they influence reported kernel speedups.
  • CPT data mixture allocation = ~75% OR + retained general corpora (reported in §4.6)
    The exact sampling allocation is called a recipe-level parameter (§3.2.1); the mixture was validated/adjusted using downstream OR performance.
  • SFT recipe hyperparameters = Stage I LR 5e-7, Stage II LR 1e-6, 220 steps, GBS 128, SEQ_LEN 8192
    These training choices were selected through experiments on OR benchmarks and affect all reported OR scores.
axioms (5)
  • domain assumption Ascend 910C profiling counters and the MFU baseline recipe accurately reflect device utilization and fair comparison.
    The 11.67% baseline and 34.22% optimized MFU rest on the correctness of the profiling tool and the open-source baseline recipe; no independent measurement is provided.
  • domain assumption The four OR benchmarks (NL4OPT, OptiBench, B4O-Feasible, B4O-ORGEval) are valid, sufficient measures of OR modeling capability.
    The paper uses the unweighted mean of these benchmarks as the overall OR score; if the benchmarks are unrepresentative or flawed, the headline gains lose meaning.
  • domain assumption The synthetic data pipeline's leakage checks prevent overlap with evaluation benchmarks.
    The paper asserts separation between data-generation assets and ORGEval assets and describes leakage controls, but provides no auditable artifact or external verification.
  • domain assumption DeepSeek-V4-Flash is representative of the V4 family so that findings transfer to DeepSeek-V4-Pro.
    The workflow is developed on Flash and then applied to Pro; the paper does not ablate the Pro CPT separately, so scale-up inference relies on this transferability assumption.
  • domain assumption The external baselines (GPT-5.4-Mini, Gemini-3-Flash, etc.) were evaluated under identical prompting and decoding settings.
    Table 15 reports comparisons to other models, but the evaluation harness is not released and the claim of identical protocol cannot be independently checked.

pith-pipeline@v1.3.0-alltime-deepseek · 48633 in / 9407 out tokens · 91166 ms · 2026-08-01T10:37:52.039408+00:00 · methodology

0 comments
read the original abstract

Full-parameter post-training of trillion-parameter-scale MoE models introduces substantial system-level challenges for large-scale distributed training, including severe memory pressure, non-overlapped communication overhead, and inefficient kernel execution. While most large-scale LLM training systems are built around GPU-based clusters, this report presents an end-to-end optimization practice on the Ascend NPU SuperPOD. Using the DeepSeek-V4 model family as the target workload, we develop a hierarchical optimization framework spanning model-level parallelism, computation-communication orchestration, and low-level kernel execution. The resulting system achieves 34.22% Model FLOPs Utilization (MFU) with a 2.93x improvement over the open-source baseline recipe while maintaining training stability. Building on this optimized infrastructure, we further establish a CPT and SFT workflow for complex Operations Research (OR) tasks. We refer to the integrated framework as SLAI T-Rex. Using DeepSeek-V4-Flash, we develop OR-oriented CPT and SFT data pipelines that combine collected domain resources with solver-verified synthetic optimization documents. The resulting dataset contains 10K high-quality SFT samples spanning four task categories and three problem representations. The specialized model achieves the highest average zero-shot Pass@1 score among the evaluated models, reaching 71.81% and outperforming GPT-5.4-Mini and the base DeepSeek-V4-Flash model by 3.98 and 11.27 percentage points, respectively. Overall, this work demonstrates a full-stack pathway from efficient trillion-parameter model post-training on Ascend infra to domain-specialized Flash models for solver-grounded mathematical modeling, advancing frontier-model systems for complex reasoning.

Figures

Figures reproduced from arXiv: 2607.20145 by Abdullin Amir, Baotian Hu, Barkova Maria, Bingwei Zhang, Chang Liu, Chenwei Wu, Cuihua Hu, Dongfang Li, Dong Zhang, Fangxue Zhang, Haizhou Li, Hongzhang Yang, Igor Vasilyev, Jiajun Liu, Jian Meng, Jie Yu, Jing Li, Jinhua Zhou, Jinmin Xiang, Junjie Huang, Lin Hu, Linxin Yang, Linyuan Qiu, Li Zeng, Mingzhe Yang, Min Liu, Min Zhang, Nosov Ivan, Qian Chen, Ruoyu Sun, Shihao Hong, Shihao Zhang, Shijun Qin, Sihang Chen, Tao Guo, Taolue Chen, Tian Ding, Tianxiang Fang, Tolstykh Vasiliy, Ushakov Anton, Weihua Dai, Weijian Sun, Weiyan Kong, Wenhan Luo, Wenlong Jin, Xianfei Jin, Xiaodong Luo, Xin Chen, Xin Zhang, Xuhui Chen, Xutong Zhao, Yangbo Guo, Yingmeng Gao, Yiting Wang, Yucheng Xie, Zhengjun Yue, Zhengxuan Lu, Zheng Zhang, Zhicheng Zhou, Zhihang Lin, Zhiquan Luo, Zhixiong Ning, Zhuohan Wang, Zirong Zeng, Ziwei Zhu.

Figure 1
Figure 1. Figure 1: (a) MFU improvements during DeepSeek-V4-Pro (1.6T) training on Ascend SuperPOD. Starting from open-sourced recipes, optimized parallelism, NPU-CPU co-optimization, and ker￾nel optimization increase MFU from 11.67% to 34.22%. (b) Overall accuracy, computed as the mean of NL4OPT (Ramamonjison et al., 2022), OptiBench (Yang et al., 2025c), Bench4Opt (B4O) -Feasible (Wang et al., 2025b), and Bench4Opt (B4O)-OR… view at source ↗
Figure 2
Figure 2. Figure 2: Latency breakdown of training trillion-parameter-class LLM model on Ascend SuperPOD NPU cluster. (a) Partition of latency in the end-to-end LLM training pipeline. (b) Averaged end-to￾end latency breakdown, including computation, communication, and NPU idle bubble. 6 [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the proposed full-stack optimization framework, which includes three cooperating layers: parallel execution strategy, kernel optimization, and hardware-affinity tuning. strategy based on dedicated kernel fusion to suppress launch overhead and eliminate redundant memory traffic, as presented in Section 2.3.3. 2.2 Optimizing Trillion-parameter LLM Training System on Ascend SuperPOD Motivated by t… view at source ↗
Figure 4
Figure 4. Figure 4: Communication pattern of the ETP token dispatcher (ETP2-EP2 example). Dispatch and combine wrap the expert grouped-GEMM with an EP all-to-all (token routing) and an ETP all-gather/reduce-scatter over the hidden dimension (Gale et al., 2023). With MoE Parallel Folding, ETP folds into EP as a single group, so one all-to-all performs dispatch and the separate ETP collectives are eliminated. doubling the resid… view at source ↗
Figure 5
Figure 5. Figure 5: Compute–communication schedule of the MoE token dispatcher. Issuing the shared-expert forward ahead of the before_ep_alltoall synchronization fills the exposed sync stall with useful computation while communication overlap is preserved. dynamic split sizes introduces a visible before_ep_alltoall synchronization stall on the device timeline. The shared expert, however, depends only on the MoE-layer input an… view at source ↗
Figure 6
Figure 6. Figure 6: Bottleneck anatomy of one representative DeepSeek-V4-Pro training step on Ascend 910C (206,503 compute kernels, 39.84 s device-side duration). Panel (a): the heavy head of architecture￾intrinsic operators by core-busy share (the tool’s “Task Duration” column)—sparse-attention back￾ward, dense matrix multiplication, and grouped MoE multiplication. Panel (b): the complementary launch-fragmentation view, cont… view at source ↗
Figure 7
Figure 7. Figure 7: The architecture of AuraKernel. Starting from a functionally correct baseline kernel, AuraKernel generates an optimized operator in three stages. (1) OR-based tiling optimization diagnoses the operator’s dominant regime and instantiates a hardware-aware parameter space, yielding an initial configuration and a set of promising optimization directions. (2) Hardware￾grounded iterative optimization runs a clos… view at source ↗
Figure 8
Figure 8. Figure 8: Operator-chain fusion atlas. Rows show four DeepSeek-V4-Pro rewrites: mHC weighted expansion, mHC mixed-precision dataflow, limited SwiGLU backward (with routing-weight gra￾dient) and unified RoPE. Columns give the emitted trace, semantic invariant, removed frontend artifacts and fused AscendC boundary. mHC fusion suite. The Manifold-Constrained Hyper-Connections (mHC) path (Xie et al., 2025) is a larger i… view at source ↗
Figure 9
Figure 9. Figure 9: Three mechanisms of the mHC weighted-expansion fusion, detailed in the text. (a) The weighted expansion bmm1[𝑡, 𝑛, 𝑑] = 𝑥 [𝑡, 𝑑] ℎ[𝑡, 𝑛] over the 𝑁 mixture channels, accumulated and emitted as BF16 in one UB-resident kernel. (b) The two-dimensional [𝑇, 𝑁, 𝐷] tiling and the three￾stage MTE2/VEC/MTE3 pipeline, where the dashed line marks the instant at which all three engines are busy. (c) The pre-projection… view at source ↗
Figure 10
Figure 10. Figure 10: Schematic of the limited-SwiGLU backward chain collapsing into one fused kernel. The eager row shows the operator types emitted by autograd: SelectV2, Cast, ConcatD, SwiGluGrad, a trailing group of comparison, mask, and fill kernels (LessEqual, GreaterEqual, LogicalAnd, Fill), and, in the MoE path, the routing-weight-gradient reduction (Mul and ReduceSum), with tile widths proportional to their per-type t… view at source ↗
Figure 11
Figure 11. Figure 11: Three mechanisms of the unified rotary-embedding operator. (a) The eager in-place slice assignment and the full-width backward scatter it forces, contrasted with the whole-row output that avoids it. (b) The real-arithmetic realization of the complex rotation, showing the cosine multiply, adjacent-pair swap, and sign-flipped sine that produce the rotated pair. (c) The per-step cosine/sine preparation chain… view at source ↗
Figure 12
Figure 12. Figure 12: Stacked error-family counts of the original checkpoint under Pass@1, 5-shot, and Pass@16. The bar heights show total failed cases, while the colored segments show how the failure composition changes. 5-shot sharply reduces protocol/API/schema failures; Pass@16 reduces total failures but leaves a large structural ORGEval segment. modeling priors, while keeping the representative cases and training implicat… view at source ↗
Figure 13
Figure 13. Figure 13: Pipeline of OR-CPT data construction. Parameterized optimization generators produce structured instances, which are verified with Gurobi, rendered into business-oriented problem statements, reconstructed into executable formulations, and filtered by contract checks, solver execution, and objective-value agreement before export. canonical LP graph, or evaluator-side template is reused when rendering CPT do… view at source ↗
Figure 14
Figure 14. Figure 14: Overview of the SFT data distillation framework for OR modeling. The pipeline converts high-quality seed samples into canonical and semantic intermediate representations, generates novel synthetic IRs, renders them into multiple problem formats, validates the corresponding answers, and feeds accepted synthetic IRs back into the next self-distillation round. levels, including surface-level problem wording … view at source ↗
Figure 15
Figure 15. Figure 15: The AI-Assisted Quality Gates workflow. This multi-stage validation process uses the Cleaner, Reviewer, and code execution Validation to ensure the quality and auditability of the final dataset. 3.4 AI-Assisted Data Construction and Refinement To scale up the data construction process described above, we employ AI-assisted tools to automate candidate generation, repair, and validation. As illustrated in … view at source ↗
Figure 16
Figure 16. Figure 16: AuraKernel optimization of the shared-key-value sparse-attention kernels on Ascend 910C. (a) Per-call forward task duration (1.23× at compression ratio 128, 1.09× at compression ratio 4). (b) Per-call backward task duration of SparseAttnSharedkvGrad (1.24× and 1.21×). (c) AIV pipeline-time decomposition of the backward kernel at compression ratio 4: the reductions concentrate in the memory-movement pipes … view at source ↗
Figure 17
Figure 17. Figure 17: AuraKernel optimization of the weight-free RMS-normalization kernels (𝑡 𝑝=2). (a) Per-call task duration speedup: forward 11.9× and backward 3.4×. (b,c) Duration-weighted AIV pipeline ratios before and after optimization: the vector-compute ratio rises (forward 27.8% → 67.5%, backward 17.8% → 32.0%) while the scalar ratio falls (forward 31.5% → 17.2%, backward 34.8% → 18.0%), showing that 2D head blocking… view at source ↗
Figure 18
Figure 18. Figure 18: AuraKernel optimization of the sparse lightning-indexer gradient on Ascend 910C. (a) Per-call backward task duration speedup, 1.16× on the full CFA path (cmp. ratio 0) and 1.21× on the SCFA path (cmp. ratio 4). (b,c) AIV pipeline-time decomposition before and after optimization at the two ratios: the kernel is scalar- and load-bound, and the reductions concentrate in the scalar and MTE2-load pipes rather … view at source ↗
Figure 19
Figure 19. Figure 19: Module task duration before (left bar) and after (right bar) operator fusion and rewrite for the seven targets, decomposed by operator category. Category durations are re-aggregated from the per-operator trace times of the pre- and post-optimization profiles. Each bar pair represents kernel￾module durations ( [PITH_FULL_IMAGE:figures/full_fig_p039_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Training stability of the 50K self-distilled SFT run on Ascend 910C NPUs. The model is trained for 800 steps with GBS=128, MBS=1, and SEQ_LEN=8192, and the training process reaches the final iteration without skipped or NaN iterations. baseline is the emitted trace slice, stage subgraph, or generic Triton kernel rather than an isolated primitive operator. 4.3 Long-term Training Stability with Ascend Super… view at source ↗
Figure 21
Figure 21. Figure 21: Ablation on data mixture under the end-to-end (CPT→SFT) pipeline [PITH_FULL_IMAGE:figures/full_fig_p045_21.png] view at source ↗
Figure 22
Figure 22. Figure 22: The trust path of a synthesized OR-CPT document. C.3 Mathematical seeds and domain contracts Seed representation. A seed is a structured optimization instance, not merely a textual question. It contains declared sets, coefficient tables, objective direction, variable domains, bounds, constraint families, and executable or LP artifacts. It also carries semantic metadata such as task family, sub￾63 [PITH_F… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

128 extracted references · 2 canonical work pages

  1. [1]

    arXiv , author =:1609.08675 , primaryclass =

    YouTube-8M: A Large-Scale Video Classification Benchmark , url =. arXiv , author =:1609.08675 , primaryclass =

    Pith/arXiv arXiv

  2. [2]

    Findings of the Association for Computational Linguistics: ACL 2025 , year =

    Autonomous Data Selection with Zero-shot Generative Classifiers for Mathematical Texts , author =. Findings of the Association for Computational Linguistics: ACL 2025 , year =

    2025

  3. [3]

    , institution =

    Li, Chao and Zhang, Yifan and Lu, Yanzhen and Yao, Andrew C. , institution =. 2025 , month = aug, url =

    2025

  4. [4]

    Wang, Yudong and Fu, Zixuan and Cai, Jie and Tang, Peijun and Lyu, Hongya and Fang, Yewei and Zheng, Zhi and Zhou, Jie and Zeng, Guoyang and Xiao, Chaojun and Han, Xu and Liu, Zhiyuan , journal =

  5. [5]

    arXiv preprint arXiv:2602.09003 , year =

    Data Science and Technology Towards AGI Part I: Tiered Data Management , author =. arXiv preprint arXiv:2602.09003 , year =

    arXiv

  6. [6]

    Long Ouyang and Jeffrey Wu and Xu Jiang and Diogo Almeida and Carroll L. Wainwright and Pamela Mishkin and Chong Zhang and Sandhini Agarwal and Katarina Slama and Alex Ray and John Schulman and Jacob Hilton and Fraser Kelton and Luke Miller and Maddie Simens and Amanda Askell and Peter Welinder and Paul F. Christiano and Jan Leike and Ryan Lowe , title =....

  7. [7]

    Guo, Daya and Yang, Dejian and Zhang, Haowei and Song, Junxiao and Wang, Peiyi and Zhu, Qihao and Xu, Runxin and Zhang, Ruoyu and Ma, Shirong and Bi, Xiao and Zhang, Xiaokang and Yu, Xingkai and Wu, Yu and Wu, Z. F. and Gou, Zhibin and Shao, Zhihong and Li, Zhuoshu and Gao, Ziyi and Liu, Aixin and Xue, Bing and Wang, Bingxuan and Wu, Bochao and Feng, Bei ...

  8. [8]

    Zhihong Shao and Peiyi Wang and Qihao Zhu and Runxin Xu and Junxiao Song and Mingchuan Zhang and Y. K. Li and Y. Wu and Daya Guo , title =. CoRR , volume =. 2024 , doi =

    2024

  9. [9]

    Le and Geoffrey E

    Noam Shazeer and Azalia Mirhoseini and Krzysztof Maziarz and Andy Davis and Quoc V. Le and Geoffrey E. Hinton and Jeff Dean , title =. 5th International Conference on Learning Representations, ICLR 2017 , publisher =. 2017 , url =

    2017

  10. [10]

    9th International Conference on Learning Representations, ICLR 2021 , publisher =

    Dmitry Lepikhin and HyoukJoong Lee and Yuanzhong Xu and Dehao Chen and Orhan Firat and Yanping Huang and Maxim Krikun and Noam Shazeer and Zhifeng Chen , title =. 9th International Conference on Learning Representations, ICLR 2021 , publisher =. 2021 , url =

    2021

  11. [11]

    Journal of Machine Learning Research , volume =

    William Fedus and Barret Zoph and Noam Shazeer , title =. Journal of Machine Learning Research , volume =. 2022 , url =

    2022

  12. [13]

    Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2020 , pages =

    Samyam Rajbhandari and Jeff Rasley and Olatunji Ruwase and Yuxiong He , title =. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2020 , pages =. 2020 , doi =

    2020

  13. [14]

    Zhang and Han Bao and Hanwei Xu and Haocheng Wang and Haowei Zhang and Honghui Ding and Huajian Xin and Huazuo Gao and Hui Li and Hui Qu and J

    DeepSeek-AI and Aixin Liu and Bei Feng and Bing Xue and Bingxuan Wang and Bochao Wu and Chengda Lu and Chenggang Zhao and Chengqi Deng and Chenyu Zhang and Chong Ruan and Damai Dai and Daya Guo and Dejian Yang and Deli Chen and Dongjie Ji and Erhang Li and Fangyun Lin and Fucong Dai and Fuli Luo and Guangbo Hao and Guanting Chen and Guowei Li and H. Zhang...

    Pith/arXiv arXiv

  14. [15]

    Operations Research , volume =

    Chenyu Huang and Zhengyang Tang and Shixi Hu and Ruoqing Jiang and Xin Zheng and Dongdong Ge and Benyou Wang and Zizhuo Wang , title =. Operations Research , volume =. 2025 , doi =

    2025

  15. [16]

    Thirty-Ninth AAAI Conference on Artificial Intelligence, AAAI 2025 , pages =

    Mahdi Mostajabdaveh and Timothy Tin Long Yu and Samarendra Chandan Bindu Dash and Rindra Ramamonjison and Jabo Serge Byusa and Giuseppe Carenini and Zirui Zhou and Yong Zhang , title =. Thirty-Ninth AAAI Conference on Artificial Intelligence, AAAI 2025 , pages =. 2025 , doi =

    2025

  16. [17]

    Proceedings of the NeurIPS 2022 Competitions Track , pages =

    NL4Opt Competition: Formulating Optimization Problems Based on Their Natural Language Descriptions , author =. Proceedings of the NeurIPS 2022 Competitions Track , pages =. 2022 , editor =

    2022

  17. [18]

    CoRR , volume =

    Zhuohan Wang and Ziwei Zhu and Ziniu Li and Congliang Chen and Yizhou Han and Yufeng Lin and Zhihang Lin and Angyang Gu and Xinglin Hu and Ruoyu Sun and Tian Ding , title =. CoRR , volume =. 2025 , doi =

    2025

  18. [19]

    2606.19348 , archivePrefix=

    DeepSeek-AI and Anyi Xu and Bangcai Lin and Bing Xue and Bingxuan Wang and Bingzheng Xu and Bochao Wu and Bowei Zhang and Chaofan Lin and Chen Dong and Chenchen Ling and Chengda Lu and Chenggang Zhao and Chengqi Deng and Chengyu Hou and Chenhao Xu and Chenze Shao and Chong Ruan and Conner Sun and Damai Dai and Daya Guo and Dejian Yang and Deli Chen and Do...

    arXiv

  19. [20]

    Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence , articleno =

    Xiao, Ziyang and Xie, Jingrong and Xu, Lilin and Guan, Shisi and Zhu, Jingyan and Han, Xiongwei and Fu, Xiaojin and Yu, WingYin and Wu, Han and Shi, Wei and Kang, Qingcan and Duan, Jiahui and Zhong, Tao and Yuan, Mingxuan and Zeng, Jia and Wang, Yuan and Chen, Gang and Zhang, Dongxiang , title =. Proceedings of the Thirty-Fourth International Joint Confer...

    doi:10.24963/ijcai.2025/1192 2025

  20. [21]

    CoRR , volume =

    Hongyang Chen and Zhongwu Sun and Hongfei Ye and Kunchi Li and Xuemin Lin , title =. CoRR , volume =. 2026 , eprinttype =. 2603.12658 , url =

    arXiv 2026

  21. [22]

    French , title =

    Robert M. French , title =. Trends in Cognitive Sciences , volume =. 1999 , doi =

    1999

  22. [23]

    Don't Stop Pretraining: Adapt Language Models to Domains and Tasks , booktitle =

    Suchin Gururangan and Ana Marasovi. Don't Stop Pretraining: Adapt Language Models to Domains and Tasks , booktitle =. 2020 , publisher =. doi:10.18653/v1/2020.acl-main.740 , url =

    doi:10.18653/v1/2020.acl-main.740 2020

  23. [24]

    CoRR , volume =

    Haoran Que and Jiaheng Liu and Ge Zhang and Chenchen Zhang and Xingwei Qu and Yinghao Ma and Feiyu Duan and Zhiqi Bai and Jiakai Wang and Yuanxing Zhang and Xu Tan and Jie Fu and Wenbo Su and Jiamang Wang and Lin Qu and Bo Zheng , title =. CoRR , volume =. 2024 , eprinttype =. 2406.01375 , url =

    Pith/arXiv arXiv 2024

  24. [25]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages =

    Jiawei Gu and Zacc Yang and Chuanghao Ding and Rui Zhao and Fei Tan , title =. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages =. 2024 , publisher =

    2024

  25. [26]

    CoRR , volume =

    Ningyuan Xi and Yetao Wu and Kun Fan and Teng Chen and Qingqing Gu and Peng Yu and Jinxian Qu and Chenxi Liu and Zhonglin Jiang and Yong Chen and others , title =. CoRR , volume =. 2024 , eprinttype =. 2409.06624 , url =

    Pith/arXiv arXiv 2024

  26. [27]

    International Conference on Learning Representations , volume=

    Mix-cpt: A domain adaptation framework via decoupling knowledge learning and format alignment , author=. International Conference on Learning Representations , volume=

  27. [28]

    Proceedings of the 42nd International Conference on Machine Learning , series =

    Hongliang Lu and Zhonglin Xie and Yaoyu Wu and Can Ren and Yuxuan Chen and Zaiwen Wen , title =. Proceedings of the 42nd International Conference on Machine Learning , series =. 2025 , url =

    2025

  28. [29]

    Bastian and Frederic Sala , title =

    Zhiqi Gao and Albert Ge and Alexander Berenbeim and Nathaniel D. Bastian and Frederic Sala , title =. CoRR , volume =. 2026 , eprinttype =. 2605.21751 , doi =

    Pith/arXiv arXiv 2026

  29. [30]

    CoRR , volume =

    Minwei Kong and Chonghe Jiang and Ao Qu and Wenbin Ouyang and Zhaoming Zeng and Xiaotong Guo and Zhekai Li and Junyi Li and Yi Fan and Xinshou Zheng and Xi Jing and Yikai Zhang and Zhiwei Liang and Seonghoo Kim and Runqing Yang and Zijian Zhou and Sirui Li and Han Zheng and Wangyang Ying and Ou Zheng and Chonghuan Wang and Jinglong Zhao and Hanzhang Qin a...

    Pith/arXiv arXiv 2026

  30. [31]

    Proceedings of the 41st International Conference on Machine Learning , series =

    Ali Ahmaditeshnizi and Wenzhi Gao and Madeleine Udell , title =. Proceedings of the 41st International Conference on Machine Learning , series =. 2024 , url =

    2024

  31. [32]

    2025 , howpublished =

    Albert Ge , title =. 2025 , howpublished =

    2025

  32. [33]

    CoRR , volume =

    Xuhan Huang and Qingning Shen and Yan Hu and Anningzhe Gao and Benyou Wang , title =. CoRR , volume =. 2024 , eprinttype =. 2405.13144 , doi =

    Pith/arXiv arXiv 2024

  33. [34]

    2026 , note =

    Gurobi Modeling Examples , howpublished =. 2026 , note =

    2026

  34. [35]

    2024 , howpublished =

    Nicholas Lownes , title =. 2024 , howpublished =

    2024

  35. [36]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Demystifying domain-adaptive post-training for financial llms , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  36. [37]

    Fu and Stefano Ermon and Atri Rudra and Christopher R\'

    Tri Dao and Daniel Y. Fu and Stefano Ermon and Atri Rudra and Christopher R\'. Advances in Neural Information Processing Systems 35 (NeurIPS 2022) , year =. 2205.14135 , eprinttype =

    Pith/arXiv arXiv 2022

  37. [38]

    The Twelfth International Conference on Learning Representations (ICLR 2024) , year =

    Tri Dao , title =. The Twelfth International Conference on Learning Representations (ICLR 2024) , year =. 2307.08691 , eprinttype =

    Pith/arXiv arXiv 2024

  38. [39]

    CoRR , volume =

    Jay Shah and Ganesh Bikshandi and Ying Zhang and Vijay Thakkar and Pradeep Ramani and Tri Dao , title =. CoRR , volume =. 2024 , url =. 2407.08608 , eprinttype =

    Pith/arXiv arXiv 2024

  39. [40]

    Philippe Tillet and H. T. Kung and David Cox , title =. Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages (MAPL 2019) , pages =. 2019 , doi =

    2019

  40. [41]

    CoRR , volume =

    Pin-Lun Hsu and Yun Dai and Vignesh Kothapalli and Qingquan Song and Shao Tang and Siyu Zhu and Steven Shimizu and Shivam Sahni and Haowen Ning and Yanning Chen , title =. CoRR , volume =. 2024 , url =. 2410.10989 , eprinttype =

    Pith/arXiv arXiv 2024

  41. [42]

    Proceedings of Machine Learning and Systems 7 (MLSys 2025) , year =

    Zihao Ye and Lequn Chen and Ruihang Lai and Wuwei Lin and Yineng Zhang and Stephanie Wang and Tianqi Chen and Baris Kasikci and Vinod Grover and Arvind Krishnamurthy and Luis Ceze , title =. Proceedings of Machine Learning and Systems 7 (MLSys 2025) , year =. 2501.01005 , eprinttype =

    Pith/arXiv arXiv 2025

  42. [43]

    CoRR , volume =

    Anjiang Wei and Tianran Sun and Yogesh Seenichamy and Hang Song and Anne Ouyang and Azalia Mirhoseini and Ke Wang and Alex Aiken , title =. CoRR , volume =. 2025 , url =. 2509.07506 , eprinttype =

    arXiv 2025

  43. [44]

    CoRR , volume =

    Juncheng Dong and Yang Yang and Tao Liu and Yang Wang and Feng Qi and Vahid Tarokh and Kaushik Rangadurai and Shuang Yang , title =. CoRR , volume =. 2025 , url =. 2510.16996 , eprinttype =

    arXiv 2025

  44. [45]

    CoRR , volume =

    Bytedance-Seed-Foundation-Code-Team and M-A-P , title =. CoRR , volume =. 2024 , url =. 2412.00535 , eprinttype =

    Pith/arXiv arXiv 2024

  45. [46]

    CoRR , volume =

    Jiehao Wu and Zixiao Huang and Wenhao Li and Chuyun Shen and Junjie Sheng and Xiangfeng Wang , title =. CoRR , volume =. 2025 , url =. 2603.23566 , eprinttype =

    Pith/arXiv arXiv 2025

  46. [47]

    2601.07160 , archivePrefix=

    Xinzi Cao and Jianyang Zhai and Pengfei Li and Zhiheng Hu and Cen Yan and Bingxu Mu and Guanghuan Fang and Bin She and Jiayu Li and Yihan Su and Dongyang Tao and Xiansong Huang and Fan Xu and Feidiao Yang and Yao Lu and Chang-Dong Wang and Yutong Lu and Weicheng Xue and Bin Zhou and Yonghong Tian , year=. 2601.07160 , archivePrefix=

    Pith/arXiv arXiv

  47. [48]

    2601.22760 , archivePrefix=

    Zhongzhen Wen and Shudi Shao and Zhong Li and Yu Ge and Tongtong Xu and Yuanyi Lin and Tian Zhang , year=. 2601.22760 , archivePrefix=

    arXiv

  48. [49]

    Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence , pages =

    Ziyang Xiao and Jingrong Xie and Lilin Xu and Shisi Guan and Jingyan Zhu and Xiongwei Han and Xiaojin Fu and WingYin Yu and Han Wu and Wei Shi and Qingcan Kang and Jiahui Duan and Tao Zhong and Mingxuan Yuan and Jia Zeng and Yuan Wang and Gang Chen and Dongxiang Zhang , title =. Proceedings of the Thirty-Fourth International Joint Conference on Artificial...

    2025

  49. [50]

    CoRR , volume =

    Yang Wang and Kai Li , title =. CoRR , volume =. 2025 , eprinttype =. 2509.18180 , doi =

    arXiv 2025

  50. [51]

    CoRR , volume =

    Tinglong Dai and David Simchi-Levi and Michelle Xiao Wu and Yao Xie , title =. CoRR , volume =. 2025 , eprinttype =. 2512.23978 , doi =

    Pith/arXiv arXiv 2025

  51. [52]

    Chain-of-Experts: When

    Ziyang Xiao and Dongxiang Zhang and Yangjun Wu and Lilin Xu and Yuan Jessica Wang and Xiongwei Han and Xiaojin Fu and Tao Zhong and Jia Zeng and Mingli Song and Gang Chen , booktitle =. Chain-of-Experts: When. 2024 , url =

    2024

  52. [53]

    Bowen Zhang and Pengcheng Luo and Genke Yang and Boon-Hee Soong and Chau Yuen , journal =

  53. [54]

    2026 , url =

    Zezhen Ding and Zhen Tan and Jiheng Zhang and Tianlong Chen , booktitle =. 2026 , url =

    2026

  54. [55]

    Ruicheng Ao and David Simchi-Levi and Xinshang Wang , journal =

  55. [56]

    arXiv preprint arXiv:2504.20073 , year=

    RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning , author=. arXiv preprint arXiv:2504.20073 , year=

    Pith/arXiv arXiv

  56. [57]

    arXiv preprint arXiv:2503.09516 , year=

    Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning , author=. arXiv preprint arXiv:2503.09516 , year=

    Pith/arXiv arXiv

  57. [58]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Deepor: A deep reasoning foundation model for optimization modeling , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  58. [59]

    Findings of the Association for Computational Linguistics: EMNLP 2025 , pages=

    SCoder: Progressive Self-Distillation for Bootstrapping Small-Scale Data Synthesizers to Empower Code LLMs , author=. Findings of the Association for Computational Linguistics: EMNLP 2025 , pages=

    2025

  59. [60]

    arXiv preprint arXiv:2605.26132 , year=

    Self-Verified Distillation: Your Language Model Is Secretly Its Own Synthetic Data Pipeline , author=. arXiv preprint arXiv:2605.26132 , year=

    Pith/arXiv arXiv

  60. [61]

    Findings of the Association for Computational Linguistics: EMNLP 2025 , pages=

    Training LLMs for Optimization Modeling via Iterative Data Synthesis and Structured Validation , author=. Findings of the Association for Computational Linguistics: EMNLP 2025 , pages=

    2025

  61. [62]

    arXiv preprint arXiv:2505.03469 , year=

    Long-short chain-of-thought mixture supervised fine-tuning eliciting efficient reasoning in large language models , author=. arXiv preprint arXiv:2505.03469 , year=

    Pith/arXiv arXiv

  62. [63]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Facilitating long context understanding via supervised chain-of-thought reasoning , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  63. [64]

    arXiv preprint arXiv:2502.03373 , year=

    Demystifying long chain-of-thought reasoning in llms , author=. arXiv preprint arXiv:2502.03373 , year=

    Pith/arXiv arXiv

  64. [65]

    Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    A two-stage adaptation of large language models for text ranking , author=. Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    2024

  65. [66]

    arXiv preprint arXiv:2510.10071 , year=

    ADEPT: Continual Pretraining via Adaptive Expansion and Dynamic Decoupled Tuning , author=. arXiv preprint arXiv:2510.10071 , year=

    arXiv

  66. [67]

    arXiv preprint arXiv:2402.17400 , year=

    Investigating continual pretraining in large language models: Insights and implications , author=. arXiv preprint arXiv:2402.17400 , year=

    Pith/arXiv arXiv

  67. [68]

    arXiv preprint arXiv:2602.22859 , year=

    From Blind Spots to Gains: Diagnostic-Driven Iterative Training for Large Multimodal Models , author=. arXiv preprint arXiv:2602.22859 , year=

    Pith/arXiv arXiv

  68. [69]

    Lane , booktitle=

    Wanru Zhao and Yihong Chen and Yuzhi Tang and Wentao Ma and Shengchao Hu and Shell Xu Hu and Alex Iacob and Abhinav Mehrotra and Nicholas D. Lane , booktitle=. Rethinking Data Curation in. 2026 , url=

    2026

  69. [70]

    Forty-second International Conference on Machine Learning , year=

    Boosting multi-domain fine-tuning of large language models through evolving interactions between samples , author=. Forty-second International Conference on Machine Learning , year=

  70. [71]

    Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 3: System Demonstrations) , pages=

    The AI committee: A multi-agent framework for automated validation and remediation of web-sourced data , author=. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 3: System Demonstrations) , pages=

  71. [72]

    ICML 2025 Workshop on Computer Use Agents , year=

    Reinforcing multi-turn reasoning in llm agents via turn-level credit assignment , author=. ICML 2025 Workshop on Computer Use Agents , year=

    2025

  72. [73]

    Yuankang Fu and Xinrong Gong and Chen Gong and Tong Zhang and Kaixiang Yang , booktitle=. Cure-. 2026 , url=

    2026

  73. [74]

    Advances in Neural Information Processing Systems , volume=

    Curriculum Design for Trajectory-Constrained Agent: Compressing Chain-of-Thought Tokens in LLMs , author=. Advances in Neural Information Processing Systems , volume=

  74. [75]

    Xing and Sham M

    Zhenting Qi and Fan Nie and Alexandre Alahi and James Zou and Himabindu Lakkaraju and Yilun Du and Eric P. Xing and Sham M. Kakade and Hanlin Zhang , booktitle=. Evo. 2026 , url=

    2026

  75. [76]

    Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V

    DCA-bench: a benchmark for dataset curation agents , author=. Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2 , pages=

  76. [77]

    2025 , eprint=

    gpt-oss-120b & gpt-oss-20b Model Card , author=. 2025 , eprint=

    2025

  77. [78]

    2602.15763 , archivePrefix=

    GLM-5-Team and Aohan Zeng and Xin Lv and Zhenyu Hou and Zhengxiao Du and Qinkai Zheng and Bin Chen and Da Yin and Chendi Ge and Chenghua Huang and Chengxing Xie and Chenzheng Zhu and Congfeng Yin and Cunxiang Wang and Gengzheng Pan and Hao Zeng and Haoke Zhang and Haoran Wang and Huilong Chen and Jiajie Zhang and Jian Jiao and Jiaqi Guo and Jingsen Wang a...

    Pith/arXiv arXiv

  78. [79]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    MAIN: Mutual Alignment Is Necessary for instruction tuning , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  79. [80]

    arXiv preprint arXiv:2406.14971 , year=

    Domain adaptation of llama3-70b-instruct through continual pre-training and model merging: A comprehensive evaluation , author=. arXiv preprint arXiv:2406.14971 , year=

    Pith/arXiv arXiv

  80. [81]

    International Conference on Learning Representations , year=

    Finetuned Language Models are Zero-Shot Learners , author=. International Conference on Learning Representations , year=

Showing first 80 references.