Pith.

Artificial Intelligence · arXiv:2609.30001 v1 · submitted 24 Sep 2026 · 34 references

Advancing Model Research in AgentX: Long-Horizon Autonomy for Industrial Recommender Systems

Shuang Yang, Zijie Zhuang, Changxin Lao, Pengbo Xu, Hanwen Xu, Yusheng Huang, Han Gao, Guanchen Wang

show 15 more authors

Tianbao Ma, Linxun Chen, Peilin Song, Xuming Wang, Chen Li, Fan Wu, Tao Wang, Zibo Zhao, Xiangyu Wu, An Liu, Fei Pan, Peng Jiang, Chen Yang, Zhaojie Liu, Wenwu Ou

The paper says

“Sustaining industrial recommendation research requires using the results of one experiment to decide what to investigate next.”

more of the abstract

We present AgentX-Model, the next generation of AgentX's model research framework, which connects proposal development and model experimentation within sandboxes defined by business inputs and prediction tasks. AgentX-Model adopts a dual-agent architecture comprising a Research Agent and a Model Agent. The Research Agent develops independently reviewed proposals from papers and experimental findings, while the Model Agent conducts multi-round investigations and returns code, measurements, and unresolved questions. Using the returned results, the Research Agent selects a starting implementation and formulates the next research question, allowing subsequent experiments to build on earlier findings. We organize this continuing research around four actions: Reproduce, Follow-up, Composition, and Diagnose. The first three actions drive routine research, while Diagnose acquires the evidence needed to choose a repair, including for issues raised by business feedback and online evaluation, such as prediction bias measured by PCOC. Across the production evaluation, 560 of 636 completed model-changing experiments recorded AUC above their business baselines. As research continued, some experiments recorded AUC above every comparable ancestor in their lineages. The five latest online A/B evaluations across different business settings reported gains including 10-15% in acquisition efficiency, 15-20% in target-segment advertising spend, and 0.3-0.8% in watch time; the watch-time model used approximately 10% fewer FLOPs and parameters. A dependency-aware historical-replay benchmark further evaluates research allocation, with initial results showing no consistent efficiency gain from more complex scheduling when agents already analyze and select concrete candidates.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 25 Sep 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. Honest industrial engineering report with modest, real novelty, but the central attribution claim is undercut by the paper's own numbers: nearly all Scenario A gain lands in the first six days, and every controlled test of the selection machinery is null. This line is wrong

4 major objections · 5 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 34 references say.

The paper says, at §4.3.1, Fig. 3 (b226, b228)

The review did not quote this passage.

Pith says: Objection 1 of 4 · major

The paper's own timeline undercuts the attribution of gains to the continuation loop. The cumulative best AUC reaches 0.797167 by day T+6 and gains only 0.000185 by T+11, then is flat through T+15; 99.2% of the final 0.023669 total gain arrives within the first six days of the fifteen-day cohort.

more of objection 1

Since nearly all measurable improvement precedes the later continuation window, the headline '+0.023669 over ~25 days' cannot be read as evidence that the research-continuation machinery produces ongoing gains. The abstract and conclusion present the raw AUC totals and the online gains without this near-term-saturation context; the revision should either report the saturation explicitly in the headline results or reframe the claim as descriptive of what the system operated on, not what it caused.

The paper says, at Table 7, §4.3.2 (b240, b241)

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.74):

“The proportions in Tables 4 and 5 are not experiment success rates: many Follow-up and Composition experiments inherit implementations that already outperform the business baseline.”

Pith says: Objection 2 of 4 · major

The continuation-specific progress signal is small. Only 7 of 77 Follow-ups (9.1%) and 5 of 120 Compositions (4.2%) in Scenario A record AUC above every comparable ancestor; i.e., 12 of 197 continuations set a new path best, and the rest inherit gains over the business baseline — a conflation the authors acknowledge in b204.

more of objection 2

This means the claim that 'continued research produced new records above all comparable ancestors in some lineages' (b35) rests on roughly 6% of the continuation sample. The revision should quantify how much of the final best-AUC gain is attributable to these 12 path-best experiments versus inherited progress, or state plainly that the framework's continuation value is not separately measurable in the current production data.

The paper says, at §4.5, Tables 10 and 12

The review did not quote this passage.

Pith says: Objection 3 of 4 · major

Every controlled comparison of the selection machinery is null or weakly negative, and this is load-bearing for the paper's third claimed contribution. No allocation policy consistently beats fixed rotation (Table 10); summaries do not reliably improve best AUC or shorten paths and even reduce above-baseline selections in one Scenario A batch (Table 12); and in 15 Scenario A replay runs no Composition is ever unlocked.

more of objection 3

These results are honestly reported, and the paper appropriately concludes that fixed rotation is competitive. But they also mean the agent-based candidate selector is a common component of all three 'agent' policies, so the comparison isolates only the allocation layer, not the contribution of agent selection; the paper should state this limitation explicitly where it claims a 'benchmark for continuing research decisions' as a contribution.

The paper says, at Table 6 and §4.1

The review did not quote this passage.

Pith says: Objection 4 of 4 · major

The online A/B evidence is presented with coarse ranges (e.g., 10–15% acquisition efficiency) and no confidence intervals, significance tests, or sample sizes, and the five Launch Reviews are the only business-impact evidence in the paper. Moreover, the reported within-implementation pooled AUC standard deviation for Scenario A is 0.002208 (§4.1), while several headline margins — the late +0.000185 gain, the C4/C5 …

more of objection 4

… deficits of 0.000328 and 0.001450, and the Composition gains in Table 7's parent-column — are at or below this noise scale. The revision should either provide significance or error-bar information for the online numbers or explicitly downgrade them to anecdotal, and should state which reported offline margins exceed run-to-run variability.

5 minor objections

The paper says, at Fig. 3(b)

Pith says: minor objection 1 of 5

The three intervals in panel (b) appear to be of different widths (the final interval spans only two days, as the caption notes), so the share changes across intervals are not directly comparable; please plot per-day shares or annotate interval widths in the figure itself.

The paper says, at §4.3.2

Pith says: minor objection 2 of 5

The paragraph on preserving the best measured round reports that 111/189 experiments exceed their first-round AUC while 91/189 finish below an earlier best, and notes the groups can overlap; reporting the joint distribution (e.g., of the 91, how many still exceed the first round) would make the preservation argument concrete.

show 3 more

The paper says, at Eq. (5)

Pith says: minor objection 3 of 5

The summation symbols in the PCOC display equation are rendered incorrectly in the submitted text; please fix the typesetting so the ratio ∑ẏ/∑y is legible.

The paper says, at Table 9

Pith says: minor objection 4 of 5

The Knowledge Transfer versus paper-derived comparison rests on 7 versus 15 unmatched experiments (b312); please label this explicitly as descriptive and add a Fisher exact test or a clear statement that no test is meaningful at this sample size.

The paper says, at §4.1, §4.3.1

Pith says: minor objection 5 of 5

The observation window is described as 'approximately 25 days' in Table 2 but the Scenario A cohort in Figure 3 covers 16 days with 453 proposals; please reconcile the two windows or state explicitly that they are different sub-populations.

What would settle it

Run a matched counterfactual control arm: over the same business settings (A–D) and a comparable observation window, randomize incoming research tasks between AgentX-Model and expert engineers, giving both arms identical baselines, experiment budgets, and human deployment gates, then report online A/B deltas with confidence intervals for each arm. If the agent arm's deltas are not statistically separable from the engineer arm's, the attribution of gains to the continuation framework collapses.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 25 Sep 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic agrees 42 s · 26,629 tokens read, 5,398 written
  3. Editor wrote the verdict above 14 s · 27,814 tokens read, 1,805 written
  4. Referee major revision: 4 major objections, 5 minor 41 s · 26,992 tokens read, 5,276 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Worth knowing: this is a competently written, unusually candid industrial report, and the novelty is real but modest. The four-action taxonomy (Reproduce, Follow-up, Composition, Diagnose), the parent-versus-strongest-ancestor comparison, and the dependency-aware replay benchmark (473 nodes, six environments) are genuinely new relative to the cited neighbors. The reporting discipline is the best thing here: the authors disclose their nulls plainly — few Compositions beat ancestors, no allocation policy consistently beats fixed rotation, summaries don't help selection, Scenario D paper-derived work gains nothing, and the calibration repair needed multiple rounds. That honesty earns real credit, and I don't read this as a fitting-as-prediction paper: gains are measured against external business baselines, so the circularity burden is low, and they explicitly caveat the historical-sampling conditioning of the replay (b341).

The soft spot is load-bearing, and the stress-test reading holds up. The headline claim is that the dual-agent continuation machinery sustains long-horizon research with business gains. The paper's own evidence doesn't support attributing those gains to the machinery. Scenario A's cumulative best AUC reaches 0.797167 by day T+6 and adds only 0.000185 through T+15 (b226, b228) — roughly 99% of the improvement precedes the continuation window being measured. Only 12 of 197 Scenario A continuations set a new path-best record (b240, b241); the rest inherit gains, a conflation the authors themselves flag (b204). And every controlled comparison of the selection/continuation layer is null or weak (Tables 10 and 12; no Composition unlocked across 15 Scenario A replay runs). The authors concede the weakest form (b204), but the consequence is stronger than they state: the data are consistent with a competent human engineer choosing among Reproduce candidates producing the same results. There is no human-led or non-agent control arm over a matched window, and the five online A/B gains are coarse ranges without intervals. No code or data is released.

Who this is for: people building agentic research harnesses for industrial ML, and anyone fielding similar 'autonomy' claims — the replay benchmark is the most reusable artifact. My recommendation: send it to peer review; it deserves referee time. But the referees should push for one of two things — either a real counterfactual arm (randomize tasks between the framework and expert engineers) or a reframed claim that stops asserting the framework caused the gains and instead reports what was built and what was observed. As written, the conclusion outruns the evidence.

The strongest objection

The paper's central claim is that the dual-agent continuation framework sustains long-horizon research with measurable business gains. The load-bearing condition is that the observed offline AUC and online A/B gains are produced by the continuation machinery rather than by the underlying model mechanisms, inherited baselines, or human-chosen deployments. The paper's own data undercut that condition in three places. First, in Scenario A the cumulative best-AUC record reaches 0.797167 on day T+6 and gains only 0.000185 by T+11, staying flat through T+15 (b226, b228); 0.023484 of the final 0.023669 total gain (99.2%) arrived within the first six days of the fifteen-day cohort, so nearly all measurable improvement precedes the later continuation window. Second, the continuation-specific progress signal is small: only 7/77 Follow-ups and 5/120 Compositions recorded AUC above every comparable ancestor (Table 7, b240, b241), i.e. 12 of 197 Scenario A continuations set new path-best records; the rest inherit gains relative to business baselines, a conflation the authors themselves flag (b204). Third, every controlled comparison of the research-selection machinery is null or weak: no allocation policy consistently beat fixed rotation (Table 10), summaries did not improve selection outcomes (Table 12), and in 15 replay runs on the Scenario A graph no Composition was ever unlocked (Section 4.5). These nulls are honestly reported, but they internally corroborate a reading in which the continuation/selection layer contributes little beyond what a human engineer selecting among Reproduce candidates would achieve. The missing piece is a counterfactual: no human-led or non-agent research control over a matched window, with the same baselines and deployment gates, exists to establish that the agent framework, rather than the specific mechanisms it happened to introduce or the favorable selection of the five reported Launch Reviews, drove the headline gains.

The reader's weakest assumption — that measured gains are attributable to the framework's continuation rather than to weak baselines, deployment selection, and the specific model mechanisms — is exactly where my reading lands, and I agree the absent human-expert counterfactual is the load-bearing gap. I sharpen it with three internal data points the paper itself provides: the Scenario A best-AUC record saturating within the first six days of the fifteen-day cohort (b226, b228), the thin new-path-best signal (7/77 Follow-ups and 5/120 Compositions, b240–b241, Table 7), and the uniformly null scheduling and summary results (Tables 10 and 12). These do not contradict the paper's honest reporting — it explicitly flags that 560/636 is not a success rate (b204) and that replay is conditioned on historical branches (b341) — but they do shift the interpretation: the sustained-operation claim is well supported, while the gains-attributable-to-continuation claim rests entirely on the missing counterfactual. The paper deserves credit for disclosing negative results (b90, b312, b360, b394, b405) and for the auditor-caught proposal revisions in Appendix A.2, which show a working verification loop. A human-expert matched control with interval estimates would settle the causal attribution; an internal audit of path-best contributions to the Scenario A delta would serve as a cheaper partial check. Since this is a structural causal-inference gap in an otherwise competent report, the CONDITIONAL verdict stands unchanged; I do not see evidence for ACCEPT (no control arm, no intervals) or for REJECT (operation is demonstrated and reporting is transparent).

The referee's report

If the framework claim holds, this is a useful industrial data point: an agent system that autonomously proposes, executes, and continues model experiments across ~25 production days, with documented throughput (262 delivered proposals in 10 days, median 33.6 min per proposal) and concrete model mechanisms that an expert team might not have tried (Appendix C). The dependency-aware replay benchmark is a genuinely reusable methodological contribution, and the paper's honesty about nulls and limitations (e.g., b204, b341, b405) is a real strength. However, the central causal claim — that the continuation machinery, rather than inherited baselines or the specific model mechanisms, produces the measured gains — is not demonstrated, and several of the paper's own numbers undercut it. The paper's value is therefore as a descriptive systems report and benchmark contribution, not as evidence that dual-agent continuation itself drives business gains.

Pith's reading of the paper (a machine's summary, not the authors')

The paper aims to show that industrial recommender-model research can run as a continuing, largely autonomous loop: a Research Agent decides what to study next from papers, prior experiment records, and business feedback, while a Model Agent implements and investigates each approved proposal through multiple rounds. The claim is that connecting these two horizons — cross-experiment planning and within-experiment investigation — is what makes long-horizon research sustainable in production. Over roughly 25 days the system completed 636 model-changing experiments, 560 of which recorded AUC above their business baselines, and the five most recent online A/B evaluations reported gains in acquisition efficiency, target-segment advertising spend, and watch time. Why it matters: if the mechanism holds, an industrial team can run a continuing model-research loop without a new human request per experiment, with people reserving approval for goals, constraints, and deployment gates.

What carries the argument. The load-bearing object is the research path: a chain of proposals and experiments connected by inheritance or composition, where each experiment retains round-specific code, measurements, and evaluation context rather than only the final revision. Three reference results — business baseline, direct parent, and strongest comparable ancestor — let the system distinguish inherited performance from a genuine new path record, which matters because 48.1% of multi-round experiments finish below an earlier best round. The four research actions give the loop its vocabulary, and the Assembler-Auditor review step inside the Research Agent is the device that forces each proposal to be testable rather than merely plausible, catching cases such as a calibration draft aimed at the wrong prediction task and a Composition that accidentally repeated one of its sources.

Core claim. The paper's central claim is that experimental results can themselves become the material for the next experiment, and that this continuation requires a specific division of labor. The Research Agent composes independently reviewed proposals — each carrying a research question, a starting implementation, a proposed modification, and an evaluation design, written P_t = (q_t, s_t, delta_t, v_t) — while the Model Agent runs multi-round investigations and returns code, measurements, and unresolved questions. Those returns form a shared research state S_t = (I_t, E_t, Q_t) that feeds the next proposal, organizing the loop into four actions: Reproduce, Follow-up, Composition, and Diagnose. The headline evidence is production-scale: 636 completed model-changing experiments over about 25 days, 560 above their business baselines; 7 of 77 Scenario A Follow-ups and 5 of 120 Compositions beating every comparable ancestor; and five recent online A/B evaluations reporting gains that include 10-15% acquisition efficiency, 15-20% target-segment advertising spend, and 0.3-0.8% watch time with about 10% fewer FLOPs and parameters on the watch-time model.

The premise it rests on. The measured offline and online gains are attributed to the dual-agent continuation loop, but the paper runs no control arm of human-led or simpler non-agent research over the same observation window, and it concedes (Section 4.3.1) that the AUC proportions are not success rates because many experiments inherit implementations that already outperform the baseline; the causal link between 'dual-agent continuation' and 'business gains' is therefore asserted, not demonstrated.

If the paper is right (Pith's inferences, not the authors')

If the cycle works as claimed, an industrial team can run continuing model research inside a business-defined sandbox without a new human request per experiment, with humans setting goals, constraints, and deployment gates.

Retaining round-specific code and measurements becomes a hard requirement for continuation: with 48.1% of multi-round experiments ending below an earlier best, an implementation-overwriting practice would discard useful starting points.

Reporting both parent and ancestor comparisons becomes necessary to state whether research is advancing: 46 of 91 Scenario A Follow-ups beat their direct parent but only 7 of 77 beat every ancestor.

Historical replay suggests that once a planning agent already selects concrete candidates, fixed task-type rotation matches or beats bandit and joint allocation, and adding compressed summaries to raw evidence yields no consistent gain.

The calibration repair case shows that business feedback can redirect a path from ranking gains toward diagnosis and conditional repair while preserving the prior ranking benefit.

The recorded gains blend the continuation machinery with the individual model mechanisms (denoising, gating, expert routing, LayerNorm normalization, quantizer exploration noise), and the paper runs no control arm of human-led or single-agent research over the same window; the most defensible reading is that the loop sustains throughput, not that it outperforms a competent engineer.

The replay result — no consistent efficiency gain from more complex allocation once agents analyze candidates — suggests the marginal value of smarter experiment-selection policies is shrinking; the bottleneck may have shifted to proposal quality and evaluation design, not scheduling.

A testable extension: over the same window, keep the planning agent but replace the independent Auditor with a human approver; if AUC-per-experiment is unchanged, the review loop adds little, and if it degrades, the Auditor is doing the load-bearing work.

The four research actions are a reusable abstraction that could transfer to other industrial settings beyond recommendation, provided they keep the same code-and-measurement retention discipline that makes continuation possible.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Sustaining industrial recommendation research requires using the results of one experiment to decide what to investigate next. We present AgentX-Model, the next generation of AgentX's model research framework, which connects proposal development and model experimentation within sandboxes defined by business inputs and prediction tasks. AgentX-Model adopts a dual-agent architecture comprising a Research Agent and a Model Agent. The Research Agent develops independently reviewed proposals from papers and experimental findings, while the Model Agent conducts multi-round investigations and returns code, measurements, and unresolved questions. Using the returned results, the Research Agent selects a starting implementation and formulates the next research question, allowing subsequent experiments to build on earlier findings. We organize this continuing research around four actions: Reproduce, Follow-up, Composition, and Diagnose. The first three actions drive routine research, while Diagnose acquires the evidence needed to choose a repair, including for issues raised by business feedback and online evaluation, such as prediction bias measured by PCOC. Across the production evaluation, 560 of 636 completed model-changing experiments recorded AUC above their business baselines. As research continued, some experiments recorded AUC above every comparable ancestor in their lineages. The five latest online A/B evaluations across different business settings reported gains including 10-15% in acquisition efficiency, 15-20% in target-segment advertising spend, and 0.3-0.8% in watch time; the watch-time model used approximately 10% fewer FLOPs and parameters. A dependency-aware historical-replay benchmark further evaluates research allocation, with initial results showing no consistent efficiency gain from more complex scheduling when agents already analyze and select concrete candidates.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-09-25

The authors' main claim, as the classifier picked it out 0.44

“The five latest online A/B evaluations across different business settings reported gains including 10-15% in acquisition efficiency, 15-20% in target-segment advertising spend, and 0.3-0.8% in watch time; the watch-time model used approximately 10% fewer FLOPs and parameters.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.08 · states limitation 0.74 · human participants 0.20 · controlled comparison 0.91 · randomized assignment 0.18 · states quantitative result 0.99 · compares with prior results 0.94 · mentions code or data availability 0.04

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Sustaining industrial recommendation research requires using the results of one experiment to decide what to investigate next. background 0.99 0.08 0.06 asserted 1.00
We present AgentX-Model, the next generation of AgentX's model research framework, which connects proposal development and model experimentation within sandboxes defined by business inputs and prediction tasks. contribution 0.99 0.86 0.09 asserted 1.00
AgentX-Model adopts a dual-agent architecture comprising a Research Agent and a Model Agent. method 0.84 0.73 0.04 asserted 1.00
The Research Agent develops independently reviewed proposals from papers and experimental findings, while the Model Agent conducts multi-round investigations and returns code, measurements, and unresolved questions. method 0.96 0.68 0.04 asserted 1.00
Using the returned results, the Research Agent selects a starting implementation and formulates the next research question, allowing subsequent experiments to build on earlier findings. method 0.98 0.75 0.10 asserted 1.00
We organize this continuing research around four actions: Reproduce, Follow-up, Composition, and Diagnose. method 0.89 0.85 0.05 asserted 1.00
The first three actions drive routine research, while Diagnose acquires the evidence needed to choose a repair, including for issues raised by business feedback and online evaluation, such as prediction bias measured by PCOC. method 0.98 0.73 0.07 asserted 1.00
Across the production evaluation, 560 of 636 completed model-changing experiments recorded AUC above their business baselines. result 1.00 0.94 0.18 asserted 1.00
As research continued, some experiments recorded AUC above every comparable ancestor in their lineages. result 1.00 0.93 0.07 asserted 0.93
The five latest online A/B evaluations across different business settings reported gains including 10-15% in acquisition efficiency, 15-20% in target-segment advertising spend, and 0.3-0.8% in watch time; the watch-time model used approximately 10% fewer FLOPs and parameters. result 1.00 0.94 0.15 asserted 1.00
A dependency-aware historical-replay benchmark further evaluates research allocation, with initial results showing no consistent efficiency gain from more complex scheduling when agents already analyze and select concrete candidates. result 0.90 0.89 0.10 asserted 0.73

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 1.00

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-25: 1 request, 8,294 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (11)
Figure 1 of the paper
Figure 1. Human–agent collaboration in AgentX-Model. The vertical line marks the interaction boundary. Automatic allocation supplies a task type for the Research Agent to consider. The blue return arrow feeds outcomes back into research, so the right-hand loop can continue within the specified goals, constraints, and resources without a new human request. This report makes three contributions: • A dual-agent framework for autonomous research. We introduce a Research Agent and a Model Agent with distinct responsibilities for cross-experiment planning and within-experiment investigation,…
Figure 2 of the paper
Figure 2. Task allocation and research decisions. Human-designed rules allocate a research-action type. The Research Agent uses available evidence to construct and refine a concrete investigation or defer. The dashed return marks the next scheduling opportunity. otherwise, the agent defers. Resource and eligibility rules constrain the candidate pool and execu￾tion. The resulting trajectories provide the candidate space and within-experiment feedback for the scheduling benchmark in Section 4.5. 4. Evaluation and Findings The evaluation follows the work from proposal production to model…
Figure 3 of the paper
Figure 3. Model and research evolution from a fixed baseline in Scenario A. (a) Points show experiment￾best AUC by result-report time; the step line tracks the cohort best from the business baseline. Experiments may inherit implementations from before the delivery window. (b) Shares use all proposals in each interval as the denominator, with one primary direction per proposal; four of seven directions are shown. 𝑇 marks the observation start. The final interval spans only two days, 𝑇 + 14 through 𝑇 + 15. derived Reproduce experiments feed successive Compositions. Follow-up F1 then records…
Figure 4 of the paper
Figure 4. An example research lineage spanning approximately 11 days in Scenario A. Each node is an experiment labeled with its best measured AUC; arrows connect parents to children. Green borders mark new path-best AUC records. Dashed boxes summarize the other path’s earlier ancestry. comparisons distinguish a local recovery from a new record along the path
Figure 5 of the paper
Figure 5. A Scenario A Composition followed by within-experiment refinement. The left panel shows the best rounds of the two direct source experiments, not successive rounds. The right panel shows the two rounds of the new joint model. The LGCD source is the new experiment’s strongest comparable ancestor. Dashed reference lines mark the business baseline and strongest-ancestor AUC; only the two Composition rounds are connected by a line. The first round reached 0.774123, below both direct sources. Its report identified spikes in the loss used to train the diffusion initializer. The Model…
Figure 6 of the paper
Figure 6. Proposal refinement inside the Research Agent. The dashed enclosure marks the core loop: construct a proposal, review it independently, and revise and resubmit in response to feedback. Assigned actions and shared evidence are inputs; delivery to the Model Agent is an exit after approval and automated checks. Execution limits are enforced by the harness. The review loop in
Figure 7 of the paper
Figure 7. From prescribed workflows to agent-driven proposal production. Both versions use language models and the same independent Auditor; the investigation and repair path is prescribed by code in (a) and selected by the agent in (b). Dashed arrows indicate revision feedback. Execution boundaries remain enforced in both versions. Development consequently shifted toward evidence-access tools, context organization, review feedback, stopping conditions, and delivery. The agent could keep reading without committing to a candidate, carry excess context, or reopen a broad investigation after a…
Figure 8 of the paper
Figure 8. Model Agent iteration and process repair. The upper loop develops a model within the reviewed experiment. Failures and first-attempt check results feed the lower repair loop, where tool and skill revisions undergo separate forms of validation before human-approved deployment to future runs. Repair acceptance rules remain fixed. Cross-task experience learning is described separately. Checking the implementation before training. Each round begins with a plan linking the intended model change to its rationale and observable tests. Structural checks verify the required elements and…
Figure 9 of the paper
Figure 9. Sample-adaptive feature modulation. Within-sample token relationships determine the adjustments supplied to the ranking backbone. 34
Figure 10 of the paper
Figure 10. Continuous features read learned discrete prototypes. The fused representations and all original tokens feed the prediction network. C.3. LGCD with AFTM: Generating Missing Semantic Profiles LGCD [20] uses diffusion to generate a user’s preferences in another domain. The Scenario A adaptation instead generates a semantic vector from business features, using existing semantic embeddings as training targets. The ranker uses the generated vectors for all users, including those without an existing semantic profile (
Figure 11 of the paper
Figure 11. Generating semantic profiles from business features. Existing profiles supervise training; generated representations serve all samples. C.4. ORQ with DIGER: Changing How a Quantizer Selects and Updates Prototypes This Composition keeps an existing quantizer’s progressive compression structure but changes how it selects and updates code vectors. Orthogonal residual quantization (ORQ) from DOS [22] first rotates features, selects dimensions for quantization, and passes the remaining dimensions and quantization residual to the next level. Concentrated code usage in the Scenario A…

Figures are the authors', reproduced from the paper.

References (34)
  1. [1] Changxin Lao et al. AgentX: Towards Agent-Driven Self-Iteration of Industrial Recommender Systems.arXiv preprint arXiv:2606.26859, 2026.https://arxiv.org/abs/2606.26859. 24 Advancing Model Research in AgentX on PithPith's reviewarXiv
  2. [2] Zijie Zhuang et al. From Trajectories to Evidence: Auditable Experimental Records for Industrial Research Agents.arXiv preprint arXiv:2608.05235, 2026.https://arxiv.org/abs/2608. 05235 on PithPith's reviewarXiv
  3. [3] Shaohua Liu et al. NOVA: A Verification-Aware Agent Harness for Architecture Evolution in Industrial Recommender Systems.arXiv preprint arXiv:2606.27243, 2026.https://arxiv. org/abs/2606.27243v3 on PithPith's reviewarXiv
  4. [4] HaoranLingetal. RecHarness: ABandit-RoutedAgenticHarnessforSelf-EvolvingRecommender Systems.arXiv preprint arXiv:2607.29241, 2026.https://arxiv.org/abs/2607.29241 on PithPith's reviewarXiv
  5. [5] Muhammad Rafay Azhar et al. CORAL: An LLM-Native Harness for Production Recommender Systems.OARS Workshop at RecSys 2026, accepted, 2026.https://arxiv.org/abs/2609. 02730 on Pith
  6. [6] Ming Li et al. Auto-RecSys: Harnessing Autonomous Research Agents for Industry-Scale Recom- mender System.arXiv preprint arXiv:2609.10922v1, 2026.https://arxiv.org/abs/2609. 10922v1 on PitharXiv
  7. [7] Moritz Baumgart et al. AutoRecLab: Describe the Experiment, Get the Code!RecSys 2026, Demo Track, to appear. arXiv:2609.21863v1, 2026. DOI (forthcoming): 10.1145/3773078.3841273. https://arxiv.org/abs/2609.21863v1 on PithPith's reviewarXiv
  8. [8] Yifan Zhang et al. Agora: Git as Shared Memory for Collective AutoResearch.arXiv preprint arXiv:2609.18094v1, 2026.https://arxiv.org/abs/2609.18094v1 on PithPith's reviewarXiv
  9. [9] Qi Zhang et al. AutoLR: Automating the Path from Research to Launch Review in Industrial Recommender Systems.arXiv preprint arXiv:2609.04871v1, 2026. https://arxiv.org/ abs/2609.04871v1 on PithPith's reviewarXiv
  10. [10] Haochen Wang, Yi Wu, Daryl Chang, Li Wei, and Lukasz Heldt. Self-Evolving Recommendation System: End-To-End Autonomous Model Optimization With LLM Agents.RecSys 2026, Industry Track, to appear. arXiv:2602.10226v3, 2026.https://arxiv.org/abs/2602.10226v3 on PithPith's reviewarXiv
  11. [11] Yuhao Wu et al. HarnessDev: Can LLMs Create and Evolve Their Own Agent Harness?arXiv preprint arXiv:2609.01437v1, 2026.https://arxiv.org/abs/2609.01437v1 on PithPith's reviewarXiv
  12. [12] Alexander Novikov et al. AlphaEvolve: A coding agent for scientific and algorithmic discovery. arXiv preprint arXiv:2506.13131, 2025.https://arxiv.org/abs/2506.13131 on PithPith's reviewarXiv
  13. [13] Eser Aygün et al. An AI system to help scientists write expert-level empirical software.Nature, 654:909–916, 2026. DOI: 10.1038/s41586-026-10658-6.https://arxiv.org/abs/2509. 06503v3 on Pithdoi:10.1038/s41586-026-10658-6.https://arxiv.org/abs/2509
  14. [14] Edan Toledo et al. AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench.Advances in Neural Information Processing Systems, 38, 2025. https://arxiv.org/abs/2507.02554v2 on PitharXiv
  15. [15] Thomas Simon Foster et al. AI Research Preference Models.arXiv preprint arXiv:2608.13940v2, 2026.https://arxiv.org/abs/2608.13940v2 on PithPith's reviewarXiv
  16. [16] Tong Zheng et al. Dream-RSI: Recursive Self-Improvement through Evolving Worlds.arXiv preprint arXiv:2609.14858v1, 2026.https://arxiv.org/abs/2609.14858v1. 25 Advancing Model Research in AgentX on PithPith's reviewarXiv
  17. [18] Songpei Xu et al. Climber: Toward Efficient Scaling Laws for Large Recommendation Models. CIKM, pages 6193–6200, 2025. DOI: 10.1145/3746252.3761561. https://arxiv.org/ abs/2502.09888 on PitharXiv
  18. [19] Mingming Ha et al. UniMixer: A Unified Architecture for Scaling Laws in Recommendation Systems.arXiv preprint arXiv:2604.00590, 2026.https://arxiv.org/abs/2604.00590 on PitharXiv
  19. [20] Ziang Lu et al. From Clues to Generation: Language-Guided Conditional Diffusion for Cross- Domain Recommendation.SIGIR, pages 1266–1276, 2026. DOI: 10.1145/3805712.3809563. https://arxiv.org/abs/2604.05365 on PitharXiv
  20. [21] Yufei Ye et al. FuXi-𝛽: Towards a Lightweight and Fast Large-Scale Generative Recommendation Model.arXiv preprint arXiv:2508.10615, 2025.https://arxiv.org/abs/2508.10615 on PithPith's reviewarXiv
show 14 more
  1. [22] Junwei Yin et al. DOS: Dual-Flow Orthogonal Semantic IDs for Recommendation in Meituan.Proceedings of the ACM Web Conference 2026 (WWW ’26), 4 pages, 2026. DOI: 10.1145/3774904.3792845.https://arxiv.org/abs/2602.04460 on PitharXiv
  2. [23] Junchen Fu et al. Differentiable Semantic ID for Generative Recommendation.SIGIR, pages 369–379, 2026. DOI: 10.1145/3805712.3809641.https://arxiv.org/abs/2601.19711 on PitharXiv
  3. [24] Yukun Zhang et al. SMES: Towards Scalable Multi-Task Recommendation via Expert Sparsity. KDD ’26, pages 8522–8531, 2026. DOI: 10.1145/3770855.3818483.https://arxiv.org/ abs/2602.09386 on PitharXiv
  4. [25] Jin-Duk Park and Won-Yong Shin. Memory Is No Longer a Bottleneck: Memory-Efficient Graph Filtering for Scalable Collaborative Filtering.IEEE Transactions on Knowledge and Data Engineering, 38(8):5269–5281, 2026. DOI: 10.1109/TKDE.2026.3704986.https://arxiv. org/abs/2606.21540 on PitharXiv
  5. [26] QingyunLiuetal. TokenMinds: PretrainedUserTokensandEmbeddingsforUserUnderstanding in Large Recommender Systems.arXiv preprint arXiv:2606.25147, 2026.https://arxiv. org/abs/2606.25147 on PithPith's reviewarXiv
  6. [27] Weidi Pan et al. RecEvolve: A Knowledge-Driven Autonomous Agent System for Recommender Systems.arXiv preprint arXiv:2609.01622, 2026.https://arxiv.org/abs/2609.01622 on PithPith's reviewarXiv
  7. [28] Jinxin Hu et al. Astar: Learning to Propose Evolution Directions for Self-Evolving Industrial AI Systems.arXiv preprint arXiv:2608.27287, 2026.https://arxiv.org/abs/2608.27287 on PithPith's reviewarXiv
  8. [29] Dongyang Ao, Kaixiang Fang, and Shijie Xu. RecSys Factory: Bounding LLM Agent Autonomy to Decision Points in the Industrial Recommender Lifecycle.arXiv preprint arXiv:2608.11241, 2026.https://arxiv.org/abs/2608.11241 on PithPith's reviewarXiv
  9. [30] Yang Xiao et al. PILOT in the Loop: Live Self-Improvement for Long-Horizon Agents.arXiv preprint arXiv:2608.26530, 2026.https://arxiv.org/abs/2608.26530 on PithPith's reviewarXiv
  10. [31] Zibo Zhao et al. RobustSGPO: Search-Space Control for Agent Harness Evolution.arXiv preprint arXiv:2609.09646v2, 2026.https://arxiv.org/abs/2609.09646v2. 26 Advancing Model Research in AgentX on PithPith's reviewarXiv
  11. [32] Guoxin Chen et al. Toward Autonomous Long-Horizon Engineering for ML Research.arXiv preprint arXiv:2604.13018, 2026.https://arxiv.org/abs/2604.13018 on PithPith's reviewarXiv
  12. [33] Mingming Zhao et al. ScienceFlow: A Long-Horizon Agent for ML Research, Scientific Discov- ery and Beyond.arXiv preprint arXiv:2608.14354, 2026.https://arxiv.org/abs/2608. 14354 on PithPith's reviewarXiv
  13. [34] Shangheng Du et al. MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery.arXiv preprint arXiv:2606.06473, 2026. https://arxiv.org/abs/ 2606.06473 on PithPith's reviewarXiv
  14. [35] Yuchen Ma et al. SkillGen: Verified Inference-Time Agent Skill Synthesis.arXiv preprint arXiv:2605.10999, 2026.https://arxiv.org/abs/2605.10999. 27 Advancing Model Research in AgentX Appendices A. Research Agent Proposal Refinement A.1. Proposal Refinement Mechanism In one cal... on PithPith's reviewarXiv
Cited by 2 Pith papers

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment