Computation and Language · arXiv:2610.17309 v2 · submitted 12 Oct 2026
StartLux-Decision: AutoThink Decision Models with Reinforcement Learning for Calibrated Deliberative Decisions
The paper says
“We introduce StartLux-Decision, decision models from 0.8B to 27B parameters that combine efficient single-pass prediction with adaptive, confidence-guided reasoning. The Instant models use bidirectional attention in full-attention layers to improve decision accuracy while keeping single-pass inference.”
more of the abstract
The AutoThink models invoke explicit reasoning for uncertain decisions and answer confident cases directly. A three-stage training pipeline establishes shared decision capabilities, specializes the two variants, and equips AutoThink with reasoning through reasoning supervision and reinforcement learning. We introduce Reinforcement Learning for Calibrated Deliberative Decisions (RLCD2), which uses a frozen model and proper scoring rules to reward improvements in decision probability quality and applies a soft penalty to overlong reasoning traces. RLCD2 raises the share of reasoning traces that terminate within the generation limit by up to 48 percentage points and cuts average reasoning length by up to 65%, while largely preserving decision accuracy. On the public subset of Decision Index 0.3, the Instant models outperform the strongest evaluated comparable-scale open-weight decision models by 7.4-16.9 points, and the 27B AutoThink model reaches 73.4. We also introduce DecisionPivot, a programmatically verifiable benchmark of computational decisions built from contrast pairs that differ in a single answer-changing detail. On DecisionPivot, AutoThink outperforms its Instant counterparts by 11.5-18.4 points in pair accuracy and leads among evaluated systems. The models can be run using frameworks such as vLLM and MLX, with quantized versions available to reduce memory usage.Inference code and links to model weights are available at https://github.com/StartLuxLabs/Startlux-Decision .
from the abstract
Pith says
Machine review · deepseek-v4-flash · 14 Oct 2026 · of version 1 of the paper; version 2 is newer · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up. A thorough, well-engineered paper with strong empirical support, though the frozen-reward staleness concern is acknowledged but not investigated. This line is wrong
3 major objections · 6 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the references say.
The paper says, at §3.1, Eq. (5); §3.2; §3.3
The review did not quote this passage.
Pith says: Objection 1 of 3 · major
The RLCD² reward uses a frozen copy of the starting model's probability readouts (p0 and p1). As the reasoning policy shifts during optimization, the distribution of generated traces may diverge from that of the frozen model, potentially making the reward signal stale.
more of objection 1
The paper does not analyze whether the frozen model's probabilities remain aligned with the current policy's decision quality, nor does it discuss the stability of the reward under distribution shift. The empirical improvements on held-out and DecisionPivot are encouraging, but the theoretical concern about reward validity under policy drift remains unaddressed. Please add a discussion or analysis of this potential issue, either by measuring alignment drift across training or by bounding it.
The paper says, at §4.1; §4.2
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.65):
“Nevertheless, calibration remains sensitive to distribution shift: maps fitted on the held-out distribution exhibit residual overconfidence of up to 6.1 percentage points when transferred to unseen datasets.”
Pith says: Objection 2 of 3 · major
The public subset of Decision Index 0.3 served as a development benchmark for model selection (data recipes, attention variants, seed averages). The authors acknowledge this, stating that DI scores reflect model selection. This fact reduces the strength of the DI 0.3 claims of outperforming reference models by 7.4–16.9 points, because the comparison is on a set that was used to guide develoment.
more of objection 2
The independent evaluation is on DecisionPivot, which was not used for any development choice. The manuscript should be explicit about this interpretation in the abstract and conclusion, not just in §4.1, to avoid overclaiming.
The paper says, at §5.2, Figure 13; §3.4, Table 3
The review did not quote this passage.
Pith says: Objection 3 of 3 · major
The fusion weight α=0.75 and the soft reasoning budget B=3,072 are optimized on held-out data from the training distribution. While this is standard practice, the sensitivity of the results to the choice of held-out set is not explored. For instance, the fusion weight that minimizes Brier on held-out data reduces DI 0.3 accuracy at the 27B scale (calibration improves but accuracy changes little).
more of objection 3
The trade-off between calibration and accuracy is acknowledged, but the authors should verify that the chosen parameters are not brittle across different held-out distributions (e.g., a separate validation set from DecisionPivot). Please provide a brief sensitivity analysis or discuss the stability of these hyperparameters.
6 minor objections
The paper says, at Figure 5, y-axis label 'Long computations'
Pith says: minor objection 1 of 6
The axis label 'Long computations' is ambiguous; it should read 'DecisionPivot pair accuracy (%)' or similar to match the table and text.
The paper says, at Table 3, column header 'Default accuracy'
Pith says: minor objection 2 of 6
The caption says 'Default-answer accuracy' but the table header abbreviates it. Please be consistent.
show 4 more
The paper says, at §5.4, first paragraph
Pith says: minor objection 3 of 6
The sentence 'Selection changes the conditional distribution of returned predictions.' could be improved by adding a reference to the optimizer's curse mentioned later.
The paper says, at Appendix D.1, Table D.1
Pith says: minor objection 4 of 6
The table footnot that 'best scores are bolded' but some benchmarks have ties. Consuder clarifying that bold indicates the best score among the listed models, with ties marked if necessary.
The paper says, at §2.3.3
Pith says: minor objection 5 of 6
The paragraph on 'Reasoning Supervision' mentions 'approximately 4,000–21,000 supervised reasoning examples per model'. The large range suggests different model sizes; please specify which size uses which approximate count.
The paper says, at §3.4, Table 3
Pith says: minor objection 6 of 6
The row 'Decision-weighted score, budget 3,072 (release)' appears twice for the 9B model (once after the 4B section and again in the 9B section). The formatting suggests the table may have been misaligned; check that all rows correspond to the correct model scale.
What would settle it
On the 2,986 held-out examples used in Table 2, compute two reward variants for each sampled trace: (1) the original frozen-model reward, and (2) a reward computed using the current (post-RL) policy’s own probability readouts instead of the frozen copy. Compare the rankings of traces within each group by the two rewards using Spearman correlation. If the average within-group Spearman ρ falls below 0.85, or if the fraction of items where the top reward trace changes exceeds 10%, the frozen reward introduces systematic bias that could degrade the policy.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 14 Oct 2026
- Reader holds up
- Skeptic agrees
- Editor wrote the verdict above
- Referee minor revision: 3 major objections, 6 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
This is a substantial piece of work. The central idea—a single-pass decision model with a confidence gate that routes hard cases to explicit reasoning, trained with an RL method that rewards improvements in probability quality rather than just answer correctness—is novel and well-motivated. The engineering is extensive: five model scales, bidirectional attention for Instant, a three-stage training pipeline, and a new benchmark (DecisionPivot) that tests contrastive reasoning. The empirical work is the paper’s strongest suit: thorough ablations, calibration studies, latency comparisons, and quantized deployment support.
What the paper does well: it ships. Models on HF, code on GitHub, inference across three backends. The RLCD² analysis (Table 2, Figure 5) convincingly shows gains in termination rate and reasoning length while holding accuracy. The policy-aware calibration section (Section 5.5) is a nice bonus—it shows the authors thought beyond benchmark scores.
Soft spots. The stress-test concern about the frozen reward model in RLCD² is real and the paper does not address it. The reward is a proper-score difference from a frozen copy; if the policy’s trace distribution shifts far enough, the frozen model’s probability readouts could become misleading guides. The paper shows the method works in the examined settings, but it would be stronger with an analysis of reward stability under drift. Also, the DI public subset was used as a development set (the authors are transparent about this), meaning those scores reflect model selection, not independent test results. DecisionPivot is independent, but it is the authors’ own benchmark—that does not undermine the results, but it means the strongest reported numbers come from a self-constructed test.
Overall, the paper is solid, not revolutionary. The architecture and training recipe are practical contributions that will be useful to the decision-model community. The paper deserves a serious referee. I would bring it to reading group and would cite it if I work on reasoning-efficient decision models.
The strongest objection
The RLCD^2 reward is computed using a frozen copy of the starting model’s probability readouts (Section 3.1, Eq. 5). The reasoning policy is optimized based on how much the frozen model’s probabilities improve after reading the generated trace. As the policy parameters change, the distribution of generated traces shifts away from that of the starting model. The frozen model’s probability estimates may then become misaligned with the actual decision quality of the current policy. The paper does not measure or discuss this drift. If the reward becomes stale, the policy could be optimized toward improving the frozen model’s scores without corresponding gains on the current model’s own outputs. The held-out accuracy and termination improvements are encouraging, but they do not rule out a latent degradation in the reliability of the reward signal. This concern is distinct from the length penalty or score choice; it concerns the fundamental validity of using an unchanging scorer on a moving distribution.
The reader correctly identified the frozen reward model as the weakest assumption. The paper’s central claim of state-of-the-art performance is well supported by independent evaluation on DecisionPivot and strong results on the public DI subset. The concern about reward staleness is theoretical and does not undermine the presented empirical results, which show clear improvements from RLCD^2. However, it is a genuine gap that limits understanding of the method’s robustness. I agree with the reader’s assessment that the overall merits justify acceptance, but the medium correctness risk is warranted. No verdict change is needed; the existing verdict already reflects this nuance.
The referee's report
If the results hold, StartLux-Decision advances the state of open-weight decision models by combining efficient single-pass inference with adaptive reasoning through a unified probability interface. RLCD² provides a principled method for optimizing reasoning policies for decision quality rather than answer correctness, and demonstrates significant gains in reasoning termination and efficiency. The DecisionPivot benchmark offers a useful complement to existing benchmarks by emphasizing sensitivity to single-detail changes. The policy-aware calibration analysis is thorough and addresses a real practical issue. The release of models and code facilitates independent scrutiny and application.
Pith's reading of the paper (a machine's summary, not the authors')
The paper introduces StartLux-Decision, a family of decision models (0.8B to 27B) that pair an efficient single-pass predictor (Instant) with a variant that selectively invokes explicit reasoning for uncertain decisions (AutoThink). To train the reasoning variant, the authors propose Reinforcement Learning for Calibrated Deliberative Decisions ($RLCD^{2}$), which rewards improvements in probabilistic decision quality—measured via proper scoring rules—rather than answer correctness alone, and imposes a soft penalty on overly long reasoning traces. $RLCD^{2}$ increases termination rates by up to 48 percentage points and reduces average reasoning length by up to 65% while largely preserving held-out accuracy. On the public subset of Decision Index 0.3, the Instant models outperform all comparable-scale open-weight decision models by 7.4–16.9 points. On the new DecisionPivot benchmark, which uses contrast pairs that differ in a single answer-changing detail, the 27B AutoThink model achieves 86.9% pair accuracy, the highest among all evaluated systems, including much larger models.
What carries the argument. The key machinery is $RLCD^{2}$ (Reinforcement Learning for Calibrated Deliberative Decisions), which extends decision-oriented RL to explicit reasoning. It uses a frozen copy of the model to compute a reward equal to the improvement in a decision-weighted proper scoring rule between pre- and post-reasoning probabilities, minus a soft length penalty. The policy is optimized with group-relative policy gradients (based on GRPO). The AutoThink variant employs a confidence gate (threshold 0.9) to decide whether to reason; when reasoning is invoked, the initial and post-reasoning probability distributions are fused via a weighted average (75% post-reasoning). The Instant variant uses bidirectional attention in full-attention layers to improve single-pass accuracy.
Core claim. The central discovery is that a decision model can be trained to efficiently allocate reasoning only when its initial confidence is low, using a reinforcement learning reward that explicitly measures improvement in probabilistic calibration after reasoning. This approach, $RLCD^{2}$, uses a frozen copy of the model to compute the difference in a decision-weighted proper scoring rule between pre- and post-reasoning probability distributions, combined with a soft penalty on tokens exceeding a budget. The resulting AutoThink models preserve the speed of single-pass prediction on confident cases and deliver substantial accuracy gains on computation-intensive decisions, all while operating at parameter scales (0.8B to 27B) that outperform larger reference models. The authors also introduce DecisionPivot, a programmatically verifiable benchmark that tests sensitivity to subtle answer-changing variations.
The premise it rests on. The training reward in $RLCD^{2}$ uses a frozen copy of the model's probability estimates as a constant signal throughout policy optimization; if the actively trained policy's distribution diverges from the frozen model's, this signal may become stale or misleading, potentially degrading decision quality or calibration, and the paper does not analyze the stability of this setup under distribution shift.
If the paper is right (Pith's inferences, not the authors')
The 4B Instant model outperforms all reference decision models on Decision Index 0.3, including much larger systems, showing that efficient single-pass architectures can be competitive when trained with bidirectional attention and the right data.
AutoThink consistently beats its Instant counterpart by 11.5–18.4 percentage points on DecisionPivot, demonstrating that adaptive reasoning is especially valuable for computation-intensive decisions like arithmetic, scheduling, and procedural verification.
RLCD^2 improves termination rate by up to 48 percentage points and reduces mean reasoning length by up to 65% at 9B scale, offering a practical method to deploy reasoning models without excessive latency or resource use.
Policy-aware calibration reduces confidence-accuracy gaps to within 2.7 percentage points across four decision pipelines, addressing a previously overlooked source of miscalibration from selection and routing.
The RLCD^2 reward relies on a frozen model's probability estimates; if the trained policy diverges substantially, the frozen readout may become a stale signal, an open stability question not addressed in the paper.
DecisionPivot's programmatic generation framework could be extended to create unlimited contrastive decision pairs, potentially providing more robust and less cheat-prone benchmarks for decision model evaluation.
The confidence gate threshold and fusion weight are fixed after tuning on held-out data; adapting these parameters dynamically per task or per instance could further improve the accuracy-efficiency trade-off.
The strong performance of small Instant models suggests that many decision tasks do not require reasoning; a hybrid deployment could use a fast Instant model as a first pass and only escalate to larger or reasoning models when confidence is low.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
We introduce StartLux-Decision, decision models from 0.8B to 27B parameters that combine efficient single-pass prediction with adaptive, confidence-guided reasoning. The Instant models use bidirectional attention in full-attention layers to improve decision accuracy while keeping single-pass inference. The AutoThink models invoke explicit reasoning for uncertain decisions and answer confident cases directly. A three-stage training pipeline establishes shared decision capabilities, specializes the two variants, and equips AutoThink with reasoning through reasoning supervision and reinforcement learning. We introduce Reinforcement Learning for Calibrated Deliberative Decisions (RLCD2), which uses a frozen model and proper scoring rules to reward improvements in decision probability quality and applies a soft penalty to overlong reasoning traces. RLCD2 raises the share of reasoning traces that terminate within the generation limit by up to 48 percentage points and cuts average reasoning length by up to 65%, while largely preserving decision accuracy. On the public subset of Decision Index 0.3, the Instant models outperform the strongest evaluated comparable-scale open-weight decision models by 7.4-16.9 points, and the 27B AutoThink model reaches 73.4. We also introduce DecisionPivot, a programmatically verifiable benchmark of computational decisions built from contrast pairs that differ in a single answer-changing detail. On DecisionPivot, AutoThink outperforms its Instant counterparts by 11.5-18.4 points in pair accuracy and leads among evaluated systems. The models can be run using frameworks such as vLLM and MLX, with quantized versions available to reduce memory usage.Inference code and links to model weights are available at https://github.com/StartLuxLabs/Startlux-Decision .
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-14
The authors' main claim, as the classifier picked it out 0.41
“On the public subset of Decision Index 0.3, the Instant models outperform the strongest evaluated comparable-scale open-weight decision models by 7.4-16.9 points, and the 27B AutoThink model reaches 73.4.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.06 · states limitation 0.17 · human participants 0.04 · controlled comparison 0.77 · randomized assignment 0.01 · states quantitative result 0.99 · compares with prior results 0.96 · mentions code or data availability 0.14
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| We introduce StartLux-Decision, decision models from 0.8B to 27B parameters that combine efficient single-pass prediction with adaptive, confidence-guided reasoning. | contribution 1.00 | 0.94 | 0.16 | asserted 1.00 |
| The Instant models use bidirectional attention in full-attention layers to improve decision accuracy while keeping single-pass inference. | method 0.95 | 0.71 | 0.04 | asserted 1.00 |
| The AutoThink models invoke explicit reasoning for uncertain decisions and answer confident cases directly. | method 0.84 | 0.60 | 0.06 | asserted 1.00 |
| A three-stage training pipeline establishes shared decision capabilities, specializes the two variants, and equips AutoThink with reasoning through reasoning supervision and reinforcement learning. | method 0.98 | 0.91 | 0.05 | asserted 1.00 |
| We introduce Reinforcement Learning for Calibrated Deliberative Decisions (RLCD$^2$), which uses a frozen model and proper scoring rules to reward improvements in decision probability quality and applies a soft penalty to overlong reasoning traces. | method 0.65 | 0.93 | 0.08 | asserted 1.00 |
| RLCD$^2$ raises the share of reasoning traces that terminate within the generation limit by up to 48 percentage points and cuts average reasoning length by up to 65%, while largely preserving decision accuracy. | result 1.00 | 0.95 | 0.22 | asserted 0.97 |
| On the public subset of Decision Index 0.3, the Instant models outperform the strongest evaluated comparable-scale open-weight decision models by 7.4-16.9 points, and the 27B AutoThink model reaches 73.4. | result 1.00 | 0.94 | 0.22 | asserted 1.00 |
| We also introduce DecisionPivot, a programmatically verifiable benchmark of computational decisions built from contrast pairs that differ in a single answer-changing detail. | contribution 0.96 | 0.94 | 0.05 | asserted 0.99 |
| On DecisionPivot, AutoThink outperforms its Instant counterparts by 11.5-18.4 points in pair accuracy and leads among evaluated systems. | result 1.00 | 0.94 | 0.15 | asserted 1.00 |
| The models run on vLLM, llama.cpp and MLX, with quantized variants for lower memory use. | method 0.81 | 0.51 | 0.02 | asserted 1.00 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.84
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-14: 1 request, 7,757 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (15)
Figures are the authors', reproduced from the paper.
Cited by 7 Pith papers
Reviewed papers on Pith that cite this one. The full citation record
- ECGJev: Question-Conditioned Option Scoring for 12-Lead ECG Interpretation cs.LG · Nov 2026
- Joint or Per-Option? Compact Encoders for Closed-Set Decisions from 17 Million to 1 Billion Parameters cs.CL · Oct 2026
- Piccolo: Co-Evolving Decision Models and Data through Recursive Self-Improvement cs.CL · Nov 2026
- Think Once, Act Fast: Cost-Effective Recursive Self-Improvement for Decision Model Harness cs.AI · Nov 2026
- SO-Arena: Benchmarking the Capability Spectrum and Reliability of System One Models cs.CL · Nov 2026
- Decoupled Predictive Decision Making: A Shared Conditional Predictor for Agents cs.AI · Nov 2026
- Understanding Jev-Style System One Models: Does Structure Reduce Selection Bias? cs.CL · Oct 2026
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment