Pith.

Artificial Intelligence · arXiv:2610.22066 v1 · submitted 15 Oct 2026 · 58 references

Prism-1: Position- and Rival-Independent Scoring for System One Models

Yao Long Teng, Jianguo Huang, Yanchen Deng, Jiayi Cai, Bo An

The paper says

“Given a state and a set of options, such as intents, tools or interface elements, System One decision models return a distribution over the options in one forward pass, without generated reasoning.”

more of the abstract

Web systems increasingly build them by reading decisions from language models. Two axioms of rational choice are natural requirements for such decisions, because in web systems the listing and the option set are determined by other components and can change or be manipulated: the order in which the options are listed should not matter, and adding an option should not change the relative odds of the others (Luce's choice axiom). We test the standard reads, namely single-choice question (SCQ) prompting, with shuffled-order training, order averaging or sorted options, and invariant set architectures, and find that each violates at least one axiom: one shuffle changes 12-20% of SCQ answers, sorting only freezes an arbitrary convention, and a well-chosen added option steers 4-34% of decisions. By Luce's theorem, a read satisfies both axioms only if it is a softmax over scores that each depend on one option alone, so in a transformer they hold by construction only if no token that scores an option sees another. We implement this as Prism-1 (Position- and Rival-Independent Scoring for System One Models): every option is read in its own block at shared positions, with the model's usual left-to-right attention, in one forward pass. On five benchmark sets with Qwen3-8B and Gemma 4 12B, SCQ prompting trained on the same data still changes 7% of its answers under one shuffle, and an added irrelevant option steers 12% of its decisions. Prism-1 stays within one point of that baseline's accuracy and is 2-4 points more accurate once only answers that survive a reordering are counted, while no shuffle changes any of its answers and no added irrelevant option moves its choice among the others.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 16 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. A provable, by-construction solution to two choice axioms for System One LLM reads, with a broad but not yet reproducible empirical package. This line is wrong

2 major objections · 7 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 58 references say.

The paper says, at §5.1, §7, Appendix B

The review did not quote this passage.

Pith says: Objection 1 of 2 · major

The empirical half of the claim — the parity of Table 1 and the order-robust accuracy gains — is not independently verifiable as submitted: no code, data pipeline, or trained checkpoints are released, and the Set-LLM baseline is a private re-implementation specified only in the 'Baselines' paragraph of Appendix B.

more of objection 1

The manuscript's detail elsewhere (exact model and dataset revisions, the SDPA causality caveat, the unpacked-reference check for Set-LLM) is exemplary and mitigates this, but I ask for an artifact release, or at minimum a complete public specification of the Set-LLM re-implementation and an explicit availability statement in the paper. This is a verifiability request: it does not call the axiomatic results or the architectural guarantee into question.

The paper says, at §5.1, §5.6, Table 1

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.62):

“Taking it mainly from the twin would need the same selective shift that lets an added option steer the choice.Prism-1trails SCQ on the held-out JevBench by 3.6 and 4.1 points, a task missing from our training data, and a read learned in fine-tuning may transfer less well than the letter read the models were instruction-tuned on.”

Pith says: Objection 2 of 2 · major

The headline that the choice axiom 'costs little or no accuracy' rests on one training budget (one epoch, LoRA rank 16 and alpha 32, 9,100 questions, §5.1), and the held-out JevBench gap (3.6 and 4.1 points behind SCQ, §7) shows that the parity is task-dependent.

more of objection 2

The claims are honestly scoped to 'in our experiments', which I credit, but one additional budget point — for example a second epoch or a different LoRA rank — would substantially strengthen the generalization of the claim; alternatively, the abstract and conclusion should state explicitly that parity is demonstrated at the evaluated budget only.

7 minor objections

The paper says, at §2

Pith says: minor objection 1 of 7

In Definition 2.2 and Proposition 2.3 the summation over S renders as an unrecognizable product-like symbol; the sums should display as standard Σ operators. The intended meaning is clear from the proofs, but the rendering should be fixed.

The paper says, at Abstract, §1

Pith says: minor objection 2 of 7

The abstract's 'one shuffle changes 12–20% of SCQ answers' mixes two conditions reported separately in §1 and Table 1 (20.0% and 11.9% untrained; 7.1% and 7.4% after training); please state which condition the 12–20% figure refers to.

show 5 more

The paper says, at §4

Pith says: minor objection 3 of 7

The layout description says every block starts at position P, but the mechanism by which position IDs are reset to P for each block inside the packed forward pass (for learned or rotary position encodings) is left implicit; one sentence on the implementation would remove the only ambiguity in the construction.

The paper says, at Appendix B

Pith says: minor objection 4 of 7

The 'Attention backend' finding that Gemma 4's SDPA path changes earlier logits by up to 17.6 when a token is appended is important for reproducibility; adding a sentence on how this was detected (for example, by comparing logits against an unpacked causal reference, as was done for Set-LLM) would make the check auditable.

The paper says, at §5.6, Table 4

Pith says: minor objection 5 of 7

The comparison-head gain (+0.9 and +0.7 points, two seeds) is the largest positive deviation from Prism-1 in Table 4, and the claim that it is 'beyond noise' would be easier to assess if the per-seed range were reported.

The paper says, at Abstract, §5.2

Pith says: minor objection 6 of 7

The abstract's '2–4 points more accurate' for order-robust accuracy sits slightly oddly with the 1.6-point gap on Gemma reported in §5.2; verify that the rounding of the two claims is consistent.

The paper says, at §2, §7

Pith says: minor objection 7 of 7

Since the suitability of the axioms is the main normative precondition for the whole approach, consider moving the scope discussion of §7 (near-duplicates, 'none of the above', 'odd one out') into the introduction; the early one-sentence marker in §2 is easy for a reader to miss.

What would settle it

Release the Prism-1 implementation and run an equivalence check on a random sample of questions: compute each option's score in the packed forward pass and in an isolated causal pass over the shared context followed by that option alone, then compare the final hidden states and scores at each block's last token. If they agree to numerical precision, the by-construction guarantee is verified in the shipped code; if they differ, the central guarantee does not hold as implemented.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 16 Oct 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic partly agrees 8 min 47 s · 25,656 tokens read, 22,845 written
  3. Editor wrote the verdict above 44 s · 27,240 tokens read, 1,093 written
  4. Referee minor revision: 2 major objections, 7 minor 3 min 21 s · 26,014 tokens read, 25,704 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Main thing to know: this paper has a real architecture-level result, and the proof part should be taken seriously. Proposition 2.4 is a clean characterization: a positive decision rule is order-invariant and satisfies Luce's choice axiom in every context iff it is a softmax over scores that depend on each option alone. The proof in Appendix A is sound, and so is the accompanying induction showing that Prism-1's block-masked layout makes each option's hidden state identical to an isolated causal pass. So the two axioms hold by construction, not by hope. That is genuinely new framing for transformer-based decision reads, and it points at something useful: if you want both guarantees, the architecture must prevent any token that scores an option from seeing another option. The empirical work is better than the usual for this area. Five benchmark sets, two open models, closed commercial APIs, an explicit noise floor for those APIs, significance testing, ablations that vary attention, read position, and loss, and a limitation section that actually names the cases where the axiom is the wrong requirement: near-duplicates, "none of the above", "odd one out", and state injection. The reader and stress-test both looked for a load-bearing flaw in the central claim and did not find one. I agree. Soft spots, in proportion. First, no code or data is shipped, and Set-LLM is a private re-implementation, so the specific accuracy and robustness numbers cannot be independently checked yet. Second, parity with trained SCQ is demonstrated at one training budget: one epoch, fixed LoRA settings. That is a normal starting point, not a fatal gap, but it limits what the accuracy comparison can support. Third, the abstract's "2-4 points more accurate" is easy to misread; in source-order accuracy Prism-1 is tied with SCQ, and the gain is specifically on answers that survive a reordering. The paper says this clearly in the body, but the framing is a bit generous. Finally, the normative premise of the choice axiom is a real scope condition: it fits routing, intent, and tool selection, and it deliberately rules out comparison-dependent tasks. The authors concede this directly, so it is a scoped limitation rather than an unaddressed one. Who this is for: anyone building decision reads from LLMs for web components, and anyone working on permutation-invariant prompting. It deserves a serious referee. I would send it to review and ask for code and data as a condition of acceptance; the axiomatic and architectural contribution stands on its own, and the empirical parity claims are worth verifying rather than dismissing.

The strongest objection

I read in good faith for a flaw in the central claim: that Proposition 2.4 characterizes order-invariant rules satisfying Luce's choice axiom, and that Prism-1 satisfies both axioms by construction. I do not find one. The proof in Appendix A is a clean application of Luce's theorem: the ratio argument in Equation (4) is correct, and the two-option sets fix the scale up to a common context term, so the score depends only on the state, question, and option. The transformer isolation argument is also sound: with the stated block-masked attention and shared starting position, each block's hidden states equal those of an isolated causal pass over the context followed by that option, so masked rivals contribute exactly zero and no score sees another option. The paper explicitly acknowledges the normative limits of the choice axiom in Section 7: near-duplicates take probability proportionally rather than from the twin, comparison-defined tasks such as 'none of the above' and 'odd one out' are outside the design, and state injection reaches every option's score. These are scoped conditions, not unaddressed gaps. The residual risks are reproducibility, because no code is shipped and Set-LLM is a private re-implementation, and the fact that parity with SCQ is demonstrated at a single training budget. Both are already reflected in the reader's CONDITIONAL verdict. I have no additional load-bearing concern.

The paper's mathematical core is standard choice theory applied cleanly, and the architecture-level argument is sound: Prism-1 computes scores of exactly the form Proposition 2.4 requires, so both axioms hold by construction. The empirical work is broad, with five benchmark sets, two models, multiple baselines, significance tests, and an unusually candid limitations section. The reader's weakest assumption, the normative status of Luce's choice axiom, is a genuine scope limitation, but the paper states it explicitly and frames the contribution conditionally, so I do not treat it as a load-bearing defect. I partially agree with the reader because the scope risk is real for comparison-defined tasks, but I disagree that it is the weakest point of the argument. The actual residual risk is empirical reproducibility: no code is released, Set-LLM is a private re-implementation, and the parity claim is measured at one training budget. That risk is already reflected in the CONDITIONAL verdict, so no adjustment is needed.

The referee's report

The central claim is sound and the paper is honestly executed. The axiomatic characterization is rigorous: the ratio argument in Equation (4) fixes the scale up to a common context term, and both directions of Propositions 2.3 and 2.4 are proved in full in Appendix A; the softmax form follows from a cited theorem, not from fitting, so the derivation has no fitted freedom. The block-isolation induction (Appendix A) is a genuine architectural insight: masked keys contribute exactly zero, and the shared starting position makes each block's computation identical to an isolated causal pass. The evaluation earns credit for exact McNemar tests, the noise-floor analysis for the hosted APIs (Table 6), the discovery and handling of Gemma 4's non-causal SDPA attention path, and negative results (gated channels, comparison heads) reported with per-seed detail rather than asserted. The main scope condition is the normative premise that Luce's axiom is the right requirement for web decisions; the paper states that this is a requirement, not a law, and Section 7 flags the near-duplicate case and comparison-defined tasks.

Pith's reading of the paper (a machine's summary, not the authors')

The paper asks what a single-pass decision model — one that returns a distribution over options without generating any reasoning — must look like if its answer must not change when the options are reordered or when an extra option is added. It proves (Proposition 2.4) that these two requirements, order invariance and Luce's choice axiom, leave exactly one possible form: a softmax over scores that each depend on the state, the question and one option alone, so in a transformer the requirements hold by construction only when no token that scores an option ever attends to a different option. The standard reads all violate this: one shuffle changes 12–20% of single-choice-question answers, sorting only freezes an arbitrary convention, and a well-chosen added option steers 4–34% of decisions. The paper's Prism-1 reads each option in its own block at shared positions in one forward pass, giving both axioms for free while matching trained single-choice prompting within one accuracy point and beating it by 1.6–3.7 points once only reordering-surviving answers are counted; this matters because web systems assemble option lists from registries, retrievers and page layouts that change or can be manipulated, so invariance by design rather than by luck is a concrete reliability and security property.

What carries the argument. The load-bearing identity is Proposition 2.4, a corollary of Luce's theorem: order invariance plus the choice axiom together force every positive decision rule into the softmax form $$R(o \mid S) = \frac{\exp(s(o))}{\sum_{o' \in S} \exp(s(o'))}$$ over option-only scores $s$, so the two axioms and the softmax-over-isolated-scores form are the same requirement. The mechanism that makes this hold inside a transformer is block-isolated causal reading: a shared context is computed once, each option's text occupies its own block starting at the same position, the attention mask lets a block attend only to the context and to itself and never to another block, and the appendix's inductive argument shows each block's hidden states equal those of an ordinary causal pass over the context followed by that single option. The score itself is a learned binary read, the log-odds of 'yes' versus 'no' at the last token of each block, and Equation (3) derives the softmax across blocks as the posterior probability that exactly one option is correct under independent per-option judgments. The named metrics that carry the evaluation are the flip rate (answers changed by one seeded shuffle), the switch rate and probability drift (answers and probability moved among the original options when one is added), and the steering rate (worst case over ten candidate added options); all of them are zero for Prism-1 by construction.

Core claim. The paper's central claim is Proposition 2.4: a decision rule that gives every option positive probability is order-invariant and satisfies Luce's choice axiom in every context if and only if it has the form $$R(o \mid x, q, \$\sigma$) = \frac{\exp(s(x,q,o))}{\sum_{o' \in S} \exp(s(x,q,o'))},$$ where the score $s$ depends on the state, the question and the option alone, and not on the listing or the rival set. Luce's theorem supplies the direction that any axiom-compliant rule is a softmax over a per-option scale, and order invariance removes the listing from the score. The architectural consequence is that in a transformer the two axioms are guaranteed by construction exactly when no token contributing to an option's score attends to another option and the option's positions do not depend on the listing; every standard read — letter-based single-choice prompting, shuffle-order training, rotation averaging, sorted options, and invariant set architectures — violates this condition. Prism-1 is the construction that satisfies it: a shared context followed by one block per option, every block starting at the same position, causal attention within each block and no attention between blocks, with score $s(o_k) = \log P(\text{yes}\mid h,b_k) - \log P(\text{no}\mid h,b_k)$ read at the block's last token and a softmax across blocks giving the decision. The empirical claim is that the constraint costs little or nothing: across five benchmark sets and two open models, Prism-1 is statistically tied with trained SCQ (79.5% against 80.4% on Gemma 4 12B, 78.0% against 77.9% on Qwen3-8B), is 1.6–3.7 points ahead once only answers right in more than one listing are counted, changes no answer under shuffling, lets no added option move probability between the original options, and processes fewer tokens than any other read measured.

The premise it rests on. The load-bearing premise is normative rather than empirical: that Luce's choice axiom — adding an option must not change the relative odds of the other options — is the right requirement to impose on a System One decision, a premise the paper itself qualifies by conceding that a near-duplicate option should arguably draw probability mostly from its twin, and that tasks such as 'none of the above' or 'odd one out', whose answer is defined by the other options, lie outside its design.

If the paper is right (Pith's inferences, not the authors')

A decision read that satisfies both axioms must be a softmax over option-only scores, so any transformer read in which an option's scoring token attends to a different option is axiom-compliant only if the model happens to ignore what it sees.

Web decisions become incrementally maintainable: a newly registered tool or intent needs only its own block given the cached shared context, and every earlier score stays valid without re-reading the set.

An injected or deceptively phrased option can win a Prism-1 decision only by being the best-scored option itself, and can never redirect the choice from one existing option to another.

The axioms cost little or no accuracy on these benchmarks: Prism-1 is statistically tied with trained single-choice prompting, ahead by 1.6 and 3.7 points in reorder-surviving accuracy, and cheapest in tokens, while mechanisms that let options compare either stayed within noise or broke the axioms.

Order-robust accuracy improves with set size: on nested Newsgroups topic sets of 5, 10 and 20 options, Prism-1 leads trained SCQ at every size by 2.4–4.1 points in reorder-robust accuracy.

One testable route toward closing the held-out gap (Prism-1 trails SCQ by 3.6–4.1 points on JevBench, which is absent from the training data) is a larger or more varied fine-tuning budget, or continued pretraining with the isolated-block objective; the paper's own explanation points at learning rather than at the architecture.

The construction transfers beyond decision reads: any ranking or selection task whose candidate list arrives from an external component — tool registries, retrievers, page layouts, recommendation pools — inherits the same two guarantees, since the Proposition 2.4 argument depends only on each candidate being judgeable on its own.

Deployments that must handle duplicate or near-duplicate options meet the main boundary of the guarantee, since a copy takes probability from every option in proportion rather than from its twin; merging near-duplicates before the read, which the paper recommends, should be part of any productisation, and the steering test itself could become a cheap standardised audit of whether a deployed decisi

A soft-masked, non-isolated read that achieved a zero flip rate and zero drift at equal accuracy would not refute Proposition 2.4, but it would show the empirical guarantees are attainable without full block isolation, narrowing the practical distance between the theorem's necessity and engineering convenience.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Given a state and a set of options, such as intents, tools or interface elements, System One decision models return a distribution over the options in one forward pass, without generated reasoning. Web systems increasingly build them by reading decisions from language models. Two axioms of rational choice are natural requirements for such decisions, because in web systems the listing and the option set are determined by other components and can change or be manipulated: the order in which the options are listed should not matter, and adding an option should not change the relative odds of the others (Luce's choice axiom). We test the standard reads, namely single-choice question (SCQ) prompting, with shuffled-order training, order averaging or sorted options, and invariant set architectures, and find that each violates at least one axiom: one shuffle changes 12-20% of SCQ answers, sorting only freezes an arbitrary convention, and a well-chosen added option steers 4-34% of decisions. By Luce's theorem, a read satisfies both axioms only if it is a softmax over scores that each depend on one option alone, so in a transformer they hold by construction only if no token that scores an option sees another. We implement this as Prism-1 (Position- and Rival-Independent Scoring for System One Models): every option is read in its own block at shared positions, with the model's usual left-to-right attention, in one forward pass. On five benchmark sets with Qwen3-8B and Gemma 4 12B, SCQ prompting trained on the same data still changes 7% of its answers under one shuffle, and an added irrelevant option steers 12% of its decisions. Prism-1 stays within one point of that baseline's accuracy and is 2-4 points more accurate once only answers that survive a reordering are counted, while no shuffle changes any of its answers and no added irrelevant option moves its choice among the others.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-16

The authors' main claim, as the classifier picked it out 1.00

“Prism-1 stays within one point of that baseline's accuracy and is 2-4 points more accurate once only answers that survive a reordering are counted, while no shuffle changes any of its answers and no added irrelevant option moves its choice among the others.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.38 · human participants 0.04 · controlled comparison 0.94 · randomized assignment 0.04 · states quantitative result 0.99 · compares with prior results 0.98 · mentions code or data availability 0.04

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Given a state and a set of options, such as intents, tools or interface elements, System One decision models return a distribution over the options in one forward pass, without generated reasoning. background 1.00 0.07 0.02 asserted 1.00
Web systems increasingly build them by reading decisions from language models. background 1.00 0.05 0.02 asserted 1.00
Two axioms of rational choice are natural requirements for such decisions, because in web systems the listing and the option set are determined by other components and can change or be manipulated: the order in which the options are listed should not matter, and adding an option should not change the relative odds of the others (Luce's choice axiom). background 0.99 0.10 0.07 asserted 0.99
We test the standard reads, namely single-choice question (SCQ) prompting, with shuffled-order training, order averaging or sorted options, and invariant set architectures, and find that each violates at least one axiom: one shuffle changes 12-20% of SCQ answers, sorting only freezes an arbitrary convention, and a well-chosen added option steers 4-34% of decisions. result 0.98 0.94 0.13 asserted 1.00
By Luce's theorem, a read satisfies both axioms only if it is a softmax over scores that each depend on one option alone, so in a transformer they hold by construction only if no token that scores an option sees another. interpretation 0.45 0.36 0.16 asserted 0.99
We implement this as Prism-1 (Position- and Rival-Independent Scoring for System One Models): every option is read in its own block at shared positions, with the model's usual left-to-right attention, in one forward pass. method 0.92 0.93 0.19 asserted 1.00
On five benchmark sets with Qwen3-8B and Gemma 4 12B, SCQ prompting trained on the same data still changes 7% of its answers under one shuffle, and an added irrelevant option steers 12% of its decisions. result 1.00 0.91 0.09 asserted 1.00
Prism-1 stays within one point of that baseline's accuracy and is 2-4 points more accurate once only answers that survive a reordering are counted, while no shuffle changes any of its answers and no added irrelevant option moves its choice among the others. result 1.00 0.95 0.75 asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.94

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-16: 1 request, 6,694 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (7)
Figure 1 of the paper
Figure 1. SCQ violates both axioms, after training and after sorting (mean over five sets). (a) Share of answers that change under
Figure 2 of the paper
Figure 2. Prism-1. (a) Every option is a block that starts at the same position 𝑃 after the shared context, and all blocks are read in one forward pass of the pretrained decoder, fine-tuned with LoRA. The score 𝑠𝑘 = log 𝑃 (yes | ℎ, 𝑏𝑘 ) − log 𝑃 (no | ℎ, 𝑏𝑘 ) is read at the last token of each block (■), a softmax over the blocks gives the decision, and training minimises − log 𝑅(𝑜𝑦 | 𝑆) for the correct option 𝑜𝑦. (b) The attention mask: the context attends causally to itself, and each block attends to the context and causally to itself, never to another option. No option’s score depends on the…
Figure 3 of the paper
Figure 3. BEV by domain: order-robust accuracy of SCQ (grey)
Figure 4 of the paper
Figure 4. Injected options: answers (%) that an added option
Figure 5 of the paper
Figure 5. The number of options and the choice axiom, with every method trained for one epoch on the same data. (a, b)
Figure 6 of the paper
Figure 6. The price of the choice axiom: accuracy relative
Figure 7 of the paper
Figure 7. What bidirectional attention among options costs. (a) Validation accuracy before and after one epoch of training, for a

Figures are the authors', reproduced from the paper.

References (58)
  1. [1] Diogo Almeida. 2026. Introducing System One Models & Jev. TypeSafe AI blog, https://typesafe.ai/blog/introducing-system-one-models-and-jev. Published 15 September 2026 on Pith
  2. [2] avbiswas. 2026. BEV Decision Mix. https://huggingface.co/datasets/avbiswas/ bev-decision. Hugging Face dataset, default configuration, revision 11ed9b5. Accessed 2026-10-12 on Pith
  3. [3] Ethan Bito, Yongli Ren, and Estrid He. 2026. One Pass, Any Order: Position- Invariant Listwise Reranking for LLM-Based Recommendation. InProceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval on Pith
  4. [4] Ralph Allan Bradley and Milton E. Terry. 1952. Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons.Biometrika39, 3/4 (1952), 324–345 on Pith
  5. [5] Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, and Hang Li. 2007. Learning to Rank: From Pairwise Approach to Listwise Approach. InProceedings of the 24th International Conference on Machine Learning. 129–136 on Pith
  6. [6] Iñigo Casanueva, Tadas Temčinas, Daniela Gerz, Matthew Henderson, and Ivan Vulić. 2020. Efficient Intent Detection with Dual Sentence Encoders. InProceed- ings of the 2nd Workshop on Natural Language Processing for Conversational AI. 38–45 on Pith
  7. [7] Nick Craswell, Onno Zoeter, Michael Taylor, and Bill Ramsey. 2008. An Exper- imental Comparison of Click Position-Bias Models. InProceedings of the 2008 International Conference on Web Search and Data Mining. 87–94 on Pith
  8. [8] Gérard Debreu. 1960. Review of R. D. Luce, Individual Choice Behavior: A Theoretical Analysis.American Economic Review50, 1 (1960), 186–188 on Pith
  9. [9] Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. 2023. Mind2Web: Towards a Generalist Agent for the Web. InAdvances in Neural Information Processing Systems, Datasets and Benchmarks Track, Vol. 36 on Pith
  10. [10] Beni Egressy and Jan Stühmer. 2025. Set-LLM: A Permutation-Invariant LLM. In Advances in Neural Information Processing Systems 38 on Pith
  11. [11] Gemma Team. 2026. Gemma 4 12B (instruction-tuned). https://huggingface.co/ google/gemma-4-12B-it. Model card, revision 707f0a3. Accessed 2026-10-05 on Pith
  12. [12] Sachin Goyal, Ziwei Ji, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar, and Vaishnavh Nagarajan. 2024. Think Before You Speak: Training Language Models With Pause Tokens. InInternational Conference on Learning Representa- tions on Pith
  13. [13] Blessed Guda, Lawrence Francis, Gabrial Zencha Ashungafac, Carlee Joe-Wong, and Moise Busogi. 2025. Quantifying and Mitigating Selection Bias in LLMs: A Transferable LoRA Fine-Tuning and Efficient Majority Voting Approach. In Proceedings of the 14th International Joint Confere... on Pith
  14. [14] Yupeng Hou, Junjie Zhang, Zihan Lin, Hongyu Lu, Ruobing Xie, Julian McAuley, and Wayne Xin Zhao. 2024. Large Language Models are Zero-Shot Rankers for Recommender Systems. InAdvances in Information Retrieval: 46th European Conference on Information Retrieval. 364–381 on Pith
  15. [15] Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. InInternational Conference on Learning Representations on Pith
  16. [16] Joel Huber, John W. Payne, and Christopher Puto. 1982. Adding Asymmetrically Dominated Alternatives: Violations of Regularity and the Similarity Hypothesis. Journal of Consumer Research9, 1 (1982), 90–98 on Pith
  17. [17] Inception. 2026. Mercury Decide. OpenRouter Decisions API, model inception/mercury-decide-20260930, https://openrouter.ai/inception/ mercury-decide-20260930. Released 30 September 2026. Accessed 2026-10-14 on Pith
  18. [18] Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, and Geri Gay on Pith
  19. [19] Sékou-Oumar Kaba, Arnab Kumar Mondal, Yan Zhang, Yoshua Bengio, and Siamak Ravanbakhsh. 2023. Equivariance with Learned Canonicalization Func- tions. InProceedings of the 40th International Conference on Machine Learning. 15546–15566 on Pith
  20. [20] Daniel Kahneman. 2011.Thinking, Fast and Slow. Farrar, Straus and Giroux on Pith
show 38 more
  1. [21] Ken Lang. 1995. NewsWeeder: Learning to Filter Netnews. InProceedings of the Twelfth International Conference on Machine Learning. 331–339 on Pith
  2. [22] LocalLLaMA. 2026. Typed Decisions. https://huggingface.co/datasets/ LocalLLaMA/typed-decisions. Hugging Face dataset. Accessed 2026-09-29 on Pith
  3. [23] R. Duncan Luce. 1959.Individual Choice Behavior: A Theoretical Analysis. Wiley on Pith
  4. [24] George Ma, Yifei Wang, Derek Lim, Stefanie Jegelka, and Yisen Wang. 2024. A Canonicalization Perspective on Invariant and Equivariant Learning. InAdvances in Neural Information Processing Systems 37. 60936–60979 on Pith
  5. [25] Daniel McFadden. 1974. Conditional Logit Analysis of Qualitative Choice Behav- ior. InFrontiers in Econometrics, Paul Zarembka (Ed.). Academic Press, 105–142 on Pith
  6. [26] Reid McIlroy-Young, Katrina Brown, Conlan Olson, Linjun Zhang, and Cynthia Dwork. 2024. Order-Independence Without Fine Tuning. InAdvances in Neural Information Processing Systems 37. Set-Based Prompting on Pith
  7. [27] Ryan L. Murphy, Balasubramaniam Srinivasan, Vinayak A. Rao, and Bruno Ribeiro. 2019. Janossy Pooling: Learning Deep Permutation-Invariant Functions for Variable-Size Inputs. In7th International Conference on Learning Representa- tions on Pith
  8. [28] Rodrigo Nogueira, Zhiying Jiang, Ronak Pradeep, and Jimmy Lin. 2020. Docu- ment Ranking with a Pretrained Sequence-to-Sequence Model. InFindings of the Association for Computational Linguistics: EMNLP 2020. 708–718 on Pith
  9. [29] Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and... on Pith
  10. [30] Pouya Pezeshkpour and Estevam Hruschka. 2024. Large Language Models Sensitivity to the Order of Options in Multiple-Choice Questions. InFindings of the Association for Computational Linguistics: NAACL 2024. 2006–2017 on Pith
  11. [31] R. L. Plackett. 1975. The Analysis of Permutations.Journal of the Royal Statistical Society: Series C (Applied Statistics)24, 2 (1975), 193–202 on Pith
  12. [32] Omri Puny, Matan Atzmon, Edward J. Smith, Ishan Misra, Aditya Grover, Heli Ben-Hamu, and Yaron Lipman. 2022. Frame Averaging for Invariant and Equi- variant Network Design. InThe Tenth International Conference on Learning Rep- resentations on Pith
  13. [33] Joshua Robinson and David Wingate. 2023. Leveraging Large Language Models for Multiple Choice Question Answering. InThe Eleventh International Conference on Learning Representations on Pith
  14. [34] Ferdinand Schlatt, Maik Fröbe, Harrisen Scells, Shengyao Zhuang, Bevan Koop- man, Guido Zuccon, Benno Stein, Martin Potthast, and Matthias Hagen. 2025. Set-Encoder: Permutation-Invariant Inter-Passage Attention for Listwise Pas- sage Re-Ranking with Cross-Encoders. InAdvances ... on Pith
  15. [35] Jiawen Shi, Zenghui Yuan, Guiyao Tie, Pan Zhou, Neil Zhenqiang Gong, and Lichao Sun. 2026. Prompt Injection Attack to Tool Selection in LLM Agents. In Network and Distributed System Security Symposium (NDSS) on Pith
  16. [36] Florian Standhartinger. 2026. JevBench. https://github.com/fstandhartinger/ jevbench. Public split. Accessed 2026-10-05 on Pith
  17. [37] Raphael Tang, Crystina Zhang, Xueguang Ma, Jimmy Lin, and Ferhan Ture on Pith
  18. [38] Torchcast AI. 2026. Torchcast Decision 12B. https://huggingface.co/torchcast- ai/torchcast-decision-12b. Model card, revision 49107b5. Accessed 2026-10-07 on Pith
  19. [39] Torchcast AI. 2026. Torchcast Decision 12B: serving code. https://github. com/Torchcast-AI/torchcast-decision-12b. Tag v1.0.0 (commit d7f8446), serv- ing/torchcast_shim.py. Accessed 2026-10-12 on Pith
  20. [40] Amos Tversky. 1972. Elimination by Aspects: A Theory of Choice.Psychological Review79, 4 (1972), 281–299 on Pith
  21. [41] Amos Tversky and Daniel Kahneman. 1986. Rational Choice and the Framing of Decisions.The Journal of Business59, 4 (1986), S251–S278 on Pith
  22. [42] TypeSafe AI. 2026. Jev 1.13 Jaggedness. TypeSafe documentation, section “Choice option order”. Accessed 2026-10-04 on Pith
  23. [43] TypeSafe AI. 2026. TypeSafe Documentation: Introduction and Primitives. https: //docs.typesafe.ai/introduction. Accessed 2026-09-29 on Pith
  24. [44] Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Lingpeng Kong, Qi Liu, Tianyu Liu, and Zhifang Sui. 2024. Large Language Models Are Not Fair Evaluators. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Vo... on Pith
  25. [45] Ziqi Wang, Hanlin Zhang, Xiner Li, Kuan-Hao Huang, Chi Han, Shuiwang Ji, Sham M. Kakade, Hao Peng, and Heng Ji. 2025. Eliminating Position Bias of Language Models: A Mechanistic Approach. InThe Thirteenth International Conference on Learning Representations on Pith
  26. [46] Sheng-Lun Wei, Cheng-Kuang Wu, Hen-Hsen Huang, and Hsin-Hsi Chen. 2024. Unveiling Selection Biases: Exploring Order and Token Sensitivity in Large Language Models. InFindings of the Association for Computational Linguistics: ACL 2024. 5598–5621 on Pith
  27. [47] Fen Xia, Tie-Yan Liu, Jue Wang, Wensheng Zhang, and Hang Li. 2008. Listwise Approach to Learning to Rank: Theory and Algorithm. InProceedings of the 25th International Conference on Machine Learning. 1192–1199 on Pith
  28. [48] An Yang et al. 2025. Qwen3 Technical Report. arXiv:2505.09388 on PithPith's reviewarXiv
  29. [49] Soyoung Yoon, Dongha Ahn, Youngwon Lee, Minkyu Jung, HyungJoo Jang, and Seung won Hwang. 2025. RoToR: Towards More Reliable Responses for Order- Invariant Inputs. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).... on Pith
  30. [50] Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Ruslan Salakhutdinov, and Alexander Smola. 2017. Deep Sets. InAdvances in Neural 9 Teng et al. Information Processing Systems 30 on Pith
  31. [51] Jiamu Zhang, Tianze Yang, Yucheng Shi, Evan Chen, Zixiang Nie, Kelly Wan, Liangjie Hong, Ninghao Liu, and Liang Wu. 2026. AnyJev Technical Report. arXiv:2610.00831 on PithPith's reviewarXiv
  32. [52] Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate Before Use: Improving Few-Shot Performance of Language Models. InProceedings of the 38th International Conference on Machine Learning. 12697–12706 on Pith
  33. [53] Chujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou, and Minlie Huang. 2024. Large Language Models Are Not Robust Multiple Choice Selectors. InThe Twelfth International Conference on Learning Representations on Pith
  34. [54] Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. 2024. WebArena: A Realistic Web Environment for Building Autonomous Agents. InInternational Conference on Learning Repr... on Pith
  35. [55] Honglei Zhuang, Zhen Qin, Rolf Jagerman, Kai Hui, Ji Ma, Jing Lu, Jianmo Ni, Xuanhui Wang, and Michael Bendersky. 2023. RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses. InProceedings of the 46th International ACM SIGIR Conference on Research and Development in Info... on Pith
  36. [56] Yongshuo Zong, Tingyang Yu, Ruchika Chavhan, Bingchen Zhao, and Timothy Hospedales. 2024. Fool Your (Vision and) Language Model with Embarrass- ingly Simple Permutations. InProceedings of the 41st International Conference on Machine Learning. A Proofs Options come from a unive... on Pith
  37. [2005] Accurately Interpreting Clickthrough Data as Implicit Feedback. InPro- ceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 154–161 on Pith
  38. [2024] Found in the Middle: Permutation Self-Consistency Improves Listwise Ranking in Large Language Models. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2327–2340 on Pith
Cited by 2 Pith papers

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment