{"id":"9202a636-a61c-4dc4-a58c-b7fd5319e33a","arxiv_id":"2610.18773","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Jeva is a decision-native architecture that encodes requests and options separately, refines only the request state against cached option memory, and uses a coverage gate to distinguish no valid answer from low confidence.","lead":"The paper introduces Jeva, a network for fast AI decisions that encodes the question and each candidate answer separately, then refines only the question side against a cached memory of answers. It reports better no-valid-answer detection, no sensitivity to answer order, and large latency savings when candidate lists recur.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"The null-aware headline rests on the Eq. (6) max-score gate, tested only on BANKING77 null sets whose construction is unspecified; if absent sets are random-intent fills rather than hard distractors, the 98.66% AUROC may measure best-candidate plausibility, not absence detection.","rationale":"I verified the theory by hand: Proposition 1's construction is arithmetically correct (attention weight t = (1 + 2e^{4/3})^{-1} follows from exp(4)/(2exp(16/3)+exp(4)); a_A = a_B makes the A–B margin ±24αt/25; the 1/5 initial advantage with 2α < 1/5 secures uniqueness), Proposition 2 follows from sigmoid algebra, and Proposition 3 is a direct permutation/reuse argument. The empirical reporting is unusually detailed: timings, batching, cache conditions, and numerical agreement are all specified, and the permutation-invariance result (mean TV 3.17e-8, zero flips) directly confirms Proposition 3, while the reuse experiment shows a bounded max probability shift (0.00736) with no decision change. The single load-bearing gap is the null gate: its input is a scalar max score, the absent-set construction on BANKING77 is unspecified, and no hard-negative or out-of-distribution null experiment appears in the paper. This is a generalization/characterization gap, not an internal inconsistency — the authors explicitly distinguish competition from coverage (Fig. 4, Prop. 2, Eq. 7) and the AUROC is threshold-free. Because the strongest claim includes 'nearly perfect null detection,' the gate's fragility to a single high-scoring distractor is precisely where the central claim is least secure; the concrete adversarial absent-set construction settles whether the concern lands. This agrees with the reader's weakest-assumption identification and does not change the verdict: CONDITIONAL remains appropriate pending the hard-negative null evaluation and a calibration-selected operating point.","tokens_in":23001,"tokens_out":12643,"duration_ms":88625,"concrete_test":"Rebuild the BANKING77 null set as follows: for each of the 2,492 answer-present requests, form the absent twin by removing the true intent and inserting the model's own top-16 predictions among the remaining intents (hard-negative variant), plus a second variant using the 16 intents nearest to the true intent under the backbone's pooled embeddings. Keep all other protocol details fixed and rerun the null evaluation. If the AUROC drops materially — e.g., below roughly 90% on the hard-negative variant — Eq. (6) is reading max plausibility rather than coverage, and the null-aware headline must be scoped to benign-null regimes. Complementarily, report the calibration of ρ on both variants and recompute the operating point with the threshold chosen on a held-out split instead of on test labels, to separate threshold-selection effects from gate quality.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central null-awareness claim — 98.66% AUROC, 93.88% TPR at 5.02% FPR on 3,080 BANKING77 requests — is produced by the coverage gate ρ = σ((max_i s_i − b_cov)/τ_cov) (Eq. 6). This gate reads only the maximum final score: it cannot see the score distribution, the margin to the runner-up, or whether the argmax candidate is genuinely adequate, and it is defeated by any absent-answer request that yields one plausible-looking distractor. The paper itself separates competition from coverage (Fig. 4, Prop. 2) but never stress-tests the gate in exactly that regime. Sections 4.2 and C.3 give counts (2,492 present, 588 absent) but no construction of the absent sets. If, as is standard, they are formed by withholding the true intent and filling with random remaining intents, the max score separates classes on simple plausibility, and the AUROC shows the gate ranks max-plausibility — not that it detects answer absence. The same scores drive both q and ρ, so nothing in the training objective (Eq. 9) forces the gate to exploit absence-specific signal; it only needs a usable threshold on max score. The reader's protocol point is also granted: the near-5% FPR point is each model's empirical 95th percentile of test answer-present scores, so TPR@5%FPR is partly threshold-selected; the threshold-free AUROC is the sounder evidence but still lives inside one uncharacterized, likely benign-null distribution. No OOD or hard-negative null evaluation is reported anywhere in the paper. The other three properties are backed by Propositions 1–3 plus direct empirical confirmation (mean TV 3.17e-8, zero flips; no decision change under caching with max |Δp| = 0.00736), so the null gate is where the central claim is least secure.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"This paper argues that current system-one decision models, which jointly encode a request and its candidates and read decisions from answer-token logits or task-specific heads, neglect four structural requirements: null awareness (distinguishing answer absence from competition among candidates), permutation invariance, option scalability, and cacheability. The authors propose Jeva, in which a shared backbone independently encodes the request and each option, a lightweight decision stack refines only the request state against an immutable and cacheable option memory via symmetric attention, and final scores s_i = z_L^T u_i feed both a conditional competition distribution q = softmax(s/tau_dec) and a coverage gate rho = sigma((max_i s_i - b_cov)/tau_cov), yielding open-set probabilities p_perp = 1 - rho and p_i = rho q_i (Eq. 6). Three propositions are stated and proved: a single decision block can reverse the preference between two retained options when a third candidate changes, which is impossible for independent scorers (Prop. 1); coverage is invariant to adding sub-maximal options, unlike a flat softmax with a null token that incurs a log-K term (Prop. 2); and permutation equivariance plus exact cache reuse hold under exact arithmetic (Prop. 3). Training uses the joint cross-entropy loss of Eq. (9), with typed Choice, Noul, and Score readouts.","tokens_in":23477,"tokens_out":12384,"duration_ms":88228,"significance":"If the reported results hold, this is a useful contribution: it isolates four design properties that are usually entangled in serialized-text decision models, offers an architecture satisfying them jointly, and gives simple, apparently correct theoretical statements whose proofs check out, together with an unusual degree of protocol transparency in Appendix C (adapter templates, cache-identity conditions, numerical configuration). The in-distribution numbers are strong and mostly consistent with the theory; for example, permutation equivariance is confirmed empirically by zero flips and a residual total variation around 3e-8, matching Proposition 3. The main risk is not the architecture itself but the evidence for the null-awareness headline, which rests on a single BANKING77 setup with an unspecified absent-set construction and a partly test-selected operating point, and which offers no hard-negative or out-of-distribution evaluation. With the requested additional null evaluations, threshold-protocol fixes, and variance reporting, the empirical claims would be substantially better supported.","major_comments":[{"comment":"The headline null-awareness evidence does not yet establish the claimed generalization because the construction of the 588 answer-absent requests is never described. If, as is common in this area, they are produced by withholding the true intent and filling the candidate list with remaining intents, then max-score plausibility alone separates the two classes, and the 98.66% AUROC may measure plausibility ranking rather than detection of answer absence. The gate in Eq. (6) reads only the maximum final score and cannot see the margin to the runner-up or the score distribution, so an absent-answer request that happens to contain one plausible distractor defeats it; the paper's own Figure 4 distinguishes competition from coverage but never tests that regime. Please specify exactly how the absent sets were generated, and add hard-negative and out-of-distribution null evaluations (for example, unanswerable requests with plausible but incorrect candidates, or null sets drawn from other domains), reporting AUROC for those settings.","section":"Section 4.2, Section C.3, Eq. (6)","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The reported 93.88% TPR at 5.02% FPR is partly a test-label-selected quantity: the paper states that \"near-5% FPR thresholds are each model's empirical 95th percentile of answer-present scores,\" so the FPR is approximately 5% by construction, the thresholds use test labels, and the models are compared at slightly different operating points. The threshold-free AUROC (98.66% versus 93.61%) is the sounder evidence and should carry the claim. Please select thresholds on a validation or calibration split, or report the full ROC with confidence intervals, so that the operating-point comparison is not selected from test data.","section":"Section 4.2","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"All headline empirical comparisons are single-run point estimates without error bars or significance tests. Several comparative claims are close enough that noise could matter, notably Typed-Decisions accuracy of 78.30% for Jeva versus 78.70% for Intern-Decision-4B, and the \"leads seven of eight slices\" summary in Table 1. Please report results over multiple seeds or at least bootstrap confidence intervals for the main metrics, and state the number of independent runs used for each table.","section":"Section 4, Tables 1 and 2","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The null-detection comparison mixes different null mechanisms: Jeva uses a coverage head trained jointly with the decision objective, typed baselines receive an auxiliary Noul question that lists all candidates, and JevK5 and CLM receive an explicit None candidate. Part of the AUROC gap could therefore reflect the adapter rather than the underlying model, which matters because null awareness is a central claimed property. Please justify that the adapter does not systematically depress baseline AUROC, for example by comparing a baseline's native calibration outputs where they exist (such as KEV's calibrated scores) against the auxiliary-Noul adapter, and report the sensitivity of the comparison to adapter wording.","section":"Section 4.2, Section C.7","authors_acknowledge":null,"authors_acknowledge_p":null}],"minor_comments":[{"comment":"The column header row (\"Typed\", \"BANKING77\", \"MASSIVE\", \"Mind2Web\", then the K=8/K=16 groups) is hard to parse; adding explicit group headers and clarifying which columns include null outcomes would improve readability.","section":"Table 1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The training-curriculum stage sizes are reported, but the data sources and mixing proportions within stages are not, which limits reproducibility of the released checkpoint.","section":"Section C.1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The auxiliary Noul and Score loss components are defined, but the combination coefficients of the primary joint cross-entropy and the auxiliary terms in the total objective are not stated.","section":"Section 3.5, Section B.8","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The maximum absolute probability difference under caching (0.00736) is reported without saying over how many requests it occurred or how the aligned-identity check treated it; a sentence clarifying whether any request approached a decision boundary would help.","section":"Section 4.4","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Several baselines and datasets are cited only to repository or blog preprints (for example, LocalLLaMA 2026, Almeida 2026, and JevK5 authors 2026); pinning committed versions and noting which baselines have peer-reviewed descriptions would aid reproducibility.","section":"References","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"major_revision"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"Two things to know. The architecture is genuinely coherent — independent request-state refinement against immutable option memory, symmetric aggregation, a max-score coverage gate, joint open-set loss. That combination is new, and three of its four headline properties hold. The fourth, null detection, is where the central claim is least secure.\n\nCredit first. Proposition 1 is a real expressivity result: one block, fixed parameters, two sets sharing options A and B, winner flips. Worked proof, with an embedding into the deployed head width. That separates Jeva from any independent scorer and is the load-bearing theoretical claim. Proposition 3's symmetry and exact-reuse argument is structurally sound — softmax over permuted keys reindexes the sum, the state is unchanged, outputs permute. The zero-flip, mean TV 3.17e-8 result is a consequence of that structure, not a fit, and the paper shows it honestly. The caching numbers (20x speedup, max |Δp| 0.00736) are plausible, and the protocol detail in Appendix C is above the genre average.\n\nThe soft spot, in proportion. The coverage gate reads only max_i s_i. It cannot see the margin to the runner-up or the score distribution, and nothing in the joint loss forces it to learn absence beyond a threshold on that max — the stress-test note lands. The paper tests it only on BANKING77, and the construction of the 588 absent sets is not specified. If, as is standard, they are the true intent withheld and filled with random remaining intents, the 98.66% AUROC measures that absent sets have lower max plausibility — genuine absence detection, not necessarily. Separately, the near-5% FPR point is each model's empirical 95th percentile of test answer-present scores. They disclose this, which earns credit, but it makes TPR@5%FPR partly threshold-selected. Both are fixable: specify the null construction, add a hard-negative or OOD null set, move the threshold onto a calibration set. None of the other three properties depends on the gate, so the paper stands, with the null headline provisional.\n\nMinor: single runs, no error bars, no shipped code. The zero-flip result doesn't need seeds, but the accuracy tables would benefit from them.\n\nWho this is for: anyone building low-latency typed decision models — tool selection, document ranking, interface routing. The four-property decomposition is a useful checklist regardless of whether Jeva wins. Send it to a serious referee with a request for the null-set construction and multi-seed numbers; I expect heavy revision on the null claim, not rejection.","headline":"A coherent decision-native architecture with a real single-block expressivity proof; three of its four structural claims hold up, and the null-detection headline needs the absent-set construction and a calibration threshold before I'd trust it.","tokens_in":23971,"tokens_out":4221,"duration_ms":31430},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"Jeva claims that separating request and option encoding, with a coverage gate over the maximum candidate score, yields order-invariant, null-aware, cacheable typed decisions that scale to thousands of options in a single scoring pass.","keywords":["system one models","decision-native architecture","typed decisions","permutation invariance","null detection","coverage gate","candidate caching","option scalability"],"falsifier":"Construct an answer-absent evaluation set in which each request deliberately contains one decoy whose score profile mimics a genuine answer (for example, selected from the model's own top-score distribution), and measure whether the coverage gate still separates absent from present requests at the reported near-5% false-positive operating point; that threshold is the empirical 95th percentile of answer-present scores on the evaluation set itself, and the paper reports no out-of-distribution evidence for the gate.","tokens_in":24451,"feed_emoji":"🎯","tokens_out":14112,"duration_ms":86156,"temperature":0.7,"pith_summary":"The paper argues that 'system one' models — which answer decision tasks by serializing the request and all candidate options into one text and reading a decision from token logits — ignore four structural facts about decision problems: the answer may be absent, the option order is meaningless, candidate counts vary, and recurring candidates should not need re-encoding for every request. It introduces Jeva, an architecture that encodes the request and each option independently, then refines only a request-state vector against the fixed option set with a single compact attention block. A separate coverage gate decides whether any candidate applies, rescaling the within-set competition by a learned presence probability $\\rho$, so 'none of the above' becomes an explicit decision rather than an artifact of normalization, and the same machinery renders Choice, Noul, and Score outputs as interpretations of one distribution. The paper claims this joint design delivers near-perfect null detection, zero decision flips under every tested candidate ordering, and exact reuse of cached option representations, while scoring full candidate sets up to 2,048 options in one pass and cutting median latency by over 20$\\times$ through cross-request caching.","feed_headline":"Zero decision flips under any candidate order","feed_subtitle":"Separating request from options yields order-stable, null-aware decisions that scale to thousands of candidates.","key_machinery":"The load-bearing mechanism is the factorization of the output distribution into coverage and competition: $p_\\perp = 1-\\rho$ and $p_i = \\rho q_i$, where $q=\\mathrm{softmax}(s/\\tau_{\\text{dec}})$ distributes probability among the candidates and $\\rho=\\sigma((\\max_i s_i - b_{\\text{cov}})/\\tau_{\\text{cov}})$ in Eq. (6) decides whether any candidate applies. This factorization defines null as answer absence rather than as a fixed score competing against the candidates; Proposition 2 shows it removes the compulsory $\\log K$ term that a flat softmax with a null score would introduce. Two further mechanisms carry the other properties: the decision block's entropy-regularized attention over the option memory, whose weights uniquely maximize a compatibility-plus-entropy objective and which lets a single block reverse a preference between two retained options when the surrounding set changes (Proposition 1), and the atomic option memory of Eq. (8), which stores each option's semantic vector and per-layer key/value projections so that cached entries are exactly identical to recomputed ones and reordering candidates merely permutes the outputs (Proposition 3).","core_discovery":"The paper's central claim is that a decision problem's structure — answer absence, unordered alternatives, variable candidate counts, and recurring candidates — should be built into the architecture rather than patched onto serialized-text prediction. Jeva encodes the request and each option independently with a shared language backbone, so option vectors and their layer projections depend on nothing but the option text; a compact decision block then refines only the request state against the immutable option memory through a symmetric attention mixer, allowing candidate interactions to change the state without materializing a $K\\times K$ interaction matrix. The final scores factor as $p_\\perp = 1-\\rho$, $p_i = \\rho q_i$, with $q=\\mathrm{softmax}(s/\\tau_{\\text{dec}})$ over candidate scores and $\\rho=\\sigma((\\max_i s_i - b_{\\text{cov}})/\\tau_{\\text{cov}})$ a coverage estimate that separates 'does any candidate apply?' from 'which one wins?'. On this basis the paper reports 98.66% null-detection AUROC on BANKING77 with 16 candidates, zero decision flips and $3.17\\times10^{-8}$ mean total variation across all tested candidate permutations, 78.40% aggregate accuracy across 500 ToolRet requests up to 2,048 candidates, and median-latency speedups of over $2\\times$ for native full-set scoring versus recursive selection at 2,048 options and over $20\\times$ for hybrid caching of newly assembled long-document sets, with decisions unchanged under caching.","pith_inferences":["Editorial inference: because the coverage gate reads only the maximum candidate score, a natural stress test is whether a richer statistic — the margin between the top two scores, or a learned function of the whole score vector — detects answer absence more reliably when decoys are adversarially engineered to score high.","Editorial inference: the joint log-loss decomposition implies that all-null training examples teach presence detection without imposing candidate rankings; a direct probe is whether $\\rho$ tracks genuine validity on datasets whose score distributions differ sharply from the one used in the reported null experiments.","Editorial inference: for agentic serving, the exact-reuse result suggests encoding a shared tool or document bank once per model version and refining only per-request states, so request latency stops growing with the size of the persistent bank.","Editorial inference: Proposition 3 holds under exact arithmetic with fixed weights, so a natural measurement is how probability differences grow under BF16 rounding and batched inference in near-tie settings; that would quantify how much of the observed $3.17\\times10^{-8}$ stability survives in production numerics."],"forward_implications":["If the architecture works as claimed, persistent candidate content — tool registries, document collections, label vocabularies — can be encoded once and reused across arbitrarily reassembled candidate sets, with per-request work reduced to state encoding and the linear-in-$K$ readout.","Permutation invariance becomes a structural property rather than an empirical tendency: reordering candidates permutes candidate probabilities but provably leaves the refined state, null probability, and typed Noul/Score outputs unchanged (Proposition 3), matching the measured zero flips and $3.17\\times10^{-8}$ mean total variation.","Null outcomes become a first-class decision: because coverage is estimated from the maximum candidate score rather than normalized against a denominator, adding options below the maximum does not itself shift the model toward or away from 'none of the above' (Proposition 2).","Set-dependent judgment is expressible with a single decision block: a change in the surrounding candidate set can reverse the preference between two retained options, a behavior no independent per-candidate scorer with a fixed tie-break can produce (Proposition 1).","At 2,048 candidates, scoring the full set in one decision-head pass beats recursive selection in both accuracy and median latency, and hybrid caching of newly assembled long-document sets yields a median-latency speedup of over $20\\times$ without changing decisions."],"supporting_citations":[{"why":"Supplies the Deep Sets foundation of shared transformations and symmetric aggregation that Jeva's permutation-invariance argument builds on.","marker":"(Zaheer et al., 2017)"},{"why":"Provides attention-based interaction among set elements, the pattern Jeva's state-refinement mixer adapts to a request-conditioned state.","marker":"(Lee et al., 2019)"},{"why":"Establishes the independent-encoding design that makes precomputed, reusable candidate representations possible.","marker":"(Karpukhin et al., 2020)"},{"why":"Demonstrates late interaction over retained representations, the retrieval precedent for Jeva's reusable candidate memory.","marker":"(Khattab and Zaharia, 2020)"},{"why":"Supplies BANKING77, the evaluation set for the null-detection claim with 3,080 matched requests and 588 answer-absent cases.","marker":"(Casanueva et al., 2020)"},{"why":"Supplies ToolRet, the diagnostic for option scalability with 500 requests in cohorts from 32 to 2,048 candidates.","marker":"(Shi et al., 2025)"},{"why":"Supplies the Typed Decisions suite used for Choice, Noul, and Score fidelity and for the permutation-stability tests.","marker":"(LocalLLaMA, 2026)"},{"why":"Supplies the MLDR long-document candidate sets used to measure cross-request hybrid-cache latency speedups.","marker":"(Chen et al., 2024)"}],"fun_headline_variants":["Zero decision flips under any tested candidate order","Null-aware, permutation-stable decisions at thousands of options","Architecture separates request and options to lock decisions","Shared caching keeps thousands of candidates decision-stable"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The load-bearing premise is that answer absence is readable from a single scalar, $\\rho = \\sigma((\\max_i s_i - b_{\\text{cov}})/\\tau_{\\text{cov}})$: if a request with no valid answer still yields a high-scoring decoy candidate, or if presence depends on information the maximum score does not capture, the coverage gate will report an answer where none exists.","fun_headline_variants_meta":{"raw":{"variants":["Zero decision flips under any tested candidate order","Null-aware, permutation-stable decisions at thousands of options","Architecture separates request and options to lock decisions","Shared caching keeps thousands of candidates decision-stable"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":0.000125,"raw_usage":{"cost":0.0001247246,"is_byok":false,"cost_details":{"upstream_inference_cost":0.0001247246,"upstream_inference_prompt_cost":3.51806e-05,"upstream_inference_completions_cost":8.9544e-05},"total_tokens":1669,"prompt_tokens":1177,"completion_tokens":492,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":988,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":442}},"tokens_in":1177,"tokens_out":492,"cost_source":"openrouter_usage","duration_ms":5294,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":442,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-10-14T07:20:31.574084+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":"Construct an answer-absent evaluation set in which each request deliberately contains one decoy whose score profile mimics a genuine answer (for example, selected from the model's own top-score distribution), and measure whether the coverage gate still separates absent from present requests at the reported near-5% false-positive operating point; that threshold is the empirical 95th percentile of answer-present scores on the evaluation set itself, and the paper reports no out-of-distribution evidence for the gate.","supporting_citations":[{"cited_title":"2020 , url =","cited_arxiv_id":null,"evidence_quote":"Demonstrates late interaction over retained representations, the retrieval precedent for Jeva's reusable candidate memory."},{"cited_title":"2024 , doi =","cited_arxiv_id":null,"evidence_quote":"Supplies the MLDR long-document candidate sets used to measure cross-request hybrid-cache latency speedups."}],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}