{"id":"d073e688-1e87-4f9e-ab96-ef948e246778","arxiv_id":"2611.17775","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"LASR is a dimensionless metric that measures action-related predictive support in world-model latents by comparing matched state, latent, and control routes.","lead":"The paper introduces LASR, a dimensionless metric that measures whether a learned latent representation in a world model provides predictive support for action-related targets beyond what the current state and non-action shortcuts already explain. It provides a controlled offline evaluation protocol for latent-action models, with two aggregate summaries (GeoLASR and WorstLASR) and a benchmark instantiation on robot manipulation data.","discovery_kind":"new_method","skeptic_critique":{"audit":{"quote":"The result remains relative to the declared audit labels, probe family, and controls, and low LASR alone does not certify causal action identity or executability.","status":"reviewer_error","question":null,"consequence":null,"objection_id":"O1","published_as":"withdrawn"},"model":"deepseek/deepseek-v4-flash","headline":"Withdrawn: Pith's reviewer misread the paper.","rationale":"","tokens_in":33618,"tokens_out":906,"duration_ms":6898,"concrete_test":"","verdict_should_be":null,"load_bearing_attack":"The paper's words: “The result remains relative to the declared audit labels, probe family, and controls, and low LASR alone does not certify causal action identity or executability.”","agreement_with_reader":null},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"The paper introduces LASR, a dimensionless metric for evaluating whether learned latent representations in world models encode action-related information beyond what is already available from the current state and non-action shortcuts. The metric compares held-out prediction risks from three matched predictor routes (state-only, latent+state, control+state) and produces per-target-fold scores aggregated into GeoLASR and WorstLASR. The authors provide theoretical justification via Bayes-risk decomposition and an oracle sandwich theorem, instantiate LASR as a benchmark on ALOHA robot data and a multi-task ManiSkill3 panel, evaluate five frozen latent-action models, and demonstrate a LASR-guided residual adapter that improves both LASR and world-model prediction metrics. The paper claims LASR detects known action information, rejects non-action shortcuts, and enables graded comparisons among imperfect representations.","tokens_in":33881,"tokens_out":2518,"duration_ms":11724,"significance":"Not published. Pith's referee wrote this assessment before its objections were checked against the paper, and 4 of its 4 major objections did not survive that check.","major_comments":[],"minor_comments":[{"quote":null,"comment":"Equation (5) is presented as a variance decomposition, but the notation E[||m_{S,Z} - m_S||^2_2] is the expected squared norm of the difference of conditional means. This is correct but could be more clearly connected to the notion of \"additional information carried by Z beyond S.\"","section":"§3.4","consequence":null,"audit_status":null,"objection_id":null,"quote_matched":false,"required_revision":null,"authors_acknowledge":null,"authors_acknowledge_p":null},{"quote":"The other half are \\emph{ranking-enriched} transitions that emphasize a harder case for the evaluator: two transitions from different episodes may be close in standardized state space under Euclidean distance but have different recorded actions.","comment":"The description of the panel composition (ordinary-reference and ranking-enriched transitions) is detailed but the rationale for the enrichment procedure could be clearer. The enrichment uses recorded-action distance, which could introduce a subtle dependence on the action labels that LASR later evaluates. The paper states this is fixed before model evaluation, but the potential for bias should be discussed.","section":"§4.1","consequence":null,"audit_status":null,"objection_id":null,"quote_matched":true,"required_revision":null,"authors_acknowledge":null,"authors_acknowledge_p":null},{"quote":"The pooled result is not universal: PegInsertionSide is a clear failure case for DiLA, LAPA, and LaWAM.","comment":"Table 8 reports task-stratified GeoLASR and shows that PegInsertionSide is a clear failure case for DiLA, LAPA, and LaWAM. This is an important negative result that should be highlighted in the main paper, as it demonstrates that LASR can identify representations that fail on specific tasks even when aggregate scores are good.","section":"Appendix H.1","consequence":null,"audit_status":null,"objection_id":null,"quote_matched":true,"required_revision":null,"authors_acknowledge":null,"authors_acknowledge_p":null},{"quote":"This null prevents using LASR as a fine-grained policy-success score.","comment":"The native-policy audit (Appendix H.5) shows that GeoLASR versus success has Spearman ρ = -0.1610 with wide confidence interval, and the prespecified within-task analysis gives ρ = +0.2327 with p=0.8992. The paper correctly states \"This null prevents using LASR as a fine-grained policy-success score.\" This is an honest and important limitation that should be mentioned earlier in the main paper, not just in the appendix.","section":"§4.4","consequence":null,"audit_status":null,"objection_id":null,"quote_matched":true,"required_revision":null,"authors_acknowledge":null,"authors_acknowledge_p":null},{"quote":"Here, $S_t$ denotes the synchronized proprioceptive or simulator state used for audit, rather than the RGB observation from which an image-based latent may be extracted.","comment":"The definition of the evaluation task uses S_t as \"the synchronized proprioceptive or simulator state used for audit, rather than the RGB observation from which an image-based latent may be extracted.\" This is an important distinction that could be emphasized more, as it means LASR requires access to ground-truth state information that may not be available in all settings.","section":"§3.1","consequence":null,"audit_status":null,"objection_id":null,"quote_matched":true,"required_revision":null,"authors_acknowledge":null,"authors_acknowledge_p":null},{"quote":"Linear probes test information accessibility, control tasks expose evaluator memorization, and conditional probes measure contributions beyond a specified reference \\citep{alain2016understanding,hewitt2019designing,hewitt2021conditional}; however, improvement over a single reference does not show that the latent outperforms both the state route and a plausible non-action control route under a matched predictor family and fitting procedure.","comment":"The related work section is comprehensive but could benefit from a more direct comparison to the conditional probing framework of Hewitt et al. (2021), which the paper cites. The paper's approach is similar in spirit (measuring usable information beyond a baseline) but applied to a different domain. A brief discussion of how LASR extends or differs from conditional probing would be helpful.","section":"§2","consequence":null,"audit_status":null,"objection_id":null,"quote_matched":true,"required_revision":null,"authors_acknowledge":null,"authors_acknowledge_p":null},{"quote":"At matched output size, the learned bottleneck retains stronger support than PCA, while its smaller version is weaker.","comment":"Table 2(a) reports that PCA moves GeoLASR and WorstLASR toward one, while a learned bottleneck retains stronger support at matched output size. This is an interesting result that suggests LASR captures preserved accessible action structure, not just dimension. The paper could discuss implications for representation learning (e.g., that linear compression may destroy action-relevant structure that nonlinear compression preserves).","section":"§4.3","consequence":null,"audit_status":null,"objection_id":null,"quote_matched":true,"required_revision":null,"authors_acknowledge":null,"authors_acknowledge_p":null},{"quote":"We therefore require outcome stress as a companion audit and do not interpret LASR alone as causal action identification or executability.","comment":"The camera and outcome interventions (Table 11) are a nice addition that tests robustness. The outcome intervention flips two cells, demonstrating that LASR can be affected by outcome leakage. The paper correctly requires outcome stress as a companion audit. This is a strength of the evaluation.","section":"Appendix H.6","consequence":null,"audit_status":null,"objection_id":null,"quote_matched":true,"required_revision":null,"authors_acknowledge":null,"authors_acknowledge_p":null},{"quote":"If any valid $\\rho_j$ is zero, the product convention sets $G=0$; the WorstLASR certificate remains unchanged and prevents a zero on one target from accepting a representation that fails another target.","comment":"The definition of GeoLASR uses the geometric mean of per-target-fold scores. The paper states \"Lower is better for both summaries.\" However, the geometric mean can be zero if any score is zero (as happens for the recorded action reference). The paper notes this in Appendix A (\"If any valid ρ_j is zero, the product convention sets G=0\"). This is fine but could be mentioned in the main text for clarity.","section":"§3.3","consequence":null,"audit_status":null,"objection_id":null,"quote_matched":true,"required_revision":null,"authors_acknowledge":null,"authors_acknowledge_p":null},{"quote":"LASR remains controlled and interpretable by fixing the targets, predictor family, and controls.","comment":"The discussion section is relatively brief. It could be expanded to include more concrete guidance on how practitioners should choose control variables and predictor families, and what validation steps are recommended before trusting LASR results.","section":"§5","consequence":null,"audit_status":null,"objection_id":null,"quote_matched":true,"required_revision":null,"authors_acknowledge":null,"authors_acknowledge_p":null}],"open_questions":null,"recommendation":"uncertain"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"This paper introduces LASR, a dimensionless metric that measures whether a learned latent representation provides predictive support for action-related targets beyond what the current state and a validation-selected non-action control can explain. The idea is well-motivated: existing metrics for latent-action models cannot isolate whether a latent that improves prediction actually encodes action information or merely captures state predictability, episode progress, or other shortcuts. LASR addresses this by comparing three matched predictor routes (state, latent, control) and defining a ratio that must be below one for the latent to pass. The paper provides a clear theoretical foundation (Bayes-risk decomposition, oracle sandwich) and a careful empirical protocol with episode-disjoint folds, validation-selected controls, and a fixed predictor family. The experiments on ALOHA and a multi-task panel show that LASR detects known action information, rejects non-action shortcuts, and produces graded rankings that correlate with downstream one-step execution accuracy. The paper also demonstrates a secondary use of LASR as a training signal for a residual adapter, with point estimates favoring the LASR-guided variant. The writing is direct and the limitations are acknowledged: LASR is relative to the declared controls, predictor family, and targets; it does not certify causal action identity or executability. The main soft spot is the scale of the evaluation. The ALOHA dataset contains only 49 episodes and 2048 transitions, and the multi-task panel is also modest. The paper would benefit from validation on larger, more diverse datasets and from uncertainty estimates for the world-model prediction improvements. The adapter results are promising but the paired intervals include zero when compared to the raw latent. Overall, this is a solid, principled contribution to a specific evaluation problem in latent-action learning. It deserves a serious referee and would benefit from broader empirical validation. I would accept it for peer review.","headline":"LASR is a well-motivated metric for evaluating whether a learned latent representation in a world model carries action-related information beyond what the current state and non-action shortcuts can explain, and the paper's empirical validation on ALOHA and multi-task panels supports its central claims, though the evaluation is limited to a small dataset and the metric's generalizability to…","tokens_in":34230,"tokens_out":474,"duration_ms":4345},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"The paper introduces LASR, a metric that measures whether a learned latent representation in a world model provides predictive support for action-related targets beyond what the current state and non-action shortcuts can explain.","keywords":["latent action","world models","representation evaluation","action-grounded latents","predictive support","offline metric","robot manipulation","ALOHA"],"falsifier":"If a latent representation that is known to contain no action information (e.g., a random noise vector or a constant) achieved a LASR score below one on the ALOHA benchmark, that would falsify the claim that LASR detects action-related predictive support beyond non-action shortcuts.","tokens_in":35311,"feed_emoji":"🤖","tokens_out":1587,"duration_ms":8759,"temperature":0.2,"pith_summary":"The paper argues that existing evaluation metrics for latent-action world models cannot distinguish whether a learned latent actually encodes action information or merely exploits predictive shortcuts like episode progress or task identity. The authors introduce the Latent Action Support Ratio (LASR), a dimensionless metric that compares held-out prediction risk from three matched routes: using only the current state, using the state plus the latent, and using the state plus a non-action control variable. A LASR score below one means the latent outperforms both the state-only and the best non-action control, providing evidence that the latent contributes action-related predictive support. The paper demonstrates LASR on robot manipulation data, showing it detects known action information, rejects non-action shortcuts, and enables graded comparisons among imperfect latent representations. The authors also use a LASR-inspired training objective to learn a lightweight residual adapter that improves the accessibility of action information in a frozen latent.","feed_headline":"Authors propose LASR metric to test whether world-model latents encode genuine action…","feed_subtitle":"The dimensionless score compares prediction with state, latent, and non-action controls to reject shortcuts like episode progress.","key_machinery":"The central object is the Latent Action Support Ratio (LASR), defined as ρ_{τ,k}(Z) = E_Z / min{E_state, E_ctrl}, where E_Z is the held-out prediction risk of a predictor using the current state S and the latent Z, E_state uses S alone, and E_ctrl uses S plus a validation-selected non-action control variable b (e.g., episode progress or episode identity). The key mechanism is the matched comparison of three predictor routes (state, latent, control) using the same predictor family (affine ridge) and loss function, with the control selected on validation data to represent a plausible non-action shortcut. The theoretical interpretation relies on a conditional Bayes-risk decomposition showing that the risk reduction from augmenting S with Z equals the expected squared distance between the conditional means with and without Z.","core_discovery":"The paper's central claim is that LASR provides a principled offline protocol for measuring whether a learned latent representation in a world model contributes action-related predictive support beyond what is already available from the current state and from plausible non-action shortcuts. The metric is defined as the ratio of the latent-augmented prediction risk to the minimum of the state-only and control-augmented risks, with a score below one indicating that the latent beats both references. The paper further claims that GeoLASR and WorstLASR aggregate per-target-fold scores to summarize typical and worst-case support, and that the metric detects known action information, rejects non-action shortcuts, and enables graded comparisons among imperfect latent representations, as demonstrated on ALOHA and multi-task panels.","headline_check":{"rule":"report_a_claim_v1","sent_back":false,"unattributed":{}},"pith_inferences":["LASR could be extended to other domains beyond robotics, such as video prediction or game-playing, where distinguishing action-related information from predictive shortcuts is similarly critical for world-model evaluation.","The metric's reliance on a fixed predictor family (affine ridge) and declared control set means that a latent could pass LASR but still fail to support more complex downstream tasks that require nonlinear action decoding or closed-loop control; the paper acknowledges this limitation.","The paired bootstrap uncertainty estimates for adapter comparisons suggest a template for reporting confidence in LASR differences, which could become standard practice in future latent-action evaluations.","The paper's demonstration that LASR correlates with one-step physical replay accuracy (branch accuracy) but not with fine-grained policy success (UniVLA native-policy audit) implies that LASR is a necessary but not sufficient condition for downstream control, and should be complemented by task-specific validation."],"forward_implications":["LASR provides a common scale for comparing different latent-action models on the same benchmark, enabling researchers to determine whether a new representation genuinely encodes action information or merely exploits predictive shortcuts.","The metric can be used to audit frozen latent representations before deploying them in downstream planning or control, rejecting representations that fail the LASR criterion (score ≥ 1) as not providing action-related support beyond state and non-action cues.","The LASR-guided residual adapter demonstrates that the metric can be used as a training signal to improve the accessibility of action information in a frozen latent, potentially leading to better world-model prediction and downstream performance.","The WorstLASR summary exposes the weakest target-fold pair, allowing practitioners to identify specific failure cases where a latent fails to provide action support, guiding targeted improvements."],"supporting_citations":[{"why":"Provides the conditional probing principle that LASR builds on: measuring usable information beyond a baseline reference.","marker":"\\citep{hewitt2021conditional}"},{"why":"Introduces control tasks for probing representations, which LASR extends by adding a non-action control route.","marker":"\\citep{hewitt2019designing}"},{"why":"Raises the fundamental question that LASR addresses: whether latent-action models learn genuine action information or exploit predictive shortcuts.","marker":"\\citep{zhang2026latent}"},{"why":"Provides the ALOHA static-battery dataset used to instantiate the LASR benchmark.","marker":"\\citep{zhao2023learning}"},{"why":"Provides the ManiSkill3 simulator used for the matched multi-task panel.","marker":"\\citep{tao2024maniskill3}"},{"why":"One of the latent-action models (LAPA) evaluated in the LASR benchmark.","marker":"\\citep{ye2025latent}"},{"why":"One of the latent-action models (UniVLA) evaluated in the LASR benchmark.","marker":"\\citep{bu2025univla}"},{"why":"One of the latent-action models (MVP-LAM) evaluated in the LASR benchmark.","marker":"\\citep{lee2026mvp}"},{"why":"One of the latent-action models (DiLA) evaluated in the LASR benchmark.","marker":"\\citep{zhang2026dila}"},{"why":"One of the latent-action models (LaWAM) evaluated in the LASR benchmark.","marker":"\\citep{chen2026lawam}"}],"cache_read_input_tokens":0,"weakest_assumption_plain":"The validity of LASR as a measure of action-grounded support depends on the chosen non-action controls (episode progress, episode identity) and predictor family (affine ridge) being sufficient to rule out all plausible non-action shortcuts that could explain away the latent's improvement.","cache_creation_input_tokens":0},"created_at":"2026-11-16T06:56:03.766084+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium+deepseek/deepseek-v4-flash+p.d60f54226f00","input_kind":"tex","input_chars":113590,"input_total_chars":113590,"passes_missing":[],"published_standing":"established_narrower","published_confidence":"moderate","objection_audit":{"version":"pith-objection-audit@1","counts":{"raised":5,"confirmed":0,"questions":0,"withdrawn":5,"moved_to_minor":0,"math_error_not_certified":0,"withdrawn_reviewer_error":5,"typesetting_minor_dropped":0,"withdrawn_already_addressed":0,"referee_assessment_unpublished":1},"open_questions":[],"standing":{"basis":["first_pass_verdict"],"standing":"established_narrower","confidence":"moderate"},"standing_rule":"1. No standing is published (not_assessed) when the review read only the abstract, when the referee pass or the objection check did not run, or when the first pass gave no verdict. 2. The first pass's verdict sets the start: ACCEPT is established, CONDITIONAL is established_narrower, REJECT is not_supported. 3. Each objection that survived the check against the paper's text caps the standing: one that breaks the central claim caps it at not_supported, one that narrows it at established_narrower, one that weakens it at supported_with_gaps. The standing is the lowest of the start and the caps. 4. A first-pass REJECT stands only if a surviving objection breaks the central claim. Without one, the standing is the lowest cap of the surviving objections, or not_assessed when none survived. 5. The confidence is the lowest of the first pass's confidence and the confidence of each objection that set the standing, and is no higher than moderate while a question is open or part of the paper's text was not supplied. 6. If any objection was withdrawn because Pith's reviewer misread the paper, the confidence is no higher than moderate, whatever the first pass said: a review that misread the paper once may have misread it elsewhere.","input_kind":"tex","nothing_to_audit":false,"bar":{"enabled":false}},"run_record":{"models":["deepseek/deepseek-v4-flash"],"passes":{"reader":{"model":"deepseek/deepseek-v4-flash","effort":"medium","cost_usd":0.003874,"tokens_in":38347,"tokens_out":3647,"duration_ms":16916,"input_sha256":"29ce797fff5e2519a3e34550840a4f5a6421f3367df8d6420d7cd23a25222163","prompt_sha256":"f9ead730038216984cfb56d7cb34332bdf691828296a1de4245f3d6667d4bf56","message_sha256":"9eeb58dc833fa4dc53138ee7f2f53959baf86feb471c43dbc2e9c96831ee6c92","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"referee":{"model":"deepseek/deepseek-v4-flash","effort":"medium","cost_usd":0.003541,"tokens_in":33881,"tokens_out":2518,"duration_ms":11724,"input_sha256":"29ce797fff5e2519a3e34550840a4f5a6421f3367df8d6420d7cd23a25222163","prompt_sha256":"a852d99aea0ddfd900e1483f76e20777aae3afde9c016d5b8d6e5db01064c728","message_sha256":"ef61e83cc1f3e601b7700d17ce27d766566b838b01f1c607719ffd460ba208e5","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"skeptic":{"model":"deepseek/deepseek-v4-flash","effort":"medium","cost_usd":0.003224,"tokens_in":33618,"tokens_out":906,"duration_ms":6898,"input_sha256":"29ce797fff5e2519a3e34550840a4f5a6421f3367df8d6420d7cd23a25222163","prompt_sha256":"8f88c0f61488b1e764c317998cddf35fffc33a1f20a3b1be54330d7843386fc7","message_sha256":"d9adc96036ef1b81839ed25b5e59edeb92f7615bb868e0c48b3891ba4cf71786","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"circularity":{"model":"deepseek/deepseek-v4-flash","effort":"medium","cost_usd":0.003114,"tokens_in":33727,"tokens_out":245,"duration_ms":3140,"input_sha256":"29ce797fff5e2519a3e34550840a4f5a6421f3367df8d6420d7cd23a25222163","prompt_sha256":"e9946e39d67b76cc60476379bc32abdc44665b5f0c438659796547bb1041a01f","message_sha256":"a1f374b2e045b0fcd09097b6afe67a409c414635fa08f15ad597dab694f6f712","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"desk_editor":{"model":"deepseek/deepseek-v4-flash","effort":"medium","cost_usd":0.003201,"tokens_in":34230,"tokens_out":474,"duration_ms":4345,"input_sha256":"29ce797fff5e2519a3e34550840a4f5a6421f3367df8d6420d7cd23a25222163","prompt_sha256":"e7211f6a3d2734c1025ce7e7e6d48f78b0944d0a32d5021f81be63b27721420a","message_sha256":"3547692743ef5b93a3849893aa60d53a23653909dc2663591185912181c4c61c","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"objection_audit":{"model":"deepseek/deepseek-v4-flash","effort":"medium","cost_usd":0.003681,"tokens_in":34921,"tokens_out":2767,"duration_ms":11668,"input_sha256":"29ce797fff5e2519a3e34550840a4f5a6421f3367df8d6420d7cd23a25222163","prompt_sha256":"6c7a6883d5517cf9e4945d83947200a36dcd152b8b56c32baf7500d8b3841d0a","message_sha256":"33be82e3ba67dc7fa033ddccf5672ff1c87b8f527c152faac40f6d22b7571ff0","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"pith_extraction":{"model":"deepseek/deepseek-v4-flash","effort":"medium","cost_usd":0.003502,"tokens_in":35311,"tokens_out":1587,"duration_ms":8759,"temperature":0.2,"input_sha256":"29ce797fff5e2519a3e34550840a4f5a6421f3367df8d6420d7cd23a25222163","prompt_sha256":"e4f56bf568e7854d0a1d2efb6ec96f8697d5132737cdb31cb7fc64acbd923974","message_sha256":"32e189df72dd58304a4bcedb87dabc60959345de502ea75dcc454403dc9933ea","cache_read_input_tokens":0,"cache_creation_input_tokens":0}},"version":"pith-run-record/v1","prompt_set_sha256":"d60f54226f003b7308ab6f0f9f2951c09c726f00cc66fc05af3456f212247e93","configured_pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium"},"basis":{"kind":"full_text","source":"recorded","chars_given":113590,"chars_total":113590},"receipt":{"review_sha256":"1cadbfb5a4a087f6a39a7b73ec2bcd5a9d3286db92daf9b20cfca829a435ed89","rule":"pith-review-sha256/v1","hashed_at":"2026-11-16T06:56:55.832801+00:00","fields":["id","arxiv_id","paper_version","created_at","pipeline_version","model_set","run_record","input_kind","input_chars","input_total_chars","passes_missing","verdict","confidence","novelty_score","correctness_risk","formal_verification","parameter_count","discovery_kind","one_line_summary","lead","skeptic_critique","referee_report","author_rebuttal","desk_editor","pith_extraction","objection_audit","published_standing","published_confidence"],"matches_current":true},"falsifier":"If a latent representation that is known to contain no action information (e.g., a random noise vector or a constant) achieved a LASR score below one on the ALOHA benchmark, that would falsify the claim that LASR detects action-related predictive support beyond non-action shortcuts.","supporting_citations":[],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":"tex"},"url":"https://pith.science/accuracy"},"review_version":1}