{"id":"e09da932-2426-499c-898a-282a004f39d7","arxiv_id":"2611.16423","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"VOKE attaches a lightweight temporal feature module to existing pose estimators to predict object velocity and acceleration, then uses those estimates to refine pose and enable kinematics-augmented reinforcement learning.","lead":"This paper presents VOKE, a system that augments existing visual pose estimators to predict an object's full kinematic state: position, rotation, linear/angular velocity, and linear/angular acceleration, from RGB video. It is relevant for dexterous robot manipulation where knowing how an object moves is as important as knowing where it is, and where explicit motion estimates improve downstream control policies.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"Spline-derived ground truth for velocity/acceleration on real datasets lacks independent validation; simulation results partially mitigate, but central claim remains partly unsubstantiated.","rationale":"The reader's weakest_assumption identifies the same concern: spline-derived ground truth may introduce error. However, the simulation results provide partial support by showing VOKE works with accurate derivatives. Still, the real dataset results are the primary evidence for the central claim, and they lack independent validation of the derivative references. The concrete test proposed would strengthen the evidence without requiring new data collection. The verdict remains CONDITIONAL (unchanged) because the reader's condition already calls for this validation; no change needed yet.","tokens_in":13930,"tokens_out":2995,"duration_ms":24478,"concrete_test":"For a random subset of frames in DexYCB and OakInk, compute the RMS residual of the cubic spline fit to the ground truth translation and rotation trajectories. For translation: compute the L2 norm between the fitted spline at each frame and the original marker-based pose. For angular velocity: compute the residual between the splined angular velocity and a centered difference of the original poses (without splining). Report these residuals relative to the magnitude of the reference velocity/acceleration. If residuals are an order of magnitude smaller than the reported MEE, the spline-derived references are reliable; otherwise, the claimed error reduction may be partly due to smoothing artifacts.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim of substantial reduction in velocity and acceleration estimation error on DexYCB and OakInk relies on spline-derived reference values for training and evaluation (Section IV-A). The only validation of these references is a MuJoCo replay that checks physical plausibility — e.g., that the object motion could be produced by physically consistent forces — but this does not verify derivative accuracy. A spline that overly smooths rapid motions could produce incorrect accelerations while remaining physically plausible, and the replay test would not detect this. The simulation dataset (Isaac Sim) provides accurate ground truth from the simulator and shows VOKE outperforms differencing, but that evaluation uses a cube object with different dynamics and occlusion patterns than the real datasets. Without quantifying the spline fit error (e.g., residual between fitted spline and original marker-based pose), the reader cannot assess whether the reported error reductions reflect true physical accuracy or merely agreement with the chosen spline model. This is load-bearing because the velocity/acceleration results are a primary contribution and the main evidence for the claimed superiority over pose differencing on real benchmarks.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"This paper presents VOKE, a framework that augments existing 6D object/hand-object pose estimators with explicit prediction of linear and angular velocity and acceleration. VOKE contains two modules: a Pluggable Kinematics Module (PKM) that regresses a motion vector from temporal visual features, and a Kinematics-guided Pose Refinement Module (KPRM) that uses the motion vector to build a second-order reference pose and refines per-frame pose estimates via an MLP. The refined pose and motion vector form a complete kinematical state. The authors evaluate VOKE on DexYCB and OakInk by attaching it to three different pose estimation backbones (gSDF, HFL-Net, HOISDF), showing consistent improvements in pose metrics and temporal stability. They also report velocity and acceleration estimation accuracy compared to smoothed pose differencing (Savitzky-Golay). A downstream application in dexterous in-hand cube reorientation (Shaddow Hand) in simlation shows that a RL policy using the complete kinematical state (KARL) converges faster and achieves higher success rate than a pose-history baseline; the policy is transferred zero-shot to a real robot with online VOKE predictions.","tokens_in":14218,"tokens_out":2365,"duration_ms":19613,"significance":"If the results hold, VOKE provides a practical way to endow existing pose estimators with direct kinematical state estimation, which could improve control and planning in robotic manipulation. The modular design (PKM, KPRM) and the systematic evaluation across multiple backbones and datasets are strengths. The work also demonstrates a complete perception-to-control pipeline in a dexterous manipulation task, with sim-to-real transfer, which is valuable. However, the significance is moderated by reliance on spline-derived ground truth for velocity/acceleration on real datasets, which undermines the strength of the comparative results on those benchmarks.","major_comments":[{"comment":"The ground truth velocity and acceleration for the DexYCB and OakInk datasets are derived by fitting cubic splines to the ground truth pose trajectories and differentiating. The authors acknowledge this (Section IV-A) and validate by replaying in MuJoCo to check 'physical meaningfulness'. However, this does not verify that the spline derivatives match the true physical velocities and accelerations. An overly smoothed spline could produce incorrect accelerations while remaining physically plausible. The simulation dataset (Isaac Sim) provides ground truth, but uses a cube with different dynamics and occlusion patterns. The paper should quantify the spline fit error (e.g., residual between fitted spline and original marker-based pose) and discuss how sensitive the results are to the choice of spline smoothing parameter. Without this, the claimed improvements in velocity/acceleration estimation over pose differencing on real datasets are not convincingly supported.","section":"Section IV-A, Paragraph 4 (Motion-quantity reference values); Table II","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The simulation dataset results (Table II, bottom row) show VOKE outperforming pose differencing. However, the reference values for the simulation are derived from the simulator (ground truth velocity) plus spline fitting for acceleration. The paper should state whether the reported 'simulation' results on velocity/acceleration use simulator-provided ground truth for velocity (and thus acceleration derived by spline). If acceleration is derived by spline, the same issue applies, albeit the underlying velocity is known exactly. Please clarify and consider using analytical derivatives for acceleration from the simulator if available.","section":"Table III; Table II (simulation dataset)","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The PKM uses the last k frames of image features. The abaltion shows that k=3 is best. However, the KPRM also uses the same k frames of pose history, and the reference pose is computed from the previous frame's network pose. This couples the frame rate to the motion prior. The paper should discuss the sensitivity to the frame rate (Δt) in Eq. (1), especially for the second-order term, and whether this was tuned or fixed.","section":"Section III-A; Table V","authors_acknowledge":null,"authors_acknowledge_p":null}],"minor_comments":[{"comment":"The phrase 'substantially reduces velocity and acceleration estimation error compared with pose differencing' appears in the abstract/introduction before the method is described. Consider rephrasing to indicate that this claim is validated later.","section":"Section I, line 70-73","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The description of the MuJoCo replay validation is vague: 'the replayed trajectories aligned well with the annotated trajectories.' Please provide a quantitative measure (e.g., positional error over time) to substantiate the claim.","section":"Section IV-A, last paragraph","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The footnote says 'Values absent from the original publications are recomputed from our inference outputs where possible.' It would be helpful to specify which metrics are recomputed versus taken from published tables, espeically for the 'orig.' rows.","section":"Table I, footnote","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"In the sentence 'Every VOKE NRMSE remains below 1, whereas Pose Diff (SG) exceeds 1 for every angular-velocity and acceleration quantity and for all four quantities in simulation,' note that NRMSE exceeding 1 indicates the estimate is worse than predicting the mean; this is a useful interpretation but should be stated explicitly.","section":"Section IV-C, sentence","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"major_revision"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"The paper tackles a real practical gap: most pose estimators only output static pose, leaving downstream controllers to differentiate noisy estimates for velocity and acceleration. VOKE adds a lightweight, pluggable module that predicts the full kinematic state directly from temporal visual features, and uses that state for pose refinement and as policy input in RL. The engineering is solid — three backbones, two real datasets, one simulation dataset, extensive ablations, and a real-robot qualitative demo. The improvements over pose differencing are large and consistent across all metrics reported.\n\nThe soft spot is the ground truth for velocity/acceleration on the real datasets (DexYCB, OakInk). Those values come from cubic splines fitted to ground-truth pose trajectories and then differentiated. The authors try to validate this with a MuJoCo replay, but that only checks physical plausibility, not derivative accuracy. If the spline smooths out rapid motions (common under occlusion or contact), the learned PKM might be fitting to smoothed values rather than true kinematics. The simulation dataset (Isaac Sim) provides exact ground truth and shows VOKE also outperforms pose differencing there, which partially mitigates the concern — but that simulation uses a cube object with different dynamics than the real hand-object interactions. So the central claim of substantial error reduction on real benchmarks rests on an assumption that is not fully quantified.\n\nThe authors acknowledge the limitation for acceleration under rapid contacts, and they release code and data, which is good. I think the paper deserves a serious referee. The method is useful, the experiments are extensive, and the modular design is practical. The main thing a reviewer should push on is a quantitative analysis of the spline fit error (e.g., residuals, comparison with optical flow on known test motions) to bound how much of the reported gain might be due to the reference rather than true physical accuracy. That said, for the simulation and pose-refinement parts, the results stand on their own.\n\nVerdict: worth engaging with. It's a competent, well-scoped contribution that advances the state of open tools for dexterous manipulation. I'd bring it to a reading group focused on perception-for-robotics.","headline":"VOKE is a well-engineered, modular framework for estimating object velocity and acceleration from video, with convincing simulation results and real-world transfer; the main caveat is that its real-dataset evaluations rely on spline-derived kinematics references whose accuracy is not independently validated.","tokens_in":14731,"tokens_out":1226,"duration_ms":10773},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"VOKE augments standard pose estimators with explicit object velocity and acceleration, improving pose accuracy and enabling better robot control.","keywords":["Visual kinematics estimation","Object velocity","Object acceleration","6D pose estimation","Dexterous manipulation","Hand-object interaction","Reinforcement learning","Robotic perception"],"falsifier":"Design an experiment where a high-speed motion capture system (e.g., Vicon or OptiTrack) records the true 6D pose, velocity, and acceleration of an object during dexterous in-hand manipulation with a robotic hand, and compare VOKE's predictions against those measurements. If VOKE's mean Euclidean errors for velocity and acceleration are not significantly lower than the pose-differencing baseline (Savitzky-Golay), the central claim of superior motion estimation would be falsified.","tokens_in":15388,"feed_emoji":"🤖","tokens_out":3545,"duration_ms":29222,"temperature":0.7,"pith_summary":"The paper argues that predicting object velocity and acceleration directly from temporal visual features, rather than differentiating noisy pose estimates, yields a more accurate and temporally consistent kinematic state. This state can be used both to refine per-frame pose estimates and to provide a better observation for reinforcement learning policies. The key claim is that the plaggable kinematics module (PKM) and kinematics-guided pose refinement module (KPRM) together let standard pose estimators produce a complete state (pose, velocity, acceleration) that is consistently more accurate than pose differencing. The authors demonstrate this on two hand-object interaction benchmarks and show that a reinforcement learning policy using this state achieves higher success in dexterous in-hand manipulation.","feed_headline":"Object motion from video: new module beats pose differencing","feed_subtitle":"VOKE predicts velocity and acceleration directly from temporal features, reducing translational RPE by up to 64% in hand-object…","key_machinery":"The framework has two core modules: a Pluggable Kinematics Module (PKM) that shares the image encoder of a base pose estimator, concatenates temporal features from the last few frames, and decodes the motion vector (linear and angular velocity and acceleration) via a lightweight MLP; and a Kinematics-guided Pose Refinement Module (KPRM) that uses this motion vector together with the previous network pose to construct a second-order motion-prior reference pose (Equation 1) and then refines the per-frame pose using a small MLP that also receives the historical pose estimates. The refined pose and motion vector constitute the complete kinematic state consumed by the downstream Kinematics-Augmented Reinforcement Learning (KARL) policy.","core_discovery":"The central discovery is that a lightweight module sharing the image encoder of a standard pose estimator and processing a short temporal window of features can predict linear and angular velocity and acceleration with lower error than third-order Savitzky-Golay differentiation of the estimated pose sequence. Additionally, the predicted motion vector can be integrated via a second-order motion prior to refine the per-frame pose, improving both spatial accuracy and temporal consistency. The complete kinematic state formed by refined pose and motion vector then provides a more informed observation for a reinforcement learning policy, leading to higher success rates in dexterous in-hand cube reorientation.","pith_inferences":["The method's reliance on spline-derived ground-truth velocity and acceleration means that on real-world data with noisy or missing ground-truth poses, the quality of the reference kinematics may degrade; true measured motion from high-speed tracking or inertial sensors could provide stronger validation.","Because PKM shares the backbone encoder, advances in pose estimation (e.g., better occlusion handling or temporal tracking) could directly improve kinematics estimation, making the approach a natural complement to ongoing work in 6D pose tracking.","The framework might generalize beyond hand-object interaction to general rigid-body manipulation (e.g., pick-and-place, stacking) where velocity and acceleration are beneficial for control or dynamics prediction.","The ablation study shows that the explicit motion vector and the second-order reference pose each contribute independently to the pose improvement; further gains might come from replacing the simple MLP refiner with a learned dynamics filter such as a Kalman filter or neural ODE."],"forward_implications":["Standard pose estimators such as gSDF, HFL-Net, and HOISDF can be augmented with PKM to output a complete kinematic state without architectural changes.","Downstream robotic controllers can use explicit velocity and acceleration observations, potentially reducing the need for temporal-history stacking or approximate differentiation.","The kinematic state improves temporal consistency of pose estimates: translational relative pose error is reduced by 41–64% and rotational relative pose error by 7.9–33.3% across backbones and datasets.","A reinforcement learning policy trained with the complete kinematic state in simulation converges faster and achieves higher success (78.90% vs 68.20%) than a policy using only pose history.","The simulation-trained policy can be deployed zero-shot on a physical robot by replacing simulator state with online VOKE predictions, demonstrating a closed perception-to-control pathway."],"supporting_citations":[{"why":"Provides the gSDF backbone: a geometry-driven hand-object reconstruction framework used as one of the three base pose estimators.","marker":"[8]"},{"why":"Provides the HFL-Net backbone: a hand-object pose estimation method using harmonious feature learning.","marker":"[9]"},{"why":"Provides the HOISDF backbone: a hand-object pose estimator using global signed distance fields.","marker":"[10]"},{"why":"Supplies the DexYCB dataset with RGB-D frames, ground-truth poses, and timestamps used for evaluation and derivation of reference kinematics.","marker":"[6]"},{"why":"Supplies the OakInk dataset used as a second evaluation benchmark.","marker":"[7]"},{"why":"Describes the Savitzky-Golay filter used as the baseline method for computing velocity and acceleration from pose sequences.","marker":"[3]"},{"why":"Provides the Isaac Sim simulation environment used for the dexterous manipulation task and policy training.","marker":"[23]"}],"fun_headline_variants":["VOKE beats pose differencing by predicting velocity directly from video","Direct kinematic estimation from video enables real-world cube reorientation","Lightweight module attaches to pose estimator and predicts full motion state","From video to pose, velocity, acceleration: VOKE does it in one pass","Kinematics-aware perception reduces error in dexterous in-hand manipulation"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The ground truth velocity and acceleration for the real-world datasets are derived by fitting cubic splines to the ground truth pose trajectories and differentiating analytically; the training and evaluation rely on the assumption that these spline-based derivatives are sufficiently accurate to serve as reference values, whereas the true physical kinematics are unknown.","fun_headline_variants_meta":{"raw":{"variants":["VOKE beats pose differencing by predicting velocity directly from video","Direct kinematic estimation from video enables real-world cube reorientation","Lightweight module attaches to pose estimator and predicts full motion state","From video to pose, velocity, acceleration: VOKE does it in one pass","Kinematics-aware perception reduces error in dexterous in-hand manipulation"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":0.001312,"raw_usage":{"cost":0.001312248,"is_byok":false,"cost_details":{"upstream_inference_cost":0.001312248,"upstream_inference_prompt_cost":0.000104328,"upstream_inference_completions_cost":0.00120792},"total_tokens":5264,"prompt_tokens":950,"completion_tokens":4314,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":256,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":4221}},"tokens_in":950,"tokens_out":4314,"cost_source":"openrouter_usage","duration_ms":33344,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":4221,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-11-13T15:14:51.170855+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":"Design an experiment where a high-speed motion capture system (e.g., Vicon or OptiTrack) records the true 6D pose, velocity, and acceleration of an object during dexterous in-hand manipulation with a robotic hand, and compare VOKE's predictions against those measurements. If VOKE's mean Euclidean errors for velocity and acceleration are not significantly lower than the pose-differencing baseline (Savitzky-Golay), the central claim of superior motion estimation would be falsified.","supporting_citations":[{"cited_title":"gSDF: Geometry- driven signed distance functions for 3D hand-object reconstruction,","cited_arxiv_id":null,"evidence_quote":"Provides the gSDF backbone: a geometry-driven hand-object reconstruction framework used as one of the three base pose estimators."},{"cited_title":"Harmonious feature learning for interactive hand-object pose estimation,","cited_arxiv_id":null,"evidence_quote":"Provides the HFL-Net backbone: a hand-object pose estimation method using harmonious feature learning."},{"cited_title":"HOISDF: Constraining 3D hand-object pose estimation with global signed distance fields,","cited_arxiv_id":null,"evidence_quote":"Provides the HOISDF backbone: a hand-object pose estimator using global signed distance fields."},{"cited_title":"DexYCB: A benchmark for capturing hand grasping of objects,","cited_arxiv_id":null,"evidence_quote":"Supplies the DexYCB dataset with RGB-D frames, ground-truth poses, and timestamps used for evaluation and derivation of reference kinematics."},{"cited_title":"OakInk: A large-scale knowledge repository for understanding hand-object interaction,","cited_arxiv_id":null,"evidence_quote":"Supplies the OakInk dataset used as a second evaluation benchmark."},{"cited_title":"Smoothing and differentiation of data by simplified least squares procedures,","cited_arxiv_id":null,"evidence_quote":"Describes the Savitzky-Golay filter used as the baseline method for computing velocity and acceleration from pose sequences."},{"cited_title":"Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning","cited_arxiv_id":"2511.04831","evidence_quote":"Provides the Isaac Sim simulation environment used for the dexterous manipulation task and policy training."}],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}