{"id":"0bdeee39-bda8-4dcb-983a-9ab3952de903","arxiv_id":"2606.27731","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"SMMD loss combines MMD with numeric distance kernels and smoothness to improve accuracy on mathematical reasoning, arithmetic, clock recognition, and chart QA across LLMs and VLMs.","lead":"The paper introduces Smooth Maximum Mean Discrepancy (SMMD), a loss that uses value-distance kernels on numeric tokens plus graph smoothness to better match LLM number predictions to targets. A smart generalist might read it to see a practical way to make AI outputs more reliable on math, time, and chart tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"No significant objection identified","rationale":"Reader correctly flagged the provisional nature of an abstract-only review and highlighted the smoothness-bias risk as the weakest assumption. Because the full manuscript is stated to be available and the claim is purely empirical with released code, the appropriate stance is to treat the reported gains as provisionally credible pending reproduction rather than to elevate an unverified bias concern to load-bearing status.","tokens_in":1678,"tokens_out":262,"duration_ms":16024,"concrete_test":"Reproduce the arithmetic and math-reasoning experiments from the released code on one backbone (e.g., Llama-3-8B) using the exact hyper-parameters and seeds reported in the paper; confirm whether mean accuracy gain over cross-entropy exceeds 2 points on both tasks.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim rests on reported accuracy gains from SMMD across four tasks and multiple backbones. The abstract states consistent improvements and notes complementary MMD+smoothness effects plus the importance of distance-based kernels; code is released. No internal inconsistency, hidden assumption, or untestable step is visible at the level of the provided description that would invalidate the headline result if the experiments are as described.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript proposes Smooth Maximum Mean Discrepancy (SMMD) as a training objective for LLMs to improve numerical prediction accuracy. SMMD extends MMD by using kernels based on value distances over numeric tokens and applies graph-based smoothness to encourage local consistency in predictions. The method is evaluated on four tasks—mathematical reasoning, arithmetic calculation, clock-time recognition, and chart question answering—using multiple open-weight LLM and VLM backbones, with claims of consistent improvements over cross-entropy and recent numeric-target losses. Analyses highlight complementary effects of MMD and smoothness, and the importance of distance-based kernels. Code is made available.","tokens_in":1755,"tokens_out":424,"duration_ms":19356,"significance":"If the empirical claims hold, this provides a principled way to incorporate numeric metric structure into LLM training, potentially enhancing performance on tasks requiring precise numeric outputs without architectural changes. The release of code supports reproducibility and further research. The approach could have broad impact in applications like math reasoning and visual QA involving numbers.","major_comments":[{"comment":"§4 (Experiments): the central claim of 'consistent improvements' across four tasks and multiple backbones is presented without reported statistical significance tests, standard deviations across seeds, or confidence intervals on the accuracy deltas; this weakens the ability to assess whether gains are reliable or could be due to variance.","section":"§4"}],"minor_comments":[{"comment":"Abstract: 'recent numeric-target losses' are referenced but not named or cited; adding the specific baselines (e.g., by name or short citation) would improve readability.","section":"Abstract"},{"comment":"§3.1 (Method): the construction of the numeric sub-vocabulary and the exact form of the distance-based kernel could include a short illustrative example with 3-4 tokens to clarify how value distances are computed.","section":"§3.1"}],"recommendation":"minor_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback and the recommendation for minor revision. We address the single major comment below and will update the manuscript accordingly.","responses":[{"response":"We agree that the absence of variability measures and significance testing limits the strength of the empirical claims. In the revised version we will rerun the key experiments with at least three random seeds, report mean accuracy ± standard deviation for each method and task, and include paired t-test p-values (or Wilcoxon signed-rank tests where appropriate) on the per-seed differences between SMMD and the baselines. These results will be added to the main tables in §4 and to the appendix.","revision_made":"yes","referee_comment":"[§4] §4 (Experiments): the central claim of 'consistent improvements' across four tasks and multiple backbones is presented without reported statistical significance tests, standard deviations across seeds, or confidence intervals on the accuracy deltas; this weakens the ability to assess whether gains are reliable or could be due to variance."}],"tokens_in":1286,"tokens_out":231,"duration_ms":9002,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main point is that this paper gives LLMs a loss term that respects the numerical values of tokens instead of treating them as flat categories. They call it Smooth MMD and add two pieces: kernels based on how far apart the actual numbers are, plus a smoothness penalty on the graph those kernels induce.\n\nWhat works is the evaluation setup. They run the method on mathematical reasoning, arithmetic, clock-time recognition, and chart QA, using several open LLMs and VLMs. The abstract says it beats cross-entropy and other recent numeric losses, with some checks that the MMD part and the smoothness part help each other and that distance kernels matter. Releasing the code at the GitHub link is the right move if anyone wants to test it themselves.\n\nThe weaker parts are the lack of detail on how big the gains actually are, whether they survive different random seeds or model scales, and whether the smoothness term creates over-smoothing on some number ranges. The abstract does not show ablations or error bars, so those questions stay open until the full results are checked. The central assumption—that the kernel graph will encourage local consistency without new biases—needs the experiments to back it up.\n\nThis is for groups that fine-tune or train LLMs for science, engineering, or any setting where the model has to output exact numbers rather than plausible text. A reader who cares about loss design for structured outputs will find the kernel construction and the task spread useful.\n\nIt should go to peer review. The method is a clear adaptation of existing tools, the experiments span multiple backbones, and the code makes verification possible.","headline":"SMMD adapts MMD with value-distance kernels and graph smoothness to better align numeric token distributions in LLMs, with reported gains on four tasks and code released.","tokens_in":2216,"tokens_out":407,"would_cite":false,"duration_ms":21240,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Smooth MMD alignment with value-distance kernels and graph smoothness improves numeric prediction accuracy in LLMs over cross-entropy.","keywords":["numerical prediction","LLM","maximum mean discrepancy","smoothness","kernel alignment","arithmetic reasoning","loss function","multimodal models"],"falsifier":"Running the four tasks with SMMD and finding no accuracy gain or outright degradation relative to cross-entropy and recent numeric losses on any backbone would falsify the improvement claim.","tokens_in":2591,"feed_emoji":"🔢","tokens_out":660,"duration_ms":26408,"temperature":0.7,"pith_summary":"Large language models often produce imprecise numeric outputs because cross-entropy treats tokens as unrelated categories rather than ordered values. The paper introduces Smooth Maximum Mean Discrepancy (SMMD), which matches predicted and target distributions using kernels based on numeric distances and then smooths residuals over the resulting token graph to enforce local consistency. Experiments across mathematical reasoning, arithmetic, clock-time, and chart question-answering tasks show consistent accuracy gains on multiple LLM and VLM backbones. Analyses indicate that the kernel-matching and smoothness terms provide complementary benefits, with kernel design choice mattering for results.","feed_headline":"SMMD loss raises numeric accuracy in LLMs","feed_subtitle":"Value-distance kernels plus graph smoothness align outputs better than cross-entropy on math, arithmetic, time, and chart tasks.","key_machinery":"Smooth Maximum Mean Discrepancy (SMMD) loss, which performs kernel matching on value-distance kernels over numeric tokens and smooths the residual over the induced kernel graph.","core_discovery":"SMMD builds on classic MMD by defining kernels over a numeric sub-vocabulary according to value distances, aligns the model's output distribution to the target via this kernel matching, and applies graph-based smoothness to the prediction-target residual over the induced kernel graph, yielding higher accuracy than cross-entropy or prior numeric losses on four numeric-target tasks.","pith_inferences":["The same kernel-plus-smoothness construction could extend to other ordered token sets such as dates or measurement units.","SMMD may reduce specific numeric hallucination patterns that standard alignment methods leave untouched.","Interactions between SMMD and post-training methods like RLHF remain untested and could either reinforce or dilute the numeric alignment.","Over-smoothing risk might appear first on tasks with wide numeric ranges or sparse target distributions."],"forward_implications":["SMMD raises accuracy on mathematical reasoning, arithmetic calculation, clock-time recognition, and chart question answering.","The MMD kernel-matching term and the smoothness term produce complementary gains.","Distance-based kernel design is required for the observed improvements.","The loss applies across multiple open-weight LLM and VLM backbones without architecture changes."],"fun_headline_variants":["SMMD aligns numeric distributions using value distance kernels","Smooth MMD improves LLM accuracy on arithmetic and math tasks","Graph smoothness and distance kernels boost numeric LLM predictions","Numeric kernel matching refines predictions in language models"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"Defining a kernel over a numeric sub-vocabulary and applying graph-based smoothness on the induced kernel graph will produce local consistency without introducing new biases or over-smoothing effects that degrade performance on some numeric ranges.","fun_headline_variants_meta":{"raw":{"variants":["SMMD aligns numeric distributions using value distance kernels","Smooth MMD improves LLM accuracy on arithmetic and math tasks","Graph smoothness and distance kernels boost numeric LLM predictions","Numeric kernel matching refines predictions in language models"]},"model":"grok-4.3","cost_usd":0.006877,"raw_usage":{"total_tokens":3095,"prompt_tokens":635,"num_sources_used":0,"completion_tokens":60,"cost_in_usd_ticks":68765500,"prompt_tokens_details":{"text_tokens":635,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2400,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":635,"tokens_out":60,"duration_ms":24978,"temperature":1.0,"reasoning_tokens":2400,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-29T04:47:31.037279+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Running the four tasks with SMMD and finding no accuracy gain or outright degradation relative to cross-entropy and recent numeric losses on any backbone would falsify the improvement claim.","supporting_citations":[],"review_version":1}