{"id":"211e0ad2-3fcd-4a7e-9f72-ae97dbe59b64","arxiv_id":"2606.30107","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"PHACT uses a propose-certify loop with deterministic derivation from fixed inputs to achieve zero false certifications in 80 adversarial trials across two models and temperatures.","lead":"The paper introduces Physics-Anchored Certification (PHACT), where language models propose physical designs but a deterministic engine alone certifies them by deriving quantities from fixed inputs. This separation aims to prevent forgery and produce reliable outputs even from unreliable models across five scientific domains.","discovery_kind":"unclear","skeptic_critique":{"model":"grok-4.3","headline":"No significant objection identified","rationale":"The reader's note on missing implementation details is accurate given the abstract-only presentation here, but the argument as stated contains no detectable flaw that would alter the unverdicted status. The empirical result and the 'by construction' claim are consistent with each other under the stated premises.","tokens_in":1592,"tokens_out":268,"duration_ms":29617,"concrete_test":"Examine the engine pseudocode or implementation description for any path that reads or incorporates a model-proposed value into the certified quantity; if none exists and all quantities are recomputed from the original fixed inputs, the construction holds.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on an empirical result (zero false certifications across 80 adversarial trials with a deliberately faulted engine) plus the construction that a deterministic engine deriving the certified quantity solely from fixed inputs cannot accept or be influenced by model-supplied values. The paper states this makes forgery impossible by construction. No internal inconsistency, circularity, or mismatch between the stated mechanism and the reported outcome is visible from the given description. The reader's weakest assumption correctly isolates the single point that would need to fail for the claim to collapse, but the provided text does not exhibit evidence that it does fail.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript introduces Physics-Anchored Certification (PHACT), a propose-certify loop in which language models generate candidate physical designs across five domains while a deterministic engine alone performs certification, returning certified, impossible, or unknown. The central claim is that deriving the certified quantity exclusively from fixed inputs (rather than accepting model-supplied values) renders forgery impossible by construction; this is supported by an empirical result of zero false certifications across eighty adversarial trials involving two models, two decoding temperatures, and a deliberately faulted engine.","tokens_in":1691,"tokens_out":377,"duration_ms":19461,"significance":"If the engine is verifiably deterministic and independent of model outputs as described, the approach supplies a structural mechanism for reliable LLM-assisted physical design that does not rely on the model's internal reliability. The empirical count of zero false positives under adversarial conditions, including an intentionally compromised engine, provides concrete evidence supporting the construction; reproducible code or machine-checked proofs of the engine would further strengthen the result.","major_comments":[],"minor_comments":[{"comment":"The description of the certification engine (likely in the methods section) should include explicit pseudocode or a small worked example showing that no model-supplied value can enter the derivation, to make the 'by construction' claim immediately verifiable without reference to external implementation.","section":null},{"comment":"Trial definitions, including how 'adversarial' prompts were constructed and what constitutes a 'false certification,' should be stated with sufficient precision to allow independent replication; the current high-level summary leaves room for selection effects.","section":null},{"comment":"Figure or table summarizing the 80 trials (model, temperature, engine fault, domain, outcome) would improve readability and allow readers to assess coverage at a glance.","section":null}],"recommendation":"minor_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the supportive review and recommendation of minor revision. The assessment correctly identifies the core contribution of PHACT as moving certification authority to a deterministic engine independent of model outputs, with the empirical result of zero false certifications under adversarial conditions. No major comments were raised in the report.","responses":[],"tokens_in":1112,"tokens_out":77,"duration_ms":13938,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The core idea here is moving the final say on physical designs out of the language model entirely. The model proposes, but a separate deterministic engine certifies the result by deriving the quantity straight from fixed inputs, returning certified, impossible, or unknown. That construction is what the paper claims makes forgery impossible by design rather than by hoping the model behaves.\n\nThe work does a solid job spelling out the propose-certify loop and running it across five domains. The test setup includes two models, two temperatures, and an intentionally faulted engine, and it reports zero false certifications. That empirical count is the main evidence offered, and it directly targets the risk of the model sneaking in bad values.\n\nThe soft spot is that everything rests on the engine truly accepting nothing from the model and deriving its output only from the fixed inputs. The abstract states this clearly, but without the actual engine code or precise trial definitions it is difficult to check for any leakage paths or selection effects in how the faults were applied. The 80 trials are a reasonable start, yet they remain an empirical sample rather than a formal guarantee.\n\nThis is aimed at engineers and researchers who already combine language models with simulators or CAD tools and want a lightweight reliability layer. It is not a deep theoretical result, but the pattern is concrete enough that a serious referee could evaluate the engine implementation and the trial protocol in detail. I would send it to peer review.","headline":"PHACT shifts authority to a deterministic engine that certifies from fixed inputs only, with zero false certifications reported across 80 adversarial trials.","tokens_in":2159,"tokens_out":359,"would_cite":false,"duration_ms":17471,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A deterministic engine that derives certification values from fixed inputs can make language model proposals for physical designs trustworthy by preventing forgery.","keywords":["language models","physical designs","certification","reliability","deterministic engine","adversarial trials","propose-certify","forgery prevention"],"falsifier":"An observation of the engine accepting a model-supplied value as the basis for certification and producing a false positive in one of the adversarial trial setups.","tokens_in":2481,"feed_emoji":"","tokens_out":519,"duration_ms":32325,"temperature":0.7,"pith_summary":"The paper aims to show that language models, despite being unreliable on their own, can contribute to reliable physical designs when their role is limited to proposing candidates. A deterministic certification engine then evaluates these proposals by calculating the relevant quantities exclusively from fixed inputs, rather than accepting any data from the model. This setup ensures that false certifications are impossible because the engine cannot be influenced or forged through model outputs. A reader would care about this because it addresses the risk of errors in AI-assisted design for fields where mistakes have physical consequences, such as engineering and scientific modeling. The method was tested extensively with adversarial examples and produced no incorrect certifications.","feed_headline":"Fixed-input engine yields zero false certifications for LM designs","feed_subtitle":"Deriving certified quantities from fixed inputs alone prevents forgery and yields reliable physical designs across multiple domains.","key_machinery":"Physics-Anchored Certification (PHACT), the propose-certify loop that isolates certification to a deterministic engine deriving values from fixed inputs alone.","core_discovery":"An unreliable language model can be made to produce reliable physical designs if the authority to assert is moved out of the model: the model proposes, and a deterministic engine alone certifies, returning certified, impossible, or unknown. We introduce Physics-Anchored Certification (PHACT), a propose-certify loop spanning five scientific domains, and identify what makes such a certificate trustworthy. A checker that accepts a model-supplied value can be forged; deriving the certified quantity from fixed inputs instead makes forgery impossible by construction. Across eighty adversarial trials spanning two models, two decoding temperatures, and a deliberately faulted engine, this contract pr","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Fixed inputs prevent forgery in LM physical designs","PHACT certifies from fixed inputs with zero errors","Model proposes engine certifies physical designs","Fixed input derivation stops certification forgery","Eighty trials show no false positives with PHACT"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The deterministic certification engine correctly derives the certified quantity solely from fixed inputs with no possibility of accepting or being influenced by model-supplied values, and the engine implementation itself cannot be forged or bypassed.","fun_headline_variants_meta":{"raw":{"variants":["Fixed inputs prevent forgery in LM physical designs","PHACT certifies from fixed inputs with zero errors","Model proposes engine certifies physical designs","Fixed input derivation stops certification forgery","Eighty trials show no false positives with PHACT"]},"model":"grok-4.3","cost_usd":0.006027,"raw_usage":{"total_tokens":2814,"prompt_tokens":591,"num_sources_used":0,"completion_tokens":65,"cost_in_usd_ticks":60274500,"prompt_tokens_details":{"text_tokens":591,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2158,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":591,"tokens_out":65,"duration_ms":21397,"temperature":1.0,"reasoning_tokens":2158,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-30T05:48:47.518861+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"An observation of the engine accepting a model-supplied value as the basis for certification and producing a false positive in one of the adversarial trial setups.","supporting_citations":[],"review_version":1}