{"id":"f310413a-353b-414e-a985-1d549b7809ad","arxiv_id":"2606.22699","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"On-device extraction of transcript, emotion, intent, fluency, facial behavior and prosody yields deception detection AUC 0.741-0.755 on the Real-life Trial Deception dataset, matching full-video Gemini and Claude while transmitting no media.","lead":"This paper shows that a compact on-device digest of transcript, emotion, facial behavior and prosody from testimony videos detects deception at AUC 0.74-0.755, matching frontier models that receive the full video. The approach keeps raw media on the device, offering a route to privacy-preserving multimodal analysis.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"Digest includes a 'deception intent filter' whose contribution is unablated, risking that reported AUC is not from the other multimodal features.","rationale":"The reader's weakest assumption (completeness of Whissle features) is adjacent but less specific; the filter introduces a potential internal circularity not isolated in the abstract. Full text may clarify the filter, but the absence of an ablation remains the load-bearing gap. This moves the verdict from UNVERDICTED to CONDITIONAL pending the check.","tokens_in":1712,"tokens_out":314,"duration_ms":36579,"concrete_test":"Retrain the small classifier on the released code using the digest with the deception intent filter column removed; if AUC falls below 0.65 while other components remain, the headline equivalence claim does not hold for the non-filter features.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim equates digest performance (AUC 0.741/0.755) to full-video Gemini performance under speaker-independent splits. The digest explicitly contains 'a deception intent filter' alongside transcript, emotion, prosody, etc. If this filter is itself a trained deception detector (or uses deception-labeled data), the matching performance could be driven by that single component rather than the claimed privacy-preserving multimodal digest. The abstract provides no ablation, no description of how the filter is implemented or trained, and no comparison of performance with vs. without it. This directly undermines the premise that the listed on-device features collectively suffice.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.3","summary":"The manuscript claims that on the Real-life Trial Deception dataset, an on-device Whissle stack extracts a compact multimodal digest (transcript, emotion, age, gender, intent distributions, a deception intent filter, fluency and rhythm, per-frame facial behaviour, and prosody) that supports deception detection. Under speaker-independent evaluation, a small classifier on the digest reaches AUC 0.741 (matching Gemini 2.5 Pro on full video) and an LLM on the digest reaches AUC 0.755 (with Claude Opus 4.8 at 7.8X fewer tokens and no media leaving the device). The paper identifies 75% accuracy as a speaker-leakage artifact and releases code and experiments.","tokens_in":1858,"tokens_out":442,"duration_ms":40663,"significance":"If the results hold after addressing the ablation concern, the work would show that competitive deception detection is possible from a compact on-device digest without transmitting raw video, offering a practical privacy benefit and substantial token reduction. The release of code supports reproducibility, and the speaker-leakage observation is a useful dataset insight.","major_comments":[{"comment":"Abstract: The digest includes 'a deception intent filter' but the abstract provides neither an ablation (performance with vs. without the filter) nor a description of how the filter is implemented or trained. If the filter is itself a trained deception detector or uses deception-labeled data, the reported AUCs (0.741/0.755) may be driven by this single component rather than the claimed multimodal features (transcript, emotion, prosody, etc.). This directly affects the central claim that the listed on-device features collectively suffice.","section":"Abstract"},{"comment":"Abstract: The speaker-independent evaluation is invoked to equate digest performance to full-video Gemini performance, yet no details are given on split construction, number of speakers, exclusion criteria, or statistical tests for the AUC comparisons. This information is required to assess whether the matching performance is robust.","section":"Abstract"}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their constructive comments, which help clarify key aspects of our work. We provide point-by-point responses below and will incorporate revisions to address the concerns raised.","responses":[{"response":"We appreciate this observation. The deception intent filter is derived from the intent distributions extracted by the on-device model and does not rely on deception-labeled data for its operation; it applies a general-purpose intent classifier to flag potentially deceptive intents based on linguistic patterns. Nevertheless, to fully address the referee's concern and strengthen the central claim, we will add both a description of the filter's implementation and an ablation study comparing performance with and without the filter to the revised manuscript. This will demonstrate that the multimodal features collectively contribute to the reported performance.","revision_made":"yes","referee_comment":"[Abstract] Abstract: The digest includes 'a deception intent filter' but the abstract provides neither an ablation (performance with vs. without the filter) nor a description of how the filter is implemented or trained. If the filter is itself a trained deception detector or uses deception-labeled data, the reported AUCs (0.741/0.755) may be driven by this single component rather than the claimed multimodal features (transcript, emotion, prosody, etc.). This directly affects the central claim that the listed on-device features collectively suffice."},{"response":"We agree that additional details on the evaluation protocol are essential for assessing robustness. The speaker-independent evaluation employs a leave-one-speaker-out strategy across the dataset's speakers, with exclusion of speakers having fewer than a minimum number of samples to ensure reliable estimates. We will include a new subsection detailing the split construction, the exact number of speakers involved, exclusion criteria, and the statistical tests used for comparing AUC values (including confidence intervals and significance testing) in the revised version of the paper.","revision_made":"yes","referee_comment":"[Abstract] Abstract: The speaker-independent evaluation is invoked to equate digest performance to full-video Gemini performance, yet no details are given on split construction, number of speakers, exclusion criteria, or statistical tests for the AUC comparisons. This information is required to assess whether the matching performance is robust."}],"tokens_in":1396,"tokens_out":472,"duration_ms":34266,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main takeaway is that a compact on-device feature set from Whissle reaches AUC 0.741 with a small classifier and 0.755 when fed to Claude, matching Gemini 2.5 Pro on the full Real-life Trial Deception videos under speaker-independent splits, all while keeping media local. They also flag the 75% accuracy number as speaker leakage and release code.\n\nWhat works is the practical framing: they list the digest components (transcript, emotion, prosody, facial behavior, fluency) and run a direct comparison to external models on a public dataset. The privacy angle is straightforward and the speaker-independent split is the right control. Releasing experiments is useful for anyone trying to replicate on-device pipelines.\n\nThe soft spot is the deception intent filter. The abstract lists it alongside the other features but gives no ablation, no training details, and no performance with versus without it. If that filter was trained on deception labels or acts as its own detector, the matching AUC could come from that single piece rather than the claimed multimodal digest. The abstract does not resolve this, so the central claim that the listed features collectively suffice rests on an untested assumption.\n\nThis is for applied multimodal privacy work or on-device inference groups. It has enough concrete numbers and a clear experimental setup to warrant referee time, even with the filter issue. I would send it for review with a request to add the ablation and clarify how the filter was built.","headline":"The paper shows an on-device digest can match full-video frontier models on deception detection AUC, but the unablated deception intent filter is a load-bearing concern that needs checking.","tokens_in":2357,"tokens_out":376,"would_cite":false,"duration_ms":11581,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Deception detection reaches the same accuracy from a compact on-device digest of video features as from the full raw video.","keywords":["deception detection","privacy-preserving","multimodal analysis","on-device processing","feature digest","speaker-independent evaluation","lie detection","video features"],"falsifier":"A fresh collection of videos from entirely new speakers in which the digest-based AUC falls clearly below the AUC obtained by sending the complete video to a multimodal model.","tokens_in":2606,"feed_emoji":"🔒","tokens_out":735,"duration_ms":31260,"temperature":0.7,"pith_summary":"The paper establishes that testimony videos can be reduced on the device to a short list of derived features before any model sees them. These features include the spoken transcript, emotion labels, age and gender estimates, intent distributions, a deception intent filter, fluency measures, per-frame facial actions, and prosody. When the resulting digest is fed to either a small classifier or a frontier language model, the area under the ROC curve reaches 0.74 to 0.755 under speaker-independent splits, matching what multimodal models achieve on the untouched video. The same accuracy figure collapses once speaker identity cannot leak between training and test examples. Because the original media never leaves the device, the approach removes the need to transmit faces and voices to third-party servers.","feed_headline":"On-device digest matches full video for lie detection","feed_subtitle":"Transcript, emotion and facial features extracted locally reach AUC 0.74 while keeping all media on the device.","key_machinery":"The compact digest of transcript, emotion, age, gender, intent distributions, deception intent filter, fluency and rhythm, per-frame facial behaviour, and prosody extracted by the on-device speech and vision stack.","core_discovery":"On the Real-life Trial Deception dataset and under speaker-independent evaluation, an on-device stack that produces a compact digest of transcript, emotion, age, gender, intent distributions, deception intent filter, fluency and rhythm, per-frame facial behaviour, and prosody supports deception classification at AUC 0.741 with a small model and AUC 0.755 when the digest is passed to a frontier LLM, matching Gemini 2.5 Pro performance on the full video while using 7.8 times fewer tokens and keeping all media local. The commonly cited 75 percent accuracy is shown to be an artifact of speaker leakage rather than genuine generalization.","pith_inferences":["The same digest format could be reused for other on-device behavioral tasks such as emotion regulation or interview coaching without cloud upload.","If the digest proves sufficient across multiple domains, video-analysis pipelines could shift from sending pixels to sending structured summaries by default.","Device makers could embed the extraction step in hardware so that even the digest never reaches application code unless the user consents."],"forward_implications":["Small classifiers trained only on the digest match the accuracy of large multimodal models that receive the full video.","Frontier language models given the digest achieve higher AUC than the same models given raw video while consuming far fewer tokens.","All video and audio can remain on the user's device throughout the detection process.","Accuracy numbers above 75 percent disappear once speaker identity is prevented from leaking across data splits."],"fun_headline_variants":["Device digest matches video lie detection","On-device features match Gemini for spotting lies","Compact local digest equals video model on deception","No media shared yet matches full video deception AUC"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The extracted features contain every piece of information in the raw video that is useful for telling truth from lies.","fun_headline_variants_meta":{"raw":{"variants":["Device digest matches video lie detection","On-device features match Gemini for spotting lies","Compact local digest equals video model on deception","No media shared yet matches full video deception AUC"]},"model":"grok-4.3","cost_usd":0.007241,"raw_usage":{"total_tokens":3337,"prompt_tokens":666,"num_sources_used":0,"completion_tokens":52,"cost_in_usd_ticks":72412000,"prompt_tokens_details":{"text_tokens":666,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2619,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":666,"tokens_out":52,"duration_ms":27428,"temperature":1.0,"reasoning_tokens":2619,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-26T10:30:10.373369+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A fresh collection of videos from entirely new speakers in which the digest-based AUC falls clearly below the AUC obtained by sending the complete video to a multimodal model.","supporting_citations":[],"review_version":1}