Pith. sign in

REVIEW 2 major objections 6 references

Catching Lies Without Sending the Video: Privacy-Preserving Multimodal Deception Detection

T0 review · 2 major / 0 minor · reviewed 2026-06-26 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Deception detection reaches the same accuracy from a compact on-device digest of video features as from the full raw video.

desk verdict The paper shows an on-device digest can match full-video frontier models on deception detection AUC, but the unablated deception intent filter is a load-bearing concern that needs checking. read the letter →

arxiv 2606.22699 v1 pith:LIHR3IYF submitted 2026-06-21 cs.CV cs.MM

Nikita Sharma , Pranav Sara , Karan Singla This is my paper · ORCID
classification cs.CVcs.MM
keywords deceptiondetectionprivacy-preservingmultimodalanalysison-deviceprocessingfeaturedigestspeaker-independentevaluationlievideofeatures
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper establishes that testimony videos can be reduced on the device to a short list of derived features before any model sees them. These features include the spoken transcript, emotion labels, age and gender estimates, intent distributions, a deception intent filter, fluency measures, per-frame facial actions, and prosody. When the resulting digest is fed to either a small classifier or a frontier language model, the area under the ROC curve reaches 0.74 to 0.755 under speaker-independent splits, matching what multimodal models achieve on the untouched video. The same accuracy figure collapses once speaker identity cannot leak between training and test examples. Because the original media never leaves the device, the approach removes the need to transmit faces and voices to third-party servers.

What carries the argument

The compact digest of transcript, emotion, age, gender, intent distributions, deception intent filter, fluency and rhythm, per-frame facial behaviour, and prosody extracted by the on-device speech and vision stack.

What would settle it

A fresh collection of videos from entirely new speakers in which the digest-based AUC falls clearly below the AUC obtained by sending the complete video to a multimodal model.

Watch

Extended reading notes

Core claim

On the Real-life Trial Deception dataset and under speaker-independent evaluation, an on-device stack that produces a compact digest of transcript, emotion, age, gender, intent distributions, deception intent filter, fluency and rhythm, per-frame facial behaviour, and prosody supports deception classification at AUC 0.741 with a small model and AUC 0.755 when the digest is passed to a frontier LLM, matching Gemini 2.5 Pro performance on the full video while using 7.8 times fewer tokens and keeping all media local. The commonly cited 75 percent accuracy is shown to be an artifact of speaker leakage rather than genuine generalization.

Load-bearing premise

The extracted features contain every piece of information in the raw video that is useful for telling truth from lies.

Editorial extensions

If this is right

  • Small classifiers trained only on the digest match the accuracy of large multimodal models that receive the full video.
  • Frontier language models given the digest achieve higher AUC than the same models given raw video while consuming far fewer tokens.
  • All video and audio can remain on the user's device throughout the detection process.
  • Accuracy numbers above 75 percent disappear once speaker identity is prevented from leaking across data splits.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same digest format could be reused for other on-device behavioral tasks such as emotion regulation or interview coaching without cloud upload.
  • If the digest proves sufficient across multiple domains, video-analysis pipelines could shift from sending pixels to sending structured summaries by default.
  • Device makers could embed the extraction step in hardware so that even the digest never reaches application code unless the user consents.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 0 minor

Summary. The manuscript claims that on the Real-life Trial Deception dataset, an on-device Whissle stack extracts a compact multimodal digest (transcript, emotion, age, gender, intent distributions, a deception intent filter, fluency and rhythm, per-frame facial behaviour, and prosody) that supports deception detection. Under speaker-independent evaluation, a small classifier on the digest reaches AUC 0.741 (matching Gemini 2.5 Pro on full video) and an LLM on the digest reaches AUC 0.755 (with Claude Opus 4.8 at 7.8X fewer tokens and no media leaving the device). The paper identifies 75% accuracy as a speaker-leakage artifact and releases code and experiments.

Significance. If the results hold after addressing the ablation concern, the work would show that competitive deception detection is possible from a compact on-device digest without transmitting raw video, offering a practical privacy benefit and substantial token reduction. The release of code supports reproducibility, and the speaker-leakage observation is a useful dataset insight.

major comments (2)
  1. [Abstract] Abstract: The digest includes 'a deception intent filter' but the abstract provides neither an ablation (performance with vs. without the filter) nor a description of how the filter is implemented or trained. If the filter is itself a trained deception detector or uses deception-labeled data, the reported AUCs (0.741/0.755) may be driven by this single component rather than the claimed multimodal features (transcript, emotion, prosody, etc.). This directly affects the central claim that the listed on-device features collectively suffice.
  2. [Abstract] Abstract: The speaker-independent evaluation is invoked to equate digest performance to full-video Gemini performance, yet no details are given on split construction, number of speakers, exclusion criteria, or statistical tests for the AUC comparisons. This information is required to assess whether the matching performance is robust.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments, which help clarify key aspects of our work. We provide point-by-point responses below and will incorporate revisions to address the concerns raised.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The digest includes 'a deception intent filter' but the abstract provides neither an ablation (performance with vs. without the filter) nor a description of how the filter is implemented or trained. If the filter is itself a trained deception detector or uses deception-labeled data, the reported AUCs (0.741/0.755) may be driven by this single component rather than the claimed multimodal features (transcript, emotion, prosody, etc.). This directly affects the central claim that the listed on-device features collectively suffice.

    Authors: We appreciate this observation. The deception intent filter is derived from the intent distributions extracted by the on-device model and does not rely on deception-labeled data for its operation; it applies a general-purpose intent classifier to flag potentially deceptive intents based on linguistic patterns. Nevertheless, to fully address the referee's concern and strengthen the central claim, we will add both a description of the filter's implementation and an ablation study comparing performance with and without the filter to the revised manuscript. This will demonstrate that the multimodal features collectively contribute to the reported performance. revision: yes

  2. Referee: [Abstract] Abstract: The speaker-independent evaluation is invoked to equate digest performance to full-video Gemini performance, yet no details are given on split construction, number of speakers, exclusion criteria, or statistical tests for the AUC comparisons. This information is required to assess whether the matching performance is robust.

    Authors: We agree that additional details on the evaluation protocol are essential for assessing robustness. The speaker-independent evaluation employs a leave-one-speaker-out strategy across the dataset's speakers, with exclusion of speakers having fewer than a minimum number of samples to ensure reliable estimates. We will include a new subsection detailing the split construction, the exact number of speakers involved, exclusion criteria, and the statistical tests used for comparing AUC values (including confidence intervals and significance testing) in the revised version of the paper. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: empirical AUC comparison on public dataset with external models

full rationale

The paper reports an empirical performance comparison: a classifier and frontier LLMs applied to an on-device feature digest achieve AUC 0.741/0.755, matching Gemini 2.5 Pro on raw video under speaker-independent splits on the Real-life Trial Deception dataset. No equations, fitted parameters, or derivations are presented that reduce the reported AUC values to inputs defined by the target result itself. The digest components (transcript, emotion, etc.) are extracted via an external on-device stack; the deception intent filter is listed but not shown via any quote to be a self-trained deception predictor whose output is then re-used as the prediction target. No self-citation chains or ansatzes are invoked to justify the central claim. This is a standard empirical result with external benchmarks and is self-contained.

Assumptions & free parameters 1 free parameters · 1 assumptions · 0 invented entities

The claim depends on the accuracy of the external Whissle feature extractor and the assumption that the Real-life Trial Deception dataset provides a representative test of deception cues independent of speaker identity.

free parameters (1)
  • small classifier hyperparameters
    The small classifier trained on the digest requires hyperparameter choices that are fitted to the evaluation data.
assumptions (1)
  • domain assumption The Whissle on-device stack extracts reliable transcript, emotion, facial behavior and prosody features from raw video and audio.
    The abstract equates digest performance to full-video performance, which presupposes accurate extraction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Catching Lies Without Sending the Video: Privacy-Preserving Multimodal Deception Detection." pith.science (2026). https://pith.science/paper/LIHR3IYF

@misc{pith2026260622699,
  author       = {Pith},
  title        = {Pith review of: Catching Lies Without Sending the Video: Privacy-Preserving Multimodal Deception Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LIHR3IYF}},
  note         = {Machine review of arXiv:2606.22699}
}
read the original abstract

Frontier multimodal models can guess whether a person is lying from a testimony video. To do so, they stream that raw face and voice to a third-party model. We ask whether the heavy media is needed at all. On the Real-life Trial Deception dataset, Whissle on-device speech and vision stack extracts a compact digest: transcript, emotion, age, gender, intent distributions, a deception intent filter, fluency and rhythm, per-frame facial behaviour, and prosody. Under speaker-independent evaluation, we report three findings. A small classifier on this digest reaches AUC 0.741, matching Gemini 2.5 Pro on full video. Handing the digest to a frontier LLM reaches AUC 0.755 with Claude Opus 4.8 at 7.8X fewer input tokens, with no media leaving the device. The reported 75% accuracy is a speaker-leakage artifact. We release code and experiments.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

6 extracted references · 1 canonical work pages

  1. [1]

    DeceptionDetection Using Real-Life Trial Data

    Pérez-Rosas,Verónica,MohamedAbouelenien,RadaMihalcea,andMihaiBurzo."DeceptionDetection Using Real-Life Trial Data."Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (ICMI), ACM, 2015

    2015

  2. [2]

    Lying Words: Predicting Deception from Linguistic Styles

    Newman, Matthew L., et al. "Lying Words: Predicting Deception from Linguistic Styles."Personality and Social Psychology Bulletin, vol. 29, no. 5, 2003

    2003

  3. [3]

    Ekman, Paul.Telling Lies: Clues to Deceit in the Marketplace, Politics, and Marriage. W. W. Norton, 2001

    2001

  4. [4]

    Bag-of-Lies: A Multimodal Dataset for Deception Detection

    Gupta, Viresh, et al. "Bag-of-Lies: A Multimodal Dataset for Deception Detection."Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), IEEE, 2019

    2019

  5. [5]

    Audio-Visual Deception Detection: DOLOS Dataset and Parameter-Efficient Crossmodal Learning

    Guo, Xiaobao, et al. "Audio-Visual Deception Detection: DOLOS Dataset and Parameter-Efficient Crossmodal Learning."Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), IEEE, 2023. 8whissle.ai

    2023

  6. [6]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

    Gemini Team, Google DeepMind. "Gemini 1.5: Unlocking Multimodal Understanding across Millions of Tokens of Context."arXiv, arXiv:2403.05530, 2024. 9whissle.ai

    work page Pith review arXiv 2024

Pith tools

Reviewed June 26, 2026 · model on record in the stance chip above.