Pith. sign in

REVIEW 4 major objections 8 minor 35 references

Beyond Visual Evidence: Revealing and Mitigating Relational Privacy Leakage in Document MLLMs

T0 review · 4 major / 8 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Document MLLMs leak correlated identity fields when visual evidence is missing, and the proposed DRUF framework suppresses this leakage.

desk verdict A real relational-leakage risk and a useful benchmark, but the unlearning claims rest on an evaluation that erases known training pairs rather than demonstrating generalization to held-out identities. read the letter →

arxiv 2608.12911 v1 pith:SAHD6IQ4 submitted 2026-08-13 cs.CV cs.CRcs.MM

Beining Xu , Hairui Wang , Jiaxin Wang , Changsheng Chen , Anirban Chakraborty This is my paper · ORCID
classification cs.CVcs.CRcs.MM
keywords MultimodalLargeLanguageModelsPrivacyLeakageMachineUnlearningKeyInformationExtractionDocumentUnderstandingRelationalDynamicIdentityDocuments
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Document MLLMs trained for key information extraction on identity documents memorize associations between fields such as family name, given name, and document number. When the input image provides no valid visual evidence, the paper argues, these models fall back on the memorized relations and jointly generate multiple sensitive fields from a single training identity — a risk it calls relational privacy leakage, which prior single-field analyses missed. To counter it, the paper builds DRUF, an unlearning framework whose forgetting objective couples the two leaked spans, and DocPrivacyBench, a benchmark that measures leakage under image-driven and prompt-driven attacks. Across three MLLMs and six unlearning baselines, DRUF reduces Image-Driven leakage accuracy from 0.642 to 0.001, brings Prompt-Driven leakage to 0.000, and keeps normal KIE extraction quality at 0.808, a gain of 4.8 percentage points over the strongest baseline.

What carries the argument

The load-bearing object is the product coupling of span-level KL divergences, $L_{\mathrm{forget}} = -K_a K_b$, where $K_a = \mathrm{KL}_{\mathrm{span}}(s_a^i)$ and $K_b = \mathrm{KL}_{\mathrm{span}}(s_b^i)$ are the average teacher–student KL shifts over the token spans of two sensitive fields jointly exposed by the model. Because $\partial L_{\mathrm{forget}}/\partial K_a = -K_b$ and $\partial L_{\mathrm{forget}}/\partial K_b = -K_a$, each span's update is weighted by the other span's shift, which couples the two fields into one relation-level forgetting target and suppresses their joint recoverability under blank or visually uninformative inputs without erasing whole documents. Around this core, DRUF adds a retain branch that aligns the student with a frozen teacher on normal KIE samples, and a dynamic forget set $F(t)$ rebuilt periodically from probing rounds so that the forgetting targets follow the model's current leakage behavior rather than a static list.

What would settle it

Train a document MLLM on DocXPand-25k, run DRUF using probing pairs detected on one split of the training identities, then evaluate leakage on a separate split of identities that never contributed to the dynamic forget set; if leakage accuracy remains near zero and KIE LC stays above 0.8, the relational-unlearning claim generalizes, and if leakage returns on those unseen identities, the reported suppression is targeted erasure of known pairs.

Watch

Extended reading notes

Core claim

The paper's central discovery is that relational privacy leakage is a distinct failure mode of document MLLMs: under weak-evidence inputs such as blank or noise-degraded document images, these models reproduce correlated sensitive field pairs — for instance, given name and document number — that co-occurred in the identities they were trained on, instead of grounding their output in the image. The proposed framework treats each exposed field pair as a coupled forgetting unit: Relational Decoupling Unlearning computes span-level KL shifts $K_a$ and $K_b$ between a frozen teacher and a trainable student over the two sensitive spans and applies the objective $L_{\mathrm{forget}} = -K_a K_b$, whose gradients weight each span by the other's shift and thereby break the stable co-generation of the pair. The forgetting targets are not preset: a dynamic set $F(t)$ collects the pairs the current model actually leaks on probing inputs and is refreshed across unlearning rounds, so the suppression tracks the model's evolving behavior. On DocPrivacyBench, the framework lowers Image-Driven leakage accuracy from 0.642 to 0.001 and Prompt-Driven leakage to 0.000 in the dual-field setting, keeps the normal KIE LC at 0.808, and improves leakage suppression by 4.8 percentage points over the strongest baseline.

Load-bearing premise

The method assumes the pairs it discovers during probing and the identities it tests afterward are not the same memorized records; if they overlap, the reported suppression is targeted deletion of known pairs, not a general ability to stop leakage.

Editorial extensions

If this is right

  • On noisier, lower-quality datasets the same model leaks far more: the strongest-leaking tested model goes from an Image-Driven Acc@1.0 of 0.004 on clean IDNet to 0.346 after noise is added, and 0.835 on DocXPand-25k.
  • Aggressive sample-level unlearning (GA, GA+KL) drives leakage to near zero but collapses normal KIE utility to LC 0.119–0.259, while static preference or fine-tuning baselines leave leakage rates of 0.175–0.676, so pair-level relational forgetting is the granularity that preserves utility.
  • A static forget set built once is not enough: with RDU but no dynamic update, residual leakage remains at 0.002 (Prompt-Driven) and 0.006 (Image-Driven), whereas refreshing the forget set reaches 0.000.
  • The probing test set should be moderate: 100 samples suppresses leakage to 0.001 with KIE LC 0.808, while 500 samples collapses KIE LC to 0.306 without reducing leakage further.
  • Training-similar face inputs leak over more unique identities (55) than unrelated photos (32), so the risk persists under both irrelevant and distribution-similar visual triggers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The benchmark and probing procedure draw on the same 5,000-sample training identity library; a held-out identity split would show whether DRUF suppresses leakage on identities it never probed or merely erases the exact pairs it was told to forget.
  • The pair-coupling objective is a general recipe for structured outputs: receipts, medical forms, and financial documents could use the same $-K_aK_b$ coupling to stop correlated fields from being co-recoverable when inputs are degraded.
  • Leakage matching at token-level similarity with two-of-three private fields means a model that shuffles values between identities or lightly paraphrases would lower the measured score without being genuinely safe; testing output membership against the full training library would close that gap.
  • Because Image-Driven attacks leak more than Prompt-Driven attacks in almost every configuration, deployment-time defenses should treat abnormal or blank images as the primary attack surface rather than focusing on prompt hardening.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. The paper studies relational privacy leakage in document-understanding multimodal LLMs (MLLMs): when an input image lacks valid visual evidence, the model may fall back on memorized field associations and jointly generate multiple correlated sensitive fields (e.g., family name and document number) from training identities. The authors propose the Dynamic Relational Unlearning Framework (DRUF), which probes the current model under blank-image inputs to build a dynamic forget set of leaked field pairs, then applies a relation-level forgetting loss (the product of span-level KL divergences) to suppress the joint generation of those pairs while a teacher–student retain branch preserves KIE utility. They also introduce DocPrivacyBench, a benchmark for measuring such leakage under Image-Driven and Prompt-Driven settings at multiple similarity thresholds, and they report experiments on three document MLLMs and six unlearning baselines. The headlined results show DRUF reducing leakage accuracy to near zero on the tested LLaVA-1.5-hf/DocXPand-25k configuration while keeping KIE LC around 0.808.

Significance. The problem is well motivated and timely: relational leakage under weak visual evidence is a realistic privacy risk for document-processing MLLMs, and the paper offers a concrete benchmark (DocPrivacyBench) and a new unlearning formulation (pair-level KL-coupled forgetting with a dynamic forget set). The release of code and benchmark is a positive artifact. If the central claims survive a stricter evaluation, the paper would provide a useful starting point for privacy-preserving document MLLMs. However, the current evidence is not yet sufficient to support the claimed generality of DRUF: the unlearning evaluation is performed on one model–dataset pair, and the probing/forgetting/evaluation identities overlap, so the reported suppression may reflect targeted erasure of the exact pairs placed in the forget set rather than a general relational unlearning capability.

major comments (4)
  1. [Section 5.3, Table 1] The claim that lower data quality generally leads to higher privacy leakage is not robust to the leakage threshold. For LLaVA-1.5-hf Image Driven, Acc@0.8 goes from 0.874 (DocXPand-25k) to 0.910 (IDNet) to 1.000 (IDNet with noise), while Acc@1.0 goes from 0.835 to 0.004 to 0.346, and LC goes from 0.972 to 0.809 to 0.882. Under the 0.8 threshold, the cleaner IDNet appears more leaky than DocXPand, whereas under the 1.0 threshold it is drastically less leaky, reversing the paper's data-quality narrative. The paper should justify the threshold choice and report the similarity-score distribution or a threshold-free metric, and it should explain the sharp transition from Acc@0.8=0.910 to Acc@0.9=0.004 for this cell.
  2. [Section 3.2, Supp D.1, Table 2] The unlearning evaluation does not separate the identities used to build the dynamic forget set from those used to score leakage. F(t) is constructed from pairs leaked by the current model on probing inputs, and the leakage evaluation matches outputs against the same training identity library; Supp D.1 states that the 6,857 probing/evaluation face crops are derived from the same 5,000 DocXPand-25k samples used to train the target model. With no identity-level split between probe/forget targets and evaluation identities, the near-zero leakage in Table 2 can be explained entirely by erasing the exact associations placed in F(t), and the Section 3.2 claim that suppression 'generalizes to other data points that have not been probed' is unsupported. Please add a held-out-identity control: train on set A, probe and forget on a subset of A, and evaluate leakage on the remaining training identities that were never probed, plus a false-positive control using identities not in training.
  3. [Section 5.4, Table 2] The unlearning comparison is run on a single model (LLaVA-1.5-hf) and a single dataset (DocXPand-25k), but the abstract states that 'we evaluate three MLLMs and six unlearning methods using this benchmark'. No unlearning results are reported for Xgen-Phi3, Idefics2, or for the IDNet datasets. The six-method comparison therefore supports conclusions only for one model–dataset pair, and the headline 4.8-percentage-point improvement over SCRUB is a single-cell result. Please either add unlearning experiments on at least one more model and one more dataset, or restrict the claims to the actual scope.
  4. [Supp F.1, Table 5] The ablation designed to show the benefit of the dynamic forget set is confounded. The 'Static Forget Set + RDU' row in Table 5 achieves Prompt-Driven/Image-Driven Acc of 0.002/0.006 with KIE LC 0.996, nearly identical to DRUF's 0.000/0.000 with KIE LC 0.996. The 'substantial degradation' (KIE LC 0.055) comes from a separate experiment with 500 static targets and 500 training steps, which is not a controlled comparison with the DRUF configuration (which uses the standard training schedule and dynamic updates at intervals). Thus the paper does not currently demonstrate that the dynamic update mechanism materially improves leakage suppression or utility preservation over a static set. Please report a matched static baseline using the same training schedule, forget-set size, and number of updates.
minor comments (8)
  1. [Table 1] The abrupt transition from Acc@0.8=0.910 to Acc@0.9=0.004 for LLaVA-1.5-hf on IDNet Image Driven is surprising; please report the similarity-score distribution and the exact matching rule at each threshold.
  2. [Table 2 caption] The parenthetical numbers, e.g., '(55)', are said in the text to be numbers of leaked pairs, but the caption does not define them; please define the notation and clarify whether these counts are distinct identities.
  3. [Figure 2] The caption contains the literal text 'family\u name' (with a backslash-u escape); this appears to be a rendering error and should be fixed.
  4. [Section 5.4] The unlearning evaluation switches to a dual-private-field setting (given name and document number) at threshold 1.0, which differs from the three-field, multi-threshold setting of Table 1; please state this explicitly in the main text and justify the change, since it complicates direct comparison of leakage rates between sections.
  5. [Abstract] The '4.8 percentage points' improvement should be identified as the Image-Driven leakage-accuracy improvement over SCRUB for LLaVA-1.5-hf on DocXPand-25k, and the scope of the claim should be stated.
  6. [Supp C.1] The heading 'Experimental Setup.' is duplicated immediately before the description of the training-process analysis.
  7. [Section 3.2, reference [7]] Reference [7] is cited to support the claim that unlearning on exposed leakage points generalizes to unprobed data points; this is a central assumption, and the citation alone does not demonstrate that the dynamic forget-set approach generalizes across identities. Please either provide direct evidence or cite a study that establishes this generalization mechanism for similar settings.
  8. [Section 3.3] The forgetting objective L_forget = -K_a K_b is non-positive, and its minimization drives the product of span-level KL divergences upward; please state the intended optimization direction explicitly and clarify that the goal is to maximize the product of the two KL shifts.

Circularity Check

2 steps flagged · score 6.0 of 10

The reported leakage suppression is measured on the same field pairs and identity pool used to build the forget set, so the headline result can be explained as targeted pair erasure rather than demonstrated relational unlearning.

  1. fitted input called prediction [Section 3.2 (dynamic forget set), Section 5.2 (cached forgetting targets), Table 2 / Section 5.1 (dual-pair leakage metric)]
    "The dynamic forget set at probing-unlearning round t is defined as the set of distinct leaked sensitive field pairs detected in the current probing round: F(t) = {(s_a, s_b) | (s_a, s_b) is detected as leaked in probing round t}. ... Because our forgetting targets are derived from the pre-unlearning evaluation stage, all methods considered in this paper use the dynamically collected cached pairs as the forgetting targets. ..."

    F(t) is built by running the current model on probing inputs and retaining exactly the field pairs it leaks; RDU's objective then applies -K_a K_b to those pairs. Table 2's headline metric counts exactly the dual-private-field pair (given name + document number) at threshold 1.0, and Section 5.2 states every method, including DRUF, receives these same dynamically collected cached pairs as forgetting targets. The reported suppression (Prompt-Driven 0.659 to 0.000, Image-Driven 0.642 to 0.001) is therefore the direct outcome of erasing the very associations used as supervision, not an independent measurement of relational unlearning on unseen pairs.

  2. other [Supplementary D.1 (test images) vs. Section 5.1 (training split)]
    "The test images are 6,857 face crops without valid visual evidence, obtained from DocXPand-25k and derived from 5,000 samples, which are used as the model input."

    Section 5.1 trains each target model on 5,000 samples from DocXPand-25k, and Supplementary D.1 uses 6,857 face crops 'derived from 5,000 samples' as the test inputs; the leakage scorer in Figure 3 matches outputs against a training identity library. No identity-level holdout between the probe/forget targets and the evaluation identities is described. The evaluation therefore does not separate the associations placed into F(t) from the associations used to score leakage, so the near-zero leakage in Table 2 can be explained entirely by targeted erasure of known memorized identities rather than by a general capability to suppress relational leakage on unseen identities.

full rationale

The paper is internally consistent and the comparison among unlearning methods is fair in a narrow sense: all baselines receive the same cached leakage targets, so their relative behavior under identical supervision is meaningful. The retain-branch utility evaluation on the normal KIE task is also an independent check and shows DRUF preserves utility better than aggressive baselines. However, the central claimed capability of DRUF is relational unlearning that generalizes beyond the probed leakage points, and that capability is not demonstrated. The dynamic forget set is constructed from the current model's own leaked pairs under probing, all methods are given those cached pairs as forgetting targets, and the leakage evaluation counts the same dual-field pair at threshold 1.0 against the same training identity pool. Supplementary D.1 confirms the test images are face crops derived from 5,000 DocXPand samples, with no held-out identity split described. The headline suppression is thus explainable as targeted pair erasure rather than as evidence of general relational unlearning. This is a partial circularity in the evaluation protocol, not a derivation that is identical to its input by definition, and no load-bearing self-citation or imported uniqueness theorem was found; a held-out identity split would be the missing control.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claims rest on hyperparameters selected by sweep or left unspecified, plus domain assumptions about what blank-image outputs mean and how span-level KL relates to joint leakage. No new physical or architectural entities are introduced.

free parameters (5)
  • lambda (retention-forgetting weight) = not reported in main text
    Balances L_retain and L_forget in the overall objective. No value, range, or selection procedure is given.
  • gamma (retain-forget sampling ratio) = 3 (default, Supp E)
    Chosen via a sweep over 1..5 (Fig 9); at gamma=3 the method achieves zero leakage and KIE 0.808, but no held-out validation is used.
  • evaluation interval Delta for dynamic forget-set updates = every 500 training steps (Supp C)
    Controls how often the forget set is rebuilt; no sensitivity analysis for Delta is reported.
  • probing test set size = 100 (default, Supp D)
    Test-set-size sweep (10/100/500) shows 100 balances leakage and KIE; enlarging to 500 collapses KIE to 0.306.
  • leakage decision in unlearning experiments = dual pair (given_name, document_number) at threshold 1.0
    The benchmark uses any-two-of-three fields at thresholds 0.8/0.9/1.0, but the headline unlearning comparison narrows to one pair and exact match, making suppression easier and the 4.8pp claim specific to that choice.
assumptions (4)
  • domain assumption Blank or noisy images carry no valid visual evidence, so any recovered identity information comes from parametric memory rather than the input image.
    Used throughout Sec 3.2 and Supp A.3; if blank images still retain statistical cues, the leakage attribution to memory is not clean.
  • domain assumption Suppressing the detected leakage pairs generalizes to unprobed samples under the same dataset and task distribution.
    Sec 3.2 invokes generalization from F(t) to other samples; the only evidence is the test-set-size experiment in Supp D, which uses the same training identity pool.
  • domain assumption The teacher-student KL divergence over the token spans of two sensitive fields is a valid measure of their joint recoverability, and maximizing the product of the two KL shifts breaks the joint generation.
    Sec 3.3 defines the loss; no formal argument or calibration connects this objective to actual joint leakage probability.
  • domain assumption Top-2-of-3 threshold matching against the training identity library correctly classifies true leakage.
    Sec 5.1 defines the metric; string similarity thresholds can overcount partial matches, as seen in IDNet Acc@0.8=0.910 but Acc@0.9=0.004.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Beyond Visual Evidence: Revealing and Mitigating Relational Privacy Leakage in Document MLLMs." pith.science (2026). https://pith.science/paper/SAHD6IQ4

@misc{pith2026260812911,
  author       = {Pith},
  title        = {Pith review of: Beyond Visual Evidence: Revealing and Mitigating Relational Privacy Leakage in Document MLLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SAHD6IQ4}},
  note         = {Machine review of arXiv:2608.12911}
}
read the original abstract

While the privacy risks of multimodal large language models (MLLMs) have drawn significant attention, the unique vulnerabilities of domain-specific MLLMs remain largely underexplored. Focusing on document understanding MLLMs for identity document processing, this paper investigates the privacy issues inherent in Key Information Extraction (KIE) tasks. We reveal that when input images lack sufficient visual evidence, these models often rely on memorized field relations from training data to infer missing content, thereby leaking multiple correlated fields containing sensitive personal information. To mitigate this risk, we make three key contributions.First, we propose the Dynamic Relational Unlearning Framework (DRUF) which comprises a Relational Decoupling Unlearning (RDU) module and a dynamic set update mechanism. It suppresses the leakage of high-risk field pairs while preserving KIE performance.Second, we introduce DocPrivacyBench, a novel benchmark to systematically evaluate a model's susceptibility to privacy leakage under conditions of absent or minimal visual evidence.Third, we evaluate three MLLMs and six unlearning methods using this benchmark, assessing both post-unlearning leakage suppression and utility preservation.Our results demonstrate that existing MLLMs consistently exhibit privacy leakage when visual evidence is scarce, particularly on noisier datasets. In contrast, DRUF outperforms the strongest baseline by improving leakage suppression by 4.8 percentage points, effectively mitigating privacy risks while maintaining robust document information extraction performance.

Figures

Figures reproduced from arXiv: 2608.12911 by the authors.

Figure 1
Figure 1. Unlike prior work[20] on single-field leakage, we show that relational privacy leakage in document MLLMs occurs when models jointly reveal correlated sensitive fields under abnormal or weak-evidence inputs. Abstract While the privacy risks of multimodal large language models (MLLMs) have drawn significant attention, the unique vulnerabilities of domain-specific MLLMs remain largely underexplored. Focusing on documen… view at source ↗
Figure 2
Figure 2. Overview of DRUF. The Retain branch uses normal document samples and a frozen teacher model to preserve the [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Overview of the protocol for document privacy leakage evaluation. The framework evaluates model leakage under both [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison of unlearning methods on [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Ablation study of privacy leakage across image [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Comparison of normal-task utility preservation and privacy leakage across GA, DRUF, and SCRUB over training [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Training process comparison of DRUF and SCRUB in terms of normal KIE capability and leakage suppression. The [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Effect of test set size on privacy leakage suppression [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Effect of 𝛾 on privacy leakage suppression and util￾ity preservation. The figure reports the leak rate, average best similarity, and KIE performance under different 𝛾 settings. Among all settings, 𝛾 = 3 achieves the best overall trade-off. E Analysis of the Sampling Ra…
Figure 10
Figure 10. Figure 10: Prompt examples for the three prompt types used in our Prompt-Driven evaluation: NIQP, SIEP, and ECAP. NIQP [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Illustration of the noise processing applied to a subset of IDNet samples. Each original document image is first [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 20 canonical work pages

  1. [1]

    Srikar Appalaraju, Bhavan Jasani, Bhargava Urala Kota, Yusheng Xie, and R Manmatha. 2021. DocFormer: End-to-end transformer for document understand- ing. InProceedings of the IEEE/CVF International Conference on Computer Vision. 993–1003

    work page 2021

  2. [2]

    Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. 2021. Extracting training data from large language models. In30th USENIX Security Symposium (USENIX Security 21). 2633–2650

    work page 2021

  3. [3]

    Tiejin Chen, Pingzhi Li, Kaixiong Zhou, Tianlong Chen, and Hua Wei. 2025. Unveiling privacy risks in multi-modal large language models: Task-specific vulnerabilities and mitigation challenges. InFindings of the Association for Com- putational Linguistics: ACL 2025. 4573–4586

    work page 2025

  4. [4]

    Tiejin Chen, Pingzhi Li, Kaixiong Zhou, Tianlong Chen, and Hua Wei. 2025. Vision language model helps private information de-identification in vision data. InFindings of the Association for Computational Linguistics: ACL 2025. 4558–4572

    work page 2025

  5. [5]

    Jiali Cheng and Hadi Amiri. 2024. Multidelete for multimodal machine unlearning. InEuropean Conference on Computer Vision. Springer, 165–184

    work page 2024

  6. [6]

    Badhan Chandra Das, M Hadi Amini, and Yanzhao Wu. 2025. Security and privacy challenges of large language models: A survey.Comput. Surveys57, 6 (2025), 1–39

    2025

  7. [7]

    Thomas De Min, Massimiliano Mancini, Stéphane Lathuilière, Subhankar Roy, and Elisa Ricci. 2024. Unlearning personal data from a single image.arXiv preprint arXiv:2407.12069(2024)

    arXiv 2024

  8. [8]

    Jérémie Dentan, Arnaud Paran, and Aymen Shabou. 2024. Reconstructing training data from document understanding models. In33rd USENIX Security Symposium (USENIX Security 24). 6813–6830

    work page 2024

Show all 35 references
  1. [9]

    Teakgyu Hong, Donghyun Kim, Mingi Ji, Wonseok Hwang, Daehyun Nam, and Sungrae Park. 2022. BROS: A pre-trained language model focusing on text and layout for better key information extraction from documents. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. ...

    2022

  2. [10]

    Zheng Huang, Kai Chen, Jianhua He, Xiang Bai, Dimosthenis Karatzas, Shijian Lu, and CV Jawahar. 2019. ICDAR2019 competition on scanned receipt OCR and information extraction. In2019 International Conference on Document Analysis and Recognition (ICDAR). IEEE, 1516–1520

    2019

  3. [11]

    Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. 2023. Knowledge unlearning for mitigating privacy risks in language models. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume ...

    2023

  4. [12]

    Jinghan Jia, Jiancheng Liu, Yihua Zhang, Parikshit Ram, Nathalie Baracaldo, and Sijia Liu. 2024. WAGLE: Strategic weight attribution for effective and modular unlearning in large language models.Advances in Neural Information Processing Systems37 (2024), 55620–55646

    2024

  5. [13]

    Meghdad Kurmanji, Peter Triantafillou, Jamie Hayes, and Eleni Triantafillou

  6. [14]

    Julien Lerouge, Guillaume Betmont, Thomas Bres, Evgeny Stepankevich, and Alexis Bergès. 2024. DocXPand-25k: A large and diverse benchmark dataset for identity documents analysis.arXiv preprint arXiv:2407.20662(2024)

    2024 arXiv

  7. [15]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. BLIP-2: Bootstrap- ping language-image pre-training with frozen image encoders and large language models. InInternational Conference on Machine Learning. PMLR, 19730–19742

    2023

  8. [16]

    Jiaqi Li, Qianshan Wei, Chuanyi Zhang, Guilin Qi, Miaozeng Du, Yongrui Chen, Sheng Bi, and Fan Liu. 2024. Single image unlearning: Efficient machine un- learning in multimodal large language models.Advances in Neural Information Processing Systems37 (2024), 35414–35453

    2024

  9. [17]

    Yulin Li, Yuxi Qian, Yuechen Yu, Xiameng Qin, Chengquan Zhang, Yan Liu, Kun Yao, Junyu Han, Jingtuo Liu, and Errui Ding. 2021. StructText: Structured text understanding with multi-modal transformers. InProceedings of the 29th ACM International Conference on Multimedia. 1912–1920

    2021

  10. [18]

    Zhan Li, Yongtao Wu, Yihang Chen, Francesco Tonin, Elias Abad Rocamora, and Volkan Cevher. 2024. Membership inference attacks against large vision- language models.Advances in Neural Information Processing Systems37 (2024), 98645–98674

    2024

  11. [19]

    Xinwei Liu, Xiaojun Jia, Yuan Xun, Siyuan Liang, and Xiaochun Cao. 2024. Mul- timodal unlearnable examples: Protecting data against multimodal contrastive learning. InProceedings of the 32nd ACM International Conference on Multimedia. 8024–8033

    2024

  12. [20]

    Francesco Pinto, Nathalie Rauschmayr, Florian Tramèr, Philip Torr, and Federico Tombari. 2024. Extracting training data from document-based VQA models.arXiv preprint arXiv:2407.08707(2024)

    2024 arXiv

  13. [21]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model.Advances in Neural Information Processing Systems36 (2023), 53728–53741

    2023

  14. [22]

    Saifullah Saifullah, Stefan Agne, Andreas Dengel, and Sheraz Ahmed. 2023. PrIeD- KIE: Towards Privacy Preserved Document Key Information Extraction.arXiv preprint arXiv:2310.03777(2023)

    2023

  15. [23]

    GM Shahariar, Zabir Al Nazi, Md Olid Hasan Bhuiyan, and Zhouxing Shi. 2026. PII-VisBench: Evaluating Personally Identifiable Information Safety in Vision Language Models Along a Continuum of Visibility.arXiv preprint arXiv:2601.05739 (2026)

    2026

  16. [24]

    Ayush Kumar Tarun, Vikram Singh Chundawat, Murari Mandal, and Mohan Kankanhalli. 2023. Deep regression unlearning. InInternational Conference on Machine Learning. PMLR, 33921–33939

    2023

  17. [25]

    Bozhong Tian, Xiaozhuan Liang, Siyuan Cheng, Qingbin Liu, Mengru Wang, Dianbo Sui, Xi Chen, Huajun Chen, and Ningyu Zhang. 2024. To forget or not? Towards practical knowledge unlearning for large language models. InFindings of the Association for Computational Linguistics: EMN...

    2024

  18. [26]

    Rubèn Tito, Khanh Nguyen, Marlon Tobaben, Raouf Kerkouche, Mohamed Ali Souibgui, Kangsoo Jung, Joonas Jälkö, Vincent Poulain D’Andecy, Aurelie Joseph, Lei Kang, et al. 2024. Privacy-aware document visual question answering. In International Conference on Document Analysis and ...

    2024

  19. [27]

    Lulu Xie, Yancheng Wang, Hong Guan, Soham Nag, Rajeev Goel, Niranjan Er- appa Narayana Swamy, Yingzhen Yang, Chaowei Xiao, Jonathan Prisby, Ross Maciejewski, and Jia Zou. 2024. IDNet: A novel identity document dataset via few-shot and quality-driven synthetic data generation. ...

    2024

  20. [28]

    Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, and Ming Zhou. 2020. LayoutLM: Pre-training of text and layout for document image understanding. InProceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1192–1200

    2020

  21. [29]

    Yuanshun Yao, Xiaojun Xu, and Yang Liu. 2024. Large language model unlearning. Advances in Neural Information Processing Systems37 (2024), 105425–105475

    2024

  22. [30]

    Mang Ye, Xuankun Rong, Wenke Huang, Bo Du, Nenghai Yu, and Dacheng Tao

  23. [31]

    Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, et al

  24. [32]

    family_name\

    Jie Zhang, Xiangkui Cao, Zhouyu Han, Shiguang Shan, and Xilin Chen. 2024. Multi-PA: A Multi-perspective Benchmark on Privacy Assessment for Large Vision-Language Models.arXiv preprint arXiv:2412.19496(2024). MM ’26, November 10–14, 2026, Rio de Janeiro, Brazil Beining Xu, Hair...

    2024

  25. [2021]

    Differentially private fine-tuning of language models.arXiv preprint arXiv:2110.06500(2021)

    2021 arXiv

  26. [2023]

    Towards unbounded machine unlearning.Advances in Neural Information Processing Systems36 (2023), 1957–1987

    2023

  27. [2025]

    A survey of safety on large vision-language models: Attacks, defenses and evaluations.arXiv preprint arXiv:2502.14881(2025)

    2025 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.