Pith. sign in

REVIEW 3 major objections 5 minor 56 references

UG-UMRE: Uncertainty-Guided Modality Augmentation and Distributional Calibration for Unified Multimodal Relation Extraction

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that replacing fixed feature vectors with Gaussian distributions, plus uncertainty-aware denoising and symmetric KL calibration, lifts unified multimodal relation extraction to new best reported F1 scores on three…

desk verdict Plausible uncertainty-aware MRE modules, but the SOTA claim rests on test-set hyperparameter selection and needs a fair evaluation protocol. read the letter →

arxiv 2608.04949 v1 pith:FP3PULYW submitted 2026-08-05 cs.CV cs.CLcs.ITcs.MMmath.IT

Bo Kong , Liruiz Jia , Yi Liang , Chao Liu , Dongfang Han , Tianwei Yan , Yuan Liu , Shengquan Liu This is my paper · ORCID
classification cs.CVcs.CLcs.ITcs.MMmath.IT
keywords UnifiedMultimodalRelationExtractionUncertainty-GuidedLearningInformationBottleneckModalNoiseProcessingGaussianFeatureModelingContrastiveDistributionalCalibration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

UG-UMRE is a method for Unified Multimodal Relation Extraction (UMRE), the task of extracting relations among textual entities and visual objects in both intra-modal and cross-modal settings. The paper's central claim is that two problems—aleatoric noise inside each modality and the distributional gap between text and image features from heterogeneous encoders—can be addressed by modeling features as Gaussian distributions rather than fixed vectors. It reports that adding its two modules to the REMOTE baseline raises F1 from 67.64 to 69.98 on UMRE, from 62.81 to 66.76 on MORE, and from 87.70 to 89.59 on MNRE, in each case a new state of the art. The authors further claim the modules are plug-and-play, since they also improve FocalMRE without changing its input interface, and that the visual branch benefits more than the textual branch from uncertainty modeling. If right, the lesson is that fusion networks fail less because of their architecture and more because of what they are fed: noisy, uncalibrated unimodal features.

What carries the argument

The central object is the probabilistic reparameterization of unimodal features. The UDUA module parameterizes text and image posteriors $p(Z_T|F'_T)\sim\mathcal N(\mu_T,\sigma_T^2 I)$ and $p(Z_V|F'_V)\sim\mathcal N(\mu_V,\sigma_V^2 I)$ through a Variational Information Bottleneck, with an InfoNCE loss whose positive samples are drawn as $\mu+\sigma\odot\epsilon$ using the reparameterization trick. The JAUA module then penalizes $\tfrac12(D_{KL}(q_T\|q_V)+D_{KL}(q_V\|q_T))$, a symmetric KL whose closed form couples both mean differences and variance ratios. This combination lets the model treat variance as a learned noise budget: high-variance samples are down-weighted in the contrastive objective and pulled toward the stable mean, while the symmetric KL forces disagreement in both location and scale to shrink.

What would settle it

Re-run the UMRE, MORE, and MNRE experiments with hyperparameters tuned exclusively on a held-out validation set, freeze them, and evaluate the test sets once; if UG-UMRE's F1 advantage over REMOTE does not remain near the reported 2.34, 3.95, and 2.25 points, the state-of-the-art claim is not supported.

Watch

Extended reading notes

Core claim

The discovery the authors assert is that prior UMRE models underperform not mainly in the fusion stage but in the unprocessed inputs: noise inside each unimodal representation and mismatched statistical properties across modalities. They claim that representing each sample's holistic semantic content as a multivariate Gaussian—mean as stable core, variance as aleatoric uncertainty—lets a variational information bottleneck strip task-irrelevant noise while an uncertainty-aware InfoNCE loss pulls sampled perturbations toward the mean, and that a symmetric bidirectional KL divergence between text and image posteriors then synchronizes cross-modal distributions. Because the KL is computed between diagonally parameterized Gaussians, it has a closed form, so the calibration is cheap. On this account the resulting shared latent space is why the subsequent multilevel cross-attention and mixture-of-experts fusion works better: the fused features inherit already denoised and aligned inputs.

Load-bearing premise

The claim that UG-UMRE beats the baselines by 2–4 F1 points rests on the assumption that the hyperparameters were chosen on a validation split rather than by scanning the test sets themselves; the paper reports test-set F1 curves in its sensitivity analysis without stating which split was used.

Editorial extensions

If this is right

  • Integrating UDUA and JAUA into the REMOTE baseline yields F1 gains of 2.34, 3.95, and 2.25 points on UMRE, MORE, and MNRE, respectively, reaching new best reported scores.
  • The modules are plug-and-play: they improve FocalMRE as well as REMOTE without changing the input/output interfaces or backbone hyperparameters, at a cost of 2.36M extra parameters and 4.13 ms per sample.
  • Denoising and calibration are complementary: removing either module lowers performance, and the visual branch loses more from removing UDUA than the textual branch, indicating visual noise is the larger obstacle.
  • Global distribution-level pre-calibration is necessary before fine-grained interaction; structural aggregation alone, as in REMOTE, leaves a residual modal gap that the t-SNE visualization shows JAUA closes.
  • Under synthetic noise, UDUA prevents catastrophic collapse, keeping 52.33% F1 at 100% noise where the baseline drops to 35.01%.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same 'denoise, calibrate, fuse' ordering may transfer to other multimodal tasks such as visual question answering, multimodal named entity recognition, or sentiment analysis, where the failure mode of noise propagation plus encoder heterogeneity is structurally similar; the paper does not test this.
  • The reported hyperparameter sensitivity fixes weights by scanning test-set F1 rather than a held-out validation split; if selection was made on the test set, the 2–4 point gains may be optimistic and need confirmation under proper validation.
  • Because JAUA acts only on holistic latent variables while preserving sequence-level features, one could ablate whether calibrating at multiple granularities, rather than just globally, yields further gains; the paper does not explore finer-grained distribution alignment.
  • The robustness to synthetic noise suggests uncertainty estimates could serve as a confidence signal to route hard examples to stronger models or to abstain on low-confidence relations, an extension the paper mentions only as future work.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This manuscript proposes UG-UMRE for unified multimodal relation extraction, with two main modules: UDUA, which models unimodal features as Gaussians via a variational information bottleneck and uses uncertainty-aware contrastive learning for denoising/augmentation, and JAUA, which aligns text and image posteriors through symmetric KL divergence for cross-modal calibration. The method is instantiated on top of the REMOTE baseline, with hierarchical cross-modal attention and an MoE fusion stage, and is evaluated on the UMRE, MORE, and MNRE benchmarks. The paper reports F1 scores of 69.98, 66.76, and 89.59, corresponding to gains of 2.34, 3.95, and 2.25 points over REMOTE, together with ablations, noise-robustness experiments, per-subtask and per-relation analyses, efficiency measurements, and case studies. The core design follows the stated paradigm of denoising first, calibrating second, and fusing last.

Significance. If the reported gains are reproducible, the paper makes a useful empirical contribution by adapting established uncertainty/IB and distribution-alignment techniques to unified multimodal relation extraction, and by demonstrating plug-and-play integration with two base models. The controlled comparison with identical inputs for REMOTE and FocalMRE, the closed-form symmetric KL objective, and the explicit noise-robustness analysis are strengths. I do not see a circularity problem: the proposed objectives are standard regularizers, and the headline numbers are benchmark F1 scores rather than quantities that reduce by construction. However, the evaluation protocol in Section 4.7 does not document a validation split, and all results appear to be single runs with no variance estimates. Since the reported margins over REMOTE are only 2 to 4 F1 points, the new-SOTA claim and the module-level conclusions are not yet established until these evaluation-protocol concerns are resolved. The paper appropriately acknowledges in its conclusion that the fixed loss weights and holistic uncertainty estimates may be less reliable for sparse relations.

major comments (3)
  1. [Section 4.7 and Section 4.1] No validation split is documented for hyperparameter selection. Section 4.7 reports sensitivity sweeps in which F1 values for UMRE, MORE, and MNRE are color-coded, and states that the model achieves optimal F1 at beta1=beta2=1, tau=0.175, lambda1=1e-3, lambda2=1e-5, and lambda3=1e-3; these are exactly the values used in Section 4.1 and for the results in Table 2. The only split explicitly identified in the surrounding experimental sections is the test set (e.g., Sections 4.4 and 4.8). If Figure 4 is computed on the test splits, then selecting the argmax over a six-dimensional hyperparameter grid on the evaluation split and reporting that same split's F1 would introduce optimistic selection bias. The claimed 2.34/3.95/2.25-point gains over REMOTE, and the ablation conclusions in Table 3, would then not be supported. Please specify the validation protocol, choose hyperparameters on a held-out split, and report a single test-set evaluation, or otherwise quantify the effect of any test-set selection.
  2. [Section 4.1, Tables 2 and 3] All experimental results appear to be from single runs, with no standard deviations, seeds, or significance tests. This is important because the method itself has stochastic components, including Gaussian reparameterization sampling in Eq. (4) and negative sampling in Eq. (5), and because several key comparisons are small: the reported gains over REMOTE are 2 to 4 F1 points, and several ablation rows differ by only 0.3 to 0.6 F1 points (for example, Table 3, w/o UDUA (Text) versus w/o UDUA (Image)). Without variance estimates or multiple seeds, it is not possible to determine whether UDUA and JAUA each provide reliable improvements or whether the differences are within run-to-run noise. Please report means and standard deviations over at least several seeds for the main comparisons and ablations.
  3. [Section 4.6] The noise-robustness analysis injects random token masking and Gaussian feature noise as proxies for aleatoric uncertainty. This is a useful sanity check, but it does not directly validate that the learned variance captures the aleatoric uncertainty described in the introduction; the same uncertainty model could also down-weight any randomly perturbed feature. The paper should either present evidence that the predicted variances correlate with the injected noise level or otherwise temper the claim that UDUA specifically filters aleatoric noise rather than acting as a general robust regularizer.
minor comments (5)
  1. [Section 3.3.2] In the paragraph following Eq. (4), the text says the perturbation is derived from the variance in Eq. (4), but Eq. (4) defines the sampling operation; the variance is introduced in the preceding paragraph and Eq. (3). Please correct this cross-reference.
  2. [Section 4.7 and Figure 4] The caption of Figure 4 says the sensitivity analysis is performed on the UMRE dataset, while the text of Section 4.7 says F1 scores for UMRE, MORE, and MNRE are color-coded. Please align the caption with the text.
  3. [Section 4.7] The text says the loss weights lambda1, lambda2, lambda3 appear in Eq. (12), but they are used in the joint objective in Eq. (13). Please correct this equation reference.
  4. [Table 6] The per-relation analysis includes relations with very small test counts, such as /per/misc/race with 2 samples and /org/misc/present_in with 1 sample, and reports F1 swings of +66.67 points on these tiny subsets. These entries are dominated by sampling noise and should either be excluded from the long-tail analysis or accompanied by confidence intervals, so that the long-tail generalization claim is not overstated.
  5. [Section 4.4] The t-SNE visualization is qualitative; the claim that JAUA makes textual and visual features tightly interwoven would be stronger with a quantitative distribution-distance metric, such as MMD or the average KL divergence before and after JAUA.

Circularity Check

1 steps flagged · score 4.0 of 10

Reported SOTA gains are selected on the evaluation split in §4.7; the module derivations themselves are not circular.

  1. fitted input called prediction [Section 4.7 (Parameter Sensitivity), with settings reused in §4.1 and results in Table 2]
    "In the visualization, F1 scores for the UMRE, MORE, and MNRE datasets are color-coded in red, blue, and yellow, respectively. The results indicate that the model achieves optimal F1 scores when β1=β2=1 (Figure 4(a)), τ=0.175 (Figure 4(b)), and λ1=1e−3, λ2=1e−5, λ3=1e−3 (Figure 4(c))."

    Section 4.7 reports F1 on the UMRE/MORE/MNRE datasets for each hyperparameter setting and identifies the values at which 'the model achieves optimal F1.' Those same values are fixed in §4.1 and yield the Table 2 gains (2.34/3.95/2.25). No separate validation split is mentioned, so the reported test F1 is the objective maximized in the sweep; the 'new SOTA' figures are the argmax of that grid, not an independent evaluation. The claim is thus partly forced by test-set selection, though the module losses are not defined in terms of the benchmark scores.

full rationale

The paper's learning objectives (Eqs. 1-13) are standard, externally grounded regularizers: VIB KL compression, InfoNCE-style contrastive loss, and symmetric KL distribution alignment. These are applied to benchmark labels that are not constructed from the model's outputs, so there is no self-definitional reduction of the proposed modules to the target metric. Self-citations in the related work and references ([17,20,21,29,48]) are not load-bearing: they support auxiliary MRE/NER applications, not the core derivation. No uniqueness theorem is invoked, and no known result is merely renamed. The only material circularity concern is the hyperparameter sensitivity protocol in §4.7, where the best F1 settings appear to be selected on the same dataset splits used to report SOTA; if the curves instead used a held-out validation split, this would not be circular. Because the central architecture content remains independent, the overall score is moderate rather than high.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the modeled Gaussian uncertainty distributions and the chosen hyperparameters, but no new physical or conceptual entities are introduced. The assumptions above are the main unproven premises, especially the evaluation protocol.

free parameters (7)
  • β1 (IB loss weight for text) = 1
    Weight of the text KL compression term in Eq. (1); selected via sensitivity analysis in Section 4.7.
  • β2 (IB loss weight for image) = 1
    Weight of the image KL compression term in Eq. (1); selected via sensitivity analysis in Section 4.7.
  • τ (SCL temperature) = 0.175
    Temperature in the InfoNCE-style contrastive loss Eq. (5); tuned in Section 4.7.
  • λ1 (IB loss weight) = 1e-3
    Weight of L_IB in the final objective Eq. (13); tuned in Section 4.7.
  • λ2 (SCL loss weight) = 1e-5
    Weight of L_SCL in the final objective Eq. (13); tuned in Section 4.7.
  • λ3 (JAUA loss weight) = 1e-3
    Weight of L_JAUA in the final objective Eq. (13); tuned in Section 4.7.
  • Latent dimension d_z = not reported
    Dimension of the shared Gaussian latent space for UDUA and JAUA; chosen by hand but its value is not given in the paper.
assumptions (4)
  • domain assumption Features from each modality can be adequately represented as diagonal Gaussian distributions learned by neural encoders.
    Core modeling premise of UDUA (Section 3.3.1); assumes aleatoric uncertainty is Gaussian and per-dimension, which may not capture structured noise.
  • domain assumption The overall relation label Y is a valid target variable U_T and U_V for both text and visual distributions in the information bottleneck objective.
    Eq. (1) sets both modality-specific targets to ground-truth labels; visual objects unrelated to the relation are still pulled toward the global label, potentially adding spurious supervision.
  • domain assumption A validation split exists and hyperparameters were selected on it, not on the test sets.
    No validation split is described; Section 4.7 and Figure 4 plot test-set F1 as a function of hyperparameters, suggesting hyperparameters were chosen using test data.
  • domain assumption The reproduced REMOTE baseline is a faithful reimplementation of the original REMOTE system.
    Table 2 compares against REMOTE marked with a dagger (reproduced by the authors) instead of the original paper's reported numbers; if the reproduction is weaker than the official model, gains are inflated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of UG-UMRE: Uncertainty-Guided Modality Augmentation and Distributional Calibration for Unified Multimodal Relation Extraction." pith.science (2026). https://pith.science/paper/FP3PULYW

@misc{pith2026260804949,
  author       = {Pith},
  title        = {Pith review of: UG-UMRE: Uncertainty-Guided Modality Augmentation and Distributional Calibration for Unified Multimodal Relation Extraction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FP3PULYW}},
  note         = {Machine review of arXiv:2608.04949}
}
read the original abstract

Unified Multimodal Relation Extraction (UMRE) aims to identify intra-modal and cross-modal relations between textual entities and visual objects. However, existing UMRE studies still encounter two critical issues: ignoring inherent aleatoric uncertainty causes noise propagation, and deep-seated heterogeneity between distinct modal distributions hinders alignment. To address these issues, we propose the Uncertainty-Guided UMRE Network (UG-UMRE). Specifically, we design an Uncertainty-Driven Unimodal Augmentation (UDUA) module, which models features as Gaussian distributions based on the Variational Information Bottleneck. By incorporating an uncertainty-aware self-supervised contrastive learning mechanism, UDUA effectively filters out noise while maintaining semantic consistency. Furthermore, we introduce the Joint Aleatoric Uncertainty Alignment (JAUA) module as a global semantic pre-calibration mechanism. JAUA leverages probabilistic distribution consistency to construct a shared latent space, eliminating the distributional gap by synchronizing cross-modal statistical properties, thereby laying a robust foundation for fine-grained interaction. Experiments on three benchmark datasets (UMRE, MORE, and MNRE) demonstrate that UG-UMRE achieves state-of-the-art performance. Further analysis validates the pluggable and effective performance of the proposed UDUA and JAUA modules.

Figures

Figures reproduced from arXiv: 2608.04949 by the authors.

Figure 1
Figure 1. (a) Examples of different multimodal relation ex [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The overall framework of UG-UMRE. 3.3.2 Uncertainty-Aware Self-Supervised Contrastive Learning. Thr￾ough VIB optimization, the obtained mean 𝜇 represents stable core semantics, while the learned variance 𝜎 2 quantifies the aleatoric uncertainty. To utilize this uncertainty for augmentation without introducing meaningless noise, we introduce a reparameterization￾based sampling strategy [19]. Specifically, for the 𝑖-t… view at source ↗
Figure 3
Figure 3. t-SNE visualization of feature distributions. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Sensitivity analysis of hyperparameters on the [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Prediction Comparison for Two Test Samples on [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 49 canonical work pages

  1. [1]

    Alemi, Ian Fischer, Joshua V

    Alexander A. Alemi, Ian Fischer, Joshua V. Dillon, and Kevin Murphy. 2017. Deep Variational Information Bottleneck. InProceedings of the ICLR. OpenReview.net

    work page 2017

  2. [2]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Ming- Hsuan Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. 2025. Qwen2.5-VL Technical...

    arXiv 2025

  3. [3]

    Xiang Chen, Ningyu Zhang, Lei Li, Shumin Deng, Chuanqi Tan, Changliang Xu, Fei Huang, Luo Si, and Huajun Chen. 2022. Hybrid Transformer with Multi-level Fusion for Multimodal Knowledge Graph Completion. InProceedings of the SIGIR. ACM, 904–915

    work page 2022

  4. [4]

    Shiyao Cui, Jiangxia Cao, Xin Cong, Jiawei Sheng, Quangang Li, Tingwen Liu, and Jinqiao Shi. 2024. Enhancing Multimodal Entity and Relation Extraction With Variational Information Bottleneck.IEEE/ACM Transactions on Audio, Speech, and Language Processing32 (2024), 1274–1285

    work page 2024

  5. [5]

    Marco Cuturi. 2013. Sinkhorn Distances: Lightspeed Computation of Optimal Transport. InProceedings of the NeurIPS. 2292–2300

    work page 2013

  6. [6]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the NAACL. Association for Computational Linguistics, 4171–4186

    work page 2019

  7. [7]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. InProceedings of the ICLR. OpenReview.net

    work page 2021

  8. [8]

    Junyu Gao, Mengyuan Chen, and Changsheng Xu. 2023. Collecting Cross-Modal Presence-Absence Evidence for Weakly-Supervised Audio-Visual Event Percep- tion. InProceedings of the CVPR. IEEE, 18827–18836

    work page 2023

Show all 56 references
  1. [9]

    Zixian Gao, Disen Hu, Xun Jiang, Huimin Lu, Heng Tao Shen, and Xing Xu. 2024. Enhanced Experts with Uncertainty-Aware Routing for Multimodal Sentiment Analysis. InProceedings of the MM. ACM, 9650–9659

    2024

  2. [10]

    Zixian Gao, Xun Jiang, Xing Xu, Fumin Shen, Yujie Li, and Heng Tao Shen. 2024. Embracing Unimodal Aleatoric Uncertainty for Robust Multimodal Fusion. In Proceedings of the CVPR. IEEE, 26866–26875

    2024

  3. [11]

    Liang He, Hongke Wang, Yongchang Cao, Zhen Wu, Jianbing Zhang, and Xinyu Dai. 2023. MORE: A Multimodal Object-Entity Relation Extraction Dataset with a Benchmark Evaluation. InProceedings of the MM. ACM, 4564–4573

    2023

  4. [12]

    Liang He, Hongke Wang, Zhen Wu, Jianbing Zhang, Xinyu Dai, and Jiajun Chen

  5. [13]

    Xinyu He, Shixin Li, Yuning Zhang, Binhe Li, Sifan Xu, and Yuqing Zhou

  6. [14]

    Hinton and Sam T

    Geoffrey E. Hinton and Sam T. Roweis. 2002. Stochastic Neighbor Embedding. In Proceedings of the NeurIPS. MIT Press, 833–840

    2002

  7. [15]

    Wenti Huang, Jiayi Chen, Junjie Li, Yiyu Mao, and Ningyi Mao. 2025. ES- MRE: Evidence subgraph enhanced reasoning for multimodal relation extraction. Knowledge-Based Systems325 (2025), 113770

    2025

  8. [16]

    Yatai Ji, Junjie Wang, Yuan Gong, Lin Zhang, Yanru Zhu, Hongfa Wang, Jiaxing Zhang, Tetsuya Sakai, and Yujiu Yang. 2023. MAP: Multimodal Uncertainty- Aware Vision-Language Pre-training Model. InProceedings of the CVPR. IEEE, 23262–23271

    2023

  9. [17]

    Liruizhi Jia, Shengquan Liu, Bo Kong, and Yuan Liu. 2025. REIA: Entity Relation Extraction Based on Interaction Policy and Data Augmentation. InProceedings of the IJCNN. IEEE, 1–8. doi:10.1109/IJCNN64981.2025.11227659

    2025

  10. [18]

    Abdelwahed Khamis, Russell Tsuchida, Mohamed Tarek, Vivien Rolland, and Lars Petersson. 2024. Scalable Optimal Transport Methods in Machine Learning: A Contemporary Survey.IEEE Transactions on Pattern Analysis and Machine Intelligence(2024), 1–20

    2024

  11. [19]

    Kingma and Max Welling

    Diederik P. Kingma and Max Welling. 2014. Auto-Encoding Variational Bayes. In Proceedings of the ICLR

    2014

  12. [20]

    Bo Kong, Shengquan Liu, Liang He, Liruizhi Jia, and Yi Liang. 2024. CSMA-CNER: Multi-modal Chinese NER Task with Cross- and Self-Modality Attention. In Proceedings of the ICME. IEEE, 1–6. doi:10.1109/ICME57554.2024.10688285

    2024

  13. [21]

    Bo Kong, Shengquan Liu, Liruizhi Jia, Yi Liang, Dongfang Han, and Xu Zhang

  14. [22]

    Zhiqiang Kou, Junyang Chen, Xin-Qiang Cai, Xiaobo Xia, Ming-Kun Xie, Dong- Dong Wu, Biao Liu, Yuheng Jia, Xin Geng, Masashi Sugiyama, and Tat-Seng Chua

  15. [23]

    Zhiqiang Kou, Si Qin, Hailin Wang, Jing Wang, Mingkun Xie, Shuo Chen, Yuheng Jia, Tongliang Liu, Masashi Sugiyama, and Xin Geng. 2025. Label Distribution Learning with Biased Annotations Assisted by Multi-Label Learning. InProceed- ings of the IJCAI

    2025

  16. [24]

    Zhiqiang Kou, Jing Wang, Yuheng Jia, Biao Liu, and Xin Geng. 2025. Instance- Dependent Inaccurate Label Distribution Learning.IEEE Transactions on Neural Networks and Learning Systems36, 1 (2025), 1425–1437. doi:10.1109/TNNLS.2023. 3329870

    2025 doi

  17. [25]

    Zhiqiang Kou, Jing Wang, Jiawei Tang, Yuheng Jia, Boyu Shi, and Xin Geng. 2024. Exploiting Multi-Label Correlation in Label Distribution Learning. InProceedings of the IJCAI. 4326–4334

    2024

  18. [26]

    Positive-Unlabeled Reinforcement Learning Distillation for On-Premise Small Models.arXiv preprint arXiv:2601.20687(2026)

    2026

  19. [27]

    Zhiqiang Kou, Yucheng Xie, Hailin Wang, Jing Wang, Mingkun Xie, Shuo Chen, Yuheng Jia, Tongliang Liu, and Xin Geng. 2025. RankMatch: A Novel Approach to Semi-Supervised Label Distribution Learning Leveraging Rank Correlation between Labels. InProceedings of the NeurIPS

    2025

  20. [28]

    Lei Li, Xiang Chen, Shuofei Qiao, Feiyu Xiong, Huajun Chen, and Ningyu Zhang

  21. [29]

    Yi Liang, Turdi Tohti, Wenpeng Hu, Tianwei Yan, Shaohuang Wang, and Askar Hamdulla. 2026. LLaMA-MoT: A Cost-Effective Framework for Visual-Linguistic Instruction Tuning Based on Multi-Head Adapters and Chain-of-Thought.Expert Systems with Applications297 (2026), 129250. doi:10...

    2026

  22. [30]

    Zhiqiang Kou, Junxiang Wu, Wenke Huang, Wenwen He, Ming-Kun Xie, Chang- wei Wang, Yuheng Jia, Di Jiang, Yang Liu, Xin Geng, and Qiang Yang. 2026. FedHarmony: Harmonizing Heterogeneous Label Correlations in Federated Multi- Label Learning. InProceedings of the CVPR. IEEE

    2026

  23. [31]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual Instruc- tion Tuning. InProceedings of the NeurIPS

    2023

  24. [32]

    Xiyang Liu, Chunming Hu, Richong Zhang, Kai Sun, Samuel Mensah, and Yongyi Mao. 2024. Multimodal Relation Extraction via a Mixture of Hierarchical Visual Context Learners. InProceedings of the WWW. ACM, 4283–4294

    2024

  25. [33]

    Llama Team. 2024. The Llama 3 Herd of Models.arXiv preprint arXiv:2407.21783 (2024)

    2024 arXiv

  26. [34]

    Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regularization. InProceedings of the ICLR. OpenReview.net

    2019

  27. [35]

    Xinkui Lin, Yongxiu Xu, Minghao Tang, Shilong Zhang, Hongbo Xu, Hao Xu, and Yubin Wang. 2025. REMOTE: A Unified Multimodal Relation Extraction Framework with Multilevel Optimal Transport and Mixture-of-Experts.arXiv preprint arXiv:2509.04844(2025)

    2025 arXiv

  28. [36]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-Resolution Image Synthesis with Latent Diffusion Models. InProceedings of the CVPR. IEEE, 10674–10685

    2022

  29. [37]

    Lin Sun, Kai Zhang, Qingyuan Li, and Renze Lou. 2024. UMIE: Unified Multimodal Information Extraction with Instruction Tuning. InProceedings of the AAAI. AAAI Press, 19062–19070

    2024

  30. [38]

    Jielong Tang, Yang Yang, Jianxing Yu, Zhen-Xing Wang, Haoyuan Liang, Liang Yao, and Jian Yin. 2025. UnCo: Uncertainty-Driven Collaborative Framework of Large and Small Models for Grounded Multimodal NER. InProceedings of the EMNLP. Association for Computational Linguistics, 7633–7651

    2025

  31. [39]

    Naftali Tishby and Noga Zaslavsky. 2015. Deep learning and the information bottleneck principle. InProceedings of the ITW. IEEE, 1–5

    2015

  32. [40]

    Sina Moradi. 2025. A Survey on Algorithmic Developments in Optimal Transport Problem with Applications.arXiv preprint arXiv:2501.06247(2025)

    2025 arXiv

  33. [41]

    Pengfei Wei, Zhaokang Huang, Hongjun Ouyang, Qintai Hu, Bi Zeng, and Guang Feng. 2024. CGI-MRE: A Comprehensive Genetic-Inspired Model For Multimodal Relation Extraction. InProceedings of the ICMR. ACM, 524–532

    2024

  34. [42]

    Junxiang Wu, Zhiqiang Kou, Hongwei Zeng, Wenke Huang, Biao Liu, Hanlin Gu, Yuheng Jia, Di Jiang, Yang Liu, and Xin Geng. 2026. Trustworthy Federated Label Distribution Learning under Annotation Quality Disparity.arXiv preprint arXiv:2605.04827(2026)

    2026 arXiv

  35. [43]

    Shengqiong Wu, Hao Fei, Yixin Cao, Lidong Bing, and Tat-Seng Chua. 2023. Information Screening whilst Exploiting! Multimodal Relation Extraction with Feature Denoising and Multimodal Topic Modeling. InProceedings of the ACL. Association for Computational Linguistics, 14734–147...

    2023

  36. [44]

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. 2024. Depth Anything V2. InProceedings of the NeurIPS

    2024

  37. [45]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024. Qwen2-VL: Enhancing Vision-Language Mode...

    2024 arXiv

  38. [46]

    Li Yuan, Yi Cai, Xudong Shen, Qing Li, Qingbao Huang, Zikun Deng, and Tao Wang. 2025. Collaborative Multi-LoRA Experts with Achievement-based Multi- Tasks Loss for Unified Multimodal Information Extraction. InProceedings of the IJCAI. 6940–6948

    2025

  39. [47]

    Zefan Zhang, Weiqi Zhang, Yanhui Li, and Tian Bai. 2024. Caption-Aware Multi- modal Relation Extraction with Mutual Information Maximization. InProceedings of the MM. ACM, 1148–1157

    2024

  40. [48]

    Changpeng Zhao, Dongfang Han, Zicheng Zuo, and Turdi Tohti. 2025. KGDB- DDI: Knowledge Graph-Based Drug Background Data Fusion Model for Drug- Drug Interaction Prediction.Artificial Intelligence in Medicine168 (2025), 103225. doi:10.1016/j.artmed.2025.103225

    2025

  41. [49]

    Changmeng Zheng, Junhao Feng, Yi Cai, Xiaoyong Wei, and Qing Li. 2023. Rethinking Multimodal Entity and Relation Extraction from a Translation Point of View. InProceedings of the ACL. Association for Computational Linguistics, 6810–6824

    2023

  42. [50]

    Yaodong Yu, Tianzhe Chu, Shengbang Tong, Ziyang Wu, Druv Pai, Sam Buchanan, and Yi Ma. 2024. Emergence of Segmentation with Minimalistic White-Box Transformers. InProceedings of the CPAL. PMLR, 72–93

    2024

  43. [51]

    Changmeng Zheng, Zhiwei Wu, Junhao Feng, Ze Fu, and Yi Cai. 2021. MNRE: A Challenge Multimodal Dataset for Neural Relation Extraction with Visual Evidence in Social Media Posts. InProceedings of the ICME. IEEE, 1–6

    2021

  44. [55]

    Changmeng Zheng, Junhao Feng, Ze Fu, Yi Cai, Qing Li, and Tao Wang. 2021. Multimodal Relation Extraction with Efficient Graph Alignment. InProceedings of the MM. ACM, 5298–5306

    2021

  45. [2023]

    InProceedings of the AAAI

    On Analyzing the Role of Image for Visual-Enhanced Relation Extraction (Student Abstract). InProceedings of the AAAI. AAAI Press, 16254–16255

  46. [2024]

    InProceedings of the MM

    Focus & Gating: A Multimodal Approach for Unveiling Relations in Noisy Social Media. InProceedings of the MM. ACM, 1379–1388

  47. [2025]

    The more quality information the better: Hierarchical generation of multi- evidence alignment and fusion model for multimodal entity and relation extrac- tion.Information Processing & Management62, 1 (2025), 103875

    2025

  48. [2026]

    doi:10.1016/j.neunet.2025.108106

    MINIGE-MNER: A Multi-Stage Interaction Network Inspired by Gene Editing for Multimodal Named Entity Recognition.Neural Networks194 (2026), 108106. doi:10.1016/j.neunet.2025.108106

    2026

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.