Pith. sign in

REVIEW 2 major objections 5 minor 65 references

Evita is a single shared backbone that fuses RGB frames and event streams by fixing geometric parallax and spectral mismatch inside every layer, setting new accuracy records at lower cost.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Evita, a unified RGB-Event backbone with geometric rectification, spectral resonance, and transient routing, plus N-ImageNetV2 pretraining, reports SOTA dense parsing with better accuracy-latency trade-offs.

T0 review reviewed 2026-07-13 challenge →

load-bearing objection Solid engineering paper: first real RGB-Event unified backbone with clean SOTA numbers and a useful pretraining set; registration quality of N-ImageNetV2 is the main unquantified premise, not a collapse of the claim. the 2 major comments →

arxiv 2607.09143 v1 pith:PQQ5ZH5I submitted 2026-07-10 cs.CV

Weaving Light and Time: Unified Harmonic-Geometric Representation Learning for Dense RGB-Event Parsing

Chenxu Peng , Chongtian zhou , Dicheng Liu , Bo-Wen Yin , Yimian Dai , Xialei Liu , Ming-Ming Cheng , Xiang Li This is my paper
classification cs.CV
keywords RGB-Event fusionunified multimodal backbonegeometric parallax rectificationharmonic spectral resonanceevent camerassemantic segmentationN-ImageNetV2dense prediction
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RGB cameras give rich texture but fail under motion and extreme light; event cameras give microsecond kinematics but almost no static detail. Dual-encoder fusion systems double the compute and still ignore the physical gaps between the two signals: small spatial offsets from asynchronous sensors, and the fact that absolute intensity and temporal contrast live in different frequency regimes. Evita is a single intertwined hierarchical backbone whose every block contains three co-learning modules that attack those gaps directly: a deformable Geometric Parallax Rectifier that warps events onto RGB structure, a Harmonic Spectral Resonance operator that transfers texture only in the complex Fourier domain, and a Transient Global Routing layer that lets event kinematics drive asymmetric attention over the RGB context. The network is pretrained on a newly registered 1.2-million-pair dataset (N-ImageNetV2) with deliberate stochastic misalignment and mixed event encodings, so the same weights later accept arbitrary event formats. On DELIVER, DDD17 and DSEC the largest variant reaches 59.57 %, 80.12 % and 76.80 % mIoU respectively while using roughly half the FLOPs of previous leaders, and the tiniest variant runs in real time on edge hardware.

Core claim

A unified encoder that embeds geometric rectification, frequency-domain texture transfer and event-driven asymmetric attention into every stage can extract invariant RGB-Event features that outperform both dual-stream and generic unified models, provided the network is pretrained on strictly registered pairs with controlled misalignment and mixed event representations.

What carries the argument

The Evita block: three parallel co-learning modules (Geometric Parallax Rectification via bounded deformable sampling, Harmonic Spectral Resonance via gated amplitude fusion in the Fourier domain, and Transient Global Routing with an additive event-derived attention bias) that jointly update dual-stream features at every hierarchical stage.

Load-bearing premise

The SuperPoint-LightGlue-RANSAC registration pipeline (with occasional human fallback) produces geometric ground truth accurate enough that the learned offsets transfer to real sensor parallax rather than merely fitting residual registration errors of the new dataset.

What would settle it

Replace the registered N-ImageNetV2 pairs with deliberately uncorrected N-ImageNet pairs (or with real multi-camera RGB-Event streams whose known physical baseline is larger than the training offsets) and measure whether the reported mIoU gains on DDD17 and DSEC collapse or remain.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The manuscript introduces Evita, claimed as the first unified backbone for dense RGB-Event semantic segmentation. It embeds three co-learning modules in every encoder stage: Geometric Parallax Rectification (cross-modal deformable alignment, Eqs. 1–2), Harmonic Spectral Resonance (amplitude-phase fusion in the Fourier domain, Eqs. 3–5), and Transient Global Routing (event-query asymmetric attention with additive kinematic bias, Eq. 6). To support pretraining, the authors construct N-ImageNetV2 (~1.2 M pairs) via SuperPoint+LightGlue+RANSAC registration with human DLT fallback (Algorithm 1) and train with stochastic event-representation mixing plus a hybrid geometric coherence loss (Eq. 8) that randomly injects misalignment. On DELIVER, DDD17 and DSEC the largest model reports new SOTA mIoU (59.57 %, 80.12 %, 76.80 %) at lower FLOPs/latency than dual-encoder and prior unified baselines; ablations isolate each module and the 0.6 alignment probability.

Significance. If the claims hold, the work supplies a practical, parameter-efficient alternative to dual-stream RGB-Event pipelines and demonstrates that explicit geometric and spectral operators can be baked into a shared hierarchical backbone. The public code, multi-scale model family (P–L), latency numbers on a single 3090, and transfer experiments to RGB-Thermal/LiDAR are concrete engineering contributions. The stochastic representation-mixing protocol is a useful, reusable idea for event-based pretraining. These strengths make the paper of clear interest to the multimodal and event-vision communities even if residual questions about registration fidelity remain.

major comments (2)
  1. §3.5 / Algorithm 1 / Eq. 8: The central transferability claim for Geometric Parallax Rectification rests on N-ImageNetV2 providing accurate geometric ground truth. The pipeline estimates a single global homography from SAE images; residual error statistics, inlier rates after adaptive threshold decay, and the fraction of human-fallback cases are never reported. Homographies cannot capture depth-dependent parallax or asynchronous temporal offsets that dominate real DDD17/DSEC pairs. Without these diagnostics it is unclear whether the learned offsets Δ and the geometric loss primarily fit residual registration artifacts rather than genuine sensor misalignment; the near-zero Δ-drop in Table 8 may therefore overstate real-world robustness. A quantitative registration-quality audit (or an ablation that replaces the learned deformable field by the pre-computed H) is needed to underwrite the cl
  2. Tables 1–2 and §4.2: All SOTA numbers are single-run point estimates with no error bars, multi-seed averages or statistical tests. Given the modest absolute margins (e.g., +0.36 mIoU over CMNeXt-B4 on DELIVER, +0.78 over OmniSegmentor on DDD17) and the free hyper-parameters listed in the pretraining protocol (alignment probability, λ, drop-path schedule), the ranking could shift under ordinary training variance. Reporting mean±std over at least three seeds for the flagship models would make the superiority claim load-bearing rather than suggestive.
minor comments (5)
  1. Abstract and §1: “The code are publicly available” → “The code is …”; several other minor grammatical slips appear throughout.
  2. Fig. 2 and Table 1: FLOPs units are inconsistent (some entries appear to be G, others raw counts); a uniform unit would aid comparison.
  3. Eq. (4): the spectral Gate is defined with AvgPool over A_r only; a short sentence clarifying why event amplitude is not also pooled would improve readability.
  4. Table 4 caption: “OminiSegmentor” is misspelled; consistent naming with the rest of the paper would help.
  5. §5.5: the MFNet/KITTI-360 transfer gains are modest (+1.1 / +1.2 mIoU); a brief discussion of domain-gap limitations would temper the universality claim.

Circularity Check

0 steps flagged

No circularity: empirical SOTA claims and robustness results are measured on external public benchmarks and are not forced by construction from the N-ImageNetV2 registration pipeline or hybrid loss.

full rationale

The paper is an empirical architecture paper whose central claims (new SOTA mIoU on DELIVER/DDD17/DSEC, superior accuracy-latency trade-off, and low degradation under synthetic misalignment) are evaluated exclusively against independent public benchmarks and published baselines. Geometric Parallax Rectification (Eqs. 1-2), Harmonic Spectral Resonance (Eqs. 3-5), Transient Global Routing (Eq. 6), and the hybrid pretraining objective (Eq. 8) are design choices whose parameters are learned by ordinary supervised optimization; none of the reported numbers reduce by algebraic identity or statistical tautology to the SuperPoint+LightGlue+RANSAC homographies of Algorithm 1. Using those homographies as training targets for the deformable offsets is standard supervised learning, not a self-definitional loop that forces the downstream metrics. There are no uniqueness theorems, fitted-then-predicted quantities, or load-bearing self-citations that close a circular chain. The derivation is therefore self-contained and externally falsifiable.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 2 invented entities

The central empirical claims rest on standard deep-learning practice plus three paper-specific design choices (deformable cross-modal warp, Fourier amplitude gating, additive event bias) and the assumption that the constructed N-ImageNetV2 alignments are accurate enough to teach real geometric robustness. Free parameters are the usual architectural and training hyperparameters; no new physical constants are introduced.

free parameters (5)
  • stochastic alignment probability = 0.6
    Chosen by ablation (peaks at 0.6); controls how often perfect vs. synthetic misalignment is shown during pretraining.
  • geometric coherence loss weight λ
    Balances classification loss against the alignment penalty in Eq. 8; value not numerically reported in the main text.
  • drop-path rates per scale (P/N/T/S/B/L) = 0.0-0.2
    Hand-set schedule 0.0/0.05/0.1/0.1/0.15/0.2 used for regularization.
  • spectral gate MLP weights W1, W2
    Learned parameters that control how much event amplitude is injected into RGB phase (Eq. 4).
  • deformable kernel size K and offset bounds
    Architectural choices that limit the magnitude of Geometric Parallax Rectification via tanh.
axioms (4)
  • domain assumption Homography estimated by SuperPoint+LightGlue+RANSAC (with human fallback) is a sufficient model of residual RGB-event geometric misalignment for pretraining.
    Algorithm 1 and §3.5; underpins both N-ImageNetV2 construction and the geometric loss target.
  • domain assumption Phase of the RGB Fourier transform carries the essential structural topology while event amplitude carries transferable high-frequency change.
    §3.3, citing FDA-style frequency adaptation; justifies the polar fusion formula (Eq. 5).
  • ad hoc to paper Event streams can be randomly projected into a finite set of tensor representations (frame, voxel, SAE, …) without destroying the invariant cross-modal features the network should learn.
    §3.1 representation repository M and stochastic mixing protocol.
  • domain assumption Standard cross-entropy on ImageNet-1K / N-ImageNetV2 classification plus the hybrid geometric penalty yields transferable dense-prediction features.
    §3.5 and §4.1 pretraining recipe.
invented entities (2)
  • Evita block (GPR + HSR + TGR) no independent evidence
    purpose: Joint geometric alignment, frequency-domain texture transfer, and event-driven asymmetric attention inside every encoder stage.
    Core architectural invention of the paper; no independent physical existence outside the network.
  • N-ImageNetV2 no independent evidence
    purpose: Provide 1.2 M strictly aligned RGB-event pairs for pretraining that removes legacy geometric parallax.
    New dataset constructed via Algorithm 1; quality depends on the registration pipeline itself.

reviewed 2026-07-13 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Weaving Light and Time: Unified Harmonic-Geometric Representation Learning for Dense RGB-Event Parsing." pith.science (2026). https://pith.science/paper/PQQ5ZH5I

@misc{pith2026260709143,
  author       = {Pith},
  title        = {Pith review of: Weaving Light and Time: Unified Harmonic-Geometric Representation Learning for Dense RGB-Event Parsing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PQQ5ZH5I}},
  note         = {Machine review of arXiv:2607.09143}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Fusing standard RGB frames with asynchronous event streams has emerged as a definitive paradigm for robust perception in degraded environments. Although unified backbones have recently gained traction in multi-modal vision, adapting them to the RGB-Event domain remains fundamentally challenging. Existing architectures either resort to decoupled dual encoders that double computational overhead, or adopt generic unified designs that fail to resolve implicit geometric parallax and cross-spectral aliasing under the extreme representational divide between dense intensity grids and sparse kinematic spikes. To transcend these bottlenecks, we present Evita, the first unified backbone specifically engineered for dedicated dense RGB-Event parsing. To achieve profound modal synergy, Evita explicitly embeds a suite of intrinsic co-learning modules directly into every encoder layer. Specifically, it features Geometric Parallax Rectification for adaptive spatial alignment, Harmonic Spectral Resonance for texture transfer exclusively in the complex frequency domain, and Transient Global Routing for event-driven asymmetric attention. To guarantee robust feature extraction against spatial misalignments and decouple representations from specific event encodings, we construct N-ImageNetV2 alongside a stochastic event representation mixing pretraining protocol, empowering the network to seamlessly accommodate arbitrary event formats in downstream tasks. Extensive evaluations across the DELIVER, DDD17, and DSEC benchmarks confirm that Evita establishes new state-of-the-art metrics while delivering a superior accuracy-latency trade-off for real-time multimodal perception.The code are publicly available at: https://github.com/chaineypung/Evita.

Figures

Figures reproduced from arXiv: 2607.09143 by Bo-Wen Yin, Chenxu Peng, Chongtian zhou, Dicheng Liu, Ming-Ming Cheng, Xialei Liu, Xiang Li, Yimian Dai.

Figure 1
Figure 1. Figure 1: Three core contributions of our work. (a) [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Performance versus computational cost on the DELIVER [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the proposed Evita framework. (a) The overall symmetric intertwined hierarchical backbone. (b) The detailed structure of the Evita block, enabling joint modality optimization. (c) The parallel routing mechanism integrating (d) Geometric Parallax Rectification and (e) Harmonic Spectral Resonance to achieve robust cross-modal feature alignment and structural frequency injection. The Transient Glo… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative comparison on the DELIVER dataset under diverse adverse weather conditions. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: The ablation study on stochastic misalignment probabil [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Visualizations of the Geometric Parallax Rectification. [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Effectiveness of the Transient Global Routing. By lever [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

65 extracted references · 4 linked inside Pith

  1. [1]

    Vatt: Transformers for multimodal self-supervised learning from raw video, audio and text.Advances in neural information processing systems, 34:24206–24221, 2021

    Hassan Akbari, Liangzhe Yuan, Rui Qian, Wei-Hong Chuang, Shih-Fu Chang, Yin Cui, and Boqing Gong. Vatt: Transformers for multimodal self-supervised learning from raw video, audio and text.Advances in neural information processing systems, 34:24206–24221, 2021

    2021

  2. [2]

    Ev-segnet: Semantic segmentation for event-based cameras

    Inigo Alonso and Ana C Murillo. Ev-segnet: Semantic segmentation for event-based cameras. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 0–0, 2019

    2019

  3. [3]

    Multimae: Multi-modal multi-task masked autoen- coders

    Roman Bachmann, David Mizrahi, Andrei Atanov, and Amir Zamir. Multimae: Multi-modal multi-task masked autoen- coders. InEuropean conference on computer vision, pages 348–367. Springer, 2022

    2022

  4. [4]

    Re-coding for uncertainties: Edge-awareness semantic concordance for re- silient event-rgb segmentation.Advances in Neural Informa- tion Processing Systems, 38:101270–101298, 2026

    Nan Bao, Yifan Zhao, Lin Zhu, and Jia Li. Re-coding for uncertainties: Edge-awareness semantic concordance for re- silient event-rgb segmentation.Advances in Neural Informa- tion Processing Systems, 38:101270–101298, 2026

    2026

  5. [5]

    Event-based visual flow.IEEE transactions on neural networks and learning systems, 25(2): 407–417, 2013

    Ryad Benosman, Charles Clercq, Xavier Lagorce, Sio-Hoi Ieng, and Chiara Bartolozzi. Event-based visual flow.IEEE transactions on neural networks and learning systems, 25(2): 407–417, 2013

    2013

  6. [6]

    Hrfuser: A multi-resolution sensor fusion archi- tecture for 2d object detection

    Tim Broedermann, Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Hrfuser: A multi-resolution sensor fusion archi- tecture for 2d object detection. In2023 IEEE 26th Inter- national Conference on Intelligent Transportation Systems (ITSC), pages 4159–4166. IEEE, 2023

    2023

  7. [7]

    Accurate event camera calibration with fourier trans- form.IEEE Transactions on Instrumentation and Measure- ment, 73:1–12, 2024

    Bolin Cai, Ami Zi, Jun Yang, Guoliang Li, Yang Zhang, Qiujie Wu, Chenen Tong, Wenxiang Liu, and Xiangcheng Chen. Accurate event camera calibration with fourier trans- form.IEEE Transactions on Instrumentation and Measure- ment, 73:1–12, 2024

    2024

  8. [8]

    Evrwkv: A continuous interactive rwkv frame- work for effective event-guided low-light image enhance- ment.IEEE Transactions on Circuits and Systems for Video Technology, 2026

    Wenjie Cai, Qingguo Meng, Zhenyu Wang, Xingbo Dong, and Zhe Jin. Evrwkv: A continuous interactive rwkv frame- work for effective event-guided low-light image enhance- ment.IEEE Transactions on Circuits and Systems for Video Technology, 2026

    2026

  9. [9]

    Explore and exploit the diverse knowledge in model zoo for domain generalization

    Yimeng Chen, Tianyang Hu, Fengwei Zhou, Zhenguo Li, and Zhi-Ming Ma. Explore and exploit the diverse knowledge in model zoo for domain generalization. InInternational Conference on Machine Learning, pages 4623–4640. PMLR, 2023

    2023

  10. [10]

    Rethinking coarse-to-fine approach in sin- gle image deblurring

    Sung-Jin Cho, Seo-Won Ji, Jun-Pyo Hong, Seung-Won Jung, and Sung-Jea Ko. Rethinking coarse-to-fine approach in sin- gle image deblurring. InProceedings of the IEEE/CVF inter- national conference on computer vision, pages 4641–4650, 2021. 11

    2021

  11. [11]

    Halsie: Hybrid ap- proach to learning segmentation by simultaneously exploit- ing image and event modalities

    Shristi Das Biswas, Adarsh Kosta, Chamika Liyanagedera, Marco Apolinario, and Kaushik Roy. Halsie: Hybrid ap- proach to learning segmentation by simultaneously exploit- ing image and event modalities. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 5964–5974, 2024

    2024

  12. [12]

    Superpoint: Self-supervised interest point detection and description

    Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabi- novich. Superpoint: Self-supervised interest point detection and description. InProceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 224–236, 2018

    2018

  13. [13]

    Event-based vision: A survey.IEEE transactions on pattern analysis and machine intelligence, 44(1):154–180, 2020

    Guillermo Gallego, Tobi Delbr¨ uck, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew J Davison, J ¨org Conradt, Kostas Daniilidis, et al. Event-based vision: A survey.IEEE transactions on pattern analysis and machine intelligence, 44(1):154–180, 2020

    2020

  14. [14]

    Pushing the lim- its of asynchronous graph-based object detection with event cameras.arXiv preprint arXiv:2211.12324, 2022

    Daniel Gehrig and Davide Scaramuzza. Pushing the lim- its of asynchronous graph-based object detection with event cameras.arXiv preprint arXiv:2211.12324, 2022

    Pith/arXiv arXiv 2022

  15. [15]

    Dsec: A stereo event camera dataset for driv- ing scenarios.IEEE Robotics and Automation Letters, 6(3): 4947–4954, 2021

    Mathias Gehrig, Willem Aarents, Daniel Gehrig, and Davide Scaramuzza. Dsec: A stereo event camera dataset for driv- ing scenarios.IEEE Robotics and Automation Letters, 6(3): 4947–4954, 2021

    2021

  16. [16]

    Omnivore: A single model for many visual modalities

    Rohit Girdhar, Mannat Singh, Nikhila Ravi, Laurens Van Der Maaten, Armand Joulin, and Ishan Misra. Omnivore: A single model for many visual modalities. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16102–16112, 2022

    2022

  17. [17]

    Mambaseg: Harness- ing mamba for accurate and efficient image-event semantic segmentation

    Fuqiang Gu, Yuanke Li, Xianlei Long, Kangping Ji, Chao Chen, Qingyi Gu, and Zhenliang Ni. Mambaseg: Harness- ing mamba for accurate and efficient image-event semantic segmentation. InProceedings of the AAAI Conference on Artificial Intelligence, pages 4302–4310, 2026

    2026

  18. [18]

    Zero-reference deep curve estimation for low-light image enhancement

    Chunle Guo, Chongyi Li, Jichang Guo, Chen Change Loy, Junhui Hou, Sam Kwong, and Runmin Cong. Zero-reference deep curve estimation for low-light image enhancement. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1780–1789, 2020

    2020

  19. [19]

    Segnext: Rethink- ing convolutional attention design for semantic segmenta- tion.Advances in neural information processing systems, 35: 1140–1156, 2022

    Meng-Hao Guo, Cheng-Ze Lu, Qibin Hou, Zhengning Liu, Ming-Ming Cheng, and Shi-Min Hu. Segnext: Rethink- ing convolutional attention design for semantic segmenta- tion.Advances in neural information processing systems, 35: 1140–1156, 2022

    2022

  20. [20]

    Xiaodong Guo, Xianda Guo, Tong Liu, Zhihong Deng, Yan- lun Peng, Xiang Li, and Wujie Zhou. Tuni: Unifying pre- training and fine-tuning with modality-aware mutual learn- ing and rectification for rgb-t semantic segmentation.IEEE Transactions on Circuits and Systems for Video Technology, 2026

    2026

  21. [21]

    Mfnet: Towards real-time se- mantic segmentation for autonomous vehicles with multi- spectral scenes

    Qishen Ha, Kohei Watanabe, Takumi Karasawa, Yoshitaka Ushiku, and Tatsuya Harada. Mfnet: Towards real-time se- mantic segmentation for autonomous vehicles with multi- spectral scenes. In2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5108–5115. IEEE, 2017

    2017

  22. [22]

    Learning to deblur and generate high frame rate video with an event camera.arXiv preprint arXiv:2003.00847, 2020

    Chen Haoyu, Teng Minggui, Shi Boxin, Wang YIzhou, and Huang Tiejun. Learning to deblur and generate high frame rate video with an event camera.arXiv preprint arXiv:2003.00847, 2020

    Pith/arXiv arXiv 2003

  23. [23]

    Geminifusion: Efficient pixel- wise multimodal fusion for vision transformer.arXiv preprint arXiv:2406.01210, 2024

    Ding Jia, Jianyuan Guo, Kai Han, Han Wu, Chao Zhang, Chang Xu, and Xinghao Chen. Geminifusion: Efficient pixel- wise multimodal fusion for vision transformer.arXiv preprint arXiv:2406.01210, 2024

    Pith/arXiv arXiv 2024

  24. [24]

    Event-based semantic segmentation with posterior attention.IEEE Transactions on Image Processing, 32:1829–1842, 2023

    Zexi Jia, Kaichao You, Weihua He, Yang Tian, Yongxiang Feng, Yaoyuan Wang, Xu Jia, Yihang Lou, Jingyi Zhang, Guoqi Li, et al. Event-based semantic segmentation with posterior attention.IEEE Transactions on Image Processing, 32:1829–1842, 2023

    2023

  25. [25]

    N-imagenet: Towards robust, fine-grained object recognition with event cameras

    Junho Kim, Jaehyeok Bae, Gangin Park, Dongsu Zhang, and Young Min Kim. N-imagenet: Towards robust, fine-grained object recognition with event cameras. InProceedings of the IEEE/CVF international conference on computer vision, pages 2146–2156, 2021

    2021

  26. [26]

    Stitchfusion: Weaving any visual modalities to enhance multimodal semantic segmentation

    Bingyu Li, Da Zhang, Zhiyuan Zhao, Junyu Gao, and Xue- long Li. Stitchfusion: Weaving any visual modalities to enhance multimodal semantic segmentation. InProceedings of the 33rd ACM International Conference on Multimedia, pages 1308–1317, 2025

    2025

  27. [27]

    Efficient event-based se- mantic segmentation via exploiting frame-event fusion: A hybrid neural network approach

    Hebei Li, Yansong Peng, Jiahui Yuan, Peixi Wu, Jin Wang, Yueyi Zhang, and Xiaoyan Sun. Efficient event-based se- mantic segmentation via exploiting frame-event fusion: A hybrid neural network approach. InProceedings of the AAAI Conference on Artificial Intelligence, pages 18296–18304, 2025

    2025

  28. [28]

    Adaptive com- plex wavelet informed transformer operator.IEEE Transac- tions on Multimedia, 2025

    Xiaotong Li, Licheng Jiao, Fang Liu, Shuyuan Yang, Hao Zhu, Xu Liu, Lingling Li, and Wenping Ma. Adaptive com- plex wavelet informed transformer operator.IEEE Transac- tions on Multimedia, 2025

    2025

  29. [29]

    Towards robust event-guided low-light image en- hancement: a large-scale real-world event-image dataset and novel approach

    Guoqiang Liang, Kanghao Chen, Hangyu Li, Yunfan Lu, and Lin Wang. Towards robust event-guided low-light image en- hancement: a large-scale real-world event-image dataset and novel approach. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23–33, 2024

    2024

  30. [30]

    Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3):3292–3310, 2022

    Yiyi Liao, Jun Xie, and Andreas Geiger. Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3):3292–3310, 2022

    2022

  31. [31]

    Lightglue: Local feature matching at light speed

    Philipp Lindenberger, Paul-Edouard Sarlin, and Marc Polle- feys. Lightglue: Local feature matching at light speed. In Proceedings of the IEEE/CVF international conference on computer vision, pages 17627–17638, 2023

    2023

  32. [32]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021

    2021

  33. [33]

    Xianlei Long, Xiaxin Zhu, Fangming Guo, Chao Chen, Xiangwei Zhu, Fuqiang Gu, Songyu Yuan, and Chunlong Zhang. Spike-brgnet: Efficient and accurate event-based se- mantic segmentation with boundary region-guided spiking neural networks.IEEE Transactions on Circuits and Systems for Video Technology, 35(3):2712–2724, 2024

    2024

  34. [34]

    Sltnet: Effi- 12 cient event-based semantic segmentation with spike-driven lightweight transformer-based networks

    Xianlei Long, Xiaxin Zhu, Fangming Guo, Wanyi Zhang, Qingyi Gu, Chao Chen, and Fuqiang Gu. Sltnet: Effi- 12 cient event-based semantic segmentation with spike-driven lightweight transformer-based networks. In2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4331–4338. IEEE, 2025

    2025

  35. [35]

    Data-driven feature tracking for event cameras

    Nico Messikommer, Carter Fang, Mathias Gehrig, and Da- vide Scaramuzza. Data-driven feature tracking for event cameras. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5642–5651, 2023

    2023

  36. [36]

    Resilient sensor fusion under adverse sensor failures via multi-modal expert fusion

    Konyul Park, Yecheol Kim, Daehun Kim, and Jun Won Choi. Resilient sensor fusion under adverse sensor failures via multi-modal expert fusion. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6720–6729, 2025

    2025

  37. [37]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, pages 8748–8763. PmLR, 2021

    2021

  38. [38]

    Acdc: The adverse conditions dataset with correspondences for semantic driving scene understanding

    Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Acdc: The adverse conditions dataset with correspondences for semantic driving scene understanding. InProceedings of the IEEE/CVF international conference on computer vision, pages 10765–10775, 2021

    2021

  39. [39]

    Efficient rgb-d se- mantic segmentation for indoor scene analysis

    Daniel Seichter, Mona K ¨ohler, Benjamin Lewandowski, Tim Wengefeld, and Horst-Michael Gross. Efficient rgb-d se- mantic segmentation for indoor scene analysis. In2021 IEEE international conference on robotics and automation (ICRA), pages 13525–13531. IEEE, 2021

    2021

  40. [40]

    Reducing the sim-to-real gap for event cameras

    Timo Stoffregen, Cedric Scheerlinck, Davide Scaramuzza, Tom Drummond, Nick Barnes, Lindsay Kleeman, and Robert Mahony. Reducing the sim-to-real gap for event cameras. In European Conference on Computer Vision, pages 534–549. Springer, 2020

    2020

  41. [41]

    Ess: Learning event-based semantic seg- mentation from still images

    Zhaoning Sun, Nico Messikommer, Daniel Gehrig, and Da- vide Scaramuzza. Ess: Learning event-based semantic seg- mentation from still images. InEuropean Conference on Computer Vision, pages 341–357. Springer, 2022

    2022

  42. [42]

    Raft: Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. InEuropean conference on com- puter vision, pages 402–419. Springer, 2020

    2020

  43. [43]

    Time lens: Event-based video frame interpolation

    Stepan Tulyakov, Daniel Gehrig, Stamatios Georgoulis, Julius Erbach, Mathias Gehrig, Yuanyou Li, and Davide Scara- muzza. Time lens: Event-based video frame interpolation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16155–16164, 2021

    2021

  44. [44]

    Event-based high dynamic range image and very high frame rate video generation using conditional generative adversarial networks

    Lin Wang, Yo-Sung Ho, Kuk-Jin Yoon, et al. Event-based high dynamic range image and very high frame rate video generation using conditional generative adversarial networks. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10081–10090, 2019

    2019

  45. [45]

    Multimodal token fusion for vision transformers

    Yikai Wang, Xinghao Chen, Lele Cao, Wenbing Huang, Fuchun Sun, and Yunhe Wang. Multimodal token fusion for vision transformers. InProceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pages 12186–12195, 2022

    2022

  46. [46]

    Generalized differentiable ransac

    Tong Wei, Yash Patel, Alexander Shekhovtsov, Jiri Matas, and Daniel Barath. Generalized differentiable ransac. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17649–17660, 2023

    2023

  47. [47]

    Event- based video reconstruction using transformer

    Wenming Weng, Yueyi Zhang, and Zhiwei Xiong. Event- based video reconstruction using transformer. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 2563–2572, 2021

    2021

  48. [48]

    Eisnet: A multi-modal fusion network for semantic seg- mentation with events and images.IEEE Transactions on Multimedia, 26:8639–8650, 2024

    Bochen Xie, Yongjian Deng, Zhanpeng Shao, and Youfu Li. Eisnet: A multi-modal fusion network for semantic seg- mentation with events and images.IEEE Transactions on Multimedia, 26:8639–8650, 2024

    2024

  49. [49]

    Cross-modal learn- ing for event-based semantic segmentation via attention soft alignment.IEEE Robotics and Automation Letters, 9(3): 2359–2366, 2024

    Chuyun Xie, Wei Gao, and Ren Guo. Cross-modal learn- ing for event-based semantic segmentation via attention soft alignment.IEEE Robotics and Automation Letters, 9(3): 2359–2366, 2024

    2024

  50. [50]

    Segformer: Simple and efficient design for semantic segmentation with transform- ers.Advances in neural information processing systems, 34: 12077–12090, 2021

    Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M Alvarez, and Ping Luo. Segformer: Simple and efficient design for semantic segmentation with transform- ers.Advances in neural information processing systems, 34: 12077–12090, 2021

    2021

  51. [51]

    Fda: Fourier domain adaptation for semantic segmentation

    Yanchao Yang and Stefano Soatto. Fda: Fourier domain adaptation for semantic segmentation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4085–4095, 2020

    2020

  52. [52]

    Sam-event-adapter: Adapting segment anything model for event-rgb semantic segmentation

    Bowen Yao, Yongjian Deng, Yuhan Liu, Hao Chen, Youfu Li, and Zhen Yang. Sam-event-adapter: Adapting segment anything model for event-rgb semantic segmentation. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 9093–9100. IEEE, 2024

    2024

  53. [53]

    Dformer: Rethinking rgbd representation learning for semantic segmentation

    Bowen Yin, Xuying Zhang, Zhong-Yu Li, Li Liu, Ming- Ming Cheng, and Qibin Hou. Dformer: Rethinking rgbd representation learning for semantic segmentation. InIn- ternational Conference on Learning Representations, pages 51803–51825, 2024

    2024

  54. [54]

    Dformerv2: Geometry self-attention for rgbd semantic segmentation

    Bo-Wen Yin, Jiao-Long Cao, Ming-Ming Cheng, and Qibin Hou. Dformerv2: Geometry self-attention for rgbd semantic segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19345– 19355, 2025

    2025

  55. [55]

    Omnisegmentor: a flex- ible multi-modal learning framework for semantic segmen- tation.Advances in Neural Information Processing Systems, 38:142674–142695, 2026

    Bo-Wen Yin, Jiao-Long Cao, Xuying Zhang, Yuming Chen, Ming-Ming Cheng, and Qibin Hou. Omnisegmentor: a flex- ible multi-modal learning framework for semantic segmen- tation.Advances in Neural Information Processing Systems, 38:142674–142695, 2026

    2026

  56. [56]

    Multi-stage progressive image restoration

    Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Shao. Multi-stage progressive image restoration. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14821–14831, 2021

    2021

  57. [57]

    Restormer: Efficient transformer for high-resolution image restoration

    Syed Waqas Zamir, Aditya Arora, Salman Khan, Mu- nawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. Restormer: Efficient transformer for high-resolution image restoration. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5728–5739, 2022

    2022

  58. [58]

    Ex- ploring event-driven dynamic context for accident scene seg- 13 mentation.IEEE Transactions on Intelligent Transportation Systems, 23(3):2606–2622, 2021

    Jiaming Zhang, Kailun Yang, and Rainer Stiefelhagen. Ex- ploring event-driven dynamic context for accident scene seg- 13 mentation.IEEE Transactions on Intelligent Transportation Systems, 23(3):2606–2622, 2021

    2021

  59. [59]

    Cmx: Cross-modal fusion for rgb-x semantic segmentation with transformers.IEEE Transactions on intelligent transportation systems, 24(12): 14679–14694, 2023

    Jiaming Zhang, Huayao Liu, Kailun Yang, Xinxin Hu, Ruip- ing Liu, and Rainer Stiefelhagen. Cmx: Cross-modal fusion for rgb-x semantic segmentation with transformers.IEEE Transactions on intelligent transportation systems, 24(12): 14679–14694, 2023

    2023

  60. [60]

    Delivering arbitrary-modal semantic segmentation

    Jiaming Zhang, Ruiping Liu, Hao Shi, Kailun Yang, Si- mon Reiß, Kunyu Peng, Haodong Fu, Kaiwei Wang, and Rainer Stiefelhagen. Delivering arbitrary-modal semantic segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1136– 1147, 2023

    2023

  61. [61]

    Rui Zhang, Luziwei Leng, Kaiwei Che, Hu Zhang, Jie Cheng, Qinghai Guo, Jianxing Liao, and Ran Cheng. Accurate and efficient event-based semantic segmentation using adaptive spiking encoder–decoder network.IEEE Transactions on Neural Networks and Learning Systems, 36(5):9326–9340, 2024

    2024

  62. [62]

    Unifying motion deblurring and frame interpolation with events

    Xiang Zhang and Lei Yu. Unifying motion deblurring and frame interpolation with events. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17765–17774, 2022

    2022

  63. [63]

    Deep learning for event-based vision: A comprehensive survey and benchmarks.arXiv preprint arXiv:2302.08890, 2023

    Xu Zheng, Yexin Liu, Yunfan Lu, Tongyan Hua, Tianbo Pan, Weiming Zhang, Dacheng Tao, and Lin Wang. Deep learning for event-based vision: A comprehensive survey and benchmarks.arXiv preprint arXiv:2302.08890, 2023

    Pith/arXiv arXiv 2023

  64. [64]

    Event-based stereo visual odometry.IEEE Transactions on Robotics, 37 (5):1433–1450, 2021

    Yi Zhou, Guillermo Gallego, and Shaojie Shen. Event-based stereo visual odometry.IEEE Transactions on Robotics, 37 (5):1433–1450, 2021

    2021

  65. [65]

    De- formable convnets v2: More deformable, better results

    Xizhou Zhu, Han Hu, Stephen Lin, and Jifeng Dai. De- formable convnets v2: More deformable, better results. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9308–9316, 2019. 14

    2019

This paper was first reviewed by grok-4.5 on July 13, 2026.