RECORD
DIAR: Deep Image Alignment and Reconstruction using Swin Transformers
Computer Vision and Pattern Recognition · Submitted 17 Oct 2023
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim The attention maps enable the model to detect relevant image content and differentiate it from outliers and artifacts.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
When taking images of some occluded content, one is often faced with the problem that every individual image frame contains unwanted artifacts, but a collection of images contains all relevant information if properly aligned and aggregated. In this paper, we attempt to build a deep learning pipeline that simultaneously aligns a sequence of distorted images and reconstructs them. We create a dataset that contains images with image distortions, such as lighting, specularities, shadows, and occlusion. We create perspective distortions with corresponding ground-truth homographies as labels. We use our dataset to train Swin transformer models to analyze sequential image data. The attention maps enable the model to detect relevant image content and differentiate it from outliers and artifacts. We further explore using neural feature maps as alternatives to classical key point detectors. The feature maps of trained convolutional layers provide dense image descriptors that can be used to find point correspondences between images. We utilize this to compute coarse image alignments and explore its limitations.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | When taking images of some occluded content, one is often faced with the problem that every individual image frame contains unwanted artifacts, but a collection of images contains all relevant information if properly aligned and aggregated. | background 1.00 | 0.07 | 0.04 | asserted 0.87 |
| s1 | In this paper, we attempt to build a deep learning pipeline that simultaneously aligns a sequence of distorted images and reconstructs them. | contribution 0.98 | 0.82 | 0.08 | asserted 0.63 |
| s2 | We create a dataset that contains images with image distortions, such as lighting, specularities, shadows, and occlusion. | method 0.67 | 0.95 | 0.02 | asserted 0.99 |
| s3 | We create perspective distortions with corresponding ground-truth homographies as labels. | method 0.70 | 0.95 | 0.02 | asserted 0.99 |
| s4 | We use our dataset to train Swin transformer models to analyze sequential image data. | method 0.98 | 0.94 | 0.03 | asserted 0.99 |
| s5 | The attention maps enable the model to detect relevant image content and differentiate it from outliers and artifacts. | interpretation 0.46 | 0.74 | 0.12 | asserted 0.99 |
| s6 | We further explore using neural feature maps as alternatives to classical key point detectors. | method 0.57 | 0.83 | 0.03 | asserted 0.50 |
| s7 | The feature maps of trained convolutional layers provide dense image descriptors that can be used to find point correspondences between images. | method 0.91 | 0.73 | 0.04 | asserted 0.95 |
| s8 | We utilize this to compute coarse image alignments and explore its limitations. | method 0.77 | 0.93 | 0.05 | asserted 0.87 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
When taking images of some occluded content, one is often faced with the problem that every individual image frame contains unwanted artifacts, but a collection of images contains all relevant information if properly aligned and aggregated. In this paper, we attempt to build a deep learning pipeline that simultaneously aligns a sequence of distorted images and reconstructs them. We create a dataset that contains images with image distortions, such as lighting, specularities, shadows, and occlusion. We create perspective distortions with corresponding ground-truth homographies as labels. We use our dataset to train Swin transformer models to analyze sequential image data. The attention maps enable the model to detect relevant image content and differentiate it from outliers and artifacts. We further explore using neural feature maps as alternatives to classical key point detectors. The feature maps of trained convolutional layers provide dense image descriptors that can be used to find point correspondences between images. We utilize this to compute coarse image alignments and explore its limitations.
Human review
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites · 4 sentences without a confident label
Cites as background 1 paper
-
have become the state-of-the-art model for sequential data [ref] .
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding · (Vision) Transformers · 0.90
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.