RECORD
Boosting Video Object Segmentation based on Scale Inconsistency
Computer Vision and Pattern Recognition · Submitted 2 May 2022
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Experiments on DAVIS 16 and DAVIS 17 datasets show that our framework can be generically applied to various VOS models and improve their performance.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
We present a refinement framework to boost the performance of pre-trained semi-supervised video object segmentation (VOS) models. Our work is based on scale inconsistency, which is motivated by the observation that existing VOS models generate inconsistent predictions from input frames with different sizes. We use the scale inconsistency as a clue to devise a pixel-level attention module that aggregates the advantages of the predictions from different-size inputs. The scale inconsistency is also used to regularize the training based on a pixel-level variance measured by an uncertainty estimation. We further present a self-supervised online adaptation, tailored for test-time optimization, that bootstraps the predictions without ground-truth masks based on the scale inconsistency. Experiments on DAVIS 16 and DAVIS 17 datasets show that our framework can be generically applied to various VOS models and improve their performance.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | We present a refinement framework to boost the performance of pre-trained semi-supervised video object segmentation (VOS) models. | contribution 1.00 | 0.92 | 0.16 | asserted 0.99 |
| s1 | Our work is based on scale inconsistency, which is motivated by the observation that existing VOS models generate inconsistent predictions from input frames with different sizes. | contribution 0.39 | 0.25 | 0.06 | asserted 0.99 |
| s2 | We use the scale inconsistency as a clue to devise a pixel-level attention module that aggregates the advantages of the predictions from different-size inputs. | method 0.95 | 0.94 | 0.05 | asserted 1.00 |
| s3 | The scale inconsistency is also used to regularize the training based on a pixel-level variance measured by an uncertainty estimation. | method 0.93 | 0.92 | 0.04 | asserted 1.00 |
| s4 | We further present a self-supervised online adaptation, tailored for test-time optimization, that bootstraps the predictions without ground-truth masks based on the scale inconsistency. | contribution 0.55 | 0.95 | 0.05 | asserted 1.00 |
| s5 | Experiments on DAVIS 16 and DAVIS 17 datasets show that our framework can be generically applied to various VOS models and improve their performance. | result 0.99 | 0.96 | 0.76 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
We present a refinement framework to boost the performance of pre-trained semi-supervised video object segmentation (VOS) models. Our work is based on scale inconsistency, which is motivated by the observation that existing VOS models generate inconsistent predictions from input frames with different sizes. We use the scale inconsistency as a clue to devise a pixel-level attention module that aggregates the advantages of the predictions from different-size inputs. The scale inconsistency is also used to regularize the training based on a pixel-level variance measured by an uncertainty estimation. We further present a self-supervised online adaptation, tailored for test-time optimization, that bootstraps the predictions without ground-truth masks based on the scale inconsistency. Experiments on DAVIS 16 and DAVIS 17 datasets show that our framework can be generically applied to various VOS models and improve their performance.
Human review
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites · 2 sentences without a confident label
Uses data or values from 1 paper
-
We adopt DAVIS 16 [ref] and DAVIS 17 [ref] dataset to evaluate the proposed method.
The 2017 DAVIS Challenge on Video Object Segmentation · Setup · 0.99
Compares with 1 paper
-
By employing the proposed method, OSVOS can outperform all existing online learning-based methods on DAVIS 16 without any post-processing that is widely adopted in existing methods [ref] .
Make One-Shot Video Object Segmentation Efficient Again · Evaluation and Comparison · 0.98
Cites as support 1 paper
-
However, as demonstrated by the works in image segmentation [ref] , simply averaging the predictions cannot effectively handle the scale inconsistency as the magnitude of inconsistency differs from pixel to pixel.
Hierarchical Multi-Scale Attention for Semantic Segmentation · Multi-scale Context Aggregation Module · 0.97
Cites as background 3 papers
-
Semi-supervised VOS [ref] is challenging as the model needs to address appearance changes, similar instances, occlusions, and scale variations based on the mask of the first frame.
The 2017 DAVIS Challenge on Video Object Segmentation · Introduction · 0.94
-
Online learning-based methods fine-tune a model on the ground-truth mask of the first frame at test time [ref] .
Online Adaptation of Convolutional Neural Networks for Video Object Segmentation · Introduction · 0.95
-
Online learning-based methods fine-tune a model on the ground-truth mask of the first frame at test time [ref] .
Make One-Shot Video Object Segmentation Efficient Again · Introduction · 0.95
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.