Pith. sign in

REVIEW 4 major objections 4 minor 47 references

A 150k-parameter segmentation model trained on automatically generated pseudo-labels matches a 217M-parameter model on aerial scenes.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 09:28 UTC pith:35VCHGKB

load-bearing objection VRE is a real, open-source engineering contribution; the paper is worth engaging with, but the headline performance claim is inherited from a sibling paper and the multi-modal attribution is untested. the 4 major comments →

arxiv 2510.14862 v2 pith:35VCHGKB submitted 2025-10-16 cs.CV cs.DC

Multi-modal video data-pipelines for machine learning with minimal human supervision

Mihai-Cristian P\^irvu , Marius Leordeanu This is my paper
classification cs.CV cs.DC
keywords multi-modal learningsemantic segmentationvideo data pipelinepseudo-labelingknowledge distillationUAV scene understandingreal-time inferencemasked autoencoders
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to show that meaningful multi-modal supervision for scene understanding can be produced without human annotation. It introduces Video Representations Extractor (VRE), an open-source pipeline that chains pre-trained experts (Mask2Former for semantics, Marigold for depth) with procedural combinations (camera normals, safe-landing maps, binary classes) over raw videos. Using VRE, it extends the Dronescapes UAV dataset from 23k to 80k frames with 13 modalities. It then trains PHG-MAE, a masked-autoencoder model that natively integrates these modalities, and distills it to a 150k-parameter network. On Dronescapes-Test this tiny model scores 53.32 mean IoU versus 53.97 for Mask2Former (217M parameters), and the small model runs in real time on commodity GPUs. The value would be: high-quality training data and a compact deployable model, both obtainable without manual labeling.

Core claim

The central claim is that an autonomous data-pipeline, VRE, can turn raw videos into a rich multi-modal training set by stacking pre-trained experts and hand-authored procedural rules, and that a model trained on this pseudo-labeled data can be distilled to under a million parameters while retaining accuracy roughly equal to a 300M-class model. Concretely, on the Dronescapes-Test benchmark the 150k-parameter PHG-MAE-Distil reaches 53.32 mean IoU, against 53.97 for the 217M-parameter Mask2Former. The paper also shows this distilled model and a DPT depth model run in streaming mode at real-time rates on a laptop RTX 4050 GPU.

What carries the argument

The load-bearing mechanism is the VRE representation graph: pre-trained neural experts (Mask2Former, Marigold) are treated as nodes, and new modalities are derived procedurally from their outputs (e.g. camera normals via SVD on depth, safe-landing masks via thresholded normals plus semantic classes). The topologically-sorted graph produces many correlated pseudo-label channels for each RGB frame. PHG-MAE, a masked autoencoder that takes these multiple modalities as input and enforces consistency across them, is trained on this data; its ensemble of randomly masked encoders and later distillation into small networks are what compress the multi-modal knowledge into a lightweight semantic segme

Load-bearing premise

The pseudo-labels generated by pre-trained experts on the eight new internet videos are accurate enough to serve as training ground truth; the paper reports no human validation of these labels, so if the experts or the hand-set thresholds are wrong on those scenes, the dataset and the distilled model inherit that error.

What would settle it

Take the eight new videos used for Dronescapes2-M+, have humans annotate even a few hundred frames, and compare VRE's pseudo-labels (Mask2Former semantics, Marigold depth, thresholded safe-landing masks) against the human annotations. If agreement is low for aerial scenes—or if the 150k model's IoU collapses on an independent aerial dataset outside Dronescapes—then the claim that no human supervision suffices is falsified.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Dense scene-understanding training data can be expanded automatically from any raw video, with no human annotation, by composing off-the-shelf experts.
  • A model under 1M parameters can achieve accuracy on a UAV benchmark comparable to a 217M model, at least on Dronescapes-Test.
  • Real-time semantic segmentation, and near-real-time depth estimation, are feasible on a consumer laptop GPU, enabling edge or on-device drone and handheld applications.
  • The same pipeline can be retargeted to other tasks and sensors by adding new representations, since VRE's graph structure is extensible.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If VRE pseudo-labels prove accurate enough, 'data' becomes a reusable asset: one could close the loop by iteratively re-labeling new videos with the distilled model plus the experts, a self-training loop that may further cut human-labeling needs.
  • The claimed '150k equals 217M' result is only demonstrated on Dronescapes-Test, which shares domain and origin with the training distribution; an external aerial benchmark would be the direct test of whether the tiny model actually generalizes.
  • The safe-landing and binary thresholds in the pipeline are hand-authored; they could be learned or calibrated on a small set of human-labeled frames, making the labels verifiable and likely more robust across domains.
  • The reported streaming FPS over TCP with raw RGB frames is likely a lower bound; switching to UDP or compressed video encoding would probably recover much of the 2–3 FPS gap observed in the cloud-processing setup.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper presents Video Representations Extractor (VRE), an open-source data-pipeline for generating multi-modal representations from raw videos with minimal human supervision, supporting batched and streaming modes and multi-GPU scheduling. As a case study, it extends the Dronescapes aerial dataset to Dronescapes2-M+ (80K frames, up to 13/15 modalities) by adding eight internet-sourced videos and using pre-trained experts (Mask2Former, Marigold) plus procedural combinations (SVD normals, thresholded safe-landing maps). It then reports, via the sibling PHG-MAE paper [33], that a 150k-parameter distilled semantic segmentation model reaches 53.32 mean IoU on Dronescapes-Test, close to Mask2Former's 53.97 at 217M parameters. The experimental sections measure throughput for batched exports and FPS for real-time and remote streaming of segmentation and depth models.

Significance. VRE is a credible, reusable engineering contribution: it is open-sourced, has a clean representation graph, re-entrant batching, and multi-GPU strategies, and the paper gives concrete deployment experiments. If the accuracy story is supported, the paper would demonstrate that automatic pseudo-label pipelines plus heavy distillation can make large-model performance accessible in real-time on commodity hardware. However, the current manuscript does not itself train or evaluate the models in Table 2, does not validate pseudo-label quality on the newly added videos, and does not isolate the contribution of VRE-generated non-semantic modalities. The significance is therefore conditional on additional validation experiments.

major comments (4)
  1. [§2.8, Table 2] The headline accuracy claim is inherited from the sibling paper [33] rather than demonstrated here. Table 2 reproduces PHG-MAE results from [33], and the present manuscript reports no training runs or accuracy measurements of its own. As a result, the abstract's statement that the authors 'show' a distilled model is competitive with ~300M-parameter models is not supported by experiments in this paper. Please either include the training/evaluation experiments needed to substantiate the claim or explicitly reframe the contribution as a deployment case study built on [33].
  2. [§2.7, Table 1] Pseudo-label accuracy on the eight new videos is not validated. The Dronescapes2-M+ semantic labels come from three Mask2Former checkpoints and depth from Marigold, with binary maps derived by hand-set thresholds; no agreement statistics against human annotation are reported for the 57K new frames. Because Dronescapes-Test is a small (116 annotated frames) benchmark from the same group, the 'fully-automated' dataset claim and the external validity of downstream results are unestablished. Provide per-modality validation on a human-annotated subset of the new videos, or temper the claims accordingly.
  3. [Table 2, §2.7] The comparison to Mask2Former is partly circular: Mask2Former generates the semantic pseudo-labels used to train the PHG-MAE/-Distil models, and Mask2Former is also the baseline in Table 2. A distilled student approaching its own teacher's IoU is an expected distillation outcome and does not, by itself, demonstrate that VRE's multi-modal data are useful. Add ablations that train on (a) semantic pseudo-labels only, (b) full VRE multi-modal set, and (c) original human-annotated Dronescapes frames, to isolate the contribution of each modality.
  4. [§2.7 safe-landing formula] The safe-landing binary map is defined by thresholds (v2>0.8, (v1+v3)<1.2, depth<=0.9) and a manually defined safe_class mapping. The text states these thresholds 'can be updated based on experiments,' but no validation protocol, sensitivity analysis, or chosen values justification is provided. These are effectively free parameters in the dataset-creation process; without a fixed protocol the derived modalities are not reproducible and their downstream contribution cannot be assessed.
minor comments (4)
  1. [Abstract, §2.8] The comparison is described as '~300M parameters' and 'almost 2 orders of magnitude larger,' but Mask2Former has 217M parameters, which is about 49x the 4.4M PHG-MAE model; please correct the magnitudes.
  2. [§2.7] Define v1, v2, v3 and spell out the safe_class mapping; the formula as written is not self-contained.
  3. [Algorithm 2.1] 'reprintopo sort' appears to be a typo for 'topo sort'.
  4. [§3.4-3.5] FPS numbers are reported as point estimates; please add error bars or repeated-run statistics to support the real-time claim.

Circularity Check

2 steps flagged

Headline model results are imported from the authors' own [33] and are partly confounded with Mask2Former pseudo-label distillation; VRE's engineering itself is self-contained.

specific steps
  1. self citation load bearing [Abstract; Section 2.8 and Table 2]
    "We then make use of PHG-MAE[33], a model specifically designed to leverage multi-modal data. We show that this model which was efficiently distilled into a low-parameter (≤1M) can have competitive results compared to models of ∼300M parameters. [...] Using the Dronescapes2 dataset, the work of PHG-MAE [33] has trained a multi-modal multi-task learning model, designed specifically for this kind of data with just 4.4M parameters."

    The abstract's headline quantitative claim is not derived in this paper: every model row in Table 2 is cited to [33], a preprint by the same two authors (Pîrvu & Leordeanu). The argument that VRE-generated multi-modal data enable competitive lightweight models therefore rests on a self-citation chain rather than on independent experiments in the present article.

  2. fitted input called prediction [Section 2.7 and Table 2]
    "semantic segmentation (3): Mask2Former on three released checkpoints [...] The -Distil variants, which are trained on top of pseudo-labels generated by the -NRand model. [...] Mask2Former [11] 217M 53.97 [...] PHG-MAE-Distil [33] 150k 53.32"

    The comparison baseline is also the source of the training labels. Dronescapes2-M+ semantics are Mask2Former outputs; the -Distil models are trained on pseudo-labels produced by PHG-MAE-NRand, which was itself trained on those Mask2Former-derived labels. The 150k model's 53.32 IoU is therefore anchored to Mask2Former's 53.97 by construction, so the near-match is a teacher-student artifact and cannot independently establish that VRE's non-semantic modalities (depth, normals, binary maps) caused the performance.

full rationale

The VRE pipeline itself (Sections 2.1-2.6, 3.1-3.5) is a self-contained engineering contribution: its batching, streaming, multi-GPU, and export designs are not circular. Circularity concerns attach to the abstract's model-performance claim, which is imported wholesale from the sibling paper [33] and then confounded by the fact that the distilled student's semantic supervision originates from Mask2Former pseudo-labels. The same-group Dronescapes-Test is an external-validity issue, not a definitional circularity. Because the paper's stated main contribution is the data-processing/deployment side, the central engineering claim retains independent content; nevertheless the headline 'competitive with ~300M models' is not independently established here and is at least partly forced by the Mask2Former teacher-student setup. Score 4 reflects this partial, load-bearing self-citation/teacher-student circularity rather than a fully definitional collapse.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 0 invented entities

The central claim rests on (i) quality of pseudo-labels from pretrained experts, (ii) hand-set thresholds for procedural modalities, (iii) reliability of the self-created Dronescapes-Test benchmark, and (iv) correctness of Table 2's numbers from the authors' own PHG-MAE paper. None of these is independently verified in this submission. No new physical or theoretical entities are introduced; intermediate modalities are procedural label definitions.

free parameters (2)
  • safe-landing formula thresholds = 0.8, 1.2, 0.9
    Section 2.7 defines safe landing as (v2>0.8)*((v1+v3)<1.2)*(depth<=0.9)*safe_class(semantic); the thresholds are hand-selected and the text says they can be updated based on experiments, with no validation on held-out data.
  • safe_class semantic mapping = not specified
    Section 2.7 maps Mask2Former semantic classes to a binary safe/unsafe class; the mapping is manually defined and no quantitative basis is given.
axioms (4)
  • domain assumption Pre-trained experts (Mask2Former, Marigold) output correct enough pseudo-labels on unseen aerial/internet videos to serve as training annotations.
    The entire Dronescapes2-M+ dataset and downstream training rely on these labels; Section 2.7 treats them as ground truth without quantitative validation against human annotation.
  • ad hoc to paper Hand-chosen thresholds in procedural combinations generalize to new scenes.
    Section 2.7 states the thresholds can be updated based on experiments; they are not calibrated on a validation set.
  • domain assumption Dronescapes-Test human-annotated labels (116 annotated frames, 5.6K total frames) are a reliable evaluation benchmark for this task.
    This benchmark comes from the authors' prior work [32]; label propagation may introduce noise, and the central model comparison uses it exclusively.
  • domain assumption Mean IoU numbers in Table 2, taken from PHG-MAE [33] by the same authors, are correct and independently reproducible.
    The paper's 'competitive results' claim rests entirely on this self-cited table; no evaluation code or training artifacts are included in this submission.

pith-pipeline@v1.3.0-alltime-deepseek · 15254 in / 14352 out tokens · 117785 ms · 2026-08-04T09:28:52.777679+00:00 · methodology

0 comments
read the original abstract

The real-world is inherently multi-modal at its core. Our tools observe and take snapshots of it, in digital form, such as videos or sounds, however much of it is lost. Similarly for actions and information passing between humans, languages are used as a written form of communication. Traditionally, Machine Learning models have been unimodal (i.e. rgb -> semantic or text -> sentiment_class). Recent trends go towards bi-modality, where images and text are learned together, however, in order to truly understand the world, we need to integrate all these independent modalities. In this work we try to combine as many visual modalities as we can using little to no human supervision. In order to do this, we use pre-trained experts and procedural combinations between them on top of raw videos using a fully autonomous data-pipeline, which we also open-source. We then make use of PHG-MAE, a model specifically designed to leverage multi-modal data. We show that this model which was efficiently distilled into a low-parameter (<1M) can have competitive results compared to models of ~300M parameters. We deploy this model and analyze the use-case of real-time semantic segmentation from handheld devices or webcams on commodity hardware. Finally, we deploy other off-the-shelf models using the same framework, such as DPT for near real-time depth estimation.

Figures

Figures reproduced from arXiv: 2510.14862 by Marius Leordeanu, Mihai-Cristian P\^irvu.

Figure 1
Figure 1. Figure 1: High-level overview of an end-to-end machine learning system: from raw data and data processing, to training and optimizing models and lastly by deploying it to interact and control a real hardware autonomously with intelligent actions. worked very well and has driven the progress of the field with results such as the AlexNet [25] classification network on the ImageNet dataset, the Trans￾former network [44… view at source ↗
Figure 2
Figure 2. Figure 2: VRE showcase. We present six exported representations on top of the RGB frame. The first two are pre-trained experts (DPT [41] and Marigold [23]). Next, we derive two camera normals representations using a SVD-based algorithm [18]. Lastly, we derive safe-landing areas by thresholding the camera normals maps like in the newly introduced Dronescapes2 [33] dataset. In this section, we’ll discuss our approach … view at source ↗
Figure 3
Figure 3. Figure 3: VRE processing strategies. Left: the standard batched strategy. We split the frames in batches and then each batch is passed through the algorithm of the representation, followed by a step where the results are stored on the disk. Right: the streaming strategy. In this mode the input is a live video stream (webcam, camera phone etc.) Each frame, or nearby ones (if needed), are processed sequentially by all… view at source ↗
Figure 4
Figure 4. Figure 4: VRE streaming architecture. We read frame by frame from the source (i.e. drone camera), process it on the VRE streaming client (i.e. cloud or local GPU), analyze the results and pass the actions to the target (i.e. drone controller). Notably, all these components can live on the same machine but they can also communicate through the network. We provide various integrations through standard Linux tools (i.e… view at source ↗
Figure 5
Figure 5. Figure 5: VRE multi-gpu batching strategies. Strategy 1: Slice the video in multiple independent chunks. Strategy 2: split the video’s representations in sub-groups. Strategy 1 is the simplest and most effective one. The video is treated as multiple independent video chunks and assign each to one VRE process and to one accelerator (GPU). This strategy has the advantage that it is consis￾tent (i.e. each GPU will fini… view at source ↗
Figure 6
Figure 6. Figure 6: The data-pipeline and PHG-MAE model on real data. Left: The process of deriving modalities as pseudo-labels from pre-trained experts using RGB only, followed by deriving new modalities from combinations of experts. Right: integration of all the new modalities in the PHG-MAE semi-supervised training and inference pipeline, with each modality being either input, inter￾mediate or output. Model Parameters Mean… view at source ↗
Figure 7
Figure 7. Figure 7: Simple export experiment. We compare three output formats for two computed representations: RGB and HSV. We observe that the duration extends both with a more complex storage (i.e. npz only vs compressed npz), with the frame resolution as well as whether we export only a binary representation or both binary and image (jpg). Inter￾estingly, the compressed export saves about 2.6x disk space (2.4GB vs 907MB o… view at source ↗
Figure 8
Figure 8. Figure 8: Batched export results (CPU vs. GPU) on a local machine with three batch sizes (1, 5, 20) and two models Depth DPT (left) and PHG-MAE￾Distil-450k (right). First, we observe that the GPU (CUDA) variant constantly outperforms the CPU one on each experiment, regardless of batch size. This is expected as machine learning models are optimized for GPU usage. For the DPT model we observe about a 5x improvement, w… view at source ↗
Figure 9
Figure 9. Figure 9: Results on running the data-pipeline on the Dronescapes2 config. Left: bar plot with the average duration of each representation per frame. Right: total duration for different number of GPUs. In the left side we provide the statistics of running the experiment on a single GPU. We observe that most of the time is spent on a single representa￾tion, namely the normals from SVD algorithm, taking an average of … view at source ↗
Figure 10
Figure 10. Figure 10: All the extracted experts and derived intermediate modalities in the data-pipeline. All are generated starting from the RGB image only. video player. In [PITH_FULL_IMAGE:figures/full_fig_p019_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Streaming the frames of a video through various ML models. Processed on a local GPU. We observe that the models have quite low variance, most of the frames take about the same amount of time regardless of the model. Notably, the PHG-MAE-Distil variants can be used for real-time segmentation, while the Depth DPT can be used for real-time depth estimation, which can enable various robotics applications, suc… view at source ↗
Figure 12
Figure 12. Figure 12: Streaming the frames of a phone camera through various ML models. Processed on a local GPU and on a remote GPU. Left: the FPS results. Right: The live-streaming setup. On the right side we can see the streaming setup: we capture the camera feed from the mobile phone. Then, we relay it to the processing GPU (local or remote). The remote machine is the same as the one used in all the experiments before, whi… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

47 extracted references · 7 linked inside Pith

  1. [1]

    Quantifying overfitting: introducing the over- fitting index

    Sanad Aburass and Maha Abu Rumman. Quantifying overfitting: introducing the over- fitting index. In2024 International Conference on Electrical, Computer and Energy Technologies (ICECET), pages 1–7. IEEE, 2024

    2024

  2. [2]

    Uav computing-assisted search and rescue mission framework for disaster and harsh environment mitigation.Drones, 6(7):154, 2022

    Saeed Hamood Alsamhi, Alexey V Shvetsov, Santosh Kumar, Svetlana V Shvetsova, Mohammed A Alhartomi, Ammar Hawbani, Navin Singh Rajput, Sumit Srivastava, Abdu Saif, and Vincent Omollo Nyangaresi. Uav computing-assisted search and rescue mission framework for disaster and harsh environment mitigation.Drones, 6(7):154, 2022

    2022

  3. [3]

    Real-time on-board deep learning fault de- tection for autonomous uav inspections.Electronics, 10(9):1091, 2021

    Naeem Ayoub and Peter Schneider-Kamp. Real-time on-board deep learning fault de- tection for autonomous uav inspections.Electronics, 10(9):1091, 2021. 22 P ˆ ırvu Mihai-Cristian, Marius Leordeanu

    2021

  4. [4]

    Multimae: Multi- modal multi-task masked autoencoders

    Roman Bachmann, David Mizrahi, Andrei Atanov, and Amir Zamir. Multimae: Multi- modal multi-task masked autoencoders. InEuropean Conference on Computer Vision, pages 348–367. Springer, 2022

    2022

  5. [5]

    wav2vec 2.0: A framework for self-supervised learning of speech representations.Advances in neural information processing systems, 33:12449–12460, 2020

    Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: A framework for self-supervised learning of speech representations.Advances in neural information processing systems, 33:12449–12460, 2020

    2020

  6. [6]

    Ofasys: A multi-modal multi-task learning system for building generalist models.arXiv preprint arXiv:2212.04408, 2022

    Jinze Bai, Rui Men, Hao Yang, Xuancheng Ren, Kai Dang, Yichang Zhang, Xiao- huan Zhou, Peng Wang, Sinan Tan, An Yang, et al. Ofasys: A multi-modal multi-task learning system for building generalist models.arXiv preprint arXiv:2212.04408, 2022

    Pith/arXiv arXiv 2022

  7. [7]

    Demystifying power and performance bottlenecks in autonomous driving systems

    Pedro HE Becker, Jose Maria Arnau, and Antonio Gonz´ alez. Demystifying power and performance bottlenecks in autonomous driving systems. In2020 IEEE International Symposium on Workload Characterization (IISWC), pages 205–215. IEEE, 2020

    2020

  8. [8]

    Findings of the 2014 workshop on statistical machine translation

    Ondˇ rej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand, et al. Findings of the 2014 workshop on statistical machine translation. InProceedings of the ninth workshop on statistical machine translation, pages 12–58, 2014

    2014

  9. [9]

    A computational approach to edge detection.IEEE Transactions on pat- tern analysis and machine intelligence, (6):679–698, 1986

    John Canny. A computational approach to edge detection.IEEE Transactions on pat- tern analysis and machine intelligence, (6):679–698, 1986

    1986

  10. [10]

    Level 2 autonomous driving on a single device: Diving into the devils of openpilot.arXiv preprint arXiv:2206.08176, 2022

    Li Chen, Tutian Tang, Zhitian Cai, Yang Li, Penghao Wu, Hongyang Li, Jianping Shi, Junchi Yan, and Yu Qiao. Level 2 autonomous driving on a single device: Diving into the devils of openpilot.arXiv preprint arXiv:2206.08176, 2022

    Pith/arXiv arXiv 2022

  11. [11]

    Masked-attention mask transformer for universal image segmentation

    Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexander Kirillov, and Rohit Gird- har. Masked-attention mask transformer for universal image segmentation. InProceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1290–1299, 2022

    2022

  12. [12]

    The cityscapes dataset for semantic urban scene understanding

    Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 3213–3223, 2016

    2016

  13. [13]

    Bert: Pre- training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre- training of deep bidirectional transformers for language understanding. InProceedings of the 2019 conference of the North American chapter of the association for compu- tational linguistics: human language technologies, volume 1 (long and short papers), pages 4171–4186, 2019

    2019

  14. [14]

    An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Pith/arXiv arXiv 2010

  15. [15]

    Peter D Dueben, Martin G Schultz, Matthew Chantry, David John Gagne, David Matthew Hall, and Amy McGovern. Challenges and benchmark datasets for machine learning in the atmospheric sciences: Definition, status, and outlook.Artificial Intelligence for the Earth Systems, 1(3):e210002, 2022

    2022

  16. [16]

    Alicevision Meshroom: An open-source 3D reconstruction pipeline

    Carsten Griwodz, Simone Gasparini, Lilian Calvet, Pierre Gurdjos, Fabien Castan, Benoit Maujean, Gregoire De Lillo, and Yann Lanthony. Alicevision Meshroom: An open-source 3D reconstruction pipeline. InProceedings of the 12th ACM Multimedia Systems Conference - MMSys ’21. ACM Press, 2021

    2021

  17. [17]

    Self-supervised learning in multi-task graphs through iterative consensus shift.arXiv preprint arXiv:2103.14417, 2021

    Emanuela Haller, Elena Burceanu, and Marius Leordeanu. Self-supervised learning in multi-task graphs through iterative consensus shift.arXiv preprint arXiv:2103.14417, 2021

    Pith/arXiv arXiv 2021

  18. [18]

    Cambridge university press, 2003

    Richard Hartley and Andrew Zisserman.Multiple view geometry in computer vision. Cambridge university press, 2003. Multi-modal video data-pipelines for machine learning with minimal human supervision 23

    2003

  19. [19]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ ar, and Ross Girshick. Masked autoencoders are scalable vision learners. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000–16009, 2022

    2022

  20. [20]

    Flood detection using real-time image segmentation from unmanned aerial vehicles on edge- computing platform.remote Sensing, 14(1):223, 2022

    Daniel Hern´ andez, Jos´ e M Cecilia, Juan-Carlos Cano, and Carlos T Calafate. Flood detection using real-time image segmentation from unmanned aerial vehicles on edge- computing platform.remote Sensing, 14(1):223, 2022

    2022

  21. [21]

    Real-time intermediate flow estimation for video frame interpolation

    Zhewei Huang, Tianyuan Zhang, Wen Heng, Boxin Shi, and Shuchang Zhou. Real-time intermediate flow estimation for video frame interpolation. InEuropean Conference on Computer Vision, pages 624–642. Springer, 2022

    2022

  22. [22]

    Software 2.0.https://web.archive.org/web/20250323195948/ https://karpathy.medium.com/software-2-0-a64152b37c35, 2025

    Andrej Karpathy. Software 2.0.https://web.archive.org/web/20250323195948/ https://karpathy.medium.com/software-2-0-a64152b37c35, 2025. [Online; ac- cessed 04-April-2025]

    arXiv 2025

  23. [23]

    Repurposing diffusion-based image generators for monocular depth estimation

    Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Metzger, Rodrigo Caye Daudt, and Konrad Schindler. Repurposing diffusion-based image generators for monocular depth estimation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9492–9502, 2024

    2024

  24. [24]

    Seg- ment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Seg- ment anything. InProceedings of the IEEE/CVF international conference on computer vision, pages 4015–4026, 2023

    2023

  25. [25]

    Imagenet classification with deep convolutional neural networks.Advances in neural information processing systems, 25, 2012

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks.Advances in neural information processing systems, 25, 2012

    2012

  26. [26]

    Semi-supervised learning for multi-task scene understand- ing by neural graph consensus

    Marius Leordeanu, Mihai Cristian P ˆ ırvu, Dragos Costea, Alina E Marcu, Emil Slusan- schi, and Rahul Sukthankar. Semi-supervised learning for multi-task scene understand- ing by neural graph consensus. InProceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 1882–1892, 2021

    2021

  27. [27]

    Generalized bound- aries from multiple image interpretations.IEEE transactions on pattern analysis and machine intelligence, 36(7):1312–1324, 2014

    Marius Leordeanu, Rahul Sukthankar, and Cristian Sminchisescu. Generalized bound- aries from multiple image interpretations.IEEE transactions on pattern analysis and machine intelligence, 36(7):1312–1324, 2014

    2014

  28. [28]

    Prophet: Realizing a pre- dictable real-time perception pipeline for autonomous vehicles

    Liangkai Liu, Zheng Dong, Yanzhi Wang, and Weisong Shi. Prophet: Realizing a pre- dictable real-time perception pipeline for autonomous vehicles. In2022 IEEE Real-Time Systems Symposium (RTSS), pages 305–317. IEEE, 2022

    2022

  29. [29]

    Unified-io 2: Scaling autoregressive multi- modal models with vision language audio and action

    Jiasen Lu, Christopher Clark, Sangho Lee, Zichen Zhang, Savya Khosla, Ryan Marten, Derek Hoiem, and Aniruddha Kembhavi. Unified-io 2: Scaling autoregressive multi- modal models with vision language audio and action. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26439–26455, 2024

    2024

  30. [30]

    Quantifying the synthetic and real domain gap in aerial scene understand- ing.arXiv preprint arXiv:2411.19913, 2024

    Alina Marcu. Quantifying the synthetic and real domain gap in aerial scene understand- ing.arXiv preprint arXiv:2411.19913, 2024

    Pith/arXiv arXiv 2024

  31. [31]

    Semantics through time: Semi-supervised segmentation of aerial videos with iterative label propagation

    Alina Marcu, Vlad Licaret, Dragos Costea, and Marius Leordeanu. Semantics through time: Semi-supervised segmentation of aerial videos with iterative label propagation. InProceedings of the Asian Conference on Computer Vision, 2020

    2020

  32. [32]

    Romanian Hub for Artificial Intelligence - HRIA

    dataset for aerial image understanding which was extended in a fully automated way using our data-pipeline:https://sites.google.com/view/ dronescapes-dataset. Finally, we discuss PHG-MAE [33], a MAE-based model which has leveraged our data-pipeline by creating an ensemble-based algorithm which operates at intermediate modalities level exported by us. This...

    2021

  33. [33]

    Self-supervised hy- pergraphs for learning multiple world interpretations

    Alina Marcu, Mihai Pirvu, Dragos Costea, Emanuela Haller, Emil Slusanschi, Ahmed Nabil Belbachir, Rahul Sukthankar, and Marius Leordeanu. Self-supervised hy- pergraphs for learning multiple world interpretations. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 983–992, 2023

    2023

  34. [34]

    Probabilistic hyper-graphs using multiple ran- domly masked autoencoders for semi-supervised multi-modal multi-task learning, 2025

    P ˆ ırvu Mihai-Cristian and Leordeanu M. Probabilistic hyper-graphs using multiple ran- domly masked autoencoders for semi-supervised multi-modal multi-task learning, 2025

    2025

  35. [35]

    4m: Massively multimodal masked modeling.Advances in Neural Information Processing Systems, 36:58363–58408, 2023

    David Mizrahi, Roman Bachmann, Oguzhan Kar, Teresa Yeo, Mingfei Gao, Afshin Dehghan, and Amir Zamir. 4m: Massively multimodal masked modeling.Advances in Neural Information Processing Systems, 36:58363–58408, 2023. 24 P ˆ ırvu Mihai-Cristian, Marius Leordeanu

    2023

  36. [36]

    Librispeech: an asr corpus based on public domain audio books

    Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. Librispeech: an asr corpus based on public domain audio books. In2015 IEEE international con- ference on acoustics, speech and signal processing (ICASSP), pages 5206–5210. IEEE, 2015

    2015

  37. [37]

    Multi-task hypergraphs for semi-supervised learning using earth observations

    Mihai Pirvu, Alina Marcu, Maria Alexandra Dobrescu, Ahmed Nabil Belbachir, and Marius Leordeanu. Multi-task hypergraphs for semi-supervised learning using earth observations. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 3404–3414, 2023

    2023

  38. [38]

    Dense extreme inception network: Towards a robust cnn model for edge detection

    Xavier Soria Poma, Edgar Riba, and Angel Sappa. Dense extreme inception network: Towards a robust cnn model for edge detection. InProceedings of the IEEE/CVF winter conference on applications of computer vision, pages 1923–1932, 2020

    1923

  39. [39]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational confer- ence on machine learning, pages 8748–8763. PmLR, 2021

    2021

  40. [40]

    Improving language understanding by generative pre-training

    Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understanding by generative pre-training. 2018

    2018

  41. [41]

    Ai and the everything in the whole wide world benchmark

    Inioluwa Deborah Raji, Emily M Bender, Amandalynne Paullada, Emily Denton, and Alex Hanna. Ai and the everything in the whole wide world benchmark. arxiv.arXiv preprint arXiv:2111.15366, 2021

    Pith/arXiv arXiv 2021

  42. [42]

    Vision transformers for dense prediction

    Ren´ e Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vision transformers for dense prediction. InProceedings of the IEEE/CVF international conference on computer vi- sion, pages 12179–12188, 2021

    2021

  43. [43]

    Raft: Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. InComputer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16, pages 402–419. Springer, 2020

    2020

  44. [44]

    Converting video formats

    Suramya Tomar. Converting video formats. ffmpeg.Linux Journal, 2006(146):10, 2006

    2006

  45. [45]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017

    2017

  46. [46]

    Taskonomy: Disentangling task transfer learning

    Amir R Zamir, Alexander Sax, William Shen, Leonidas J Guibas, Jitendra Malik, and Silvio Savarese. Taskonomy: Disentangling task transfer learning. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 3712–3722, 2018

    2018

  47. [47]

    Fast segment anything.arXiv preprint arXiv:2306.12156, 2023

    Xu Zhao, Wenchao Ding, Yongqi An, Yinglong Du, Tao Yu, Min Li, Ming Tang, and Jinqiao Wang. Fast segment anything.arXiv preprint arXiv:2306.12156, 2023

    Pith/arXiv arXiv 2023