Pith. sign in

REVIEW 21 references

Extract and Merge: Merging extracted humans from different images utilizing Mask R-CNN

T0 review · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read An application that extracts detected people from multiple images or videos with Mask R-CNN and composites them onto a new background, layer by layer.

arxiv 1908.00398 v1 pith:NAULETV5 submitted 2019-08-01 cs.CV cs.GRcs.LG

Asati Minkesh , Kraisittipong Worranitta , Miyachi Taizo This is my paper · ORCID
classification cs.CVcs.GRcs.LG
keywords imagesapplicationdifferentmaskextracthumanmergingr-cnn
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The authors describe a pipeline that takes two or more images (or videos) and a background image as input. First, Mask R-CNN, a pre-trained instance segmentation model, detects all objects in each input and generates a pixel-level mask for each object. The pipeline then filters out everything that is not a person, and if the user wants fewer people than detected, it selects the people with the largest bounding-box area. Finally, the selected people are copied, mask pixel by mask pixel, onto the background image. Because every input is resized to the same dimensions before processing, the mask coordinates align and the paste operation is a direct pixel replacement. For video inputs, the application processes frames until the longest input video ends, so the output video matches that length. The paper shows qualitative examples of single-person and multi-person extraction and merging. It reports a speed of five frames per second on an Nvidia Titan V, but provides no quantitative evaluation of segmentation accuracy or output quality. The core technique is a straightforward use of existing Mask R-CNN features, with no changes to the network architecture.
Extended reading notes

Core claim

The application can extract selected human instances from multiple input images or videos and merge them into a new background layer by layer, running at five frames per second without adding overhead to Mask R-CNN. This is stated in the abstract and in Section 5.3, where examples of single-person, multi-person, and video merging are shown.

Load-bearing premise

The whole pipeline depends on the pre-trained COCO Mask R-CNN producing accurate person masks on arbitrary user inputs, including crowded or low-quality images; no evaluation of mask quality is provided, so the compositing quality rests on this unmeasured assumption. This enters in Section 5.2, where the authors use pre-trained weights from the Matterport Mask R-CNN implementation.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no free parameters or invented entities. It leans on pre-trained Mask R-CNN generalization, the area-based selection heuristic, and uniform resizing as unstated domain assumptions.

assumptions (3)
  • domain assumption Pre-trained Mask R-CNN trained on COCO generalizes to the user-provided images and videos in the application.
    The pipeline relies on this without testing. Location: Section 5.2, 'We used pre-trained weights of this network that was trained on MS COCO dataset.'
  • domain assumption Bounding-box area is a good proxy for which people are significant enough to extract.
    The second stage selects people by area size, assuming larger people are the intended subjects. Location: Section 4, second stage.
  • domain assumption Resizing all inputs to the same size preserves sufficient visual quality for compositing.
    The pipeline resizes all images and background to the same size before processing, which can distort aspect ratios but is assumed to be acceptable. Location: Section 4, 'we resized all input images and background image to the same size.'

how reviews work

0 comments
Cite this review

Pith. "Pith review of Extract and Merge: Merging extracted humans from different images utilizing Mask R-CNN." pith.science (2026). https://pith.science/paper/NAULETV5

@misc{pith2026190800398,
  author       = {Pith},
  title        = {Pith review of: Extract and Merge: Merging extracted humans from different images utilizing Mask R-CNN},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NAULETV5}},
  note         = {Machine review of arXiv:1908.00398}
}
read the original abstract

Selecting human objects out of the various type of objects in images and merging them with other scenes is manual and day-to-day work for photo editors. Although recently Adobe photoshop released "select subject" tool which automatically selects the foreground object in an image, but still requires fine manual tweaking separately. In this work, we proposed an application utilizing Mask R-CNN (for object detection and mask segmentation) that can extract human instances from multiple images and merge them with a new background. This application does not add any overhead to Mask R-CNN, running at 5 frames per second. It can extract human instances from any number of images or videos from merging them together. We also structured the code to accept videos of different lengths as input and length of the output-video will be equal to the longest input-video. We wanted to create a simple yet effective application that can serve as a base for photo editing and do most time-consuming work automatically, so, editors can focus more on the design part. Other application could be to group people together in a single picture with a new background from different images which could not be physically together. We are showing single-person and multi-person extraction and placement in two different backgrounds. Also, we are showing a video example with single-person extraction.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 21 canonical work pages

  1. [1]

    Fully convolutional instance -aware semantic segmentation,

    Y. Li, H. Qi, J. Dai, X. Ji, and Y. Wei. “Fully convolutional instance -aware semantic segmentation,” In CVPR, 2017

    work page 2017

  2. [2]

    R -FCN: Object detection via region -based fully convolutional networks,

    J. Dai, Y. Li, K. He and J. Sun. “R -FCN: Object detection via region -based fully convolutional networks,” In NIPS, 2016

    work page 2016

  3. [3]

    Mask-RCNN,

    K. He, G. Gkioxari, P. Dollar and R. Girshick. “Mask-RCNN,” In ICCV, 2017

    work page 2017

  4. [4]

    Instance -sensitive fully convolutional networks,

    J. Dai, K. He, Y. Li, S. Ren, and J. Sun. “Instance -sensitive fully convolutional networks,” In ECCV, 2016

    work page 2016

  5. [5]

    Selective Search for Object Recognition. International Journal of Computer Vision,

    J. R. R. Uijlings, K. E. A. van de Sande, T. Gevers and A. W. M. Smeulders. “Selective Search for Object Recognition. International Journal of Computer Vision,” In IJCV, 2013

    work page 2013

  6. [6]

    Rich feature hierarchies for accurate object detection and semantic segmentation,

    R. Girshick, J. Donahue, T. Darrell, and J. Malik. “Rich feature hierarchies for accurate object detection and semantic segmentation,” In CVPR, 2014

    work page 2014

  7. [7]

    Gradient -Based Learning Applied to Document Recognition,

    Y. LeCun, L. Bottou , Y. Bengio and P. Haffner “Gradient -Based Learning Applied to Document Recognition,” In Proc. of the IEEE, 1998s

    work page

  8. [8]

    ImageNet classification with deep convolutional neural networks,

    A. Krizhevsky, I. Sutskever, and G. Hinton. “ImageNet classification with deep convolutional neural networks,” In NIPS, 2012

    work page 2012

Show all 21 references
  1. [9]

    Faster R -CNN: Towards Real - Time Object Detection with Region Proposal Networks,

    S. Ren, K. He, R . Girshick, and J. Sun. “Faster R -CNN: Towards Real - Time Object Detection with Region Proposal Networks,” In NIPS, 2015

    2015

  2. [10]

    You Only Look Once: Unified, Real - Time Object Detection,

    J. Redmon, S. Divvala, R. Girshick and A. Farhadi. “You Only Look Once: Unified, Real - Time Object Detection,” In CVPR, 2016

    2016

  3. [11]

    SSD: Single Shot MultiBox Detector,

    W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu and A. C. Berg. “SSD: Single Shot MultiBox Detector,” In ECCV, 2016

    2016

  4. [12]

    Microsoft COCO: Common Objects in Context,

    T. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, C. L. Zitnick. “Microsoft COCO: Common Objects in Context,” In ECCV, Part V, LNCS 8693, pp. 740– 755, 2014

    2014

  5. [13]

    ImageNet: A Large - Scale Hierarchical Image Database,

    J. Deng, W. Dong, R. Socher, L. -J. Li, K. Li and L. Fei -Fei. “ImageNet: A Large - Scale Hierarchical Image Database,” In CVPR, 2009

    2009

  6. [14]

    Fast R-CNN,

    R. Girshick. "Fast R-CNN," In ICCV, 2015

    2015

  7. [15]

    Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow,

    Matterport Inc. “Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow,” Retrieved 2017.From https://github.com/matterport/Mask_RCNN

    2017

  8. [16]

    Utilizing Mask R -CNN for Detection and Segmentation of Oral Diseases,

    R. Anantharaman, M. Velazquez and Y. Lee “Utilizing Mask R -CNN for Detection and Segmentation of Oral Diseases,” In BIBM, 2018

    2018

  9. [17]

    Robot-Assisted Training in Laparoscopy Using Deep Reinforcement Learning

    X. Tan, C. -B. Chng, Y. Su, K. -B. Lim and C. -K. Chui . “ Robot-Assisted Training in Laparoscopy Using Deep Reinforcement Learning ”, In IEEE Robotics and Automation Letters 4, 2019

    2019

  10. [18]

    Inshore Ship Detection Bas ed on Mask R- CNN,

    S. Nie, Z. Jiang, H. Zhang, B. Cai and Y. Yao. “Inshore Ship Detection Bas ed on Mask R- CNN,” In IGARSS, 2018

    2018

  11. [19]

    Image blending with Mask R-CNN and OpenCV,

    HiuKim Yuen. "Image blending with Mask R-CNN and OpenCV," Retrieved July 22, 2018 from https://medium.com/softmind-engineering/image-blending-with-mask-r-cnn-and- opencv-eb5ac521f920

    2018

  12. [20]

    Stuart Weitzman Boots, Designe r Bags, and Outfits with Mask R - CNN,

    Michael Sugimura. "Stuart Weitzman Boots, Designe r Bags, and Outfits with Mask R - CNN," Retrieved October 8, 2018 from https://towardsdatascience.com/stuart-weitzman- boots-designer-bags-and-outfits-with-mask-r-cnn-92a267a02819

    2018

  13. [21]

    Image to OSM,

    Jason Remillard. “Image to OSM,” Retrieved February 3,2018. from https://github.com/jremillard/images-to-osm

    2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.