Multi-Stream Single Shot Spatial-Temporal Action Detection

Benyuan Liu; Pengfei Zhang; Yu Cao

REVIEW 4 major objections 5 minor 26 references

Multi-Stream Single Shot Spatial-Temporal Action Detection

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Adding two 3D ConvNet streams to an SSD action detector yields a 71.30 frame-mAP on UCF101-24, the best one-stage result reported.

desk verdict A plausible but unverifiable engineering extension: useful ablation, but the headline SOTA number cannot be trusted without code, exact protocol, and variance estimates. read the letter →

arxiv 1908.08178 v1 pith:DBR7JUES submitted 2019-08-22 cs.CV

Pengfei Zhang , Yu Cao , Benyuan Liu This is my paper · ORCID

classification cs.CV

keywords actiondetectionspatial-temporallocalization3DconvolutionalneuralnetworksSSDopticalflowmulti-streamfusionUCF101-24videounderstanding

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a single-pass (one-stage) action detector can beat prior one-stage methods by running four complementary streams at once: a 2D RGB stream for the current frame, a 2D optical-flow stream for short-term motion, and two 3D ConvNet streams that ingest eight-frame clips of RGB and optical flow to capture longer context. On the UCF101-24 benchmark, the fused four-stream model reports 71.30 frame-mAP at IoU 0.5, which the authors state is the best one-stage result on that dataset. The practical payoff would be a detector that localizes actions in space and time in one forward pass, without a separate region-proposal stage, making online or real-time video analysis more feasible.

What carries the argument

The machinery is a four-stream SSD: each stream uses a truncated VGG-16 backbone converted to convolutional layers, plus eight extra layers and detection heads, while the 3D streams are created by inflating all 2D kernels to 3x3x3, repeating pretrained weights along the temporal dimension, and passing feature maps through temporal mean pooling to collapse the frame dimension before SSD-style box prediction. Late fusion keeps one stream's box regressions and averages the class confidence scores across streams. This design lets a single pass combine frame-level spatial accuracy with temporal context from past frames.

What would settle it

Run the same four-stream configuration several times on UCF101-24 split 1 with the same hyperparameters and record the spread of frame-mAP; if the best run is not reliably above the 67.81 frame-mAP of the paper's 2D SSD baseline, the claimed four-stream advantage collapses.

Watch

Extended reading notes

Core claim

The central discovery is that combining short-term 2D appearance and motion streams with long-term 3D appearance and motion streams in a single SSD detector consistently improves frame-level and video-level action localization over any two-stream variant. Under late fusion, each stream contributes differently: 2D RGB is the best appearance stream because it gives clean spatial features of the current frame, while 3D optical flow is the best motion stream, adding 1.66 frame-mAP over 2D optical flow. The paper also reports category-level patterns: 3D RGB tolerates moving-camera (active-background) videos better than optical flow, whereas optical flow wins on fixed-background videos with fast short actions. The best configuration fuses all four streams and reaches 71.30 frame-mAP, surpassing prior one-stage detectors; the authors additionally claim this is the first system to combine 3D CNNs with SSD for action detection.

Load-bearing premise

The single reported frame-mAP of 71.30 on one split of UCF101-24, with no variance estimate, is treated as a stable measurement; if the number fluctuates across training runs, the state-of-the-art claim does not stand.

Editorial extensions

If this is right

Any two-stream action detector can adopt the 3D streams and temporal pooling directly, since the paper presents them as plug-in additions.
A one-stage detector built this way runs localization and classification in a single pass, making it better suited than two-stage proposal-based detectors for online and real-time settings.
3D RGB as a motion stream is a useful fallback when optical flow is unreliable, such as in active-background videos.
The best stream combination depends on video category, so the four-stream fusion is a robust default that adapts to intra-class variability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the reported 71.30 frame-mAP holds up under replication, the practical implication is that one-stage detectors need not trade away temporal reasoning: fusing cheap long-range 3D streams can close much of the gap to two-stage tube detectors.
A direct testable extension would be swapping the 2D appearance stream for a modern lightweight backbone; the paper's ablation suggests the 3D motion streams, not the 2D appearance stream, carry most of the temporal gain.
The category-level split between active-background and fixed-background videos implies a dataset-adaptive fusion policy: weighting 3D RGB higher when camera motion is detected could outperform uniform late fusion.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a modest engineering extension, not a breakthrough, and the headline result is not independently checkable from the paper. The useful part is the ablation: sorting 2D and 3D RGB/optical-flow streams into short-term and long-term roles, with a temporal pooling layer that connects an inflated 3D VGG backbone to SSD detection heads. The empirical story is self-consistent, and the per-class analysis of active versus fixed background videos gives a credible explanation for why 3D RGB helps in some cases and optical flow in others. Single-stream 3D beats 2D, and the four-stream late fusion reaches 71.30 frame-mAP on UCF101-24 split 1. That is a real measurement, not a fitted constant.

The soft spots are mostly about verifiability. No code or weights are released, and the evaluation metric is under-specified: no statement on how frames without ground-truth boxes are handled, how duplicate detections are matched, or how AP is aggregated over classes. The compared methods are tightly bunched (67.10–67.81), so a difference of a few mAP from a protocol quirk could flip the state-of-the-art claim. There are no error bars or repeated runs; 71.28 and 71.30 are different architectural configurations, not repeats of the same run, so the headline number includes a small post-hoc selection. The baselines are partly borrowed rather than re-run, which is acceptable for an ablation but weak for a SOTA claim. Finally, the novelty statement “first to combine 3D CNN and SSD” does not survive contact with the cited Hou et al. C3D one-stage detector; the authors dismiss it as offline, but the claim should be softened.

None of this makes the paper nonsense. The architecture is plausible, the ablation is informative, and the 3D-stream improvements over 2D are consistent across table rows. The central claim is probably true, but it is not proven by the evidence in hand. This is a classic “send to peer review with a demand for code and exact evaluation details” case, rather than a desk reject or an accept. The right reader is a practitioner building one-stage action detectors who wants a baseline fusion recipe; the wrong reader is someone looking for a verified SOTA number. I would not cite it in the next year until the code appears, but I would not mind seeing it discussed in a reading group.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes a multi-stream single-shot action detector that combines two 2D streams (RGB and optical flow) with two 3D streams (RGB and optical flow), fusing their outputs with late fusion inside an SSD framework. The architecture is described in Section 3, and experiments on UCF101-24 split 1 are reported in Section 4. The paper claims that the four-stream model reaches 71.30 frame-mAP at IoU 0.5, which it states is the state-of-the-art result among one-stage methods, and that this is the first system to combine 3D CNNs with SSD for action detection. The paper also presents ablations showing that 3D streams improve over their 2D counterparts and that different stream combinations perform differently for active-background versus fixed-background videos.

Significance. If the reported number is reproducible, the paper provides a useful empirical demonstration that 3D temporal streams can be integrated into a one-stage detector and that their combination with 2D appearance streams improves frame-level action detection on a public benchmark. The paper deserves credit for performing a systematic ablation over stream combinations and for reporting results on UCF101-24 using standard metrics. However, the central claim rests on a single mAP value with no variance estimate, and the evaluation protocol is not specified in enough detail to rule out small but consequential implementation differences. The practical contribution is therefore moderate: the architecture is a straightforward combination of existing components, but the empirical findings are potentially valuable to practitioners.

major comments (4)

[Section 4.1 (Evaluation metrics)] The frame-mAP definition is incomplete. Section 4.1 states only that a detection is correct if the IoU with ground truth exceeds a threshold and the category is correct; it does not specify how duplicate detections are matched to a single ground-truth box, how frames without ground-truth instances are handled, how average precision is aggregated over the 24 classes, or whether tube linking is applied before computing the frame-mAP. Because the three prior results in Table 3 (67.10, 67.3, 67.81) are tightly clustered, a systematic difference in the evaluation implementation of even a few mAP could change the ranking. Please provide the exact protocol or release the evaluation code.
[Table 3 and Abstract] The headline 71.30 frame-mAP is a single run on a single split with no error bars, and the two four-stream rows (71.28 and 71.30) are not repeated runs of the same configuration but different choices of the appearance stream. Reporting the higher of the two as the abstract's headline result therefore incorporates a small post-hoc selection, and a 0.02 mAP margin is within typical run-to-run variation. Please report results from multiple runs, provide error bars or significance estimates, and state whether the appearance-stream choice was made before or after evaluating all fusion configurations.
[Section 3 (Model Description) and Section 4 (Experiments)] Training and inference details are too sparse for reproducibility. The paper states N=8 frames and describes 2D-to-3D kernel inflation and temporal pooling, but omits the learning rate schedule, batch size, number of training iterations, input resolution, optical flow stack representation, and the exact placement of the temporal pooling layer relative to the inflated VGG backbone. Without these details, a reader cannot verify that the reported improvement over the 2D baseline is not due to an implementation choice, and the experiments cannot be independently reproduced.
[Section 1 (Introduction) and Section 5 (Conclusions)] The paper motivates its design by the speed advantage of one-stage detectors, but it reports no inference time, FLOPs, or latency comparison against two-stage methods. Since the proposed model runs four streams and requires optical flow estimation for every input frame, the claimed practical speed advantage is not established by the reported experiments. Please add a runtime measurement or explicitly state that speed is not evaluated.

minor comments (5)

[Abstract] The abstract contains wording issues: '3D Convolutional Neural Networks (CNNs) based single shot detector' should be 'a 3D CNN-based single-shot detector', and 'the first system that combined 3D CNN and SSD' should be 'the first system to combine 3D CNN and SSD'.
[Section 4.2 (Two-Stream)] The sentence reporting a 2.23% improvement for the three-stream model does not state the baseline explicitly; it should say 'over Singh et al.'s 2D SSD baseline' to match Table 3.
[Figure 3] The per-class average precision differences in Figure 3 are difficult to read because the 24 class names are not legible; please list the classes in a table or use annotations that remain readable in print.
[Section 2 (Related Work)] Reference [10] is the AVA dataset paper, and the sentence 'The most recent state-of-the-art result is achieved by Gu et al. [10] based on I3D and faster-RCNN' is misleading because AVA is not a UCF101-24 benchmark result; please cite a method that actually reports state-of-the-art on UCF101-24.
[Tables 1-3] The metric names are inconsistent: the tables use 'f.-mAP' while the text uses 'frame-mAP'. Please unify the notation throughout.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the central result is an empirical measurement on a public benchmark with external baselines, not a derivation that reduces to its inputs.

full rationale

This is an empirical systems paper. The central claim is a reported frame-mAP of 71.30% on UCF101-24 split 1, obtained by running the proposed four-stream SSD detector and comparing it with previously published one-stage methods. There is no mathematical derivation chain in which an output is defined in terms of an input or in which a fitted parameter is later renamed as a prediction. The authors do not cite their own prior work for any load-bearing premise; the baselines are external works (Singh et al., Kalogeiton et al., Hou et al.), and the evaluation metric is a standard mAP computation on a public dataset. The concern that the headline number is not independently reproducible due to missing code, unspecified protocol details, and lack of variance is a correctness/reproducibility issue, not circularity: an unreported or noisy measurement is not a self-referential derivation. The paper is self-contained against external benchmarks, so no circular step is present and the circularity score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper's central claim depends on standard empirical assumptions in video action detection: the benchmark annotations are reliable, optical flow is a valid motion cue, ImageNet-pretrained weights transfer to 3D by inflation, and equal-weight late fusion is a reasonable integration. No invented entities or model-specific fitted constants beyond hand-set hyperparameters are introduced.

free parameters (2)

N (input frame count for 3D streams) = 8
Chosen by hand in Section 3: 'We set N = 8 frames in the experiment.' The long-term context captured by the 3D streams depends on this window size, and no sensitivity analysis is provided.
Late fusion weights = Equal weighting (1/n streams)
In the Fusion Method section, each box's confidence is the average of corresponding boxes across streams. Equal weights are selected by design, not learned or ablated; alternative weightings are not tested.

assumptions (4)

domain assumption UCF101-24 split 1 annotations and evaluation metrics are reliable
Section 4.1 uses the first split and standard frame/video mAP; the paper assumes this benchmark measures action detection quality.
domain assumption Optical flow from Brox et al. is a valid motion representation
Section 3 uses Brox optical flow as the motion stream input; the paper does not validate this choice against other flow methods.
domain assumption ImageNet-pretrained VGG-16 weights transfer to 3D via kernel inflation
Section 3 initializes 3D kernels by repeating 2D weights; the paper assumes this initialization is effective and does not ablate it.
domain assumption Equal-weight late fusion preserves detector accuracy
Section 3's fusion keeps appearance-stream boxes and averages scores; this heuristic is assumed to be valid without comparison to learned fusion.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Multi-Stream Single Shot Spatial-Temporal Action Detection." pith.science (2026). https://pith.science/paper/DBR7JUES

@misc{pith2026190808178,
  author       = {Pith},
  title        = {Pith review of: Multi-Stream Single Shot Spatial-Temporal Action Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DBR7JUES}},
  note         = {Machine review of arXiv:1908.08178}
}

read the original abstract

We present a 3D Convolutional Neural Networks (CNNs) based single shot detector for spatial-temporal action detection tasks. Our model includes: (1) two short-term appearance and motion streams, with single RGB and optical flow image input separately, in order to capture the spatial and temporal information for the current frame; (2) two long-term 3D ConvNet based stream, working on sequences of continuous RGB and optical flow images to capture the context from past frames. Our model achieves strong performance for action detection in video and can be easily integrated into any current two-stream action detection methods. We report a frame-mAP of 71.30% on the challenging UCF101-24 actions dataset, achieving the state-of-the-art result of the one-stage methods. To the best of our knowledge, our work is the first system that combined 3D CNN and SSD in action detection tasks.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 25 canonical work pages

[1]

Multi-Stream Single Shot Spatial-Temporal Action Detection

INTRODUCTION The objective of action detection is to recognize and local- ize all the human action instances in a given video across both space and time. It is a fundamental task for video un- derstanding and important for practical applications such as video surveillance and human-robot interaction. Action de- tection is a challenging problem due to two ...

work page Pith review arXiv 2019
[2]

Gkioxari and Ma- lik [9] applied a two-stream R-CNN based framework to pro- duce frame level detections, and then linked the result to tubes with a dynamic programming method

RELATED WORK Our research builds on previous works in two ﬁelds: Spatial-temporal action localization. Gkioxari and Ma- lik [9] applied a two-stream R-CNN based framework to pro- duce frame level detections, and then linked the result to tubes with a dynamic programming method. Weinzaepfel et al. [4] extracted EdgeBoxes as the action proposals and then us...

work page
[3]

The architecture of our model is il- lustrated in Fig.1

MODEL DESCRIPTION Multi-stream model. The architecture of our model is il- lustrated in Fig.1. Our model consists of 4 streams: 2D and 3D RGB streams, 2D and 3D optical ﬂow streams. The con- ventional 2D RGB and optical ﬂow streams are employed to capture the short-term spatial-temporal features, meanwhile, 3D streams are added to learn long-term features...

work page
[4]

2: Details of the inﬂated 3D backbone

EXPERIMENTS To evaluate the performance of 3D SSD stream, we exam- ine different stream combinations and their detection accu- 2 Fig. 2: Details of the inﬂated 3D backbone. racy on the UCF101-24 dataset. Singh’s 2D SSD real-time framework [12] is used as a baseline. We keep their fusion and linking methods unchanged and focus on the performance improvemen...

work page
[5]

We present an empirical study of the properties of the combinations of 2D RGB, 2D OF, 3D RGB and 3D OF streams

CONCLUSIONS AND FUTURE PLANS This paper introduced a multi-stream action detector which achieves state-of-the-art results of the one-stage methods on UCF101-24 dataset. We present an empirical study of the properties of the combinations of 2D RGB, 2D OF, 3D RGB and 3D OF streams. Based on the results of those experi- ments, the following conclusions could...

work page
[6]

Ucf101: A dataset of 101 human actions classes from videos in the wild,

K. Soomro, A. R. Zamir, and M. Shah, “Ucf101: A dataset of 101 human actions classes from videos in the wild,” CoRR, vol. abs/1212.0402, 2012

arXiv 2012
[7]

Two-stream convo- lutional networks for action recognition in videos,

K. Simonyan and A. Zisserman, “Two-stream convo- lutional networks for action recognition in videos,” in Advances in Neural Information Processing Systems 27, Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, Eds., pp. 568–576. Curran Asso- ciates, Inc., 2014

work page 2014
[8]

Action recognition with im- proved trajectories,

H. Wang and C. Schmid, “Action recognition with im- proved trajectories,” in 2013 IEEE International Con- ference on Computer Vision, Dec 2013, pp. 3551–3558

work page 2013

Show all 26 references

[9]

Learn- ing to track for spatio-temporal action localization,

P. Weinzaepfel, Z. Harchaoui, and C. Schmid, “Learn- ing to track for spatio-temporal action localization,” in 2015 IEEE International Conference on Computer Vi- sion (ICCV), Dec 2015, pp. 3164–3172

2015
[10]

Large-scale video classiﬁca- tion with convolutional neural networks,

A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Suk- thankar, and L. Fei-Fei, “Large-scale video classiﬁca- tion with convolutional neural networks,” in 2014 IEEE Conference on Computer Vision and Pattern Recogni- tion, June 2014, pp. 1725–1732

2014
[11]

Quo vadis, action recog- nition? a new model and the kinetics dataset,

J. Carreira and A. Zisserman, “Quo vadis, action recog- nition? a new model and the kinetics dataset,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017, pp. 4724–4733

2017
[12]

Learning spatio-temporal representation with pseudo-3d residual networks,

Z. Qiu, T. Yao, and T. Mei, “Learning spatio-temporal representation with pseudo-3d residual networks,” in 2017 IEEE International Conference on Computer Vi- sion (ICCV), Oct 2017, pp. 5534–5542

2017
[13]

Multi-region two-stream R- CNN for action detection,

X. Peng and C. Schmid, “Multi-region two-stream R- CNN for action detection,” in ECCV - European Con- ference on Computer Vision , Amsterdam, Netherlands, Oct. 2016, vol. 9908 of Lecture Notes in Computer Sci- ence, pp. 744–759, Springer

2016
[14]

Finding action tubes,

G. Gkioxari and J. Malik, “Finding action tubes,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015, pp. 759–768

2015
[15]

Ava: A video dataset of spatio-temporally localized atomic visual ac- tions,

C. Gu, C. Sun, D. A. Ross, C. V ondrick, C. Panto- faru, Y . Li, S. Vijayanarasimhan, G. Toderici, S. Ricco, R. Sukthankar, C. Schmid, and J. Malik, “Ava: A video dataset of spatio-temporally localized atomic visual ac- tions,” in 2018 IEEE/CVF Conference on Computer Vision an...

2018
[16]

Deep learning for detecting multiple space-time action tubes in videos,

S. Saha, G. Singh, M. Sapienza, P. H. S. Torr, and F. Cuzzolin, “Deep learning for detecting multiple space-time action tubes in videos,” inProceedings of the British Machine Vision Conference 2016, BMVC 2016, York, UK, September 19-22, 2016

2016
[17]

Online real-time multiple spatiotemporal action localisation and prediction,

G. Singh, S. Saha, M. Sapienza, P. Torr, and F. Cuz- zolin, “Online real-time multiple spatiotemporal action localisation and prediction,” in2017 IEEE International Conference on Computer Vision (ICCV) , Oct 2017, pp. 3657–3666

2017
[18]

Action tubelet detector for spatio-temporal action localization,

V . Kalogeiton, P. Weinzaepfel, V . Ferrari, and C. Schmid, “Action tubelet detector for spatio-temporal action localization,” in2017 IEEE International Confer- ence on Computer Vision (ICCV) , Oct 2017, pp. 4415– 4423

2017
[19]

SSD: single shot multibox de- tector,

W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. E. Reed, C. Fu, and A. C. Berg, “SSD: single shot multibox de- tector,” in ECCV (1). 2016, vol. 9905 of Lecture Notes in Computer Science, pp. 21–37, Springer

2016
[20]

3d convolutional neural networks for human action recognition,

S. Ji, W. Xu, M. Yang, and K. Yu, “3d convolutional neural networks for human action recognition,” IEEE Transactions on Pattern Analysis and Machine Intelli- gence, vol. 35, no. 1, pp. 221–231, Jan 2013

2013
[21]

Learning spatiotemporal features with 3d convolutional networks,

D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, “Learning spatiotemporal features with 3d convolutional networks,” in 2015 IEEE International Conference on Computer Vision (ICCV) , Dec 2015, pp. 4489–4497

2015
[22]

An end-to-end 3d con- volutional neural network for action detection and seg- mentation in videos,

R. Hou, C. Chen, and M. Shah, “An end-to-end 3d con- volutional neural network for action detection and seg- mentation in videos,” arXiv preprint arXiv:1712.01111, 2017

2017 arXiv
[23]

High accuracy optical ﬂow estimation based on a the- ory for warping,

T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, “High accuracy optical ﬂow estimation based on a the- ory for warping,” in European Conf. on Computer Vi- sion, Prague, Czech Republic, 2004, vol. 3024, pp. 25– 36

2004
[24]

Con- volutional two-stream network fusion for video action recognition,

C. Feichtenhofer, A. Pinz, and A. Zisserman, “Con- volutional two-stream network fusion for video action recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016, pp. 1933–1941

2016
[25]

Tube convolutional neural network (t-cnn) for action detection in videos,

R. Hou, C. Chen, and M. Shah, “Tube convolutional neural network (t-cnn) for action detection in videos,” in 2017 IEEE International Conference on Computer Vi- sion (ICCV), Oct 2017, pp. 5823–5832

2017
[26]

YOLO9000: better, faster, stronger,

J. Redmon and A. Farhadi, “YOLO9000: better, faster, stronger,” in CVPR. 2017, pp. 6517–6525, IEEE Com- puter Society. 5

2017

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Multi-Stream Single Shot Spatial-Temporal Action Detection

INTRODUCTION The objective of action detection is to recognize and local- ize all the human action instances in a given video across both space and time. It is a fundamental task for video un- derstanding and important for practical applications such as video surveillance and human-robot interaction. Action de- tection is a challenging problem due to two ...

work page Pith review arXiv 2019

[2] [2]

Gkioxari and Ma- lik [9] applied a two-stream R-CNN based framework to pro- duce frame level detections, and then linked the result to tubes with a dynamic programming method

RELATED WORK Our research builds on previous works in two ﬁelds: Spatial-temporal action localization. Gkioxari and Ma- lik [9] applied a two-stream R-CNN based framework to pro- duce frame level detections, and then linked the result to tubes with a dynamic programming method. Weinzaepfel et al. [4] extracted EdgeBoxes as the action proposals and then us...

work page

[3] [3]

The architecture of our model is il- lustrated in Fig.1

MODEL DESCRIPTION Multi-stream model. The architecture of our model is il- lustrated in Fig.1. Our model consists of 4 streams: 2D and 3D RGB streams, 2D and 3D optical ﬂow streams. The con- ventional 2D RGB and optical ﬂow streams are employed to capture the short-term spatial-temporal features, meanwhile, 3D streams are added to learn long-term features...

work page

[4] [4]

2: Details of the inﬂated 3D backbone

EXPERIMENTS To evaluate the performance of 3D SSD stream, we exam- ine different stream combinations and their detection accu- 2 Fig. 2: Details of the inﬂated 3D backbone. racy on the UCF101-24 dataset. Singh’s 2D SSD real-time framework [12] is used as a baseline. We keep their fusion and linking methods unchanged and focus on the performance improvemen...

work page

[5] [5]

We present an empirical study of the properties of the combinations of 2D RGB, 2D OF, 3D RGB and 3D OF streams

CONCLUSIONS AND FUTURE PLANS This paper introduced a multi-stream action detector which achieves state-of-the-art results of the one-stage methods on UCF101-24 dataset. We present an empirical study of the properties of the combinations of 2D RGB, 2D OF, 3D RGB and 3D OF streams. Based on the results of those experi- ments, the following conclusions could...

work page

[6] [6]

Ucf101: A dataset of 101 human actions classes from videos in the wild,

K. Soomro, A. R. Zamir, and M. Shah, “Ucf101: A dataset of 101 human actions classes from videos in the wild,” CoRR, vol. abs/1212.0402, 2012

arXiv 2012

[7] [7]

Two-stream convo- lutional networks for action recognition in videos,

K. Simonyan and A. Zisserman, “Two-stream convo- lutional networks for action recognition in videos,” in Advances in Neural Information Processing Systems 27, Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, Eds., pp. 568–576. Curran Asso- ciates, Inc., 2014

work page 2014

[8] [8]

Action recognition with im- proved trajectories,

H. Wang and C. Schmid, “Action recognition with im- proved trajectories,” in 2013 IEEE International Con- ference on Computer Vision, Dec 2013, pp. 3551–3558

work page 2013