Pith. sign in

REVIEW 5 major objections 6 minor 21 references

What goes around comes around: Cycle-Consistency-based Short-Term Motion Prediction for Anomaly Detection using Generative Adversarial Networks

T0 review · 5 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read A single-frame GAN trained with cycle consistency detects surveillance anomalies at 98.0% AUC on UCSD Ped2, cutting the error of prior GAN-based methods by about 42.8%.

desk verdict Ablation shows cycle-consistency and morphological post-processing help on Ped2, but test-set-driven choices and missing sensitivity analysis keep the headline numbers from being definitive. read the letter →

arxiv 1908.03055 v1 pith:DE3HP4L5 submitted 2019-08-08 cs.CV

classification cs.CV
keywords anomalydetectionvideosurveillancegenerativeadversarialnetworkscycle-consistencylossopticalflowmorphologicalnoisesuppressionshort-termmotionpredictionUCSDPed2
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a GAN that predicts short-term motion from a single frame can spot anomalies in static surveillance video. Its answer is yes: translating a camera frame into an optical-flow map, and back, with a cycle-consistency loss that requires the round trip to reconstruct the input, produces much cleaner anomaly heat maps. Adding a morphological closing-and-opening step that deletes small isolated differences raises performance further. The final least-squares GAN configuration reaches 98.0% AUC on the UCSD Ped2 benchmark and cuts the anomaly-detection error of prior GAN-based methods by about 42.8%, while the same extensions improve two different GAN baselines by 2.0 and 2.4 percentage points.

What carries the argument

The central machinery is a pair of conditional GAN generators, $G_{A\to B}$ and $G_{B\to A}$, that map between the appearance domain (camera frames) and the motion domain (dense optical-flow maps represented as three-channel HSI images). The generators are trained with the pix2pix objective (conditional adversarial loss plus an L1 pixel loss) and, in the extended model, a cycle-consistency loss $L_{\mathrm{cyc}} = \mathbb{E}_a[\|G_{B\to A}(G_{A\to B}(a)) - a\|_1]$ that enforces round-trip reconstruction. At inference the predicted flow and predicted frame are compared with the real optical flow and the real frame; differences are computed in the feature space of VGG-16 layer conv3-3, squared, and summed into a heat map. The anomaly score is the heat map's root mean square. A morphological closing followed by opening with a $7\times 7$ all-ones kernel suppresses small noise regions, under the assumption that anomalies occupy larger connected areas than prediction noise.

What would settle it

Take the final LSGAN cycle-consistent model and run it on anomaly types whose heat-map signatures are small or fragmented—for example, a small distant object, a briefly visible anomaly, or an anomaly occupying only a few pixels—comparing AUC with and without the morphological opening. If the opening removes the anomaly's region or lowers AUC on those cases, the spatial-size assumption fails and the noise-suppression gain is limited to anomalies with large heat-map footprints.

Watch

Extended reading notes

Core claim

The paper's claim is that the key to better GAN-based anomaly detection is not more temporal context but a more faithful motion model. Their cross-channel generator learns to transfer appearance into motion and motion into appearance; adding a cycle-consistency loss—the L1 distance between an input and its round-trip reconstruction—removes much of the spurious motion prediction at pedestrian extremities that made normal frames look anomalous. Morphological noise suppression then removes residual small-area differences from the heat map. The result is that the best model, an LSGAN with cycle consistency and noise suppression, detects anomalies at 98.0% AUC on UCSD Ped2, outperforming GAN baselines that use longer input sequences and reducing the error of prior GAN-based methods by about 42.8%.

Load-bearing premise

The noise-suppression step assumes true anomalies produce spatially larger connected regions of difference in the heat map than ordinary prediction noise, so morphological opening with a 7x7 kernel removes the noise without removing the anomaly.

Editorial extensions

If this is right

  • If the central claim holds, a single input frame plus its optical flow is enough for competitive anomaly detection on static cameras, so methods do not need multi-frame stacks or recurrent temporal layers to reach state-of-the-art GAN results.
  • The cycle-consistency extension and the noise-suppression step each improve both a VanillaGAN and an LSGAN baseline, so the recipe transfers across GAN loss choices.
  • Anomaly detection quality depends strongly on which optical-flow estimator supplies motion ground truth; the estimator that preserves high-frequency discontinuities outperforms a smooth one and a deep-network one on this benchmark.
  • At roughly 40 ms per timestep, the resulting detector runs faster than real time at typical surveillance frame rates, which supports deployment on static CCTV setups.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The cycle-consistency loss is tested only inside the cross-channel architecture; a natural extension is to add the same round-trip constraint to other predictive anomaly detectors, such as future-frame prediction networks, where normal motion errors are also a known source of false positives.
  • Because the noise-suppression step is a fixed 7x7 morphological opening, a learned denoiser trained on normal-scene heat maps could replace it and potentially remove the spatial-size assumption while keeping the gain.
  • The error-reduction figure of 42.8% is measured against prior GAN-based methods on Ped2; whether it holds on Ped1, on anomalies with small or fragmented footprints, or on non-static cameras is left open by the paper.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes a GAN-based anomaly detection method for static surveillance cameras, building on the cross-channel appearance-to-motion framework of Ravanbakhsh et al. It adds a cycle-consistency loss during training and a morphological closing/opening post-processing step at inference to suppress prediction noise. The authors evaluate three optical flow algorithms (Brox, Farneback, FlowNet2), several VGG-16 feature layers for semantic difference computation, and two GAN losses (VanillaGAN and LSGAN) on the UCSD Ped2 dataset. The reported best model achieves 98.0% AUC and is claimed to reduce the anomaly detection error of GAN-based methods by about 42.8%, with the cycle-consistency and noise-suppression extensions improving the two tested baselines by 2.0 and 2.4 percentage points, respectively.

Significance. If the empirical claims were fully supported, the paper would make a modest contribution to the video anomaly detection literature: it demonstrates that cycle-consistency can reduce false positives in cross-channel GAN prediction and that a simple post-processing step can improve AUC on a standard benchmark. The paper is generally well organized, the ablation structure is clear, and the use of a public dataset is a strength. However, the central quantitative claims rest on a single test set used for model selection, single training runs without error bars, and a noise-suppression mechanism whose underlying assumption is in tension with observations reported in the paper itself. The claimed 42.8% error reduction is also not representative of the four GAN-based comparators listed in Table 4, but rather matches improvement over a single method. These issues are load-bearing for the stated contributions.

major comments (5)
  1. [Tables 1 and 2; Sections 3.1 and 4.3] The VGG-16 feature layer (Table 1) and the optical flow algorithm (Table 2) are both selected by comparing AUC on the UCSD Ped2 test set, and the final comparison in Table 3 and Table 4 is reported on the same test set. Because no held-out validation split or nested evaluation is used, the reported 98.0% AUC and the claimed gains of the extensions incorporate selection bias and do not estimate true out-of-sample performance. Please add a validation split, cross-validation, or an explicit protocol in which all configuration choices are fixed before the final test evaluation.
  2. [Table 3; Section 4.4] The paper reports single AUC values for each configuration with no multiple seeds, confidence intervals, or statistical tests. The increments attributed to noise suppression are only 0.9 and 0.4 percentage points for VanillaGAN and LSGAN, respectively, and the cycle-consistency gains are 1.1 and 2.2 percentage points. GAN training is stochastic, so differences of this size can easily fall within run-to-run variance. Without repeated runs and a measure of dispersion, the central claim that the extensions improve performance is not statistically established.
  3. [Sections 3.3, 4.2, and 4.3] The noise-suppression step binarizes the heat map by clipping every Δij > 0 to 1 and then applies closing and opening with a 7×7 all-ones structuring element. The kernel size and threshold are indeed stated in the manuscript, but the assumption in Section 3.3 is that anomalies occupy spatially larger regions than prediction noise. This assumption is not directly supported: the end of Section 4.3 observes that normal-object noise in frame-domain heat maps is distributed over large areas of small differences, which is not cleanly separated from the 'large region' criterion. No connected-component size statistics, kernel-size sensitivity analysis, or per-anomaly event evaluation is provided. The 0.4–0.9 percentage point gains from noise suppression therefore remain untested with respect to the mechanism claimed.
  4. [Abstract and Section 5] The statement that the final method 'reduced the anomaly detection error for GAN-based methods by about 42.8%' is misleading. The value 42.8% corresponds exactly to the error reduction relative to Lee et al. (96.5% AUC, i.e., 3.5% error, versus 2.0% error for the proposed method) in Table 4, not to an average over the GAN-based methods listed there. For example, relative to Ravanbakhsh et al. [17] (93.5% AUC), the error reduction is about 30.8%. The reference comparator should be stated explicitly and the generalized phrasing removed or justified.
  5. [Section 5 and Table 3] The conclusion says the extensions improve the two baseline setups by 2.0% and 2.4%, respectively. According to Table 3, the LSGAN improvement from baseline to the full cycle-consistency-plus-noise-suppression model is 98.0 − 95.4 = 2.6 percentage points, while cycle-consistency alone gives 2.2 percentage points. The 2.4% figure does not match any value in Table 3. Please clarify which comparison is intended and correct the inconsistency.
minor comments (6)
  1. [Abstract and throughout] There are several grammatical and typographical errors, including 'With these extension' in the abstract, 'The strength of GANs lay' in Section 4.5, and 'imate-to-optical-flow' in Section 4.4. These should be corrected.
  2. [Table 1] The VGG layer evaluation in Table 1 is performed only with the generator GB→A (flow-to-frame direction), but the final model uses heat maps from both translation directions. The transferability of the conv(3-3) choice to the fused detection pipeline is not justified, and the spread of AUC values across layers (57.0–78.5) suggests the choice may be sensitive to noise.
  3. [Section 4.2] Implementation details omit several hyperparameters that are needed for reproducibility, including learning rate, batch size, the values of λL1, λcyc, and λh, the dropout behavior, and the hardware environment used for the 40 ms runtime claim.
  4. [Section 4.3] The observation that fusion 'does not yield a further improvement' is based on a single AUC comparison without error bars; the statement that fusion builds 'approximately the mean' of the two ROC curves is qualitative and should be supported by point-wise ROC statistics or removed.
  5. [Section 4.4] The sentence 'the noise suppression has no significant drawbacks' is not supported by any per-event or per-anomaly analysis; an aggregate AUC value alone cannot establish the absence of drawbacks for specific anomaly types.
  6. [Section 5] The statement that the method outperforms 'other state-of-the-art methods based on GANs' is too strong, since Table 4 compares only four earlier GAN-based methods from 2017 and 2018 and omits more recent anomaly detection approaches.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: all quantitative claims are empirical benchmark results; no step reduces to its own inputs or relies on a load-bearing self-citation.

full rationale

The paper makes no derivation claim that reduces to its inputs. The method explicitly adapts the cross-channel GAN of Ravanbakhsh et al. [17] and the cycle-consistency loss of Zhu et al. [21]; these are external, acknowledged baselines, not the authors' own earlier work, so there is no self-citation chain carrying the load. The reported improvements (cycle-consistency and morphological noise suppression) are measured AUC values on UCSD Ped2 (Tables 3 and 4), not quantities that are equal to the loss terms or to the morphological operation by construction. Section 3.3's assumption that anomalies occupy spatially larger connected heat-map regions is an empirical premise for the post-processing; it is not used to define the anomaly score (Eq. 5) nor to force the measured gain, and the paper does not claim it as a theorem. The choice of VGG layer, optical-flow method, and kernel size is made on the same benchmark, which is a methodological/test-set reuse concern, but it does not constitute definitional circularity, fitted-input-called-prediction, or author-imported uniqueness. Therefore no circular step is exhibited, and the score is 0.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central claim is an empirical performance gain, so the ledger captures the hyperparameters and modeling choices that were fit to the Ped2 benchmark. The paper does not introduce new theoretical entities or physical quantities.

free parameters (7)
  • Cycle-consistency loss weight lambda_cyc = not reported
    Added to the GAN and L1 losses in Eqs. 7-8; the value is not given in the paper and would need tuning to reproduce the reported AUC.
  • L1 loss weight lambda_L1 = not reported
    Weight of the pixel-wise L1 term in Eqs. 7-8; not specified.
  • Fusion weight lambda_h = not reported
    Weight for combining normalized heat maps from the two translation directions in Section 3.1; not specified.
  • Morphological kernel size = 7x7
    Kernel size for closing and opening in Section 4.2, chosen without a reported sweep; results may depend on it.
  • VGG-16 feature layer = conv (3-3)
    Selected in Table 1 as the best of 13 layers by test-set AUC on Ped2; this is a selection on the benchmark.
  • Optical flow algorithm = Brox et al. [1]
    Selected in Table 2 as best of three methods by test-set AUC; the paper's improvements are reported only for this flow input.
  • Training epochs = 10
    Fixed without reported ablation; early stopping or longer training could change results.
assumptions (5)
  • domain assumption Anomalies in surveillance produce spatially larger heat-map difference regions than noise, so opening removes noise without removing true anomalies.
    Stated in Section 3.3 and it is the load-bearing premise for the morphological noise-suppression contribution.
  • domain assumption Optical flow maps encoded as 3-channel HSI images retain sufficient information for a GAN to translate between appearance and motion domains.
    Used in Section 3.1 to define the cross-channel transfer; if flow visualization loses information, the whole training signal is degraded.
  • domain assumption The training set distribution of normal frames and flows lets a GAN learn a faithful model; anomalies are out-of-distribution and will be poorly translated.
    Standard assumption for GAN-based anomaly detection, invoked in Sections 1 and 3.1.
  • standard math Adversarial training with the stated losses converges to a useful generator and discriminator equilibrium.
    Standard ML assumption underlying all GAN experiments; not proven for this setup.
  • domain assumption Pretrained VGG-16 features trained on ImageNet provide a meaningful semantic difference measure for unseen surveillance frames.
    Used in Section 3.1 for the semantic anomaly score; no fine-tuning or domain adaptation is described.

how reviews work

0 comments
Cite this review

Pith. "Pith review of What goes around comes around: Cycle-Consistency-based Short-Term Motion Prediction for Anomaly Detection using Generative Adversarial Networks." pith.science (2026). https://pith.science/paper/DE3HP4L5

@misc{pith2026190803055,
  author       = {Pith},
  title        = {Pith review of: What goes around comes around: Cycle-Consistency-based Short-Term Motion Prediction for Anomaly Detection using Generative Adversarial Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DE3HP4L5}},
  note         = {Machine review of arXiv:1908.03055}
}
read the original abstract

Anomaly detection plays in many fields of research, along with the strongly related task of outlier detection, a very important role. Especially within the context of the automated analysis of video material recorded by surveillance cameras, abnormal situations can be of very different nature. For this purpose this work investigates Generative-Adversarial-Network-based methods (GAN) for anomaly detection related to surveillance applications. The focus is on the usage of static camera setups, since this kind of camera is one of the most often used and belongs to the lower price segment. In order to address this task, multiple subtasks are evaluated, including the influence of existing optical flow methods for the incorporation of short-term temporal information, different forms of network setups and losses for GANs, and the use of morphological operations for further performance improvement. With these extension we achieved up to 2.4% better results. Furthermore, the final method reduced the anomaly detection error for GAN-based methods by about 42.8%.

Figures

Figures reproduced from arXiv: 1908.03055 by the authors.

Figure 1
Figure 1. Examples for a normal and abnormal situa￾tion represented in the Ped2 [13] dataset. The image on the left-hand side is considered as normal since only pedes￾trians occur. On the right-hand side the occurrence of a ve￾hicle is considered as anomaly. to keep an eye on every single camera. This leads to the de￾mand for automatic systems that assist people in such data intensive situations. However, most systems that de… view at source ↗
Figure 2
Figure 2. Schematic overview of our GAN training. During the training process of our GAN approach two generators are trained. This schematic highlights the direction from appearance to motion, where the generators GA→B and GB→A are trained one for each transformation direction between motion and appearance domain. The procedure for the direction from motion to appearance (see faded part) is the same. only with real frames and… view at source ↗
Figure 3
Figure 3. Schematic of the anomaly detection workflow during inference. Given a video frame Ft, the generator predicts dense optical flow Oet which is compared to the actual optical flow Ot. The result is a heat map ∆t. After some processing (cf. Section 3.3) ∆t is refined in order to eliminate interfering noise. Based on the resulting value of the anomaly score function α(∆∗ t ) the frame Ft is finally categorized as normal … view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Visualized flow maps of different calculation methods. The flow map calculated according to [1] is visualized on the left-hand side, the one according to [2] in the middle and the one according to [6] on the right-hand side. Color intensities were adjusted to improve t…
Figure 5
Figure 5. Figure 5: Heat maps for baseline and extensions. In the first row, the heat map of the baseline model is visu￾alized. The second row contains the heat map of the same scene computed by the model with cycle-consistency exten￾sion. The third row contains the heat map of the model …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 17 canonical work pages

  1. [17]

    Abnormal Event Detection in Videos using Generative Adversarial Nets

    M. Ravanbakhsh, M. Nabi, E. Sangineto, L. Marcenaro, C. S. Regazzoni, and N. Sebe. Abnormal event detection in videos using generative adversarial nets. CoRR, abs/1708.09644,

  2. [1]

    T. Brox, A. Bruhn, N. Papenberg, and J. Weickert. High ac- curacy optical flow estimation based on a theory for warp- ing. In T. Pajdla and J. Matas, editors, Computer Vi- sion - ECCV 2004 , pages 25–36, Berlin, Heidelberg, 2004. Springer Berlin Heidelberg. 5, 6

  3. [2]

    Farneback

    G. Farneback. Two-frame motion estimation based on poly- nomial expansion. In J. Bigun and T. Gustavsson, editors, Image Analysis, pages 363–370, Berlin, Heidelberg, 2003. Springer Berlin Heidelberg. 5, 6

  4. [3]

    Goodfellow, J

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y . Bengio. Gen- erative adversarial nets. In Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, edi- tors, Advances in Neural Information Processing Systems 27, pages 2672–2680. Curran Associates, Inc., 2014. 3

  5. [4]

    Hasan, J

    M. Hasan, J. Choi, J. Neumann, A. K. Roy-Chowdhury, and L. S. Davis. Learning temporal regularity in video se- quences. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016. 2

  6. [5]

    X. Hu, S. Hu, Y . Huang, H. Zhang, and H. Wu. Video anomaly detection using deep incremental slow feature anal- ysis network. IET Computer Vision, 10(4):258–265, 2016. 2

  7. [6]

    E. Ilg, N. Mayer, T. Saikia, M. Keuper, A. Dosovitskiy, and T. Brox. Flownet 2.0: Evolution of optical flow estimation with deep networks. In IEEE Conference on Computer Vi- sion and Pattern Recognition (CVPR), 2017. 5, 6, 8

  8. [7]

    Isola, J

    P. Isola, J. Zhu, T. Zhou, and A. A. Efros. Image-to-image translation with conditional adversarial networks. In 2017 IEEE Conference on Computer Vision and Pattern Recog- nition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017 , pages 5967–5976, 2017. 3, 4, 5

Show all 21 references
  1. [8]

    D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In ICLR, 2015. 5

  2. [9]

    B. R. Kiran, D. M. Thomas, and R. Parakkal. An overview of deep learning based methods for unsupervised and semi- supervised anomaly detection in videos. Journal of Imaging, 4(2), 2018. 2

  3. [10]

    Krizhevsky, I

    A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 25, pages 1097–1105. Curran Associates, In...

  4. [11]

    S. Lee, H. G. Kim, and Y . M. Ro. Stan: Spatio- temporal ad- versarial networks for abnormal event detection. 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1323–1327, 2018. 2, 4, 7

  5. [12]

    W. Liu, W. Luo, D. Lian, and S. Gao. Future frame pre- diction for anomaly detection – a new baseline. In 2018 IEEE Conference on Computer Vision and Pattern Recog- nition (CVPR), 2018. 2, 4, 5, 7

  6. [13]

    Mahadevan, W

    V . Mahadevan, W. Li, V . Bhalodia, and N. Vasconcelos. Anomaly detection in crowded scenes. In 2010 IEEE Com- puter Society Conference on Computer Vision and Pattern Recognition, pages 1975–1981, June 2010. 1, 5

  7. [14]

    X. Mao, Q. Li, H. Xie, R. Y . Lau, Z. Wang, and S. Paul Smol- ley. Least squares generative adversarial networks. In The IEEE International Conference on Computer Vision (ICCV), Oct 2017. 3

  8. [15]

    Mathieu, C

    M. Mathieu, C. Couprie, and Y . LeCun. Deep multi- scale video prediction beyond mean square error. CoRR, abs/1511.05440, 2015. 2

  9. [16]

    J. R. Medel and A. E. Savakis. Anomaly detection in video using predictive convolutional long short-term memory net- works. CoRR, abs/1612.00390, 2016. 2

  10. [18]

    Ravanbakhsh, E

    M. Ravanbakhsh, E. Sangineto, M. Nabi, and N. Sebe. Train- ing adversarial discriminators for cross-channel abnormal event detection in crowds. CoRR, abs/1706.07680, 2017. 2, 4, 7

  11. [19]

    Simonyan and A

    K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015. 4

  12. [20]

    D. Xu, Y . Yan, E. Ricci, and N. Sebe. Detecting anomalous events in videos by learning deep representations of appear- ance and motion. Computer Vision and Image Understand- ing, 156:117 – 127, 2017. Image and Video Understanding in Big Data. 2

  13. [21]

    J. Zhu, T. Park, P. Isola, and A. A. Efros. Unpaired image- to-image translation using cycle-consistent adversarial net- works. CoRR, abs/1703.10593, 2017. 4, 5

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.