Pith. sign in

REVIEW 4 major objections 3 minor 26 references

Joint Training of Image Generator and Detector for Road Defect Detection

T0 review · 4 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Jointly training an image generator with a detector beats prior road-defect detection while using under 20% of the parameters.

desk verdict The claimed joint-training mechanism isn't what the experiments run; the practical result may still be useful, but the paper's key novelty is unverified. read the letter →

arxiv 2509.03465 v1 pith:RYIFS6QB submitted 2025-09-03 cs.CV

Kuan-Chuan Peng This is my paper
classification cs.CV
keywords roaddefectdetectiongenerativedataaugmentationjointtraininghardexamplesynthesisdualdiscriminatorsCLIP-basedFIDlossRDD2022edgedeployment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes JTGD, a method that jointly trains an image generator and a defect detector so the generator learns to synthesize harder, more realistic road-defect images that improve the detector's generalization. The key idea is to let the detector's own losses on real and synthetic images feed back into the generator's objective, adding a hard-example synthesis loss on top of a dual-discriminator GAN and a CLIP-based Fréchet Inception Distance loss. On the RDD2022 benchmark using the CRDDC 2022 held-out test server, JTGD reaches an overall F1 of 63.13 with a 49M-parameter InternImage-T detector, beating the 59.20 of the Faster Swin baseline at 253M parameters, without any ensemble or test-time augmentation. Each added component—the generator, the FID loss, and the hard-example loss—contributes a small additive gain, and the method uses under 20% of the baseline's test-time parameters, making it suited to edge deployment.

What carries the argument

The load-bearing object is the hard-example synthesis loss Lh = -Ldf - Ldr, which couples the generator to the detector: Ldf and Ldr are the same detection loss applied to synthetic and real images, and minimizing their negative sum pushes the generator to create defects that degrade the detector's current performance. This signal is surrounded by two quality-enforcing mechanisms: dual discriminators (image-level Di and patch-level Dp) that keep both the whole image and the defect patch plausible, and a CLIP-feature Fréchet Inception Distance loss that keeps the synthetic distribution close to the real one. The generator is a modified CycleGAN that turns a clean road image plus a mask into a

What would settle it

Train JTGD with the hard-example loss replaced by its opposite sign (maximizing Ldf + Ldr), or a variant using only Ldf without Ldr, and measure F1 on RDD2022. If the opposite-sign or one-sided variant yields the same F1 improvement, or if visual inspection of the generator's outputs under Lh shows artifacts or unlearnable patterns, the claim that harder-but-realistic synthesis drives the gain would be falsified. A controlled comparison against a separately-trained generator with equal FID and the same augmentation budget would isolate the joint-training effect.

Watch

Extended reading notes

Core claim

The central claim is that road-defect detection can be improved more efficiently by making the data-augmentation generator aware of the detector's objective. Rather than training a GAN once and freezing it, JTGD optimizes the generator with three objectives: fooling an image discriminator and a patch discriminator, matching the CLIP-feature statistics of real and synthetic defect images, and producing images the current detector fails to detect (hard-example synthesis loss, Lh = -Ldf - Ldr). The generator is conditioned on a defect mask and a clean road image, so the detector's losses provide dense supervision for where synthetic defects are placed. Retraining the detector on real plus synth

Load-bearing premise

The load-bearing premise is that minimizing Lh = -Ldf - Ldr forces the generator to synthesize defects that are harder but still realistic and useful; if the hard examples become degenerate or unlearnable artifacts, the joint-training gain would not be attributable to the paper's mechanism.

Editorial extensions

If this is right

  • Without ensembles or test-time augmentation, JTGD can run on edge hardware with a 49M-parameter detector, so the same accuracy gain is available in deployment.
  • The dual-discriminator and CLIP-FID losses are backbone-agnostic, so they can improve data augmentation for other detection tasks that suffer from scarce defect examples.
  • The ablation shows each component adds a positive increment (generator +1.05, FID +0.42, hard-example +0.19 F1), suggesting the design is modular and extensible.
  • Because the generator is conditioned on a mask specifying defect type and location, practitioners can control which defect classes get augmented, potentially addressing class imbalance.
  • The method reduces test-time parameters to under 20% of the state of the art while improving accuracy, shifting the trade-off toward lightweight deployment.
  • The author states the dual-discriminator design and CLIP-based FID loss can serve as general enhancements for detection tasks beyond road defects.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: The country-wise gains vary widely (Norway +12.6 F1, USA -0.5), so the benefit likely depends on how much the hard-example signal aligns with the test distribution; a stratified analysis by defect type and country could reveal which conditions the joint loss helps.
  • Inference: Since E2 (InternImage-T without generator) already outperforms Faster Swin by +2.27 F1, part of the headline gain comes from the backbone change; the marginal contribution of joint training over a separately-trained generator with the same backbone may be smaller than the E5-E1 gap suggests.
  • Inference: The hard-example loss minimizes the negative sum of detection losses, which could also be interpreted as an adversarial regularization; comparing against an alternative that maximizes classifier uncertainty or mines hard examples from real data would test whether the generative component is essential.
  • Inference: The CLIP-based FID loss may be less effective for defect types whose appearance is far from natural image semantics; a synthetic-to-real gap in crack textures could be measured by FID per defect class.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper proposes JTGD, a method for road defect detection without ensemble or test-time augmentation. A CycleGAN-style generator with dual (image and patch) discriminators is augmented with a CLIP-based FID loss and a hard-example synthesis loss Lh = -Ldf - Ldr, intended to let the generator produce synthetic defect images that challenge the detector. On the RDD2022/CRDDC2022 benchmark, JTGD (InternImage-T, 49M parameters) reports 63.13 overall F1 versus 59.20 for Faster Swin (253M parameters), with ablations (E2-E5) intended to show the contribution of each component. The central claim is that joint generator-detector training, improved synthetic image quality, and data augmentation produce the performance gain.

Significance. If the joint-training mechanism were actually exercised by the experiments, this would be a useful contribution: it demonstrates a lightweight detector that outperforms a much larger state-of-the-art system under the no-ensemble/no-TTA constraint, with numbers reported on the held-out challenge server. The parameter-efficiency comparison is clear and the CLIP-FID and dual-discriminator ideas are worth investigating. However, the manuscript as written does not establish the central mechanism: the experimental section describes a sequential pipeline that cannot use Lh, and the only isolation of Lh is a 0.19 F1 gap with no error bars. No code or checkpoints are provided, so the reproducibility of the benchmark results and the actual training schedule cannot be independently checked.

major comments (4)
  1. [Sec. 2 (Eq. 3) vs. Sec. 3] The training protocol described in Sec. 3 is sequential, not joint: the generator is trained 'from scratch with dual discriminators and LFID until convergence' before the detector is pretrained and finetuned with synthesized images. Under this schedule, Ldf and Ldr in Eq. (3) are never computed during generator training, so Lh cannot influence the generator. The +0.19 F1 gain of E5 over E4 in Table 2 therefore cannot be attributed to the proposed hard-example loss. The authors must supply the exact joint-training algorithm, including update timing and how detector gradients reach the generator, or remove this claim from the contributions.
  2. [Eq. (3)] Even if a joint update schedule were used, Lh contains -Ldr, whose gradient with respect to the generator parameters is zero because Ldr is computed on fixed real images. Only -Ldf can affect the generator. The paper does not acknowledge this inert term; as written, the hard-example loss is at best only partially effective. The authors should reformulate or explicitly state which terms actually backpropagate to the generator.
  3. [Table 2] The paper reports a single F1 value per configuration, with no repeated runs, error bars, or code. The only experiment isolating Lh (E5 vs. E4) is a difference of +0.19 F1, which is small relative to likely seed/ordering variance. Given the schedule contradiction in Sec. 3, this is insufficient evidence for the central joint-training claim. Additional seeds or a controlled experimental protocol are needed before this can be accepted.
  4. [Table 2 / Sec. 2] The 'dual discriminators' are listed as a contribution, but the ablation does not isolate them: every experiment that includes the generator (E3-E5) uses both Di and Dp. There is no condition with the generator and only the patch discriminator. The paper should either provide such an ablation or temper the claim that the image discriminator is individually responsible for the reported gains.
minor comments (3)
  1. [Throughout] There are several typos and formatting issues: 'T raining', 'T able', 'Apendix', 'parm.', and inconsistent reference to 'Table 1' in Sec. 4 where Table 2 is meant (the sentence 'E5 versus E4 in Table 1' should be Table 2).
  2. [Sec. 2 / Appendix A4] The proposed loss is called 'CLIP-based Frechet Inception Distance loss' even though the features are CLIP, not Inception. This is confusing; consider naming it 'CLIP-FID loss' and clarifying the modification from the standard ImageNet-based FID.
  3. [Sec. 3 / Appendix A3] The generator and detector are both said to be trained 'until convergence,' but no stopping criteria or epoch counts are given. The confidence threshold is chosen on validation data sampled from the training set; details of this split and the threshold search are missing.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity: held-out F1 results are not construction-equivalent to inputs. One minor self-citation is present, and a Sec. 2 vs Sec. 3 implementation contradiction is flagged as a correctness concern, not a circularity.

full rationale

The paper's central F1 numbers come from the CRDDC 2022 challenge server on a held-out test set; they are measured, not derived from fitted constants. The hard-example loss Lh (Eq. 3) is a training objective, and the detector is evaluated on unseen data, so the reported gains are not definitionally forced. The dual-discriminator and CLIP-FID losses are architectural/loss choices, not redefinitions of the evaluation metric. The only self-citation is Ref. [18] (author's own work), cited for the practical motivation that edge devices have limited resources; it is not load-bearing for the method or results. A real concern appears in the text: Sec. 2 claims joint training with Lh, while Sec. 3 describes a sequential pipeline (generator trained to convergence with dual discriminators and LFID, then detector finetuned), which would make Lh ineffective and the E5-vs-E4 +0.19 F1 gap unexplained. This is an implementation/validity inconsistency, not a circular reduction of outputs to inputs, and does not raise the circularity score beyond the minor self-citation level.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical entities, particles, forces, or conserved quantities. The central empirical claim rests on hand-set loss weights, random synthesis parameters, and assumptions that CLIP-FID optimization and detector-feedback hard examples transfer to held-out test performance.

free parameters (4)
  • wh (hard-example loss weight) = 1
    Appendix A3 states 'We empirically set wh = 1 and wFID = 0.1 such that the ranges of Lh and LFID are comparable'; no sensitivity analysis is reported.
  • wFID (CLIP-FID loss weight) = 0.1
    Appendix A3 states the weight was empirically set; no sweep or validation curve is provided.
  • detector confidence threshold = not specified
    Appendix A3 says the threshold is chosen via validation data; threshold selection is a free choice that directly affects F1.
  • synthetic data augmentation amount and schedule = not specified
    Appendix A3 says the detector is finetuned with real and synthesized data until convergence, but does not state how many synthetic images are added or their sampling schedule.
assumptions (4)
  • domain assumption Minimizing CLIP-feature FID improves perceptual quality and downstream detector performance.
    Sec. 2 and Appendix A4 justify LFID by citing [13,16]; the link between a lower FID on the synthesized set and better detection is shown only qualitatively and on one run.
  • domain assumption Randomly placed defect boxes in YOLOP drivable areas form a useful training distribution for real road defects.
    Appendix A2 describes random locations, random aspect ratios, random sizes, and random defect types; no attempt is made to match real defect statistics beyond staying inside drivable areas.
  • domain assumption Hard-example synthesis via Lh = -Ldf - Ldr improves a retrained detector rather than producing degenerate images.
    Sec. 2 after Eq. 3 states this as an expectation; no analysis of the generated hard examples or training stability is provided.
  • domain assumption CycleGAN defaults and the released InternImage pretrained checkpoint transfer to the mask-conditioned road defect synthesis task.
    Sec. 3 says the authors follow default settings of InternImage and CycleGAN unless otherwise specified, but the mask-conditioned input is a nontrivial adaptation that is not validated against alternatives.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Joint Training of Image Generator and Detector for Road Defect Detection." pith.science (2026). https://pith.science/paper/RYIFS6QB

@misc{pith2026250903465,
  author       = {Pith},
  title        = {Pith review of: Joint Training of Image Generator and Detector for Road Defect Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RYIFS6QB}},
  note         = {Machine review of arXiv:2509.03465}
}
read the original abstract

Road defect detection is important for road authorities to reduce the vehicle damage caused by road defects. Considering the practical scenarios where the defect detectors are typically deployed on edge devices with limited memory and computational resource, we aim at performing road defect detection without using ensemble-based methods or test-time augmentation (TTA). To this end, we propose to Jointly Train the image Generator and Detector for road defect detection (dubbed as JTGD). We design the dual discriminators for the generative model to enforce both the synthesized defect patches and overall images to look plausible. The synthesized image quality is improved by our proposed CLIP-based Fr\'echet Inception Distance loss. The generative model in JTGD is trained jointly with the detector to encourage the generative model to synthesize harder examples for the detector. Since harder synthesized images of better quality caused by the aforesaid design are used in the data augmentation, JTGD outperforms the state-of-the-art method in the RDD2022 road defect detection benchmark across various countries under the condition of no ensemble and TTA. JTGD only uses less than 20% of the number of parameters compared with the competing baseline, which makes it more suitable for deployment on edge devices in practice.

Figures

Figures reproduced from arXiv: 2509.03465 by the authors.

Figure 1
Figure 1. The overview of our method JTGD, where we propose to use the dual discriminators (Di and Dp) and the Frechet inception ´ distance loss LFID computed using the CLIP features (frozen) to im￾prove the quality of the images synthesized by the generator ψ for data augmentation when training the defect detector D. We train ψ and D jointly such that ψ can learn to synthesize harder examples based on D’s detection results, … view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

26 extracted references · 24 canonical work pages

  1. [1]

    Internimage pre-trained model.https://tinyurl.com/ 2p825mt6. 3

    work page

  2. [2]

    https : / / tinyurl

    News from the Federal. https : / / tinyurl . com / r39bdj8a, . 1

    work page

  3. [3]

    AAA Newsroom.https://tinyurl.com/ys9mn8b4, . 1

    work page

  4. [4]

    https://pytorch.org/

    PyT orch. https://pytorch.org/. 1

    work page

  5. [5]

    D. Arya, H. Maeda, S. K. Ghosh, D. T oshniwal, H. Omata, T . Kashiyama, and Y . Sekimoto. Crowdsensing-based road damage detection challenge (CRDDC-2022). arXiv preprint arXiv:2211.11362, 2022. 1, 3, 4

    work page Pith review arXiv 2022

  6. [6]

    D. Arya, H. Maeda, S. K. Ghosh, D. T oshniwal, and Y . Sekimoto. RDD2022: A multi-national image dataset for automatic road damage detection.arXiv preprint arXiv:2209.08538, 2022. 2, 3, 1

    arXiv 2022

  7. [7]

    Bhavsar, A

    M. Bhavsar, A. Alfarrarjeh, U. Baranwal, and S. H. Kim. Country-specific ensemble learning: A deep learning approach for road damage detection. In IEEE International Conference on Big Data (Big Data), 2022. 1

    work page 2022

  8. [8]

    Chakraborty, U

    T . Chakraborty, U. R. K S, S. M. Naik, M. Panja, and B. Manvitha. Ten years of generative adversarial nets (GANs): A survey of the state-of-the-art.Machine Learning: Science and T echnology, 5(1), 2024. 1

    work page 2024

Show all 26 references
  1. [9]

    W . Ding, X. Zhao, B. Zhu, Y . Du, G. Zhu, T . Y u, L. Li, and J. W ang. An ensemble of one-stage and two-stage detectors approach for road damage detection. In IEEE International Conference on Big Data (Big Data), 2022. 1, 3, 4

    2022

  2. [10]

    Williams C

    V an Gool L. Williams C. K. I. Winn J. Everingham, M. and A. Zisserman. The P ASCAL visual object classes (VOC) challenge. IJCV, 2010. 3

    2010

  3. [11]

    Heusel, H

    M. Heusel, H. Ramsauer, T . Unterthiner, B. Nessler, and S. Hochreiter. GANs trained by a two time-scale update rule converge to a local nash equilibrium. InNIPS, 2017. 2

    2017

  4. [12]

    Jeong and J

    D. Jeong and J. Kim. Road damage detection using yolo with image tiling about multi-source images. InIEEE International Conference on Big Data (Big Data), 2022. 1

    2022

  5. [13]

    Kynk¨a¨anniemi, T

    T . Kynk¨a¨anniemi, T . Karras, M. Aittala, T . Aila, and J. Lehtinen. The role of imagenet classes in Fr´echet inception distance. In ICLR, 2023. 3

    2023

  6. [14]

    T .-Y . Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P . Perona, D. Ramanan, C. L. Zitnick, and P . Doll´ar. Microsoft COCO: Common objects in context. InECCV, 2014. 3

    2014

  7. [15]

    Maeda, T

    H. Maeda, T . Kashiyama, Y . Sekimoto, T . Seto, and H. Omata. Generative adversarial network for road damage detection. In Computer-Aided Civil and Infrastructure Engineering, 2020. 1, 2

    2020

  8. [16]

    Mathiasen and F

    A. Mathiasen and F . Hvilshøj. Backpropagating through fr´echet inception distance.arXiv preprint arXiv:2009.14075, 2021. 3

    2009 arXiv

  9. [17]

    A. M. Okran, M. Abdel-Nasser, H. A. Rashwan, and D. Puig. Effective deep learning-based ensemble model for road crack detection. In IEEE International Conference on Big Data (Big Data), 2022. 1

    2022

  10. [18]

    K.-C. Peng. Iterative self knowledge distillation — from pothole classification to fine-grained and covid recognition. InICASSP,

  11. [19]

    Radford, J

    A. Radford, J. W . Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P . Mishkin, J. Clark, G. Krueger, and I. Sutskever. Learning transferable visual models from natural language supervision. InICML, 2021. 2, 3

    2021

  12. [20]

    Russakovsky, J

    O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. ImageNet large scale visual recognition challenge.IJCV, 115(3):211–252,

  13. [21]

    P . K. Saha and Y . Sekimoto. Road damage detection for multiple countries. In IEEE International Conference on Big Data (Big Data), 2022. 1

    2022

  14. [22]

    W ang, Y

    S. W ang, Y . T ang, X. Liao, J. He, H. Feng, H. Jiao, X. Su, and Q. Y uan. An ensemble learning approach with multi-depth attention mechanism for road damage detection. InIEEE International Conference on Big Data (Big Data), 2022. 1

    2022

  15. [23]

    W ang, J

    W . W ang, J. Dai, Z. Chen, Z. Huang, Z. Li, X. Zhu, X. Hu, T . Lu, L. Lu, H. Li, X. W ang, and Y . Qiao. InternImage: Exploring large-scale vision foundation models with deformable convolutions. InCVPR, 2023. 3, 4

    2023

  16. [24]

    D. Wu, M. Liao, W . Zhang, X. W ang, X. Bai, W . Cheng, and W . Liu. YOLOP: Y ou only look once for panoptic driving perception. Machine Intelligence Research, 2022. 1, 2

    2022

  17. [25]

    Zhong, J

    J. Zhong, J. Huyan, W . Zhang, H. Cheng, J. Zhang, Z. T ong, X. Jiang, and B. Huang. A deeper generative adversarial network for grooved cement concrete pavement crack detection. In Engineering Applications of Artificial Intelligence, 2023. 1, 2

    2023

  18. [26]

    J.-Y . Zhu, T . Park, P . Isola, and A. A. Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In ICCV, 2017. 3, 1 5 Appendix The appendix is organized as follows: • In Sec. A1, we provide the statistics of the RDD 2022 dataset [6] and the d...

    2017

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.