Pith. sign in

REVIEW 2 major objections 6 minor 22 references

Atlas: A Dataset and Benchmark for E-commerce Clothing Product Categorization

T0 review · 2 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read This paper introduces Atlas, a public dataset of 186,150 clothing images with a 52-leaf taxonomy, and benchmarks image-only categorization at a 0.92 micro F-score.

desk verdict A useful dataset release whose benchmark number is questionable until the split is clarified. read the letter →

arxiv 1908.08984 v1 pith:YQBG4ANL submitted 2019-08-12 cs.CV cs.LGstat.ML

classification cs.CVcs.LGstat.ML
keywords productcategorizatione-commercetaxonomyclothingimagesimageclassificationsequence-to-sequenceattentionmechanismdatasetbenchmarkhierarchical
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces Atlas, a public dataset of 186,150 clothing product images labeled with category paths in a three-level taxonomy ending in 52 leaf nodes. It argues that image-based product categorization can be benchmarked reproducibly on this dataset, and reports that a ResNet-34 classifier reaches a micro F-score of 0.92 on the test split, with an attention-based sequence model at 0.90. The contribution is as much the collection methodology as the numbers: crawling retailer listings, manually mapping store breadcrumbs to a unified taxonomy, and filtering out zoomed or detail images with a small CNN trained on visually inspected examples. A sympathetic reader would care because public, real-world product taxonomy data is scarce and stores use incompatible taxonomies, so a shared benchmark lets competing approaches be compared on the same categories.

What carries the argument

The load-bearing object is the Atlas dataset itself: 186,150 images with 52 category paths under a three-level taxonomy built by clustering similar retailer breadcrumbs. Two mechanisms carry the benchmark. First, a two-class CNN filter, called the Zoomed Versus Normal model, is trained on visually inspected thumbnails to remove close-up or cropped images that would otherwise corrupt the dataset. Second, the benchmark models: a ResNet-34 classifier with adaptive pooling, and an attention-based sequence-to-sequence model in which a ResNet-101 encoder produces a $14 \times 14 \times 2048$ feature map and an LSTM decoder with soft attention predicts the category path as a token sequence from a start marker to an end marker, using beam search at inference.

What would settle it

Take a random sample of Atlas test images, have independent annotators assign the 52 leaf categories without seeing the stored labels, and measure agreement against the dataset labels; low agreement would mean the reported F-scores partly reflect label noise rather than categorization accuracy.

Watch

Extended reading notes

Core claim

The central claim is that Atlas is a clean, usable dataset for e-commerce clothing categorization and that standard visual models perform well on it: the ResNet-34 image classifier obtains a micro F-score of 0.92 over 52 leaf categories, while the attention-based encoder-decoder, which predicts the category path level by level from a ResNet-101 image encoding, obtains 0.90. The paper also claims the attention model offers an explanation for predictions by focusing on different image regions, such as the face for gender and the garment area for later levels, and that it can propose new category paths, though only 2 of 5 generated paths were valid, so manual inspection is needed. This establishes a baseline rather than a solution; the authors position Atlas as an invitation for the community to improve on these numbers.

Load-bearing premise

The dataset labels are derived from retailer website breadcrumbs that were manually mapped to the paper's taxonomy; if any mapped breadcrumb is wrong or the mapping is inconsistent across stores, the labels are noisy and the reported F-scores no longer measure what they appear to.

Editorial extensions

If this is right

  • Future methods can be compared directly on fixed 65/5/30 splits, since Atlas fixes the taxonomy and the test set.
  • The zoomed-versus-normal filtering step shows that a small binary CNN can remove unrepresentative product images with an average F-score of 0.98, a reusable cleaning step.
  • The attention-based sequence model reaches a 0.90 micro F-score and produces attention maps that indicate which image region supports each category level.
  • The sequence model generates novel category paths, but only 2 of 5 it produced were valid, so manual review is required before enriching the taxonomy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the labels survive independent audit, the 0.92 image-only baseline suggests the ceiling for 52 well-separated clothing categories is already close to practical accuracy, and further gains should come from difficult confusable pairs rather than model capacity.
  • Constraining the decoder to output only legal child categories would directly test whether invalid path generation is avoidable; this is measurable against the paper's reported 2-of-5 validity.
  • Because the taxonomy includes Indian ethnic-wear categories, Atlas can serve as a domain-shift test for models trained on Western fashion images.
  • Product titles and prices were collected alongside images but are not used by the benchmark models; a multimodal model using both could be compared against the 0.92 image-only number.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper introduces Atlas, a clothing product image dataset of 186,150 images organized into a three-level taxonomy with 52 leaf category paths, collected by crawling Indian e-commerce product listings and manually mapping store breadcrumbs to a custom taxonomy. The authors describe a zoomed-vs-normal image filtering step, then present two benchmarks: a ResNet-34 image classifier and an attention-based CNN-LSTM sequence-to-sequence model that predicts category paths. On a 65/5/30 split, they report a micro F-score of 0.92 for the classifier and 0.90 for the sequence model. The dataset, code, and pre-trained models are publicly released.

Significance. If the reported performance is a valid estimate of generalization to new products, Atlas would be a useful, reproducible benchmark for image-based e-commerce product categorization, and the public release of data, code, and training configurations is a concrete strength. The paper also provides a plausible, extensible methodology for building similar taxonomy datasets. However, the central empirical claim depends on two load-bearing assumptions: that the train/test split prevents leakage of the same product across splits, and that the crawled breadcrumb-derived labels are accurate. These are asserted but not demonstrated, so the headline 0.92 F-score should currently be treated as unverified.

major comments (2)
  1. [Section 6, with Section 3.3] The reported benchmark numbers may not measure generalization to new products because the split appears to be image-level, not product-level. Section 6 states that the train/validation/test split was produced by stratified random sampling on the dataset, giving 119,155 training images, 11,147 validation images, and 55,848 test images; Section 3.3 explicitly notes that an e-commerce listing often contains several images of a single product. If multiple images of the same product are distributed across train and test, the model can exploit near-duplicate appearance, and the reported 0.92 micro-F becomes an overestimate of performance on unseen products. The paper does not mention product IDs, deduplication, or a product-level split. Please report whether the splits are grouped by product, and either provide a product-level split or demonstrate that no product appears in both training and test sets.
  2. [Section 3.2] The ground-truth labels are created by manually mapping each retailer's breadcrumb path to the authors' custom taxonomy, but no label-quality measurement is reported. If retailer breadcrumbs are wrong or the manual mapping is inconsistent, the labels are noisy and the F-scores do not measure what they claim. The paper should report an audit of the labeling process, such as inter-annotator agreement on a sample, a manual verification of a random subset, or a quantitative comparison against an independent label source. Without this, the 'high-quality dataset' claim is not yet supported.
minor comments (6)
  1. [Section 3, first paragraph] The text says 'Our dataset contains data corresponding to 52 products' but the rest of the paper consistently describes 52 category paths (leaf nodes), not 52 products. This should be corrected to avoid confusion.
  2. [Section 6] Please define micro-F precisely: is it computed on the leaf-level category, on full path accuracy, or on per-level token predictions? For the sequence model, a path is only correct if all levels are correct, so the metric definition matters for interpreting the 0.92 versus 0.90 comparison.
  3. [Section 6] The results are reported as single numbers without error bars or repeated runs. Given that the two models differ by only two points, a small number of independent runs with mean and standard deviation would make the benchmark comparison more informative.
  4. [Section 3.3] The zoomed-vs-normal filter is a key data-cleaning step, but the paper does not report the size or construction of the test set used for Table 1, nor how the 6005 normal and 1054 zoomed training images were split. Please clarify whether the filter's test set is disjoint from the final Atlas train/test split.
  5. [Section 5.1] The beam width is said to be chosen 'based on our experiments,' but no sweep or resulting F-scores are reported. Please provide the evaluated beam widths and the corresponding performance, or state that the choice had negligible effect.
  6. [Section 3.3] There is a typo: 'contors' should be 'contours.'

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the benchmark is an empirical measurement on a held-out test set.

full rationale

The paper's central claim is an empirical benchmark: ResNet-34 and Seq2Seq models are trained on a training split and scored on a held-out test split (Section 6), with no parameter fitted to the test labels. The dataset labels are produced from external retailer breadcrumbs manually mapped to the authors' taxonomy (Sections 3.1-3.2), not from the benchmark models, so label construction does not predetermine the reported F-scores. There are no load-bearing self-citations or imported uniqueness theorems. Concerns raised in review, such as possible product-level leakage from image-level splitting or label noise from breadcrumb mapping, are data-quality and evaluation-validity issues rather than circularity: they do not make the reported numbers equal to an input by construction. Therefore the paper warrants a 0 circularity score.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claims rest on the dataset being correctly labeled and split. Those assumptions are not externally verified in the paper.

free parameters (4)
  • Beam width = 5
    Selected by experiments for the sequence decoder; changing it changes the reported F-score.
  • SVM regularization C = 0.0001
    Chosen via grid search for the zoomed-versus-normal filter; affects which images are kept in the dataset.
  • Encoder/decoder learning rates = 1e-4 and 4e-4
    Fixed training hyperparameters; the benchmark result is conditional on them.
  • Zoomed-vs-normal filter training set composition = 6,005 normal and 1,054 zoomed
    The filter training set is manually selected and imbalanced; this choice affects downstream dataset composition.
assumptions (4)
  • domain assumption Retailer breadcrumbs and the manually constructed mapping to the custom taxonomy provide correct category labels.
    Invoked in Section 3.2; no external label audit is performed.
  • domain assumption The zoomed-versus-normal CNN trained on 6,005 and 1,054 manually labeled images reliably cleans the full dataset.
    Invoked in Section 3.3; errors in this filter propagate into the released labels.
  • domain assumption Stratified image-level splitting produces independent train and test sets.
    Invoked in Section 6; if multiple images of the same product appear across splits, reported results would be inflated.
  • domain assumption Pretrained ImageNet weights transfer appropriately to clothing images.
    Used in Sections 4.1 and 4.2; standard practice but not verified on this domain.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Atlas: A Dataset and Benchmark for E-commerce Clothing Product Categorization." pith.science (2026). https://pith.science/paper/YQBG4ANL

@misc{pith2026190808984,
  author       = {Pith},
  title        = {Pith review of: Atlas: A Dataset and Benchmark for E-commerce Clothing Product Categorization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YQBG4ANL}},
  note         = {Machine review of arXiv:1908.08984}
}
read the original abstract

In E-commerce, it is a common practice to organize the product catalog using product taxonomy. This enables the buyer to easily locate the item they are looking for and also to explore various items available under a category. Product taxonomy is a tree structure with 3 or more levels of depth and several leaf nodes. Product categorization is a large scale classification task that assigns a category path to a particular product. Research in this area is restricted by the unavailability of good real-world datasets and the variations in taxonomy due to the absence of a standard across the different e-commerce stores. In this paper, we introduce a high-quality product taxonomy dataset focusing on clothing products which contain 186,150 images under clothing category with 3 levels and 52 leaf nodes in the taxonomy. We explain the methodology used to collect and label this dataset. Further, we establish the benchmark by comparing image classification and Attention based Sequence models for predicting the category path. Our benchmark model reaches a micro f-score of 0.92 on the test set. The dataset, code and pre-trained models are publicly available at \url{https://github.com/vumaasha/atlas}. We invite the community to improve upon these baselines.

Figures

Figures reproduced from arXiv: 1908.08984 by the authors.

Figure 1
Figure 1. Examples of (a) Zoomed(dirty) and (b) Normal(clean) images from our Atlas dataset. The Zoomed images show close-ups of the apparel or cropped versions of the image that make it difficult to recognize the product, whereas the Normal images show figures with the entire product visible [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Architecture of Zoomed Vs Normal Model tative image of the product. Some images might display packaging, installation instructions, etc. In the case of clothing, we found that many product listings also included zoomed in images that display intrinsic details such as the texture of the fabric, brand labels, button, and pocket styles. Without the context of the product listing, it would be even hard for a human to id… view at source ↗
Figure 3
Figure 3. A sample of category paths predicted on test dataset by our model. We can observe how the Attention focuses on different sections of the image while generating each category level. For example, the face is being focused to predict the first category level - gender. encoding produced by the Encoder will have the dimensions: batch size,14,14, 2048. Recurrent Neural Networks(RNN) are popular for sequential classificati… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Encoder - Decoder with Attention Network 5 Training 5.1 Model hyperparameters Zoomed Vs Normal LinearSVM Model We trained LinearSVM available in Scikit-learn with C set to 0.0001, class weight set to ’balanced’ using hinge loss. The optimal C value was identified using…
Figure 5
Figure 5. Figure 5: F-scores of our benchmark models over leaf level categories ordered by their sample size. Note that the sample size in x axis is in log scale 7 Conclusion and Future Works This paper introduces Atlas, a fashion apparel dataset with 186,150 apparel images along with the…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 19 canonical work pages

  1. [1]

    In: Asian conference on computer vi- sion

    Bossard, L., Dantone, M., Leistner, C., Wengert, C., Quack, T., Van Gool, L.: Apparel classification with style. In: Asian conference on computer vi- sion. pp. 321–335. Springer (2012)

  2. [2]

    In: European conference on computer vision

    Chen, H., Gallagher, A., Girod, B.: Describing clothing by semantic at- tributes. In: European conference on computer vision. pp. 609–623. Springer (2012)

  3. [3]

    In: Proceedings of the 15th Conference of the European Chapter of the As- sociation for Computational Linguistics: Volume 1, Long Papers

    Das, P., Xia, Y., Levine, A., Di Fabbrizio, G., Datta, A.: Web-scale language-independent cataloging of noisy product listings for e-commerce. In: Proceedings of the 15th Conference of the European Chapter of the As- sociation for Computational Linguistics: Volume 1, Long Papers. vol. 1, pp. 969–979 (2017)

  4. [4]

    In: Proceedings of the 5th international conference on business information systems (2002)

    Ding, Y., Korotkiy, M., Omelayenko, B., Kartseva, V., Zykov, V., Klein, M., Schulten, E., Fensel, D.: Goldenbullet: Automated classification of product data in e-commerce. In: Proceedings of the 5th international conference on business information systems (2002)

  5. [5]

    In: Proceed- ings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval

    Dumais, S., Chen, H.: Hierarchical classification of web content. In: Proceed- ings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval. pp. 256–263. ACM (2000)

  6. [6]

    In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Min- ing

    Ha, J.W., Pyo, H., Kim, J.: Large-scale item categorization in e-commerce using multiple recurrent neural networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Min- ing. pp. 107–115. ACM (2016)

  7. [7]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recog- nition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)

  8. [8]

    In: eCOM@SIGIR

    Hiramatsu, M., Wakabayashi, K.: Encoder-decoder neural networks for tax- onomy classification. In: eCOM@SIGIR. CEUR Workshop Proceedings, vol. 2319. CEUR-WS.org (2018)

Show all 22 references
  1. [9]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Karpathy, A., Fei-Fei, L.: Deep visual-semantic alignments for generating image descriptions. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3128–3137 (2015)

  2. [10]

    In: European conference on computer vision

    Kiapour, M.H., Yamaguchi, K., Berg, A.C., Berg, T.L.: Hipster wars: Dis- covering elements of fashion styles. In: European conference on computer vision. pp. 472–488. Springer (2014)

  3. [11]

    In: Proceedings of the 2015 Conference of the North Ameri- can Chapter of the Association for Computational Linguistics: Human Lan- guage Technologies

    Kozareva, Z.: Everyone likes shopping! multi-class product categorization for e-commerce. In: Proceedings of the 2015 Conference of the North Ameri- can Chapter of the Association for Computational Linguistics: Human Lan- guage Technologies. pp. 1329–1333 (2015)

  4. [12]

    In: Advances in neural information pro- cessing systems

    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in neural information pro- cessing systems. pp. 1097–1105 (2012) 12 V. Umaashankar et al

  5. [13]

    In: eCOM@SIGIR

    Li, M.Y., Kok, S., Kok, S.: Unconstrained product categorization with sequence-to-sequence models. In: eCOM@SIGIR. CEUR Workshop Pro- ceedings, vol. 2319. CEUR-WS.org (2018)

  6. [14]

    arXiv preprint arXiv:1812.05774 (2018)

    Li, M.Y., Kok, S., Tan, L.: Don’t classify, translate: Multi-level e- commerce product categorization via machine translation. arXiv preprint arXiv:1812.05774 (2018)

  7. [15]

    In: eCOM@SIGIR

    Lin, Y., Das, P., Datta, A.: Overview of the SIGIR 2018 ecom rakuten data challenge. In: eCOM@SIGIR. CEUR Workshop Proceedings, vol. 2319. CEUR-WS.org (2018)

  8. [16]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Liu, Z., Luo, P., Qiu, S., Wang, X., Tang, X.: Deepfashion: Powering robust clothes recognition and retrieval with rich annotations. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1096– 1104 (2016)

  9. [17]

    In: Proceedings of the 38th Interna- tional ACM SIGIR Conference on Research and Development in Informa- tion Retrieval

    McAuley, J., Targett, C., Shi, Q., Van Den Hengel, A.: Image-based recom- mendations on styles and substitutes. In: Proceedings of the 38th Interna- tional ACM SIGIR Conference on Research and Development in Informa- tion Retrieval. pp. 43–52. ACM (2015)

  10. [18]

    In: Proceedings of the 21st ACM international conference on Information and knowledge management

    Shen, D., Ruvini, J.D., Sarwar, B.: Large-scale item categorization for e- commerce. In: Proceedings of the 21st ACM international conference on Information and knowledge management. pp. 595–604. ACM (2012)

  11. [19]

    In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV)

    Smith, L.N.: Cyclical learning rates for training neural networks. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV). pp. 464–472. IEEE (2017)

  12. [20]

    Neural computation 1(2), 270–280 (1989)

    Williams, R.J., Zipser, D.: A learning algorithm for continually running fully recurrent neural networks. Neural computation 1(2), 270–280 (1989)

  13. [21]

    arXiv preprint arXiv:1609.08144 (2016)

    Wu, Y., Schuster, M., Chen, Z., Le, Q.V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al.: Google’s neural ma- chine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144 (2016)

  14. [22]

    In: International conference on machine learning

    Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhudinov, R., Zemel, R., Bengio, Y.: Show, attend and tell: Neural image caption generation with visual attention. In: International conference on machine learning. pp. 2048–2057 (2015)

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.