{"id":"07611b13-197b-493d-8cc5-009023fffd4e","arxiv_id":"2411.14737","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A sales-weighted feature influence score, a Random Forest popularity predictor, and diffusion-based image edits are combined to rank fashion features, with a human survey that only weakly confirms the ranking.","lead":"This paper proposes an influence score that ranks fashion design features using past sales data, and a Random Forest model that predicts sales class from product images, captions, and metadata. The authors validate the score by removing high- and low-scoring features from images with diffusion models and comparing human preferences, but the evidence is thin and partly circular.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The ablation's causal inference rests entirely on the unverified assumption that diffusion edits remove only the targeted feature; the paper's own caveats and appendix images suggest other attributes change, so Tables 3-4 do not establish feature influence.","rationale":"Agree with the reader: the weakest assumption is edit fidelity. The central claim is that the influence score identifies features that affect product popularity, and the only manipulation-based evidence is Experiment 2. That evidence requires surgical edits. The paper gives no check that the edits are surgical, and there is internal and visual reason to doubt it: Section 4 acknowledges style-inconsistency risk for Firefly, and the appendix figures show changes beyond the named attribute. Without a fidelity check, the FDP score gaps in Table 3 and the human preferences in Table 4 could be caused by editing artifacts, style drift, or correlated attribute changes rather than by the feature itself. The near-chance human results for bad features and the absence of confidence intervals make the ablation even harder to interpret, but the fidelity problem is the most load-bearing because it undercuts the entire experimental design. If the proposed audit passes, the central claim would be substantially strengthened; if it fails, the REJECT verdict is clearly correct. Therefore the reader's verdict should remain unchanged.","tokens_in":15362,"tokens_out":6714,"duration_ms":67587,"concrete_test":"Run a pre-registered fidelity audit on all 18 image pairs used in Tables 3 and 4: for each pair, have at least three blinded annotators (or a validated attribute-detection model) independently answer (a) whether the target feature is absent from the edited image and (b) whether every item on a fixed checklist of non-target attributes (colors, fabric texture, background, silhouette, other visible design elements) is unchanged. Keep only pairs that pass both checks, then recompute Table 3 and Table 4 on the retained pairs. If no pair passes, or if the retained results change materially, the ablation does not isolate the target feature and the central claim lacks support.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing assumption is the edit-fidelity counterfactual underlying Experiment 2: after InstructPix2pix-Distill or Adobe Firefly removes a good or bad feature, the edited image differs from the original only in that feature. Under that assumption, FDP score changes in Table 3 and human preferences in Table 4 measure the target feature's influence; without it, they measure editing artifacts and correlated visual changes. The paper provides no fidelity verification, no attribute classifier, no blinded annotation, and no embedding-distance check on non-target attributes. Section 4 itself warns that Firefly can generate content that does not fit well with the rest of the product, leading to style inconsistencies, and the appendix pairs visibly differ beyond the named feature. Because the influence score (Eq. 2) is validated almost entirely through this ablation, a confounded edit leaves the central claim without empirical support.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an 'influence score' for fashion product features, defined as the average min-max normalized sales of products containing the feature plus a frequency regularization term (Eq. 2), and a Fashion Demand Predictor (FDP) that classifies products into three sales classes from image, text, and tabular data. The authors validate the influence score through an ablation study in which a small set of 'good' and 'bad' features are removed from product images using diffusion models, and the resulting images are scored by the FDP and by human raters. The FDP reaches 81% accuracy on the 3-class sales task, and the paper claims that removing good features lowers predicted popularity while removing bad features raises it. The paper also compares FDP with a LLaVA baseline on a triplet ranking task.","tokens_in":15499,"tokens_out":8595,"duration_ms":80198,"significance":"The problem is timely and the idea of leveraging sales data to score visual features is appealing. The FDP model's accuracy is decent, and the combination of automated feature scoring with image editing is an interesting direction. However, the validation of the central influence score is not convincing: it relies on internal consistency with the FDP, which is trained on the same sales labels, and on a human study that shows only weak support for the 'bad' feature removal. The paper also provides no evidence that the diffusion edits isolate the targeted feature. If the influence score were validated with independent held-out data and faithful edits, it could be a useful contribution; as presented, the evidence does not support the claims.","major_comments":[{"comment":"The influence score and the FDP are both computed from the same sales labels: the former as the average normalized sales of products containing a feature, the latter trained on quantile-based sales classes. Table 3 therefore tests whether two models of the same target agree; it cannot serve as an independent validation of the influence score. The human survey in Table 4 is independent, but for 'bad' features the modified image is preferred in only 5 of 9 cases (with 1 tie), which is close to chance and does not establish the claim that the influence score identifies features whose removal improves popularity.","section":"§3.3, Eq. (2); §5.1, Experiment 2"},{"comment":"The ablation assumes that InstructPix2pix-Distill and Adobe Firefly remove exactly the targeted feature and leave all other attributes unchanged. The paper provides no fidelity check (e.g., an attribute classifier or human annotation verifying that only the target changed). The text itself notes that Firefly 'can generate content that does not fit well with the rest of the product' (Section 4), and the appendix examples (Figures 4–6) show apparent changes in style and other attributes. Without such verification, the score changes in Tables 3 and 4 cannot be attributed to the intended feature.","section":"§4; Tables 3–4"},{"comment":"The evaluation uses only 9 features per side, each with a single product. No error bars, confidence intervals, or significance tests are reported. In Table 3, some differences are very small (e.g., 'Stripped Finish': 1.345 vs 1.342), and in Table 4 the 'bad' feature results are mixed (5 AI, 3 original, 1 tie). This sample size and effect size do not support the strong claim that the influence score reliably identifies good and bad features.","section":"§5.2, Tables 3–4"},{"comment":"The regularization term λ·NP(pi) is added after min-max normalization of feature frequencies, so higher-frequency features receive a larger positive increment; yet the text states that a positive λ 'penalizes features with lower frequencies.' This is at best a relative penalty, and the choice λ=0.15 is not justified by any sensitivity analysis. Since the influence score is the central quantity, its definition should be clarified and its hyperparameters analyzed.","section":"§3.3, Eq. (2)"}],"minor_comments":[{"comment":"'Stripped Finish' appears in the table but the figure caption and the surrounding text refer to a 'striped pattern'; the spelling should be consistent ('Striped Finish').","section":"Table 3 and Figure 6"},{"comment":"The paper states there are 22 categorical and 13 numerical features in Section 3.1, but later refers to '36 categorical and numeric product data features' in Section 3.4; the discrepancy should be resolved.","section":"§3.1 and §3.4"},{"comment":"The abstract says products are 'enhanced with' good features, but the experiments only remove features; this wording is misleading and should be corrected.","section":"Abstract and §5"},{"comment":"The Kendall tau formula sums over triples with a constant denominator; the notation could be simplified for readability, but this does not affect the results.","section":"§5.1, Eq. (5)"}],"recommendation":"reject","confidential_remarks":"The paper has a promising idea, but the validation is fundamentally circular and the ablation is confounded by unverified edits. The human survey results for 'bad' features are too weak to rescue the central claim. I would consider a revised paper that (1) computes influence scores on a training split and evaluates on held-out data, (2) reports significance tests and confidence intervals, (3) performs a fidelity check on the image edits, and (4) tempers claims about automation and causality. As submitted, it does not meet the bar for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this paper has a coherent, practical pipeline and a real industrial dataset, but the central claim—that the influence score identifies causal drivers of fashion popularity—is not supported by the validation. The load-bearing counterfactual assumption is that the diffusion edits remove only the targeted feature, and the paper gives no fidelity check; its own caveats and appendix images suggest other attributes change. Without that, Tables 3 and 4 measure editing artifacts as much as feature influence.\n\nWhat's genuinely new: the assembly of a sales-frequency influence score, a Random Forest popularity classifier on multimodal embeddings, and diffusion-based counterfactual editing with human ranking. I don't know of prior work doing exactly this combination. The FDP's 0.81 accuracy on a 3-class sales task is respectable, and the triplet ranking experiment against Llava is a reasonable sanity check, though it largely confirms the classifier learned its training target.\n\nWhere it gets soft, in proportion: the influence score is computed from labels, the FDP is trained on the same labels, and then the FDP is used to validate the influence score. That's internal consistency, not external validation. The ablation uses 9 good and 9 bad features, hand-picked for editability, with no error bars or significance tests. The human survey for 'bad' features shows only 5/9 preferring the modified version, with 3 preferring original and 1 tie—basically chance. The paper acknowledges Firefly can produce style inconsistencies, and the appendix pairs visibly differ beyond the named feature. These are fixable but they are load-bearing.\n\nThe writing is clear, the dataset is real (8,503 products from a fast fashion company), and the authors are honest about some limitations, though not about edit fidelity or circularity.\n\nWho's this for? Groups building practical design-assistance tools for e-commerce will find the framework appealing, but they should not adopt the influence score as validated. For a general CV audience, it's a cautionary example of counterfactual evaluation.\n\nMy recommendation: it deserves peer review, not a desk reject, because the problem is real and the framework has promise. But I'd send it with an expectation of major revision: pre-register a random feature sample, add edit-fidelity checks (attribute classifiers or embedding distance on non-target attributes), report significance, and either get much stronger human data or soften the causal language.","headline":"A promising pipeline undermined by unverified counterfactual edits and a largely circular validation; the influence score remains unproven.","tokens_in":16050,"tokens_out":2570,"would_cite":false,"duration_ms":25533,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A sales-and-frequency score can identify which fashion design features move product popularity, and the paper confirms it by AI-editing images and asking human raters.","keywords":["fashion image analysis","influence score","demand forecasting","diffusion-based image editing","ablation study","human preference survey","sales class classification","feature importance ranking"],"falsifier":"Run a controlled ablation where the edit is verified to change only the target feature—e.g., a pixel-level or perceptual similarity test showing the rest of the image is unchanged—and check whether the influence-score ranking still predicts the direction of FDP and human preference changes; if it does not, the causal claim fails. A quicker check: permute feature-sales associations and recompute influence scores; if random 'good'/'bad' labels predict human preferences as well as the real ones, the score is capturing catalog correlation rather than causal influence.","tokens_in":15101,"feed_emoji":"👗","tokens_out":10149,"duration_ms":91668,"temperature":0.7,"pith_summary":"The paper tries to establish that the design features written in a fashion product's caption can be ranked by how much they matter to buyers, using only historical sales data and how often each feature appears. It proposes an influence score for each feature, builds a classifier called the Fashion Demand Predictor (FDP) that predicts a product's sales class from its image, text, and metadata with over 80 percent accuracy, and then uses text-prompted image editing to remove top-ranked and bottom-ranked features from real product photos. The authors argue that the ablation results—removing high-scoring features lowers predicted popularity while removing low-scoring features raises it—agree with human preference surveys, so the score is a usable guide for design and marketing decisions. If right, the framework gives fashion teams an automated, data-driven way to decide which features to keep, add, or drop before a product ships.","feed_headline":"Sales history ranks which fashion features matter most","feed_subtitle":"A score trained on 8,503 products and validated with AI edits tells designers what to keep.","key_machinery":"The central object is the influence score, Eq. (2): $$\\operatorname{Influence}(f_i) = \\frac{1}{|S_i|}\\sum_{s\\in S_i} N_S(s) + \\$\\lambda$ \\cdot N_P(p_i),$$ the average normalized sales of products containing feature $f_i$ plus a frequency-regularization term weighted by $\\lambda = 0.15$. The score converts sales history into a per-feature popularity label that ranks 1,147 features from 'good' to 'bad.' The validation machinery is the Fashion Demand Predictor (FDP), a Random Forest classifier that maps image embeddings (FashionCLIP), text embeddings (Sentence-BERT), and tabular product attributes to one of three equal-quantile sales classes; its softmax probabilities are turned into a scalar popularity score via Eq. (4). The third piece is the ablation loop: diffusion-based editors remove a targeted feature from an original product image, and the change in FDP score and in human preference between original and edited versions is used to confirm the feature's influence.","core_discovery":"At the heart of the paper is the influence score for each of 1,147 caption-derived design features: $$\\operatorname{Influence}(f_i) = \\frac{1}{|S_i|}\\sum_{s\\in S_i} N_S(s) + \\$\\lambda$ \\cdot N_P(p_i),$$ with $\\lambda = 0.15$, where the first term is the average min-max-normalized sales of products containing the feature and the second penalizes rarely seen features. High-scoring features are labeled 'good' and low-scoring 'bad.' To validate the ranking, the authors train the Fashion Demand Predictor (FDP), a Random Forest classifier that takes FashionCLIP image embeddings, Sentence-BERT caption embeddings, and 36 categorical or numeric product attributes and assigns each product to one of three equal-quantile sales classes with 81% test accuracy. They then remove a targeted feature from real product images with two diffusion-based editors, InstructPix2pix-Distill and Adobe Firefly, and compare the original and edited versions using the FDP's scalar popularity score $s_j = \\sum_{i=1}^{3} P(C_{ji})\\cdot i$ and human preference surveys. The paper reports that removing 'good' features lowers FDP scores in all nine tested cases and human preference in seven of nine, while removing 'bad' features raises FDP scores in seven of nine and human preference in five of nine; this agreement is taken as evidence that the influence score captures features that genuinely drive demand.","pith_inferences":["The influence score is a frequency-adjusted conditional average of past sales, so it describes which features the training catalog's customers bought, not why they bought; the causal reading rests entirely on how clean the diffusion edits are.","The same score-and-ablate recipe should transfer to any product domain with attribute text and sales data—furniture, consumer electronics, packaged goods—where the limiting step is editing images so that only one attribute changes.","Replacing the fixed $\\lambda = 0.15$ with an empirical-Bayes shrinkage that downweights high-variance rare features could stabilize the ranking of features seen in only a handful of products, which the current frequency penalty only partially addresses."],"forward_implications":["Designers could rank candidate features by influence score before production, using only the historical catalog's sales and captions, and use the ranking to prioritize high-scoring features and drop low-scoring ones.","The Fashion Demand Predictor offers a fast, cheap proxy for market response: training takes about 15 minutes on a CPU over 8,503 products, compared with several GPU-hours to fine-tune a multimodal LLM.","The same ablation pipeline can be reused as a quality check for AI-edited product images: if removing a supposedly 'good' feature does not lower the FDP score, the edit or the score needs scrutiny.","The method is automated end-to-end—caption cleaning, synonym clustering, scoring, and prediction—so it can be rerun as new sales data arrive, keeping the feature ranking current."],"supporting_citations":[{"why":"Supplies the pretrained image encoder whose embeddings feed the FDP classifier; the model's visual input depends on it.","marker":"[10]"},{"why":"Encodes each product caption into a dense vector that the FDP uses alongside image embeddings.","marker":"[50]"},{"why":"The ensemble classifier chosen for FDP; its 81% three-class accuracy is the performance claim that anchors the validation.","marker":"[42]"},{"why":"One of the two diffusion tools used to remove targeted features in the ablation study.","marker":"[45]"},{"why":"The other editing tool, used for region-specific feature removal in the ablation study.","marker":"[2]"},{"why":"Basis for MinHashLSH clustering that groups synonymous caption phrases into features.","marker":"[6]"},{"why":"Provides the locality-sensitive hashing procedure that makes synonym grouping scalable and yields the 1,147-feature set.","marker":"[18]"},{"why":"Establishes the paper's core proxy: consumer ratings (popularity) correspond with higher sales, justifying sales as the target variable.","marker":"[41]"},{"why":"Supplies the rank-correlation metric used to score FDP and human rankings in Experiment 1.","marker":"[27]"}],"fun_headline_variants":["AI edits prove which fashion features move sales","Influence score ranks fashion features by proven sales impact","Diffusion-based test validates fashion feature popularity","Sales history + AI = fashion feature importance score"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument depends on the image edits removing exactly the targeted feature and nothing else, because the paper provides no fidelity check that the edited images differ only in that feature; if the edits change style, color, or other attributes, the ablation results do not isolate the feature's influence.","fun_headline_variants_meta":{"raw":{"variants":["AI edits prove which fashion features move sales","Influence score ranks fashion features by proven sales impact","Diffusion-based test validates fashion feature popularity","Sales history + AI = fashion feature importance score"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000301,"raw_usage":{"total_tokens":1782,"prompt_tokens":1038,"completion_tokens":744,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":654,"completion_tokens_details":{"reasoning_tokens":686}},"tokens_in":654,"tokens_out":744,"duration_ms":8946,"temperature":1.0,"reasoning_tokens":686,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:57:10.932561+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a controlled ablation where the edit is verified to change only the target feature—e.g., a pixel-level or perceptual similarity test showing the rest of the image is unchanged—and check whether the influence-score ranking still predicts the direction of FDP and human preference changes; if it does not, the causal claim fails. A quicker check: permute feature-sales associations and recompute influence scores; if random 'good'/'bad' labels predict human preferences as well as the real ones, the score is capturing catalog correlation rather than causal influence.","supporting_citations":[{"cited_title":"Fashionclip: Connecting lan- guage and images for product representations","cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained image encoder whose embeddings feed the FDP classifier; the model's visual input depends on it."},{"cited_title":"A re- view on random forest: An ensemble classifier","cited_arxiv_id":null,"evidence_quote":"The ensemble classifier chosen for FDP; its 81% three-class accuracy is the performance claim that anchors the validation."},{"cited_title":"Instruct-pix2pix-distill: A distillation framework for pix2pix","cited_arxiv_id":null,"evidence_quote":"One of the two diffusion tools used to remove targeted features in the ablation study."},{"cited_title":"Adobe introduces firefly image 3 foundation model to take creative exploration and ideation to new heights","cited_arxiv_id":null,"evidence_quote":"The other editing tool, used for region-specific feature removal in the ablation study."},{"cited_title":"On the resemblance and containment of documents","cited_arxiv_id":null,"evidence_quote":"Basis for MinHashLSH clustering that groups synonymous caption phrases into features."},{"cited_title":"Simi- larity search in high dimensions via hashing","cited_arxiv_id":null,"evidence_quote":"Provides the locality-sensitive hashing procedure that makes synonym grouping scalable and yields the 1,147-feature set."},{"cited_title":"On- line consumers reviews: Examining the moderating effects of product type and product popularity on the review impact on sales","cited_arxiv_id":null,"evidence_quote":"Establishes the paper's core proxy: consumer ratings (popularity) correspond with higher sales, justifying sales as the target variable."},{"cited_title":"A new measure of rank correlation","cited_arxiv_id":null,"evidence_quote":"Supplies the rank-correlation metric used to score FDP and human rankings in Experiment 1."}],"review_version":1}