{"id":"e29153f7-c266-4997-8529-079fd1ff54fe","arxiv_id":"1908.01308","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Combining zero-padding with ROI pooling allows batch training on full-resolution photos, and adding a one-hot theme embedding improves aesthetic distribution prediction on AVA and Photo.net.","lead":"This paper presents a deep learning method for rating photo aesthetics that works on full-resolution images of any size by padding them and using ROI pooling to isolate the original image. It also feeds the photo's challenge theme into the network, showing that knowing the theme improves the predicted rating distributions.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theme labels at test time likely leak per-theme rating statistics, confounding the state-of-the-art comparison.","rationale":"The reader's conditional verdict is sound, but the most load-bearing weakness is not the ROI-pooling boundary artifact. That effect is real yet likely minor: ROI pooling is applied after only three convolutional layers, so the contaminated band is a few pixels wide, and training with padded inputs can partially absorb it. The theme-label leakage is more serious because it directly undermines the headline comparison: the winning model uses an input signal (theme) that no baseline receives, and the largest reported gain (std-dev SRCC jumping from 0.3424 to 0.6918) is exactly what a per-theme empirical prior would produce. The paper does not report whether the same themes appear in train and test, does not compare against a theme-conditioned baseline, and does not test held-out themes, so the state-of-the-art claim is not yet established as a fair, architecture-driven improvement. I also flag the missing aspect-ratio encoding advertised in the abstract but absent from the method, and the lack of error bars and code, as supporting reasons to keep the verdict conditional rather than accepting the claims at face value.","tokens_in":14547,"tokens_out":14767,"duration_ms":160104,"concrete_test":"Construct a theme-prior baseline: for each AVA test image, predict the empirical score distribution of training images belonging to the same challenge theme (and separately the per-theme mean and std). Compute SRCC and EMD using the same protocol as Table III. If this baseline approaches or exceeds the reported std-dev SRCC of 0.6918, the theme module is storing per-theme statistics rather than adapting visual features, and the SOTA comparison must be rerun against theme-conditioned baselines and a theme-disjoint train/test split.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the proposed model reaches state-of-the-art results on AVA (Abstract; Table II). The model behind those numbers is \"Pad+ROI+Theme\", which receives the AVA challenge theme as a one-hot input (Section III-B). The paper uses the standard AVA partition \"as in [40]\" (Section IV-A), which is image-wise rather than theme-wise; as a result, themes in the test set have many training images with known rating distributions. The one-hot theme embedding is therefore able to memorize per-theme mean score and rating spread, and this is consistent with the ablation in Table III: adding theme raises std-dev SRCC from 0.3424 to 0.6918 while mean SRCC rises from 0.7438 to 0.7611. None of the compared methods (NIMA, GPF-CNN, Hosu et al., etc.) receive theme labels, so the comparison is not apples-to-apples. The paper reports no theme-conditioned baseline and no evaluation with held-out themes, so we cannot tell whether the gain comes from learning visual criteria per theme or simply from reading off per-theme statistics. Separately, the abstract promises an aspect-ratio encoding fused with visual features, but Section III contains no such encoding, further suggesting the manuscript overstates what is actually implemented.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a full-resolution aesthetic quality assessment method. Arbitrary-size images are zero-padded to a uniform size and passed through Inception-v3, whose first pooling layer is replaced by ROI pooling so that features are pooled only from the original image region. The authors argue this preserves aspect ratio and resolution while eliminating padding side effects. A second contribution is a theme-aware branch: in the AVA dataset, the challenge theme is encoded as a one-hot vector, projected to 256 dimensions, concatenated with visual features, and used to predict an aesthetic score distribution under an EMD loss. The method is evaluated on AVA and Photo.net for distribution prediction and mean/std-dev score prediction, with ablations over input transformation, theme information, feature-map size, and data augmentation.","tokens_in":14763,"tokens_out":6566,"duration_ms":64592,"significance":"If the claims hold, the ROI-padding scheme is a practically useful way to train on full-resolution, variable-aspect-ratio images in batches, and the theme-aware formulation highlights a real source of criterion bias in crowdsourced aesthetic ratings. The paper has clear strengths: the architecture is simple and well motivated, the ablation study covers the main design choices (Table III), the feature-map-size analysis (Table IV) is informative, and the ROI-align comparison supports the quantization-error discussion. The main risk is that the headline state-of-the-art comparisons are confounded by theme-label information that competing methods do not receive, and the theoretical claim that padding side effects are fully eliminated is not exact. The reported gains, especially on standard-deviation prediction, are large enough that the confound must be resolved before the central claim can be accepted.","major_comments":[{"comment":"Section III-B introduces a theme branch fed with the one-hot challenge theme, and the experiments use the standard image-wise AVA split (Section IV-A). Under this split, a large fraction of the test-set themes also occur in training, so the fully connected theme embedding can memorize per-theme rating statistics (mean and, especially, variance) rather than learning a visual criterion per theme. The ablation in Table III is consistent with this: adding theme raises the std-dev SRCC from 0.3424 to 0.6918 while the mean SRCC increases only from 0.7438 to 0.7611. Because none of the compared methods in Tables I and II receive theme labels, the state-of-the-art comparisons are not apples-to-apples. I request a held-out-theme evaluation (e.g., no test theme seen in training) and a theme-conditioned baseline that gives the same one-hot theme input to a standard architecture, so the contribution of the visual theme-aware mechanism can be separated from per-theme label statistics.","section":"III-B, Tables I-III"},{"comment":"Equation (4), P(Ac,R)=P(aimg), treats the image and padding regions as cleanly separable at the ROI-pooling layer, but the convolutional layers between the padded input and the ROI-pooling layer have receptive fields that cross the image/padding boundary. Zero-valued padding therefore influences activations inside the image region near its border, and this boundary contamination is propagated into the pooled features. The claim that ROI pooling 'eliminates the side effects of padding' and that the network predicts 'based on only image features' is thus only approximately true. Please quantify the boundary effect (e.g., by ablating with non-zero padding values or by comparing interior and border crops) or soften the claim accordingly.","section":"III-A, Eq. (4)"},{"comment":"The abstract states that 'the image aspect ratios are encoded and fused with visual features to remedy the shape information loss of RoM pooling,' but Section III contains no aspect-ratio encoding, no aspect-ratio feature, and no experiment manipulating aspect-ratio information. The only auxiliary input described is the theme one-hot vector (Section III-B). Either implement and evaluate the aspect-ratio encoding or remove this claim from the abstract and the list of contributions.","section":"Abstract, Section III"}],"minor_comments":[{"comment":"All reported metrics are single-run point estimates. Please report standard deviations across multiple runs or state how many runs were averaged, especially because several gaps over baselines are small (e.g., Table II, mean SRCC 0.7611 vs. Hosu et al. 0.7450).","section":"Tables I-III"},{"comment":"The EMD value reported for Kong et al. [43] is unexpected because [43] is a ranking/attribute method rather than a distribution predictor; please clarify how this EMD was computed.","section":"Table VI, Section IV-G"},{"comment":"The abstract uses 'region of image (RoM) pooling' while the rest of the paper uses 'ROI (region of interest) pooling'; please make the terminology consistent.","section":"Abstract"},{"comment":"There are typographical errors, including 'wuth' for 'with' in Section III-D and 'roi poooling' in Section II-B; please proofread the manuscript.","section":"III-D, II-B"},{"comment":"The paper says the AVA split is 'as in [40]' but does not describe the number of training and test images; for reproducibility, give the exact split statistics or release the split.","section":"IV-A"},{"comment":"The EMD formula is numbered Eq. (7), but the text refers to 'equ (4)'; please fix the cross-reference.","section":"III-C"}],"recommendation":"major_revision","confidential_remarks":"The primary concern is the theme-leakage confound in the state-of-the-art comparison. If the authors can show that the theme advantage persists when test themes are unseen during training, or when a baseline receives the same theme input, I would be willing to accept the central claim. Otherwise the paper should restrict its claims to settings where theme labels are available at test time. The aspect-ratio encoding discrepancy in the abstract also needs to be resolved."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. The padding+ROI pooling trick for training on full-resolution images is a clean, practical contribution and the ablations support it. But the theme-aware model, which generates the headline numbers, is confounded: at test time the one-hot theme embedding can effectively look up the average score and spread of that theme from training images, since the split is image-wise and themes repeat. That explains the enormous jump in std-dev SRCC (0.34 to 0.69) when theme is added. None of the compared methods get theme labels, so the comparison in Table II is not apples-to-apples.\n\nWhat's good: Running Inception-v3 on padded originals and cutting off the padding with ROI pooling is a sensible adaptation of Fast R-CNN, and their ablation shows it beats resize and crop baselines on mean SRCC (0.74 vs ~0.70). The discussion of why ROI align vs pooling barely matters is reasonable, and the size analysis (Table IV) supports the full-resolution claim. That core idea is worth publishing.\n\nThe soft spots are real, though. The theme pollution is the big one. To fix it they'd need to test on held-out themes or compare against a trivial model that predicts the average ground-truth distribution per theme, or at least show the visual features actually change per theme. The abstract promises an aspect-ratio encoding fused with visual features, but Section III has no such encoding — the paper says one thing and implements another. All metrics are single-run point estimates, no error bars, no significance tests. The split is just \"as in [40]\", which is vague, and code isn't released. The Photo.net comparison uses their own random split, so the published numbers aren't directly comparable. These don't sink the ROI-pooling idea, but they sink the SOTA claim.\n\nWho's this for? Researchers working on AQA with multi-size inputs. The ROI-pooling adaptation is reusable, and the theme-leakage failure mode is a useful cautionary tale for anyone using metadata as a conditioning input.\n\nRecommendation: send it to peer review, but the reviewers should demand a theme-free comparison and a held-out-theme experiment. If the theme gain disappears, the paper can still be accepted on the strength of the ROI-pooling contribution alone, with the theme part toned down.","headline":"A clever ROI-pooling idea for full-resolution AQA, but the theme-aware model leaks per-theme label statistics and the SOTA claim doesn't survive.","tokens_in":15303,"tokens_out":2882,"would_cite":false,"duration_ms":28349,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Full-resolution aesthetic scoring is possible for arbitrary-sized photos: pad the image to a common canvas, let a region-of-interest pooling layer discard the padding before classification, and condition the prediction on the photo's…","keywords":["aesthetic quality assessment","full-resolution image","ROI pooling","image padding","theme-aware model","aesthetic distribution prediction","earth mover distance","AVA dataset"],"falsifier":"Run the early convolutional layers of the same network on a photograph alone and on that photograph zero-padded to the training canvas, then compare the two feature maps inside the photograph's rectangle; any difference above numerical roundoff shows that padding still shapes the features the ROI pooling layer keeps.","tokens_in":14328,"feed_emoji":"🖼️","tokens_out":9930,"duration_ms":99169,"temperature":0.7,"pith_summary":"Aesthetic quality assessment usually forces photos into a fixed input rectangle, and resizing, cropping, or naive padding distorts the composition being judged. This paper claims a way around that constraint: pad every photo to one canvas, run a fixed-input network, then use region-of-interest pooling to drop the padded region so only the original full-resolution pixels reach the classifier. It also argues that the same photo earns different scores under different contest themes, and that feeding theme information into the network alongside visual features improves the predicted score distribution. On AVA and Photo.net, the reported distribution distances, correlations, and mean-score errors are better than the prior methods it compares against, with theme information giving the largest boost to predicting how spread out the votes are.","feed_headline":"Full-resolution photos get fairer aesthetic scores via ROI pooling","feed_subtitle":"A padding-and-pooling trick preserves original pixels and theme context, outperforming prior methods on two standard datasets.","key_machinery":"The mechanism that carries the argument is ROI pooling on padded full-resolution inputs: zero-padding equalizes image sizes for batching, and the ROI pooling layer, placed where Inception-v3's first pooling layer would be, max-pools only the feature-map region corresponding to the original image, producing a uniform output while discarding padding activations. The second mechanism is theme conditioning, where a 1397-way one-hot theme code is passed through a fully connected layer to 256 dimensions and concatenated with visual features before the final classifier. The third is the loss: earth mover distance between the predicted and ground-truth cumulative score distributions, chosen because ordered score bins make cumulative distance more meaningful than raw cross-entropy.","core_discovery":"The central claim is that the fixed-input restriction of deep aesthetic networks is best handled at the feature-map level, not the pixel level, and that the aesthetic criterion itself is theme-dependent. The authors pad each photo to an 800-by-800 canvas, let the early layers of Inception-v3 process the canvas, then apply ROI pooling only to the rectangle containing the true image; in their words, this cuts off the forward propagation of padding features. They double the usual pooled size to 146 by 146 to relax the information bottleneck. A one-hot contest-theme code is reduced to 256 dimensions and concatenated with the visual features, so the network can learn different criteria for different themes. On Photo.net, which lacks theme labels, the ROI-pooling part alone still improves over the prior distribution model. The claimed outcome is that this model sets the best published numbers on AVA and Photo.net for aesthetic distribution prediction and mean-score regression.","pith_inferences":["The same padding-then-ROI construction should transfer to other image-level regression problems with mixed aspect ratios, such as general image quality, document quality, or medical-image scoring; the main risk would be early-layer padding contamination.","A direct check of the paper's central assumption is to compare early convolutional feature maps of a photo run alone versus the same photo on a padded canvas; if the overlapping region differs by more than roundoff, padding still shapes the pooled features.","The theme branch could be extended from a one-hot challenge index to textual or attribute-based theme descriptions, which would let the model score photos for unseen themes by interpolating between known criteria.","Editorial flag: the abstract promises an attention-based fusion module and aspect-ratio encoding, but the method text describes only concatenation of visual and theme features; those components need to be specified before the reported architecture is reproducible."],"forward_implications":["Arbitrary-aspect-ratio photos can be scored end to end in normal batches, without resizing, cropping, multi-size training, or batch-size-one training.","Theme-aware conditioning substantially improves standard-deviation prediction, so the model captures some of the spread of human opinions, not only the average score.","Both full-resolution inputs and larger pooled feature maps improve SRCC and distribution distances, confirming that information loss from fixed small inputs hurts aesthetic judgments.","Because ROI align and ROI pooling perform nearly identically here, quantization error from pooling whole-image regions is not a practical concern for this use."],"supporting_citations":[{"why":"Supplies the ROI pooling operation that crops padded features before the classifier.","marker":"[15]"},{"why":"Defines the distribution-prediction baseline and the EMD loss formulation used for training and comparison.","marker":"[18]"},{"why":"Provides the AVA images, challenge themes, and vote distributions that drive the main experiments.","marker":"[40]"},{"why":"Inception-v3 backbone network whose first pooling layer is replaced by ROI pooling.","marker":"[39]"},{"why":"ROI align alternative used to show whole-image pooling quantization error is negligible.","marker":"[35]"},{"why":"Strong prior distribution-prediction method on AVA and Photo.net that the results must beat.","marker":"[19]"},{"why":"Semantic-aware hybrid network baseline for distribution prediction with KL and chi-square metrics.","marker":"[14]"},{"why":"Multi-level spatially pooled feature method whose Inception-v3 result and augmentation strategy are compared.","marker":"[31]"},{"why":"Earlier semantic-label AQA work that motivates conditioning aesthetic assessment on extra information.","marker":"[16]"}],"fun_headline_variants":["ROI pooling avoids pixel damage in full-res aesthetic scoring","Theme-aware aesthetic scores without resizing photos","Padding plus ROI pooling lets full-res photos be judged fairly","Full-resolution aesthetic prediction with a padding-pooling trick"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument assumes that cropping the feature map to the original image's rectangle after the first layers completely removes the influence of the zero border, yet the padding runs through several convolutional layers first and may have already leaked into the image's own features.","fun_headline_variants_meta":{"raw":{"variants":["ROI pooling avoids pixel damage in full-res aesthetic scoring","Theme-aware aesthetic scores without resizing photos","Padding plus ROI pooling lets full-res photos be judged fairly","Full-resolution aesthetic prediction with a padding-pooling trick"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000733,"raw_usage":{"total_tokens":3274,"prompt_tokens":939,"completion_tokens":2335,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":555,"completion_tokens_details":{"reasoning_tokens":2271}},"tokens_in":555,"tokens_out":2335,"duration_ms":19602,"temperature":1.0,"reasoning_tokens":2271,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T15:16:57.889052+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the early convolutional layers of the same network on a photograph alone and on that photograph zero-padded to the training canvas, then compare the two feature maps inside the photograph's rectangle; any difference above numerical roundoff shows that padding still shapes the features the ROI pooling layer keeps.","supporting_citations":[{"cited_title":"Nima: Neural image assessment","cited_arxiv_id":null,"evidence_quote":"Defines the distribution-prediction baseline and the EMD loss formulation used for training and comparison."},{"cited_title":"Ava: A large- scale database for aesthetic visual analysis","cited_arxiv_id":null,"evidence_quote":"Provides the AVA images, challenge themes, and vote distributions that drive the main experiments."},{"cited_title":"A gated peripheral- foveal convolutional neural network for unified image aesthetic predic- tion","cited_arxiv_id":null,"evidence_quote":"Strong prior distribution-prediction method on AVA and Photo.net that the results must beat."},{"cited_title":"Distribution-oriented aesthetics assessment with semantic-aware hybrid network","cited_arxiv_id":null,"evidence_quote":"Semantic-aware hybrid network baseline for distribution prediction with KL and chi-square metrics."},{"cited_title":"Deep aesthetic quality assessment with semantic information","cited_arxiv_id":null,"evidence_quote":"Earlier semantic-label AQA work that motivates conditioning aesthetic assessment on extra information."}],"review_version":1}