{"id":"72f4ec7a-721a-4a7a-9c0c-ca04c92fc9a7","arxiv_id":"2607.15752","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"PRAC mines preference-rich images and merges LoRA adapters from aesthetically similar users to achieve state-of-the-art personalized aesthetic rating prediction.","lead":"This paper introduces PRAC, a method that personalizes photo-aesthetic ratings by first picking the images most revealing of a user's taste and then merging model adjustments from users with similar preferences. Across four benchmarks it reports state-of-the-art accuracy with only 10–100 user ratings, plus textual reasons for rating disagreements.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"PDM is undefined for three of four benchmark datasets; PRAC's reported gains on FLICKR-AES/AADB/REAL-CUR are not reproducible from the described method.","rationale":"The reader's weakest assumption identified PDM as relying on an unvalidated mapping from textual profiles to real aesthetic judgments, and noted that PDM cannot be computed for datasets without user profiles. My stress-test focuses on the latter, more concrete problem: the paper's own method definition makes PDM impossible on three of the four datasets where results are reported. This is an internal inconsistency between Section 3.2 and the experiments, not a matter of MLLM capability. It directly affects the interpretation of the central 'consistent SOTA' claim because the components credited for the gains are not specified for most benchmarks. The reader's verdict of CONDITIONAL remains appropriate; my analysis strengthens the condition by specifying exactly what must be clarified or validated. No new verdict change is needed, hence UNCHANGED. I did not identify a more severe flaw (e.g., evidence of data leakage or contradictory numbers), so a rejection is not warranted by this concern alone.","tokens_in":15058,"tokens_out":4905,"duration_ms":40676,"concrete_test":"Re-run the Table 6 ablation on FLICKR-AES with PreferSelect implemented as described when no user profile exists: set PDM=0 so Pscore=(1−α)·CCM with α=0.3, and compare to the reported 100-shot SRCC of 0.778. If the CCM-only variant reproduces 0.778, the paper must disclose that PDM is not used for profile-less datasets; if it does not reproduce, the reported result cannot be obtained from the described method.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—PRAC 'consistently achieves the best performance across all datasets'—relies on PreferSelect's Personalized Deviation Metric (PDM), defined in Eq. 3 as the KL divergence between the generic aesthetic distribution p_i and the profile-conditioned distribution p_i^u obtained from the prompt 'You are a <User Profile>'. PDM therefore requires user profiles. Section 4.1 states that only PARA provides rich user profiles; FLICKR-AES, AADB, and REAL-CUR do not. Yet Tables 3–5 report PRAC results on all three, and Table 6 attributes significant improvements to PreferSelect on those datasets. The paper never specifies how PDM is computed without profiles. If PDM is omitted, Pscore (Eq. 4) reduces to (1−α)·CCM, making the α=0.3 tuning from Figure 5 meaningless for those datasets. If an implicit proxy is used, it is not described. Either way, the method as written cannot be applied to three of four evaluation datasets, and the 'consistent' gains cannot be traced to the claimed dual-metric sample mining. This is a concrete reproducibility gap, not merely an assumption about MLLM alignment.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PRAC, an MLLM-based framework for personalized image aesthetic assessment (PIAA). PRAC has two main components: PreferSelect, which ranks a user's available annotated images by a preference-richness score combining Collective Controversy (CCM) and Personalized Deviation (PDM), and PreferMerge, which finds 'aesthetically-resonant' users via Fisher-information-derived preference embeddings and merges their LoRA adapters to improve the target user's model. The method is evaluated on PARA, FLICKR-AES, AADB, and REAL-CUR in 10-shot and 100-shot settings, reporting SRCC gains over prior PIAA methods, with ablations and computational-cost comparisons. The central claim is that PRAC consistently achieves the best performance across all four benchmarks.","tokens_in":15375,"tokens_out":3944,"duration_ms":32214,"significance":"If the method is fully specified and the reported gains are reproducible, PRAC would be a meaningful advance in few-shot PIAA: it reframes sample selection as a dual-metric preference-richness problem and introduces cohort-based LoRA merging, which is a timely and interpretable use of MLLMs for aesthetic personalization. The paper provides broad comparisons on four datasets, ablations of both components, qualitative rationales for divergent judgments, and a computational-cost analysis. The idea that certain images are more informative for personalization, and that similar users can be found from gradient-based preference embeddings, is well motivated by cognitive-aesthetics literature. However, the main results currently rest on a reproducibility gap: PDM is defined only when rich user profiles exist, yet three of the four benchmark datasets lack such profiles, and the manuscript does not explain how PreferSelect operates there. This must be resolved before the cross-dataset claims can be accepted.","major_comments":[{"comment":"PDM is defined in Eq. (3) as the KL divergence between the generic aesthetic distribution and a distribution obtained from the profile-conditioned prompt 'You are a <User Profile>'. Section 4.1 states that only PARA provides rich user profiles; FLICKR-AES, AADB, and REAL-CUR do not. Nevertheless, Tables 3–5 report PRAC on all three datasets, and Table 6 attributes substantial gains to PreferSelect on those datasets. The manuscript does not specify how PDM is computed in the absence of profiles. If PDM is omitted, Pscore in Eq. (4) reduces to (1−α)·CCM, making the tuned α=0.3 from Figure 5 meaningless for those datasets; if an implicit proxy is used, it is not described. As written, the method cannot be applied to three of the four evaluation benchmarks, so the abstract's claim of 'consistent' state-of-the-art performance and the attribution of gains to the dual-metric PreferSelect are no","section":"Sec. 3.2, Eqs. (3)–(4); Tables 3–6"},{"comment":"The hyperparameters α, K, and β are selected via ablations on PARA, but no held-out validation split is described. Since PARA has fixed testing users (Table 2), tuning on the same testing distribution risks selection on the evaluation set. In addition, no error bars, standard deviations, or significance tests are reported on any main table. The baseline in Table 6 is said to be averaged over 10 random runs, but the variance is not given. This matters for small margins such as the 10-shot PARA comparison (PRAC 0.707 vs PIAA-MIR 0.702) and the ablation increment 0.640 vs 0.634 in Table 6. Please report mean±std over repeated trials and define a validation protocol for hyperparameter selection.","section":"Sec. 4.3, Figures 5–6"},{"comment":"PreferMerge requires solving a combinatorial subset-selection problem over the User Pool in Eq. (7), but the manuscript does not state how this optimization is performed in practice. It also does not specify whether the Fisher Information Matrix F_u is approximated by a diagonal or empirical Fisher, how the preference embedding is reduced to a comparable vector, or how the merging weights w_i in Eq. (8) are computed from the similarities in Eq. (6). These implementation details are necessary to reproduce the cohort-merging step, particularly on PARA with 398 training users, and to understand the computational cost reported in Figure 9.","section":"Sec. 3.3, Eqs. (5)–(8)"}],"minor_comments":[{"comment":"The rows 'PARA (unconditional)', 'PARA (artistic)', 'PARA (photographic)', and 'PARA (personality)' are listed among methods but appear to be variants or protocol settings, not prior PIAA methods. Please clarify in the caption or re-organize the table.","section":"Table 2 caption"},{"comment":"The subset notation should use S ⊆ P rather than S ⊂ P, since S can equal the full pool if K equals |P|.","section":"Eq. (7)"},{"comment":"Reference [16] gives the arXiv ID as '2307.117606'; the correct EmotionPrompt ID appears to be '2307.11760'. Please verify.","section":"References"},{"comment":"For REAL-CUR, the table shows no training images/users, so the evaluation is inherently cross-database. The description in Section 4.1 could state this more explicitly, since the later cross-database experiment relies on it.","section":"Sec. 4.1, REAL-CUR"},{"comment":"The conclusion calls PRAC 'the first MLLM-based PIAA model'; this is a strong novelty claim that is not formally supported by the related-work survey. Consider softening it.","section":"Sec. 5"}],"recommendation":"major_revision","confidential_remarks":"The decisive issue is the PDM gap for FLICKR-AES, AADB, and REAL-CUR. If the authors cannot provide a concrete description of how PreferSelect runs on profile-free datasets, the cross-dataset results and the abstract's 'consistent' claim should be substantially revised. The other major concern is statistical rigor: the reported gains are often small in absolute terms and no significance testing is provided. With those issues addressed, the paper could make a solid contribution to the PIAA literature."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"PRAC is worth a serious referee: it is a competent MLLM-based personalization method with consistent gains on four PIAA benchmarks, but the write-up has a real reproducibility gap — the Personalized Deviation Metric needs user profiles, and three of the four datasets don't have them.\n\nWhat's genuinely new is the packaging: PreferSelect ranks a user's few annotated images by a mix of collective controversy (standard deviation of a generic aesthetic distribution) and personalized deviation (KL from a profile-conditioned MLLM prompt), then PreferMerge fine-tunes LoRAs on those samples, uses Fisher-information gradients as preference embeddings, and merges the LoRAs of the K most similar users. That combination is not in the prior PIAA literature. The ablations are decent: each component adds a few points of SRCC on each dataset, and the four benchmark numbers are consistently above the previous best (e.g., 0.707/0.733 vs 0.702/0.716 on PARA). The qualitative examples and compute comparison are a plus.\n\nThe soft spots are proportionate. The biggest one is that PDM, as defined in Eq. 3, cannot be computed for FLICKR-AES, AADB, or REAL-CUR because they have no user profiles. Yet Tables 3–5 report full PRAC results there, and Table 6 attributes improvement to PreferSelect on those datasets. If PDM is omitted, Pscore in Eq. 4 collapses to (1−α)CCM, and the α=0.3 tuning in Figure 5 loses its meaning for those datasets. The paper never says how they handled this — maybe they used a default profile or dropped PDM and re-tuned α, but that is not in the text. This is a concrete reproducibility gap, not a philosophical worry. A reviewer should ask for the exact protocol.\n\nTwo smaller things: no error bars or significance tests anywhere, so the headline 'consistently best' rests on raw SRCC deltas that are sometimes small (e.g., 0.692 vs 0.668 on FLICKR-AES). And the hyperparameters α, K, β are chosen via ablation on PARA with no described held-out split, so there is at least a selection-risk smell, though the gains on other datasets make it plausible the method generalizes.\n\nOverall, the paper is coherent on its own terms and the authors clearly know the literature. It just needs to specify how PreferSelect runs without profiles, and ideally add variance estimates. I'd send it to peer review, and I'd want those issues resolved before trusting the numbers.","headline":"Solid PIAA method with consistent gains, but the profile-dependent PDM is undefined for three of the four benchmarks, so the reported results are not yet reproducible.","tokens_in":15817,"tokens_out":2507,"would_cite":true,"duration_ms":20833,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PRAC improves personalized aesthetic rating prediction by mining preference-rich images and merging similar users' models.","keywords":["personalized image aesthetic assessment","multimodal large language model","preference-rich sample mining","cohort merging","few-shot learning","Fisher information matrix","LoRA","subjective aesthetics"],"falsifier":"Swap user profiles between users during PreferSelect: if SRCC does not drop meaningfully, PDM is not using profile information and the reported gains must come from somewhere else. A cleaner direct check is to have users rate a large pool of images and test whether PDM-ranked images match images where users actually disagree.","tokens_in":14980,"feed_emoji":"🎨","tokens_out":5542,"duration_ms":42908,"temperature":0.7,"pith_summary":"This paper argues that the bottleneck in personalized image aesthetic assessment under few-shot conditions is not just model capacity but data selection: some images carry far more information about a particular user's taste than others, and some other users' tastes are far more relevant than the rest. To exploit this, it introduces PRAC, a two-stage procedure built on a multimodal large language model. PreferSelect scores each candidate image by collective controversy (how much public ratings diverge) plus personalized deviation (how far the user's predicted rating is from consensus), then fine-tunes a user-specific LoRA on the top-scoring images. PreferMerge represents each user by the Fisher information of their LoRA gradients, finds a small cohort of aesthetically similar users, and merges their LoRA weights. Across PARA, FLICKR-AES, AADB, and REAL-CUR, PRAC reports the best Spearman correlation in both 10-shot and 100-shot settings, and it also produces textual rationales for divergent judgments.","feed_headline":"Mine divisive images, merge user cohorts for best taste prediction","feed_subtitle":"A two-step MLLM pipeline beats previous personalized aesthetic rating methods on four benchmarks with only 10-100 user labels.","key_machinery":"The engine is a two-metric selection score, Pscore, combined with a cohort-merging rule. Pscore combines CCM, the standard deviation of the model's predicted public aesthetic rating distribution, with PDM, the KL divergence between the generic distribution and the distribution the model gives when the prompt is conditioned on the user's textual profile (age, gender, expertise, Big-Five traits). The selected high-scoring images form the user's query set. Each user is then represented by a Fisher Information Matrix computed from the gradients of a LoRA fine-tuned on that query set; cosine similarity between these matrices measures cross-user taste similarity. PreferMerge selects K cohort membe","core_discovery":"PRAC, a multimodal large language model personalized aesthetic assessment framework, reports state-of-the-art Spearman rank correlation on all four benchmarks. On PARA it reaches 0.707 (10-shot) and 0.733 (100-shot), beating previous bests of 0.702 and 0.716; on FLICKR-AES 0.692/0.778 versus 0.668/0.748; on AADB 0.597/0.671; and on the cross-database REAL-CUR test 0.585/0.631. The claim is that these gains come from two targeted operations: selecting images whose ratings best expose a user's taste (high collective controversy plus high personalized deviation from public consensus), and merging the fine-tuned models of a small cohort of users whose preference embeddings are most similar, whil","pith_inferences":["If profile-conditioned distributions approximate true user taste, the same PDM-style metric could rank content for other subjective tasks such as recommendation or moderation, where disagreement is informative.","A natural extension is an online version: update a user's Fisher-embedding as new ratings arrive, recompute the cohort, and re-merge LoRAs, turning the few-shot model into a continuously adapting one.","Because Fisher-information similarity does not depend on the concrete rating scale, the cohort-merging step could in principle operate across datasets with different scales, enabling cross-platform personalization.","A direct test would compare cohort selection by Fisher-embedding similarity against selection by observed agreement on a held-out set of divisive images; if the two disagree often, the embedding similarity may be capturing model artifacts rather than taste."],"forward_implications":["With as few as 10 user-rated images, PRAC beats previous personalized aesthetic assessment methods on PARA, FLICKR-AES, and AADB in the paper's experiments.","At 100 shots, the margin over prior best grows on all four benchmarks, indicating that both sample selection and cohort merging scale with data.","PreferSelect alone improves over randomly sampled fine-tuning; adding PreferMerge adds a further gain on every dataset, so both mechanisms are load-bearing.","The cross-database results show that a model trained on one dataset's users transfers to unseen users on other datasets, suggesting the pipeline captures transferable preference structure.","Because the MLLM is prompted with a user profile, PRAC can provide a written rationale for a rating, making the prediction interpretable."],"fun_headline_variants":["Few user labels: mine controversial images, merge taste cohorts for PIAA","Mine disagreeable photos, merge like-minded users to predict individual aesthetics","PRAC: use disputed images and merge similar users for personalized aesthetics","Disputed images expose taste; merging cohorts personalizes aesthetics with few labels","Controversial images beat average taste; merge models of like-minded raters"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The Personalized Deviation Metric assumes that prompting the generic predictor with a user's demographic and personality profile yields a rating distribution close enough to that user's real judgments to identify which images are actually informative.","fun_headline_variants_meta":{"raw":{"variants":["Few user labels: mine controversial images, merge taste cohorts for PIAA","Mine disagreeable photos, merge like-minded users to predict individual aesthetics","PRAC: use disputed images and merge similar users for personalized aesthetics","Disputed images expose taste; merging cohorts personalizes aesthetics with few labels","Controversial images beat average taste; merge models of like-minded raters"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001139,"raw_usage":{"total_tokens":4571,"prompt_tokens":758,"completion_tokens":3813,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":502,"completion_tokens_details":{"reasoning_tokens":3727}},"tokens_in":502,"tokens_out":3813,"duration_ms":20044,"temperature":1.0,"reasoning_tokens":3727,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T22:25:01.040352+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Swap user profiles between users during PreferSelect: if SRCC does not drop meaningfully, PDM is not using profile information and the reported gains must come from somewhere else. A cleaner direct check is to have users rate a large pool of images and test whether PDM-ranked images match images where users actually disagree.","supporting_citations":[],"review_version":1}