Pith. sign in

REVIEW 4 major objections 5 minor 51 references

Discrete Prompt Tuning via Recursive Utilization of Black-box Multimodal Large Language Model for Personalized Visual Emotion Recognition

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Per-user natural-language prompts, refined through a recursive generate-score-refine loop, shift a black-box multimodal model's emotion predictions toward an individual viewer's own labels — the paper's route to personalized visual emotion

desk verdict Honest OPRO-style application to personalized VER; small gains, no significance tests, but worth a referee. read the letter →

arxiv 2509.04480 v1 pith:CCIZXOAG submitted 2025-08-30 cs.CL cs.LG

Ryo Takahashi , Naoki Saito , Keisuke Maeda , Takahiro Ogawa , Miki Haseyama This is my paper
classification cs.CLcs.LG
keywords visualemotionrecognitionpersonalizedmultimodallargelanguagemodelsdiscreteprompttuningblack-boxoptimizationengineeringmajorityvoting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Visual emotion recognition is inherently personal: the same photo can excite one viewer and frighten another, yet multimodal large language models (MLLMs) are trained on general opinions and default to majority reactions. The paper's claim is that this individual bias can be corrected without fine-tuning or seeing the model's internals, by tuning the natural-language prompt itself. An LLM proposes candidate prompts; the black-box MLLM scores each prompt by how often its predicted emotion matches the user's own labels on a training slice of that user's photos; and the best- and worst-scoring prompts are handed back to the LLM to write better ones. After several rounds, the highest-scoring prompts are each run on the target image, majority-voted into a final label, and this recursive discrete prompt tuning reaches 44.9 percent accuracy on 15 Affection users — against 39.1 percent for a zero-shot MLLM prompt and 40.6 percent for a fixed one — while keeping personalization human-readable and requiring no gradient access. The paper itself notes the gains are uneven across emotions: anger barely improves, which it attributes to class imbalance.

What carries the argument

The load-bearing object is the recursive discrete prompt-tuning loop of Algorithm 1, built from two parts. First, an accuracy scorer ACC(p) (Eq. 2): each candidate prompt runs through the MLLM on the user's training images, and the fraction of the user's labels it matches is its score. Second, the LLM modifier t_mod(Ppos, Pneg): the LLM receives the top-k and worst-k prompts with their scores and returns T new prompts, drawn closer to the high-scoring examples and away from the low-scoring ones — an implicit gradient in language space. Iteration builds a prompt pool; the H highest-scoring prompts run on the target image and are majority-voted. The scorer keeps the method black-box compatible

What would settle it

Re-run the full tuning loop on the same 15 Affection users with a different random 30/70 percent split per user. If the tuned prompts' held-out accuracy advantage over the fixed initial prompt (CM4) and the single tuned prompt without voting (CM7) does not reproduce across resplits — or if per-user gains do not exceed the reported between-user standard deviation of about 9.6 percentage points — the gains are artifacts of the training-slice ranking rather than true personalization.

Watch

Extended reading notes

Core claim

The central claim: a black-box multimodal LLM can be personalized per viewer purely by discrete prompt tuning — rewriting the instruction text — in a recursive loop powered by the LLM's self-correction. Each candidate prompt is scored by how often the model's predictions match the user's own labels on a slice of that user's photos. The top-k and worst-k prompts, with their scores, return to the LLM, which writes new prompts more like the good ones and less like the bad ones — an 'implicit gradient' toward the user's optimal prompt. On 15 Affection users this beat every baseline and ablation, indicating genuinely personal prompts rather than random winners.

Load-bearing premise

Each user's final prompts are chosen by ranking candidate prompts on recognition accuracy over a 30% training slice of that user's photos (about 260 images), and the method's effect rests on that ranking transferring to the user's remaining 70% of images — if high training accuracy reflects overfitting to the slice rather than stable user-specific signal, the reported gains over the ablations could be selection noise.

Editorial extensions

If this is right

  • Personalized emotion recognition becomes available for closed, API-only multimodal models: no parameter access, no loss gradients, and no soft-prompt embeddings are required.
  • The tuned prompts beat a zero-shot MLLM prompt (44.9% vs 39.1% accuracy) and a fixed initial prompt (40.6%), so the gain comes from the iterative refinement itself rather than from switching to an MLLM.
  • Prompts tuned to another user underperform user-specific prompts (CM6, 40.4%), so the tuned prompts carry genuinely individual information, not just a generically good instruction.
  • Running five optimized prompts and majority-voting outperforms a single tuned prompt by about 1.8 accuracy points (43.1% → 44.9%), a stability gain the authors attribute to averaging away per-prompt misclassifications.
  • The final prompts remain readable natural language with role instructions (e.g., 'as a choreographer expressing feelings through movement'), so what changed for each user can be inspected and edited by a human.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The decisive check the paper does not run is split-robustness: prompts are chosen by accuracy on one 30% slice of each user's photos, so rerunning the loop on different random splits would show whether the ranking signal is stable or partly selection noise; this is directly testable with the same dataset and models.
  • The same loop should transfer to other subjective visual judgments where a viewer's own response is ground truth — aesthetic preference, humor, trustworthiness, or 'would this appeal to me' — since the mechanism only needs per-user labels and an MLLM that can score prompts.
  • The method's ceiling is set by the two models' abilities rather than by the procedure: a stronger refining LLM or a stronger recognition MLLM should improve the resulting prompts without any change to the algorithm, a scaling prediction that could be tested by swapping either model.
  • Because winning prompts drift toward role-playing instructions (choreographer, emotional interpreter), the loop appears to discover personas as a way around the MLLM's majority-opinion default; making persona induction an explicit objective is a natural, testable extension.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a discrete prompt tuning method for personalized visual emotion recognition (VER) using a black-box multimodal large language model (MLLM). An LLM generates candidate natural-language prompts; each prompt is scored by its accuracy on 30% of a user's training images (Eq. 2); the top-k and worst-k prompts are fed back to the LLM to generate modified prompts iteratively (Algorithm 1); at test time, the H highest-scoring prompts are used for recognition and a majority vote yields the final label (Eqs. 5-6). Experiments on 15 Affection users report accuracy 44.9% ± 9.62%, ECC 63.4% ± 6.61%, outperforming CNN/Transformer baselines (CM1-CM2), a zero-shot MLLM (CM3), and ablation variants (CM4-CM7). The authors conclude that recursive discrete prompt tuning adapts the MLLM to individual users without access to gradients or model parameters.

Significance. If the empirical claims hold, the method offers a practical, interpretable, gradient-free way to personalize black-box MLLMs for VER, and the algorithmic template (LLM-guided prompt generation, evaluation, and iterative refinement) could transfer to other affective computing tasks. The paper has notable strengths: a clear algorithm (Algorithm 1), a held-out test protocol, multiple ablation conditions (CM4-CM7) that isolate initial prompts, one-iteration tuning, cross-user transfer, and majority voting, plus an explicit limitations section. However, the central comparative claims are not yet compelling because the reported gains over the strongest ablations are small relative to per-user variability, no significance testing is reported, and there is no same-budget random-search control that would distinguish the benefit of recursive refinement from the effect of selecting among many random prompts. These are fixable with additional analysis and one baseline experiment.

major comments (4)
  1. [§IV-B, Table 3] The central claim that the proposed method achieves the highest accuracy and ECC rests on mean differences that are not accompanied by any significance testing. The proposed method beats CM7 by only 1.8 accuracy points (44.9 vs 43.1) and 1.6 ECC points (63.4 vs 61.8) while the per-user standard deviations are 9.62 and 6.61, respectively; the gain over CM5 is 3.3/2.5 points. With 15 users and seven comparisons, these gaps may be within noise. Please report per-user paired tests or confidence intervals for the proposed method against CM5 and CM7, and correct for multiple comparisons. Without this, the headline 'highest performance' is not established.
  2. [§III-B, §III-D, Algorithm 1 (Eqs. 2, 5)] The load-bearing premise is that ACC(p) computed on the 30% per-user training split is a reliable selector for the H prompts used on the held-out 70%. Because the pool grows and the algorithm always keeps the argmax over that pool, training accuracy necessarily rises with iterations even if the iterative feedback adds no signal; Fig. 2 shows only training accuracy. The claim in §IV-B that the improvement is 'not simply due to selection of high-accuracy prompts from a set of randomly generated candidates' needs a control that spends the same number of MLLM evaluations on randomly generated prompts (with the same majority-voting protocol) and compares test accuracy. Please add this random-search baseline, and ideally a validation-split-based selection, to separate the contribution of recursive utilization from selection noise.
  3. [§IV-A, hyperparameters N, T, k, I1, I2, I3, H] The method has seven free hyperparameters (N=6, T=5, k=3, I1=20, I2=2, I3=3, H=5), but no sensitivity analysis is reported. Since the final majority vote over H prompts is a major source of the reported gain over CM7, and k controls the LLM's feedback, the results may depend nontrivially on these choices. Please report ablations over at least H and k (and ideally I1), or provide a justification that the chosen values are not performance-critical. Currently the only iteration-related ablation is CM5, which is insufficient to assess robustness.
  4. [§IV-A, user selection] The Affection user pool is restricted to 15 users who provided 'multiple images for each emotion label'; individuals with small or extremely large response counts were excluded. This filters the population and may affect the difficulty of personalization and the magnitude of the reported gains. The paper should state how many users were excluded, analyze the sensitivity of the conclusions to this filter, or at least discuss the potential bias in the limitations section. As written, the personalization claim is limited to a selected moderate-response subset rather than to Affection users generally.
minor comments (5)
  1. [§IV-A, CM5 vs CM7] The definitions of CM5 and CM7 are ambiguous: CM5 is 'a prompt modified in one iteration' but §IV-B states CM5 'included a majority voting approach,' while CM7 is 'a single modified prompt without majority voting.' Please clarify exactly which prompts and voting rule each ablation uses, and make Table 3 annotations consistent.
  2. [Fig. 2 / Table 4] Figure 2 and the accuracies in Table 4 are training-set accuracies. This should be stated explicitly in the captions and in the text, because the reader may otherwise mistake them for held-out performance.
  3. [§IV-A / reproducibility] Please report the exact GPT-4o version, decoding parameters (e.g., temperature), and any random-seed handling for LLM and MLLM inference. Without these, the iterative prompt-generation process is not reproducible.
  4. [§III-C] The phrase 'implicit gradients indicate the direction toward the optimal prompt' is metaphorical and could be misread as a formal gradient in prompt space. Consider rephrasing to describe the LLM's refinement feedback more literally.
  5. [References] Minor formatting issues: 'Y ou' appears in several references (e.g., [33], [35]), 'Proceeding' is inconsistently used, and 'Mikel's wheel' should probably be 'Mikels' wheel' to match [46]. These should be corrected.

Circularity Check

1 steps flagged · score 4.0 of 10

Held-out test evaluation prevents circularity in the headline result, but the Figure 2 evidence for recursive refinement is self-confirming: the plotted training accuracy is the argmax objective over an expanding prompt pool, so its rise is definitional.

  1. other [Section IV-B, Figure 2 discussion; Algorithm 1, lines 7-10 and 18]
    "This transition confirms that the performance improvement in the proposed method is not simply due to the selection of high-accuracy prompts from a set of randomly generated candidates but rather is attributable to the introduction of discrete prompt tuning."

    Algorithm 1 grows Prank on every iteration (line 10: Prank ← Prank ∪ Pmod) and finally selects popt_h = arg max_{pall_m} ACC(pall_m) (Eq. 5, line 18). Therefore the training accuracy of the selected prompt is, by construction, the maximum over a monotonically expanding pool: it cannot decrease even if the modified prompts Pmod are random. Figure 2 plots exactly this selection objective, so it cannot distinguish LLM-guided refinement from mere selection over more candidates. The paper uses this curve to rule out 'the selection of high-accuracy prompts,' but the curve is forced by that selection rule, making the evidence self-confirming.

full rationale

The central quantitative claim is not circular: prompts are scored on the 30% training split via Eq. 2, and the final accuracy, ECC, and EMC in Table 3 are computed on the held-out 70% test split. No test labels enter Algorithm 1, Eq. 5, or the prompt-selection loop, so the headline comparison against CM1-CM7 is an independent empirical evaluation. The paper's framework is based on the external method [29], not on a self-citation chain, and the only prior-work citation from the same group ([11]) is not load-bearing. The genuinely circular piece is the Figure 2 argument: because the plotted training accuracy is the argmax over an expanding prompt pool, its monotone increase is guaranteed by the algorithm's own selection rule and cannot support the claim that recursive discrete prompt tuning, rather than evaluating many prompts, drives the improvement. This is an internal-evidence circularity; the main held-out result retains independent content, hence a moderate rather than extreme score.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on hand-set engineering choices (hyperparameters and prompt templates, Tables 1 and 2) and one empirical assumption: that training-subset accuracy ranks prompts that hold up on held-out images. The method introduces no new entities; 'implicit gradients' in Section III-C is a metaphor for the LLM feedback loop, not a separate mechanism with independent evidence. The EMC metric constant C=4 in Eq. (11) is inherited from prior work [51].

free parameters (2)
  • Prompt-tuning hyperparameters (N, T, k, I1, I2, I3, H) = N=6, T=5, k=3, I1=20, I2=2, I3=3, H=5
    Chosen by hand without sensitivity analysis; no ablation shows the result is robust to these settings (Section IV-A).
  • Per-user optimized prompt set (the fitted object) = e.g., 'As a choreographer expressing feelings through movement...' with 45.4% training accuracy (Table 4)
    Each user's prompts are selected by argmax over training ACC (Algorithm 1, line 18), so the discrete prompt is the fitted object of the method; its held-out test performance is the claim.
assumptions (4)
  • domain assumption ACC(p) on the per-user training subset is a reliable signal for held-out test performance
    All prompt selection uses Eq. (2) on 30% of each user's images; if this ranking overfits, the test gains reported in Section IV-B may be selection noise. No overfitting analysis is provided.
  • domain assumption LLM refinement guided by top-k and worst-k prompts plus accuracy scores improves MLLM performance (the 'implicit gradient' assumption)
    Core mechanism in Section III-C and Table 2; assumes GPT-4o rewrites genuinely increase similarity to good prompts rather than merely resampling the same prompt space.
  • domain assumption MLLMs trained on general data favor majority viewpoints, which is the cause of poor personalized VER
    Motivational premise from refs [22]-[24] in Section I; not directly tested, but it motivates the method rather than appearing in the evaluation.
  • domain assumption Affection dataset emotion labels (Mikels wheel, 8 classes) are valid per-user ground truth
    Dataset premise from [45], [46] in Section IV-A; non-target labels output by the MLLM (e.g., 'happy', 'sad') are counted as incorrect.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Discrete Prompt Tuning via Recursive Utilization of Black-box Multimodal Large Language Model for Personalized Visual Emotion Recognition." pith.science (2026). https://pith.science/paper/CCIZXOAG

@misc{pith2026250904480,
  author       = {Pith},
  title        = {Pith review of: Discrete Prompt Tuning via Recursive Utilization of Black-box Multimodal Large Language Model for Personalized Visual Emotion Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CCIZXOAG}},
  note         = {Machine review of arXiv:2509.04480}
}
read the original abstract

Visual Emotion Recognition (VER) is an important research topic due to its wide range of applications, including opinion mining and advertisement design. Extending this capability to recognize emotions at the individual level further broadens its potential applications. Recently, Multimodal Large Language Models (MLLMs) have attracted increasing attention and demonstrated performance comparable to that of conventional VER methods. However, MLLMs are trained on large and diverse datasets containing general opinions, which causes them to favor majority viewpoints and familiar patterns. This tendency limits their performance in a personalized VER, which is crucial for practical and real-world applications, and indicates a key area for improvement. To address this limitation, the proposed method employs discrete prompt tuning inspired by the process of humans' prompt engineering to adapt the VER task to each individual. Our method selects the best natural language representation from the generated prompts and uses it to update the prompt for the realization of accurate personalized VER.

Figures

Figures reproduced from arXiv: 2509.04480 by the authors.

Figure 1
Figure 1. FIGURE 1: Overview of the proposed method. The proposed [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. FIGURE 2: Transition of recognition accuracy of training [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. FIGURE 3: Examples of target images and their VER results by proposed and comparative methods. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: FIGURE 4: Confusion matrices of VER results obtained from proposed method and CM4. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

51 extracted references · 45 canonical work pages

  1. [29]

    S. Liu, S. Y u, Z. Lin, D. Pathak, and D. Ramanan, ‘‘Language models as black-box optimizers for vision-language models,’’ in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 12 687–12 697

    work page 2024

  2. [1]

    X. Zhu, L. Li, W. Zhang, T. Rao, M. Xu, Q. Huang, and D. Xu, ‘‘Depen- dency exploitation: a unified cnn-rnn approach for visual emotion recog- nition,’’ in Proceedings of the International Joint Conference on Artificial Intelligence, 2017, pp. 3595–3601

    work page 2017

  3. [2]

    L. Xu, Z. Wang, B. Wu, and S. Lui, ‘‘MDAN: Multi-level dependent attention network for visual emotion analysis,’’ in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 9479–9488

    work page 2022

  4. [3]

    Xie, C.-J

    H. Xie, C.-J. Peng, Y .-W. Tseng, H.-J. Chen, C.-F. Hsu, H.-H. Shuai, and W.-H. Cheng, ‘‘Emovit: Revolutionizing emotion insights with visual instruction tuning,’’ in Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, 2024, pp. 26 596–26 605

    work page 2024

  5. [4]

    J. Zhu, S. Zhao, J. Jiang, Z. Xu, W. Tang, and H. Y ao, ‘‘Learning class prototypes for visual emotion recognition,’’ in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing , 2025, pp. 1–5

    work page 2025

  6. [5]

    Poels and S

    K. Poels and S. Dewitte, ‘‘How to capture the heart? reviewing 20 years of emotion measurement in advertising,’’ Journal of Advertising Research, vol. 46, no. 1, pp. 18–37, 2006

    work page 2006

  7. [6]

    C. E. Lopez and C. S. Tucker, ‘‘From mining affective states to mining facial keypoint data: The quest towards personalized feedback,’’ inInterna- tional Design Engineering Technical Conferences and Computers and In- formation in Engineering Conference, vol. 58110, 2017, p. V001T02A039

    work page 2017

  8. [7]

    S. C. Guntuku, D. Preotiuc-Pietro, J. C. Eichstaedt, and L. H. Ungar, ‘‘What twitter profile and posted images reveal about depression and anxiety,’’ in Proceedings of the international AAAI conference on web and social media, vol. 13, 2019, pp. 236–246

    work page 2019

Show all 51 references
  1. [8]

    L. Pang, S. Zhu, and C.-W. Ngo, ‘‘Deep multimodal learning for affective analysis and retrieval,’’ IEEE Transactions on Multimedia, vol. 17, no. 11, pp. 2008–2020, 2015

    2008

  2. [9]

    S. C. Guntuku, J. T. Zhou, S. Roy, W. Lin, and I. W. Tsang, ‘‘Who likes what and, why?’insights into modeling users’ personality based on image ‘likes,’’IEEE Transactions on Affective Computing, vol. 9, no. 1, pp. 130– 143, 2016

    2016

  3. [10]

    Jaiswal, S

    S. Jaiswal, S. Virmani, V . Sethi, K. De, and P . P . Roy, ‘‘An intelligent recommendation system using gaze and emotion detection,’’ Multimedia Tools and Applications, vol. 78, pp. 14 231–14 250, 2019

    2019

  4. [11]

    Moroto, Y

    Y . Moroto, Y . Y e, K. Maeda, T. Ogawa, and M. Haseyama, ‘‘Zero-shot vi- sual sentiment prediction via cross-domain knowledge distillation,’’ IEEE Open Journal of Signal Processing , vol. 5, pp. 177–185, 2023

    2023

  5. [12]

    Kim, Y .-S

    H.-R. Kim, Y .-S. Kim, S. J. Kim, and I.-K. Lee, ‘‘Building Emotional Machines: Recognizing image emotions through deep neural networks,’’ IEEE Transactions on Multimedia, vol. 20, no. 11, pp. 2980–2992, 2018

    2018

  6. [13]

    L. Li, X. Zhu, Y . Hao, S. Wang, X. Gao, and Q. Huang, ‘‘A hierarchical CNN-RNN approach for visual emotion classification,’’ACM Transactions on Multimedia Computing, Communications, and Applications , vol. 15, no. 3s, pp. 1–17, 2019

    2019

  7. [14]

    C. Wu, J. Lei, Q. Zheng, W. Zhao, W. Lin, X. Zhang, X. Zhou, Z. Zhao, Y . Zhang, Y . Wanget al., ‘‘Can GPT-4v (ision) serve medical applications? case studies on GPT-4v for multimodal medical diagnosis,’’ arXiv preprint arXiv:2310.09909, 2023

    2023 arXiv

  8. [15]

    Zhang, K

    C. Zhang, K. Lin, Z. Y ang, J. Wang, L. Li, C.-C. Lin, Z. Liu, and L. Wang, ‘‘MM-Narrator: Narrating long-form videos with multimodal in-context learning,’’ in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 13 647–13 657

    2024

  9. [16]

    Tzelepi and V

    M. Tzelepi and V . Mezaris, ‘‘Disturbing image detection using LMM- elicited emotion embeddings,’’ in Proceedings of the IEEE International Conference on Image Processing Challenges and Workshops , 2024, pp. 4191–4196

    2024

  10. [17]

    Nadeem, S

    M. Nadeem, S. S. Sohail, L. Javed, F. Anwer, A. K. J. Saudagar, and K. Muhammad, ‘‘Vision-enabled large language and deep learning models for image-based emotion recognition,’’Cognitive Computation, vol. 13, pp. 1–14, 2024

    2024

  11. [18]

    Z. Lian, L. Sun, H. Sun, K. Chen, Z. Wen, H. Gu, B. Liu, and J. Tao, ‘‘GPT-4v with Emotion: A zero-shot benchmark for generalized emotion recognition,’’ Information Fusion, vol. 108, pp. 102 367–102 380, 2024

    2024

  12. [19]

    F. Bai, Y . Du, T. Huang, M. Q.-H. Meng, and B. Zhao, ‘‘M3D: Advancing 3D medical image analysis with multi-modal large language models,’’ arXiv preprint arXiv:2404.00578, 2024

    2024 arXiv

  13. [20]

    Zhang, D

    R. Zhang, D. Jiang, Y . Zhang, H. Lin, Z. Guo, P . Qiu, A. Zhou, P . Lu, K.-W. Chang, Y . Qiao, P . Gao, and H. Li, ‘‘MA THVERSE: Does your multi-modal LLM truly see the diagrams in visual math problems?’’ in Proceedings of the European Conference on Computer Vision , 2025, pp...

    2025

  14. [21]

    H. Lu, X. Niu, J. Wang, Y . Wang, Q. Hu, J. Tang, Y . Zhang, K. Y uan, B. Huang, Z. Y u, D. He, S. Deng, H. Chen, Y . Chen, and S. Shan, ‘‘GPT as psychologist? preliminary evaluations for GPT-4v on visual affective VOLUME 11, 2023 9 R. Takahashi et al.: Discrete Prompt Tuning ...

    2023

  15. [22]

    Mirchandani, F

    S. Mirchandani, F. Xia, P . Florence, B. Ichter, D. Driess, M. G. Arenas, K. Rao, D. Sadigh, and A. Zeng, ‘‘Large language models as general pattern machines,’’ inProceedings of The Conference on Robot Learning, vol. 229, 2023, pp. 2498–2518

    2023

  16. [23]

    J. Li, Y . Cao, S. Huang, and J. Chen, ‘‘Formality is favored: Unraveling the learning preferences of large language models on data with conflicting knowledge,’’ in Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2024, pp. 5307–5320

    2024

  17. [24]

    Sheng, K.-W

    E. Sheng, K.-W. Chang, P . Natarajan, and N. Peng, ‘‘The woman worked as a babysitter: On biases in language generation,’’ in Proceedings of the Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing...

    2019

  18. [25]

    X. Liu, K. Ji, Y . Fu, W. Tam, Z. Du, Z. Y ang, and J. Tang, ‘‘P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks,’’ in Proceedings of the Annual Meeting of the Association for Computational Linguistics, 2022, pp. 61–68

    2022

  19. [26]

    X. L. Li and P . Liang, ‘‘Prefix-tuning: Optimizing continuous prompts for generation,’’ in Proceedings of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing, 2021, pp. 4582–4597

    2021

  20. [27]

    Y . Gu, X. Han, Z. Liu, and M. Huang, ‘‘PPT: Pre-trained prompt tuning for few-shot learning,’’ in Proceedings of the Annual Meeting of the Associa- tion for Computational Linguistics , 2022, pp. 8410–8423

    2022

  21. [28]

    Lester, R

    B. Lester, R. Al-Rfou, and N. Constant, ‘‘The power of scale for parameter- efficient prompt tuning,’’ in Proceedings of the Conference on Empirical Methods in Natural Language Processing , 2021, pp. 3045–3059

    2021

  22. [30]

    S. Diao, Z. Huang, R. Xu, X. Li, Y . Lin, X. Zhou, and T. Zhang, ‘‘Black- box prompt learning for pre-trained language models,’’ arXiv preprint arXiv:2201.08531, 2022

    2022 arXiv

  23. [31]

    Pryzant, D

    R. Pryzant, D. Iter, J. Li, Y . Lee, C. Zhu, and M. Zeng, ‘‘Automatic prompt optimization with ‘‘gradient descent’’ and beam search,’’ in Proceedings of the Conference on Empirical Methods in Natural Language Processing , 2023, pp. 7957–7968

    2023

  24. [32]

    K.-C. Peng, T. Chen, A. Sadovnik, and A. C. Gallagher, ‘‘A mixed bag of emotions: Model, predict, and transfer emotion distributions,’’ in Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2015, pp. 860–868

    2015

  25. [33]

    Q. Y ou, J. Luo, H. Jin, and J. Y ang, ‘‘Building a large scale dataset for image emotion recognition: The fine print and the benchmark,’’ in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 30, no. 1, 2016

    2016

  26. [34]

    T. Chen, D. Borth, T. Darrell, and S.-F. Chang, ‘‘DeepSentiBank: Visual sentiment concept classification with deep convolutional neural networks,’’ arXiv preprint arXiv:1410.8586, 2014

    2014 arXiv

  27. [35]

    Q. Y ou, J. Luo, H. Jin, and J. Y ang, ‘‘Robust image sentiment analysis using progressively trained and domain transferred deep networks,’’ in Proceedings of the AAAI conference on Artificial Intelligence , vol. 29, no. 1, 2015

    2015

  28. [36]

    D. She, J. Y ang, M.-M. Cheng, Y .-K. Lai, P . L. Rosin, and L. Wang, ‘‘WSCNet: Weakly supervised coupled networks for visual sentiment clas- sification and detection,’’IEEE Transactions on Multimedia, vol. 22, no. 5, pp. 1358–1371, 2020

    2020

  29. [37]

    S. Deng, L. Wu, G. Shi, L. Xing, W. Hu, H. Zhang, and Y . Xiang, ‘‘Simple but powerful, a language-supervised method for image emotion classification,’’ IEEE Transactions on Affective Computing , vol. 14, no. 4, 2023

    2023

  30. [38]

    Bustos, C

    C. Bustos, C. Civit, B. Du, A. Solé-Ribalta, and A. Lapedriza, ‘‘On the use of vision-language models for visual sentiment analysis: a study on clip,’’ in Proceedings of the International Conference on Affective Computing and Intelligent Interaction, 2023, pp. 1–8

    2023

  31. [39]

    B. Wang, G. Tu, B. Liang, Z. Bai, M. Y ang, X. Zeng, L. Y ao, and R. Xu, ‘‘Enhancing emotion reasoning for image multi-emotion prediction,’’ in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 2025, pp. 1–5

    2025

  32. [40]

    Q. Xu, Y . Wei, S. Y uan, J. Wu, L. Wang, and C. Wu, ‘‘Learning emotional prompt features with multiple views for visual emotion analysis,’’ Infor- mation Fusion, vol. 108, p. 102366, 2024

    2024

  33. [41]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P . Mishkin, J. Clark, G. Krueger, and I. Sutskever, ‘‘Learning transferable visual models from natural language supervision,’’ in Proceedings of the International Conference on Machine Lea...

    2021

  34. [42]

    S. Deng, L. Wu, G. Shi, L. Xing, M. Jian, Y . Xiang, and R. Dong, ‘‘Learning to compose diversified prompts for image emotion classification,’’Compu- tational Visual Media, vol. 10, no. 6, pp. 1169–1183, 2024

    2024

  35. [43]

    W. Dai, J. Li, D. LI, A. Tiong, J. Zhao, W. Wang, B. Li, P . N. Fung, and S. Hoi, ‘‘InstructBLIP: Towards general-purpose vision-language models with instruction tuning,’’ in Proceedings of the Advances in Neural Infor- mation Processing Systems, vol. 36, 2023, pp. 49 250–49 267

    2023

  36. [44]

    Cheng, X

    J. Cheng, X. Liu, K. Zheng, P . Ke, H. Wang, Y . Dong, J. Tang, and M. Huang, ‘‘Black-box prompt optimization: Aligning large language models without model training,’’ arXiv preprint arXiv:2311.04155, 2024

    2024 arXiv

  37. [45]

    Achlioptas, M

    P . Achlioptas, M. Ovsjanikov, L. Guibas, and S. Tulyakov, ‘‘Affection: Learning affective explanations for real-world visual data,’’ inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 6641–6651

    2023

  38. [46]

    J. A. Mikels, B. L. Fredrickson, G. R. Larkin, C. M. Lindberg, S. J. Maglio, and P . A. Reuter-Lorenz, ‘‘Emotional category data on images from the international affective picture system,’’Behavior research methods, vol. 37, no. 4, pp. 626–630, 2005

    2005

  39. [47]

    Korovina, M

    O. Korovina, M. Baez, and F. Casati, ‘‘Reliability of crowdsourcing as a method for collecting emotions labels on pictures,’’ BMC research notes, vol. 12, pp. 1–6, 2019

    2019

  40. [48]

    Huang, J

    X. Huang, J. Xu, W. Zheng, Q. Mao, and A. Dhall, ‘‘A survey of deep learning for group-level emotion recognition,’’ arXiv preprint arXiv:2408.15276, 2024

    2024 arXiv

  41. [49]

    H. Liu, C. Li, Q. Wu, and Y . J. Lee, ‘‘Visual instruction tuning,’’ in Proceedings of the Advances in Neural Information Processing Systems , vol. 36, 2023, pp. 34 892–34 916

    2023

  42. [50]

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, ‘‘Swin transformer: Hierarchical vision transformer using shifted windows,’’ in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 10 012–10 022

    2021

  43. [51]

    C. Zhao, J. Shi, L. Nie, and J. Y ang, ‘‘To err like human: Affective bias- inspired measures for visual emotion recognition evaluation,’’ in Proceed- ing of the Advances in Neural Information Processing Systems , 2024, pp. 134 747–134 769. RYO TAKAHASHI (Graduate Student Memb...

    2024

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.