Pith. sign in

REVIEW 4 major objections 6 minor 68 references

ArtAnno: Annotating Implicit Semantics in Artworks through LLM Agent-Driven Bidirectional Human-AI Augmentation

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A bidirectional human-AI augmentation loop, implemented as the multi-agent system ArtAnno, cuts artwork annotation time nearly in half and raises expert label agreement from 73% to 90%.

desk verdict A real system with a coherent bidirectional-augmentation framework, but the headline efficiency gain is confounded by the within-subject design and the ablation isolating evolution is non-significant. read the letter →

arxiv 2608.05026 v1 pith:IGTNDOH3 submitted 2026-08-05 cs.HC cs.AI

Xiaoyan Gu , Yifang Wang , Wenqing Zheng , Haozhong Liu , Yixia Zheng , Peiyi Jiang , Wenjie Ning , Wei Zhang
show 1 more author
Wei Chen
This is my paper
classification cs.HCcs.AI
keywords artworkannotationimplicitsemanticsbidirectionalhuman-AIaugmentationmulti-agentsystemslargelanguagemodelsskilllibraryevolutioncollaborationculturalknowledge
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Artwork annotation normally requires reading what pixels alone do not show: culturally grounded symbols, such as spilled salt in "The Last Supper" signaling betrayal, a task that demands both time and art-historical knowledge. The paper sets out to show that a bidirectional human-AI loop can remove that barrier: an AI subsystem proactively recommends labels, clusters, and evidence while a second subsystem distills the annotator's corrections and searches into reusable skills and knowledge that improve later recommendations. This claim is operationalized in ArtAnno, a multi-agent system built on the proposed BiHAA (Bidirectional Human-AI Augmentation) framework. In a controlled study with 12 annotators who lacked art expertise, the full system reduced mean annotation time from 30.92 to 15.75 minutes, a roughly 50% reduction (p=.00049), and raised agreement with expert reference labels from 22/30 (73%) to 27/30 (90%). If the claim holds, knowledge-intensive annotation becomes accessible to non-experts, and the act of annotating generates reusable expertise instead of consuming it.

What carries the argument

The load-bearing mechanism is the bidirectional loop itself, which the paper names BiHAA: two multi-agent subsystems that write into each other. In one direction, a Skill Recommendation Agent selects relevant skills, a Cluster and Pattern Mining Agent groups images and summarizes shared patterns, a Label Recommendation Agent matches cluster patterns against label definitions to produce ranked suggestions with rationales, and a Chatbot Agent answers verification questions; a priority score $P(c)=\lambda_{1}S_{\text{uncertainty}}(c)+\lambda_{2}S_{\text{task}}(c)+\lambda_{3}S_{\text{impact}}(c)$ decides which clusters are surfaced first. In the other direction, a Behavior Mining Agent extracts candidate skills from interaction logs, a Skill Generation Agent writes each as a structured document (Overview, Preconditions, Workflow, Best Practices, Examples, Troubleshooting), a Skill Management Agent merges near-duplicate skills by structure alignment and module-level diffing, and a Summary Agent with a Memory Management Agent update the knowledge base. The loop closes because the skills and knowledge written by the second module feed the recommendations and prioritization of the first, which is what makes the claimed gains cumulative across rounds within a session.

What would settle it

Run the system on a fresh set of artworks with the skill library pre-seeded with a deliberately wrong but plausible skill (for example, treating front-back spatial arrangement as implying subordination, the very error the paper's Case 1 corrects). If a new annotator who follows the recommended labels ends up with lower agreement against the expert reference than an unseeded control group, the evolution loop demonstrably propagates error rather than filtering it; if agreement stays at the unseeded level, human verification is doing the filtering that the paper assumes.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that the familiar one-way division—AI assisting humans, or humans calibrating AI in separate stages—is the real bottleneck, and that a reciprocal loop removes it. ArtAnno embodies the loop as two coupled modules: the Proactive Agentic Support Module supplies cluster-level label recommendations with rationales, background knowledge, and visual-pattern evidence, while the Interaction-Driven Evolution Module mines logged user actions into structured skills, merges overlapping skills with a structure-aware algorithm, and deposits domain knowledge into a knowledge base. The main empirical results are that mean annotation time falls from 30.92 to 15.75 minutes (p=.00049) and label agreement with the expert reference rises from 73% to 90%; an ablation shows the evolution module alone contributes a smaller, non-significant time gain (17.25 to 15.75 minutes, p=.088) but a significant gain in user-rated experience. The case studies supply the qualitative mechanism: a user's correction about spatial arrangement versus functional action becomes a reusable skill that later guides another user to a correct equal-status reading, and a cross-cultural misunderstanding about skulls in Mexican posters refines an explanation skill into a context-aware, contrastive form.

Load-bearing premise

The system's evolution module assumes that logged interactions contain enough correct signal to distill into reusable skills, but because those logs carry no correctness markers, a wrong label or faulty correction can be distilled, stored, and recommended again as if it were expertise—and this can happen even while the proactive-support half of the system keeps delivering genuine efficiency gains.

Editorial extensions

If this is right

  • Annotators without art training can reach near-expert labeling of implicit cultural semantics: agreement with the expert reference rose from 22/30 (73%) to 27/30 (90%) in roughly half the time.
  • The evolution module consolidates rather than accumulates: 112 raw candidate skills merged down to 31, a 72% reduction, while users reported no extra operational burden.
  • Learned skills transfer within a session: the skill distilled from one user's correction about spatial position versus functional action guided a later user to a correct equal-status reading of a similar composition.
  • Recommendation quality improves without retraining: users who corrected or questioned the system reported that earlier errors did not reappear in later tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test the paper does not run: seed the skill library with a deliberately wrong skill (say, the Western 'skulls mean horror' reading from the poster case) and check whether fresh annotators who trust the recommendations propagate the error; if they do, the loop amplifies exactly the bias the paper names in its limitations.
  • The headline speedup bundles proactive support and evolution together, and because the evolution-only contrast was not statistically significant on time (p=.088), whether fast annotation persists across sessions is an open question the paper's within-session data cannot yet answer.
  • If the behavior-mining pipeline is as domain-agnostic as the paper argues, the same loop should transfer to medical image or historical document annotation with comparable margins; that is a testable prediction the paper leaves open.
  • A side effect the paper does not measure: novices who follow expert-derived skills may internalize the heuristics themselves, so the annotation session could double as a training intervention even if the AI were later removed.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces BiHAA, a bidirectional human-AI augmentation framework for artwork annotation, and ArtAnno, a multi-agent system implementing it with proactive agent support and interaction-driven evolution. The authors report a formative study with 20 annotators, a within-subject user study with 12 participants across three conditions, and two case studies. The main empirical claims are that ArtAnno reduces annotation time by nearly 50% versus a baseline (M=15.75 vs 30.92 minutes, p=.00049) and improves label agreement from 73% to 90% (p=.00049), while an ablation shows a non-significant time improvement (C2 vs C3, p=.088) but a claimed significant user-experience benefit. The paper also claims the system accumulates reusable knowledge and skills within a session.

Significance. The contribution is potentially valuable: it operationalizes a closed-loop HAI framework, uses a structured skill library, and evaluates with human participants and expert-resolved reference labels. The formative study and the explicit limitations section are strengths. However, the load-bearing efficiency and knowledge-accumulation claims are not currently supported by the reported statistics and experimental design. The central comparison suffers from a within-subject learning confound, and the evolution-module benefit rests on ambiguous and non-significant ablation results. If these issues are resolved, the system's direction is significant for HCI and computational art research.

major comments (4)
  1. [§5.1 Efficiency] The headline comparison between C3 and C1 is reported without naming the statistical tests, test statistics, standard deviations, confidence intervals, or effect sizes. The same p-value, p=.00049, is reported for both annotation time and label agreement; with N=12 this is surprising and suggests either a reporting error or an underspecified analysis. Please report exact tests (e.g., paired t-test or Wilcoxon signed-rank test), test statistics, per-condition M and SD, and effect sizes for both measures, and clarify whether the two reported p-values came from the same or different analyses.
  2. [§5.1 Data and Task] The within-subject design confounds the headline comparison. Participants completed all three conditions, and C1 allowed external search engines and AI tools including GPT-5, while C2 and C3 disabled external tools. The three image groups are described only as 'comparable difficulty' with no supporting evidence, and no per-order statistics or condition-by-order interaction are reported. Any practice effect or transfer of domain knowledge from earlier conditions to later ones can affect conditions unequally, and the claimed 50% time reduction cannot be separated from these effects. Please provide per-condition and per-order descriptive statistics and either a condition-by-order analysis or a first-condition-only comparison.
  3. [§5.1 System Evolution] The ablation evidence for the evolution module is not sufficient. C2 vs C3 annotation time is reported as non-significant (p=.088), yet the text states 'C3 was rated significantly higher than C2 (p=.00049)' without identifying the dependent variable, the questionnaire items, the scale, or the test used. The skill-count statistics (112 raw, 31 retained) demonstrate that the pipeline operates but do not measure whether the produced skills are correct or useful. Please specify the measure and test, report per-condition means and SDs, and provide quantitative evidence of skill quality or correctness.
  4. [§6.3 Limitations] The limitation that interaction sequences carry no correctness markers and that noisy actions such as mislabeling may propagate into learned skills is directly relevant to the knowledge-accumulation claim. The case studies are anecdotal and do not quantify whether the system's self-generated and self-merged skills are accurate. As a result, the claim that the system 'enable[s] knowledge accumulation' is not yet established. Please add expert validation or an error-propagation analysis for the learned skills, or substantially weaken the claim.
minor comments (6)
  1. [§5.1 Data and Task] The phrase 'comparable difficulty' should be operationalized; please include difficulty ratings or baseline annotation times for each image group.
  2. [§4.3.1 Eq. (1)] The lambda weights are compared across three configurations, but the threshold theta in Algorithm 1 is not specified or subjected to sensitivity analysis.
  3. [§5.1 System Usability] The overall SUS score is reported as 84.17, but per-condition SUS scores and standard deviations are not reported, making it impossible to compare usability across C1, C2, and C3.
  4. [Running header] The running header on several pages reads 'Trovato et al.' although the author list is Gu et al.; this appears to be a template error that should be corrected.
  5. [Figure 4] Figure 4 uses stacked bars with small counts; the figure would benefit from explicit counts and standard deviations for each Likert item.
  6. [Figures 2 and 5] The extracted interface text contains repeated 'NgO' tokens (e.g., Fig. 2 and Fig. 5); these appear to be rendering artifacts and should be cleaned.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the headline efficiency gain is a measured outcome against an external baseline, and the self-referential skill-count evidence is a validity limitation, not a circular derivation.

full rationale

The paper's central quantitative claims are empirical measurements rather than derivations from the system's own assumptions. The efficiency result (M=15.75 vs 30.92 minutes, p=.00049) is a user-study observation comparing the full system against a baseline that allows external search/AI tools, and label agreement is scored against expert-resolved reference labels constructed independently of the system. No equation in the paper derives these numbers from fitted parameters, and no prediction is renamed from an input. The evolution-module evidence is weaker: Section 5.1 cites the system's own count of 112 raw skills reduced to 31 as evidence of consolidation, and Section 6.3 explicitly concedes that "interaction sequences carry no correctness markers, noisy actions such as mislabeling may propagate into learned skills." That is a legitimate validity limitation and a reason to treat the knowledge-accumulation claim cautiously, but it is not circularity under the definitions used here, because the claim is not obtained by construction from its inputs and the module's benefit also rests on subjective UX ratings and case-based examples. Self-citations (e.g., refs. [9], [39], [41], [42], [57]) appear in background and motivation sections and are not load-bearing for the headline efficiency comparison, which is anchored by the external baseline and expert labels. The within-subject design and shared GPT-5 usage between C1 and C3 could confound the time comparison, but that is an experimental-control concern rather than a circular derivation. Overall, the paper's main empirical claims are self-contained against external benchmarks and do not reduce to their own inputs.

Assumptions & free parameters 2 free parameters · 4 assumptions · 1 invented entities

The system's effectiveness rests on LLM reliability, equivalence of the three image groups, correct expert ground truth, and the assumption that interaction trajectories are meaningful without correctness markers. Two hand-set parameters, cluster priority weights and the skill merging threshold, affect prioritization and knowledge consolidation. The structured skill document is an internal construct without independent evidence of correctness.

free parameters (2)
  • Cluster priority weights lambda1, lambda2, lambda3 = 0.4, 0.3, 0.3
    Set by comparing three hand-picked configurations in Section 4.3.1, Eq. 1. The authors report top-3 labels are over 75% consistent across settings, so the weights influence which clusters are surfaced first rather than the central efficiency outcome.
  • Skill merging threshold theta = not specified
    Algorithm 1 requires a similarity threshold theta to decide whether a new skill merges with an existing one or is stored separately. No value or calibration procedure is given, so the parameter is effectively free and directly affects the reported 72% skill reduction.
assumptions (4)
  • domain assumption LLM (GPT-5) generates reliable image descriptions, cluster patterns, and label rationales from text and bounding boxes.
    Every stage of the Proactive Agentic Support Module depends on LLM output quality, but no audit of generated descriptions or rationales is reported beyond user ratings.
  • domain assumption User interactions recorded during annotation are representative of expert knowledge and are mostly correct.
    The Interaction-Driven Evolution Module distills trajectories into skills without correctness markers; Section 6.3 admits mislabeling could propagate into learned skills.
  • ad hoc to paper The three image groups used in the user study have comparable annotation difficulty.
    Section 5.1 states groups were of comparable difficulty, but no metric, pilot data, or difficulty calibration is provided. If the baseline group was harder, the time and agreement differences would be confounded.
  • domain assumption The two expert-resolved reference labels are a valid ground truth for implicit semantics.
    Section 5.1 uses expert labels to compute agreement, but no inter-annotator reliability or validation of the reference set is reported.
invented entities (1)
  • Structured skill document with Overview, Preconditions, Workflow, Best Practices, Examples, and Troubleshooting modules
    purpose: Internal representation of distilled annotation procedures, used by the Skill Management Agent for merging and reuse.
    Introduced in Section 4.3.2 as a new knowledge representation. It has no external validation showing that a skill document corresponds to correct expert procedure; Section 6.3 notes correctness markers are absent.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ArtAnno: Annotating Implicit Semantics in Artworks through LLM Agent-Driven Bidirectional Human-AI Augmentation." pith.science (2026). https://pith.science/paper/IGTNDOH3

@misc{pith2026260805026,
  author       = {Pith},
  title        = {Pith review of: ArtAnno: Annotating Implicit Semantics in Artworks through LLM Agent-Driven Bidirectional Human-AI Augmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IGTNDOH3}},
  note         = {Machine review of arXiv:2608.05026}
}
read the original abstract

High-quality annotation of artworks is essential for computational art research, yet extracting implicit semantics remains challenging due to the reliance on culturally grounded meanings and deep contextual knowledge behind the images. Current AI-assisted annotation tools often lack assistance or rely on one-way workflows where experts have to perform extra manual calibrations to improve AI models, resulting in limited efficiency. To address this, we propose Bidirectional Human-AI Augmentation(BiHAA), a closed-loop framework in which skills and domain knowledge base evolve through real-time interaction and bidirectional HAI augmentation. Informed by a formative study with 20 artwork annotators from different backgrounds, we implement this framework in ArtAnno, an artwork annotation system driven by a multi-agent architecture. The system includes a Proactive Agentic Support Module, where AI augments humans through semantic mining and label suggestion, and an Interaction-Driven Evolution Module, where human expertise continuously enhances the AI through distilling annotation trajectories into reusable experience. Evaluation through a user study and two case studies demonstrates that our framework and system improve annotation efficiency, enable knowledge accumulation, and reduce the effort of information seeking and verification for annotators with limited domain expertise. We conclude by discussing broader implications and future directions.

Figures

Figures reproduced from arXiv: 2608.05026 by the authors.

Figure 1
Figure 1. Comparison between the Bidirectional Human-AI Augmentation Framework (BiHAA) and traditional frame￾works. (A-B) Left block represents input, right one represents output. (C) Both blocks serve as input and output. system records user actions and Human-AI interaction processes as logs and further extracts expert heuristics from these logs, trans￾forming them into reusable skills [19] and knowledge that are inte￾grated… view at source ↗
Figure 2
Figure 2. ArtAnno System Interface for Annotation of Female Identity and Experience Accumulation in Traditional Chinese [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Overview of the ArtAnno System. The system integrates two core components: (A) Proactive Agentic Support, [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Results of the system requirement questionnaire. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Comparison of skills before and after annotation. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

68 extracted references · 37 canonical work pages

  1. [1]

    Panos Achlioptas, Maks Ovsjanikov, Kilichbek Haydarov, Mohamed Elhoseiny, and Leonidas Guibas. 2021. ArtEmis: Affective Language for Visual Art. arXiv:2101.07396 [cs.CV] https://arxiv.org/abs/2101.07396

    arXiv 2021

  2. [2]

    Huan ang Gao, Jiayi Geng, Wenyue Hua, Mengkang Hu, Xinzhe Juan, Hongzhang Liu, Shilong Liu, Jiahao Qiu, Xuan Qi, Yiran Wu, Hongru Wang, Han Xiao, Yuhang Zhou, Shaokun Zhang, Jiayi Zhang, Jinyu Xiang, Yixiong Fang, Qiwen Zhao, Dongrui Liu, Qihan Ren, Cheng Qian, Zhenhailong Wang, Minda Hu, Huazheng Wang, Qingyun Wu, Heng Ji, and Mengdi Wang. 2026. A Survey...

    arXiv 2026

  3. [3]

    Franziska Becker, Steffen Koch, and Tanja Blascheck. 2025. AnnoLens: Explo- ration and Annotation through Lens-Based Guidance. InIEEE Visualization and Visual Analytics (VIS). 241–245. doi:10.1109/VIS60296.2025.00054

    arXiv 2025

  4. [4]

    Benato, Jancarlo F

    Bárbara C. Benato, Jancarlo F. Gomes, Alexandru C. Telea, and Alexandre X. Fal- cão. 2021. Semi-Automatic Data Annotation Guided by Feature Space Projection. Pattern Recogn.109, C (2021), 11 pages. doi:10.1016/j.patcog.2020.107612

    work page arXiv 2021

  5. [5]

    1972.Ways of Seeing

    John Berger. 1972.Ways of Seeing. Penguin Books. UIST ’26, November 2–5, 2026, GM Renaissance Center in Detroit, MI Trovato et al

    work page 1972

  6. [6]

    Islem Bouzenia and Michael Pradel. 2025. Understanding Software Engineering Agents: A Study of Thought-Action-Result Trajectories. arXiv:2506.18824 [cs.SE] https://arxiv.org/abs/2506.18824

    arXiv 2025

  7. [7]

    John Brooke. 1996. SUS: A Quick and Dirty Usability Scale. InUsability Evaluation in Industry, P. W. Jordan, B. Thomas, B. A. Weerdmeester, and I. L. McClelland (Eds.). Taylor & Francis, 189–194

    work page 1996

  8. [8]

    Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoubhik Debnath, Ronghang Hu, Didac Suris, Chaitanya Ryali, Kalyan Vasudev Alwala, Haitham Khedr, An- drew Huang, Jie Lei, Tengyu Ma, Baishan Guo, Arpit Kalla, Markus Marks, Joseph Greer, Meng Wang, Peize Sun, Roman Rädle, Triantafyllos Afouras, Effrosyni Mavroudi, Katherine Xu, Tsung-Han Wu, Yu Zhou, Lilian...

    arXiv 2025

Show all 68 references
  1. [9]

    Xiaojiao Chen, Wenru Qi, Yulian Yang, Xiaosong Wang, and Wei Chen. 2025. From Perception to Reflection: A Layered Framework for Aesthetic Education in the Digital Design of Ancient Painting.Visual Informatics9, 4 (2025), 100290. doi:10.1016/j.visinf.2025.100290

    2025

  2. [10]

    Tianheng Cheng, Lin Song, Yixiao Ge, Wenyu Liu, Xinggang Wang, and Ying Shan. 2024. YOLO-World: Real-Time Open-Vocabulary Object Detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 16901–16911. https://arxiv.org/abs/2401.17270

    2024 arXiv

  3. [11]

    Zichen Cheng, Ziyue Lin, Yihang Yang, Zhongyu Wei, and Siming Chen. 2025. Interactive Simulation and Visual Analysis of Social Media Event Dynamics with LLM-Based Multi-Agent Modeling.Visual Informatics9, 3 (2025), 100260. doi:10.1016/j.visinf.2025.100260

    2025

  4. [12]

    CVAT.ai Corporation. 2024. Computer Vision Annotation Tool (CVAT). https: //github.com/cvat-ai/cvat. doi:10.5281/zenodo.10977499

    2024 doi

  5. [13]

    Owen Pickrell, Beata Fonferko-Shadrach, Carys Jones, Ashley Akbari, Simon Thompson, and Arron Lacey

    Samuel Dobbie, Huw Strafford, W. Owen Pickrell, Beata Fonferko-Shadrach, Carys Jones, Ashley Akbari, Simon Thompson, and Arron Lacey. 2021. Markup: A Web-Based Annotation Tool Powered by Active Learning.Frontiers in Digital HealthVolume 3 - 2021 (2021). doi:10.3389/fdgth.2021.598916

    2021

  6. [14]

    Emilie Dubourg, Vincent Thouzeau, and Nicolas Baumard. 2024. A Step-by-Step Method for Cultural Annotation by LLMs.Frontiers in Artificial Intelligence7 (2024), 1365508. doi:10.3389/frai.2024.1365508

    2024

  7. [15]

    Abhishek Dutta and Andrew Zisserman. 2017. VGG Image Annotator (VIA). https://www.robots.ox.ac.uk/~vgg/software/via/. Accessed: 2026

    2017

  8. [16]

    Nicola Fanelli, Gennaro Vessio, and Giovanna Castellano. 2025. ArtSeek: Deep Artwork Understanding via Multimodal In-Context Reasoning and Late Interac- tion Retrieval. arXiv:2507.21917 [cs.CV] https://arxiv.org/abs/2507.21917

    2025 arXiv

  9. [17]

    Mingmeng Geng and Roberto Trotta. 2025. Human-LLM Coevolution: Evidence from Academic Writing. InFindings of the Association for Computational Lin- guistics (ACL), Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Moham- mad Taher Pilehvar (Eds.). Vienna, Austria, 12689–1269...

    2025 doi

  10. [18]

    Anders Högberg. 2026. Becoming Human in the Age of AI: Cognitive Co- Evolutionary Processes.Frontiers in PsychologyVolume 16 - 2025 (2026). doi:10.3389/fpsyg.2025.1734048

    2026

  11. [19]

    Guanyu Jiang, Zhaochen Su, Xiaoye Qu, and Yi R. Fung. 2026. XSkill: Continual Learning from Experience and Skills in Multimodal Agents. arXiv:2603.12056 [cs.AI] https://arxiv.org/abs/2603.12056

    2026 arXiv

  12. [20]

    Ehinger, and Jey Han Lau

    Yanbei Jiang, Krista A. Ehinger, and Jey Han Lau. 2024. KALE: An Artwork Image Captioning System Augmented with Heterogeneous Graph. InProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence(Jeju, Korea) (IJCAI). Article 848, 9 pages. doi:10.2...

    2024 doi

  13. [21]

    Nyasha Kadenhe, Mohamed Al Musleh, and Allan Lompot. 2025. Human-AI Co-Design and Co-Creation: A Review of Emerging Approaches, Challenges, and Future Directions. InProceedings of the 2025 AAAI Summer Symposium Series: Human-AI Collaboration: Exploring Diversity of Human Cogni...

    2025 doi

  14. [22]

    Jia Kaixin, Zhu Kewen, Deng Huanghuang, Qiu Yiwu, Ding Shiying, Ding Chenyang, Ning Zou, and Li Zejian. 2025. Knowledge Graph for Intelligent Generation of Artistic Image Creation: Constructing a New Annotation Hierar- chy. arXiv:2511.03585 [cs.HC] https://arxiv.org/abs/2511.03585

    2025

  15. [23]

    J. F. Kelley. 1984. An Iterative Design Methodology for User -Friendly Natural-Language Office Information Applications.ACM Transactions on Office Information Systems2, 1 (1984), 26–41. doi:10.1145/357417.357420

    1984

  16. [24]

    Chris Kelly, Luhui Hu, Bang Yang, Yu Tian, Deshun Yang, Cindy Yang, Zaoshan Huang, Zihao Li, Jiayin Hu, and Yuexian Zou. 2024. VisionGPT: Vision-Language Understanding Agent Using Generalized Multimodal Framework.arXiv preprint arXiv:2403.09027(2024). https://arxiv.org/abs/2403.09027

    2024 arXiv

  17. [25]

    I Keep Six Honest Serving Men

    Rudyard Kipling. 1902.Just So Stories. Macmillan and Co. Contains poem “I Keep Six Honest Serving Men”, source of the 5W1H / Kipling method

    1902

  18. [26]

    Jincheng Li, Jinpeng Wu, Shaocong Tan, Lin Du, Yu Zhang, Chaofan Yang, Jiadi Zhang, Rebecca Ruige Xu, Rui Shi, Lu Bai, and Xiaoru Yuan. 2026. Calli-VA: A Visual Analytics System for Analyzing and Comparing Chinese Calligraphic Styles.IEEE Transactions on Visualization and Comp...

    2026

  19. [27]

    Rui Li, Xiaofei Jia, Changle Zhou, and Junsong Zhang. 2022. Reconfiguration of the Brain During Aesthetic Experience on Chinese Calligraphy: Using Brain Complex Networks.Visual Informatics6, 1 (2022), 35–46. doi:10.1016/j.visinf. 2022.02.002

    2022 doi

  20. [28]

    Yijie Lian, Jianing Hao, Wei Zeng, and Qiong Luo. 2025. A Survey of Visual Insight Mining: Connecting Data and Insights via Visualization.Visual Informatics9, 4 (2025), 100271. doi:10.1016/j.visinf.2025.100271

    2025

  21. [29]

    Lawrence Zitnick

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. Microsoft COCO: Common Objects in Context. InEuropean Conference on Computer Vision (ECCV). 740–755. https://doi.org/10.1007/978-3-319-10602-1_48

    2014 doi

  22. [30]

    Yiren Liu, Si Chen, Haocong Cheng, Mengxia Yu, Xiao Ran, Andrew Mo, Yiliu Tang, and Yun Huang. 2024. How AI Processing Delays Foster Creativity: Explor- ing Research Question Co-Creation with an LLM-based Agent. InProceedings of the 2024 CHI Conference on Human Factors in Comp...

    2024

  23. [31]

    Jun Ma, Yuting He, Feifei Li, Lin Han, Chenyu You, and Bo Wang. 2024. Segment Anything in Medical Images.Nature Communications15 (2024), 654. doi:10.1038/ s41467-024-44824-z

    2024

  24. [32]

    Jun Ma, Zongxin Yang, Sumin Kim, Bihui Chen, Mohammed Baharoon, Adibvafa Fallahpour, Reza Asakereh, Hongwei Lyu, and Bo Wang. 2025. MedSAM2: Seg- ment Anything in 3D Medical Images and Videos. arXiv:2504.03600 [eess.IV] https://arxiv.org/abs/2504.03600

    2025 arXiv

  25. [33]

    W. J. Thomas Mitchell. 1994.Picture Theory(2 ed.). University of Chicago Press

    1994

  26. [34]

    Madhawa Perera, Md Zakir Hossain, Alexander Krumpholz, and Tom Gedeon

  27. [35]

    Maosheng Qin, Renyu Zhu, Mingxuan Xia, Chenkai Chen, Zhen Zhu, Min- min Lin, Junbo Zhao, Lu Xu, Changjie Fan, Runze Wu, and Haobo Wang

  28. [36]

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross Girshick, Piotr Dollár, and Christoph Feichtenh...

    2024 arXiv

  29. [37]

    arXiv:2509.14030 [cs.AI] https://arxiv.org/abs/2509.14030

    CrowdAgent: Multi-Agent Managed Multi-Source Annotation System. arXiv:2509.14030 [cs.AI] https://arxiv.org/abs/2509.14030

  30. [38]

    Guus Schreiber, Alia Amin, Mark van Assem, Victor de Boer, Lynda Hardman, Michiel Hildebrand, Laura Hollink, Zhisheng Huang, Janneke van Kersen, Marco de Niet, Borys Omelayenko, Jacco van Ossenbruggen, Ronny Siebes, Jos Taekema, Jan Wielemaker, and Bob Wielinga. 2006. Multimed...

    2006 doi

  31. [39]

    Artem Reshetnikov, Maria-Cristina Marinescu, and Joaquim More Lopez. 2022. DEArt: Dataset of European Art. InComputer Vision – ECCV Workshops: Tel A viv, Israel, Proceedings, Part I(Tel Aviv, Israel). 218–233. doi:10.1007/978-3-031-25056- 9_15

    2022 doi

  32. [40]

    Hanning Shao and Xiaoru Yuan. 2025. CataAnno: An Ancient Catalog Annotator for Annotation Cleaning by Recommendation. 31, 1 (2025), 404–414. doi:10.1109/ TVCG.2024.3456379

    2025

  33. [41]

    Erzhuo Shao, Yifang Wang, Yifan Qian, Zhenyu Pan, Han Liu, and Dashun Wang

  34. [42]

    Chuhan Shi, Xiaoquan Ren, Yifang Wang, Junze Li, Yushi Sun, Yawen Luo, and Rui Sheng. 2026. A Survey of Human-AI Collaboration for Scientific Discovery. (2026). https://doi.org/10.20944/preprints202601.0405.v2

    2026

  35. [44]

    Leixian Shen, Yifang Wang, Huamin Qu, Xing Xie, and Haotian Li. 2026. Interaction-Augmented Instruction: Modeling the Synergy of Prompts and Inter- actions in Human-GenAI Collaboration. InProceedings of the CHI Conference on Human Factors in Computing Systems. doi:10.1145/3772...

    2026

  36. [45]

    Ravidu Suien Rammuni Silva. 2025. AI-ArtBench. doi:10.21227/typ6-a328

    2025 doi

  37. [46]

    Hosking, Rachel W

    James Simpson, Gaurav Patil, Hamish Stening, Ayman Bin Kamruddin, Daniel Somerville, Sigrid Seage, Patrick Nalepka, Mark Dras, Simon G. Hosking, Rachel W. Kallen, Michael J. Richardson, and Deborah Richards. 2026. Can an AI Agent Lead Human Teams?Computers in Human Behavior: A...

    2026

  38. [47]

    Donghoon Shin, Daniel Lee, Gary Hsieh, and Gromit Yeuk-Yin Chan. 2025. Poster- Mate: Audience-driven Collaborative Persona Agents for Poster Design. InPro- ceedings of the 38th Annual ACM Symposium on User Interface Software and Technology (UIST). Article 201, 20 pages. doi:10...

    2025

  39. [48]

    Peter Steinberger. 2026. OpenClaw: Open -Source Autonomous LLM Agent. https://openclaw.ai/

    2026

  40. [49]

    Tan Tang, Yanhong Wu, Junming Gao, Kejia Ruan, Yanjie Zhang, Shuainan Ye, Yingcai Wu, and Xiaojiao Chen. 2024. ArtEyer: Enriching GPT-Based Agents with Contextual Data Visualizations for Fine Art Authentication.Visual Informatics8, 4 (2024), 48–59. doi:10.1016/j.visinf.2024.11.001

    2024 doi

  41. [50]

    Matthias Springstein, Stefanie Schneider, Javad Rahnama, Julian Stalter, Maxi- milian Kristen, Eric Müller-Budack, and Ralph Ewerth. 2024. Visual Narratives: Large-scale Hierarchical Classification of Art-historical Images. InIEEE/CVF Winter Conference on Applications of Compu...

    2024

  42. [51]

    Maxim Tkachenko, Mikhail Malyuk, Andrey Holmanyuk, and Nikolai Liubimov

  43. [52]

    Imad Eddine Toubal, Aditya Avinash, Neil Gordon Alldrin, Jan Dlabal, Wenlei Zhou, Enming Luo, Otilia Stretcu, Hao Xiong, Chun-Ta Lu, Howard Zhou, Ranjay Krishna, Ariel Fuxman, and Tom Duerig. 2024. Modeling Collaborator: Enabling Subjective Vision Classification With Minimal H...

    2024 arXiv

  44. [53]

    Yuying Tang, Xinyi Chen, Haotian Li, Xing Xie, Xiaojuan Ma, and Huamin Qu

  45. [54]

    arXiv:2602.05854 [cs.HC] https://arxiv.org/abs/2602.05854

    DuoDrama: Supporting Screenplay Refinement Through LLM-Assisted Human Reflection. arXiv:2602.05854 [cs.HC] https://arxiv.org/abs/2602.05854

  46. [55]

    Xinru Wang, Hannah Kim, Sajjadur Rahman, Kushan Mitra, and Zhengjie Miao

  47. [56]

    https://github.com/heartexlabs/label- studio

    Label Studio: Data Labeling Software. https://github.com/heartexlabs/label- studio

  48. [57]

    Yifang Wang, Rui Sheng, Erzhuo Shao, Yifan Qian, Haotian Li, Nan Cao, and Dashun Wang. 2026. Figures as Interfaces: Toward LLM-Native Artifacts for Scientific Discovery.arXiv preprint arXiv:2604.08491(2026). https://doi.org/10. 48550/arXiv.2604.08491

    2026 doi arXiv

  49. [58]

    Victoria Uren, Philipp Cimiano, José Iria, Siegfried Handschuh, Maria Vargas-Vera, Enrico Motta, and Fabio Ciravegna. 2006. Semantic Annotation for Knowledge Management: Requirements and a Survey of the State of the Art.Web Semant.4, 1 (Jan. 2006), 14–28. doi:10.1016/j.websem....

    2006 doi

  50. [59]

    He Wang, Yang Ouyang, Yuchen Wu, Chang Jiang, Lixia Jin, Yuanwu Cao, and Quan Li. 2025. KMTLabeler: An Interactive Knowledge-Assisted Labeling Tool for Medical Text Classification.IEEE Transactions on Visualization and Computer Graphics31, 9 (2025), 4493–4510. doi:10.1109/TVCG...

    2025

  51. [60]

    Xiwei Xuan, Xiaoqi Wang, Wenbin He, Jorge Piazentin Ono, Liang Gou, Kwan- Liu Ma, and Liu Ren. 2025. VISTA: A Visual Analytics Framework to Enhance Foundation Model-Generated Data Labels.IEEE Transactions on Visualization and Computer Graphics31, 10 (2025), 6991–7003. doi:10.1...

    2025

  52. [61]

    Yutao Yang, Junsong Li, Qianjun Pan, Bihao Zhan, Yuxuan Cai, Lin Du, Jie Zhou, Kai Chen, Qin Chen, Xin Li, Bo Zhang, and Liang He. 2026. AutoSkill: Experience- Driven Lifelong Learning via Skill Self-Evolution. arXiv:2603.01145 [cs.AI] https: //arxiv.org/abs/2603.01145

    2026

  53. [62]

    Yiwei Wang, Muhao Chen, Nanyun Peng, and Kai-Wei Chang. 2024. DeepEdit: Knowledge Editing as Decoding with Constraints.ArXivabs/2401.10471 (2024). https://api.semanticscholar.org/CorpusID:267060897

    2024 arXiv

  54. [63]

    Hanyu Zhou and Gim Hee Lee. 2025. LLaVA-4D: Embedding SpatioTemporal Prompt into LMMs for 4D Scene Understanding. arXiv:2505.12253 [cs.CV] https: //arxiv.org/abs/2505.12253

    2025 arXiv

  55. [64]

    Zhenyuan Wang, Qing Zhao, Yue Zhang, Jinhui Zhang, Guihua Shan, Xiao Zhou, and Dong Tian. 2025. ClayVolume: A Progressive Refinement Interaction System for Immersive Visualization.Visual Informatics9, 1 (2025), 71–83. doi:10.1016/j. visinf.2025.01.003

    2025 doi

  56. [65]

    Zhishang Xiang, Chengyi Yang, Zerui Chen, et al. 2026. A Systematic Survey of Self-Evolving Agents: From Model-Centric to Environment-Driven Co-Evolution. TechRxiv(2026). https://www.techrxiv.org/doi/pdf/10.36227/techrxiv.177203250. 05832634/v2

    2026 doi

  57. [68]

    Yuanhao Zhao, Mingming Jia, Genyun Sun, and Aizhu Zhang. 2025. PAMSNet: A Point Annotation-Driven Multi-Source Network for Remote Sensing Semantic Segmentation.ISPRS Journal of Photogrammetry and Remote Sensing229 (2025), 1–16. doi:10.1016/j.isprsjprs.2025.07.035

    2025 doi

  58. [2024]

    InProceedings of the CHI Conference on Human Factors in Computing Systems (CHI)

    Human-LLM Collaborative Annotation Through Effective Verification of LLM Labels. InProceedings of the CHI Conference on Human Factors in Computing Systems (CHI). Article 303, 21 pages. doi:10.1145/3613904.3641960

  59. [2025]

    InCompanion Proceedings of the 27th International Conference on Multimodal Interaction (ICMI Companion)

    Developing Multimodal Human-AI Interaction Systems Using Multi-Agent Frameworks. InCompanion Proceedings of the 27th International Conference on Multimodal Interaction (ICMI Companion). 67–69. doi:10.1145/3747327.3762827

  60. [2026]

    Nature Computational Science6, 3 (2026), 301–315

    SciSciGPT: Advancing Human–AI Collaboration in the Science of Science. Nature Computational Science6, 3 (2026), 301–315. https://doi.org/10.1038/s43588- 025-00906-6

    2026 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.