REVIEW 4 major objections 5 minor 1 cited by
VisTCP: A Visualization Framework to Construct Knowledge-Graph-Based Representation for Traditional Chinese Painting
T0 review · 4 major / 5 minor · reviewed 2026-07-08 · grok-4.5
Pith's one-line read VisTCP lets art historians build trustworthy knowledge-graph representations of Traditional Chinese Paintings through human-in-the-loop model refinement.
desk verdict Solid domain HITL system for TCP knowledge graphs; the joint-embedding refinement loop is the real contribution, but “trustworthy” is overclaimed on qualitative evidence alone. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The joint-embedding visualization view that places expert annotations and model predictions in a shared space so users can see uncertainty, correct the structured representation, and iteratively improve the TCP-oriented model.
What would settle it
On a held-out set of Traditional Chinese Paintings, measure whether VisTCP graphs improve object-and-relation coverage, inter-expert agreement, or research utility relative to unaided expert annotation and to standard vision models; a null or negative result would falsify the claim.
Extended reading notes
Core claim
A human-in-the-loop visualization framework can produce trustworthy knowledge-graph structured representations of Traditional Chinese Paintings by combining a TCP-oriented extraction model trained on expert labels, a joint-embedding view that surfaces expert-versus-model differences, and iterative expert refinement of the resulting graph.
Load-bearing premise
A taxonomy and training labels produced with three domain experts are complete and stable enough that a model trained on them, plus iterative human correction shown in joint embeddings, will yield representations that generalize as trustworthy across a wider corpus of Traditional Chinese Paintings.
Editorial extensions
If this is right
- Art historians can obtain consistent, machine-readable knowledge graphs of TCP objects and relationships instead of relying only on free-text notes.
- Archaeology and art-history studies gain a reusable semantic layer for comparing motifs, events, and compositions across paintings.
- Model uncertainty becomes visible rather than hidden, so experts know where their domain knowledge must intervene.
- Each round of expert correction can retrain the extractor, gradually reducing the effort needed for new paintings.
Reading between the lines
- The same taxonomy-plus-joint-embedding loop could transfer to other heritage image domains whose visual vocabulary diverges from natural-image benchmarks.
- If the taxonomy is published, it becomes a community resource for labeling larger TCP datasets even outside this tool.
- Without reported quantitative accuracy or baseline comparisons, adoption will hinge more on expert qualitative trust than on measured gains.
- The joint-embedding view itself is a reusable pattern for any human-in-the-loop structured-representation task where model and expert disagree on rare classes.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes VisTCP, a human-in-the-loop visualization framework for constructing knowledge-graph structured representations of Traditional Chinese Paintings (TCPs). Motivated by the claim that generic image structured-representation methods fail on TCPs (domain shift from natural images; hard even for experts), the authors first run a pilot study with three domain experts to build a TCP-oriented semantic taxonomy, then train a structured-representation model on expert annotations. A joint-embedding visualization view surfaces differences between expert labels and model predictions so that experts can iteratively refine outputs and, in principle, improve the model. Effectiveness is argued via a case study, a usage scenario, and expert interviews on a real TCP dataset, with the central claim that VisTCP yields trustworthy KG-style representations usable for archaeology and art-history research.
Significance. If the central claim holds, VisTCP would be a useful contribution at the intersection of visualization, digital humanities, and cultural-heritage computing: a domain-specific taxonomy plus a disagreement-aware HITL loop for a corpus that resists off-the-shelf scene-graph methods. The joint-embedding view of expert-versus-model differences is a concrete design idea that could transfer to other expert-scarce annotation settings. The work is primarily a systems/design contribution rather than a new learning algorithm; its significance therefore rests on whether the evaluation actually establishes trustworthiness and generalizability of the resulting representations beyond the study participants.
major comments (4)
- The abstract and evaluation design (case study, usage scenario, expert interviews) do not report quantitative extraction quality for the TCP-oriented structured-representation model—e.g., object/relation precision, recall, or F1 on held-out paintings, nor any measure of representation completeness against expert gold. The central claim of “trustworthy” KG representations is load-bearing and is not established by qualitative evidence alone; without held-out accuracy numbers it is unclear whether the model-plus-HITL loop improves correctness or mainly re-expresses the same experts’ judgments.
- Taxonomy construction, training labels, iterative refinement, and interview-based effectiveness evidence all appear to draw on a three-expert pool (pilot study). This creates a methodological circularity risk for the trustworthiness claim: the same narrow consensus that defines the ontology and labels also supplies the evidence that the refined outputs are good. The manuscript needs either (a) inter-annotator agreement on taxonomy and labels from a larger or held-out expert set, or (b) external validation (additional experts or art-historical ground truth not involved in taxonomy design) before the representations can be treated as stable across a broader TCP corpus.
- No controlled baseline or ablation is described comparing, for example, generic scene-graph / open-vocabulary methods, model-only extraction, pure expert annotation, and the full VisTCP HITL loop on the same paintings. Without such comparisons it is hard to attribute gains to the TCP-oriented taxonomy, the joint-embedding view, or iterative refinement, and hard to quantify how much the framework improves over existing structured-representation pipelines that the introduction claims “perform poorly on TCPs.”
- Dataset scale and split protocol are not stated in the evaluation summary (number of paintings, objects/relations annotated, train/val/test or leave-painting-out design). For a claim that the trained model plus joint-embedding refinement generalizes as trustworthy structured representation, the manuscript must report corpus size, annotation volume, and how generalization beyond the annotated set was tested; otherwise the case study and usage scenario remain anecdotal relative to the stated research goal.
minor comments (5)
- Clarify early what “structured representation” concretely means in the KG (node/edge types, event vs. object relations, multi-instance handling) so that later claims about semantic understanding are checkable against a fixed schema.
- The phrase “trustworthy structured representations” is strong; either define operational criteria (e.g., expert-verified precision thresholds, stability under re-annotation) or soften the claim language to match the qualitative evidence actually provided.
- Describe the joint-embedding view’s construction more precisely (what is embedded—objects, relations, full graphs; which similarity space; how uncertainty is encoded) so that other systems can reproduce or compare the design.
- State whether iterative expert refinements are used only for one-off correction of the current painting or are fed back to retrain/fine-tune the model, and under what schedule; the abstract implies both “refine the structured representation” and “iterative optimization of the model.”
- Position related work more sharply against prior cultural-heritage KG / painting annotation tools and against recent open-vocabulary detection and scene-graph models, so the TCP-specific gap is measurable rather than asserted.
Simulated Author's Rebuttal
We thank the referee for a careful and constructive review. The four major comments correctly identify that our central claim of trustworthy TCP structured representations currently rests too heavily on qualitative evidence (case study, usage scenario, expert interviews) and on a small expert pool. We agree that the manuscript must report quantitative extraction quality, clarify dataset scale and splits, reduce circularity risk around the three-expert consensus, and provide controlled comparisons against generic structured-representation pipelines. We will revise the evaluation section accordingly and temper claims where evidence remains qualitative. Below we respond point by point and indicate the concrete revisions planned for the next version.
read point-by-point responses
-
Referee: The abstract and evaluation design (case study, usage scenario, expert interviews) do not report quantitative extraction quality for the TCP-oriented structured-representation model—e.g., object/relation precision, recall, or F1 on held-out paintings, nor any measure of representation completeness against expert gold. The central claim of “trustworthy” KG representations is load-bearing and is not established by qualitative evidence alone; without held-out accuracy numbers it is unclear whether the model-plus-HITL loop improves correctness or mainly re-expresses the same experts’ judgments.
Authors: We agree that the trustworthiness claim is load-bearing and that qualitative evidence alone is insufficient. The current manuscript emphasizes the HITL workflow and expert feedback; it does not report held-out object/relation precision, recall, or F1, nor a completeness measure against expert gold. In revision we will (1) add a quantitative evaluation of the TCP-oriented structured-representation model on a held-out painting set (object detection and relation extraction P/R/F1, plus a simple completeness metric relative to expert gold graphs); (2) report before/after metrics for representations refined in the joint-embedding HITL loop so that gains from iterative refinement are separated from model-only output; and (3) revise the abstract and claims so that “trustworthy” is scoped to what the numbers and expert validation jointly support, rather than implied by qualitative evidence alone. Where sample size limits statistical strength, we will state that limitation explicitly. revision: yes
-
Referee: Taxonomy construction, training labels, iterative refinement, and interview-based effectiveness evidence all appear to draw on a three-expert pool (pilot study). This creates a methodological circularity risk for the trustworthiness claim: the same narrow consensus that defines the ontology and labels also supplies the evidence that the refined outputs are good. The manuscript needs either (a) inter-annotator agreement on taxonomy and labels from a larger or held-out expert set, or (b) external validation (additional experts or art-historical ground truth not involved in taxonomy design) before the representations can be treated as stable across a broader TCP corpus.
Authors: The referee is right that relying on the same three-expert pool for taxonomy design, labeling, refinement, and interview-based effectiveness creates a circularity risk. We will address this in two ways. First, we will report inter-annotator agreement (e.g., pairwise agreement / Cohen’s or Fleiss’ kappa where applicable) on taxonomy categories and on object/relation labels among the participating experts, and we will document how disagreements were resolved. Second, we will add external validation: at least one additional domain expert (or a small held-out expert set) who did not design the taxonomy will review a sample of model and HITL-refined graphs, and we will report their agreement with the refined representations and any systematic disagreements. We will also expand the Limitations section to state that the taxonomy reflects a small expert consensus and that broader multi-institution validation remains future work. We cannot fully eliminate the small-pool constraint within this revision cycle, but the IAA numbers plus held-out expert review will make the stability claim more honest and testable. revision: yes
-
Referee: No controlled baseline or ablation is described comparing, for example, generic scene-graph / open-vocabulary methods, model-only extraction, pure expert annotation, and the full VisTCP HITL loop on the same paintings. Without such comparisons it is hard to attribute gains to the TCP-oriented taxonomy, the joint-embedding view, or iterative refinement, and hard to quantify how much the framework improves over existing structured-representation pipelines that the introduction claims “perform poorly on TCPs.”
Authors: We agree that the introduction’s claim that generic structured-representation methods perform poorly on TCPs is not backed by controlled comparisons in the current evaluation, and that attribution of gains to the taxonomy, joint-embedding view, and iterative refinement is therefore weak. In revision we will add a controlled comparison on a fixed set of paintings covering: (i) at least one generic scene-graph or open-vocabulary structured-representation baseline (as used or cited in the related work); (ii) our TCP-oriented model alone (no HITL); (iii) pure expert annotation time/quality where feasible; and (iv) the full VisTCP HITL loop. We will report the same object/relation quality metrics (and, where appropriate, expert preference or correction effort) across these conditions so that improvements can be attributed more clearly. If pure expert annotation of the full set is too costly, we will use a stratified subset and report effort (time, number of corrections) for model-only vs. HITL. Claims in the introduction and discussion will be rewritten to match what these comparisons actually show. revision: yes
-
Referee: Dataset scale and split protocol are not stated in the evaluation summary (number of paintings, objects/relations annotated, train/val/test or leave-painting-out design). For a claim that the trained model plus joint-embedding refinement generalizes as trustworthy structured representation, the manuscript must report corpus size, annotation volume, and how generalization beyond the annotated set was tested; otherwise the case study and usage scenario remain anecdotal relative to the stated research goal.
Authors: This is a clear omission and we will fix it. The revised manuscript will report: total number of TCP images in the corpus; number of paintings annotated for training and evaluation; counts of annotated objects, relations, and taxonomy categories; and the train/validation/test (or leave-painting-out) split protocol used for the structured-representation model. We will also state how held-out paintings used in the quantitative evaluation and in the case study / usage scenario relate to the training set, so that generalization beyond the annotated set is explicit rather than implied. If the corpus is modest, we will say so and frame the contribution as a systems/design study with preliminary generalization evidence rather than as large-scale corpus-wide validation. revision: yes
Circularity Check
No significant circularity: VisTCP is a systems/HCI framework paper whose claims rest on pilot taxonomy, model training, and qualitative evaluation rather than a derivation that collapses to its inputs.
full rationale
This is a visualization/HCI systems paper, not a first-principles or fitted-equation derivation. The claimed pipeline is: (1) pilot study with three experts builds a TCP semantic taxonomy; (2) expert-annotated data train a structured-representation model; (3) a joint-embedding view surfaces expert–model differences so experts can refine labels and iteratively improve the model; (4) effectiveness is argued via case study, usage scenario, and expert interviews. None of the six circularity patterns apply in a load-bearing way. There is no equation that defines a quantity in terms of the quantity it claims to predict; no free parameter fitted to data and then renamed a prediction; no uniqueness theorem or ansatz imported from the authors’ prior work to force the result; and no renaming of a known empirical law as a new derivation. Self-citation, if any, is ordinary related-work framing and is not used to prove the central trustworthiness claim. Residual methodological risk (small expert pool both defining the taxonomy and supplying interview evidence) is self-reinforcement of evaluation, not definitional circularity of a derivation chain. Per the analyzer rules, honest non-finding is the correct outcome: score 0, empty steps.
Assumptions & free parameters
assumptions (4)
- domain assumption A semantic taxonomy of TCP objects and relations elicited from three domain experts is adequate to ground structured representation of TCPs.
- domain assumption Expert annotations of TCPs constitute reliable supervision for a structured-representation model despite acknowledged difficulty even for experts.
- domain assumption Case study, usage scenario, and expert interviews suffice to demonstrate effectiveness and trustworthiness of the structured representations.
- standard math Standard supervised structured-prediction / scene-graph style learning applies once TCP-specific labels exist.
invented entities (2)
-
VisTCP joint-embedding visualization view (expert annotations vs model predictions)
-
TCP-oriented semantic taxonomy
Cite this review
Pith. "Pith review of VisTCP: A Visualization Framework to Construct Knowledge-Graph-Based Representation for Traditional Chinese Painting." pith.science (2026). https://pith.science/paper/N7AFH22V
@misc{pith2026260705841,
author = {Pith},
title = {Pith review of: VisTCP: A Visualization Framework to Construct Knowledge-Graph-Based Representation for Traditional Chinese Painting},
year = {2026},
howpublished = {\url{https://pith.science/paper/N7AFH22V}},
note = {Machine review of arXiv:2607.05841}
}
read the original abstract
Structured representation can characterize semantic objects and relationships in images. It provides a possible effective way for the semantic understanding of Traditional Chinese Paintings (TCPs) to better support archaeology and art history research. However, most image-oriented structured representation methods perform poorly on TCPs, due to two major challenges: 1) the objects and events of TCPs exhibit substantial differences from modern natural images, which results in semantic misunderstandings of TCPs; and 2) it is difficult to achieve accurate identification of ancient objects and events in TCPs, even for domain experts.In this paper, we propose VisTCP, a visualization framework that combines a TCP-oriented intelligent model and expert knowledge, which enables art historians to achieve trustworthy structured representations of TCPs in a human-in-the-loop manner. Firstly, we conduct a pilot study with three domain experts to build a semantic taxonomy of TCPs. Then, expert-annotated data are used to train a TCP-oriented structured representation model, which can automatically extract meaningful objects and their relationships in TCPs. To inform users of the model uncertainty, we design a joint embedding visualization view to show the differences between expert annotations and model predictions. This allows users to refine the structured representation based on their domain knowledge, enabling iterative optimization of the model. Finally, we conduct a case study, a usage scenario, and expert interviews on a real dataset to demonstrate the effectiveness of VisTCP in supporting the structured representation and semantic understanding of TCPs.
Figures
Figures from the paper (3 more)
Forward citations
Cited by 1 Pith paper
-
CompoVista: A Composition-Graph-Based Visual Analytics System for Compositional Analysis of Traditional Chinese Paintings
CompoVista introduces a four-layer Composition Graph and canvas-based visual analytics system enabling structure-aware retrieval, cohort comparison, and evidence inspection for Traditional Chinese Paintings.
Reference graph
Works this paper leans on
-
[1]
Scene graph generation by iterative message passing,
D. Xu, Y . Zhu, C. B. Choy, and L. Fei-Fei, “Scene graph generation by iterative message passing,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 5410–5419
work page 2017
-
[2]
Unifying large language models and knowledge graphs: A roadmap,
S. Pan, L. Luo, Y . Wang, C. Chen, J. Wang, and X. Wu, “Unifying large language models and knowledge graphs: A roadmap,”IEEE Transactions on Knowledge and Data Engineering, pp. 1–20, 2024
work page 2024
-
[3]
Vip-cnn: Visual phrase guided convolutional neural network,
Y . Li, W. Ouyang, X. Wang, and X. Tang, “Vip-cnn: Visual phrase guided convolutional neural network,” in2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 7244– 7253
work page 2017
-
[4]
Graph-structured repre- sentations for visual question answering,
D. Teney, L. Liu, and A. Van Den Hengel, “Graph-structured repre- sentations for visual question answering,” in2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 3233– 3241
work page 2017
-
[5]
Align r-cnn: A pair- wise head network for visual relationship detection,
M. Tajrobehkar, K. Tang, H. Zhang, and J.-H. Lim, “Align r-cnn: A pair- wise head network for visual relationship detection,”IEEE transactions on multimedia, vol. 24, pp. 1266–1276, 2021
work page 2021
-
[6]
Unbiased scene graph generation from biased training,
K. Tang, Y . Niu, J. Huang, J. Shi, and H. Zhang, “Unbiased scene graph generation from biased training,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 3716– 3725
work page 2020
-
[7]
S. McCausland and Y . Hwang,On Telling Images of China: Essays in Narrative Painting and Visual Culture. Hong Kong University Press, 2013
work page 2013
-
[8]
Modelling traditional chinese paintings for content-based image classification and retrieval,
D. Zhang, B. Pham, and Y . Li, “Modelling traditional chinese paintings for content-based image classification and retrieval,” in10th Interna- tional Multimedia Modelling Conference, 2004. Proceedings.IEEE, 2004, pp. 258–264
work page 2004
Show all 27 references
-
[9]
A study on a content-based image retrieval technique for chinese paintings,
C.-C. Hung, “A study on a content-based image retrieval technique for chinese paintings,”The Electronic Library, vol. 36, no. 1, pp. 172–188, Feb. 2018
2018
-
[10]
The classification of traditional chinese painting based on cnn,
Q. Meng, H. Zhang, M. Zhou, S. Zhao, and P. Zhou, “The classification of traditional chinese painting based on cnn,” inCloud Computing and Security, X. Sun, Z. Pan, and E. Bertino, Eds. Cham: Springer International Publishing, 2018, pp. 232–241
2018
-
[11]
The elements extraction on traditional chinese paintings based on object detection,
Q. Meng, K. Li, M. Zhou, and H. Zhang, “The elements extraction on traditional chinese paintings based on object detection,” inProceedings of the 2019 2nd Artificial Intelligence and Cloud Computing Conference. New York, NY , USA: Association for Computing Machinery, 2019, pp. 111–116
2019
-
[12]
Dct–cnn-based classification method for the gongbi and xieyi techniques of chinese ink-wash paintings,
W. Jiang, Z. Wang, J. S. Jin, Y . Han, and M. Sun, “Dct–cnn-based classification method for the gongbi and xieyi techniques of chinese ink-wash paintings,”Neurocomputing, vol. 330, pp. 280–286, 2019. [Online]. Available: https://www.sciencedirect.com/science/article/pii/ S0925...
2019
-
[13]
Scrolltimes: Tracing the provenance of paintings as a window into history,
W. Zhang, J. K. Wong, Y . Chen, A. Jia, L. Wang, J. Zhang, L. Cheng, and W. Chen, “Scrolltimes: Tracing the provenance of paintings as a window into history,”CoRR, vol. abs/2306.08834, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2306.08834
-
[14]
Data-driven npr illustrations of natural flows in chinese painting,
Y .-C. Lai, B.-A. Chen, K.-W. Chen, W.-L. Si, C.-Y . Yao, and E. Zhang, “Data-driven npr illustrations of natural flows in chinese painting,”IEEE Transactions on Visualization and Computer Graphics, vol. 23, no. 12, pp. 2535–2549, 2017
2017
-
[15]
Annotating traditional chinese paintings for immersive virtual exhibition,
W. Ma, Y . Wang, Y .-Q. Xu, Q. Li, X. Ma, and W. Gao, “Annotating traditional chinese paintings for immersive virtual exhibition,”J. Comput. Cult. Herit., vol. 5, no. 2, aug 2012. [Online]. Available: https://doi.org/10.1145/2307723.2307725
2012 doi
-
[16]
Visual genome: Connecting language and vision using crowdsourced dense image annotations,
R. Krishna, Y . Zhu, O. Groth, J. Johnson, K. Hata, J. Kravitz, S. Chen, Y . Kalantidis, L.-J. Li, D. A. Shamma, M. S. Bernstein, and L. Fei-Fei, “Visual genome: Connecting language and vision using crowdsourced dense image annotations,”Int. J. Comput. Vision, vol. 123, no. 1,...
2017 doi
-
[17]
Computational approaches for traditional chinese painting: From the
W. Zhang, J. Zhang, K. Wong, Y . Wang, Y . Feng, L. Wang, and W. Chen, “Computational approaches for traditional chinese painting: From the ”six principles of painting” perspective,”CoRR, vol. abs/2307.14227,
- [18]
-
[19]
Retainvis: Visual analytics with interpretable and interactive recurrent neural networks on electronic medical records,
B. C. Kwon, M.-J. Choi, J. T. Kim, E. Choi, Y . B. Kim, S. Kwon, J. Sun, and J. Choo, “Retainvis: Visual analytics with interpretable and interactive recurrent neural networks on electronic medical records,” IEEE Transactions on Visualization and Computer Graphics, vol. 25, no...
2019
-
[20]
Protosteer: Steering deep sequence model with prototypes,
Y . Ming, P. Xu, F. Cheng, H. Qu, and L. Ren, “Protosteer: Steering deep sequence model with prototypes,”IEEE Transactions on Visualization and Computer Graphics, vol. 26, no. 1, pp. 238–248, 2020
2020
-
[21]
Drava: Aligning human concepts with machine learning latent dimensions for the visual exploration of small multiples,
Q. Wang, S. L’Yi, and N. Gehlenborg, “Drava: Aligning human concepts with machine learning latent dimensions for the visual exploration of small multiples,” ser. CHI ’23. New York, NY , USA: Association for Computing Machinery, 2023. [Online]. Available: https://doi.org/10.114...
2023 doi
-
[22]
Irvine: A design study on analyzing corre- lation patterns of electrical engines,
J. Eirich, J. Bonart, D. J ¨ackle, M. Sedlmair, U. Schmid, K. Fischbach, T. Schreck, and J. Bernard, “Irvine: A design study on analyzing corre- lation patterns of electrical engines,”IEEE Transactions on Visualization and Computer Graphics, vol. 28, no. 1, pp. 11–21, 2022
2022
-
[23]
Rclens: Interactive rare category exploration and identification,
H. Lin, S. Gao, D. Gotz, F. Du, J. He, and N. Cao, “Rclens: Interactive rare category exploration and identification,”IEEE Transactions on Visualization and Computer Graphics, vol. 24, no. 7, pp. 2223–2237, 2018
2018
-
[24]
Chinese iconography thesaurus,
POSTHUMANITIES, “Chinese iconography thesaurus,” https://chineseiconography.org/
-
[25]
Mask r-cnn,
K. He, G. Gkioxari, P. Doll ´ar, and R. Girshick, “Mask r-cnn,” in2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2980–2988
2017
-
[26]
Real: A representative error-driven approach for active learning,
C. Chen, Y . Wang, L. Liao, Y . Chen, and X. Du, “Real: A representative error-driven approach for active learning,” inMachine Learning and Knowledge Discovery in Databases: Research Track: European Conference, ECML PKDD 2023, Turin, Italy, September 18–22, 2023, Proceedings, ...
2023
-
[27]
Available: https://doi.org/10.1007/978-3-031-43412-9 2
[Online]. Available: https://doi.org/10.1007/978-3-031-43412-9 2
Reviewed July 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.