On Explaining Visual Captioning with Hybrid Markov Logic Networks

Deepak Venugopal; Monika Shah; Somdeb Sarkhel

REVIEW 5 major objections 5 minor 36 references

On Explaining Visual Captioning with Hybrid Markov Logic Networks

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A hybrid Markov logic network can pick out the training examples that plausibly drove a model to generate a given image caption.

desk verdict A useful caption-level explanation pipeline that is over-sold as model-level; the stress-test is right. read the letter →

arxiv 2507.21246 v1 pith:UXUYQNEU submitted 2025-07-28 cs.CV cs.AI

Monika Shah , Somdeb Sarkhel , Deepak Venugopal This is my paper · ORCID

classification cs.CVcs.AI

keywords visualcaptioningexplainableAIhybridMarkovlogicnetworksexample-basedexplanationsCLIPembeddingsvirtualevidenceimportanceweightinguserstudy

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a captioning model's output can be explained by pointing to specific training examples that plausibly influenced the caption's generation. The authors build a Hybrid Markov Logic Network (HMLN), a probabilistic knowledge base combining symbolic text predicates with real-valued CLIP similarities, to model the distribution over training instances, then condition that distribution on the generated caption and measure how the distribution shifts. Training examples whose conditional marginals move most are presented as positive, negative, or neutral explanations. A user study with Amazon Mechanical Turk workers and with AI-trained participants indicates these example-based explanations are interpretable, and the same machinery is used to compare four captioning models, with AoANet rated the most explainable. This matters because standard captioning metrics compare generated captions with human references and do not reveal how a model integrates visual and linguistic knowledge.

What carries the argument

The central object is the Hybrid Markov Logic Network (HMLN), a log-linear model over first-order logic formulas that mix symbolic predicates with real-valued functions. Two template formulas drive the explanation: a conjunctive property C that scores when two predicates co-occur in a training caption, and an explanation property I that scores when one predicate explains away the other, with CLIP cosine similarities between text predicates and images supplying the real-valued terms. Weights are learned by contrastive divergence, and explaining a caption means conditioning the learned distribution on virtual evidence derived from the generated caption, then comparing importance-weighted marginals with unconditioned marginals via Hellinger distance to quantify the bias each training example introduces.

What would settle it

Retrain a captioning model twice, once on the full training set and once with the training examples the HMLN labels as most positively biasing removed; if the generated caption for the test image does not change in probability or content, while removing other examples does change it, the HMLN's ranking is not tracking the model's actual learned dependencies.

Watch

Extended reading notes

Core claim

The central claim is that conditioning an HMLN distribution over training instances on a generated caption identifies training examples that plausibly explain the caption's generation, even though the captioning model's parameters, activations, and training objective are never inspected. The distribution is defined over predicates extracted from human captions, parameterized with CLIP-based real-valued terms, and conditioned through virtual evidence derived from the generated caption. The shift between the prior and evidence-conditioned distributions, measured by Hellinger distance on importance-weighted marginals, ranks training examples by how much they may have biased the model. In the authors' experiments, this form of explanation was rated as interpretable by both non-technical and technical users, and the ranking made AoANet the most explainable of the four captioning models compared (SGAE, AoANet, X-LAN, and M2 Transformer).

Load-bearing premise

The whole explanation pipeline never looks at the captioning model itself; it assumes CLIP's image-text similarities stand in for the model's learned associations, so if a captioning model learned different associations than CLIP encodes, the selected 'explanatory' training examples would be visually plausible without actually explaining the model's output.

Editorial extensions

If this is right

Explanations can be produced for any captioning model without access to its internals, using only its generated captions, the training captions, and CLIP embeddings.
Explainability becomes a measurable axis on which captioning models can be compared; in the paper's study, AoANet received the highest explanation ratings.
The bias scores correlate with human caption quality as measured by CLIPScore, suggesting the selected examples align with images humans find describable.
The framework is intended to extend to other generative multimodal tasks such as visual question answering, as the authors state.
Attention-based explanations did not show a significant difference between high- and low-attention object pairs in the user study, whereas the HMLN explanations did.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension is to validate the selected examples causally by retraining a captioning model without the top positively biased training examples and checking whether the target caption becomes less likely; the paper does not perform such an ablation.
Because the method relies on CLIP similarity as a proxy for the captioning model's learned associations, it is likely to work best when the captioning model's image-text alignment resembles CLIP's; for models trained on very different distributions, the 'explanatory' examples may be visually plausible but not causally responsible.
The Hellinger-distance ranking could be repurposed as a dataset diagnostic: training examples that consistently receive negative bias for a class of images may signal spurious correlations or underrepresented combinations in the training data.
The framework's comparisons across models are only as meaningful as the user-study instrument; a more objective grounding would come from replacing Likert ratings with forced-choice tests or with behavioral measures such as how well users can predict a model's caption from the explanation.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know upfront: this paper explains captions, not models. The stress-test note holds up fully on reading. The pipeline (Sections 3.2 and 3.3) builds potentials and virtual evidence from CLIP embeddings and the caption string only. The captioning model is never queried—no parameters, activations, gradients, or training objective. Two different models producing the same caption get identical explanations. So the AMT study measures how interpretable the caption grounding is, not how interpretable the model is, and the “AoA is most explainable” comparison is really a comparison of the captions those models happened to produce. This is not a fatal flaw if the framing is fixed: the method is a legitimate way to produce visually grounded, example-based explanations of why a caption describes an image. But the current language about “how relevant examples from the training data could have influenced the generation” is overreach without a sufficiency argument.

What is genuinely new: the specific combination of HMLNs with query-specific learning, CLIP-based real-valued potentials, virtual evidence from the generated caption, and importance-weighted Hellinger distances to select contrasting training examples. That is a coherent, novel pipeline. The AMT study is substantial (1000 workers, plus a smaller technical-user group), and the attention baseline is a useful comparison. The paper does make a real contribution to example-based XAI for captions, just not the one the abstract advertises.

The soft spots are real but uneven. The circularity in Section 4.4 is genuine—CLIPScore and the HMLN potentials both rely on CLIP—but it only weakens the secondary bias-quantification claim; the primary interpretability claim rests on the user study, which is independent. The formal slips, such as Eq. (5) mixing T and n indices, the confusing max(W,1) “clipping” (it is a lower bound, not an upper bound), and the missing weight w_f in the Section 3.2 potential expression, are minor but should be cleaned up. No code or data is released, which hurts reproducibility. The comparative claim about AoA relies on differences of ~0.05 on a 5-point scale with some non-significant t-test pairs; that should be softened.

My verdict: this deserves peer review, but it needs major revision on framing and honesty of claims. If the authors reframe to “caption-level explanations” and either remove or heavily qualify the model-comparison language, the contribution is solid and useful for the XAI and captioning subfields. As it stands, the central explanatory claim is unsupported.

Recommendation: send to review, but expect the referee to demand a substantial rewrite of the claims and an explicit acknowledgment of the model-agnostic nature.

Referee Report

5 major / 5 minor

Summary. This paper proposes a Hybrid Markov Logic Network (HMLN) framework for generating example-based explanations of image captions. The authors learn a distribution over relational predicates extracted from training captions, condition on a generated caption through CLIP-based virtual evidence, and use importance weighting together with Hellinger distances to select training examples that introduce positive bias, negative bias, or minimal bias relative to the test instance. The framework is evaluated through an Amazon Mechanical Turk study with 1000 workers and a smaller technical-user study across four captioning models (SGAE, AoANet, X-LAN, and M2 Transformer), with an attention-based explainer as a baseline. The paper reports that users find the explanations interpretable and concludes that AoANet is the most explainable model among the four.

Significance. If the central claim were established, the paper would contribute a novel neuro-symbolic, example-based explanation format for captioning that could be useful to end users and for comparing systems. The main strengths are the principled probabilistic formulation (HMLNs with learned potentials, importance weighting, and virtual evidence), the use of a real user study with both lay and technical participants, the comparison against an attention baseline, and the reliance on publicly available models, data, and CLIP embeddings. However, the central explanatory claim is not currently supported by the method's inputs: the pipeline never accesses the captioning model being explained, and the evaluation conflates caption-level quality or interpretability with model-level explainability. The paper therefore needs substantial revision before the model-comparison conclusions can be accepted.

major comments (5)

[§3.3, Algorithm 1/2; §4.3] The framework never conditions on the captioning model being explained. Algorithm 1 receives only the HMLN structure, training data, and caption C; Algorithm 2 uses the caption's ground predicates and CLIP similarities between the test image and those predicates. No parameter, activation, gradient, or training objective of SGAE, AoA, X-LAN, or M2 Transformer is used. Consequently, any two captioning models that emit the same caption for the same test image yield identical explanations and identical bias quantifiers. The §4.3 conclusion that 'AoA was the most explainable model' is therefore a statement about the captions (or about how well those captions match CLIP-grounded training examples), not about the models. The paper must either justify that the generated caption is a sufficient statistic for the model's learned associations or condition on model-specific quantities before the model-comparison claim can be accepted.
[§3.3, Eq. (5)] The importance-weight clipping is implemented as max(W^(j),1), but the preceding text says clipping is used to 'upper bound the weights'; max with 1 is a lower bound, not an upper bound. The intended clipping is presumably min(W^(j), c) or a normalized truncation, and the choice of c should be reported. The equation also mixes indices: the numerator sums over t = 1..T with an undefined j in I(I_i^(j)), while the denominator sums over j = 1..n of max(W^(j),1). Because these estimates feed the Hellinger distances used to select the three explanation examples in §4.1, the indexing and clipping rule need to be corrected.
[§3.2, Real-Valued Terms] The potential value for the C term is stated as min{σ(ε−g1), σ(ε−g2)}, but Eq. (2) defines potentials as exp(w_f · s_i(x)); the C potential should therefore be exp(w_f · C_j(x)) = (min{σ(ε−g1), σ(ε−g2)})^{w_f}. Omitting the learned weight w_f changes the likelihood in Eq. (2) and the gradient in Eq. (4), and it is not a purely cosmetic issue because the weights are learned and shared across groundings.
[§4.4] The validation of the bias-quantification measure is partly circular. The HMLN potentials (§3.2) and the virtual evidence (§3.3) are constructed from CLIP cosine similarities between images and text, and CLIPScore is also a CLIP-based image-caption similarity. A high correlation between the maximum Hellinger distance and CLIPScore may simply reflect the shared CLIP embedding geometry, rather than the explanatory quality of the selected training examples. This does not establish that the selected examples explain the captioning model's generation behavior; a non-CLIP reference metric or a model-specific ground truth (e.g., influence functions or controlled training-set perturbation) is needed.
[§4.3, Table 4] The significance summary and Table 4 are internally inconsistent. The text states that differences were statistically significant except for M2 paired with XLAN and SGAE, but the table contains entries such as AoA vs XLAN = 0.912, which would not be significant at p < 0.05 for any reasonable sample size, and the red/non-red marking is not explained in the caption. As reported, the statistics do not support the conclusion that AoA is the most explainable model.

minor comments (5)

[Abstract] The phrase 'may not provide a user with a deep insights' should be 'may not provide a user with deep insight'; the abstract also states the hypothesis about training-data influence as a fact, whereas the body of the paper more cautiously frames it as a hypothesis.
[§3.3, Virtual Evidence] The expression ψ(x_i) = e^{−log σ(ε−d_i)} simplifies to 1/σ(ε−d_i) = 1 + e^{d_i−ε}, which is not a bounded soft-evidence value; the sign convention is unexplained and appears inconsistent with the C-term's σ(ε−g) formulation, so the intended form or a justification for the unbounded factor should be provided.
[§3.2, Real-Valued Terms] The parameter a appears in the equation log σ(g_j(X)−ε) = −log(1+e^{a(ε−g_j(X))}) but is never defined or set; the text reports ε = 0.7 but gives no sensitivity analysis for ε or for the Hellinger-distance threshold 0.75 used in §4.4.
[§4.3] The paired t-test is described without specifying whether the 1000 AMT workers were the same individuals across the four models; a paired test requires matched observations, and if the worker groups were independent, an unpaired test would be appropriate. The question and instruction wording should also be reported in full to allow replication.
[§4.4, Figure 4] The x-axis is described as 'negative log-sigmoid of the average human-written CLIPScores (closer to 0 is a better score),' but the transformation is not defined in the text and the 'better score' convention is ambiguous; reporting the raw CLIPScore or a standard transformation would make the figure easier to interpret.

Circularity Check

2 steps flagged · score 4.0 of 10

The explanation pipeline is driven only by the generated caption plus CLIP embeddings, so model-level explanations and the CLIPScore-based validation reduce to shared CLIP inputs.

other [Section 3.3, Algorithm 1]
"Algorithm 1: Learning Potentials Input: HMLN structureH, Training dataD, captionC generated for test imageI Output: Learned distributionP(·) with potentialsφ1...φn"

Neither Algorithm 1 nor Algorithm 2 receives the captioning model's parameters, activations, gradients, hidden states, or training objective. The real-valued potentials use CLIP cosine distances and the virtual evidence ψ(x) is CLIP similarity between the test image and the caption predicates. Thus the selected 'explanatory' training examples are by construction a function of (caption C, test image I, training captions, CLIP) and not of the captioning model. Any two captioning models that emit the same C for the same I receive identical explanations, and the §4.3 comparison of models is really a comparison of their captions under CLIP-grounded example selection.
other [Section 4.4]
"We used CLIPScore [10] on the ground-truth captions for the test data, i.e., captions written by humans and computed the average score over the 5 captions. We consider the maximum Hellinger's distance between prior and conditionals (considering only conditional distributions larger than a threshold of 0.75), namely, the training examples that are more useful in learning the caption, and compare the distances in these examples with the average CLIPScore."

The Hellinger distances being validated are computed from potentials whose real-valued terms are CLIP cosine distances (I_j = -(g_j(X1)-g_j(X2))^2 and C_j = min log σ(ε - g_j(...))) and from virtual evidence ψ(x_i) = e^{-log σ(ε - d_i)} with d_i the CLIP cosine similarity between the test image and the predicate. CLIPScore, by the paper's own §1.1 description, is 'CLIP to measure coherence between the visual representation of image and the textual representation of the caption.' Both sides of the Fig. 4 comparison are therefore built from the same CLIP embeddings, so the reported positive relationship is partly a shared-measure correlation rather than an independent confirmation that the distances capture the captioning model's generation process.

full rationale

The HMLN machinery itself is internally consistent: weights are learned from the training data by contrastive divergence, and no equation is a tautology of its output. However, the central explanatory target is the captioning model, and the method never conditions on that model: Algorithms 1 and 2 consume only the HMLN structure, training data, test image, and the generated caption. Consequently the model-level 'explanation' and the cross-model explainability ranking reduce to CLIP-based caption-to-training-example matching; two models emitting the same caption would receive identical explanations by construction. Section 4.4's CLIPScore validation partially shares this same CLIP representation with the HMLN potentials and virtual evidence, so the correlation there is not independent. The AMT user study provides external evidence that users find the selected examples interpretable, which prevents a fully circular verdict and keeps the score moderate; the central model-specific explanatory claim remains unestablished.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on assumptions that CLIP similarity tracks captioning-model associations, that conditional shift in an HMLN quantifies learning bias, and that several hand-chosen thresholds are stable. None of these is independently verified in the paper.

free parameters (6)

epsilon threshold in C property and virtual evidence = 0.7
Set because it 'indicated that the text was a good explanation of the image'; no sensitivity analysis is reported.
maximum conjunctive property chain length = 2
Chosen because longer chains lack coherence, without quantitative evidence.
number of explanation examples per test instance = 3
Fixed as one positive-bias, one negative-bias, and one least-bias example in Section 4.1.
Hellinger distance threshold = 0.75
Used in Section 4.4 to restrict which training examples enter the bias-quantification comparison; no justification is given.
learning rate for HMLN weight learning = 0.01
Implementation detail in Section 4.1 with no sensitivity analysis.
importance weight clipping rule = max(W,1) in Eq. (5)
The text says upper-bound clipping, but max(W,1) is a lower-bound rule; the actual clipping parameter is ambiguous.

assumptions (5)

standard math Gibbs sampling converges to the HMLN posterior after burn-in, and thinning makes samples approximately independent.
Invoked in Sections 2.2 and 3.3 to justify the Monte Carlo estimators in Eqs. (5) and (6).
domain assumption A generated caption's relations are virtual evidence, a model preference rather than direct observations.
Stated in Section 3.3 as the basis for the conditional distribution used to rank training examples.
domain assumption CLIP cosine similarity between text predicates and images measures the semantic match relevant to captioning.
Used in Section 3.2 to convert neural embeddings into real-valued terms inside the HMLN potentials.
domain assumption Query-specific closed-world normalization: only groundings containing test-image objects are relevant.
Introduced in Section 3.2 to ignore other groundings when learning query-specific parameters.
ad hoc to paper Explanations can be generated without accessing the captioning model's parameters, activations, or training objective.
The entire pipeline uses CLIP, scene graphs, and HMLN fitting, and never queries the model being explained; this is the key external-validity assumption.

how reviews work

0 comments

Cite this review

Pith. "Pith review of On Explaining Visual Captioning with Hybrid Markov Logic Networks." pith.science (2026). https://pith.science/paper/UXUYQNEU

@misc{pith2026250721246,
  author       = {Pith},
  title        = {Pith review of: On Explaining Visual Captioning with Hybrid Markov Logic Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UXUYQNEU}},
  note         = {Machine review of arXiv:2507.21246}
}

read the original abstract

Deep Neural Networks (DNNs) have made tremendous progress in multimodal tasks such as image captioning. However, explaining/interpreting how these models integrate visual information, language information and knowledge representation to generate meaningful captions remains a challenging problem. Standard metrics to measure performance typically rely on comparing generated captions with human-written ones that may not provide a user with a deep insights into this integration. In this work, we develop a novel explanation framework that is easily interpretable based on Hybrid Markov Logic Networks (HMLNs) - a language that can combine symbolic rules with real-valued functions - where we hypothesize how relevant examples from the training data could have influenced the generation of the observed caption. To do this, we learn a HMLN distribution over the training instances and infer the shift in distributions over these instances when we condition on the generated sample which allows us to quantify which examples may have been a source of richer information to generate the observed caption. Our experiments on captions generated for several state-of-the-art captioning models using Amazon Mechanical Turk illustrate the interpretability of our explanations, and allow us to compare these models along the dimension of explainability.

Figures

Figures reproduced from arXiv: 2507.21246 by the authors.

**Figure 2.** Three training instances are shown in the top row. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** (a) Example explanation used in the user study (b) comparing responses from AMT users across models (higher is better) (c) [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: The Y-axis shows the maximum Hellinger’s distance between the prior and the conditionals and the X-axis shows the negative [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 30 canonical work pages

[1]

Stephen H Bach, Matthias Broecheler, Bert Huang, and Lise Getoor. 2017. Hinge- loss markov random fields and probabilistic soft logic. Journal of Machine Learning Research 18, 109 (2017), 1–67

work page 2017
[2]

Adrien Bibal, Rémi Cardon, David Alfter, Rodrigo Wilkens, Xiaoou Wang, Thomas François, and Patrick Watrin. 2022. Is attention explanation? an introduc- tion to the debate. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 3889–3900

work page 2022
[3]

George Casella and Edward I George. 1992. Explaining the Gibbs sampler. The American Statistician 46, 3 (1992), 167–174

work page 1992
[4]

Anton Chechetka. 2011. Query-Specific Learning and Inference for Probabilistic Graphical Models. Carnegie Mellon University

work page 2011
[5]

Marcella Cornia, Matteo Stefanini, Lorenzo Baraldi, and Rita Cucchiara. 2020. Meshed-memory transformer for image captioning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10578–10587

work page 2020
[6]

Artur d’Avila Garcez and Luis C Lamb. 2023. Neurosymbolic AI: The 3 rd wave. Artificial Intelligence Review 56, 11 (2023), 12387–12406

work page 2023
[7]

Víctor González-Castro, Rocío Alaiz-Rodríguez, and Enrique Alegre. 2013. Class distribution estimation based on the Hellinger distance. Information Sciences 218 (2013), 146–164

work page 2013
[8]

Aditya Grover, Jiaming Song, Ashish Kapoor, Kenneth Tran, Alekh Agarwal, Eric J Horvitz, and Stefano Ermon. 2019. Bias correction of learned genera- tive models using likelihood-free importance weighting. Advances in neural information processing systems 32 (2019)

work page 2019

Show all 36 references

[9]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR

2016
[10]

Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi
[11]

Geoffrey E Hinton. 2002. Training products of experts by minimizing contrastive divergence. Neural computation 14, 8 (2002), 1771–1800

2002
[12]

Lun Huang, Wenmin Wang, Jie Chen, and Xiao-Yong Wei. 2019. Attention on attention for image captioning. In ICCV

2019
[13]

Maximilian Ilse, Jakub Tomczak, and Max Welling. 2018. Attention-based deep multiple instance learning. In ICML

2018
[14]

Andrej Karpathy and Li Fei-Fei. 2015. Deep visual-semantic alignments for generating image descriptions. In CVPR

2015
[15]

Durk P Kingma, Shakir Mohamed, Danilo Jimenez Rezende, and Max Welling
[16]

Qing Li, Jianlong Fu, Dongfei Yu, Tao Mei, and Jiebo Luo. 2018. Tell-and-answer: Towards explainable visual question answering using attributes and captions.arXiv preprint arXiv:1801.09041 (2018)

2018 arXiv
[17]

Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. Advances in neural information processing systems 30 (2017)

2017
[18]

Pranava Madhyastha, Josiah Wang, and Lucia Specia. 2019. VIFIDEL: Evaluating the Visual Fidelity of Image Descriptions. In ACL

2019
[19]

Tim Miller. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence 267 (2019), 1–38

2019
[20]

Yingwei Pan, Ting Yao, Yehao Li, and Tao Mei. 2020. X-linear attention networks for image captioning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10971–10980

2020
[21]

Ankur Parikh, Hoifung Poon, and Kristina Toutanova. 2015. Grounded semantic parsing for complex knowledge extraction. InProceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 756–766

2015
[22]

Domingos Pedro and D Lowd. 2009. Markov logic: An interface layer for artificial intelligence. Synth. Lect. Artific. Intell. Mach. Learn 3, 1 (2009), 1–155

2009
[23]

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In ICML

2021
[24]

Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. InNeurIPS

2015
[25]

Why should i trust you?

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. " Why should i trust you?" Explaining the predictions of any classifier. InProceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 1135–1144

2016
[26]

Sebastian Schuster, Ranjay Krishna, Angel Chang, Li Fei-Fei, and Christopher D Manning. 2015. Generating semantically precise scene graphs from textual de- scriptions for improved image retrieval. In Workshop on vision and language

2015
[27]

Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedan- tam, Devi Parikh, and Dhruv Batra. 2017. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE inter- national conference on computer vision. 618–626

2017
[28]

Gal Shalev, Gabi Shalev, and Joseph Keshet. 2022. A baseline for detecting out-of-distribution examples in image captioning. In Proceedings of the 30th ACM International Conference on Multimedia. 4175–4184

2022
[29]

Zhan Shi, Xu Zhou, Xipeng Qiu, and Xiaodan Zhu. 2020. Improving image captioning with better use of captions. In ACL

2020
[30]

Parag Singla and Pedro Domingos. 2005. Discriminative training of Markov logic networks. In AAAI, V ol. 5. 868–873

2005
[31]

Jiamei Sun, Sebastian Lapuschkin, Wojciech Samek, and Alexander Binder. 2020. Understanding image captioning models beyond visualizing attention. arXiv preprint arXiv:2001.01037 (2020)

2020 arXiv
[32]

Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In International conference on machine learning . PMLR, 3319–3328

2017
[33]

Jue Wang and Pedro M Domingos. 2008. Hybrid Markov Logic Networks.. In AAAI, V ol. 8. 1106–1111

2008
[34]

Jialin Wu and Raymond J Mooney. 2018. Faithful multimodal explanation for visual question answering. arXiv preprint arXiv:1809.02805 (2018)

2018 arXiv
[2014]

Advances in neural information processing systems 27 (2014)

Semi-supervised learning with deep generative models. Advances in neural information processing systems 27 (2014)

2014
[2021]

In EMNLP

CLIPScore: A Reference-free Evaluation Metric for Image Captioning. In EMNLP

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Stephen H Bach, Matthias Broecheler, Bert Huang, and Lise Getoor. 2017. Hinge- loss markov random fields and probabilistic soft logic. Journal of Machine Learning Research 18, 109 (2017), 1–67

work page 2017

[2] [2]

Adrien Bibal, Rémi Cardon, David Alfter, Rodrigo Wilkens, Xiaoou Wang, Thomas François, and Patrick Watrin. 2022. Is attention explanation? an introduc- tion to the debate. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 3889–3900

work page 2022

[3] [3]

George Casella and Edward I George. 1992. Explaining the Gibbs sampler. The American Statistician 46, 3 (1992), 167–174

work page 1992

[4] [4]

Anton Chechetka. 2011. Query-Specific Learning and Inference for Probabilistic Graphical Models. Carnegie Mellon University

work page 2011

[5] [5]

Marcella Cornia, Matteo Stefanini, Lorenzo Baraldi, and Rita Cucchiara. 2020. Meshed-memory transformer for image captioning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10578–10587

work page 2020

[6] [6]

Artur d’Avila Garcez and Luis C Lamb. 2023. Neurosymbolic AI: The 3 rd wave. Artificial Intelligence Review 56, 11 (2023), 12387–12406

work page 2023

[7] [7]

Víctor González-Castro, Rocío Alaiz-Rodríguez, and Enrique Alegre. 2013. Class distribution estimation based on the Hellinger distance. Information Sciences 218 (2013), 146–164

work page 2013

[8] [8]

Aditya Grover, Jiaming Song, Ashish Kapoor, Kenneth Tran, Alekh Agarwal, Eric J Horvitz, and Stefano Ermon. 2019. Bias correction of learned genera- tive models using likelihood-free importance weighting. Advances in neural information processing systems 32 (2019)

work page 2019