Pith. sign in

REVIEW 2 major objections 40 references

Falcon: Functional Assembly and Language for Compositional Reasoning in X-ray

T0 review · 2 major / 0 minor · reviewed 2026-06-30 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Falcon improves compositional threat reasoning in X-ray images by injecting structured safety states into language models.

desk verdict Falcon frames compositional threat reasoning in X-ray screening via an injected structured safety state, but the abstract gives no results or ablations to show whether the structure actually changes model behavior. read the letter →

arxiv 2606.25701 v2 pith:25ATGYAZ submitted 2026-06-24 cs.CV

Yonathan Michael , Mohamad Alansari , Natnael Takele , Andreas Henschel , Naoufel Werghi This is my paper · ORCID
classification cs.CV
keywords compositionalthreatreasoningX-raybaggagescreeningmultimodalmodelsfunctionalcompatibilitystructuredsafetystateFalcon-Xbenchmarkvision-languageassessment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that vision-language models focus on individual objects and therefore miss threats that arise only from the functional compatibility of multiple components in X-ray baggage scans. It introduces Falcon, which extracts segmentation-aware region features and assembles them into an explicit structured safety state that records component presence, pairwise functional relations, and scene-level risk. This state is supplied to the language model as an intermediate interface so that reasoning respects relational constraints rather than appearance alone. On the new Falcon-X benchmark the method yields more coherent threat assessments than standard multimodal models, which adapt to visual patterns but fail at compositional safety inference.

What carries the argument

The structured safety state (component presence, pairwise functional compatibility, scene-level risk) injected as an explicit intermediate interface to the language model.

What would settle it

A controlled test in which the structured state explicitly signals incompatible components and high risk, yet the model still produces an inconsistent low-risk output.

Watch

Extended reading notes

Core claim

Falcon abstracts segmentation-aware region features into a structured safety state capturing component presence, pairwise functional compatibility, and scene-level risk. This structured representation is injected into the language model as an explicit intermediate interface, encouraging relationally consistent and safety-aware reasoning about threats that emerge from combinations of spatially dispersed components rather than from any single object.

Load-bearing premise

That the language model will actually use the injected structured safety state for reasoning rather than learning to ignore or override it.

Editorial extensions

If this is right

  • Existing multimodal models adapt to appearance but struggle with compositional safety reasoning.
  • Falcon improves functional grounding of components in cluttered X-ray imagery.
  • Falcon produces more coherent threat assessments than appearance-only baselines.
  • Compositional safety reasoning constitutes a distinct evaluation paradigm separate from standard object detection.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same state-injection pattern could be tested in other relational safety domains such as industrial inspection or medical imaging.
  • An ablation that removes the explicit state after training would reveal whether the model has internalized the relational constraints or continues to rely on the injection.
  • Extending the structured state to include temporal relations could address video-based screening scenarios.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 0 minor

Summary. The paper introduces Falcon, a multimodal framework for compositional threat reasoning in X-ray baggage screening. It abstracts segmentation-aware region features into an explicit structured safety state (component presence, pairwise functional compatibility, scene-level risk) that is injected into the language model as an intermediate interface. The authors also present the Falcon-X benchmark unifying dense grounding with structured supervision over component completeness and risk inference. The central claim is that existing multimodal models adapt to appearance but struggle with compositional safety reasoning, while Falcon improves functional grounding and produces more coherent threat assessments, establishing compositional safety reasoning as a distinct evaluation paradigm.

Significance. If the empirical claims hold with rigorous validation, the work could define a new evaluation axis for multimodal systems focused on relational and functional reasoning in safety-critical domains rather than object-centric detection. The explicit injection of a structured safety state is a concrete architectural choice that could be tested for whether it enforces consistency beyond what appearance-only models achieve.

major comments (2)
  1. [Abstract] Abstract: the assertion that 'Experiments show that while existing multimodal models adapt to appearance, they struggle with compositional safety reasoning. Falcon improves functional grounding...' supplies no quantitative results, ablation studies, error analysis, dataset statistics, or baseline comparisons. Without these, the central claim that Falcon produces more coherent threat assessments cannot be evaluated.
  2. [Abstract] Abstract (and implied methods): the design assumes that explicit injection of the structured safety state will produce relationally consistent reasoning rather than being overridden or ignored by the language model. No implementation details, training procedure, or ablation isolating the effect of the injected state versus appearance features are referenced, leaving the weakest assumption untested.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their thoughtful feedback on our manuscript. We address each major comment below with references to the full paper content.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the assertion that 'Experiments show that while existing multimodal models adapt to appearance, they struggle with compositional safety reasoning. Falcon improves functional grounding...' supplies no quantitative results, ablation studies, error analysis, dataset statistics, or baseline comparisons. Without these, the central claim that Falcon produces more coherent threat assessments cannot be evaluated.

    Authors: The abstract is a concise high-level summary and does not contain quantitative details by standard convention. The full manuscript provides these elements in Section 4 (Experiments) and the appendix: quantitative results and baseline comparisons on Falcon-X, ablation studies on the structured safety state components, error analysis of threat assessments, and dataset statistics including component distributions and risk annotations. These support the central claim of improved functional grounding and coherent assessments. revision: partial

  2. Referee: [Abstract] Abstract (and implied methods): the design assumes that explicit injection of the structured safety state will produce relationally consistent reasoning rather than being overridden or ignored by the language model. No implementation details, training procedure, or ablation isolating the effect of the injected state versus appearance features are referenced, leaving the weakest assumption untested.

    Authors: The Methods section (Section 3) details the abstraction of segmentation-aware region features into the structured safety state (component presence, pairwise functional compatibility, scene-level risk) and its explicit injection into the language model as an intermediate interface. The training procedure, including fine-tuning with Falcon-X's structured supervision, is described. Ablation studies isolating the injected safety state versus appearance-only features appear in Section 4, showing gains in relational consistency. If these sections require expansion for clarity, we will revise accordingly. revision: no

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity detected

full rationale

The provided manuscript text contains no equations, derivations, parameter fittings, or formal predictions that could reduce to their own inputs by construction. The central contribution is a new multimodal framework and benchmark whose claims rest on experimental comparisons against appearance-only baselines; these evaluations are independent of any self-referential definitions or self-citation chains. No load-bearing steps match the enumerated circularity patterns.

Assumptions & free parameters 0 free parameters · 1 assumptions · 1 invented entities

Only the abstract is available, so the ledger records only the high-level premises stated in the text; no implementation-level parameters or proofs can be audited.

assumptions (1)
  • domain assumption Threat often emerges not from a single object but from the functional compatibility of spatially dispersed components.
    Explicitly stated as the motivation for formalizing compositional threat reasoning.
invented entities (1)
  • structured safety state
    purpose: Captures component presence, pairwise functional compatibility, and scene-level risk as an explicit intermediate interface for the language model.
    Introduced as the core abstraction of the Falcon framework; no independent evidence outside the paper is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Falcon: Functional Assembly and Language for Compositional Reasoning in X-ray." pith.science (2026). https://pith.science/paper/25ATGYAZ

@misc{pith2026260625701,
  author       = {Pith},
  title        = {Pith review of: Falcon: Functional Assembly and Language for Compositional Reasoning in X-ray},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/25ATGYAZ}},
  note         = {Machine review of arXiv:2606.25701}
}
read the original abstract

Conventional vision-language models are largely object-centric, focusing on detecting and describing individual entities. In safety-critical X-ray baggage screening, however, threat often emerges not from a single object but from the functional compatibility of spatially dispersed components, such as batteries, detonators, and explosive charges. We formalize this setting as \emph{compositional threat reasoning}, where risk is modeled as a relational property of grounded regions rather than an independent detection outcome. We introduce \textbf{Falcon}, a multimodal framework that abstracts segmentation-aware region features into a structured safety state capturing component presence, pairwise functional compatibility, and scene-level risk. This structured representation is injected into the language model as an explicit intermediate interface, encouraging relationally consistent and safety-aware reasoning. To evaluate this problem, we present \textbf{Falcon-X}, a benchmark that unifies dense grounding with structured supervision over component completeness and risk inference in cluttered X-ray imagery. Experiments show that while existing multimodal models adapt to appearance, they struggle with compositional safety reasoning. Falcon improves functional grounding and produces more coherent threat assessments, establishing compositional safety reasoning as a distinct evaluation paradigm for multimodal systems.

Figures

Figures reproduced from arXiv: 2606.25701 by the authors.

Figure 1
Figure 1. Falcon enables segmentation-aware functional threat reasoning in X-ray im￾agery. It performs instance grounding, component presence recognition, referring func￾tional grounding and more, while generating grounded and domain-aware natural￾language explanations. Falcon reasons over spatially distant components of IED under heavy clutter, supporting structured threat analysis beyond object detection. Abstract. Conventi… view at source ↗
Figure 2
Figure 2. Qualitative comparison of structured threat reasoning. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Overview of the Falcon-X data collection pipeline. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Falcon-X Task Suite. Left: Example of multi-level tasks on Falcon-X base image. Right: Samples of compositional grounding on controlled synthetic Falcon-X images for functional reasoning. 5 Falcon-X Task Suite Falcon-X introduces a hierarchical evaluation framework for…
Figure 5
Figure 5. Figure 5: Falcon architecture. Segmentation-aware perception extracts mask-aligned region embeddings, which are aggregated into structured component slots by the SSA to predict presence, functional links, and scene risk. The resulting structured tokens are fused with visual and …
Figure 6
Figure 6. Figure 6: Falcon qualitative results on compositional grounding and scene-level reasoning. 7.4 Ablations We conduct controlled ablations to quantify the contribution of (i) structured prediction heads and (ii) perception versus reasoning bottlenecks. All variants are evaluated o…
Figure 7
Figure 7. Figure 7: Acquisition setup and collection protocol. [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Examples of counterfactual synthetic samples used in Falcon-X. Starting from a real X-ray image (left), controlled variants are generated by removing one functional component using mask-guided inpainting. From left to right, each row shows: the orig￾inal image, missing…
Figure 9
Figure 9. Figure 9: Qualitative examples of Falcon on RFG. Given a functionally constrained query, the model grounds all visible components that could jointly participate in a potential IED assembly. VQA [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]
Figure 10
Figure 10. Figure 10: Qualitative examples of Falcon on domain-specific VQA in X-ray [PITH_FULL_IMAGE:figures/full_fig_p023_10.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 40 canonical work pages

  1. [1]

    In: ICCV (2025)

    Cai, Z., Ke, F., Jahangard, S., Garcia de la Banda, M., Haffari, R., Stuckey, P.J., Rezatofighi, H.: Naver: A neuro-symbolic compositional automaton for vi- sual grounding with explicit logic reasoning. In: ICCV (2025)

    work page 2025

  2. [2]

    Knowledge- Based Systems (2022)

    Chang,A.,Zhang,Y.,Zhang,S.,Zhong,L.,Zhang,L.:Detectingprohibitedobjects with physical size constraint from cluttered x-ray baggage images. Knowledge- Based Systems (2022)

    work page 2022

  3. [3]

    Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic

    Chen, K., Zhang, Z., Zeng, W., Zhang, R., Zhu, F., Zhao, R.: Shikra: Unleash- ing multimodal llm’s referential dialogue magic. arXiv preprint arXiv:2306.15195 (2023)

    work page Pith review arXiv 2023

  4. [4]

    Chiang, W.L., Li, Z., Lin, Z., Sheng, Y., Wu, Z., Zhang, H., Zheng, L., Zhuang, S., Zhuang, Y., Gonzalez, J.E., Stoica, I., Xing, E.P.: Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality (March 2023),https://lmsys.org/ blog/2023-03-30-vicuna/

    work page 2023

  5. [5]

    United States Patent Application US20160161228A1 (June 2016)

    Eshetu, A., Burton, T.B., Howell, J.D., Rutter, M.F., Winnett, T.J.: Inert ied training kits. United States Patent Application US20160161228A1 (June 2016)

    work page 2016

  6. [6]

    In: ICCV (2025)

    Garcia-Fernandez,P.,Vaquero,L.,Liu,M.,Xue,F.,Cores,D.,Sebe,N.,Mucientes, M., Ricci, E.: Superpowering open-vocabulary object detectors for x-ray vision. In: ICCV (2025)

    work page 2025

  7. [7]

    CIPAE (2023)

    He, C., Mu, T., Ren, W., Zhao, B.: Lpixray: A large-scale logistics prohibited item x-ray dataset for the application of deep learning in security inspection. CIPAE (2023)

    work page 2023

  8. [8]

    In: ICCV (2017) 16 Y

    He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask r-cnn. In: ICCV (2017) 16 Y. Michael, M. Alansari et al

    work page 2017

Show all 40 references
  1. [9]

    In: ICLR (2022)

    Hu, E.J., yelong shen, Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. In: ICLR (2022)

    2022

  2. [10]

    In: ICCV (2025)

    Lafon, M., Karmim, Y., Silva-Rodríguez, J., Couairon, P., Rambour, C., Fournier- Sniehotta, R., Ayed, I.B., Dolz, J., Thome, N.: Vilu: Learning vision-language uncertainties for failure prediction. In: ICCV (2025)

    2025

  3. [11]

    In: CVPR (2024)

    Lai, X., Tian, Z., Chen, Y., Li, Y., Yuan, Y., Liu, S., Jia, J.: Lisa: Reasoning segmentation via large language model. In: CVPR (2024)

    2024

  4. [12]

    In: ICML (2023)

    Li, J., Li, D., Savarese, S., Hoi, S.: Blip-2: bootstrapping language-image pre- training with frozen image encoders and large language models. In: ICML (2023)

    2023

  5. [13]

    TNNLS36(2024)

    Li, M., Jia, T., Wang, H., Ma, B., Lu, H., Lin, S., Cai, D., Chen, D.: Ao-detr: Anti-overlapping detr for x-ray prohibited items detection. TNNLS36(2024)

    2024

  6. [14]

    In: ECCV (2014)

    Lin, T.Y., Maire, M., Belongie, S.J., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: ECCV (2014)

    2014

  7. [15]

    In: CVPR (2023)

    Liu, C., Ding, H., Jiang, X.: GRES: Generalized referring expression segmentation. In: CVPR (2023)

    2023

  8. [16]

    Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning (2023)

    2023

  9. [17]

    ITIFS19, 3866–3878 (2024)

    Ma, B., Jia, T., Li, M., Wu, S., Wang, H., Chen, D.: Toward dual-view x-ray baggage inspection: A large-scale benchmark and adaptive hierarchical cross re- finement for prohibited item discovery. ITIFS19, 3866–3878 (2024)

    2024

  10. [18]

    ITMM25, 4374–4386 (2022)

    Ma, B., Jia, T., Su, M., Jia, X., Chen, D., Zhang, Y.: Automated segmentation of prohibited items in x-ray baggage images using dense de-overlap attention snake. ITMM25, 4374–4386 (2022)

    2022

  11. [19]

    In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G

    Ma, C., Jiang, Y., Wu, J., Yuan, Z., Qi, X.: Groma: Localized visual tokenization for grounding multimodal large language models. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds.) Computer Vision – ECCV 2024. pp. 417–435. Springer Nature Sw...

    2024

  12. [20]

    In: ICLR (2019)

    Mao, J., Gan, C., Kohli, P., Tenenbaum, J.B., Wu, J.: The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision. In: ICLR (2019)

    2019

  13. [21]

    Journal of Nondestructive Evaluation (2015)

    Mery, D., Riffo, V., Zscherpel, U., Mondragón, G., Lillo, I., Zuccar, I., Lobel, H., Carrasco, M.: Gdxray: The database of x-ray images for nondestructive testing. Journal of Nondestructive Evaluation (2015)

    2015

  14. [22]

    In: CVPR (2019)

    Miao, C., Xie, L., Wan, F., Su, c., Liu, H., Jiao, j., Ye, Q.: Sixray: A large-scale security inspection x-ray benchmark for prohibited item discovery in overlapping images. In: CVPR (2019)

    2019

  15. [23]

    In- formation Processing & Management63(1) (2026)

    Michael, Y., Alansari, M., Ahmed, A., Werghi, N., Henschel, A.: X-ssl: Self- supervised x-ray threat detection with zero-shot and multi-modal learning. In- formation Processing & Management63(1) (2026)

    2026

  16. [24]

    Transactions on Machine Learning Research (2024)

    Oquab, M., Darcet, T., Moutakanni, T., Vo, H.V., Szafraniec, M., Khalidov, V., Fernandez, P., HAZIZA, D., Massa, F., El-Nouby, A., Assran, M., Ballas, N., Galuba, W., Howes, R., Huang, P.Y., Li, S.W., Misra, I., Rabbat, M., Sharma, V., Synnaeve, G., Xu, H., Jegou, H., Mairal, ...

    2024

  17. [25]

    ArXiv abs/2306.14824(2023)

    Peng, Z., Wang, W., Dong, L., Hao, Y., Huang, S., Ma, S., Wei, F.: Kosmos-2: Grounding multimodal large language models to the world. ArXiv abs/2306.14824(2023)

    2023 arXiv

  18. [26]

    In: ICML (2021) Falcon 17

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transferable visual models from natural language supervision. In: ICML (2021) Falcon 17

    2021

  19. [27]

    CVPR (2024)

    Rasheed, H., Maaz, M., Shaji, S., Shaker, A., Khan, S., Cholakkal, H., Anwer, R.M., Xing, E., Yang, M.H., Khan, F.S.: Glamm: Pixel grounding large multimodal model. CVPR (2024)

    2024

  20. [28]

    In: ICLR (2026)

    Robinson,I.,Robicheaux,P.,Popov,M.,Ramanan,D.,Peri,N.:RF-DETR:Neural architecture search for real-time detection transformers. In: ICLR (2026)

    2026

  21. [29]

    In: CVPR (2022)

    Tao, R., Li, H., Wang, T., Wei, Y., Ding, Y., Bowei Jin and, H.Z., Liu, X., Liu, A.: Exploring endogenous shift for cross-domain detection: A large-scale benchmark and perturbation suppression network. In: CVPR (2022)

    2022

  22. [30]

    In: ICCV (2021)

    Tao, R., Wei, Y., Jiang, X., Li, H., Qin, H., Wang, J., Ma, Y., Zhang, L., Liu, X.: Towards real-world x-ray security inspection: A high-quality benchmark and lateral inhibition module for prohibited items detection. In: ICCV (2021)

    2021

  23. [31]

    In: ICCV (2021)

    Tao, R., Wei, Y., Jiang, X., Li, H., Qin, H., Wang, J., Ma, Y., Zhang, L., Liu*, X.: Towards real-world x-ray security inspection: A high-quality benchmark and lateral inhibition module for prohibited items detection. In: ICCV (2021)

    2021

  24. [32]

    In: CVPR (2025)

    Velayudhan, D., Ahmed, A., Alansari, M., Gour, N., Behouch, A., Hassan, T., Wasim, S.T., Maalej, N., Naseer, M., Gall, J., et al.: Sting-bee: Towards vision- language model for real-world x-ray baggage security inspection. In: CVPR (2025)

    2025

  25. [33]

    ACM Computing Surveys55(8) (2022)

    Velayudhan, D., Hassan, T., Damiani, E., Werghi, N.: Recent advances in bag- gage threat detection: A comprehensive and systematic survey. ACM Computing Surveys55(8) (2022)

    2022

  26. [34]

    In: ACMMM (2020)

    Wei, Y., Tao, R., Wu, Z., Ma, Y., Zhang, L., Liu, X.: Occluded prohibited items detection:Anx-raysecurityinspectionbenchmarkandde-occlusionattentionmod- ule. In: ACMMM (2020)

    2020

  27. [35]

    arXiv (2025)

    Yuan, H., Li, X., Zhang, T., Huang, Z., Xu, S., Ji, S., Tong, Y., Qi, L., Feng, J., Yang, M.H.: Sa2va: Marrying sam2 with llava for dense grounded understanding of images and videos. arXiv (2025)

    2025

  28. [36]

    IJCV (2023)

    Zhang, L., Jiang, L., Ji, R., Fan, H.: Pidray: A large-scale x-ray benchmark for real-world prohibited item detection. IJCV (2023)

    2023

  29. [37]

    In: Del Bue, A., Canton, C., Pont-Tuset, J., Tommasi, T

    Zhang, S., Sun, P., Chen, S., Xiao, M., Shao, W., Zhang, W., Liu, Y., Chen, K., Luo, P.: Gpt4roi: Instruction tuning large language model on region-of-interest. In: Del Bue, A., Canton, C., Pont-Tuset, J., Tommasi, T. (eds.) Computer Vision – ECCV 2024 Workshops. pp. 52–70. Sp...

    2024

  30. [38]

    ITIFS17, 998–1009 (2022)

    Zhao, C., Zhu, L., Dou, S., Deng, W., Wang, L.: Detecting overlapped objects in x-ray security imagery by a label-aware mechanism. ITIFS17, 998–1009 (2022)

    2022

  31. [39]

    scene-caption

    Zhu, D., Chen, J., Shen, X., Li, X., Elhoseiny, M.: Minigpt-4: Enhancing vision- language understanding with advanced large language models. In: ICLR (2024) Falcon 1 Appendix –Additional Details on Falcon-X (section 9) –Additional Details on Falcon-X Task Suite Generation (sec...

    2024

  32. [40]

    Which components could possibly form an IED. Ground all that apply

    Scene captions must describe only visible components. 2. Referring instructions must be uniquely resolvable from spatial metadata. 3. VQA questions must be answerable from structured annotations. 4. Functional links must be inferred using spatial proximity and component types....

Pith tools

Reviewed June 30, 2026 · model on record in the stance chip above.