Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

LifeIR at the NTCIR-18 Lifelog-6 Task

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The authors claim that every added module in their lifelog image-search pipeline improves retrieval, with a final mAP of 0.2652 on the NTCIR-18 Lifelog-6 test topics.

desk verdict A competent NTCIR system paper whose per-stage effectiveness claim is contradicted by its own Table 3; the final result is plausible but the ablation logic needs an honest rewrite. read the letter →

arxiv 2505.20987 v1 pith:ZDXOLKMC submitted 2025-05-27 cs.IR

Jiahan Chen , Da Li , Keping Bi This is my paper
classification cs.IR
keywords lifelogretrievalcross-modalCLIPmultimodallargelanguagemodelqueryrewritingevent-basedcandidateexpansionNTCIR-18Lifelog-6automaticsystem
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper describes an automated retrieval system for finding historical moments in a personal lifelog archive from a text query, and claims that each of its stages—blur filtering, query rewriting, event-based candidate expansion, and multimodal-LLM reranking—contributes to better retrieval quality. The final system reports mAP 0.2652 and P@10 0.3038 on the 26 topics of the NTCIR-18 Lifelog-6 LSAT task. The paper's contribution is a modular, fully automatic recipe that combines a pretrained image-text model with a large language model for query refinement and a multimodal LLM for final relevance judgment.

What carries the argument

The pipeline's central objects are: (1) CLIP's shared image-text embedding space for initial relevance scoring via cosine similarity; (2) a visual event defined as a sequence of temporally adjacent images with high CLIP similarity, used to compute event-level query similarity; (3) LLM-based query rewriting that compresses the topic's title, description, and narrative into a first-person query of at most 30 words; (4) event-based candidate expansion that takes the top-100 events, finds peak hour-level time intervals, and expands each peak by 80 preceding and 80 following images using timestamp-named files; and (5) Qwen2-VL as a multimodal large language model that reranks candidates by judging query-image relevance, optionally augmented with location metadata. Each component is designed to address one of the three lifelog characteristics: heterogeneity, temporal connection, and query intent specificity.

What would settle it

Observe that LSAT04 (temporal expansion added) scores mAP 0.1887, below LSAT03's 0.1905, already contradicting the claim. A decisive test would be a repeated run (multiple seeds or topic-wise significance testing) of the same two configurations, or an LSAT01-plus-temporal-expansion run without MLLM filtering; if temporal expansion does not reliably beat the configuration without it, the 'every stage helps' conclusion collapses.

Watch

Extended reading notes

Core claim

The paper sets out to show that a five-stage automatic pipeline—blur filtering, CLIP-based retrieval, LLM query rewriting, event-based candidate expansion, and MLLM posterior filtering—handles the known-item lifelog search task, and that each added stage independently improves retrieval quality. Its reported results on the 26 LSAT topics are mAP 0.2652, P@10 0.3038, and R@10 0.2617. The paper attributes the gains to sharper query intent after rewriting, better recall from exploiting temporal continuity of events, and noise suppression from multimodal reranking.

Load-bearing premise

The load-bearing premise is that the single submitted run for each configuration cleanly isolates the effect of the newly added module; the paper's own numbers show that adding temporal expansion lowers mAP and other metrics from LSAT03 to LSAT04, so this premise fails for at least one stage.

Editorial extensions

If this is right

  • If the pipeline's per-stage gains are real, then each module is a drop-in addition to a CLIP-based retrieval system, not a change that requires retraining.
  • The reported mAP 0.2652, P@10 0.3038, and R@10 0.2617 would be the system's expected performance on the 26 NTCIR-18 topics.
  • The approach suggests that temporal coherence can be exploited cheaply through file-naming order rather than through expensive video-level models.
  • For lifelog retrieval practice, the pipeline would mean that image-caption embeddings, an LLM rewrite step, and a multimodal reranker are sufficient for a fully automatic known-item search.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own Table 3 cuts against its 'every stage helps' claim: LSAT04 (temporal expansion added on top of LSAT03) drops to mAP 0.1887 from 0.1905 and also lowers P@100 and R@10, so a single-run ablation cannot cleanly attribute the later gains to the earlier-stated logic.
  • A natural experiment would be to rerun LSAT03 and LSAT04 multiple times, or to add temporal expansion to LSAT01 directly; until then the effectiveness of the temporal-expansion stage is unsupported.
  • The multi-round event expansion (LSAT05 to LSAT06) accounts for the biggest single jump, suggesting the iterative query update, not the temporal peak expansion, carries the recall gain.
  • The design relies on timestamp-based filenames; applying the temporal expansion to archives without such naming, or with irregular capture intervals, would likely erase the gain—a testable boundary condition.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper reports the LifeIR system's participation in the NTCIR-18 Lifelog-6 LSAT automatic subtask. The proposed pipeline consists of blurred-image filtering, LLM-based query rewriting, CLIP-based retrieval, event-based candidate-set expansion, and MLLM-based re-ranking/filtering. The authors submitted five valid runs and report that their final run (LSAT06) achieves mAP 0.2652, P@10 0.3038, and R@10 0.2617 on 26 topics. The abstract and Section 4.2 claim that the results demonstrate the effectiveness of each stage and the entire pipeline.

Significance. If fully supported, the paper would provide a useful component-level validation of common design choices in lifelog retrieval (query rewriting, event expansion, MLLM re-ranking) on a standard shared-task benchmark. Strengths include the use of official relevance judgments, a clear description of the pipeline, and a complete report of all submitted runs rather than only the best one. However, the stage-level claim is not currently established by the reported numbers, because the only quasi-isolated ablation in Table 3 shows a performance drop on multiple metrics and no statistical uncertainty is reported.

major comments (3)
  1. [Section 4.2 / Table 3] The claim that 'the addition of each of our methods has improved the evaluation' is contradicted by the paper's own results. Comparing LSAT03 to LSAT04, which adds temporal-based candidate expansion to an otherwise identical pipeline, mAP falls from 0.1905 to 0.1887, P@10 from 0.2769 to 0.2731, P@100 from 0.1373 to 0.1323, and R@10 from 0.1696 to 0.1693. Similarly, adding MLLM filtering (LSAT01 vs LSAT03) improves mAP and P@10 but decreases P@100 (0.1496 to 0.1373) and R@10 (0.1741 to 0.1696). The abstract's statement that 'the effectiveness of each stage' is shown is not supported by these numbers; please either revise the claim or supply evidence that the differences are significant and specific to each module.
  2. [Section 4.2 / Table 2] The run sequence is not an incremental ablation. LSAT04 to LSAT05 replaces temporal-based expansion with single-round event-based expansion rather than adding a component, and according to the text in Section 4.2 LSAT05 also includes metadata integration not listed in Table 2. Therefore the improvement from LSAT04 to LSAT05 cannot be attributed to event-based expansion alone. A proper ablation would keep all other components fixed while toggling one module, or the paper should explicitly frame these runs as system variants rather than a controlled ablation.
  3. [Table 3 / Section 4.2] All runs are single submissions with no variance estimates, confidence intervals, or significance tests. Because the differences between adjacent runs are small in several cases (e.g., mAP 0.1905 vs. 0.1887) and the evaluation has only 26 topics, the observed differences may be within noise. Please provide per-topic scores, bootstrap intervals, or a paired test (e.g., a permutation test on mAP and P@10) to support the stage-level attribution.
minor comments (4)
  1. [Section 1] The task description contains typos: 'know-item search task' should be 'known-item search task' (the correct form appears later in the same section), and 'lifelong retrieval' should be 'lifelog retrieval'.
  2. [Figure 2] The caption reads 'A Case of Rewriting Rewriting' and should be corrected, for example to 'A Case of Query Rewriting'.
  3. [Section 4.3 / Reference [10]] Reference [10] is cited for CLIP's limitations in fine-grained visual understanding, but [10] is a paper about a prognostic staging system for hepatocellular carcinoma; the citation does not appear to support the claim.
  4. [Section 3.1] The phrase 'The dataset provided over 725k images' is awkward and ambiguous; consider 'The dataset contains over 725,000 images' and clarify whether the count is before or after blur filtering.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found; the effectiveness claims are judged against external NTCIR relevance judgments, with only a non-circular ablation-validity caveat from Table 3.

full rationale

The paper makes no mathematical derivation whose outputs are fed back into its inputs: each module (blur filtering, CLIP retrieval, query rewriting, temporal/event-based expansion, Qwen2-VL posterior filtering) is evaluated against the NTCIR-18 organizer's external relevance judgments, so no fitted parameter is renamed as a prediction and no self-defined quantity is used to prove itself. The central claim, 'The evaluation results of our submissions have shown the effectiveness of each stage and the entire pipeline,' is an empirical submission comparison rather than a derived result, and the references it relies on (CLIP, Qwen2-VL, prior NTCIR and LSC overviews) are all external; there are no self-citations, no imported uniqueness theorem, and no ansatz smuggled in from the authors' own prior work. One data-quality caveat is worth flagging explicitly at Section 4.2 and Table 3 but it is not circularity: adding temporal expansion (LSAT04 vs LSAT03) lowers mAP from 0.1905 to 0.1887 and P@10 from 0.2769 to 0.2731, and adding MLLM filtering (LSAT03 vs LSAT01) lowers P@100 and R@10. That is a soundness problem about single-run, no-variance ablation comparisons, not a case in which a result reduces by construction to its own inputs. Because no circular step can be quoted and exhibited, the correct score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper's results depend mainly on hand-chosen pipeline hyperparameters (window sizes, top-k counts, prompt constraints) and on domain assumptions about lifelog temporal coherence and the reliability of CLIP and Qwen2-VL. No new theoretical entities or fitted scientific parameters are introduced.

free parameters (5)
  • blur threshold = not reported (mean edge-weight of manually judged blurry sample)
    Section 3.1: images sampled and manually judged blurry; their mean edge weight sets the filter threshold; threshold is hand-set from data.
  • temporal expansion window = 80 preceding and 80 subsequent images
    Section 3.4: fixed window size chosen for expanding candidate set around peak intervals; no tuning analysis reported.
  • event candidate set size = top 100 events
    Section 3.4: top-100 events taken for initial retrieval and expansion rounds; fixed by hand.
  • query expansion image count = top 5 images
    Section 3.4: features of top 5 images summed with query for the new representation.
  • rewritten query length limit = 30 words
    Section 3.3: prompt constrains rewritten query to at most 30 words to suit CLIP.
assumptions (5)
  • domain assumption Lifelog images exhibit temporal continuity, so temporally adjacent images are semantically related.
    Section 3.4: the event-based expansion relies on this to expand candidates with nearby timestamps.
  • domain assumption CLIP embeddings capture text-image similarity sufficient for first-stage retrieval.
    Section 3.2: CLIP is used without fine-tuning; the paper assumes its pretrained joint embedding is adequate.
  • domain assumption Qwen2-VL's relevance judgments are reliable enough to filter and rerank candidates.
    Section 3.5: the MLLM is the final filter; no validation of its judgments beyond the final metric.
  • domain assumption Image file names encode capture timestamps that can be used to order images.
    Section 3.4: temporal expansion depends on timestamp-based naming conventions.
  • domain assumption The 26 NTCIR-18 queries and relevance judgments are a fair evaluation of the system.
    Section 4: results rely on official topics and judgments; the paper does not scrutinize the annotation quality.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LifeIR at the NTCIR-18 Lifelog-6 Task." pith.science (2026). https://pith.science/paper/ZDXOLKMC

@misc{pith2026250520987,
  author       = {Pith},
  title        = {Pith review of: LifeIR at the NTCIR-18 Lifelog-6 Task},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZDXOLKMC}},
  note         = {Machine review of arXiv:2505.20987}
}
read the original abstract

In recent years, sharing lifelogs recorded through wearable devices such as sports watches and GoPros, has gained significant popularity. Lifelogs involve various types of information, including images, videos, and GPS data, revealing users' lifestyles, dietary patterns, and physical activities. The Lifelog Semantic Access Task(LSAT) in the NTCIR-18 Lifelog-6 Challenge focuses on retrieving relevant images from a large scale of users' lifelogs based on textual queries describing an action or event. It serves users' need to find images about a scenario in the historical moments of their lifelogs. We propose a multi-stage pipeline for this task of searching images with texts, addressing various challenges in lifelog retrieval. Our pipeline includes: filtering blurred images, rewriting queries to make intents clearer, extending the candidate set based on events to include images with temporal connections, and reranking results using a multimodal large language model(MLLM) with stronger relevance judgment capabilities. The evaluation results of our submissions have shown the effectiveness of each stage and the entire pipeline.

Figures

Figures reproduced from arXiv: 2505.20987 by the authors.

Figure 1
Figure 1. Overview of Our Method. encompasses information such as time, physical activities, biomet￾rics, and locations when the image was taken. The visual concepts include detected scenes and concepts for each image. All images in the dataset were obtained from one GoPro wearer. There are a total of 26 topics in the NTCIR-18 Lifelog-6, con￾sisting of 13 ad-hoc topics and 13 known-item topics. All topics are made up of Title… view at source ↗
Figure 2
Figure 2. A Case of Rewriting Rewriting. The italicized text is the output of the LLM during the rewrite process. 3.4 Event-based Candidate Set Expansion Temporal connection. In the NTCIR-18 Lifelog-6, Queries are often aimed at searching for one or more semantic events. Semantic events are moments in everyday life that demonstrate a specific activity. They are usually temporally continuous, depending on the duration of the a… view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Reconstructing Content with Collaborative Attention for Universal Multimodal Representation Learning

    cs.IR 2026-03 conditional novelty 5.0 of 10

    CoCoA forces an MLLM to reconstruct masked text through a single EOS token, improving multimodal embedding quality on MMEB-V1 and matching MoCa at 3B with far less pretraining data.

Reference graph

Works this paper leans on

30 extracted references · 16 canonical work pages · cited by 1 Pith paper

  1. [10]

    Masatoshi Kudo, Hobyung Chung, and Yukio Osaki. 2003. Prognostic staging system for hepatocellular carcinoma (CLIP score): its value and limitations, and a proposal for a new staging system, the Japan Integrated Staging Score (JIS score). Journal of gastroenterology 38 (2003), 207–215

    work page 2003

  2. [2]

    Ahmed Alateeq, Mark Roantree, and Cathal Gurrin. 2023. Voxento 4.0: A More Flexible Visualisation and Control for Lifelogs. In Proceedings of the 6th Annual ACM Lifelog Search Challenge (Thessaloniki, Greece) (LSC ’23). Association for Computing Machinery, New York, NY, USA, 7–12. https://doi.org/10.1145/ 3592573.3593097

    work page arXiv 2023

  3. [3]

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al

  4. [4]

    Youngmin Baek, Bado Lee, Dongyoon Han, Sangdoo Yun, and Hwalsuk Lee. 2019. Character Region Awareness for Text Detection. CoRR abs/1904.01941 (2019). arXiv:1904.01941 http://arxiv.org/abs/1904.01941

    arXiv 2019

  5. [5]

    John Canny. 1986. A computational approach to edge detection.IEEE Transactions on pattern analysis and machine intelligence 6 (1986), 679–698

    1986

  6. [6]

    Weizhi Chen, Jingbo Chen, Yupeng Deng, Jiansheng Chen, Yuman Feng, Zhi- hao Xi, Diyou Liu, Kai Li, and Yu Meng. 2025. LRSCLIP: A Vision-Language Foundation Model for Aligning Remote Sensing Image with Longer Text. arXiv:2503.19311 [cs.CV] https://arxiv.org/abs/2503.19311

    work page arXiv 2025

  7. [7]

    Cathal Gurrin, Björn Þór Jónsson, Duc Tien Dang Nguyen, Graham Healy, Jakub Lokoc, Liting Zhou, Luca Rossetto, Minh-Triet Tran, Wolfgang Hürst, Werner Bailer, and Klaus Schoeffmann. 2023. Introduction to the Sixth An- nual Lifelog Search Challenge, LSC’23. In Proceedings of the 2023 ACM In- ternational Conference on Multimedia Retrieval (Thessaloniki, Gre...

    work page arXiv 2023

  8. [8]

    Smeaton, and Aiden R

    Cathal Gurrin, Alan F. Smeaton, and Aiden R. Doherty. 2014. LifeLogging: Personal Big Data. Foundations and Trends® in Information Retrieval 8, 1 (2014), 1–125. https://doi.org/10.1561/1500000033

    work page doi:10.1561/1500000033 2014

Show all 30 references
  1. [9]

    Václav Jirkovsk `y, Marek Obitko, and Vladimír Mařík. 2016. Understanding data heterogeneity in the context of cyber-physical systems integration. IEEE Transactions on Industrial Informatics 13, 2 (2016), 660–667

    2016

  2. [11]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning . PMLR, 19730–19742

    2023

  3. [12]

    Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang

  4. [13]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual in- struction tuning. Advances in neural information processing systems 36 (2023), 34892–34916

    2023

  5. [14]

    Tien-Thanh Nguyen-Dang, Xuan-Dang Thai, Gia-Huy Vuong, Van-Son Ho, Minh- Triet Tran, Van-Tu Ninh, Minh-Khoi Pham, Tu-Khiem Le, and Graham Healy

  6. [15]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. arXiv:2103.000...

    2021 arXiv

  7. [16]

    Ricardo Ribeiro, Alina Trifan, and António JR Neves. 2022. Lifelog retrieval from daily digital data: narrative review. JMIR mHealth and uHealth 10, 5 (2022), e30517

    2022

  8. [17]

    Klaus Schoeffmann. 2023. lifeXplore at the Lifelog Search Challenge 2023. In Proceedings of the 6th Annual ACM Lifelog Search Challenge (Thessaloniki, Greece) (LSC ’23). Association for Computing Machinery, New York, NY, USA, 53–58. https://doi.org/10.1145/3592573.3593105

    2023

  9. [18]

    Vishnu Sreekumar, Simon Dennis, Isidoros Doxas, Yuwen Zhuang, and Mikhail Belkin. 2014. The geometry and dynamics of lifelogs: discovering the organiza- tional principles of human experience. PloS one 9, 5 (2014), e97166

    2014

  10. [19]

    Jinyu Tian, Jiantao Zhou, and Jia Duan. 2021. Probabilistic selective encryption of convolutional neural networks for hierarchical services. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 2205–2214

    2021

  11. [20]

    Quang-Linh Tran, Binh Nguyen, Gareth Jones, and Cathal Gurrin. 2024. Memo- riEase at the NTCIR-17 Lifelog-5 Task. (2024)

    2024

  12. [21]

    Chien-Yao Wang, Alexey Bochkovskiy, and Hong-Yuan Mark Liao. 2022. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv:2207.02696 [cs.CV] https://arxiv.org/abs/2207.02696

    2022 arXiv

  13. [22]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. 2024. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191 (2024)

    2024 arXiv

  14. [23]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Cheng- peng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jin...

    2024 arXiv

  15. [24]

    Beichen Zhang, Pan Zhang, Xiaoyi Dong, Yuhang Zang, and Jiaqi Wang. 2024. Long-clip: Unlocking the long-text capability of clip. In European Conference on Computer Vision. Springer, 310–325

    2024

  16. [25]

    Liting Zhou, Cathal Gurrin, Duc-Tien Dang-Nguyen, Graham Healy, Chenyang Lyu, Tianbo Ji, Longyue Wang, Joho Hideo, Ly-Duyen Tran, and Naushad Alam

  17. [26]

    Liting Zhou, Quang-Linh Tran, Hoang Bao Le, Duc-Tien Dang-Nguyen, Graham Healy, ChenYang Lyu, Tianbo Ji, Longyue Wang, Hideo Joho, Ly-Duyen Tran, and Cathal Gurrin. 2025. Overview of the NTCIR-18 Lifelog-6 Task. In Proceedings of the 18th NTCIR Conference on Evaluation of Info...

    2025

  18. [27]

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. Minigpt-4: Enhancing vision-language understanding with advanced large lan- guage models. arXiv preprint arXiv:2304.10592 (2023)

    2023 arXiv

  19. [29]

    In Proceedings of the 17th NTCIR Conference on Evaluation of Information Access Technologies

    Overview of the NTCIR-17 Lifelog-5 Task. In Proceedings of the 17th NTCIR Conference on Evaluation of Information Access Technologies. https://doi. org/10.20736/0002001329

    doi

  20. [2019]

    arXiv preprint arXiv:1908.03557 (2019)

    Visualbert: A simple and performant baseline for vision and language. arXiv preprint arXiv:1908.03557 (2019)

    2019 arXiv

  21. [2022]

    Advances in neural information processing systems 35 (2022), 23716–23736

    Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems 35 (2022), 23716–23736

    2022

  22. [2023]

    In Proceedings of the 6th Annual ACM Lifelog Search Challenge

    LifeInsight: an interactive lifelog retrieval system with comprehensive spatial insights and query assistance. In Proceedings of the 6th Annual ACM Lifelog Search Challenge. 59–64

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.