REVIEW 4 major objections 3 minor 31 references
CountingFruit: Language-Guided 3D Fruit Counting with Semantic Gaussian Splatting
T0 review · 4 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper claims that a prompt-based 3D Gaussian scene can count fruit instances accurately and quickly, without retraining or per-frame segmentation, and reports up to 99.7% recall across nine orchard datasets.
desk verdict Strong applied counting system with a load-bearing gap: the prompt-to-3D feature projection is never specified, so peer review is warranted with major revision. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central machinery is the language-augmented Gaussian. Each Gaussian carries geometry and color plus a three-dimensional semantic code per hierarchical level (subpart, part, whole), trained so that its $\alpha$-blended rendering matches compressed CLIP features of masked image regions. Counting then works by prompt filtering: a Gaussian is kept only if its cosine similarity to the best positive prompt exceeds $\tau_{\mathrm{pos}}$ and its similarity to the best negative prompt stays below $\tau_{\mathrm{neg}}$. The kept Gaussians are sampled into a dense point cloud with per-point density proportional to $\det(\Sigma_i)^{1/2}\alpha_i$, and instances are recovered by DBSCAN clustering with fruit-template splitting. This machinery converts an open-vocabulary language query into a geometric count.
What would settle it
The direct check is to reproduce Equations (10)-(11) on a released model: take a 512-dimensional CLIP text embedding for the prompt, map it into the paper's 3-dimensional code space with the projection the authors supply, and test whether the dual-threshold filter still separates fruit from foliage on a held-out tree. If no text-side projection is implemented, the filtering criterion is not executable as written; a second check is to run the full pipeline on a new tree with human-verified counts and see whether recall stays near the reported 99% or collapses.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that a 3D Gaussian scene can double as a language-addressable count map. Each Gaussian is augmented with feature vectors learned so that rendered semantic features match compressed CLIP features extracted from multi-level masks; at inference, positive and negative text prompts are compared to these codes under a dual-threshold cosine rule. The retained Gaussians are sampled into colored point clouds with density proportional to their covariance and opacity, then split into fruit instances by DBSCAN followed by template-guided splitting. The paper reports that this language-guided filtering, rather than the geometric reconstruction alone, is what lifts recall: in the ablation, removing the semantic embedding drops recall by 15 to 28 percentage points depending on the dataset.
Load-bearing premise
The load-bearing premise is that a text prompt's CLIP embedding can be compared by cosine similarity to the three-dimensional semantic code stored in each Gaussian, even though the paper compresses only image-side features and does not describe how the 512-dimensional text vector is projected into that same three-dimensional space.
Editorial extensions
If this is right
- Fruit counting becomes a prompt: after one reconstruction, the same scene can answer different queries ('apple', 'green apple', 'foliage') by changing text prompts, with no retraining.
- The pipeline removes the multi-view 2D segmentation fusion step, so counting errors from inconsistent per-frame masks are replaced by 3D semantic filtering and geometric clustering.
- Rendering at hundreds to over 1,000 FPS means the scene representation is fast enough for on-robot perception while retaining dense 3D structure.
- The same language-augmented Gaussians support open-vocabulary retrieval of non-fruit structures, so the framework extends beyond counting to general prompt-driven 3D querying.
Reading between the lines
- Inference: the text-side projection is the piece most likely to be supplied by implementation details; making it explicit and testing robustness to prompt rewording would settle whether the reported recalls come from the language channel or from geometric clustering.
- Inference: the dual thresholds are manually chosen, so a natural extension is to calibrate them per orchard from a handful of labelled trees and report sensitivity curves, which the paper does not provide.
- Inference: because the semantic codes are prompt-agnostic, the same reconstructed scene could be reused for other commodity-like objects (flowers, nuts, twigs), a hypothesis the paper states as motivation but does not test.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents FruitLangGS, a 3D Gaussian Splatting pipeline for prompt-driven fruit counting in orchard scenes. It reconstructs a scene with adaptive-density Gaussians, attaches to each Gaussian a three-dimensional semantic code learned by compressing masked CLIP features through a scene-specific autoencoder at three segmentation levels, and at inference filters Gaussians by dual-threshold cosine similarity to positive and negative text prompts. Surviving Gaussians are sampled into point clouds and clustered with DBSCAN plus template-based splitting to obtain instance counts. Experiments on nine orchard datasets compare rendering quality and speed against NeRF, FruitNeRF, Taming3DGS, and 3DGS; counting recall against NeRF+Ours, 3DGS+Ours, and FruitNeRF; and ablations remove the semantic embedding or replace it with FruitNeRF's Fruit Field. The paper claims consistent recall superiority, up to 99.7% recall, and the highest rendering speeds.
Significance. If the results hold, FruitLangGS would be a useful demonstration that a single prompt-conditioned 3D Gaussian scene can support open-vocabulary fruit counting without per-frame masks or retraining, with rendering-speed advantages over NeRF-based fruit counters. The paper's strengths include evaluation on nine real-world datasets, comparison against several backbones, and ablations that isolate the semantic embedding. However, the central mechanism is not fully specified and the reported metric set is incomplete: no precision or F1, no error bars or repeated runs, and no end-to-end timing for the counting pipeline. The significance of the claimed recall gains therefore depends on clarifications that cannot be verified from the manuscript as written.
major comments (4)
- [Section 3.3, Eq. (10)] The prompt filtering mechanism is undefined as written. Eq. (10) states that unit-normalized text features t_j lie in R^d, and Sec. 3.2 sets d=3, but the CLIP-ViT-B/16 text encoder used in Table 1 outputs 512-dimensional vectors and no text compression or projection is described anywhere in the paper. The autoencoder Psi composed with E in Eq. (6) is trained to reconstruct masked CLIP image features, not text features, so passing text through it is not justified and is never stated. Unless the authors specify how prompt embeddings are mapped to R^3 and validate that this mapping preserves text-image alignment in the three-dimensional space, the cosine similarities in Eqs. (10)-(11) compare vectors in different spaces and the language-guided filtering central to the recall claims cannot be executed.
- [Section 4.3, Table 3] The central claim that FruitLangGS 'consistently outperforms existing pipelines in instance counting recall' is not supported by Table 3. FruitNeRF (U-Net-B) achieves higher recall on Tree02 (99.1% versus 97.3%) and on Fuji-SfM (100.3% versus 99.2%), and ties on Tree01 (96.6% versus 96.6%). The 100.3% recall on Fuji-SfM corresponds to 1459 predictions against 1455 ground-truth instances, which indicates oversegmentation rather than a clean recall advantage; reporting only recall, without precision or F1, cannot distinguish high recall from overcounting. The paper should report precision and F1, error bars or repeated-seed statistics, and a matching procedure that prevents a predicted count above ground truth from being scored as greater than 100% recall.
- [Section 3.3, Eq. (11) and Table 4] The dual-threshold filter and clustering are controlled by manually selected parameters (tau_pos, tau_neg, DBSCAN eps and min_samples, template-splitting thresholds), and the paper gives no selection procedure, search range, or sensitivity analysis. Without a stated protocol for choosing these values, the reported recall could reflect fitting to these nine datasets. In addition, the first ablation row in Table 4 sets f_i=0 during filtering and clustering; under Eq. (11) with positive tau_pos, all cosine similarities to positive prompts are zero, so the filtered set is empty unless thresholds are changed or filtering is bypassed, neither of which is described. The ablation should specify exactly how the geometric-only variant operates.
- [Section 4.2, Table 2] The FPS numbers measure Gaussian rasterization speed only, not the end-to-end counting latency. The abstract and introduction call FruitLangGS a real-time fruit counting system, but the semantic filtering, point-cloud sampling, DBSCAN, and template-guided splitting in Section 3.3 are not timed anywhere. Without end-to-end latency measurements (prompt to count), the 'real-time' claim and the comparison to 'slow inference' of prior pipelines are unsubstantiated; FPS of rendering alone does not establish that counting operates in real time.
minor comments (3)
- [Eq. (9)] Eq. (9) says L_sem is 'the semantic embedding loss from Eq. (6)', but Eq. (6) is the autoencoder loss L_AE; the semantic rendering loss is defined in Eq. (8). Please correct the cross-reference and the notation.
- [Section 3.3, Eq. (12)] Eq. (12) samples points proportional to det(Sigma_i)^{1/2} times alpha_i, but the actual number of sampled points per Gaussian is not specified; this density directly affects DBSCAN and therefore the final count. Please state the normalization or constant used.
- [Section 3.2] The choice d=3 is asserted to preserve discriminative capacity, but no ablation or supporting experiment for this dimensionality is provided; an ablation over d would strengthen the claim.
Circularity Check
No significant circularity; the main gap is an underspecified text projection in Eq. 10, which is a correctness issue, not a circular derivation.
full rationale
Walking the derivation chain, I find no circular reduction. The counting pipeline is evaluated against external instance-level ground truth (FruitNeRF benchmark and PFuji-Size datasets); GT counts do not appear anywhere in the objective functions (Eqs. 2, 6, 8, 9) or in the filtering criterion (Eq. 11), and the final count (Eq. 13) is a clustering output, not the GT count itself. Semantic features are trained by reconstructing CLIP image features extracted from segmentation masks (Eqs. 5-8), so the language-conditioned embeddings are not defined in terms of the reported recall. The only flagged issue is an omitted projection: Eq. 10 writes CLIP text features as t_j in R^d with d=3 after Sec. 3.2 sets d=3, but the paper never specifies how the 512-dimensional CLIP text embedding is reduced to R^3; this makes the filtering specification non-executable as written. That is a correctness/executability gap, not a circularity, because no equation or fitted parameter forces the predicted count to equal its input. Manual thresholds and clustering hyperparameters are tuning risks, not definitional circularity, and no load-bearing self-citations appear.
Assumptions & free parameters
free parameters (5)
- tau_pos (positive similarity threshold) =
not reported
- tau_neg (negative similarity threshold) =
not reported
- DBSCAN eps and min_samples =
not reported
- Fruit template and splitting thresholds =
not reported
- Latent dimension d=3 and loss weights =
d=3; lambda_sem and lambda_reg not specified
assumptions (4)
- domain assumption CLIP image and text embeddings remain comparable after compressing image features to 3 dimensions and leaving text features in CLIP's native space, or after an unstated mapping.
- domain assumption The pretrained segmentation model provides accurate subpart, part, and whole masks so masked CLIP features are trustworthy semantic targets.
- domain assumption Gaussians with high semantic score and high opacity correspond to fruit surface points, and clustering these points separates individual fruits.
- domain assumption Photometric multi-view reconstruction with 3DGS produces geometry accurate enough for counting.
Cite this review
Pith. "Pith review of CountingFruit: Language-Guided 3D Fruit Counting with Semantic Gaussian Splatting." pith.science (2026). https://pith.science/paper/7YANCQEX
@misc{pith2026250601109,
author = {Pith},
title = {Pith review of: CountingFruit: Language-Guided 3D Fruit Counting with Semantic Gaussian Splatting},
year = {2026},
howpublished = {\url{https://pith.science/paper/7YANCQEX}},
note = {Machine review of arXiv:2506.01109}
}
read the original abstract
Accurate 3D fruit counting in orchards is challenging due to heavy occlusion, semantic ambiguity between fruits and surrounding structures, and the high computational cost of volumetric reconstruction. Existing pipelines often rely on multi-view 2D segmentation and dense volumetric sampling, which lead to accumulated fusion errors and slow inference. We introduce FruitLangGS, a language-guided 3D fruit counting framework that reconstructs orchard-scale scenes using an adaptive-density Gaussian Splatting pipeline with radius-aware pruning and tile-based rasterization, enabling scalable 3D representation. During inference, compressed CLIP-aligned semantic vectors embedded in each Gaussian are filtered via a dual-threshold cosine similarity mechanism, retrieving Gaussians relevant to target prompts while suppressing common distractors (e.g., foliage), without requiring retraining or image-space masks. The selected Gaussians are then sampled into dense point clouds and clustered geometrically to estimate fruit instances, remaining robust under severe occlusion and viewpoint variation. Experiments on nine different orchard-scale datasets demonstrate that FruitLangGS consistently outperforms existing pipelines in instance counting recall, avoiding multi-view segmentation fusion errors and achieving up to 99.7% recall on Pfuji-Size_Orch2018 orchard dataset. Ablation studies further confirm that language-conditioned semantic embedding and dual-threshold prompt filtering are essential for suppressing distractors and improving counting accuracy under heavy occlusion. Beyond fruit counting, the same framework enables prompt-driven 3D semantic retrieval without retraining, highlighting the potential of language-guided 3D perception for scalable agricultural scene understanding.
Reference graph
Works this paper leans on
-
[1]
INTRODUCTION The global transition toward smart agriculture has led to in- creasing demand for scalable, autonomous systems that can sense, reason, and act in dynamic outdoor environments [1]. In this context, fruit counting plays a crucial role in yield prediction, robotic harvesting, and orchard monitoring, sup- porting critical decision-making in preci...
work page Pith review arXiv 2025
-
[2]
RELATED WORK 2.1. Fruit Counting in Precision Agriculture Fruit counting is essential in precision agriculture, support- ing yield prediction, harvest planning, and crop-load man- agement. Early methods relied on 2D color features [5, 6], assuming strong contrast between fruits and foliage. How- ever, natural orchard environments often suffer from unstabl...
-
[3]
The over- all pipeline is shown in Fig
METHOD FruitLangGS is proposed as a framework for real-time, open- vocabulary fruit counting in orchard environments. The over- all pipeline is shown in Fig. 2. We detail each module in the following sections. 3.1. Scene Reconstruction with Adaptive Splatting In orchard-scale fruit counting, a suitable scene represen- tation must reconcile two contrasting...
-
[4]
EXPERIMENTS AND RESULTS We conduct comprehensive experiments to evaluate the effec- tiveness, efficiency, and modularity of FruitLangGS across diverse orchard scenarios. Our objectives are: (1) to assess the reconstruction quality and rendering speed of our adap- tive Gaussian Splatting pipeline against neural and explicit baselines; (2) to evaluate count...
-
[5]
w/o Language-Conditioned Embedding
Progressive densification, opacity pruning, and geometric regularization are applied. Semantic filtering, sampling, and clustering hyperparameters are listed in Table 1. 4.2. Reconstruction Quality and Efficiency We first evaluate the 3D reconstruction performance of FruitLangGS in terms of both visual fidelity and render- ing efficiency. As summarized in...
-
[6]
CONCLUSION AND FUTURE WORK FruitLangGS introduces a real-time framework for open- vocabulary 3D fruit counting by coupling adaptive Gaussian rendering with language-guided semantic filtering. This de- sign eliminates the need for category-specific training or per-frame annotations, enabling prompt-driven counting di- rectly on reconstructed scenes. Experi...
-
[7]
Precision farming for sustainabil- ity: An agricultural intelligence model,
Vinod Chandra SS, Anand Hareendran, and Ghas- san Faisal Albaaji, “Precision farming for sustainabil- ity: An agricultural intelligence model,”Computers and Electronics in Agriculture, vol. 226, pp. 109386, 2024
work page 2024
-
[8]
A review on precision agriculture nav- igating the future of farming with ai and iot,
ME Krishnababu, B Rama Devi, Amitabh Soni, Chan- dan Kumar Panigrahi, B Sudeepthi, Abhinav Rathi, and Anoop Shukla, “A review on precision agriculture nav- igating the future of farming with ai and iot,”Asian Journal of Soil Science and Plant Nutrition, vol. 10, no. 2, pp. 336–349, 2024
work page 2024
Show all 31 references
-
[9]
Iot and ai for next-generation farming: Opportunities, chal- lenges, and outlook,
Nadir Ali Bhangar and Abul Kashem Shahriyar, “Iot and ai for next-generation farming: Opportunities, chal- lenges, and outlook,”International Journal of Sustain- able Infrastructure for Cities and Societies, vol. 8, no. 2, pp. 14–26, 2023
2023
-
[10]
Developing machine vision in tree-fruit applications—fruit count, fruit size and branch avoidance in automated harvesting,
Chiranjivi Neupane, Kerry B Walsh, Rafael Goulart, and Anand Koirala, “Developing machine vision in tree-fruit applications—fruit count, fruit size and branch avoidance in automated harvesting,”Sensors, vol. 24, no. 17, pp. 5593, 2024
2024
-
[11]
Automatic fruit recognition and counting from multiple images,
Y Song, CA Glasbey, GW Horgan, Gerrit Polder, JA Dieleman, and GW AM Van der Heijden, “Automatic fruit recognition and counting from multiple images,” Biosystems Engineering, vol. 118, pp. 203–215, 2014
2014
-
[12]
Deep count: fruit counting based on deep simulated learning,
Maryam Rahnemoonfar and Clay Sheppard, “Deep count: fruit counting based on deep simulated learning,” Sensors, vol. 17, no. 4, pp. 905, 2017
2017
-
[13]
Agrocounters—a reposi- tory for counting objects in images in the agricultural domain by using deep-learning algorithms: Framework and evaluation,
Guy Farjon and Yael Edan, “Agrocounters—a reposi- tory for counting objects in images in the agricultural domain by using deep-learning algorithms: Framework and evaluation,”Computers and Electronics in Agricul- ture, vol. 222, pp. 108988, 2024
2024
-
[14]
Robust fruit counting: Combining deep learning, tracking, and struc- ture from motion,
Xu Liu, Steven W Chen, Shreyas Aditya, Nivedha Sivakumar, Sandeep Dcunha, Chao Qu, Camillo J Tay- lor, Jnaneshwar Das, and Vijay Kumar, “Robust fruit counting: Combining deep learning, tracking, and struc- ture from motion,” in2018 IEEE/RSJ international Conference on intellig...
2018
-
[15]
Fruitnerf: A unified neural radiance field based fruit counting framework,
Lukas Meyer, Andreas Gilson, Ute Schmid, and Marc Stamminger, “Fruitnerf: A unified neural radiance field based fruit counting framework,” in2024 IEEE/RSJ In- ternational Conference on Intelligent Robots and Sys- tems (IROS). IEEE, 2024, pp. 1–8
2024
-
[16]
Fruitnerf++: A general- ized multi-fruit counting method utilizing contrastive learning and neural radiance fields,
Lukas Meyer, Andrei-Timotei Ardelean, Tim Weyrich, and Marc Stamminger, “Fruitnerf++: A general- ized multi-fruit counting method utilizing contrastive learning and neural radiance fields,”arXiv preprint arXiv:2505.19863, 2025
2025 arXiv
-
[17]
A weakly- supervised approach for flower/fruit counting in apple orchards,
Uddhav Bhattarai and Manoj Karkee, “A weakly- supervised approach for flower/fruit counting in apple orchards,”Computers in Industry, vol. 138, pp. 103635, 2022
2022
-
[18]
Tracking and counting of tomato at different growth period using an improving yolo-deepsort network for inspection robot,
Yuhao Ge, Sen Lin, Yunhe Zhang, Zuolin Li, Hong- tai Cheng, Jing Dong, Shanshan Shao, Jin Zhang, Xi- angyu Qi, and Zedong Wu, “Tracking and counting of tomato at different growth period using an improving yolo-deepsort network for inspection robot,”Machines, vol. 10, no. 6, pp...
2022
-
[19]
You only look once: Unified, real-time object detection,
Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi, “You only look once: Unified, real-time object detection,” inProceedings of the IEEE confer- ence on computer vision and pattern recognition, 2016, pp. 779–788
2016
-
[20]
Colour-agnostic shape-based 3d fruit detection for crop harvesting robots,
Ehud Barnea, Rotem Mairon, and Ohad Ben-Shahar, “Colour-agnostic shape-based 3d fruit detection for crop harvesting robots,”Biosystems Engineering, vol. 146, pp. 57–70, 2016
2016
-
[21]
High-precision fruit localization using active laser-camera scanning: Robust laser line extraction for 2d-3d transformation,
Pengyu Chu, Zhaojian Li, Kaixiang Zhang, Kyle Lam- mers, and Renfu Lu, “High-precision fruit localization using active laser-camera scanning: Robust laser line extraction for 2d-3d transformation,”Smart Agricul- tural Technology, vol. 7, pp. 100391, 2024
2024
-
[22]
Nerf: Representing scenes as neural radiance fields for view synthesis,
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,”Communications of the ACM, vol. 65, no. 1, pp. 99–106, 2021
2021
-
[23]
3d gaussian splatting for real-time radiance field rendering,
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkuehler, and George Drettakis, “3d gaussian splatting for real-time radiance field rendering,”ACM Trans. Graph., vol. 42, no. 4, July 2023
2023
-
[24]
Adr- gaussian: Accelerating gaussian splatting with adaptive radius,
Xinzhe Wang, Ran Yi, and Lizhuang Ma, “Adr- gaussian: Accelerating gaussian splatting with adaptive radius,” inSIGGRAPH Asia 2024 Conference Papers, 2024, pp. 1–10
2024
-
[25]
Taming 3dgs: High-quality radiance fields with limited resources,
Saswat Subhajyoti Mallick, Rahul Goel, Bernhard Kerbl, Markus Steinberger, Francisco Vicente Carrasco, and Fernando De La Torre, “Taming 3dgs: High-quality radiance fields with limited resources,” inSIGGRAPH Asia 2024 Conference Papers, 2024, pp. 1–11
2024
-
[26]
Learning transferable visual models from natural lan- guage supervision,
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- try, Amanda Askell, Pamela Mishkin, Jack Clark, et al., “Learning transferable visual models from natural lan- guage supervision,” inInternational conference on ma- chine lear...
2021
-
[27]
Langsplat: 3d language gaussian splatting,
Minghan Qin, Wanhua Li, Jiawei Zhou, Haoqian Wang, and Hanspeter Pfister, “Langsplat: 3d language gaussian splatting,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 20051–20060
2024
-
[28]
Segment anything,
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al., “Segment anything,” inProceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 4015–4026
2023
-
[29]
A density-based algorithm for discovering clusters in large spatial databases with noise,
Martin Ester, Hans-Peter Kriegel, J ¨org Sander, Xiaowei Xu, et al., “A density-based algorithm for discovering clusters in large spatial databases with noise,” inkdd, 1996, vol. 96, pp. 226–231
1996
-
[30]
Jordi Gen ´e-Mola, Ricardo Sanz-Cortiella, Joan R Rosell-Polo, Alexandre Escol `a, and Eduard Grego- rio, “Pfuji-size dataset: A collection of images and photogrammetry-derived 3d point clouds with ground truth annotations for fuji apple detection and size esti- mation in fiel...
2021
-
[31]
Track- ing and counting apples in orchards under intermittent occlusions and low frame rates,
Gonc ¸alo P. Matos, Carlos Santiago, Jo˜ao P. Costeira, Ri- cardo L. Saldanha, and Ernesto M. Morgado, “Track- ing and counting apples in orchards under intermittent occlusions and low frame rates,” in2024 IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion Worksh...
2024
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.