{"id":"f87d3662-fd64-4e24-8f35-9ae3a7ae8d80","arxiv_id":"2501.09456","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Aperture shape and f-number from 1.8 to 3.4 show no statistically significant effect on YOLOv8 detection precision in simulated automotive images, with degradation only at 48 dB gain.","lead":"This paper found that in synthetic automotive scenes, aperture shape (circular, plus-shaped, vertical or horizontal slit) made no statistically significant difference to YOLOv8 detection precision after gain compensation, and precision only degraded at the highest simulated gain. The result, produced with a depth-aware PSF simulation pipeline, suggests smaller and cheaper camera apertures could be viable for automotive detection tasks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The null aperture-effect conclusion rests on failure to reject in underpowered Welch t-tests on five non-independent folds; without an equivalence test, 'no significant difference' is not supported as evidence of absence.","rationale":"The reader's verdict correctly flags the absence of an equivalence test and the simulation-fidelity assumption. My specific concern is internal to the statistical analysis: the Welch t-test on five folds treats non-independent fold measurements as independent samples, and non-rejection is used to support a positive claim of indifference. This is directly relevant to the strongest claim, which is a null result. The simulation-fidelity issue is external validity; if the statistical test is invalid, the central claim is unsupported even within the simulation. The paper has clear strengths: a large synthetic dataset, per-pixel depth-based PSF filtering, and a measured camera noise curve. The K-fold procedure and the reporting of t-statistics and p-values are transparent. However, the inference from 'p > 0.05' to 'no significant difference in performance' is a classic logical error that a concrete equivalence test would settle. I agree with the reader's CONDITIONAL verdict: the paper should be accepted only with a corrected statistical analysis and more careful wording, or the practical conclusions should be qualified accordingly. Since the reader already set CONDITIONAL, my concern does not change the verdict; it sharpens the conditions for acceptance.","tokens_in":15441,"tokens_out":6499,"duration_ms":69666,"concrete_test":"Re-analyze the per-fold mAP values (K=5) for each aperture pair at each gain and bbox size using paired t-tests on the within-fold differences, and run a two-one-sided t-test (TOST) equivalence procedure with a pre-specified equivalence bound, e.g., Δ = 0.01 mAP for tiny bboxes and Δ = 0.005 for medium/large bboxes. If the 90% confidence interval for the mean difference lies entirely within ±Δ in all comparisons, the 'no significant difference' conclusion is upgraded to 'statistically equivalent'; if any interval exceeds the bound, the conclusion must be weakened to 'no statistically significant difference detected,' and the practical claim about f/1.8-f/3.4 should be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section III-A (Eqs. 8-15) and Tables III-VII, the authors run Welch's two-sample t-test on K=5 fold mAP values for each aperture pair. This test is used to support the central 'no statistically significant difference' claim. Two problems make this inference load-bearing. First, the five folds are not independent: each fold's training set shares 80% of the same source images, and the mAP values for different apertures at the same gain are computed on the same test images. Treating them as independent samples in an unpaired t-test inflates the effective degrees of freedom and can distort p-values; a paired analysis across folds is the appropriate design for comparing two conditions measured on the same folds. Second, a non-significant t-test does not establish equivalence: it only says the data are consistent with a null effect, not that the effect is absent. With n=5 and the large fold-to-fold variance visible for tiny and small bboxes in Figs. 15-16, the tests are likely underpowered for any practically meaningful mAP difference. Section IV then overreaches by concluding 'the results indicate no significant difference in detection performance across f-numbers ranging from f/1.8 to f/3.4' and suggesting cost savings in lens specifications. This is a load-bearing flaw because the paper's central claim is precisely a null effect; if the tests are too weak, the claimed indifference could be a false negative. The reader's simulation-fidelity concern is also real, but the statistical issue undermines the claim even on the paper's own terms.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies how aperture size and shape of an automotive camera affect deep-learning object detection (traffic signs, speed signs, traffic lights). Synthetic 'pinhole' images are convolved with Zemax ray-traced PSF kernels at 19 depth ranges, gain-dependent AWGN is added using a measured camera noise curve, and YOLOv8n detectors are trained and evaluated per aperture and gain level. Four apertures are compared (circular f/1.8, plus f/2.4, horizontal and vertical slit f/3.4) at four gain levels (0, 30, 40, 48 dB) with 5-fold cross-validation. The authors conclude that there is no statistically significant difference in detection precision among aperture shapes and across f/1.8 to f/3.4, and they suggest a possible cost saving from smaller diameter optics.","tokens_in":15781,"tokens_out":4791,"duration_ms":51042,"significance":"The paper addresses a question of practical relevance to automotive camera design: whether moderate increases in f-number or non-circular aperture shapes degrade downstream object detection. The pipeline is a methodological contribution: it converts abstract optical simulations into a large, annotated, depth-aware synthetic dataset with realistic PSF effects, and the comparative setup across 80 trained networks is a substantial computational effort. The main claim, if rigorously supported, could inform cost-driven lens specifications. However, the statistical evidence for the null result is currently weak, and the simulation lacks real-image validation, so the strength of the claim exceeds what the presented analysis can support.","major_comments":[{"comment":"The alternative hypothesis in Eq. (9), H_a: \\bar\\mu_1,i \\neq \\bar\\mu_2,i \\neq \\bar\\mu_3,i \\neq \\bar\\mu_4,i, is not the logical negation of the null hypothesis in Eq. (8), which states all four means are equal. The correct alternative is 'at least one pair of means differs.' Moreover, the pairwise Welch's t-test in Eq. (12) tests only one pair of apertures, so the hypotheses as written do not correspond to the tests actually performed. The same issue applies to the gain hypotheses in Eqs. (10)-(11). This is a formal error that should be corrected because the entire statistical analysis is founded on the statement of these hypotheses.","section":"Section III-A, Eq. (9) and Eq. (11)"},{"comment":"The inference 'no statistically significant difference' is based on failure to reject in Welch's t-tests with n=5 fold means. These five means are not independent samples: each fold's training set shares 80% of its images with every other fold, and all folds use images from the same synthetic source. Treating them as independent samples in an unpaired t-test inflates the effective degrees of freedom and can distort p-values. A paired analysis across folds, or a mixed-effects model with fold as a random effect, is more appropriate. More critically, a non-significant p-value does not establish equivalence; with only five folds and the large fold-to-fold variance visible for tiny and small bounding boxes in Figures 15-16, the tests are underpowered for the small mAP differences that matter. The authors should report effect sizes and confidence intervals, and should consider a formal equivalence test (e.g., two one-sided tests, TOST) with a pre-specified margin. As written, the central conclusion in Section IV that 'no significant difference' exists is not supported by the statistics.","section":"Section III-A, Tables III-VII"},{"comment":"The load-bearing premise of the paper is that the simulated PSF kernels and the AWGN model faithfully represent real camera behavior. The PSF kernels come from Zemax for a specific 16mm lens and a specific windshield model, and the noise is fit to dark-frame measurements of one camera (DFK37BUX252). No comparison is made to real images captured with these apertures, nor to images from another camera model. The noise model is also additive white Gaussian noise only, ignoring shot noise and spatial non-uniformities that are present in real automotive cameras. Therefore, the null result could be a property of the simulator rather than a property of physical imaging systems. I recommend either adding a real-image validation experiment (even a limited one, e.g., with one aperture pair and a static scene) or substantially tempering the conclusions to explicitly state that the findings hold only for the simulated model and list the conditions under which they might transfer.","section":"Section II-B and II-C, Simulation Validation"},{"comment":"The metric mAP is defined in Eq. (5) as the mean over classes of |TP_c| / (|TP_c| + |FP_c|), which is average precision at a single confidence score threshold, not mean average precision (the area under the precision-recall curve). This is inconsistent with the statement that cocoeval.py was used to compute the reported values, as cocoeval computes AP as an integral over recall levels. If the paper reports COCO mAP, Eq. (5) should be replaced by the standard definition; if Eq. (5) is what was actually computed, then the results are not comparable to COCO mAP and the authors should rename the metric. The metric definition is central because every result in the paper is expressed in these units.","section":"Section III, Eq. (5)"},{"comment":"The paper states that 384 pairwise tests are required for the aperture comparisons and 'a similar amount' for the gain comparisons, but then reports only selected examples (Tables III-VII) and draws the general conclusion that 'the null hypothesis cannot be rejected for any of the cases.' This makes the global claim unverifiable from the text. Additionally, no correction for multiple comparisons is applied despite the large number of tests. I recommend reporting the full set of test results (e.g., in a supplementary table) and applying or justifying a multiple-comparison correction, or limiting the conclusions to the reported bbox classes and gain levels.","section":"Section III-A, Hypothesis Testing Procedure"}],"minor_comments":[{"comment":"The captions of Tables VI and VII say 'APERTURE SIGNIFICANCE' but the columns compare gain levels for a fixed aperture; they should read 'GAIN SIGNIFICANCE'.","section":"Tables VI and VII"},{"comment":"The phrase 'For brevity, we will only present selected test examples' is acceptable, but the paper should indicate which of the 384 tests are shown and whether the omitted tests are consistent with the reported ones.","section":"Section III-A, 'selected test examples'"},{"comment":"The notation |classes| is used but classes are not defined in a set; consider using |C| with C as the set of classes.","section":"Eq. (5) and text"},{"comment":"The gain factor in dB is computed for an ideal aperture-area ratio, but the authors do not discuss that real lens transmittance and vignetting may make the effective f-number different; this could be a caveat in interpreting the aperture comparison.","section":"Section II-C, Table I"},{"comment":"The class distribution histogram would benefit from labels on the axes and a legend describing the class groups, as the current text references ranges of class indices that are hard to read from the figure.","section":"Section II-E, Fig. 14"}],"recommendation":"major_revision","confidential_remarks":"The paper's core finding—that aperture shape and moderate f-number increases do not degrade YOLOv8 detection—is interesting and would fit the journal's scope if properly supported. The main risks are statistical and methodological: the equivalence claim rests on underpowered non-independent t-tests, and the simulation fidelity is unvalidated against real images. These are fixable with additional analysis and a more cautious framing, so I recommend major revision rather than rejection. I would encourage the editor to ask for a more complete statistical treatment and, if feasible, a modest real-camera validation experiment."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Best quick take: this is a real piece of engineering work. They built a plausible PSF-emulation pipeline (Zemax ray tracing, per-color, per-depth PSF kernels, measured camera noise), applied it to 41k synthetic automotive images, trained 80 YOLOv8 models, and asked a sensible question: does aperture shape or f-number up to f/3.4 hurt detection? The answer they get is no, not in these simulations, and the null is consistent across four shapes and four gain levels. The MTF analysis showing the plus aperture is sharper but does not translate to better detection is a nice, honest point.\n\nThe soft spots matter. The central claim is framed as 'no statistically significant difference,' but that is a failure to reject with n=5 fold means. The folds are likely correlated (the same test images are reused across apertures, and training sets overlap), and an unpaired Welch test treats them as independent; a paired test would be more appropriate and would have even less power. More importantly, a null result needs an equivalence test or confidence intervals to support 'no meaningful difference.' The paper doesn't provide that. Also, the alternative hypothesis in Eq. (9) is misstated. For the gain comparisons, the 48 dB degradation is well supported; the aperture comparisons are indeed accompanied by overlapping error bars across many conditions, so the finding is plausible, but it is not proven as strongly as the conclusion states.\n\nSecond, there is no real-camera validation. The PSF kernels come from Zemax for a specific reference lens and windshield, and the noise model is AWGN fitted from dark frames. That is reasonable for a comparative study, but it means the null could be a property of the simulation. The authors acknowledge this indirectly by calling it a numerical analysis, but the conclusion about cost-saving in lens specifications is a forward-looking inference that would need at least one real-aperture check.\n\nWho is this for? Engineers choosing camera modules for perception systems, and researchers who want a reproducible baseline for how much optical blur synthetic-to-real pipelines should bother modeling. It deserves a serious referee: the contribution is useful, the pipeline is described in enough detail to rebuild, and the flaws are addressable (add equivalence testing, report effect sizes, consider paired tests, and add a real-camera sanity check). I would accept it for review with a request for major revisions on the statistics.","headline":"A solid, rebuildable simulation study with a practical null result, but the 'no difference' claim outruns the statistics and the lack of real-camera validation.","tokens_in":16304,"tokens_out":3397,"would_cite":true,"duration_ms":36318,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that aperture shape and f-number from f/1.8 to f/3.4 make no statistically significant difference to YOLOv8 detection precision on simulated automotive signs and lights.","keywords":["point spread function","automotive object detection","aperture shape","f-number","synthetic images","YOLOv8","mean average precision","camera noise"],"falsifier":"Capture the same scene with four cameras identical except for circular, plus, vertical-slit, and horizontal-slit apertures at the matching f-numbers and gains, then run the same YOLOv8 evaluation; if real-image mAP differences exceed the paper's 5-fold error bars, the simulated null result would be a simulation artifact rather than a physical property.","tokens_in":15259,"feed_emoji":"🚗","tokens_out":11378,"duration_ms":103496,"temperature":0.7,"pith_summary":"Working with synthetic automotive scenes, the paper asks whether the shape or size of a camera aperture changes how well a deep detector recognizes speed signs, traffic signs, and traffic lights. It answers no for the configurations tested: across circular, plus-shaped, vertical-slit, and horizontal-slit apertures spanning f/1.8 to f/3.4, detection precision differences are statistically indistinguishable. The proposed method turns 'pinhole' synthetic images into realistic camera images by convolving depth-dependent point-spread-function (PSF) kernels from ray tracing and adding aperture-dependent noise from a measured camera gain curve. If the simulation faithfully represents real optics, the result matters because it suggests lens designers can use smaller, cheaper optics without sacrificing detector accuracy in this operating range.","feed_headline":"Aperture shape barely moves car-vision detection accuracy","feed_subtitle":"Simulated tests across four aperture shapes and f/1.8–f/3.4 show detection precision stays flat; only 48 dB gain hurts.","key_machinery":"The load-bearing object is the PSF-emulation pipeline. For each color channel and each of 19 distance bins from 10 to 100 meters, ray tracing produces a spatially varying PSF kernel over a 51x51 block grid; the pipeline masks a synthetic pinhole image by its per-pixel depth map, convolves each depth slice with the corresponding kernel, reconstructs the full image, then adds zero-mean white Gaussian noise whose standard deviation is interpolated from a measured camera-noise-versus-gain curve. This machinery converts 41,000 synthetic images into 16 replicas—four apertures times four gain levels—isolating aperture effects from noise effects.","core_discovery":"On its own terms, the paper establishes that for YOLOv8-based detection of European-format traffic signs, speed signs, and traffic lights, neither aperture shape nor moderate aperture size changes mean average precision (mAP). Two-sample t-tests for unequal variances on 5-fold cross-validation show no aperture pair reaching $p<0.05$ for tiny and small bounding boxes, and the paper's stated conclusion extends the null result across f/1.8 to f/3.4. A plus-shaped aperture with higher modulation transfer function (MTF) than the circular reference does not translate into higher mAP, so optical sharpness and learned detection performance decouple in this regime. Added noise from gain compensation is tolerated up to 40 dB; at 48 dB every aperture loses precision significantly.","pith_inferences":["Beyond the paper's claims, if the simulation-to-real gap is small, the cost argument is real: higher-f-number lenses with smaller diameters could cut optics cost, but the paper's own spot-size scaling warns that beyond f/3.4 smaller pixels become diffraction-limited, so the savings may not extend to high-resolution sensors.","Beyond the paper's claims, the null result is conditional on the 10-to-100-meter range and these three object classes; far-field tiny objects, low-contrast scenes, or detectors with different inductive biases could expose aperture effects the current test cannot resolve.","A direct real-image validation—mounting cameras with each aperture behind a windshield on the same routes and comparing real mAP against the simulated replicas—would settle whether the null effect is real or an artifact of the PSF and noise model.","A natural next experiment is to vary f-number continuously beyond f/3.4 and to include defocus from windshield curvature, since the simulation includes lateral color and windshield refraction but restricts depth to discrete bins."],"forward_implications":["A camera designer can move from f/1.8 to f/3.4 circular optics, or to plus or slit shapes, without expecting a statistically meaningful change in detection mAP for these object classes.","The superior MTF of a non-circular aperture is not itself a reason to adopt it; detector precision does not follow optical sharpness in the tested range.","Detection precision is effectively flat from 0 to 40 dB of camera gain, and drops only when gain reaches 48 dB.","The depth-aware PSF filtering pipeline can make synthetic training images optically realistic enough to run comparative aperture studies without building multiple physical camera rigs."],"supporting_citations":[{"why":"Motivates the aperture-shape question by showing animal pupil shapes are adapted to different visual tasks.","marker":"[3]"},{"why":"Defines the YOLOv8n network and training pipeline used for all 100 detection classes.","marker":"[5]"},{"why":"Supplies the synthetic traffic-sign and speed-sign images that form the pinhole dataset.","marker":"[8]"},{"why":"Supplies the synthetic traffic-light images used alongside the sign images.","marker":"[9]"},{"why":"Defines the bounding-box size thresholds used to stratify mAP results.","marker":"[10]"},{"why":"Provides the reference camera's sensor parameters and the measured noise-versus-gain curve used to fit additive noise.","marker":"[11]"},{"why":"Supplies the K-fold cross-validation methodology that generates the five mAP values per condition.","marker":"[19]"},{"why":"Provides the diffraction-limited spot-size relation used to connect f-number to pixel-size tradeoffs.","marker":"[20]"},{"why":"Supplies the two-sample t-test statistic with unequal-variances degrees of freedom used for all pairwise significance tests.","marker":"[22]"},{"why":"Justifies the unequal-variances t-test as robust to Type I error, supporting the significance conclusions.","marker":"[23]"}],"fun_headline_variants":["Aperture shape? Negligible for car-vision detection","Car-vision accuracy immune to aperture shape and size","Aperture tweaks don't move automotive detection mAP","Lens aperture: little effect on traffic sign recognition","f/1.8–f/3.4: detection accuracy flat, noise hurts at 48 dB"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument stands on the assumption that ray-traced PSF kernels plus the fitted additive-noise model reproduce what a real automotive camera behind a windshield actually does, and the paper includes no real-image validation of that equivalence.","fun_headline_variants_meta":{"raw":{"variants":["Aperture shape? Negligible for car-vision detection","Car-vision accuracy immune to aperture shape and size","Aperture tweaks don't move automotive detection mAP","Lens aperture: little effect on traffic sign recognition","f/1.8–f/3.4: detection accuracy flat, noise hurts at 48 dB"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00073,"raw_usage":{"total_tokens":3175,"prompt_tokens":757,"completion_tokens":2418,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":373,"completion_tokens_details":{"reasoning_tokens":2325}},"tokens_in":373,"tokens_out":2418,"duration_ms":17350,"temperature":1.0,"reasoning_tokens":2325,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:00:44.165476+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Capture the same scene with four cameras identical except for circular, plus, vertical-slit, and horizontal-slit apertures at the matching f-numbers and gains, then run the same YOLOv8 evaluation; if real-image mAP differences exceed the paper's 5-fold error bars, the simulated null result would be a simulation artifact rather than a physical property.","supporting_citations":[{"cited_title":"Why do animal eyes have pupils of different shapes?","cited_arxiv_id":null,"evidence_quote":"Motivates the aperture-shape question by showing animal pupil shapes are adapted to different visual tasks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the YOLOv8n network and training pipeline used for all 100 detection classes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the synthetic traffic-sign and speed-sign images that form the pinhole dataset."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the synthetic traffic-light images used alongside the sign images."},{"cited_title":"theimagingsource.com/en-us/product/industrial/37u/dfk37bux252/ IEEE TRANSACTIONS ON INTELLIGENT VEHICLES, VOL","cited_arxiv_id":null,"evidence_quote":"Provides the reference camera's sensor parameters and the measured noise-versus-gain curve used to fit additive noise."},{"cited_title":"Refaeilzadeh, L","cited_arxiv_id":null,"evidence_quote":"Supplies the K-fold cross-validation methodology that generates the five mAP values per condition."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the diffraction-limited spot-size relation used to connect f-number to pixel-size tradeoffs."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the two-sample t-test statistic with unequal-variances degrees of freedom used for all pairwise significance tests."},{"cited_title":"Why Welch’s test is Type I error robust,","cited_arxiv_id":null,"evidence_quote":"Justifies the unequal-variances t-test as robust to Type I error, supporting the significance conclusions."}],"review_version":1}