{"id":"d26ddb68-0145-4e77-b6db-b98b700cd0d6","arxiv_id":"1908.10780","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A simulation-based sensitivity study shows that the D-Egg module's downward-facing LEDs and a forward/backward PMT ratio likelihood can recover the bubble-column size and effective scattering length in IceCube hole ice within a coarse MC grid.","lead":"This paper reports a Monte Carlo study of using downward-facing LEDs on IceCube's new D-Egg optical modules to measure the properties of refrozen drill-hole ice. If the method works as simulated, it could reduce a key systematic uncertainty in neutrino direction reconstruction and improve the physics reach of the IceCube Upgrade.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Parameter precision is asserted from MC truth recovery, yet the likelihood scan treats simulated data as test data, so the quoted ±20% and ±5 cm accuracy is not a true coverage statement without a calibration ensemble.","rationale":"The reader identified the simplified hole-ice model (uniform centered bubble column, no cabling or misalignment) as the weakest assumption. I agree that this is a substantive limitation, but I see a more immediate, load-bearing concern in the statistical construction of the precision claim. The strongest claim in the paper—and in the reader's summary—is that D-Egg LEDs can determine λe to ±20% and D to ±5 cm. That number comes from the binning of a likelihood scan in which the test data are drawn from the same simulation as the templates. The paper explicitly notes that the 3σ region is smaller than the bin size and that future finer sampling will help, which confirms that the quoted precision is not a calibrated uncertainty. In addition, Equation 4.1 applies a Poisson likelihood to ratio histograms; the ratio of two Poisson means is not Poisson, and the event count per ratio bin is not specified, so the likelihood itself is not a valid statistical model for the stated observable. This is a correctness risk that the reader's summary touches on (through 'MC-closure result') but does not identify as the primary issue. My recommended verdict remains CONDITIONAL: the paper is a useful work-in-progress sensitivity study, but the headline precision must be backed by a coverage study or an explicit likelihood calibration before it can be used for detector design. I would not move to REJECT because the Monte Carlo closure is demonstrated and the paper honestly lists the missing systematics; the issue is the interpretation of the reported precision.","tokens_in":6010,"tokens_out":1619,"duration_ms":15455,"concrete_test":"Perform a coverage test: generate 200–1000 pseudo-experiments with true parameters drawn uniformly across the simulated grid, adding realistic per-event Poisson fluctuations in NPE and Gaussian LED-intensity variations. For each pseudo-experiment, run the full likelihood fit with the same binning and nuisance treatment, and compute the fraction of fits whose 3σ (or 68%) regions contain the true (λe, D). If the coverage is well below the nominal level, then the quoted ±20% and ±5 cm must be reinterpreted as bin-resolution limits, not as measurement uncertainties.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central quantitative claim (recovery of λe to ±20% and D to ±5 cm) is inferred from a likelihood ratio scan (Figure 7) in which pseudo-data are generated from the same Geant4 simulation and binning used to build the template. That is a closure test, not an uncertainty quantification. The quoted precision is the bin size; the paper states 'the size of the 3σ region is smaller than the current bin size,' but the 3σ region is derived from a likelihood ratio with no coverage calibration, no nuisance-parameter marginalization beyond fixing r and φ, and no treatment of Poisson fluctuations in the templates. Moreover, the likelihood in Eq. 4.1 multiplies a binned Poisson likelihood over ratio histograms that are ratios of two Poisson variables; the Poisson likelihood used is not strictly correct for binned ratios, and the number of 'events' in each ratio bin is not defined. The paper also excludes bulk-ice scattering, D-Egg misalignment, cabling shadowing, and non-uniform bubble-column structure; these are listed as future work, yet the ±20% and ±5 cm numbers are presented in the abstract and summary as if they were measurement capabilities. Thus the most load-bearing concern is not that the model is simplified, but that the reported precision is a bin-limited closure result rather than an estimated measurement uncertainty.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This ICRC 2019 proceedings paper from the IceCube Collaboration presents a Monte Carlo sensitivity study for measuring hole-ice properties with the D-Egg optical modules planned for the IceCube Upgrade. The proposed method uses four downward-facing calibration LEDs on an upper D-Egg and records the photoelectron yields at the upward-facing PMT of the lower D-Egg (forward PMT) and the upward-facing PMT of the upper D-Egg (backward PMT). The ratio of these yields is used as an observable to suppress the absolute LED-intensity uncertainty. Photon propagation is simulated with Geant4 over a grid of bubble-column effective scattering lengths lambda_e, bubble-column diameters D, and module offsets (r, phi). A binned Poisson likelihood over ratio histograms is constructed, and the authors report that likelihood fits recover the Monte Carlo truth and conclude that lambda_e can be determined to within +/-20% and D to within +/-5 cm. The paper explicitly states that the results are limited by the chosen binning and that systematic effects from bulk-ice scattering, D-Egg misalignment, and cabling obstruction are not included.","tokens_in":6392,"tokens_out":3071,"duration_ms":33054,"significance":"The proposed calibration concept is timely and relevant for the IceCube Upgrade, where hole-ice uncertainty is a known systematic for low-energy neutrino reconstruction. The use of the forward-to-backward PMT ratio is a sensible way to reduce LED-intensity calibration uncertainty, and the simulation incorporates realistic PMT quantum efficiency, collection efficiency, and charge resolution. The paper is honest in labeling the study as a Monte Carlo sensitivity study and in listing omitted systematics. The main value is as a baseline for future in-situ measurements. Its principal weakness is that the quoted precision is a bin-limited closure result rather than a calibrated measurement uncertainty, and the likelihood used for the ratio observable is not derived from a well-defined generative model.","major_comments":[{"comment":"The central quantitative claim that lambda_e can be determined to within +/-20% and D to within +/-5 cm is not supported by a proper uncertainty quantification. The likelihood scan uses pseudo-data generated from the same Geant4 model and the same binning used to build the template histograms, so recovering the MC truth is expected closure behavior. The paper states that \"the size of the 3sigma region is smaller than the current bin size,\" which means the quoted precision is essentially the grid spacing of the scan, not a calibrated confidence interval. To make the stated precision claim valid, the authors should either perform an ensemble of pseudo-experiments and demonstrate coverage of the quoted intervals, or explicitly rephrase the claim as a bin-resolution-limited sensitivity estimate.","section":"Section 4, Figure 7"},{"comment":"The binned Poisson likelihood in Eq. (4.1) is applied to histograms of the ratio of NPE at the forward and backward PMTs, but the ratio of two Poisson-distributed counts is not itself Poisson distributed, and the manuscript does not define what \"number of events\" (n_i) means for a ratio histogram. Without a generative model for how individual entries in the ratio histogram are produced and counted, the Poisson likelihood is only a heuristic. The authors should either define the ratio-bin count model explicitly or use a likelihood based on the actual sampling distribution of the ratio, e.g., through a Monte Carlo-calibrated likelihood or an unbinned likelihood.","section":"Section 4, Eq. (4.1)"},{"comment":"The abstract and summary present the +/-20% and +/-5 cm numbers as measurement capabilities without the caveats that the paper itself lists in the final paragraph of Section 4: bulk-ice scattering, D-Egg alignment uncertainty, and cabling obstruction are not included. Since these effects are directly relevant to in-situ deployment, the precision claims should be qualified throughout the paper as idealized, simulation-only sensitivities. Otherwise the summary overstates what the study actually demonstrates.","section":"Section 4, final paragraph; Section 5"}],"minor_comments":[{"comment":"The text says the bulk ice \"does not scatter photons,\" while Section 1 states that the bulk ice has an approximately 20 m scattering length. The simulation approximation should be stated explicitly as neglecting bulk-ice scattering for the sensitivity study, not as a property of real ice.","section":"Section 3.1"},{"comment":"The notation in Eq. (4.1) is under-specified: the LED index l, the bin index i, the meaning of Nbin = 10, and the construction of the ratio histograms from NPE values should be defined in the text. The bold Greek symbols theta and nu also render awkwardly and should be replaced with standard vector notation.","section":"Section 4, Eq. (4.1)"},{"comment":"The color scale label \"log(L/L)\" appears to be missing a subscript or normalization; it should be something like \"log(L/L_max)\" or \"-2 Delta log L,\" and the caption should state how the white star marks the best-fit bin and how the 3sigma contour is defined.","section":"Figure 7"},{"comment":"The sentence stating that these results \"plan to be applied to the entire IceCube data collected over 10 years\" needs a brief justification or a qualifier, since D-Eggs are only being deployed in the Upgrade and transferring their calibration to legacy IceCube data will require additional assumptions about hole-ice properties over time and depth.","section":"Section 2"}],"recommendation":"major_revision","confidential_remarks":"This is a short conference proceedings paper, and the scope is appropriate for ICRC. The main issue is that the headline precision numbers are presented as established measurement capabilities when they are bin-limited closure estimates from an idealized simulation. I believe the paper can be made acceptable by softening the claims and clarifying the likelihood construction, but the current wording overstates what the simulation demonstrates."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is a conference write-up from ICRC2019, and it is exactly what it looks like: a Monte Carlo sensitivity study for measuring the bubble-column diameter and effective scattering length with the D-Egg downward-facing LEDs. The new bit is the specific geometry and the use of the forward/backward PMT NPE ratio as an intensity-robust observable, plus a likelihood fit on top. It is a well-scoped baseline, and the authors are upfront that this is work in progress.\n\nWhat it does well: the ratio construction sensibly removes the dominant LED-intensity systematic. The PMT response is treated with laboratory-measured quantum and collection efficiencies. The MC-closure figures show the best-fit point lands in the true bin for the tested parameter space. The text repeatedly says 'potential' and 'work in progress.'\n\nThe soft spots are quantitative, not qualitative. The headline precision—±20% in λe and ±5 cm in D—is essentially the bin size. The likelihood-ratio scan uses pseudo-data generated from the same Geant4 simulation that produced the templates, so this is a closure test, not an uncertainty estimate. To make a claim like 'determine within ±20%' you need an ensemble of pseudo-experiments and a coverage check; the paper does not provide that. The Poisson likelihood on ratio histograms is also questionable because each bin is a ratio of two Poisson variables—the number of 'events' per bin is not well defined. That may not change the best-fit location, but it could distort the likelihood shape and therefore the confidence regions. Beyond that, the model excludes bulk-ice scattering, D-Egg misalignment, cabling shadows, and non-uniform bubble-column structure. Those are listed as future work, which is honest, but it means the abstract's precision numbers are not yet tied to a measurement capability.\n\nNone of this is fatal. The central conclusion—that the proposed observable has enough sensitivity to strongly constrain hole-ice parameters—is plausible and likely correct. But the specific numbers in the abstract and summary should be read as binning-limited sensitivity in a simplified simulation, not as calibrated uncertainty.\n\nWho is this for? Anyone working on IceCube Upgrade calibration or systematic uncertainties in neutrino reconstruction. It deserves a serious referee, but the referee should ask for a coverage study, a proper likelihood for ratio observables, and a quantified list of systematics before the precision numbers appear in a citable journal version.\n\nMy recommendation: send it to peer review as a proceedings-style paper or technical note, but require the authors to soften the abstract and add fake-data coverage checks.","headline":"Honest, clearly scoped MC sensitivity study for D-Egg hole-ice calibration; the precision claims are bin-limited closure results, not real measurement uncertainties.","tokens_in":6790,"tokens_out":2107,"would_cite":false,"duration_ms":22022,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A Monte Carlo study shows that D-Egg downward LEDs, using the forward-to-backward photoelectron ratio in a likelihood fit, can recover the hole-ice bubble column's effective scattering length to within ±20% and diameter to within ±5 cm.","keywords":["IceCube Upgrade","D-Egg optical module","hole ice calibration","bubble column","LED flasher","effective scattering length","neutrino reconstruction","Monte Carlo sensitivity study"],"falsifier":"After deployment, compare likelihood-fit values of $\\lambda_e$ and $D$ from real D-Egg LED flashes against an independent optical probe of the same hole ice—for example, photographic images of the bubble column or scattering measurements from neighboring strings; a systematic disagreement larger than the quoted $\\pm 20\\%$ in scattering length or $\\pm 5$ cm in diameter would show the simulation geometry does not transfer to the ice.","tokens_in":5797,"feed_emoji":"💡","tokens_out":8244,"duration_ms":83598,"temperature":0.7,"pith_summary":"This paper seeks to establish that the downward-facing calibration LEDs on the D-Egg optical modules for the IceCube Upgrade can measure the optical properties of the refrozen drill-hole ice from the recorded light alone. Using a Monte Carlo simulation of photon propagation, the authors build a binned Poisson likelihood from the ratio of photoelectrons seen by the forward and backward photomultipliers, and show that the best-fit point lands in the Monte Carlo truth bin for every tested combination of bubble-column size, scattering, and module position. The projected precision is $\\pm 20\\%$ on the effective scattering length in the bubble column and $\\pm 5$ cm on its diameter. A sympathetic reader should care because hole ice is currently a sizable systematic for low-energy neutrino reconstruction, and the same fitted parameters could be applied to the full decade of IceCube data.","feed_headline":"Simulation: D-Egg LEDs can measure refrozen ice to ±5 cm","feed_subtitle":"Light ratios from two photomultipliers recover scattering length to ±20%, easing a neutrino-reconstruction systematic.","key_machinery":"The load-bearing observable is the forward-to-backward photoelectron ratio: for each of the four downward-pointing LEDs, the upward-facing PMT in the lower module sees mostly direct light, while the upward-facing PMT in the upper module sees light that has been Mie-scattered inside the bubble column, so the ratio depends on both $D$ and $\\lambda_e$ but divides out the LED intensity. The paper forms binned ratio histograms from the four LEDs and fits them with a binned Poisson likelihood whose parameters are $\\lambda_e$, $D$, $r$, and $\\varphi$, treating $r$ and $\\varphi$ as nuisance parameters. The scattering is parameterized by the effective scattering length $\\lambda_e = \\lambda_s/(1-\\langle\\cos\\theta\\rangle)$, with $\\langle\\cos\\theta\\rangle = 0.95$ in the Henyey-Greenstein function.","core_discovery":"The central claim is that the ratio of photoelectrons at two vertically separated PMTs encodes the two hole-ice parameters—bubble-column diameter $D$ and effective scattering length $\\lambda_e$—while cancelling the unknown absolute LED intensity. The paper supports this by simulating two perfectly aligned D-Egg modules in a 60 cm borehole, with Geant4 photon propagation and Henyey-Greenstein scattering, and by fitting the simulated photoelectron-ratio histograms with a binned Poisson likelihood over $\\lambda_e$, $D$, the module offset $r$, and the rotation $\\varphi$. The fit minimum coincides with the Monte Carlo truth for all tested cases, and the contour size is limited by the chosen binning rather than by statistical fluctuations. From this the paper concludes that in-situ D-Egg measurements can determine $\\lambda_e$ to within $\\pm 20\\%$ and $D$ to within $\\pm 5$ cm.","pith_inferences":["The paper leaves the precision bound set by its grid binning; a finer grid in the same likelihood setup is a natural extension and could yield tighter constraints than the quoted $\\pm 20\\%$ and $\\pm 5$ cm.","The same ratio-observable idea could be applied to any two photosensors that view a common pulsed light source through a scattering medium, wherever the main calibration nuisance is the absolute source brightness.","A testable follow-up, not claimed in the paper, is to overlay the fitted bubble-column extent on camera images of the same hole ice once real D-Egg data exist; a mismatch would indicate the centered-uniform-column model needs refinement."],"forward_implications":["If the quoted precision transfers to the deployed detector, the hole-ice systematic in low-energy neutrino reconstruction would be substantially reduced.","The same fitted hole-ice parameters can be applied retroactively to the existing decade of IceCube data, since the Upgrade holes sit in the same ice.","IceCube-Gen2 can reuse the identical D-Egg LED calibration hardware and analysis without new design work.","With modules spaced 2.7 m apart along the string, the measurement yields a far denser map of bubble-column properties with depth than the current 17 m-spaced DOMs allow."],"supporting_citations":[{"why":"It supplies the baseline bulk-ice scattering length of about 20 m that makes the hole ice a distinct optical medium.","marker":"[2]"},{"why":"It documents that previous measurements of hole-ice properties were inconclusive, motivating the D-Egg LED approach.","marker":"[3]"},{"why":"It defines the IceCube Upgrade deployment geometry, including the roughly 3 m vertical spacing used in the simulation.","marker":"[5]"},{"why":"It introduces the D-Egg optical module whose downward LEDs and upward-facing PMTs are the subject of the study.","marker":"[6]"},{"why":"It provides the Geant4 photon-propagation engine that generates the simulated photoelectron maps.","marker":"[8]"},{"why":"It supplies the Henyey-Greenstein scattering function used to parameterize Mie scattering in the bubble column.","marker":"[9]"},{"why":"It supplies the laboratory-measured PMT quantum efficiency and collection efficiency that make the simulation realistic.","marker":"[10]"},{"why":"It specifies the LED angular distribution with a 120-degree viewing angle used in the simulation geometry.","marker":"[11]"}],"fun_headline_variants":["Simulation: D-Egg LED ratio measures ice to ±5 cm","Simulation: PMT ratio from D-Egg recovers scattering to ±20%","Simulation: D-Egg LEDs gauge hole-ice size and scattering","Simulation: D-Egg light ratio yields ±5 cm ice precision"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire sensitivity estimate rests on the assumption that each drill hole can be modeled as a centered, uniform bubble column of adjustable diameter inside a 60 cm borehole, with the outer clear ice scattering exactly like bulk ice and no cables or module misalignment blocking the light.","fun_headline_variants_meta":{"raw":{"variants":["Simulation: D-Egg LED ratio measures ice to ±5 cm","Simulation: PMT ratio from D-Egg recovers scattering to ±20%","Simulation: D-Egg LEDs gauge hole-ice size and scattering","Simulation: D-Egg light ratio yields ±5 cm ice precision"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001471,"raw_usage":{"total_tokens":5945,"prompt_tokens":1006,"completion_tokens":4939,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":622,"completion_tokens_details":{"reasoning_tokens":4857}},"tokens_in":622,"tokens_out":4939,"duration_ms":35141,"temperature":1.0,"reasoning_tokens":4857,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T10:34:08.476710+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"After deployment, compare likelihood-fit values of $\\lambda_e$ and $D$ from real D-Egg LED flashes against an independent optical probe of the same hole ice—for example, photographic images of the bubble column or scattering measurements from neighboring strings; a systematic disagreement larger than the quoted $\\pm 20\\%$ in scattering length or $\\pm 5$ cm in diameter would show the simulation geometry does not transfer to the ice.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It documents that previous measurements of hole-ice properties were inconclusive, motivating the D-Egg LED approach."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It defines the IceCube Upgrade deployment geometry, including the roughly 3 m vertical spacing used in the simulation."},{"cited_title":"Allison et al., Nucl","cited_arxiv_id":null,"evidence_quote":"It supplies the Henyey-Greenstein scattering function used to parameterize Mie scattering in the bubble column."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It supplies the laboratory-measured PMT quantum efficiency and collection efficiency that make the simulation realistic."},{"cited_title":"Makino, EPJ Web Conf","cited_arxiv_id":null,"evidence_quote":"It specifies the LED angular distribution with a 120-degree viewing angle used in the simulation geometry."}],"review_version":1}