{"id":"66b02a87-3402-46a4-80f0-9efe10a25cd8","arxiv_id":"2412.04197","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":10,"one_line_summary":"GARFIELD is an interactive R toolkit that indexes ultrafast electron diffraction patterns from imperfect quasi-single crystals by matching both spot positions and intensities.","lead":"This paper presents GARFIELD, an interactive software toolkit that indexes ultrafast electron diffraction images from imperfect molecular crystals by matching both spot positions and intensities. It is aimed at structural dynamics experiments where standard auto-indexing tools fail on blurred, overlapping Bragg spots.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Tangent-plane mosaicity approximation is least reliable exactly where GARFIELD claims to operate: fitted σ_mos reaches 12.7°/13.3° in rubrene slice 1, beyond the paper's own 5–10° validity range, yet no exact validation of the intensity decomposition is provided.","rationale":"The reader's CONDITIONAL verdict is appropriate. My pass converges on the same weakest link—the infinitesimal-rotation/tangent-plane treatment of mosaicity—but sharpens it with a concrete internal tension: the paper's own validity statement (σ_mos ≤ ~5°, possibly >10° for moderate anisotropy) is violated by the rubrene example used to demonstrate the package (σ_mos ≈ 12.7°/13.3°). Since no independent exact computation is reported for these values, the error magnitude in the intensity estimates used for filtering and decomposition is unknown. I do not see a reason to reject: the paper is transparent about the approximation, the software is distributed, the worked examples are plausible, and the decomposition output is presented as estimates. The condition should be that the authors provide a numerical validation of the intensity model at high, anisotropic mosaicity, e.g., by comparing with exact integration on synthetic data or by benchmarking partial intensities against an independent model. I therefore keep the reader's CONDITIONAL/UNCHANGED verdict; the concern is a load-bearing correctness risk that should be verified before the quantitative decomposition claim is relied upon.","tokens_in":32295,"tokens_out":5590,"duration_ms":60147,"concrete_test":"Synthetic validation for the rubrene slice 1 parameters: generate ground-truth spot positions and partial intensities by direct numerical integration of the full rotation distribution over SO(3) (Monte Carlo or adaptive quadrature on rotation vectors, without the tangent-plane/marginalization step), using the fitted σ_mos = 12.7°, 13.3°, shape-transform parameters, and Ewald-geometry values. Feed exactly these spot positions and intensities to GARFIELD's GRID SCAN and GEOFIT; if the recovered orientation changes or the per-reflection intensity contributions differ from ground truth by more than the stated tolerance for intensity estimates (e.g., >10% relative on spots with multiple contributors), the tangent-plane approximation is the binding limitation in the claimed application regime.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section IV C 2 derives the mosaicity-induced intensity profile by marginalizing P_ω over rotations about each reciprocal-lattice vector and identifying the marginal with the tangent-plane distribution (Eqs. 12–14). This is exact only for infinitesimal rotations; Figure 5 shows that for finite rotations the correct conditioning is along curved lines in (ω1, ω2), and the paper states that errors increase with anisotropy and that σ_mos up to about 5° are acceptable, with >10° only tolerable under moderate anisotropy. GARFIELD's two advertised functions—intensity-based filtering of candidate orientations (GRID SCAN/GEOFIT) and decomposition of overlapping spots into individual reflection contributions (claim B)—both depend on these predicted intensities. The application examples sit at the edge or beyond this stated envelope: rubrene slice 1 uses fitted σ_mos = 12.7° and 13.3°, slice 2 uses 7.4° and 9.0° (Supplementary §2.3.3), and the main-text example fits σ1 = σ2 = 4.7° (Fig. 7). No exact or numerically integrated calculation of the marginal/conditional intensity profile is reported for these high-mosaicity, anisotropic cases. Consequently, the quantitative accuracy of the per-reflection intensity decomposition—the part of the central claim needed for dynamics analysis—is unverified in the regime the tool is designed for. This is a correctness risk, not a stylistic concern: biased intensity predictions could both mis-rank orientations and mis-assign fractional contributions in overlapped spots.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript introduces GARFIELD, an interactive R-based software toolkit for indexing ultrafast electron diffraction (UED) patterns of imperfect quasi-single crystals. The program combines position and intensity information in a kinematical diffraction model with Gaussian models for mosaicity, beam divergence, bandwidth, and finite-size broadening. It provides two core tools: GRID SCAN, a global search over crystal orientations using a figure of merit that includes intensities, and GEOFIT, a nonlinear least-squares fitting tool that optimizes up to 22 parameters including orientation, beam center, mosaicity, and spot intensities. The central claims are (A) that GARFIELD can index blurred and overlapping UED reflections in cases where conventional geometry-only indexing tools fail, and (B) that it can decompose overlapping Bragg spots into individual reflection contributions useful for time-resolved structural dynamics. The paper includes a detailed model description, cost-function definitions, and application examples on three compounds (κ-(BEDT-TTF)2Cu[N(CN)2]Br, TBAI3, and rubrene), with reported R-factors of 14.9%, 26%, and 16% respectively.","tokens_in":32658,"tokens_out":2897,"duration_ms":31523,"significance":"If the claims hold, GARFIELD fills a genuine niche: UED data from thin, mosaic, low-symmetry crystals are often too blurred for standard auto-indexing tools, and the interactive, intensity-aware, mosaicity-aware approach is a plausible and potentially valuable alternative. The paper's strengths include a clear statement of design principles (use all available information, model fuzziness), an explicit and mostly well-documented mathematical model, a freely available open-source implementation (GPLv3) with a graphical user interface, and application to real experimental data from multiple compounds. The self-identified limitations—kinematical approximation, Gaussian profile assumptions, fixed σ3 values in some fits, and unexplained rubrene spots—are openly discussed. However, the validation is largely in-sample and the intensity-decomposition capability, which is central to claim B, is not independently validated in the high-mosaicity regime where GARFIELD is intended to operate.","major_comments":[{"comment":"The intensity predictions rest on the tangent-plane approximation obtained by marginalizing the mosaicity distribution over rotations about each reciprocal-lattice vector. The paper itself states that this is 'the most critical step in estimating diffraction intensities' and that σ_mos up to about 5° are acceptable, with values above 10° tolerable only under moderate anisotropy. Yet the rubrene applications (Supplementary §2.3.3) use fitted σ_mos values of 12.7° and 13.3° for slice 1 and 7.4° and 9.0° for slice 2, with σ3 fixed. No exact or numerically integrated calculation of the marginal/conditional intensity profile is reported for these high-mosaicity, anisotropic cases. Because claim B (decomposition of overlapping spots into individual reflection contributions) depends directly on these predicted intensities, the accuracy of the decomposition in the tool's intended operating regime is unverified. I recommend adding a synthetic-data validation: generate diffraction patterns with known orientation distributions (including anisotropic and high-σ_mos cases), then compare GARFIELD's per-reflection intensity decomposition against direct numerical integration of the full mosaicity distribution (without the tangent-plane approximation). This would establish the error bounds of the decomposition as a function of σ_mos and anisotropy.","section":"§IV C 2, Eqs. (12)–(14) and Fig. 5"},{"comment":"The evaluation is in-sample. All application examples fit model parameters to the same observed diffraction patterns that are then used for the reported R-factors and visual comparisons; the simulated images are predictions from the fitted parameters, so the agreement is a fit-quality metric rather than an independent test. The paper's appeal to visual comparison as a 'special kind of cross-validation' (Section II) is informal. This matters for claim B because overfitting could produce a misleading decomposition of overlapping spots. I suggest adding an out-of-sample test: for the TBAI3 tilt series, fit parameters to one or two images and compare predictions (positions and intensities) to the remaining recorded images; alternatively, use synthetic data with known ground truth for the individual reflection contributions.","section":"Section V and Supplementary applications"},{"comment":"The position that GARFIELD is needed because 'other indexing tools are ineffective' is not substantiated by any quantitative comparison. The paper does not report attempts to index the example patterns with established tools such as MOSFLM, DIALS, XDS, or serial-crystallography indexers, nor does it quantify how many spots those tools would fail to index. Since the stated contribution is explicitly relative to existing tools, a head-to-head benchmark (or at least a documented report that such tools fail on these data) is needed to support the significance claim. This is not a correctness issue with the model, but it is load-bearing for the paper's advertised advantage.","section":"Introduction and Conclusion"}],"minor_comments":[{"comment":"There are several typographical errors: 'If the the unit cell parameters' (Section II), 'there are reasons to belief' (Section IV C), 'posssible' (Section V), and 'correspont' (Section IV C 2). These should be corrected.","section":"General"},{"comment":"The notation for covariance matrices is confusing: Σ_ω^(xyz), Σ_ω^(123), and Σ_ω^(12) appear in quick succession. A table or more explicit subscript convention would help the reader track which coordinate system is meant.","section":"§IV C 2"},{"comment":"The caption of Figure 5 says 'A couple of rotations with axes perpendicular to an RLP ... are marked by a series of points' but the figure shows many more than a couple; rephrase to 'A set of rotations'.","section":"§IV C 2, Fig. 5"},{"comment":"The paragraph introducing P_h states that it 'occupies a two-dimensional surface given by a sphere of radius h'; since the distribution is on a sphere in three-dimensional reciprocal space, it may be clearer to call it a two-dimensional manifold embedded in three dimensions, to avoid confusion with a planar surface.","section":"§IV C 2"},{"comment":"The supplementary text refers to 'CmpdA' in Figure S1 and S2 legends, while the main text discusses Me4P[Pt(dmit)2]2 and other compounds; please ensure the naming is consistent or explained.","section":"Supplementary §2.3"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid software-toolkit contribution with a well-presented model, and the stress-test concern about the tangent-plane approximation is real but fixable within the manuscript's scope; adding synthetic-data validation and an out-of-sample test would materially strengthen the central claims. The lack of any quantitative comparison to existing auto-indexing tools is a scope/significance issue that the editor may want to weigh for a methods journal such as Structural Dynamics. The manuscript has no obvious integrity problems; the limitations are openly acknowledged."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"GARFIELD is a real, usable toolkit for a genuine pain point: indexing UED patterns from mosaic, low-quality molecular crystals that standard auto-indexers often reject. The paper is worth reading, but don't take the intensity decomposition at face value in the high-mosaicity regime; that part is under-validated.\n\nWhat is new: the combination of an intensity-aware cost function, an interactive GUI, and Gaussian-broadened kinematical modeling is a genuine contribution. The code and data are released under GPLv3, the wiki documentation looks thorough, and the three application examples (kappa-(BEDT-TTF)2Cu[N(CN)2]Br, TBAI3, rubrene) show real utility. The paper is also admirably honest about its limits: the kinematical approximation, the Gaussian broadening assumptions, the fixed sigma_3 values, and the unexplained rubrene spots are all discussed openly. That honesty earns credit.\n\nThe stress-test concern is legitimate. The tangent-plane marginalization over rotations about the reciprocal lattice vector is the most critical step in intensity estimation, and the paper itself says errors grow with anisotropy and sigma_mos. Rubrene slice 1 fits sigma_mos of 12.7 and 13.3 degrees, which is beyond the paper's own comfortable 5-10 degree envelope (they allow >10 only under moderate anisotropy). No exact or numerically integrated calculation is provided for these cases, so the per-reflection intensity decomposition—claim B, crucial for dynamics—lacks validation in exactly the regime the tool is designed for. That is a real soft spot, not a nitpick.\n\nTwo more concerns, in proportion. The paper says other indexing tools are ineffective but provides no benchmark against, say, TakeTwo or pinkIndexer on the same patterns; the comparative claim is asserted. Fitted parameters come without error bars, and some exclusions (masking, discarding spots) are post hoc, so the reported R-factors are best-case rather than predictive. The evaluation is partly in-sample: simulated patterns use parameters fitted to the same observed patterns. None of this is disqualifying, but it limits how much weight the numerical demonstrations can carry.\n\nThe central argument—that intensity information plus interactivity helps index challenging UED data—does hold up. The TBAI3 re-analysis confirms an earlier orientation with a precursor, which is a nice external anchor. And again, the authors shipped code and were straight about the model's limits.\n\nWho this is for: UED experimentalists dealing with imperfect single-crystal samples. It deserves a serious referee. I would send it to peer review with the expectation that the authors either benchmark against existing indexers or provide a numerical validation of the tangent-plane approximation in the high-mosaicity regime.","headline":"Useful, honest UED indexing toolkit with a real niche, but the intensity-decomposition claim is under-validated at the high mosaicity it targets.","tokens_in":33184,"tokens_out":2603,"would_cite":true,"duration_ms":27928,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a software toolkit called GARFIELD can interpret ultrafast electron diffraction patterns that defeat geometry-based auto-indexers, using reflection intensities as well as positions and explicitly modeling the…","keywords":["ultrafast electron diffraction","crystal indexing","mosaicity","Bragg spot decomposition","kinematical diffraction model","graphical user interface","nonlinear least squares"],"falsifier":"Simulate a UED diffraction image from a known crystal structure with a deliberately anisotropic mosaic spread of $\\sigma_{\\mathrm{mos}}=10^\\circ$ to $20^\\circ$, run GRID SCAN and GEOFIT on the synthetic reduced data, and compare the recovered orientation and per-reflection intensity decomposition with the ground truth; a substantial disagreement at large mosaic spread, while a small-spread control recovers correctly, would show that the tangent-plane assumption is the limiting step.","tokens_in":32074,"feed_emoji":"🔬","tokens_out":7755,"duration_ms":77558,"temperature":0.7,"pith_summary":"The paper claims that a software toolkit called GARFIELD can interpret ultrafast electron diffraction patterns that defeat geometry-based auto-indexers: patterns from low-symmetry molecular crystals whose spots are blurred, elongated, and overlapping because the sample is a mosaic of slightly misaligned crystalline domains. The proposed remedy is to use the intensities of the Bragg spots, not only their positions, and to model the physical sources of fuzziness explicitly. If the claim holds, a UED time series only needs one representative image indexed, and merged Bragg spots can be resolved into the individual $hkl$ reflection contributions that pump–probe dynamics analysis actually needs. The paper further claims that an interactive workflow with visual comparison of simulated and recorded diffraction helps prevent wrong indexings from being accepted.","feed_headline":"GARFIELD uses intensity, not just geometry, to index messy UED data","feed_subtitle":"A new toolkit models blurred, overlapping Bragg spots so one ultrafast diffraction image can be indexed per time series.","key_machinery":"The load-bearing object is a probabilistic kinematical diffraction model in which each reciprocal lattice point is expanded into a smooth Gaussian distribution by combining four independent effects: finite coherent domain size, orientation spread (mosaicity), beam divergence, and energy bandwidth. For mosaicity, the model uses the infinitesimal-rotation tangent-plane approximation: the rotation distribution is marginalized over rotations about the reciprocal lattice vector, turning a sphere-cap distribution into a flat bivariate normal in the tangent plane. The expected intensity of a reflection is then obtained by evaluating this normal distribution along the trace of the nearly planar Ewald sphere, with geometric corrections for larger rotations. Around this model sit two search tools: GRID SCAN, which tests up to 17,694,720 orientations on a uniformly sampled sphere to produce candidate orientations ranked by a figure of merit, and GEOFIT, which refines up to 22 parameters by nonlinear least squares against a cost function that combines intensity residuals, optional position residuals, and spot “ambits” that assign predicted reflections to observed spots.","core_discovery":"The paper presents GARFIELD, an interactive toolkit that finds the Laue indices of Bragg reflections in ultrafast electron diffraction images of imperfect quasi-single crystals and quantifies how much each reflection contributes when several overlap into one Bragg spot. Its central claim is that such indexing can succeed where position-only tools fail because the analysis exploits reflection intensities as a filter and because the prediction model accounts for the blurring effects that dominate UED data: domain orientation spread (mosaicity), finite coherent domain size, beam divergence, and energy bandwidth. With a known crystal structure supplied as input, GARFIELD predicts spot positions and intensities in a kinematical approximation, assigns predicted reflections to observed spots through circular “ambits,” and ranks candidate orientations by a cost function that combines intensity residuals with optional position residuals. The user explores parameter sets interactively, starting from a uniform grid search over the full rotation space and refining up to 22 parameters by nonlinear least-squares fitting.","pith_inferences":["The same intensity-plus-blurring strategy could plausibly be applied to other snapshot electron diffraction geometries, such as serial electron diffraction of nanocrystals, where blurred and overlapping spots also defeat geometry-based indexing.","The per-reflection intensity decomposition could be tested quantitatively on tilt-series or precession data in which the same overlapped reflections are measured separated, giving an external check the paper does not perform.","If the tangent-plane approximation is the bottleneck, a straightforward extension would replace the marginalization over rotations about the lattice vector with a numerical integration over the full rotation distribution, at the cost of the speed that the current Gaussian algebra provides."],"forward_implications":["Because each frame in a UED time series is nearly identical, indexing one representative image per series indexes the whole series, reducing the job to a handful of images.","For high-mosaicity samples, many observed spots are superpositions; GARFIELD returns up to five contributing $hkl$ reflections per spot with estimated intensities, so apparent intensity changes in a pump–probe experiment can be traced to actual structural reflections.","Using intensities as a filter makes the orientation search robust to moderately inaccurate intensity predictions: errors reduce the selectivity of the filter rather than shifting the solution to a wrong indexing.","The model's predictions are detailed enough that weak unexpected spots can be singled out for further study, as in the rubrene examples where unexplained reflections point to a possible mixture of polymorphs."],"supporting_citations":[{"why":"supplies the ultrafast electron diffraction data on Me4P[Pt(dmit)2]2 used to demonstrate the workflow and figures.","marker":"36"},{"why":"defines the CIF input format that supplies the known crystal structure required by GARFIELD.","marker":"37"},{"why":"provides the uniform sphere sampling that underlies the grid search over all crystal orientations.","marker":"44"},{"why":"support the argument that dynamical scattering is suppressed in highly textured crystals, justifying the kinematical model.","marker":"51,52"},{"why":"provide the electron scattering factor parameterizations used to compute reflection intensities.","marker":"53,54"},{"why":"introduces the mosaic-block concept of orientation spread that the paper adopts and generalizes.","marker":"55"},{"why":"establishes the Gaussian approximation for reflection partiality that GARFIELD's intensity model extends.","marker":"56"}],"fun_headline_variants":["GARFIELD uses intensity to index messy UED data","GARFIELD models blurred spots to index imperfect crystals","Intensity-based indexing for messy ultrafast diffraction","GARFIELD resolves overlapping Bragg spots with intensity","GARFIELD: intensity-aware indexing for messy crystals"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The intensity estimates that discriminate between candidate orientations rest on assuming that the spread of domain orientations is small and Gaussian enough that rotating a reciprocal-lattice point can be approximated by moving it in a flat tangent plane while averaging away rotations about the lattice vector; if the mosaic spread is large or strongly anisotropic, the predicted intensities may be too inaccurate to discriminate solutions.","fun_headline_variants_meta":{"raw":{"variants":["GARFIELD uses intensity to index messy UED data","GARFIELD models blurred spots to index imperfect crystals","Intensity-based indexing for messy ultrafast diffraction","GARFIELD resolves overlapping Bragg spots with intensity","GARFIELD: intensity-aware indexing for messy crystals"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000674,"raw_usage":{"total_tokens":3041,"prompt_tokens":889,"completion_tokens":2152,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":505,"completion_tokens_details":{"reasoning_tokens":2091}},"tokens_in":505,"tokens_out":2152,"duration_ms":16041,"temperature":1.0,"reasoning_tokens":2091,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T21:39:18.762377+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Simulate a UED diffraction image from a known crystal structure with a deliberately anisotropic mosaic spread of $\\sigma_{\\mathrm{mos}}=10^\\circ$ to $20^\\circ$, run GRID SCAN and GEOFIT on the synthetic reduced data, and compare the recovered orientation and per-reflection intensity decomposition with the ground truth; a substantial disagreement at large mosaic spread, while a small-spread control recovers correctly, would show that the tangent-plane assumption is the limiting step.","supporting_citations":[{"cited_title":"Ishikawa , author S","cited_arxiv_id":null,"evidence_quote":"supplies the ultrafast electron diffraction data on Me4P[Pt(dmit)2]2 used to demonstrate the workflow and figures."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"defines the CIF input format that supplies the known crystal structure required by GARFIELD."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"provides the uniform sphere sampling that underlies the grid search over all crystal orientations."},{"cited_title":"Darwin ,\\ title title XCII","cited_arxiv_id":null,"evidence_quote":"introduces the mosaic-block concept of orientation spread that the paper adopts and generalizes."},{"cited_title":"Brehm , author T","cited_arxiv_id":null,"evidence_quote":"establishes the Gaussian approximation for reflection partiality that GARFIELD's intensity model extends."}],"review_version":1}