{"id":"02a28210-03b6-4fad-b027-ef3e7cd98d44","arxiv_id":"2502.04486","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"SFit implements the Gauss-Newton approximation for chi-square minimization and, in KMTNet point-lens fits, reports fewer false success and failure flags than BFGS while using fewer evaluations than Nelder-Mead.","lead":"This paper introduces SFit, a Python minimizer that estimates the curvature of a chi-square surface using only first derivatives of the model, and tests it by fitting 1,716 KMTNet microlensing events. The authors report that SFit uses fewer evaluations than Nelder-Mead and reports success more reliably than BFGS, though it finds the minimum on fewer events overall.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Benchmark comparison is asymmetric: SFit fits flux parameters as free variables while SciPy baselines use exact linear fits for the same parameters, confounding speed and reliability claims.","rationale":"The reader's weakest assumption—that the benchmark comparison is fair—is exactly the load-bearing concern. The paper's own Section 4.2 describes different parameterizations for SFit versus the SciPy algorithms, confounding both the evaluation-count comparison and the reliability assessment. The reliability metric itself is also unconventional: Table 1 classifies outcomes by the algorithm's reported success flag, so 'more reliable' means the flag is accurate, not that the minimum is found more often. The paper's Summary acknowledges BFGS finds the best-fitting model almost 100% of the time, while SFit succeeds on 83%, which undercuts the abstract's unqualified 'more reliable' phrasing. The proposed test—equalizing the parameter treatment—would directly determine whether the advertised advantages survive a fair comparison. Since the core algorithm is valid and the paper could be revised to qualify its claims, the reader's CONDITIONAL verdict remains appropriate; no change is needed.","tokens_in":6318,"tokens_out":6287,"duration_ms":64348,"concrete_test":"On a random subset of 200 events, rerun the benchmark with identical parameterization: for SFit, optimize only t0, u0, tE and solve for flux parameters by linear least squares at each iteration (as done for the SciPy baselines); for BFGS, optionally also include flux parameters as free variables as a cross-check. If SFit's Δχ²<1 success rate remains below BFGS's (~99.9% vs 83%) under equalized treatment, the headline 'more reliable' claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that SFit is 'more reliable than the BFGS gradient method' rests on a benchmark in which the algorithms solve different problems. Section 4.2 states that for the three SciPy algorithms, flux parameters are linear-fit at each iteration (optimizing only t0, u0, tE), while for SFit the flux parameters are included as free parameters in the minimizer. Thus Table 2's function-evaluation counts compare a 3-parameter problem with exact linear subproblem solves to a (3+2N)-parameter problem, and Table 1's reliability classification is defined by the algorithm's own success flag, not by the actual success rate. By the Δχ²<1 criterion, BFGS finds the best minimum on 1714/1716 events, SFit on 1429/1716. The abstract's 'more reliable' is therefore only true under a flag-accuracy definition; under the standard 'finds the minimum' definition, BFGS is more reliable. The asymmetry is load-bearing because the advertised advantage may be an artifact of giving SFit a harder problem.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an optimization algorithm for minimizing chi-square in which the Hessian is approximated by products of first derivatives of the model (Eq. 8), following the approach sketched in Gould (2003). The authors implement this as the Python package SFit, apply it to point-lens microlensing light curves, and benchmark it against BFGS, Nelder-Mead, and Newton-CG from SciPy on 1716 KMTNet 2018 events. The paper's central claim is that, for this application, SFit is faster than Nelder-Mead and more reliable than BFGS, while Newton-CG is ineffective. The derivation is standard but clearly presented; the contribution lies in the implementation and the large real-data performance test.","tokens_in":6537,"tokens_out":3708,"duration_ms":38499,"significance":"If the performance comparison were fair, the paper would be a useful contribution to the microlensing fitting toolbox: it provides a public implementation, estimates parameter uncertainties from the approximate Hessian, and tests the method on a large sample of real survey data rather than on synthetic examples. These are genuine strengths. However, the central comparative claim is weakened by the asymmetric benchmark design: the SciPy baselines solve a smaller effective problem than SFit, and the 'reliability' definition is based on the algorithms' own success flags rather than solely on whether the minimum was actually found. As a result, the advertised advantage over BFGS is not established by the reported tests.","major_comments":[{"comment":"The benchmark comparison is asymmetric. For the three SciPy algorithms, the flux parameters fS,k and fB,k are removed by an exact linear fit at every iteration, so the optimizer solves a 3-parameter problem in (t0, u0, tE). For SFit, the flux parameters are included as free parameters, making it a (3+2N)-parameter problem where N is the number of datasets. Therefore Table 2's function-evaluation counts compare two different optimization problems and cannot directly support the speed claims, and Table 1's reliability comparisons conflate algorithmic reliability with problem dimensionality. The authors should rerun the comparison with SFit also linear-fitting the flux parameters at each iteration, or with the SciPy algorithms also treating the flux parameters as free parameters, and report both variants.","section":"Section 4.2, Tables 1 and 2"},{"comment":"The abstract's claim that SFit is 'more reliable than the BFGS gradient method' is only true under a definition of reliability based on the algorithm's self-reported success flag. Using the paper's own criterion for actually finding the minimum (Delta chi^2 < 1), BFGS reaches the best minimum on 1714 of 1716 events (1172 reported successes plus 542 reported failures), whereas SFit reaches it on 1429 of 1716 events (1425 reported successes plus 4 reported failures). Under the metric that matters to a user, BFGS is more reliable. The claim should be rephrased to 'more reliable success reporting' or the primary reliability metric should be changed to whether the minimum was found, independent of the algorithm's flag.","section":"Section 4.3, Table 1, and Section 5"},{"comment":"The classification of fits as 'true positives', 'false positives', etc. uses the algorithm's own success flag together with the Delta chi^2 < 1 criterion. Because the success flags of the SciPy algorithms are known to be unreliable in this application (as the paper itself shows, e.g., 542 BFGS failures nevertheless found the minimum), the four-way classification is informative only about flag behavior, not about actual fitting performance. The paper should present the actual success rate (fraction of events with Delta chi^2 < 1 to the best minimum) as the headline reliability metric, with the flag-based classification as a secondary diagnostic.","section":"Section 4.3, Table 1"}],"minor_comments":[{"comment":"The header 'Evalutions' should be corrected to 'Evaluations'.","section":"Table 2"},{"comment":"The text states that tol = 1e-5 for all algorithms, but the meaning of tol differs across SciPy methods (e.g., gtol for BFGS, xatol/fatol for Nelder-Mead). Please specify the option passed for each method so the comparison is reproducible.","section":"Section 4.2"},{"comment":"The adaptive step-size procedure for epsilon is described only qualitatively ('starts at 0.001 and becomes larger as the minimum is approached'). Since this directly affects the number of function evaluations and convergence behavior, please give the update rule or refer to the relevant code lines.","section":"Section 3.1"},{"comment":"The captions do not state the color convention (purple = reported success, red = reported failure), although the text refers to these colors. Please add the color legend to the captions.","section":"Figures 1-3"}],"recommendation":"major_revision","confidential_remarks":"The main issue is the fairness of the benchmark. I would encourage the editor to ask the authors to provide a matched comparison in which SFit uses the same linear-fit treatment of flux parameters as the SciPy baselines, or to clearly qualify all comparative claims as holding only for the specific parametrization tested. The derivation and code are sound, and the paper is worth publishing after this point is addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I've read Yee & Gould's 'An Alternate Method for Minimizing χ2'. The honest summary is: the algorithm is not new—Eq. 8 is the textbook Gauss-Newton approximation of the Hessian—but the software package and the large KMTNet benchmark are real, reproducible contributions. The derivation is correct, the code is on GitHub, and the test on 1716 events is substantial. That's the actual value here.\n\nThe soft spots are in the benchmark and the abstract's claims. Section 4.2 tells you that for the three SciPy minimizers the flux parameters are linearly fitted at each iteration, while for SFit they're free parameters in the minimizer. So you're comparing a 3-parameter problem with exact subproblem solves to a (3+2N)-parameter problem. That asymmetry confounds the function-evaluation counts in Table 2 and undermines the 'more reliable than BFGS' claim. Table 1's reliability is defined by the algorithm's own success flag, not by whether the minimum was found. Under the paper's own Δχ²<1 criterion, BFGS finds the best minimum on 1714/1716 events and SFit on 1429/1716. So the 'more reliable' claim flips under the standard definition. They should compare with scipy.optimize.least_squares, which is the natural Gauss-Newton implementation, and report both flag accuracy and minimum-finding rates.\n\nThe paper also doesn't cite the standard Gauss-Newton method; it traces the idea to Gould (2003). That's a real omission, minor in substance but it overstates novelty.\n\nIf revised, the paper could be a decent tools note: the package is useful, the uncertainty estimates from the approximate Hessian are convenient, and the event sample is a solid stress test. As written, the headline conclusion doesn't survive a fair comparison. A serious referee should engage with it, but it needs major revision on the benchmark design and framing.","headline":"A useful microlensing fitting package with a benchmark whose headline claim—'more reliable than BFGS'—doesn't hold up to a fair comparison.","tokens_in":7031,"tokens_out":3212,"would_cite":false,"duration_ms":30989,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper introduces SFit, a χ² minimizer that approximates the Hessian with products of first derivatives, and reports that on 1,716 KMTNet microlensing light curves it fits faster than Nelder-Mead and with fewer misleading…","keywords":["chi-square minimization","Newton's method","first-derivative Hessian approximation","microlensing light-curve fitting","KMTNet","point-lens model","parameter uncertainties","sfit_minimize"],"falsifier":"Re-run the same 1,716-event sample with identical flux-parameter handling for every algorithm—for instance, linear-subtract the flux parameters for SFit too—and compare reported success flags and evaluation counts. If SFit no longer shows a lower false-positive/false-negative total or a speed advantage over BFGS and Nelder-Mead, the paper's central comparison collapses.","tokens_in":6103,"feed_emoji":"🔭","tokens_out":8740,"duration_ms":82573,"temperature":0.7,"pith_summary":"The paper introduces SFit, a Python minimizer for $\\chi^2$ that builds a Newton-style step without computing second derivatives: it approximates the matrix of second derivatives of $\\chi^2$ by products of first derivatives of the model, and uses the same products to estimate parameter uncertainties. The authors argue that this makes large-scale model fitting more practical because first derivatives are more stable than second derivatives. They test SFit by fitting point-lens microlensing models to 1,716 light curves from the 2018 KMTNet database and claim that SFit is faster than the Nelder-Mead simplex method and more reliable than the BFGS gradient method, while Newton-CG performs poorly. If true, the result gives microlensing analysts a minimizer that rarely misreports whether a fit has succeeded and that returns uncertainties for free, at moderate computational cost.","feed_headline":"First derivatives only: a faster, more reliable microlensing fitter","feed_subtitle":"SFit reaches the same minima as gradient methods, reports fewer false failures, and returns parameter uncertainties for each fit.","key_machinery":"The load-bearing object is the derivative-product approximation in Equation 8, which replaces the true Hessian (the matrix of second derivatives of $\\chi^2$) with $-2\\sum_k \\sigma_k^{-2}\\,(\\partial F/\\partial A_i)(\\partial F/\\partial A_j)$. This matrix is exactly the Hessian for linear models because the residual-weighted second-derivative term vanishes; the paper keeps the same form for nonlinear models as an approximation. The machinery inverts that approximate Hessian to get a step direction, applies an adaptive step-size multiplier, and reads parameter uncertainties from the diagonal of the inverse matrix. For microlensing, the point-lens model and its derivatives are supplied by the MulensModel package, with the two flux parameters per dataset included as free parameters for SFit but fitted by linear least squares inside each evaluation for the SciPy comparison algorithms.","core_discovery":"The paper's central claim is that a minimizer based on the approximation $\\frac{\\partial^2\\chi^2}{\\partial A_i\\partial A_j}\\approx -2\\sum_k \\frac{1}{\\sigma_k^2}\\frac{\\partial F(x_k)}{\\partial A_i}\\frac{\\partial F(x_k)}{\\partial A_j}$ can serve as a practical Newton-type method. The step is $\\Delta_i=\\sum_j C_{ij}D_j$ with $C=B^{-1}$ and $B$ built from those derivative products, and a small adaptive step-size parameter $\\epsilon$ controls how far the step is taken. On the paper's benchmark of 1,716 point-lens microlensing events, SFit reports success on 1,425 events and every one of those reports is a true positive, whereas BFGS reaches the best of the four minima on nearly all events but reports failure on 32 percent of fits; Nelder-Mead succeeds on 98 percent of events but needs roughly 2.5 times as many function evaluations; Newton-CG has high rates of both false successes and false failures. The paper therefore concludes that SFit is faster than Nelder-Mead and more reliable than BFGS for this application.","pith_inferences":["The comparison embeds the flux parameters differently for SFit than for the SciPy algorithms, so the speed and reliability rankings are not yet separated from parameterization; equalizing that treatment would be a direct test of the paper's headline advantage.","The same derivative-product Hessian approximation could be tried on any least-squares problem where second derivatives are expensive or noisy, but its usefulness likely hinges on how small the neglected residual-weighted term is near the minimum.","Extending the benchmark to finite-source or binary-lens microlensing models would show whether the fixed grid and adaptive step size remain stable on model surfaces with strong degeneracies."],"forward_implications":["Point-lens microlensing fits can be run with an algorithm that never reports a false success, eliminating a common source of wasted follow-up effort.","Users can obtain parameter uncertainties from the same derivative products used for the minimization, with no extra Hessian calculation.","The benchmark numbers give a concrete cost comparison: median 167 $\\chi^2$ evaluations for SFit, 34 for BFGS, and 419 for Nelder-Mead on this sample.","Failure flags from BFGS and Nelder-Mead should not be read as 'no minimum found,' because most of their reported failures still reach the best minimum.","Newton-CG should be avoided for this kind of light-curve fitting."],"supporting_citations":[{"why":"Gives the derivation and the linear-fit solution that the algorithm generalizes.","marker":"Gould 2003"},{"why":"Defines the point-lens model that all fits target.","marker":"Paczynski 1986"},{"why":"Provides the MulensModel light-curve calculations, derivatives, and jacobians used in the fits.","marker":"Poleski & Yee 2019"},{"why":"Source of the BFGS and Newton-CG minimization algorithms used as baselines.","marker":"Nocedal & Wright 2006"},{"why":"Reference for the Nelder-Mead variant used as a simplex baseline.","marker":"Gao & Han 2012"},{"why":"Describes KMTNet, the survey that supplied the 2018 light-curve data.","marker":"Kim et al. 2016"},{"why":"Describes the pySIS photometry and reduction data used for the light curves.","marker":"Albrow et al. 2009"},{"why":"EventFinder grid used to set the starting time of peak for each fit.","marker":"Kim et al. 2018a"}],"fun_headline_variants":["SFit: faster than Nelder-Mead, more reliable than BFGS","New chi^2 minimizer beats gradient methods on microlensing","First-derivative Hessian speeds up microlensing fits","Adaptive step-size minimizer outperforms SciPy optimizers","SFit mimics Newton's method using only first derivatives"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison is fair: SFit fits the brightness-scale and background parameters as free parameters, the SciPy algorithms fit those same parameters with a separate linear fit at every step, and success is defined by reaching the best of the four fits within a $\\chi^2$ difference of 1.","fun_headline_variants_meta":{"raw":{"variants":["SFit: faster than Nelder-Mead, more reliable than BFGS","New chi^2 minimizer beats gradient methods on microlensing","First-derivative Hessian speeds up microlensing fits","Adaptive step-size minimizer outperforms SciPy optimizers","SFit mimics Newton's method using only first derivatives"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000747,"raw_usage":{"total_tokens":3325,"prompt_tokens":940,"completion_tokens":2385,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":556,"completion_tokens_details":{"reasoning_tokens":2298}},"tokens_in":556,"tokens_out":2385,"duration_ms":18089,"temperature":1.0,"reasoning_tokens":2298,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T22:33:12.813246+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same 1,716-event sample with identical flux-parameter handling for every algorithm—for instance, linear-subtract the flux parameters for SFit too—and compare reported success flags and evaluation counts. If SFit no longer shows a lower false-positive/false-negative total or a speed advantage over BFGS and Nelder-Mead, the paper's central comparison collapses.","supporting_citations":[{"cited_title":"chi^2 and Linear Fits","cited_arxiv_id":"astro-ph/0310577","evidence_quote":"Gives the derivation and the linear-fit solution that the algorithm generalizes."},{"cited_title":"2016, Journal of Korean Astronomical Society, 49, 37, doi: 10.5303/JKAS.2016.49.1.037 7","cited_arxiv_id":null,"evidence_quote":"Describes KMTNet, the survey that supplied the 2018 light-curve data."},{"cited_title":"D., Horne, K., Bramich, D","cited_arxiv_id":null,"evidence_quote":"Describes the pySIS photometry and reduction data used for the light curves."}],"review_version":1}