{"id":"c5cd317e-cc3d-4e1c-89c4-eb2a8dc46e26","arxiv_id":"2501.05914","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"SCAN and r2SCAN predict the Neel temperatures of 48 antiferromagnetic insulators with mean absolute percentage errors of 23% and 22% and Pearson correlations of 0.97 and 0.98 against experiment.","lead":"This paper compares two modern density functional theory functionals, SCAN and r2SCAN, on 48 antiferromagnetic materials by computing magnetic exchange couplings and simulating their ordering temperatures. It reports that both functionals match experiment far better than older GGA and GGA+U methods, making them promising for high-throughput magnetic materials screening.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline outperformance of SCAN/r2SCAN over GGA and GGA+U rests on a cross-set MAPE comparison that is not controlled on the same 48 materials.","rationale":"The reader's weakest_assumption identified exactly this issue: the headline comparison to GGA and GGA+U is not controlled across the same material set. I agree that this is the most load-bearing concern. The correlation coefficients for SCAN and r2SCAN are strong, and the MAPE values are informative, but the relative claim 'greatly outperform GGA and GGA+U' depends on comparing like with like. With only 27 of 29 previous compounds overlapping and 21 new compounds lacking any GGA/GGA+U benchmark in this paper, the reported gap of 87% vs 23% could be distorted by compositional differences. The paper does not provide per-material GGA/GGA+U predictions for the new compounds, so the reader cannot check whether the 21 additions are systematically easier or harder for meta-GGA. This does not invalidate the paper's useful benchmark, but it does mean the central comparative claim is conditional on the overlap-set comparison being representative. A targeted recomputation on the 27 common compounds would settle the issue. I therefore agree with the reader's conditional verdict and do not recommend changing it.","tokens_in":23339,"tokens_out":3824,"duration_ms":38651,"concrete_test":"Recompute the MAPE and Pearson r for SCAN and r2SCAN restricted to the 27 compounds that overlap with ref [22], using the predicted and experimental values already in Table II, and compare with the GGA and GGA+U MAPE values from ref [22] computed on those same 27 compounds. If the restricted meta-GGA MAPE remains close to 22-23% and the GGA/GGA+U MAPE on the same 27 set remains near 87%/54%, the claim is supported. If the restricted meta-GGA MAPE rises substantially (e.g., above 30%) or the GGA MAPE on the 27-set falls well below 87%, the headline outperformance is inflated. A stronger check is to run the same GGA and GGA+U workflow used in ref [22] on the 21 additional materials and compare per-material errors across the full 48-compound set.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central quantitative claim is that SCAN and r2SCAN greatly outperform GGA and GGA+U, with MAPE values of 23% and 22% versus 87% and 54%. However, these numbers are not computed on the same set of compounds. Section II.B states that only 27 of the 29 compounds from the previous GGA/GGA+U study [22] are included, plus 21 new compounds, and Figure 2's inset explicitly labels the GGA and GGA+U data as taken from ref [22]. Thus the comparison is like-for-like only for 27 compounds, while the 21 additional materials have no GGA or GGA+U results in this paper. The exclusions also change the set: LiCoPO4 was dropped for convergence issues and KMnSb because of no definitive experimental TN. If the added materials are ones where meta-GGA happens to be accurate, or where GGA/GGA+U would be especially poor, the reported 3-4x improvement overstates the real advantage. The Pearson coefficients of 0.97/0.98 are robust for the meta-GGA functionals themselves, but they do not establish superiority over GGA/GGA+U. Because the abstract and conclusions present the outperformance as the main result, this uncontrolled comparison is the most load-bearing weak point.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper benchmarks the SCAN and r2SCAN meta-GGA functionals for predicting Néel temperatures of 48 insulating 3d antiferromagnetic materials. The workflow combines DFT total energies of multiple magnetic configurations, least-squares fitting of Heisenberg exchange parameters, and classical Monte Carlo simulations of the resulting spin model. The authors report mean absolute percentage errors of 23% (SCAN) and 22% (r2SCAN) relative to experiment, with Pearson correlation coefficients of 0.97 and 0.98, and compare these against previously published GGA and GGA+U results from a 29-material study. They further use AFM-FM energy differences from small supercells to argue that the HSE06 hybrid functional underestimates transition temperatures relative to the meta-GGA functionals.","tokens_in":117,"tokens_out":3355,"duration_ms":85176,"significance":"If the claims are supported, this is a valuable benchmark: it is a large, chemically diverse dataset; the transition temperatures are forward predictions from a Heisenberg model and Monte Carlo, with no fitting to experimental T_N; the exchange parameters are tabulated in the supplement; and the VASP/FHI-aims cross-check in Table I is a useful validation of the pseudopotential workflow. The paper also makes an honest effort to monitor exchange-parameter convergence as the number of magnetic configurations increases. However, the headline claim that SCAN and r2SCAN greatly outperform GGA and GGA+U currently rests on a cross-set comparison that is not controlled on the same materials, and the HSE06 conclusion is based on energy differences rather than simulated transition temperatures. These issues make the central comparative claim weaker than the abstract suggests.","major_comments":[{"comment":"The central comparison of MAPE values is not apples-to-apples. The SCAN and r2SCAN MAPE values of 23% and 22% are computed on the present 48-material set, whereas the GGA and GGA+U MAPE values of 87% and 54% are taken from reference [22], which used a 29-material set; only 27 of those 29 materials are in the present set, and the 21 additional materials have no GGA or GGA+U results in this paper. If the additional materials happen to be easier for meta-GGA functionals, the reported improvement is inflated. The abstract and conclusions present this comparison as the main result, so the authors should either recompute GGA and GGA+U on the full 48-material set or at least report the common-27-subset MAPE for all functionals.","section":"Section II.B and Figure 2 inset"},{"comment":"The conclusion that HSE06 underestimates transition temperatures is an extrapolation from the AFM-FM energy difference per magnetic atom in small supercells (Figure 4), not from Monte Carlo simulations of HSE06 exchange parameters. An energy difference between two ordered states is a useful indicator but is not a transition temperature; moreover, some compounds are explicitly excluded because small supercells incorrectly stabilize ferromagnetism. The text softens this with 'expected to underestimate', but the abstract states it as a finding. The authors should either run the full MC workflow with HSE06 couplings on a tractable subset or restrict the claim to energy differences.","section":"Section III.C"},{"comment":"The exclusion of metallic magnetic configurations is a modeling choice whose effect on the exchange parameters is demonstrated for only one compound (MnTe). Because the metallic versus insulating character of a configuration can depend on the functional, this exclusion could bias the comparison between functionals. The authors should provide a sensitivity analysis showing that the extracted J values and resulting T_C are robust across a broader set of compounds when metallic configurations are included or excluded, or otherwise justify why the exclusion is unbiased.","section":"Section III.A and Figure 1(b)"}],"minor_comments":[{"comment":"The text refers to 'CrF4' as one of the compounds with maximum errors; this appears to be a typo for CrF2, which is the compound listed in Figure 3 and Table II.","section":"Section III.B"},{"comment":"The Pearson correlation coefficients are reported as 97% and 98%, but Pearson r is dimensionless and should be written as 0.97 and 0.98.","section":"Abstract and Section III.B"},{"comment":"The placeholder 'URL-will-be-inserted-by-publisher' should be replaced with the actual supplemental material identifier or repository link.","section":"Reference [76]"},{"comment":"Several compounds have multiple experimental T_N values (e.g., CrCl2, MnSe, Fe2TeO6, K2NiF4). The authors should state how a single experimental value was selected for the MAPE calculation, or report the sensitivity of MAPE to the experimental choice.","section":"Table II and Section III.B"},{"comment":"The legend labels the DFT method as 'GGA', but the text and context suggest this is PBE; the authors should specify the exact functional and note whether the GGA and HSE06 energy differences were computed with the same supercells and cutoff as the SCAN and r2SCAN calculations.","section":"Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a DFT benchmarking journal and the dataset is potentially useful. The main obstacle is the uncontrolled comparison with reference [22]; this is fixable by recomputing GGA/GGA+U on the same set or by restricting the headline claim. The HSE06 claim also needs to be downgraded unless MC simulations are performed. I do not see a circularity problem: the workflow does not fit to experimental transition temperatures."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here is my take. The genuinely new thing is the 48-material benchmark of SCAN and r2SCAN for Neel temperatures, with the full exchange-parameter tables in the supplement. That is real work and useful: DFT + Heisenberg + classical Monte Carlo is standard, but no one had done it at this scale for meta-GGAs, and the Pearson correlations of 0.97/0.98 on the meta-GGA numbers are credible. They also cross-checked VASP against FHI-aims for eight compounds, which is the right kind of sanity check.\n\nThe main soft spot is exactly the one the stress test flags. The abstract and conclusions say SCAN/r2SCAN 'greatly outperform' GGA and GGA+U, but the GGA/GGA+U MAPEs come from a previous 29-compound study, not from the same 48 compounds here. Only 27 overlap; 21 are new. If the extra materials are easy cases, the 87%/54% versus 22%/23% gap overstates the advantage. This is not a deal-breaker for the value of the meta-GGA data, but the headline claim needs to be re-run on a common set, or the comparison should be restricted to the overlap. As written, it is uncontrolled.\n\nTwo smaller things. First, the HSE06 conclusion is inferred from AFM-FM energy differences in small supercells, not from MC; the text says 'expected to underestimate,' which is honest, but the abstract states the finding more flatly. Second, the experimental TN selection is under-specified: Table II lists multiple values for many compounds, and the paper never states which value goes into MAPE and Pearson. That matters for a benchmark.\n\nThe exclusions and convergence handling are modeling choices, not fits to experimental TNs. Nothing in the workflow fits Tc to experiment, so there is no circularity. The lack of input files and code is a minor limitation, but the exchange tables are enough to reproduce the MC step. Citation pattern is fine; the self-citations to their earlier GGA benchmark are directly relevant.\n\nOverall: this paper is worth engaging. The central practical message — that meta-GGAs track experimental Neel temperatures well for insulating 3d antiferromagnets — probably holds, but it needs a same-set baseline before it can be stated as strongly as it is. I would send it to review, mainly because a serious referee can force the controlled comparison and the experimental value specification. I would not cite the 87% versus 23% claim as-is, but I would cite the exchange-parameter benchmark.","headline":"Useful meta-GGA benchmark for 48 antiferromagnets, but the headline outperformance over GGA/GGA+U rests on an uncontrolled cross-set comparison.","tokens_in":24122,"tokens_out":3587,"would_cite":true,"duration_ms":32735,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":["71.15.Mb","75.10.Hk","75.30.Et","75.50.Ee"],"model":"deepseek-v4-flash","headline":"By mapping density functional energies onto a Heisenberg Hamiltonian and running classical Monte Carlo, SCAN and r2SCAN predict the Néel temperatures of 48 insulating antiferromagnets with mean absolute errors of 23% and 22%…","keywords":["antiferromagnetism","Néel temperature","SCAN functional","r2SCAN functional","meta-GGA","Heisenberg exchange couplings","classical Monte Carlo","HSE06 hybrid functional"],"falsifier":"Recompute GGA and GGA+U transitions for all 48 materials using the same supercell, least-squares, and Monte Carlo pipeline; if their mean absolute percentage errors on the full set fall close to the 22-23% reported for the meta-GGA functionals, the headline improvement over GGA and GGA+U would be an artifact of comparing different material sets.","tokens_in":93,"feed_emoji":"🧲","tokens_out":8449,"duration_ms":128127,"temperature":0.7,"pith_summary":"The paper asks whether two meta-GGA exchange-correlation functionals, SCAN and its regularized successor r2SCAN, can predict the temperature at which an antiferromagnetic insulator loses its magnetic order. The procedure is: compute total energies of many spin arrangements with density functional theory, fit the results to a classical Heisenberg Hamiltonian, and feed the fitted exchange couplings into Monte Carlo simulations. Across 48 insulating 3d antiferromagnets, the predicted Néel temperatures come within a mean absolute percentage error of 23% for SCAN and 22% for r2SCAN, with Pearson correlations of 0.97 and 0.98 against experiment. If this holds, materials scientists can estimate magnetic ordering temperatures without the per-material Hubbard-$U$ tuning that GGA+U requires, and the expensive hybrid functional HSE06 looks unnecessary for this property class.","feed_headline":"SCAN and r2SCAN predict magnetic ordering temperatures to ~22 percent","feed_subtitle":"Across 48 antiferromagnets, the two meta-GGA functionals beat GGA and GGA+U by more than half the typical error.","key_machinery":"The load-bearing object is the mapping from density functional total energies to a classical Heisenberg Hamiltonian, $\\hat{H} = -\\frac{1}{2}\\sum_{i,j} J_{ij} \\hat{\\mathbf{S}}_i \\cdot \\hat{\\mathbf{S}}_j$, with unit spins on magnetic sites and exchange couplings $J_{ij}$ out to the needed neighbour shell. To get reliable couplings, the paper uses about three times the minimum number of magnetic configurations and extracts the $J_{ij}$ by least-squares fitting, excluding configurations that turn metallic during self-consistency because they spoil convergence. The fitted couplings are then inserted into classical Monte Carlo simulations with parallel tempering on supercells of at least 2000 magnetic sites. SCAN and r2SCAN enter at the first step, supplying the total energies that the fit converts into couplings, so the entire accuracy story rests on how well these functionals describe the energy differences between spin arrangements.","core_discovery":"The central discovery is that the meta-GGA functionals SCAN and r2SCAN, despite having no empirical fit to magnetic data, produce Néel temperatures for a diverse 48-material set of insulating 3d antiferromagnets that track experiment across nearly three decades in temperature, from tens of kelvin to over a thousand kelvin. The authors report mean absolute percentage errors of 23% for SCAN and 22% for r2SCAN, with Pearson correlation coefficients of 0.97 and 0.98, compared with reported errors of 87% for GGA and 54% for GGA+U from an earlier 29-material benchmark. They also find that the hybrid functional HSE06 yields smaller energy differences between antiferromagnetic and ferromagnetic configurations than SCAN and r2SCAN, and therefore infer it underestimates transition temperatures relative to experiment. For one compound, CrF2, r2SCAN incorrectly predicts ferromagnetic rather than antiferromagnetic order, a failure flagged for the otherwise similar functional.","pith_inferences":["Editorial inference: the same pipeline applied to 4d/5d or weakly itinerant magnets, which the 3d insulating selection deliberately excludes, would test whether the meta-GGA advantage survives spin-orbit coupling and longer-range interactions.","Editorial inference: the two large disagreements between the functionals, MnTe (232 K versus 356 K) and CrF2 (AFM versus FM), are natural targets for wavefunction benchmarks such as coupled-cluster or quantum Monte Carlo to decide which functional is closer to reality.","Editorial inference: because the absolute errors remain about one fifth of the transition temperature, the fitted exchange parameters may be more transferable than the temperatures themselves; using them as training data for machine-learned spin Hamiltonians is a plausible next step."],"forward_implications":["A screening workflow built on r2SCAN total energies, a Heisenberg fit, and classical Monte Carlo can estimate magnetic ordering temperatures of insulating 3d antiferromagnets with typical errors of about 22%, without Hubbard-$U$ tuning.","The systematic opposite biases of GGA, which overestimates, and GGA+U, which underestimates, reported in the earlier study are largely removed by the meta-GGA functionals, which scatter roughly evenly above and below experiment.","For most of the 48 materials, SCAN and r2SCAN agree: 77% of the compounds have lower SCAN than r2SCAN transition temperatures, but the distribution is centered near a ratio of 1.","HSE06, despite being more expensive, is not the right tool for this property: its smaller AFM-FM energy differences place its predicted transition temperatures below both meta-GGA predictions and experiment.","The high Pearson correlation and the absence of a fitted empirical parameter make r2SCAN a candidate for high-throughput and machine-learning-assisted prediction of magnetic transition temperatures."],"supporting_citations":[{"why":"Provides the GGA and GGA+U benchmark error rates (87% and 54%) and the 29-compound predecessor data with which 27 of the present 48 materials overlap.","marker":"[22]"},{"why":"Defines the SCAN meta-GGA exchange-correlation functional whose performance is the subject of the study.","marker":"[11]"},{"why":"Defines the r2SCAN functional, the regularized SCAN variant that the study evaluates alongside SCAN.","marker":"[15]"},{"why":"Establishes the Heisenberg Hamiltonian mapping used to extract exchange couplings from total energies.","marker":"[26]"},{"why":"Motivates the least-squares fitting of exchange couplings from a larger-than-minimal set of magnetic configurations.","marker":"[27]"},{"why":"Supplies the classical Monte Carlo code with parallel tempering that turns fitted couplings into Neel temperatures.","marker":"[28]"},{"why":"Generates the optimized supercells large enough to contain exchange interactions out to the required neighbor shell.","marker":"[24]"},{"why":"Provides the all-electron code used to validate the plane-wave pseudopotential SCAN couplings for a subset of simple materials.","marker":"[75]"}],"fun_headline_variants":["SCAN and r2SCAN predict Neel temps within 22-23% error","Meta-GGA beats GGA and GGA+U for magnetic ordering temperatures","r2SCAN and SCAN: 0.97-0.98 correlation with experimental Neel temps","No empirical magnetic fitting: SCAN and r2SCAN get 22% error on Neel temps","Antiferromagnets: SCAN and r2SCAN halve GGA+U error on Neel temps"],"cache_read_input_tokens":26240,"weakest_assumption_plain":"The outperformance claim assumes that the 87% and 54% error rates from the earlier 29-material GGA and GGA+U study can be compared directly with the 23% and 22% errors on the present 48-material set, even though 21 of the 48 materials were not in that earlier study and have no GGA or GGA+U data here.","fun_headline_variants_meta":{"raw":{"variants":["SCAN and r2SCAN predict Neel temps within 22-23% error","Meta-GGA beats GGA and GGA+U for magnetic ordering temperatures","r2SCAN and SCAN: 0.97-0.98 correlation with experimental Neel temps","No empirical magnetic fitting: SCAN and r2SCAN get 22% error on Neel temps","Antiferromagnets: SCAN and r2SCAN halve GGA+U error on Neel temps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001503,"raw_usage":{"total_tokens":6057,"prompt_tokens":1000,"completion_tokens":5057,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":616,"completion_tokens_details":{"reasoning_tokens":4931}},"tokens_in":616,"tokens_out":5057,"duration_ms":37045,"temperature":1.0,"reasoning_tokens":4931,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:05:54.847431+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute GGA and GGA+U transitions for all 48 materials using the same supercell, least-squares, and Monte Carlo pipeline; if their mean absolute percentage errors on the full set fall close to the 22-23% reported for the meta-GGA functionals, the headline improvement over GGA and GGA+U would be an artifact of comparing different material sets.","supporting_citations":[{"cited_title":"Optimizing Supercell Structures for Heisenberg Exchange Interaction Calculations","cited_arxiv_id":"2410.14356","evidence_quote":"Generates the optimized supercells large enough to contain exchange interactions out to the required neighbor shell."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the all-electron code used to validate the plane-wave pseudopotential SCAN couplings for a subset of simple materials."}],"review_version":1}