{"id":"4b54f556-ab8e-4e5e-b93b-3fb9bbf9705d","arxiv_id":"2506.21426","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Using a random-network baseline, the study shows that Budapest firms' systemic risk fell below null-model expectations after COVID-19, with importers and exporters affecting local risk in opposite ways.","lead":"This paper tracks how much damage each company in Budapest's supplier network could cause if it failed, year by year from 2015 to 2022. It finds that after the COVID-19 pandemic, the network became less fragile than a random baseline predicts, and that imports and exports pull local systemic risk in opposite directions.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Post-2020 empirical-null ESRI gap may stem from the s-GM's known upward bias and the 2018 coverage break, not from adaptive rewiring; the paper does not quantify the bias's contribution.","rationale":"The reader's weakest assumption points to the null model's upward bias and the 2018 data break as confounds for the post-2020 divergence; this is exactly the load-bearing issue I identify. The paper's own text supports the concern: it notes the model overestimates ESRI for the most risky firms, that the model-empirical gap appears from 2018 onward, and that the deviation increases in 2020-2022. The central claim of adaptive rewiring requires that the post-2020 gap is not simply an extrapolation of a coverage/composition-driven bias. My proposed balanced-panel test would settle this. The descriptive findings about the evolution of ESRI and the regression determinants remain interesting and are not invalidated by this concern, so the CONDITIONAL verdict remains appropriate; the conditions should explicitly include this counterfactual check.","tokens_in":23250,"tokens_out":3254,"duration_ms":36047,"concrete_test":"Recompute Fig. 3C on the balanced panel of firms present in both 2015-2017 and 2020-2022 (or, failing that, restrict the 2020-2022 sample to firms satisfying the pre-2018 employee/customer filters used before the RTIR coverage expansion). If the post-2020 empirical-minus-null ESRI gap shrinks to the level of the 2018-2019 baseline gap, the reported divergence is explained by the 2018 data-coverage break combined with the s-GM's known upward bias, not by adaptive rewiring.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that empirical ESRI aligns with the null until 2020 and then becomes significantly lower, reflecting adaptive rewiring (Fig. 3C). However, the divergence begins in 2018, exactly when RTIR/VAT reporting changes introduce many small firms into the sample, and the authors state that the s-GM 'overestimates ESRI of the most risky firms' (Fig. 3B) and that 'from 2018 onward, the model ESRI becomes significantly higher than the real one', with the deviation increasing in 2020-2022. Because the s-GM preserves strengths but randomizes topology, its upward bias for high-risk firms grows with the number of low-connectivity small firms that become disconnected in model samples (ref. 62); the 2018-2022 samples contain far more such firms. Thus the post-2020 gap may be a continuation of a bias that is already present in 2018-2019 under normal conditions, driven by data-coverage and composition changes rather than by pandemic-era rewiring. The paper does not quantify the contribution of this bias to the mean gap, nor does it compare the 2020-2022 gap against an extrapolation of the 2018-2019 bias trend. The regression includes the null ESRI as a covariate, but that controls for cross-sectional levels, not for the year-specific mean shift that drives Fig. 3C. Without a counterfactual that holds firm composition and coverage fixed, the attribution 'reflecting the emergence of a more resilient economy driven by firms' adaptive behavior' is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies the evolution of firm-level systemic risk (ESRI) in the Budapest production network from 2015 to 2022, using VAT transaction data. The authors compute ESRI for each firm and benchmark the empirical values against an ensemble of stripe-corrected gravity model (s-GM) null networks that preserve each firm's sector-level total input and output flows. They report that the set of highest-risk firms changes during COVID-19, that the empirical mean ESRI aligns with the null until 2020 but becomes significantly lower afterwards, and that international trade becomes a significant predictor of firm-level ESRI during the pandemic. Regression analysis with firm fixed effects and year dummies is used to relate ESRI to null-model ESRI, local trade strengths, essentiality, and import/export volumes.","tokens_in":23579,"tokens_out":3500,"duration_ms":38816,"significance":"If the central claim is correct, the paper would provide one of the first longitudinal accounts of how a production network's systemic risk responds to a major shock through adaptive rewiring. The dataset is rich, the temporal span is unusual, and the use of a constrained null model to benchmark empirical ESRI is methodologically appropriate in principle. The paper also conducts extensive robustness checks (pooled OLS, cross-sectional regressions, supplementary analyses) and is transparent about computational limitations. However, the main empirical-null comparison is vulnerable to a known bias in the null model and a concurrent data-coverage break, and the regression design does not fully separate the COVID-specific shift from earlier level changes. These issues are central to the paper's headline claim, so the current evidence is insufficient to establish the adaptive-rewiring narrative.","major_comments":[{"comment":"The central claim that empirical ESRI becomes significantly smaller than the null after 2020 is undermined by the 2018 data-coverage break and the known upward bias of the s-GM for high-risk firms. The authors themselves note that the model 'overestimates ESRI of the most risky firms' (Fig. 3B) and that 'from 2018 onward, the model ESRI becomes significantly higher than the real one.' The 2018 jump coincides with the RTIR/VAT reporting changes that introduce many small firms, and the s-GM's bias grows when low-connectivity small firms become disconnected in model samples (ref. 62). The paper does not quantify how much of the 2020-2022 gap is explained by this bias or compare the pandemic-period gap against an extrapolation of the 2018-2019 bias trend. Without such a counterfactual, the attribution of the divergence to adaptive rewiring is unsupported.","section":"Empirical vs null model values of ESRI (Fig. 3C)"},{"comment":"The regression analysis includes ESRImodel as a covariate, which controls for cross-sectional levels but not for the year-specific mean shift that drives Figure 3C. The pooled OLS regressions in Supplementary S16 show year dummies that decline monotonically from 2018 onward (-0.027 in 2018, -0.041 in 2019, -0.043 in 2020, -0.047 in 2021, -0.053 in 2022), meaning the empirical-null gap is already present before the pandemic. The interpretation that the import/export coefficients are COVID-specific therefore relies on a before/after split that is not cleanly identified by the data, because the post-2018 sample composition changes confound the pandemic effect.","section":"Regression framework (Table 1; Supplementary S16)"},{"comment":"The s-GM preserves, on average, each firm's total out-strength and in-strength by sector, and the ESRI definition (Eq. 1) weights firms' output reductions by their out-strengths. The pre-2020 agreement between empirical and null mean ESRI is therefore partly a calibration artifact: the null model is built to reproduce the very strength sequences that dominate ESRI. This does not invalidate the null-model comparison, but it means the normal-times alignment cannot serve as independent evidence that the s-GM is an unbiased benchmark during the crisis. To support the adaptive-rewiring claim, the authors should report an analysis that holds firm composition fixed (for example by reweighting or subsetting the 2018-2022 samples to match the 2015-2017 firm distribution) and show that the divergence persists.","section":"Methods, Eq. (4) and Eq. (1)"}],"minor_comments":[{"comment":"The 2017 pre-filtering transaction count (25,494) is an order of magnitude smaller than the 2016 (225,165) and 2018 (1,373,207) values; this appears to be a typo and should be corrected or explained.","section":"Table 2"},{"comment":"The caption refers to a shaded area corresponding to standard deviations but does not specify whether the shading is shown for both empirical and null series; please clarify in the figure or legend.","section":"Figure 3C"},{"comment":"The text says that upstream ESRI shows 'large positive variations' starting from 2020, but earlier in the same paragraph it states that the plateau shape remains consistent until 2019; the transition from stability to step-like structure is described only verbally and would benefit from a quantitative measure of the variation.","section":"Results, 'Sector composition of ESRI plateaux'"},{"comment":"Reference 62 is cited for the claim that small firms may become disconnected in s-GM samples, but that paper addresses critical density for network reconstruction more generally; citing a more directly relevant source or expanding the explanation would help the reader.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and important question, and the dataset is a clear strength. The main barrier is not the method per se but the identification: the 2018 coverage break and the s-GM's known bias are acknowledged in the text, yet the paper proceeds to interpret the resulting gap as evidence of adaptive rewiring without quantifying these confounds. I would encourage the editor to request a revised version that explicitly addresses this with a counterfactual or a bias-quantification exercise; the current version's headline claim is not yet supported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hi [Name],\n\nShort version: this is a careful, useful empirical study of firm-level systemic risk in Budapest's VAT network from 2015 to 2022. The descriptive findings are solid — the plateau of riskiest firms changes character during COVID, the postal sector rises to prominence, and import/export become significant predictors of ESRI only in the crisis years. That's genuinely new for a European capital. The authors also deserve credit for using the s-GM null model and for being upfront about its limitations and the 2018 data break.\n\nThe trouble is that the paper's main interpretive claim — that the post-2020 drop in empirical ESRI relative to the null reflects adaptive rewiring and a more resilient economy — is not supported by the evidence as presented. The gap between empirical and model ESRI starts in 2018, exactly when the RTIR reporting change brings many small, low-connectivity firms into the sample. The authors themselves note the s-GM overestimates ESRI for the riskiest firms because small firms get disconnected in the model ensemble (ref 62). That bias grows as more small firms appear, so the 2020-2022 widening may be a continuation of a composition effect, not a pandemic response. The paper never quantifies how much of the gap the known bias explains, nor does it compare the crisis divergence against an extrapolation of the 2018-2019 trend. The regression includes the null ESRI as a covariate, but that controls for cross-sectional level, not the year-specific mean shift that drives Figure 3C.\n\nThere's also a milder circularity concern: the s-GM constrains sector-level degree and strength sequences, which are exactly the quantities that ESRI depends on, so the good normal-times match is partly built in. That doesn't sink the descriptive results, but it does mean the null comparison is a weaker benchmark than the framing suggests.\n\nNone of this invalidates the paper. The plateau composition shift and the trade regression results stand on their own. But the \"adaptive rewiring\" rhetoric goes beyond what the data can bear without a counterfactual that holds firm composition and coverage fixed, or at least a sensitivity analysis removing the 2018-2019 bias trend.\n\nWorth a serious referee — it's a solid dataset, novel time span, and the questions matter. I'd send it out, but with a clear request to quantify the bias, soften the causal language, and make the code/data available for reproducibility.\n\nBest,\n[Your name]","headline":"Solid descriptive study of Budapest production networks; the headline resilience-rewiring claim outruns the evidence because the empirical-null gap starts in 2018, before COVID.","tokens_in":24138,"tokens_out":3027,"would_cite":false,"duration_ms":32229,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":["89.65.Gh"],"model":"deepseek-v4-flash","headline":"Using yearly snapshots of the Budapest production network, this paper argues that firms' rewiring of supply links during COVID-19 made the economy more resilient than a randomized network with the same firm-level constraints would predict.","keywords":["systemic risk","production networks","firm-level data","null models","maximum entropy","COVID-19","supply chain rewiring","international trade"],"falsifier":"Re-run the 2020–2022 comparison after removing the null model's known upward bias for the most connected firms, for instance by restricting the comparison to firms matched on size and sector or by explicitly subtracting the bias, and see whether empirical systemic risk still falls significantly below the corrected benchmark; if the gap disappears, the adaptive-rewiring interpretation is falsified. A complementary check measures actual supplier and customer turnover per firm in the pandemic years and asks whether the firms whose risk fell most are the ones whose link portfolios demonstrably changed.","tokens_in":23042,"feed_emoji":"🔗","tokens_out":13217,"duration_ms":132582,"temperature":0.7,"pith_summary":"The paper asks whether firms can make an economy safer by actively reconfiguring their supply relationships during a crisis, and measures this in the Budapest production network year by year from 2015 to 2022. Each firm carries a systemic-risk score: the total output the local economy would lose if that firm failed, computed by simulating the propagation of its failure through suppliers and customers. The paper compares these empirical scores against a maximum-entropy null model that randomizes the network while preserving every firm's sector-level inputs and outputs, a stand-in for a market that adapts no further than its production structure dictates. Empirical risk tracks the null benchmark closely until 2020, then drops significantly below it even as the benchmark rises, which the authors read as evidence that firms rewired their links under pandemic pressure into configurations that spread damage less. A sympathetic reader would care because the result suggests resilience is something firms actively produce in a crisis, not just a fixed property of the network, and because the gap between real and randomized networks gives a generic way to spot that adaptation.","feed_headline":"COVID-era rewiring cut systemic risk below random-network levels","feed_subtitle":"The real network became safer than randomized ones in 2020-2022, a sign firms were actively rewiring.","key_machinery":"Two objects carry the argument. The first is the Economic Systemic Risk Index (ESRI), which assigns each firm the output-weighted total production loss of the network when that firm fails: $\\mathrm{ESRI}_i = \\sum_j \\frac{s^{\\mathrm{out}}_j}{\\sum_l s^{\\mathrm{out}}_l}\\left[1 - h_j(n^*)\\right]$, where $h_j(n^*)$ is the fraction of original production firm $j$ can still maintain after upstream and downstream shocks have iterated to a stable state under a generalized Leontief production function. The second is the stripe-corrected gravity model (s-GM), a heuristic maximum-entropy null model that generates an ensemble of randomized networks (100 per year) while preserving each firm's total output and its input quantities by sector; the ensemble incarnates a Walrasian-equilibrium assumption in which agents care only about final allocations, not the network that realizes them. The paper's detection strategy is to compare each year's empirical ESRI distribution against this equilibrium benchmark, and then to regress empirical ESRI on the null-model value plus local-trade and international-trade variables with firm fixed effects, so that the regression residuals reveal which firm attributes carry the part of systemic risk the null model cannot explain.","core_discovery":"The paper's central claim is that firm-level systemic risk in the Hungarian production network behaves like the random-network benchmark during normal times and then becomes significantly smaller than it from 2020 onward, even though the benchmark itself keeps rising. Because the null model already respects each firm's production structure, total sales and sector-level purchases are preserved, this divergence cannot be explained by the growth of firms or the entry of small firms alone; the authors attribute it to adaptive behavior, namely firms finding alternative suppliers and customers under pandemic restrictions and import bans, and thereby damping how far a failure would propagate. Accompanying the divergence is a structural shift in the composition of the riskiest firms: the plateau of top-0.1% ESRI firms becomes dominated by firms that enable exchange itself, most prominently postal services, a reconfiguration the null model does not reproduce. The regression analysis rounds out the picture, showing that international trade volumes, insignificant before 2020, become strong predictors of firm-level systemic risk during the crisis, with imports and exports exerting opposing effects through the supply and demand channels respectively.","pith_inferences":["A direct test the paper motivates but does not run: measure supplier and customer turnover for each firm in 2020–2022 and verify that the firms whose ESRI fell most relative to the null model are the same firms whose link portfolios demonstrably changed; that would make the rewiring mechanism observable rather than inferred.","The adaptive-rewiring reading predicts a cross-country gradient: economies whose pandemic restrictions most severely constrained imports and exports should show the largest empirical-versus-null ESRI gaps, since forced import substitution would drive more rewiring.","Part of the post-2020 gap could be mechanical rather than adaptive: the null model is known to overestimate the risk of the most connected firms, and the 2018 VAT reporting reform changed which firms appear in the data, so quantifying how much of the divergence survives those corrections would sharpen or shrink the rewiring conclusion.","The paper's framing suggests a monitoring design consequence: after a crisis rewires a network, the newly central exchange-enabling firms become single points of failure, so resilience monitoring should follow the post-shock network rather than the pre-crisis list of high-risk firms."],"forward_implications":["Systemic risk is time-dependent: the pandemic years show that a firm's danger to the economy can be reduced by its own rewiring, so pre-crisis network measurements alone will overstate post-crisis risk.","The gap between an empirical network and its sector-constrained random benchmark becomes a working indicator of adaptation: when observed risk falls below the null expectation, purposeful restructuring rather than random churn is the likely cause.","The identity of the most dangerous firms can change abruptly in a crisis: postal services and other exchange-enabling sectors rising into the top-risk plateau means crisis monitoring should track the connectors, not just the traditional giants.","International trade affects local systemic risk mainly when trade itself is disrupted: import and export volumes become significant predictors of firm risk during COVID-19, with imports complementing domestic supply and exports substituting for local revenue pushing risk in opposite directions.","If the divergence is real, average-risk statistics understate the story: the economically meaningful signal is risk relative to the equilibrium benchmark, which fell even while the absolute number of firms and transactions kept growing."],"supporting_citations":[{"why":"Defines the Economic Systemic Risk Index and its shock-propagation algorithm, the measure of firm-level risk the whole paper relies on.","marker":"[36]"},{"why":"Introduces the stripe-corrected gravity model, the maximum-entropy null model that produces the randomized benchmark networks.","marker":"[54]"},{"why":"Shows the s-GM reproduces individual-firm ESRI values in normal times, the prior result that justifies using it as the pre-crisis benchmark.","marker":"[55]"},{"why":"Supplies the essentiality matrix of products and the Leontief-style shock-propagation setup used inside the ESRI computation.","marker":"[17]"},{"why":"Provides the exponential random graph / maximum entropy formalism that gives the null model its status as a rigorous null hypothesis.","marker":"[47]"},{"why":"Argues that supply-chain rewiring mitigates systemic risk, the mechanism the paper invokes to explain the post-2020 divergence.","marker":"[66]"},{"why":"Documents the Hungarian firm-to-firm VAT transaction dataset from which the yearly Budapest production networks are built.","marker":"[58]"}],"fun_headline_variants":["Real supply networks beat random ones during COVID","Firm rewiring made Hungary's network safer than random","Post-2020 risk fell below random as firms adapted","Imports and exports tug systemic risk in opposite directions","Pandemic rewiring reshuffled riskiest firms"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that the post-2020 drop in systemic risk comes from firms' adaptive rewiring depends on treating the randomized null model as a fair benchmark: if that model's known tendency to overestimate the risk of the biggest firms, or the 2018 VAT reporting change that expanded the dataset, explains the gap instead, the rewiring conclusion loses its footing.","fun_headline_variants_meta":{"raw":{"variants":["Real supply networks beat random ones during COVID","Firm rewiring made Hungary's network safer than random","Post-2020 risk fell below random as firms adapted","Imports and exports tug systemic risk in opposite directions","Pandemic rewiring reshuffled riskiest firms"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000227,"raw_usage":{"total_tokens":1505,"prompt_tokens":1010,"completion_tokens":495,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":626,"completion_tokens_details":{"reasoning_tokens":433}},"tokens_in":626,"tokens_out":495,"duration_ms":6100,"temperature":1.0,"reasoning_tokens":433,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:26:50.158938+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the 2020–2022 comparison after removing the null model's known upward bias for the most connected firms, for instance by restricting the comparison to firms matched on size and sector or by explicitly subtracting the bias, and see whether empirical systemic risk still falls significantly below the corrected benchmark; if the gap disappears, the adaptive-rewiring interpretation is falsified. A complementary check measures actual supplier and customer turnover per firm in the pandemic years and asks whether the firms whose risk fell most are the ones whose link portfolios demonstrably changed.","supporting_citations":[{"cited_title":"Inferring firm-level supply chain networks with realistic systemic risk from industry sector-level data","cited_arxiv_id":"2408.02467","evidence_quote":"Shows the s-GM reproduces individual-firm ESRI values in normal times, the prior result that justifies using it as the pre-crisis benchmark."},{"cited_title":"Production networks and epidemic spreading: How to restart the UK economy?","cited_arxiv_id":"2005.10585","evidence_quote":"Supplies the essentiality matrix of products and the Leontief-style shock-propagation setup used inside the ESRI computation."},{"cited_title":"Systemic risk mitigation in supply chains through network rewiring","cited_arxiv_id":"2504.12955","evidence_quote":"Argues that supply-chain rewiring mitigates systemic risk, the mechanism the paper invokes to explain the post-2020 divergence."},{"cited_title":"& Stancsics, M","cited_arxiv_id":null,"evidence_quote":"Documents the Hungarian firm-to-firm VAT transaction dataset from which the yearly Budapest production networks are built."}],"review_version":1}