{"id":"0a9f9671-30a7-43e3-adc7-a2b5eacfe110","arxiv_id":"2501.15825","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"The paper models which ties are missing in covert networks using exponential random graph models, then simulates different missingness mechanisms on six real networks to show their effects on estimates of network structure.","lead":"This paper treats missing data in covert criminal and terrorist networks as a hidden network in its own right, modeling which ties are unobserved with exponential random graph models. Simulating different missingness mechanisms on six real covert networks shows that common shortcuts, like assuming missing ties are absent, can distort conclusions about network structure.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1's parameter restriction θ≠0 ⇒ MNAR is false: an ERGM statistic can depend only on the observed part of X and still have θ≠0, so the central nesting claim is overstated.","rationale":"The reader's weakest assumption concerns the use of six empirical covert networks as true ground truth. That is a fair limitation and is explicitly disclosed in Section 6, but it is a standard simulation-device limitation rather than an internal inconsistency: the qualitative claims about bias under MNAR and zero imputation would likely survive replacement of the reference networks with other plausible structures. The more load-bearing issue is the paper's formal taxonomy, because the central claim in Section 1 and Table 1 is that ERGMs over the missingness indicator matrix nest MCAR, MAR, and MNAR through parameter restrictions. The two-dyad counterexample in the attack shows that θ≠0 does not imply MNAR: a term of the form d_ij(1-d_ik)x_ik uses only observed entries of X, so the resulting mechanism satisfies the paper's own MAR definition while having a nonzero network-interaction parameter. This matters substantively because the MAR/MNAR distinction determines whether the face-value likelihood in Equation (6) is valid; mislabelling MAR as MNAR would send users away from an appropriate estimator. The paper's own Section 5.2 MAR construction is effectively of this form, but it is never connected to Equation (4) or Table 1, and the simulation study only includes covariate-dependent MAR and self-censoring MNAR mechanisms. The omission is fixable, so the verdict remains conditional: the manuscript should revise the parameter-restriction table, allow f(D,X) terms that select on observed entries, and add a network-dependent MAR condition to the simulations. I do not see a basis for rejecting the broader framework, and the extensive simulation grid and transparent assumptions are genuine strengths, though the lack of public code remains an additional verification concern.","tokens_in":25902,"tokens_out":16323,"duration_ms":169892,"concrete_test":"Enumerate all D realizations for two dyads (ij,ik) in a three-node network and evaluate the exact ERGM P(D|X) ∝ exp(ψ(d_ij+d_ik) + θ[d_ij(1-d_ik)x_ik + d_ik(1-d_ij)x_ij]) for two X matrices that agree on the entries observed under a fixed D and differ only on the missing entry. If P(D|X) is identical for all such pairs, the mechanism is MAR with θ≠0, directly falsifying Table 1. As a corroborating check, simulate this mechanism on Network 4 and re-estimate the network model with the ergm package's MAR routine; if the estimator remains unbiased under this mechanism, the paper's classification of all θ≠0 as MNAR is empirically refuted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Equation (4) defines f(D,X) as any joint function of D and X, and Table 1 asserts θ≠0 ⇒ MNAR. This is not implied and is false. Consider an ERGM over two dyads with the statistic s(D,X)=d_ij(1-d_ik)x_ik + d_ik(1-d_ij)x_ij, where d and x are binary. For every realization of D, s depends on X only on dyads with d=0, i.e. only on the observed part of X. Hence P(D|X)=P(D|X_obs), which is MAR by the paper's own Section 5.2 definition, while θ≠0. This is not a remote construction: it is the same pattern as the paper's bivariate MAR example (Section 5.2, g10(x_ik,ψ)), yet that mechanism is never represented in Equation (4)/Table 1 and is not included in the simulation study, whose MNAR conditions use self-censoring entrainment (Table 3, d_ij x_ij). The consequence is that a user following Table 1 would label a genuinely MAR mechanism as MNAR, and would incorrectly conclude that the face-value likelihood of Equation (6)–(7) is unavailable. The Discussion claim that MAR cannot arise from dependence within the missingness mechanism is also false under this construction. The framework is repairable by allowing restricted f(D,X) terms that select observed entries, but as stated the central claim is unsupported.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes to model missing tie-variable indicators D in covert networks with an exponential random graph model, P(D|X) = exp(ψ'f(D) + β'f(D,C) + θ'f(D,X) - κ), arguing that this framework nests MCAR, MAR, and MNAR through parameter restrictions (Table 1). It then reports a Monte Carlo study that degrades six empirical covert networks under four missingness models (independent, latent-space, endogenous MCAR ERGM, and MNAR ERGM), re-estimates ERGMs under a MAR assumption ('Miss') or zero imputation ('Zero'), and compares failure rates and relative bias. A follow-up inspection varies the entrainment parameter θ1 and degree-covariate parameter θ2 to show their effects on density, centralization, and triadic closure estimates.","tokens_in":26215,"tokens_out":8445,"duration_ms":79137,"significance":"If the nesting claim were correct, this would be a useful framework for formalizing and simulating missingness mechanisms in network analysis, especially for covert networks whose observations are layered and biased. The paper's strengths include a clear mathematical formulation, the explicit use of an ERGM for the missingness indicator matrix, a simulation across six real covert networks, and practically relevant findings about the behavior of the 'ergm' package under missing data. However, the central classification result in Table 1 is not generally true: a nonzero θ can index a mechanism that depends only on the observed part of X and is therefore MAR, not MNAR. The simulation also treats ERGM fits to the (possibly missing) empirical networks as ground truth, and it excludes the most degenerate network from the bias plots. These issues make the paper's theoretical claims and some of its empirical conclusions (as stated) unsupported, though the framework appears repairable.","major_comments":[{"comment":"The classification in Table 1 (θ ≠ 0 ⇒ MNAR) is not implied by Eq. (4). For a two-dyad ERGM on D with statistic s(D,X) = d_ij(1−d_ik)x_ik + d_ik(1−d_ij)x_ij, the statistic is nonzero only on dyads whose X entries are observed (d = 0), so P(D|X) = P(D|X_obs) for every realization of D. This is MAR by the paper's own definition in Eq. (5), yet θ ≠ 0. This is not a remote construction: Section 5.2's bivariate MAR example g10(x_ik, ψ) has exactly this pattern, and that mechanism is never represented in Table 1 or in the simulation's 'MNAR' conditions (Table 3), which use self-censoring terms d_ij x_ij. Consequently, a user following Table 1 would label a genuinely MAR mechanism as MNAR and would incorrectly conclude that the face-value likelihood of Eqs. (6)–(7) is unavailable. The Discussion's claim (Section 8) that MAR cannot arise from dependence within the missingness mechanism is also false under this construction. The framework is repairable by restricting f(D,X) to statistics that select observed entries (or by redefining the Table 1 conditions), but as stated the central nesting claim is unsupported.","section":"§4.2–5.2, Eq. (4), Table 1"},{"comment":"The simulation treats the six empirical covert networks as the true networks (Section 6: 'We assume that the empirical covert networks are the true network') and the complete-data ERGM estimates as the true generative parameters. Since the paper's own introduction argues that covert network data are partial, censored, and multiply constructed, these reference networks are themselves likely to contain missingness. The relative-bias and failure-rate results therefore characterize the behavior of estimators relative to an assumed ground truth, not relative to the actual covert networks. The paper explicitly acknowledges the assumption, but the substantive conclusions in Section 8 ('the true network structures ... have the greatest effects on the relative biases') should be phrased as conditional on that assumption, with a discussion of how violations of the assumption would affect the reported magnitudes.","section":"§6, §7.2, Fig. 3"},{"comment":"Network 3 is excluded from all relative-bias plots (Figure 3 note: 'Network 3 is absent in these plots due to model estimation problems'). This is the network with the most severe estimation failure rates. The cross-network comparison of biases therefore omits the most extreme case, which is directly relevant to the paper's claim that true network structure drives bias patterns. The authors should either present the available partial information for Network 3 (e.g., estimates from conditions that did converge) or explicitly discuss how its exclusion affects each cross-network conclusion.","section":"§7.2, Fig. 3"},{"comment":"The paper does not ship code or data, despite the simulation being the paper's central evidence: Section 7 says code is 'available on request from the authors,' and Section 9 repeats 'available at request.' For a statistical methods paper, this is insufficient for reproducibility. Moreover, the simulation details are incomplete: the independent and latent-space missingness models listed in Section 6 are not fully specified (their parameter values and how the 10/35/60% missingness proportions are targeted are not given), and Table 3 provides parameter values only for the two ERGM missingness models. Without these details, the failure-rate and relative-bias results cannot be independently verified.","section":"§7, §9"}],"minor_comments":[{"comment":"The notation X_obs and X_mis is used before being formally defined; Section 5.2 defines X_obs = (X_ij : d_ij = 0) and X_mis = (X_ij : d_ij = 1), but this should appear earlier, at the first use in Section 3.","section":"§5.2"},{"comment":"The relative bias rBias = (η̃ − η)/η can be unstable when the baseline parameter η is close to zero (e.g., Table 2, Network 6, absdiff(Arrests) = −0.01). The paper should state how near-zero η values were handled or use an alternative metric.","section":"§7.2, Eq. (rBias)"},{"comment":"The independent and latent-space missingness models are mentioned as part of the four models in Section 6, but Figure 3's caption and text do not clearly identify which models are included in each panel. The caption should name all four missingness models and the subset of parameters shown.","section":"§6, §7.3"},{"comment":"The term 'entrainment' is used to describe the parameter θ1 but is never explicitly defined in the text; the paper should define it at first use, e.g., as the effect of the true tie variable x_ij on the missingness indicator d_ij.","section":"§7.3, Table 4"},{"comment":"The review of cognitive biases in crime investigation is lengthy and not directly used in the model or simulation; condensing it would sharpen the connection between the motivating examples and the formal framework.","section":"§2.1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within scope for stat.ME and network science, and the core idea of modeling missingness indicators with an ERGM is worth pursuing. The main obstruction is the incorrect MAR/MNAR classification in Table 1, which is fixable but requires reworking the definition of f(D,X) and the Discussion's claims. I would also encourage the editor to require a code repository and full simulation specifications before acceptance, as the paper's evidence is entirely simulation-based. The citation pattern appears appropriate and the paper engages with prior work on missing network data."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper before you read it. First, the core idea is genuinely useful: model the missingness indicator matrix D as an ERGM, so missingness can have endogenous dependence and can be conditioned on the true network. That is a real step beyond the ad hoc mechanisms used in the Smith–Moody–Morgan studies, and the simulation work is substantial. Second, the paper's central classification claim — Table 1, θ≠0 ⇒ MNAR — is simply wrong, and the flaw is not deep in the weeds; it is contradicted by the paper's own MAR example in Section 5.2.\n\nThe counterexample is the one the stress-test gives: take an ERGM for D with a statistic like d_ij(1−d_ik)x_ik + d_ik(1−d_ij)x_ij. For any realized D, this depends on X only on dyads with d=0, i.e. only on the observed part of the network. By the paper's own definition in Eq. (5), that is MAR, and yet the parameter multiplying it is θ ≠ 0. So the table is classifying a genuine MAR mechanism as MNAR. The fix is easy: allow f(D,X) statistics that select observed entries, and revise the table and the Discussion accordingly. But as it stands, a user following Table 1 will draw the wrong conclusion about when the face-value likelihood is available.\n\nWhat the paper does well: the general model in Eq. (4) is a clean way to think about missingness mechanisms, and the simulation study across six covert networks is thorough. The findings — that MNAR inflates variance and biases GWESP estimates, that the density parameter is surprisingly stable under entrainment, and that the degree-covariate effect has a phase transition around θ2=0 — are plausible and worth knowing. The authors are honest about the assumption that the empirical networks are the true networks, and they do not oversell their results.\n\nSoft spots, in proportion: the Table 1 problem is the major one and needs to be fixed before publication. Next is reproducibility — the paper says code and data are “available on request.” For a simulation study with 7200 degraded networks, that is not good enough; the code should be archived. Network 3 is absent from the relative-bias plots, which they acknowledge; that is a minor issue. The true-network assumption is a standard limitation, not a flaw.\n\nWho is this for: network methodologists and anyone who analyses covert or illicit networks and has to defend assumptions about missingness. It deserves serious refereeing — I would send it to review, with the expectation of a major revision centered on the missingness-assumption classification and code release.\n\nBest.","headline":"Modeling missingness indicators as an ERGM is a real step forward, but Table 1's classification is false and needs repair before the paper can be accepted.","tokens_in":26721,"tokens_out":4465,"would_cite":true,"duration_ms":40826,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The pattern of missing ties in covert network data is itself a network, and an ERGM fitted to the missingness indicators turns MCAR, MAR, and MNAR into parameter restrictions.","keywords":["covert networks","missing network data","missingness model","exponential random graph model","missing not at random","network simulation","social network analysis"],"falsifier":"Construct a fully observed network with known ground truth (for example, a complete communication log or a simulated network with a known generative process), generate missingness by a mechanism that depends only on observed information, such as sequential snowball sampling with all sampled nodes' ties observed, and fit an ERGM to the resulting missingness indicators. If an ERGM with endogenous dependence terms, like geometrically weighted degree or clustering, fits the missingness pattern substantially better than an independent model, the paper's claim that MAR cannot be motivated through endogenous missingness dependence would be falsified.","tokens_in":25686,"feed_emoji":"🕸️","tokens_out":12795,"duration_ms":107590,"temperature":0.7,"pith_summary":"Covert network data are missing in patterns shaped by secrecy, surveillance priorities, and investigative habits, not by chance. This paper proposes modelling the missingness itself: since the matrix of missingness indicators has the same binary, symmetric form as the network's adjacency matrix, an exponential random graph model (ERGM) can be fitted to the pattern of missing ties, with parameters for missing ties clustering together, dependence on observed attributes, and dependence on the true network. Inside this model, missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR) become special cases obtained by setting certain parameters to zero. Simulations that degrade six empirical covert networks show that edge-targeted missingness can severely bias estimated triadic closure, that standard errors inflate under MNAR, and that zero-imputing missing ties artificially shrinks uncertainty. The payoff for a reader is a way to make the missingness assumption explicit, to simulate realistic data-collection biases, and to move beyond the MAR assumption built into current estimation software.","feed_headline":"Modelling missing ties exposes bias in covert-network estimates","feed_subtitle":"A statistical model of which ties go missing turns the three missing-data assumptions into parameter choices.","key_machinery":"The carrying object is the missingness indicator matrix $D$, a binary symmetric matrix with $d_{ij}=1$ when the tie between $i$ and $j$ is unobserved; it occupies exactly the same sample space as the adjacency matrix $X$. The exponential random graph model (ERGM) is the machinery applied to this matrix: it is a probability distribution over binary networks in which the log-probability of a network is a linear combination of network statistics, $\\log \\Pr(Y) \\propto \\eta^\\top z(Y)$, with a normalising constant that makes the distribution proper. Chosen statistics $z(D)$ encode the assumed observational biases: edges for baseline missingness, geometrically weighted degree and clustering terms for missingness that propagates through shared nodes or triads, entrainment terms $d_{ij}x_{ij}$ for missingness that targets ties rather than nulls, and node or dyad covariates for observable drivers of invisibility. Because the same family is used for the data model and the missingness model, the paper can compare MCAR, MAR, and MNAR as parameter restrictions and simulate their consequences by sampling from the fitted missingness ERGM.","core_discovery":"The paper's central claim is that an exponential random graph model applied to the missingness indicator matrix $D$ gives a generative account of why ties in covert networks go missing. With $d_{ij}=1$ marking an unobserved tie, the model takes the form $\\Pr(D\\mid X,\\beta,\\psi,\\theta)=\\exp(\\psi^\\top f(D)+\\beta^\\top f(D,C)+\\theta^\\top f(D,X)-\\kappa(\\beta,\\psi,\\theta))$, where the $\\psi$ terms make missing ties depend on other missing ties, the $\\beta$ terms make them depend on measured attributes, and the $\\theta$ terms make them depend on the true network $X$. Setting $\\theta=0$ yields MCAR, $\\theta=0,\\beta\\neq0$ yields MAR, and $\\theta\\neq0$ yields MNAR, so the usual missingness taxonomy is a set of parameter restrictions on one model. The paper then shows, across 7,200 degraded versions of six covert networks, that assuming MAR when the true mechanism is MNAR attenuates estimated triadic clustering, inflates standard errors, and that treating missing ties as zeroes narrows uncertainty that should remain wide.","pith_inferences":["Editorial inference: the same missingness-model machinery applies outside covert settings; survey non-response, organisational rosters, and interaction logs all produce node-clustered missingness, and fitting an ERGM to $D$ would let researchers test that clustering directly rather than assume independence.","Editorial inference: the observed sharp transition in inferred centralisation as the degree-based missingness parameter crosses zero suggests there may be observation-effort thresholds that regime-shift the apparent network structure; a testable consequence is that moderate increases in surveillance could produce qualitative rather than gradual changes in estimated network metrics.","Editorial inference: because the simulations assume the six published covert networks are the true networks, the numerical bias magnitudes should be read as relative illustrations; converting them into operational error bounds would require ground-truth networks, for instance complete communication logs from a closed criminal case, degraded and re-estimated in the same way.","Editorial inference: one could use the missingness ERGM as a prior or penalty in estimation, treating the missingness parameters as sensitivity hyperparameters; this would give a practical route to estimating network models that are robust across a range of MNAR mechanisms rather than only under MAR."],"forward_implications":["When missing ties are set to zero before estimation, the analysis implicitly asserts that every missing tie is a null tie; the paper shows this narrows uncertainty and drives coefficients toward zero as missingness grows.","Under edge-targeted missingness (MNAR with positive entrainment), estimated triadic closure is biased downward even when estimation follows the standard MAR-based routine, so substantive claims about clustering in covert networks should be treated as sensitive to the missingness assumption.","A richer model specification with attribute covariates greatly lowers estimation failure rates for a given network, but it does not remove the bias introduced by the missingness mechanism; the structure of the true network is what drives the bias.","Because the missingness model is generative, it can be used before data collection to simulate the consequences of surveillance prioritisation, informant coverage, or record availability, and after data collection as a sensitivity-analysis tool.","The framework is designed to extend Bayesian data augmentation for partially observed networks beyond the MAR/ignorability assumption, which would allow MNAR mechanisms to be incorporated directly into estimation of the network-generating model."],"supporting_citations":[{"why":"This reference supplies the Markov dependence assumption that the paper extends from tie variables to the missingness indicators.","marker":"Frank & Strauss, 1986"},{"why":"This reference provides the alternating or geometrically weighted statistics used to specify stable ERGMs for both the data and the missingness model.","marker":"Snijders et al., 2006"},{"why":"This reference establishes the face-value likelihood and MAR estimation framework that the simulation uses when missingness is represented as missing.","marker":"Handcock & Gile, 2010"},{"why":"This reference supplies the MCAR/MAR/MNAR taxonomy and ignorability conditions that the paper translates into ERGM parameter restrictions.","marker":"Little and Rubin, 1987"},{"why":"This reference provides the Bayesian data-augmentation approach for partially observed networks that the paper identifies as the route to incorporating non-ignorable missingness.","marker":"Koskinen et al., 2013"},{"why":"This reference defines the null-tie imputation procedures that the paper contrasts with explicit missingness models.","marker":"Huisman, 2009"},{"why":"This reference is the earlier Monte Carlo study of missing network data whose simulation design the paper replicates and extends.","marker":"Kossinets, 2006"},{"why":"This reference simulates non-random missing data and network measurement effects, providing the prior work that the paper's MNAR modelling builds on.","marker":"Smith et al., 2017"},{"why":"This reference documents ERGM degeneracy problems that the paper draws on to interpret estimation failure rates.","marker":"Handcock, 2003"}],"fun_headline_variants":["Silent ties skew covert network estimates","Model missing ties or misread crime networks","Missingness modeling uncovers bias in dark nets","Why hidden gaps in dark net data matter"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's load-bearing premise is that the six empirical covert networks are the true networks and that the ERGM fitted to each complete network gives the true generative parameters; if those public networks are themselves partial or biased constructions, the simulated biases and failure rates describe the models' behaviour relative to an assumed ground truth rather than to real covert networks.","fun_headline_variants_meta":{"raw":{"variants":["Silent ties skew covert network estimates","Model missing ties or misread crime networks","Missingness modeling uncovers bias in dark nets","Why hidden gaps in dark net data matter"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00074,"raw_usage":{"total_tokens":3267,"prompt_tokens":874,"completion_tokens":2393,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":490,"completion_tokens_details":{"reasoning_tokens":2338}},"tokens_in":490,"tokens_out":2393,"duration_ms":21590,"temperature":1.0,"reasoning_tokens":2338,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T13:54:36.327816+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a fully observed network with known ground truth (for example, a complete communication log or a simulated network with a known generative process), generate missingness by a mechanism that depends only on observed information, such as sequential snowball sampling with all sampled nodes' ties observed, and fit an ERGM to the resulting missingness indicators. If an ERGM with endogenous dependence terms, like geometrically weighted degree or clustering, fits the missingness pattern substantially better than an independent model, the paper's claim that MAR cannot be motivated through endogenous missingness dependence would be falsified.","supporting_citations":[],"review_version":1}