{"id":"95d2859a-c3cb-45ec-ac21-52a3182c6529","arxiv_id":"2606.19775","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":1,"one_line_summary":"β-sampling experiments on 45 networks show that the structural properties of missing links and the choice of validation-set sampling strategy substantially affect link prediction accuracy and differ from random or consistent sampling.","lead":"This paper studies how the choice of which links to hide as the probe set and validation set affects the measured accuracy of link prediction algorithms on incomplete networks. It introduces β-sampling and reports that both the structure of the hidden links and the second-stage sampling method change performance substantially across 45 real networks.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"β-sampling may entangle intended degree-product variation with correlated structural confounders across the 45 networks","rationale":"The reader's weakest_assumption directly identifies the isolation problem that the empirical design must solve; the abstract-only review correctly left the verdict open because that isolation is not demonstrated. Full-text details on controls or auxiliary statistics would be needed to close the gap; absent them the concern remains load-bearing for the headline claim.","tokens_in":1769,"tokens_out":328,"duration_ms":10968,"concrete_test":"For each of the 45 networks and each tested β, compute the average local clustering coefficient and average edge betweenness of the probe-set links; if these quantities trend monotonically with β, re-run the link-prediction experiments on probe sets that are explicitly matched on both degree-product and clustering (via stratified or propensity-score sampling) and test whether the accuracy-vs-β relationship survives.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central empirical claim requires that probe sets generated by p ∝ (d_u d_v)^β differ cleanly in the structural property of interest (degree-product of missing links) while other factors remain comparable. In real networks, high-(d_u d_v) links systematically co-vary with clustering, betweenness, or community membership; nothing in the described procedure (or the abstract) indicates explicit matching or regression controls for these. Consequently, accuracy differences attributed to “structural characteristics of missing links” could instead trace to these uncontrolled covariates. The second-stage sampling result inherits the same risk once the probe set itself is not isolated.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper proposes a β-sampling scheme for dividing observed links into training, validation, and probe sets in link prediction, with sampling probability p ∝ (d_u d_v)^β. Experiments on 45 real-world networks are used to claim that structural characteristics of missing links (varied via probe sets) substantially affect prediction accuracy, that links between high-degree nodes are easier to predict, and that second-stage sampling for the validation set exerts significant influence even with fixed probe sets, with optimal strategies differing from random and consistent sampling.","tokens_in":1899,"tokens_out":517,"duration_ms":25238,"significance":"If the empirical effects can be isolated from confounders, the work would demonstrate that standard evaluation practices in link prediction are sensitive to sampling choices and could motivate more careful benchmark design. The scale of 45 networks provides breadth, but the absence of statistical controls and the risk of entangled structural factors limit the strength of the conclusions.","major_comments":[{"comment":"Abstract: the claim that 'structural characteristics of missing links, as simulated via varying probe sets, substantially impact prediction accuracy' lacks any reported statistical details, error bars, dataset list, p-values, or controls for multiple testing across 45 networks, making the 'substantial impact' assertion unverifiable from the given evidence.","section":"Abstract"},{"comment":"β-sampling definition and experimental setup: varying the single parameter β in p ∝ (d_u d_v)^β does not demonstrably isolate the intended degree-product property, because high-(d_u d_v) links systematically co-vary with clustering, betweenness, and community structure in real networks; no matching, stratification, or regression controls are described to rule out these confounders.","section":"Method / Experimental Setup"},{"comment":"Results on second-stage sampling: the claim that 'even with a fixed probe set, second-stage sampling still exerts a significant influence on prediction accuracy' inherits the same isolation problem once the probe set itself is generated by β-sampling, and no explicit test is reported showing independence from the first-stage structural variation.","section":"Results"}],"minor_comments":[{"comment":"Abstract: the definitions of 'random sampling' and 'consistent sampling' are given inline but would benefit from explicit mathematical notation when first introduced in the main text.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the detailed and constructive comments, which help clarify the need for greater statistical transparency and discussion of potential confounders. We respond to each major comment below and indicate the revisions that will be incorporated.","responses":[{"response":"We agree that the abstract would be strengthened by supporting statistical information. In the revised manuscript we will add error bars to all reported accuracy metrics, include a table summarizing the 45 networks, and report mean accuracy differences with standard deviations across networks to quantify the impact. We will also note the consistency of trends rather than relying on per-network p-values, thereby addressing multiple-testing concerns while keeping the focus on the observed empirical patterns.","revision_made":"yes","referee_comment":"[Abstract] Abstract: the claim that 'structural characteristics of missing links, as simulated via varying probe sets, substantially impact prediction accuracy' lacks any reported statistical details, error bars, dataset list, p-values, or controls for multiple testing across 45 networks, making the 'substantial impact' assertion unverifiable from the given evidence."},{"response":"We acknowledge that degree-product correlates with other structural features in real networks and that β-sampling does not fully isolate the degree-product effect. Our contribution is empirical: systematically varying the degree-product distribution of missing links via β produces consistent changes in prediction accuracy across 45 networks. We will add an explicit discussion of potential confounders and note that future work could employ matching or regression controls. The current design prioritizes breadth over synthetic isolation.","revision_made":"partial","referee_comment":"[Method / Experimental Setup] β-sampling definition and experimental setup: varying the single parameter β in p ∝ (d_u d_v)^β does not demonstrably isolate the intended degree-product property, because high-(d_u d_v) links systematically co-vary with clustering, betweenness, and community structure in real networks; no matching, stratification, or regression controls are described to rule out these confounders."},{"response":"The second-stage experiments already hold the probe set fixed while varying only the validation-set sampling strategy. To further demonstrate robustness, we will add results for the same second-stage variations applied to probe sets generated under multiple distinct β values and report that the influence of second-stage sampling persists. This clarification and additional figures will be included in the revised results section.","revision_made":"yes","referee_comment":"[Results] Results on second-stage sampling: the claim that 'even with a fixed probe set, second-stage sampling still exerts a significant influence on prediction accuracy' inherits the same isolation problem once the probe set itself is generated by β-sampling, and no explicit test is reported showing independence from the first-stage structural variation."}],"tokens_in":1445,"tokens_out":587,"duration_ms":28574,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper's central observation is that the way you pick the probe set and then the validation set changes measured accuracy, and their β-sampling lets you tune the degree product of the held-out links. On 45 networks they report that probe sets biased toward high-degree-product edges are easier to predict and that the best second-stage sampling rule is neither random nor the consistent-sampling baseline. That is the actual new piece: a simple parametric knob and an empirical demonstration that the two sampling stages are not interchangeable.\n\nThe work is useful for anyone who runs link-prediction benchmarks, because it shows that standard random splits can hide or exaggerate performance differences. The 45-network scale is respectable and the distinction between random and consistent sampling is worth checking.\n\nThe soft spot is the lack of any reported controls or diagnostics for the obvious confounders. In real networks, degree-product correlates with clustering, community structure, and betweenness; nothing in the abstract indicates they matched or regressed those out. If the accuracy gaps shrink once those are held fixed, the claim that “structural characteristics of missing links” drive the result becomes harder to sustain. The abstract also gives no error bars, no multiple-testing correction, and no list of the networks, so the size and reliability of the reported effects cannot be judged from what is shown.\n\nThis is the kind of methodological note that belongs in a methods-focused venue or as a short note in a network-science journal. A serious referee should see it to verify the controls and the statistical reporting, but it is not a foundational result. I would bring it to a reading group for the sampling discussion and would not cite it in my own work unless the full experiments hold up after the confounder check.","headline":"β-sampling highlights that probe-set and validation-set choices affect link-prediction scores, but the experiments need tighter controls before the degree-product claim can be taken at face value.","tokens_in":2379,"tokens_out":428,"would_cite":false,"duration_ms":16436,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"The structural features of missing links, set by how probe sets are sampled, control link prediction accuracy.","keywords":["link prediction","sampling strategy","probe set","validation set","network incompleteness","degree bias","β-sampling"],"falsifier":"If prediction accuracy shows no systematic change when β is varied across a fresh collection of networks, or if second-stage sampling effects disappear under controlled tests that hold all other factors fixed.","tokens_in":2663,"feed_emoji":"🔗","tokens_out":588,"duration_ms":17587,"temperature":0.7,"pith_summary":"The paper examines two sampling steps when preparing data for link prediction: first choosing the probe set of missing links, then choosing a validation set from the rest. It introduces β-sampling, which sets the chance of picking a link proportional to the product of its endpoints' degrees raised to β. Tests on 45 networks show that probe sets biased toward high-degree connections yield higher prediction scores. With the probe set held fixed, the second sampling step for validation still changes results, and the best choice differs from both random selection and from matching the probe set's structure.","feed_headline":"Degree-biased probe sampling changes link prediction accuracy","feed_subtitle":"High-degree missing links prove easier to predict; validation sampling exerts its own separate effect on 45 networks.","key_machinery":"β-sampling, a scheme that draws each observed link into the probe set with probability proportional to (degree_u × degree_v)^β, which tunes the degree bias of the simulated missing links.","core_discovery":"Using β-sampling to vary the probe set shows that prediction accuracy rises when missing links preferentially connect high-degree nodes. Even with a fixed probe set, different second-stage sampling strategies for the validation set produce measurably different accuracies, and the strategy that works best is neither random sampling nor consistent sampling that matches structural traits.","pith_inferences":["Real missing links in networks may follow particular degree patterns, so results could differ when those patterns are known.","Algorithms might be tuned or selected according to the expected β value of the missing data.","Benchmark suites could usefully include multiple probe sets generated at different β values."],"forward_implications":["Prediction performance improves when missing links tend to join high-degree nodes.","Even a fixed probe set leaves room for second-stage sampling to shift measured accuracy.","The best validation-set sampling rule is not the same as random selection or consistent structural matching.","Evaluation protocols must account for both sampling stages rather than treating them as neutral."],"fun_headline_variants":["β-sampling ties prediction to high degree node connections","Validation sampling impacts accuracy beyond the probe set","Probe sampling by β affects link prediction accuracy","Different second stage sampling produces different accuracies"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"Changing the single parameter β cleanly alters only the structural properties of the probe set without introducing other confounding differences across the 45 networks.","fun_headline_variants_meta":{"raw":{"variants":["β-sampling ties prediction to high degree node connections","Validation sampling impacts accuracy beyond the probe set","Probe sampling by β affects link prediction accuracy","Different second stage sampling produces different accuracies"]},"model":"grok-4.3","cost_usd":0.012465,"raw_usage":{"total_tokens":5418,"prompt_tokens":649,"num_sources_used":0,"completion_tokens":54,"cost_in_usd_ticks":124649500,"prompt_tokens_details":{"text_tokens":649,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":4715,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":649,"tokens_out":54,"duration_ms":38895,"temperature":1.0,"reasoning_tokens":4715,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-26T15:30:58.958620+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"If prediction accuracy shows no systematic change when β is varied across a fresh collection of networks, or if second-stage sampling effects disappear under controlled tests that hold all other factors fixed.","supporting_citations":[],"review_version":1}