{"id":"fa7881fb-ef0d-4d09-9322-127fe645afc9","arxiv_id":"2411.16145","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Per-snapshot NC-LID negatively correlates with dynamic graph embedding quality for most of 10 real-world networks, identifying nodes with weak embeddings.","lead":"Graphs that change over time, like email networks or school contact patterns, are converted into numeric embeddings for machine learning. This paper shows that a measure called NC-LID, computed on each time snapshot, tends to flag nodes whose embeddings are poor, a first step toward smarter dynamic graph embeddings.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Pooled node-snapshot observations violate MWU independence and inflate significance; per-snapshot NC-LID also ignores temporal structure, so the headline correlation may be an artifact of pooling.","rationale":"The paper is a focused empirical study and the contribution is clearly a signal rather than a method. The evidence is consistent negative correlations and MWU differences on pooled node-snapshot observations, but the statistical design has a serious flaw: observations are not independent because the same node contributes to many snapshots. This inflates the effective sample size and makes both the Spearman p-values and MWU p-values unreliable as evidence for the claim. The reader identified this exact weakness in Section 3 and Section 4, so I agree. I also note the post-hoc exclusion of ia-realitycall, where F1 is essentially random (F1=0.0362) and the correlation direction is positive, meaning the exclusion is motivated by the very property being studied; this deserves a sensitivity analysis rather than dismissal. Because the paper does not yet build a LID-aware method, the practical claim is modest, but the empirical support needs to be re-established under a correct statistical treatment. CONDITIONAL is the appropriate verdict: the paper should be accepted only after the pooled-analysis concern is addressed, either by node-level aggregation or by a mixed-effects model accounting for node-level dependence.","tokens_in":8169,"tokens_out":1465,"duration_ms":13208,"concrete_test":"For each network, compute node-level F1 and NC-LID only from the last snapshot (or only from the snapshot achieving the best F1), giving one independent observation per node per network. Re-run the Spearman correlation and Mann-Whitney U tests on these node-level values, and also fit a mixed-effects or clustered model with node as a random effect on the pooled snapshot data. If the 6-network significant negative correlations and 6-of-8 significant MWU differences do not survive at p<0.05, the pooled analysis was the source of the signal.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that NC-LID computed per snapshot on static graphs indicates nodes whose dynnode2vec embeddings poorly preserve temporal graph structure. The strongest statistical evidence is the Spearman correlations and MWU tests in Section 4, computed over pooled node-snapshot observations. The load-bearing problem is that these observations are not independent: the same node appears in multiple snapshots with highly correlated embeddings and NC-LID values, so the effective sample size is far smaller than the pooled count. Violating the MWU independence assumption inflates significance, and the claim that significant differences exist in 6 of 8 networks is therefore unverified. In addition, the per-snapshot NC-LID ignores temporal paths, event ordering, and temporal neighborhoods, so \"temporal graph structure\" may not be what is being tested. The post-hoc exclusion of ia-realitycall (Section 4) removes a network with near-random embeddings and an F1 correlation opposite to the claimed direction; without a justified or pre-specified exclusion rule, this weakens robustness. The strongest check is therefore to recompute the correlation and MWU tests on one observation per node per network, using only the last snapshot or a within-node temporally aggregated NC-LID over snapshots; if the negative correlations and significant MWU differences do not survive, the empirical support for NC-LID as an indicator of weak dynamic embeddings collapses.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an adaptation of the static graph measure NC-LID (local intrinsic dimensionality based on natural communities) to dynamic graphs, and an evaluation methodology for testing whether NC-LID identifies nodes whose dynamic graph embeddings poorly preserve local structure. The authors apply dynnode2vec to ten real-world dynamic networks, computing per-snapshot NC-LID values and comparing them with node-level F1 scores from graph reconstruction. They report significant negative Spearman correlations between NC-LID and F1 for six networks, and Mann-Whitney U tests showing significantly lower F1 for high-NC-LID nodes in six of eight networks with significant differences. The paper concludes that NC-LID is a promising indicator for guiding LID-aware dynamic embedding methods.","tokens_in":8407,"tokens_out":3748,"duration_ms":35293,"significance":"If the reported relationship holds, the result is a useful first step toward adaptive, LID-aware dynamic graph embedding, with a concrete evaluation methodology and a clearly stated future extension to modify random-walk sampling based on NC-LID. The use of graph reconstruction F1 as an external criterion is a strength: the evaluation metric is computed independently of NC-LID, so the main correlation claim is not circular. The paper is also transparent about its data sources and hyperparameter choices. However, the statistical support needs methodological strengthening before the central claim can be accepted.","major_comments":[{"comment":"The correlations and Mann-Whitney U tests are computed over pooled node-snapshot observations, i.e., each node contributes one observation in every snapshot in which it appears. This violates the independence assumption of both Spearman's rank correlation and the MWU test. Nodes recur across snapshots with highly persistent embedding vectors and NC-LID values, so the effective sample size is much smaller than the pooled count and the reported p-values (e.g., in Table 3) are unreliable. Please reanalyze with one observation per node (for example, using the last snapshot, or per-node averaged NC-LID and F1 across snapshots), or use a mixed-effects model or block bootstrap that accounts for node-level clustering, and report whether the significant negative correlations and MWU differences survive.","section":"Section 3, Spearman correlations and MWU tests"},{"comment":"The paper sets aside ia-realitycall from the interpretation of Figure 3 because its dynnode2vec embeddings have extremely low F1 (0.0362 in Table 2), stating that \"the correlation results obtained for this network can not be used to derive a meaningful conclusion.\" This exclusion is introduced after observing that ia-realitycall exhibits a positive correlation, opposite to the claimed direction. Without a pre-specified, uniformly applied quality threshold, this post-hoc removal weakens the robustness of the central claim. Please either include ia-realitycall in the summary statistics or justify the exclusion with an objective criterion (e.g., a minimum F1 threshold) applied to all networks before looking at the correlations.","section":"Section 4, exclusion of ia-realitycall"},{"comment":"The NC-LID variant used in the experiments is the static, per-snapshot definition: it is computed independently on each snapshot using shortest-path distances and natural communities. This measure does not incorporate temporal paths, event ordering, or temporal neighborhoods. The abstract and conclusions state that NC-LID indicates nodes whose embeddings \"do not tend to preserve temporal graph structure well,\" but the experiments actually test preservation of static connectedness within snapshot graphs. Please either align the language with what is measured, or additionally evaluate the continuous-time adaptation based on temporal distances described in Section 2, and clarify which notion of \"temporal structure\" the reported correlations support.","section":"Section 2 and Section 4"}],"minor_comments":[{"comment":"There are several typos: \"exibited\" should be \"exhibited\", \"Spreman\" should be \"Spearman\", \"Dijsktra\" should be \"Dijkstra\", and \"shorest-path\" should be \"shortest-path\".","section":"Throughout"},{"comment":"Figure 3 shows the Spearman correlations but no numerical values or p-values. Adding per-network sample sizes, exact correlation coefficients, and significance markers would make the results easier to assess, especially given the pooling issue raised above.","section":"Figure 3"},{"comment":"The probabilities of superiority PS(H) and PS(L) do not sum to 1 for several rows (e.g., ia-realitycall: 0.1701 + 0.0817 = 0.2518). This presumably reflects ties in F1 scores; please state explicitly how ties are treated in the computation of these probabilities.","section":"Table 3"},{"comment":"The hyperparameter tuning description says p and q are chosen from {0.25, 0.50, 1, 2, 4} and the number of walks per node and walk length are fixed, but it does not report the final selected hyperparameters for each network beyond those in Table 2. Listing the tuned p/q values for all networks would improve reproducibility.","section":"Section 3"}],"recommendation":"major_revision","confidential_remarks":"The core idea is incremental but potentially useful. The main technical problem is the pooled statistical testing, which is load-bearing for the paper's central claim; if the requested reanalysis fails to preserve the significance, the claim would need to be substantially weakened. The post-hoc exclusion of ia-realitycall also needs a principled justification. The paper is otherwise clearly written and the experimental setup is transparent."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Fairly straightforward paper: it takes NC-LID, a static-graph local intrinsic dimensionality measure, computes it per snapshot for ten dynamic networks, and correlates it with the quality of dynnode2vec embeddings measured by graph reconstruction F1. The new content is the correlation study itself; the adaptation is exactly what you'd expect. That's fine as a first step, and the paper is honest about it.\n\nWhat the paper does well: the experimental design is clear, the datasets are public and standard, and the authors report per-network details. The main observation—that in six of ten networks, nodes with higher NC-LID tend to have lower F1—is plausible and worth checking. They also run Mann-Whitney tests comparing high- vs low-NC-LID nodes, and the directions line up with the correlations.\n\nThe soft spot is statistical. The correlations and MWU tests pool all node-snapshot observations, so the same node contributes many times, with embeddings and NC-LID values that are highly correlated across snapshots. That inflates the effective sample size and makes the p-values look far stronger than they are. The MWU assumption of independent observations is violated. Recomputing with one observation per node (e.g., last snapshot or a node-level aggregate) is the obvious check. The paper also doesn't correct for multiple testing across ten networks, and the exclusion of ia-realitycall is post hoc—though the stated reason, that the embedding itself is essentially random, is defensible. A second issue is that per-snapshot NC-LID ignores temporal paths and event ordering, so the phrase 'temporal graph structure' is doing more work than the measure does.\n\nIf the per-node analysis still shows a negative association in most networks, the paper is a useful empirical result for the dynamic-graph-embedding community. As it stands, the central claim is plausible but the evidence is weaker than reported. No code or data is released, which makes the numbers harder to check.\n\nI'd send it to review, but with a clear request for a proper independence-aware analysis. The paper is a legitimate contribution to a narrow subfield, and the authors are not overclaiming—they call it a first step, which it is.","headline":"Plausible empirical finding that per-snapshot NC-LID correlates negatively with dynnode2vec embedding quality, but the pooled node-snapshot statistics overstate significance.","tokens_in":8940,"tokens_out":2401,"would_cite":false,"duration_ms":22722,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that NC-LID, a local intrinsic dimensionality measure for static graphs, can identify nodes whose dynamic graph embedding vectors fail to preserve temporal structure.","keywords":["dynamic graphs","graph embeddings","local intrinsic dimensionality","NC-LID","dynnode2vec","random walks","link reconstruction","temporal networks"],"falsifier":"Run the same pipeline on a synthetic dynamic graph with known community evolution and controlled random-walk embeddings; compute per-node NC-LID and F1 for link reconstruction. If the Spearman correlation between NC-LID and F1 is non-negative, or if high-NC-LID nodes do not have lower F1, the claimed indicator property fails.","tokens_in":7984,"feed_emoji":"🕸️","tokens_out":4502,"duration_ms":38776,"temperature":0.7,"pith_summary":"The paper asks whether NC-LID, a measure of local intrinsic dimensionality originally defined for static graphs, can identify nodes whose dynamic graph embeddings are poorly constructed. Using dynnode2vec on 10 real-world dynamic networks, the authors find significant negative Spearman correlations between NC-LID and the F1 score of link reconstruction in 6 networks, with coefficients from -0.19 to -0.41. Mann-Whitney tests show that in 6 of the 8 networks with significant differences, high-NC-LID nodes have significantly lower F1 than low-NC-LID nodes. The authors conclude that NC-LID is a good indicator of nodes whose embedding vectors fail to preserve temporal graph structure, and that this is a first step toward LID-aware dynamic graph embedding algorithms.","feed_headline":"NC-LID flags nodes with poor dynamic embeddings","feed_subtitle":"A local-dimensionality score predicts where dynnode2vec embeddings lose temporal structure across real networks.","key_machinery":"The central object is NC-LID(n) = $-\\ln\\left(\\frac{|S|}{D(n,k)}\\right)$, where $S$ is the natural community of node $n$ found by a fitness-based algorithm, $k$ is the largest shortest-path distance from $n$ to any node in $S$, and $D(n,k)$ counts nodes within $k$ hops. It measures how irregularly shaped the node's intrinsic locality is: low values mean shortest-path distance cleanly separates the community from the rest of the graph, high values mean the community blends into surrounding structure. The paper's methodology relies on this scalar as a per-node, per-snapshot signal and correlates it with reconstruction-based quality metrics (precision, recall, F1) of dynnode2vec embeddings, then compares F1 of high- versus low-NC-LID groups with Mann-Whitney U tests.","core_discovery":"The central claim is that NC-LID, computed independently on each snapshot of a dynamic graph, serves as a proxy for where dynnode2vec's embeddings lose local structure. Nodes with high NC-LID have natural communities that are hard to separate from the rest of the graph by shortest-path distance; the paper shows these nodes tend to have lower precision, recall, and F1 in graph reconstruction from embeddings. This relationship is not universal: two networks show no significant F1 difference, and one network's embeddings are so poor that its correlations are uninformative. The authors read the overall pattern as evidence that dynamic embedding methods can be improved by steering random-walk sampling toward high-NC-LID nodes.","pith_inferences":["An editorial inference is that the mechanism behind the correlation may be static community complexity rather than temporality: the same negative relationship might appear for static node2vec embeddings on each snapshot, which would mean the dynamic setting adds little beyond what static NC-LID already predicts.","A testable extension is to compare pooled per-snapshot NC-LID with NC-LID computed on the time-aggregated graph; if the aggregated version loses the correlation, event ordering is the informative part.","Because node-snapshot observations are pooled, the reported p-values treat repeated appearances of the same node as independent; a node-level bootstrap or mixed-effects model would give a stricter significance check."],"forward_implications":["If NC-LID reliably marks weak nodes, dynnode2vec and similar random-walk methods can allocate more random walks to high-NC-LID nodes without increasing the total walk budget.","Random-walk bias parameters $p$ and $q$ could be personalized by NC-LID to keep walks inside complex natural communities longer, potentially improving the embeddings of those nodes.","The same methodology transfers to any dynamic embedding method built on discrete snapshots, since per-snapshot NC-LID needs no modification.","In continuous-time representations, replacing shortest-path distance with temporal distance would extend the indicator to event-based dynamic graphs.","Link-reconstruction F1 can serve as a cheap, task-agnostic node-level quality monitor for dynamic embeddings."],"supporting_citations":[{"why":"Defines NC-LID for static graphs and proposes the LID-aware graph embedding approach this paper adapts.","marker":"[12]"},{"why":"Shows LID-aware node2vec improves intrinsic embedding quality, supplying the reconstruction-based evaluation logic.","marker":"[13]"},{"why":"Provides dynnode2vec, the dynamic random-walk embedding method whose node-level quality is correlated with NC-LID.","marker":"[8]"},{"why":"Defines node2vec, the static embedding model that dynnode2vec extends and whose hyperparameters are tuned.","marker":"[2]"},{"why":"Supplies the fitness-based natural community detection algorithm that defines intrinsic localities in NC-LID.","marker":"[7]"},{"why":"Gives the formal LID definition from the ball-expanding model that NC-LID builds on.","marker":"[4]"},{"why":"Provides the Mann-Whitney U test used to compare F1 scores of high- and low-NC-LID nodes.","marker":"[9]"},{"why":"Defines temporal paths and distances used for the continuous-time adaptation of NC-LID.","marker":"[10]"}],"fun_headline_variants":["NC-LID predicts weak nodes in dynamic graph embeddings","Dimensionality score spots dynamic embeddings losing structure","NC-LID exposes nodes that break dynamic embedding fidelity","Local dimensionality predicts where dynamic graph embeddings fail"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The analysis assumes that NC-LID computed separately on each static snapshot captures the structure relevant to how well dynnode2vec preserves temporal structure, and that treating every node-snapshot observation as independent gives valid significance levels.","fun_headline_variants_meta":{"raw":{"variants":["NC-LID predicts weak nodes in dynamic graph embeddings","Dimensionality score spots dynamic embeddings losing structure","NC-LID exposes nodes that break dynamic embedding fidelity","Local dimensionality predicts where dynamic graph embeddings fail"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000498,"raw_usage":{"total_tokens":2372,"prompt_tokens":809,"completion_tokens":1563,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":425,"completion_tokens_details":{"reasoning_tokens":1503}},"tokens_in":425,"tokens_out":1563,"duration_ms":11881,"temperature":1.0,"reasoning_tokens":1503,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:29:28.698691+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same pipeline on a synthetic dynamic graph with known community evolution and controlled random-walk embeddings; compute per-node NC-LID and F1 for link reconstruction. If the Spearman correlation between NC-LID and F1 is non-negative, or if high-NC-LID nodes do not have lower F1, the claimed indicator property fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines NC-LID for static graphs and proposes the LID-aware graph embedding approach this paper adapts."},{"cited_title":"New Journal of Physics 11(3), 033,015 (2009)","cited_arxiv_id":null,"evidence_quote":"Supplies the fitness-based natural community detection algorithm that defines intrinsic localities in NC-LID."},{"cited_title":"In: 2013 IEEE 13th International Conference on Data Mining Workshops, pp","cited_arxiv_id":null,"evidence_quote":"Gives the formal LID definition from the ball-expanding model that NC-LID builds on."}],"review_version":1}