Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Visualizing Distributed Traces in Aggregate

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that grouping traces by shared services or graph structure and showing one aggregate view per group lets developers analyze a distributed trace dataset without reading every trace.

desk verdict An honest, reimplementable trace-aggregation prototype whose reported success actually reduces to exact service-set equality, not similarity grouping. read the letter →

arxiv 2412.07036 v1 pith:7CDPWIMI submitted 2024-12-09 cs.DC

Adrita Samanta , Henry Han , Darby Huye , Lan Liu , Zhaoqi Zhang , Raja R. Sambasivan This is my paper · ORCID
classification cs.DC
keywords distributedtracingtraceaggregationJaccardsimilaritydisjointsetunionrepresentativetracesvisualizationsimilargroupingsystemsdebugging
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that developers can understand a distributed trace dataset by reading a small number of representative traces instead of combing through every trace. It defines trace similarity by the set of services a trace touches or by the exact structure of the trace graph, connects traces whose Jaccard similarity clears a tunable threshold, and uses disjoint set union to form groups. For each group it picks the trace with the most similarity links as the representative, and visualizes the group as an aggregate graph that shows which services appear in every trace and which service-to-service calls are most frequent. If the method holds up, a developer debugging a large trace dataset would start from one view per workflow instead of sampling individual traces.

What carries the argument

The central machinery is Jaccardian similarity combined with disjoint set union. Each trace is encoded as a set, either of service names or of graph edges; similarity between two traces is the size of their intersection divided by the size of their union; a chosen threshold turns pairwise similarities into edges of a trace similarity graph; union-find collects the connected components; and the trace with the highest degree in a component becomes its representative. The aggregate trace data structure is the visualization layer: a graph whose node colors and sizes encode how many group members contain a given service and whose edge thickness encodes how many times one service calls another across the group.

What would settle it

Take a dataset whose traces carry ground-truth labels of complete workflows, including both cache-hit and cache-miss executions, and run the preprocessing rule that deletes any trace whose edge set is a subset of another trace's edge set; if any labeled-complete traces are deleted, the filter is removing valid workflows. As a second check, compare the method's groups to a manual grouping by request type on the same dataset and count mismatches, which would show whether service-set similarity is too coarse to separate distinct workflows.

Watch

Extended reading notes

Core claim

The discovery is a concrete pipeline: encode each trace as a set of service names or as a set of graph edges; compute Jaccard similarity between encodings; connect traces whose similarity passes a threshold; apply disjoint set union to find connected components; choose the highest-degree trace in each component as the representative; and show the component as an aggregate graph in which node color marks services present in all traces and edge thickness marks call frequency. The paper reports that on a hand-built set of 24 traces, the grouping by service sets exactly matched a manual categorization, and that a binary-search procedure found thresholds producing 5 groups when 6 were requested and 11 groups when 11 were requested. The same pipeline removes a trace whose edge set is a subset of another trace's edge set, on the assumption that the smaller trace is an incomplete recording of the larger one.

Load-bearing premise

The load-bearing premise is that when one trace's edge set is a subset of another trace's edge set, the smaller trace is an incomplete version of the larger one and can be removed; the paper itself notes this is false for workflows where a shorter trace is a complete outcome, such as cache hit versus cache miss.

Editorial extensions

If this is right

  • A developer would inspect one representative trace per group, so the number of views needed to understand a workload tracks the number of distinct workflows rather than the number of traces collected.
  • The similarity threshold becomes a zoom control: lower values merge workflows into coarser groups, higher values split them, and the binary-search procedure finds a threshold that meets a target group count.
  • The aggregate graph shows both the common path and the variation around it, revealing which services are always present and which calls are frequent across a group.
  • The subset-based preprocessing shrinks the trace set before grouping, which reduces the pairwise comparisons the later steps must run.
  • The paper's own limitation note shows the preprocessing can delete complete short workflows, such as cache hits, because it treats every subgraph as an incomplete record.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Applied to a production dataset, the service-set definition would probably merge traces that share services but perform different request types, so groups may hide semantic differences that a structure-based or request-type-aware similarity would separate.
  • The subset rule can be tested directly: with ground-truth workflow labels, count how many labeled-complete traces are removed; any such removal shows the filter is discarding valid workflows.
  • The same grouping machinery could be inverted for anomaly detection, treating traces far from their group's representative as outliers rather than selecting the representative as the typical view.
  • Because pairwise Jaccard comparison is quadratic in the number of traces, using this on millions of traces would likely require sampling or indexing before the union-find step; the small synthetic tests do not yet show behavior at that scale.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a pipeline for aggregating distributed traces: it filters out traces whose edge sets are subsets of other traces, groups traces by Jaccard similarity of service-name sets (Definition 1) or exact structure (Definition 3), selects one representative trace per group, and builds aggregate graph visualizations. The evaluation is performed on two hand-created JSON datasets (24 and 32 traces) using a default threshold of 0.8 and a binary-search procedure that targets a user-specified number of groups. The authors report exact agreement with their own manual grouping, but no measurements on real trace datasets are provided.

Significance. If validated on realistic data, the approach could make trace triage more efficient by reducing the number of traces a developer must read. The paper ships reproducible source code and data, and it provides an honest limitation discussion in Section 7. However, the current evaluation is too weak to support the central claim: the default threshold makes the 'similarity' grouping collapse to exact service-set equality on the toy datasets, the threshold-search procedure tunes only to a user-supplied group count, and the preprocessing filter is conceded to be unsound for legitimate short workflows. The central algorithm is coherent, but the evidence for its effectiveness is missing.

major comments (4)
  1. [§6.2 (Default Threshold Experiment)] The reported 'exact match' to the manual grouping is fully explained by the choice of threshold 0.8: in the 24-trace sample, any two traces with different service sets have Jaccard similarity at most 3/4, so the similarity graph contains edges only between traces with identical service-name sets. Consequently, the experiment demonstrates exact service-set equality clustering, not the similarity-based grouping claimed in the abstract. This is a load-bearing gap in the effectiveness evaluation.
  2. [§4.2.3 (Similarity Threshold)] The 'optimal threshold' procedure optimizes only toward a user-specified number of groups (goalGroups), not toward any independent ground truth of correct grouping. The Section 6.3 result that a threshold of 0.625 yields 5 trace groups for goalGroups=6 thus shows only that the binary search can hit a user preference; it does not establish that the resulting groups are behaviorally similar or meaningful. Manual checking of the authors' own synthetic traces does not provide independent validation.
  3. [§4.1 (Preprocessing Traces) and §7 (Future Work)] The preprocessing rule removes any trace whose edge set is a subset of another trace's edge set, on the assumption that the smaller trace is incomplete. Section 7 explicitly concedes that this is false for workflows such as cache hit versus cache miss, where a shorter trace is a complete valid outcome. Since Section 8 nonetheless claims to have successfully filtered incomplete traces, the paper is internally inconsistent: the claimed success of the first research goal is contradicted by the acknowledged limitation of its only mechanism.
  4. [§8 (Conclusion)] The conclusion states that the authors 'used our third definition to group similar traces based on the exact structure of the trace graph,' but Section 6 reports results only for Definition 1 (service sets). No experiment or evaluation for Definition 3 is presented anywhere in the paper, so this claim is unsupported. The paper should either report such results or remove the claim.
minor comments (4)
  1. [§2.4] The Jaccard formula is referenced ('In other words:') but the equation is missing from the manuscript; please add the explicit formula.
  2. [§6.2.1] Some references to external material are vague, e.g., 'slide 5 on the sample traces page above', and the GitHub repository link should be made persistent (e.g., via a DOI) and cited properly.
  3. [§6.2.2] The statement that the algorithm's time complexity is O(n^2) is inconsistent with the later qualifier that DSU and the similarity graph can be larger; please state the complexity of the full pipeline more carefully.
  4. [Throughout] There are numerous typographical and grammatical issues (e.g., 'prepro-cess' at the start of §4, inconsistent capitalization of 'Jaccardian' vs 'Jaccard') that should be corrected by careful copyediting.

Circularity Check

2 steps flagged · score 5.0 of 10

The central effectiveness claims are self-definitional: the grouping 'success' is an implementation check of Definition 1 under a threshold that permits only exact service-set equality, and the incomplete-trace filter is defined as the very subgraph test it runs.

  1. self definitional [Section 6.2.1, Results of Effectiveness; Section 4.2.2, Applying Chosen Definition]
    "we manually applied trace definition 1 (categorization based on the names of the services)... So, our grouping of traces exactly matched the grouping that we manually made."

    The manual labels were created by applying Definition 1, which is the same service-set criterion the algorithm uses. At the default threshold of 0.8, no two traces with different service sets in the 24-trace sample have Jaccard similarity above 3/4, so the similarity graph connects only traces with identical service-set lists. The reported 'exact match' is therefore guaranteed by construction; it verifies that the implementation reproduces Definition 1, not that traces that are merely similar (rather than identical in service set) are grouped in a behaviorally useful way.

  2. self definitional [Section 4.1, Preprocessing Traces; Section 7, Future Work; Section 8, Conclusion]
    "We defined an incomplete trace to be a subgraph of a more complete version of the trace. ... Some workflows in a system terminate before other workflows in certain scenarios (like a cache hit vs miss) but it doesn’t mean that the terminated trace is incomplete."

    The paper defines 'incomplete' as being a subgraph and then detects incompleteness by checking whether one trace's edge set is a subset of another's. Under that definition, the preprocessing step cannot fail to 'filter out incomplete traces,' so the Section 8 claim that the program 'was able to filter out incomplete traces' restates the definition rather than demonstrating empirical success. The Section 7 admission that a shorter trace can be a complete workflow (cache hit vs miss) shows that the definitional test does not track actual incompleteness, confirming the circularity of the claim.

full rationale

This paper is an engineering proposal rather than a formal derivation, and there is no self-citation chain or imported uniqueness theorem: references to prior work are contextual, and the DSU, Jaccard, and graph-tool pieces are ordinary implementation tools. However, the two central effectiveness claims are closed under the paper's own definitions. The grouping evaluation uses manual labels that are exactly Definition 1's service-set categories, and the chosen threshold 0.8 is above the maximum Jaccard value (3/4) between any two distinct service sets in the sample, so the algorithm can only connect traces with identical service sets; the 'exact match' is a self-consistency check, not a validation of similarity-based grouping. The threshold-search procedure is openly a tuning procedure aimed at a user-specified group count rather than an external ground truth, so I do not count it as a separate circular step. The incomplete-trace filter is defined as the subgraph test itself, making its reported success definitional, and the authors themselves concede in Section 7 that the subgraph relation does not imply incompleteness for complete-but-shorter workflows. Weighing these definitional closures against the otherwise non-circular implementation work, the score is 5: partial circularity in the central evaluation and claims, with the visualization and grouping mechanics themselves not circular.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claims rest on a user-chosen similarity threshold, a goal group count for threshold tuning, a set of domain assumptions about what similarity means, and a preprocessing rule that equates edge-subset with incompleteness. The training data for the reported optimal thresholds are the authors' own manually created traces.

free parameters (2)
  • similarity threshold = 0.8 default; 0.625 for goal=6; 0.75 for goal=11
    User-specified threshold in Section 4.2.2 controls whether traces are considered similar; the 'optimal' value is fitted to a goal number of groups in Sections 4.2.3 and 6.3.
  • goal number of trace groups (goalGroups) = 6 and 11 in Section 6.3
    User-supplied target for binary search threshold selection; the reported 'optimal threshold' is defined relative to this goal rather than an external quality metric.
assumptions (4)
  • domain assumption Jaccard similarity over service sets (Definition 1) or edge sets (Definition 3) captures trace similarity
    Adopted in Sections 2.4 and 4.2.2; no justification that Jaccard on these encodings corresponds to behavioral similarity for debugging.
  • ad hoc to paper If trace A's edges are a subset of trace B's edges, A is an incomplete version of B and can be removed
    Used in Section 4.1 preprocessing; the authors themselves identify counterexamples (cache hit vs. miss) in Section 7, so this axiom is load-bearing and known to be false in general.
  • domain assumption Connected components of the Jaccard similarity graph correspond to meaningful trace groups
    Used in Section 4.2.2 step 5; transitivity of the similarity threshold is assumed without discussion.
  • domain assumption The trace with highest degree is the most representative of its group
    Used in Section 4.2.2 step 6; unweighted degree is a heuristic proxy for centrality, not established as representative.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Visualizing Distributed Traces in Aggregate." pith.science (2026). https://pith.science/paper/7CDPWIMI

@misc{pith2026241207036,
  author       = {Pith},
  title        = {Pith review of: Visualizing Distributed Traces in Aggregate},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7CDPWIMI}},
  note         = {Machine review of arXiv:2412.07036}
}
read the original abstract

Distributed systems are comprised of many components that communicate together to form an application. Distributed tracing gives us visibility into these complex interactions, but it can be difficult to reason about the system's behavior, even with traces. Systems collect large amounts of tracing data even with low sampling rates. Even when there are patterns in the system, it is often difficult to detect similarities in traces since current tools mainly allow developers to visualize individual traces. Debugging and system optimization is difficult for developers without an understanding of the whole trace dataset. In order to help present these similarities, this paper proposes a method to aggregate traces in a way that groups together and visualizes similar traces. We do so by assigning a few traces that are representative of each set. We suggest that traces can be grouped based on how many services they share, how many levels the graph has, how structurally similar they are, or how close their latencies are. We also develop an aggregate trace data structure as a way to comprehensively visualize these groups and a method for filtering out incomplete traces if a more complete version of the trace exists. The unique traces of each group are especially useful to developers for troubleshooting. Overall, our approach allows for a more efficient method of analyzing system behavior.

Figures

Figures reproduced from arXiv: 2412.07036 by the authors.

Figure 1
Figure 1. Tprof Aggregate trace of 3 traces. tracing data. 2.3 Prior Work Previous work has been done with the goal of reducing the time and effort needed for going through many distributed traces. One such tool is tprof [3], a performance profiler that creates an aggregate trace by taking the average of subspan durations and timings. For example, in [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. Preprocessing traces. Example of two traces, [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 5
Figure 5. Example traces 1 and 2 with service names. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figures from the paper (6 more)
Figure 6
Figure 6. Figure 6: Group using similarity threshold under 3 [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 8
Figure 8. Figure 8: Representative output for traces 1 and 2. [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]
Figure 9
Figure 9. Figure 9: Span library for trace services. We go through each trace’s services to create a set of all the different services, which in this case are [’Front End’, ’Feed’, ’Friends Database’, ’Post’, ’Friends’]. Then, we find which services are present in each trace and add them …
Figure 11
Figure 11. Figure 11: Single group visualization of Trace 1 and [PITH_FULL_IMAGE:figures/full_fig_p007_11.png]
Figure 10
Figure 10. Figure 10: Visualization output of node names, node [PITH_FULL_IMAGE:figures/full_fig_p007_10.png]
Figure 13
Figure 13. Figure 13: Example JSON format. JSON format of both traces can be found here: trace1.json - 24 traces sample dataset trace2.json - 32 traces sample dataset 6.1.3 Running Experiments. Our code is a single Python program so we run the program with the following command (Main.py is…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Enabling Multi-Dimensional Distributed Trace Comparison with Contrast

    cs.DC 2026-07 conditional novelty 6.0 of 10

    A mergeable trace-summary representation (TPO) lets operators dynamically compare arbitrary groups of traces across structural, timing, critical-path, and semantic dimensions, powering a visual and an LLM-based compar...

Reference graph

Works this paper leans on

8 extracted references · 7 canonical work pages · cited by 1 Pith paper

  1. [1]

    Introduction to Disjoint Set (Union-Find Algo- rithm)

    GeeksForGeeks. Introduction to Disjoint Set (Union-Find Algo- rithm). https://www.geeksforgeeks.org/introduction-to-disjoint-set- data-structure-or-union-find-algorithm/

    work page

  2. [2]

    SampleHST: Efficient On-the-Fly Selection of Distributed Traces, September 2022

    Gias, Alim Ul, Gao Yicheng, Sheldon, Matthew, Perusquía, José A., Owen O’Brien, and Giuliano Casale. SampleHST: Efficient On-the-Fly Selection of Distributed Traces, September 2022. https://doi.org/10. 5281/zenodo.7117741

    work page 2022

  3. [3]

    tprof: Performance profiling via struc- tural aggregation and automated analysis of distributed systems traces

    Lexiang Huang and Timothy Zhu. tprof: Performance profiling via struc- tural aggregation and automated analysis of distributed systems traces. In SoCC’21: Proceedings of the 12th Symposium on Cloud Computing , 2021

    work page 2021

  4. [4]

    Sambasivan

    Darby Huye, Yuri Shkuro, and Raja R. Sambasivan. Lifting the veil on meta’s microservice architecture: Analyses and topology and request workflows. In ATC’23: Proceedings of the 2023 USENIX Annual Technical Conference, 2023

    work page 2023

  5. [5]

    Jaccard Similarity

    Fatih Karabiber. Jaccard Similarity. https://www.learndatasci.com/ glossary/jaccard-similarity/

    work page

  6. [6]

    Sifter: Scalable sampling for distributed traces, without feature engineering

    Pedro Las-Casas, Giorgi Papakerashvili, Vaastav Anand, and Jonathan Mac. Sifter: Scalable sampling for distributed traces, without feature engineering. In SoCC’19: Proceedings of the Ninth Symposium on Cloud Computing, 2019

    work page 2019

  7. [7]

    Tiago P. Peixoto. The graph-tool python library, 2014. Available at: http://figshare.com/articles/graph_tool/1164194

    arXiv 2014

  8. [8]

    Zeno: Diagnosing performance problems with temporal provenance

    Yang Wu, Ang Chen, and Linh Thi Xuan Phan. Zeno: Diagnosing performance problems with temporal provenance. In 16th USENIX Symposium on Networked Systems Design and Implementation (NSDI 19) , pages 395–420, Boston, MA, February 2019. USENIX Association

    work page 2019

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.