{"id":"546b10cf-8268-40b7-bef9-4da655a53217","arxiv_id":"2412.00898","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Using MPC and program-analysis filters, the authors build a privacy-preserving SCA pipeline that keeps F1 around 0.81 and cuts MPC overhead from 184x to 23x.","lead":"Software composition analysis usually forces companies to upload their source code to a vendor's server. This paper studies and builds privacy-preserving versions, finding that multi-party computation gives the strongest protection and that program-analysis filters can cut its huge overhead to about 8.5% of the naive private pipeline.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Vendor-side privacy is not cryptographically guaranteed: the paper's own 1% model-stealing experiment shows measurable leakage, so 'without sacrificing privacy' is unsupported for the vendor.","rationale":"The reader identified the same load-bearing concern: the vendor's privacy relies on the embedding model remaining secret, but the client can query the model through MPC and the paper's own experiment demonstrates partial extraction at 1% of the database. This is the weakest link in the central claim because the abstract explicitly promises no sacrifice to 'SCA's privacy guarantee,' and the paper motivates the work by protecting both customer and vendor assets. The optimization contribution is orthogonal to this issue: the filters do not alter the baseline MPC privacy posture, so they do not introduce the vulnerability, but the paper still presents SAFE SCA as a complete privacy-preserving solution. A concrete scaling test of the model-stealing attack would settle whether the concern is a real failure or a bounded residual risk. The other potential issue, the possibly misleading '8.5% overhead' phrasing (Table III's 8.47% is the time ratio relative to unoptimized SAFE SCA, not the overhead relative to non-private CENTRIS), is a presentation problem rather than a correctness failure because the underlying measurements are reported. The reader's CONDITIONAL verdict already requires bounding the model-stealing risk, so this stress-test does not change the verdict; it sharpens the specific condition.","tokens_in":24200,"tokens_out":7025,"duration_ms":63178,"concrete_test":"Run the model-stealing attack at increasing fractions of the OSS database (1%, 5%, 10%, 25%) via the MPC oracle and measure the stolen model's F1 on the 14-project benchmark; if F1 approaches 0.802 within a feasible query budget, the vendor privacy claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that SAFE SCA preserves privacy without sacrifice depends on protecting the vendor's OSS database and embedding model. In Sec. IV-C, the paper argues that the downloaded OSS vector database is harmless 'without the embedding model,' which the client supposedly cannot access. But under the stated semi-honest threat model, the client can use the MPC protocol as an oracle: it submits arbitrary function tensors and receives plaintext embeddings, then uses these input-output pairs to approximate the model. The paper itself reports that stealing only 1% of the OSS database yields an F1 of 0.589 in about three days, showing measurable model leakage. Scaling this attack with more queries, parallelization, or better query selection could let the client exploit the full downloaded database, defeating the vendor's primary asset protection. The paper dismisses this as 'orthogonal' and suggests external defenses (query limiting, periodic model updates) without implementing or evaluating them. Thus the phrase 'without sacrificing SCA's privacy guarantee' is not supported for the vendor side; the guarantee is only as strong as the unquantified difficulty of model stealing.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents the first systematic study of privacy-preserving software composition analysis (SCA). It identifies the privacy assets of both the client (source/binary code, SCA report) and the vendor (OSS database, embedding model), evaluates three privacy-protection setups (private deployment, SBB-based, and MPC-based) and proposes an optimized MPC-based framework called SAFE SCA. SAFE SCA adds three client-side filters—a symbol filter, an informative source function filter, and an assembly function filter—to reduce the number of functions that need expensive encrypted computation. On a 14-project benchmark, the authors report reducing the MPC-based SCA time from 195,125 seconds to 16,527 seconds (8.47%) while maintaining F1 around 0.809, close to the non-private CENTRIS (DPCNN) baseline at 0.802. The paper also reports an SBB-based leakage experiment and argues that MPC offers the strongest privacy guarantee.","tokens_in":24564,"tokens_out":4183,"duration_ms":42082,"significance":"If the central claim is sustained, the work would be a meaningful step toward practical privacy-preserving SCA: the 11.8x speedup over unoptimized MPC and 23x overhead over non-private CENTRIS are directly measured, and the artifact is publicly released, which supports reproducibility. However, the significance is moderated by two concerns. First, the vendor-side privacy guarantee is not cryptographically established: the client downloads the full OSS vector database and obtains plaintext embeddings through MPC queries, making model extraction a concrete threat that the paper's own 1% stealing experiment partially demonstrates. Second, the accuracy comparison relies on a small 14-project dataset with hyperparameter β selected on the same evaluation set, so the reported F1 values are partly in-sample fits. These issues affect the paper's headline claim that the optimizations come 'without sacrificing privacy guarantee or accuracy,' but they do not invalidate the measured overhead reduction.","major_comments":[{"comment":"The claim that MPC-based SCA achieves privacy 'without sacrificing SCA's privacy guarantee' is not supported for the vendor side. In Sec. IV-C, the paper argues that the downloaded OSS vector database is harmless because the client lacks the embedding model, but the client can use the MPC protocol as an oracle: it submits arbitrary function tensors and receives plaintext embeddings. Under the stated semi-honest threat model, the client may use these input-output pairs to approximate the model. The paper's own experiment shows that stealing 1% of the OSS database yields F1 = 0.589 in about three days, which is measurable leakage. Calling this 'orthogonal' and citing external defenses (query limiting, periodic model updates) is not sufficient, since those defenses are neither implemented nor evaluated. The vendor-side privacy guarantee should either be substantiated with an implemented defense or explicitly weakened in the abstract and conclusion.","section":"Abstract and Sec. IV-C"},{"comment":"The reported accuracy of SAFE SCA, F1 = 0.809, is obtained by selecting β = 40 as the value giving the highest F1 on the same 14-project evaluation set (Fig. 8), and this same in-sample result is then presented in Table III as SAFE SCA's accuracy. This makes the reported F1 partly a fitted value. The paper notes that performance is not sensitive to β, which mitigates the concern, but it would be more convincing to use a separate tuning set or cross-validation. Additionally, the 14-project dataset is small and no variance or significance tests are reported, so the claim that accuracy is not sacrificed relative to CENTRIS (DPCNN) should be stated with appropriate caution.","section":"Sec. VII-A and Table III"}],"minor_comments":[{"comment":"The caption of Fig. 4 contains the stray text 'Fig 4：Try enlarging the font', which appears to be a formatting artifact and should be removed.","section":"Fig. 4"},{"comment":"The sentence 'without the embedding model, which is exclusive to the client, the client cannot exploit the DB' appears to contain a typo: the model is exclusive to the server, not the client. The wording should be corrected to avoid confusion.","section":"Sec. IV-C"},{"comment":"The weight formula w = LoC/(5N−1) is introduced without a derivation or ablation study; a brief explanation of the chosen constants (5 and −1) would improve clarity.","section":"Sec. IV-B"},{"comment":"The hyperparameters θ1 and θ2 are both fixed at 0.02 and the sensitivity analysis is only referenced on the project website; including at least a brief sensitivity plot or table in the paper would make the evaluation more self-contained.","section":"Sec. VII"}],"recommendation":"major_revision","confidential_remarks":"The paper's main contribution is the measured overhead reduction, which appears sound. The main obstacle to acceptance is the unsupported vendor-side privacy claim in the abstract: the model-stealing experiment in Sec. IV-C directly contradicts the 'without sacrificing privacy guarantee' phrasing. I would favor revision over rejection because the overhead result is valuable and the privacy claim can likely be fixed by either implementing/evaluating a defense or substantially qualifying the claim. The accuracy overfitting issue (β selected on the evaluation set) also needs to be addressed, ideally with cross-validation or a separate tuning split."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a solid applied systems paper with one overclaimed sentence. The genuinely new part is the empirical landscape of privacy-preserving SCA plus SAFE SCA's three client-side filters that cut the MPC-based SCA cost from 195,125s to 16,527s (8.5%, 11.8x) on 14 real projects, with F1 0.809 versus 0.802 for non-private CENTRIS(DPCNN). The overhead reduction is measured directly, not derived, and the artifact is released. That deserves real credit.\n\nThe soft spots are mostly about the strength of the claims, not the engineering. The vendor-side privacy guarantee in Sec. IV-C is the weakest link. The client downloads the whole OSS vector database, and under the stated semi-honest model can query the MPC protocol as an oracle to learn embedding model outputs. The paper's own proof-of-concept, stealing 1% of the database, reaches F1 0.589 in about three days. That is measurable model leakage, so \"without sacrificing SCA's privacy guarantee\" is not supported for the vendor. Calling it orthogonal is fair only if you accept external defenses the paper does not implement or measure. The stress-test note lands here.\n\nOn evaluation: beta=40 is chosen from Fig. 8 on the same 14-project benchmark, so the reported F1 is partly a fitted value. The paper says performance is not sensitive to beta (F1 stays above 0.78 for beta > 5), which softens the concern, but reporting variance across projects and using a small held-out selection would still help. The dataset is small (14 binaries, manually constructed ground truth) and there are no error bars or significance tests. That is a moderate weakness, not a fatal one.\n\nThe \"first study\" framing is a bit strong given PP-CSA and other privacy-preserving program analysis works, but the specific SCA instantiation and the optimization filters are new. The comparison with BinaryAI is reasonable. Self-citations are mostly to the same group's prior binary analysis work; no red flag there.\n\nWho is this for: researchers and practitioners in software supply-chain security who want a concrete MPC-based SCA prototype and a map of trade-offs. It deserves a serious referee, with the vendor-privacy claim and the evaluation methodology as the main revision targets.","headline":"Solid applied systems paper: the measured 11.8x overhead reduction for MPC-based SCA is real, but the vendor-side privacy guarantee is overclaimed and the evaluation has a tuning-on-the-test-set issue.","tokens_in":24988,"tokens_out":1578,"would_cite":false,"duration_ms":15421,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Software composition analysis can be made private without an accuracy or cost blow-up by sending fewer than 5% of functions through encrypted computation.","keywords":["privacy-preserving software composition analysis","multi-party computation","code similarity","program analysis filters","open-source software reuse","code embedding","secure computation","software supply chain security"],"falsifier":"Have a client issue queries over a much larger fraction of the OSS database and train a stand-in embedding model on the downloaded vectors paired with public OSS source code, then measure whether the stand-in reaches the plaintext model's F1 of about 0.802. If it does, the claim that the vendor's database is protected 'without sacrificing privacy' is false for the vendor.","tokens_in":23979,"feed_emoji":"🔒","tokens_out":8955,"duration_ms":73101,"temperature":0.7,"pith_summary":"Software composition analysis (SCA) identifies which open-source components a program reuses, but today's services require uploading the whole codebase to a vendor, exposing proprietary code and the SCA report. The paper argues that multi-party computation (MPC) gives both customers and vendors the strongest privacy protection among the studied options, at the price of a 184-fold slowdown. It then shows that the slowdown is mostly wasted work: three client-side filters, namely symbol matching with call-graph checks, selection of complex source functions, and selection of representative assembly functions, reduce the number of functions sent into encrypted computation to under 5% of the total. On a 14-project benchmark the optimized pipeline runs in 16,527 seconds instead of 195,125 seconds (8.5% of the cost) while achieving F1 0.809, essentially matching the non-private baseline's 0.802. The claim is that practical, accuracy-preserving privacy-preserving SCA is achievable when both sides are semi-honest, meaning they follow the protocol but try to learn from what they see.","feed_headline":"Privacy-preserving SCA overhead cut to 8.5%","feed_subtitle":"Client-side filters shrink encrypted computation to under 5% of functions, keeping F1 at 0.809 versus 0.802 non-private.","key_machinery":"The load-bearing mechanism is the filter-then-encrypt pipeline of SAFE SCA. Three client-side filters decide which functions ever touch CrypTen, the secret-sharing multi-party computation framework that encrypts client tensors and the vendor's embedding model during inference. The symbol filter first rules out functions copied from OSS projects by matching complex function names and requiring the matched functions to share the same call-graph edge, reaching F1 0.547 on its own; the informative source function filter keeps only functions with the lowest maintainability indices; and the assembly function filter selects disassembled functions whose names occur in the OSS database while down-weighting very long functions. The net effect is a regression of y = 0.048x + 77 selected functions versus x total functions, i.e., under 5% of functions carry the entire encrypted workload.","core_discovery":"The central discovery is that the cost of MPC-based SCA is not a fixed tax: it scales with the number of functions sent through the cryptographic protocol, and most functions do not need to be sent at all. By filtering on the client side before encryption, matching complex symbols plus call-graph edges, keeping only low-maintainability source functions, and keeping only assembly functions whose names appear in OSS projects and whose basic-block counts are tractable, the proposed SAFE SCA pipeline encrypts only about 4.8% of the input functions. The paper reports that this brings end-to-end SCA time from 195,125 seconds to 16,527 seconds on a 14-project dataset, while F1 rises slightly to 0.809 from the unfiltered version's 0.791 and compared with 0.802 for the non-private CENTRIS(DPCNN). The same experiments show the SBB-based alternative leaks the SCA report to a curious server, so MPC remains the only studied approach that protects both the customer's code and the vendor's database and model.","pith_inferences":["The 8.5% figure is tied to this 14-project benchmark and this particular embedding stack; the durable insight is the selection-rate law, under 5% of functions, which should transfer to larger databases but deserves re-measurement at industrial scale.","The vendor-side privacy guarantee depends on the embedding model staying secret. The paper's own experiment shows that stealing 1% of the OSS database to approximate the model reaches only 0.589 F1 in about three days; scaling that attack, for example by pairing the downloaded vector database with public OSS source code to train a surrogate embedder, is a concrete way to test whether the guarantee","The same filter-then-encrypt pattern could apply to other privacy-preserving code analyses, such as call-stack or vulnerability scanning, wherever a small informative subset of functions carries most of the signal.","The semi-honest assumption is the ceiling of the guarantee: a malicious client who deviates from the filtering protocol could query more functions or craft inputs to extract more information, so higher-assurance deployments would need a malicious-secure MPC variant."],"forward_implications":["An industrial SCA vendor can offer a privacy-preserving service at roughly 20 minutes per analyzed binary on average, instead of hours, while keeping accuracy at the level of the non-private state of the art.","Customers no longer have to choose between leaking source code to the vendor and forcing the vendor to hand over its OSS database: both assets stay protected under MPC.","The SBB-based alternative is unsuitable when the vendor must not learn the SCA report, since a curious server can predict the report from mutated hashes with F1 close to the client's.","The approach extends source-based SCA to statically linked libraries: the assembly filter and binary embedding model lift recall from 0.747 to 0.791 when binary code with debug information is available.","The commit-based signature generation avoids the prohibitive cost of tagging every release, making large-scale OSS database updates tractable."],"supporting_citations":[{"why":"Supplies the CENTRIS SCA framework that all variants build on, including code segmentation and OSS-identification logic.","marker":"[90]"},{"why":"Provides the multi-party computation framework used to encrypt client tensors and the model for the MPC-based solutions.","marker":"[50]"},{"why":"Defines similarity-based bucketization (SBB), the alternative privacy protocol whose leakage the paper measures and rejects.","marker":"[41]"},{"why":"Defines the DPCNN architecture used as the source embedding model in CENTRIS(DPCNN) and SAFE SCA.","marker":"[46]"},{"why":"Defines TLSH, the locality-sensitive hash used by the CENTRIS(TLSH) and SBB-based baselines.","marker":"[72]"},{"why":"Guides OSS database construction, dependency analysis, and handling of overlapping signatures, informing the paper's database.","marker":"[44]"},{"why":"Introduces the maintainability-index-based selection of fingerprinting functions that inspires the informative source function filter.","marker":"[91]"},{"why":"Provides the similarity-search index used to match embeddings efficiently against the OSS database.","marker":"[45]"}],"fun_headline_variants":["MPC-based SCA overhead slashed to 8.5%","Client-side filtering cuts SCA crypto cost 12x","Privacy-preserving SCA now only 8.5% overhead","Filter first, encrypt later for fast private SCA","Encrypt only 4.8% of functions for private SCA"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The vendor's database stays secret only because the client never gets the embedding model that would make those downloaded vectors useful for code lookups; if a client can reconstruct or steal that model from query results, the vendor-side privacy guarantee collapses.","fun_headline_variants_meta":{"raw":{"variants":["MPC-based SCA overhead slashed to 8.5%","Client-side filtering cuts SCA crypto cost 12x","Privacy-preserving SCA now only 8.5% overhead","Filter first, encrypt later for fast private SCA","Encrypt only 4.8% of functions for private SCA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000702,"raw_usage":{"total_tokens":3222,"prompt_tokens":1053,"completion_tokens":2169,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":669,"completion_tokens_details":{"reasoning_tokens":2081}},"tokens_in":669,"tokens_out":2169,"duration_ms":16465,"temperature":1.0,"reasoning_tokens":2081,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T04:53:03.041424+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have a client issue queries over a much larger fraction of the OSS database and train a stand-in embedding model on the downloaded vectors paired with public OSS source code, then measure whether the stand-in reaches the plaintext model's F1 of about 0.802. If it does, the claim that the vendor's database is protected 'without sacrificing privacy' is false for the vendor.","supporting_citations":[{"cited_title":"Centris: A precise and scalable approach for identifying modified open-source software reuse,","cited_arxiv_id":null,"evidence_quote":"Supplies the CENTRIS SCA framework that all variants build on, including code segmentation and OSS-identification logic."},{"cited_title":"Increasing adversarial uncertainty to scale private similarity testing,","cited_arxiv_id":null,"evidence_quote":"Defines similarity-based bucketization (SBB), the alternative privacy protocol whose leakage the paper measures and rejects."},{"cited_title":"Deep pyramid convolutional neural networks for text categorization,","cited_arxiv_id":null,"evidence_quote":"Defines the DPCNN architecture used as the source embedding model in CENTRIS(DPCNN) and SAFE SCA."},{"cited_title":"Tlsh–a locality sensitive hash,","cited_arxiv_id":null,"evidence_quote":"Defines TLSH, the locality-sensitive hash used by the CENTRIS(TLSH) and SBB-based baselines."},{"cited_title":"Third-party library dependency for large-scale sca in the c/c++ ecosystem: How far are we?","cited_arxiv_id":null,"evidence_quote":"Guides OSS database construction, dependency analysis, and handling of overlapping signatures, informing the paper's database."},{"cited_title":"Ossfp: Precise and scalable c/c++ third-party library detection using fingerprinting functions,","cited_arxiv_id":null,"evidence_quote":"Introduces the maintainability-index-based selection of fingerprinting functions that inspires the informative source function filter."},{"cited_title":"Billion-scale similarity search with GPUs,","cited_arxiv_id":null,"evidence_quote":"Provides the similarity-search index used to match embeddings efficiently against the OSS database."}],"review_version":1}