REVIEW 3 major objections 4 minor 35 references
Privacy-Preserving Tensor Factorization for Collaborative Health Data Analysis
T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read DPFact enables collaborative phenotyping across hospitals with formal differential privacy, and reports better accuracy and lower communication cost than federated tensor-factorization baselines.
desk verdict A well-engineered federated tensor factorization paper whose central privacy claim is invalidated by a data-dependent sensitivity bound. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing machinery is the pairing of elastic averaging SGD with zCDP-based Gaussian output perturbation. Local sites run τ passes of permutation SGD, update the patient factor A[t] via proximal gradient with $\ell^2$,1 soft-thresholding, and update the feature factors B[t] and C[t] with an elastic pull γ toward the global factors. Before upload, each feature factor is perturbed with noise from $\mathcal{N}(0, \Delta_2^2/(2\rho))$, where the sensitivity is taken as $\Delta_2(f) = 2\tau L\eta$ and L is the Lipschitz constant of the local objective gradient, computed as the Frobenius norm of $(A^{[t]\top}A^{[t]})*(C^{[t]\top}C^{[t]})+\gamma I$ in Appendix A.1. The server aggregates the noisy factors into the global B and C via a gradient update. Privacy accounting relies on serial composition across the two feature factor matrices per epoch and parallel composition across sites, converting the zCDP budget into a standard (ε,δ)-DP guarantee at convergence.
What would settle it
Take two neighboring local tensors that differ in only one patient's entry, run the Appendix A.1 calculation for the Lipschitz constant $L = \|(A^\top A)*(C^\top C)+\gamma I\|_F$, and compare the resulting sensitivity $\Delta_2(f)=2\tau L\eta$; if the two runs give different noise scales, the claimed $(\varepsilon,\delta)$-differential privacy of Theorem 4.1 does not follow from the stated calibration, because the noise is then a function of the private data.
Extended reading notes
Core claim
The paper's central claim is that DPFact—a collaborative tensor factorization built on elastic averaging SGD with Gaussian output perturbation—can decompose horizontally partitioned EHR tensors into shared and site-specific computational phenotypes under a formal differential-privacy guarantee, and that under strict privacy budgets it is more accurate and communication-efficient than the leading federated tensor-factorization baseline. Each site runs τ local passes, updates its patient factor matrix with a proximal step for $\ell^2$,1 regularization, and sends only perturbed versions of the two feature factor matrices to a semi-trusted server; the patient factor matrix is never shared. The privacy analysis uses zero-concentrated differential privacy and parallel composition to obtain Theorem 4.1: Algorithm 1 is (ε,δ)-differentially private when the per-epoch, per-factor budget is $\rho = \varepsilon^2/(8E\log(1/\delta))$, where E is the number of epochs until convergence. The experimental section reports lower RMSE and communication cost than the federated baseline and a centralized ALS baseline, and phenotype discovery that includes patterns specific to the neonatal ICU.
Load-bearing premise
The formal privacy guarantee rests on the noise scale being a fixed public quantity, but the paper computes it from learned factor matrices, which depend on the very patient records being protected.
Editorial extensions
If this is right
- A multi-site consortium could run phenotyping without raw EHR transfer; only noisy feature factor matrices would leave each hospital.
- The total privacy loss can be fixed in advance: choosing a per-epoch budget ρ and knowing the number of epochs E gives an (ε,δ) guarantee before training starts.
- The same run produces both global phenotypes and site-specific ones, because the l2,1 penalty can switch off a phenotype at any site.
- Communication cost stays bounded by the two feature factor matrices per round, which is why DPFact reports lower byte cost than an ADMM-style federated tensor-factorization baseline.
- The privacy/utility trade-off is tunable: larger budgets give factor match scores closer to the non-private model while retaining a formal guarantee.
Reading between the lines
- An extension not pursued in the paper: the same collaborative structure applies to any horizontally partitioned multi-way dataset—medical imaging, sensor networks, or retail—where shared and site-specific latent factors are meaningful.
- A consequence the authors do not spell out: the l2,1 proximal step gives a built-in diagnostic, since a site whose patient-factor column is driven to zero is effectively reporting that a phenotype is absent in that population, which could be read as a population-health comparison across sites.
- A testable extension: vary the per-epoch privacy budget adaptively during training, allocating more budget early when gradients are large and less later, rather than fixing one ρ for all epochs.
- A neighboring direction: replace the zCDP composition with Rényi differential privacy accounting, which could tighten the final (ε,δ) conversion for the same noise scale.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DPFact, a differentially private collaborative tensor factorization method for computational phenotyping from horizontally partitioned EHR tensors. Each hospital runs τ passes of permutation-based SGD on its local tensor, updates patient and feature factor matrices, adds Gaussian noise to the feature factor matrices under zero-concentrated differential privacy (zCDP), and sends the perturbed factors to a semi-trusted server, which aggregates them into global factors using an EASGD-style update. The main theoretical claim (Theorem 4.1) is that the whole procedure is (ε,δ)-differentially private with per-epoch, per-factor budget ρ = ε²/(8E log(1/δ)), where E is the number of epochs at convergence. Experiments on MIMIC-III, CMS DE-SynPUF, and a synthetic dataset compare DPFact with CP-ALS, centralized SGD, and the federated TRIP method, reporting lower communication cost and competitive or better AUC, plus qualitative phenotype discovery. The stress-test concern about data-dependent sensitivity is confirmed by the manuscript itself: Appendix A.1 computes the Lipschitz constant L from the learned factor matrices.
Significance. If its privacy guarantee were valid, DPFact would be a useful contribution: it combines zCDP-based output perturbation with a communication-efficient EASGD formulation, and the l2,1-regularization provides a plausible mechanism for discovering site-specific phenotypes in heterogeneous populations. The empirical evaluation is careful and reproducible in spirit: code is publicly released, three datasets (two real, one synthetic) are used, and the factor-match-score analysis in Figure 3(b) gives a concrete picture of the privacy-utility trade-off. These strengths do not compensate for the fact that the central claim, the formal differential privacy guarantee, rests on a sensitivity bound computed from the private data itself; as stated, Theorem 4.1 is not proven and the mechanism as described does not clearly provide (ε,δ)-DP.
major comments (3)
- [§4.2.1, Appendix A.1, Theorem 4.1] The sensitivity bound Δ2(f) = 2τLη used to calibrate the Gaussian noise is not a valid worst-case sensitivity because L is data-dependent. Appendix A.1 computes L as the Frobenius norm of (A[t]^T A[t])*(C[t]^T C[t]) + γI, and A[t] and C[t] are themselves learned from the private local tensor O[t]. Algorithm 1 contains no clipping, no projection, and no bounded initialization of the factor matrices; the optimization domain is unbounded, so no finite uniform bound on L holds over all neighboring datasets. Proposition 2.7 requires the noise to be calibrated from the worst-case sensitivity sup_{D,D'} ‖f(D) − f(D')‖, which is independent of the realized dataset; calibrating σ from the empirical Lipschitz constant observed during training makes the noise distribution a function of the private data, so the (ε,δ)-DP claim in Theorem 4.1 does not follow from the stated construction. This is a load-bearing gap because the formal privacy guarantee is the paper's central claimed contribution.
- [Theorem 4.1, §4.2.2] The composition count E in Theorem 4.1 is the number of epochs at which the algorithm happens to converge, which is a data-dependent stopping time. The per-epoch budget ρ = ε²/(8E log(1/δ)) is therefore chosen after observing the run, and the proof in §4.2.2 composes 2Eρb mechanisms with E itself dependent on the data. For a valid statement of differential privacy, E must be a fixed, a priori bound on the number of released mechanisms (or the guarantee must be stated as a function of a separately bounded E); otherwise the privacy loss of the realized run is not bounded by the claimed ε. The reported total budgets in §5.4.4, e.g., (1.2, 10⁻⁴) for MIMIC-III, are computed from empirical convergence points and hence inherit this problem.
- [§4.1.1–4.1.2, Appendix A.1] Even accepting the formula Δ2(f) = 2τLη, the sensitivity analysis concerns only the local objective (15) with two of the three factor matrices held fixed. In the actual algorithm, the feature-factor updates (16)–(17) depend on A[t], which is itself updated over the τ passes through (13) and the proximal step (14); the Lipschitz constant computed in Appendix A.1 fixes A[t] and C[t] at their current values and therefore does not bound the end-to-end sensitivity of the released B[t] with respect to a change in one entry of O[t]. The proof gives no argument covering the coupled updates, so the claimed bound on the released factors is not established even in principle.
minor comments (4)
- [§4.1.3] The text states that the Gaussian mechanism adds noise with standard deviation σ = Δ2²/(2ρ); this quantity has the units of variance, and Algorithm 1 line 12 correctly draws noise from N(0, Δ2²/(2ρ)). The text should say variance, or should set σ = Δ2/√(2ρ).
- [§2.3, Proposition 2.10] The parallel composition statement as written gives (1/T)Σρ_t; the standard zCDP parallel composition guarantees max_t ρ_t, and the average form is not a valid general composition bound (it coincides with the maximum only in the symmetric application used in §4.2.2). Please restate the proposition correctly or explicitly restrict it to the special case actually used.
- [§4.2.1] The phrase 'the objective function (15) satisfies L-Lipschitz, with Lipschitz constant L the tight upper bound of the gradient' conflates Lipschitz continuity of the function with Lipschitz continuity of the gradient; what is needed in the formula Δ2(f) = 2τLη is L-smoothness of f.
- [§5.4.5, Tables 2 and 8] The communication-cost table for 1, 5, and 10 sites appears twice (as Table 2 in §5.4.5 and as Table 8 in Appendix A.3), and the text in §5.4.5 refers to 'Table 8' when the nearby table is labeled Table 2. Please remove the duplicate and fix the cross-reference.
Circularity Check
Theorem 4.1's privacy guarantee is self-referential: the noise is calibrated using a Lipschitz constant computed from the private factor matrices, and the per-epoch budget uses the data-dependent convergence epoch.
-
self definitional
[Section 4.2.1 and Appendix A.1 (Eq. 21-22)]
"the L2-sensitivity of this optimization problem in (15) is calculated as ∆2(f) = 2τLη[29]. ... By combining the results in(21) and (22), we thus get theLipschitz constant of∇f(B[t]) as the Frobenius norm of (A[t]⊤A[t])∗( C[t]⊤C[t]) +γIj ."
The Lipschitz constant L is computed as the Frobenius norm of (A[t]^T A[t])*(C[t]^T C[t]) + γI, where A[t] and C[t] are the local factor matrices learned from the private tensor O[t] via updates (13)-(17). Algorithm 1 contains no clipping or projection step, so L is a data-dependent quantity. The claimed sensitivity Δ2(f)=2τLη is therefore not the worst-case supremum over neighboring datasets required by Definition 2.4; it is evaluated at the realized factors. The Gaussian noise added in Algorithm 1 is calibrated from this data-dependent L, so the asserted differential privacy guarantee is constructed from the very private data it is supposed to protect, rather than from a fixed a priori bound.
-
self definitional
[Theorem 4.1, Section 4.2.2]
"Algorithm 1 is (ε,δ)-differentially private if we choose the input privacy budget for each factor matrix per epoch as ρ= ε^2/(8E log(1/δ)) where E is the number of epochs when the algorithm is converged."
The per-epoch privacy budget ρ is set using E, the realized number of epochs until Algorithm 1 converges on the actual private tensors. E is a data-dependent stopping time, not a fixed quantity chosen before seeing the data. Thus the noise variance per epoch is also determined by the observed run. This makes the total privacy claim self-referential: the guarantee is evaluated after the data-dependent trajectory is known, compounding the failure of the sensitivity bound in Section 4.2.1.
full rationale
The empirical accuracy and communication-cost comparisons in Section 5 are not circular: hyperparameter tuning and benchmark comparisons are standard practice, and the utility claims would stand or fall independently. The circularity is confined to the formal privacy proof, but it is load-bearing for the paper's central contribution of a rigorous differential privacy guarantee. Section 4.2.1 defines the L2 sensitivity as Δ2(f)=2τLη, and Appendix A.1 computes L from the factor matrices A[t], C[t] that are themselves learned from the private data. Since Algorithm 1 has no clipping, projection, or other bound making L a global worst-case constant, this is not a valid sensitivity bound under Definition 2.4. The Gaussian mechanism in Algorithm 1 is then calibrated from this data-dependent L, and Theorem 4.1 further chooses the per-epoch budget using the data-dependent convergence epoch E. The privacy guarantee therefore reduces to a statement about the realized data rather than a fixed worst-case analysis. I assign a score of 7 rather than 8 because the non-privacy contributions retain independent content; the privacy proof itself, however, is substantially self-referential and does not establish the claimed guarantee.
Assumptions & free parameters
free parameters (7)
- γ (quadratic penalty) =
5 (MIMIC-III), grid {2,5,8,10}
- μ (l2,1 regularization) =
1,1.8,3.2,1.8,1.5,0.6 per MIMIC-III ICU; 2 CMS; 0.5 synthetic
- η (learning rate) =
1e-2 (MIMIC-III), 1e-3 (CMS), 1e-2 (synthetic)
- ρ (per-epoch, per-factor zCDP budget) =
1e-3
- L (Lipschitz constant for sensitivity) =
computed from ||(A^T A)*(C^T C)+γI||_F
- R (rank) =
50
- τ (local P-SGD passes per epoch) =
not reported
assumptions (5)
- standard math L2 sensitivity of τ-pass P-SGD is Δ2(f)=2τLη (Wu et al. [29])
- standard math zCDP serial and parallel composition propositions hold as stated (Propositions 2.9 and 2.10)
- domain assumption All sites share the same feature factor matrices B and C
- ad hoc to paper The number of epochs E used in the privacy budget is the post-hoc convergence point
- domain assumption The factor norms are bounded a priori so that L is a fixed sensitivity constant
Cite this review
Pith. "Pith review of Privacy-Preserving Tensor Factorization for Collaborative Health Data Analysis." pith.science (2026). https://pith.science/paper/UW3WICJV
@misc{pith2026190809888,
author = {Pith},
title = {Pith review of: Privacy-Preserving Tensor Factorization for Collaborative Health Data Analysis},
year = {2026},
howpublished = {\url{https://pith.science/paper/UW3WICJV}},
note = {Machine review of arXiv:1908.09888}
}
read the original abstract
Tensor factorization has been demonstrated as an efficient approach for computational phenotyping, where massive electronic health records (EHRs) are converted to concise and meaningful clinical concepts. While distributing the tensor factorization tasks to local sites can avoid direct data sharing, it still requires the exchange of intermediary results which could reveal sensitive patient information. Therefore, the challenge is how to jointly decompose the tensor under rigorous and principled privacy constraints, while still support the model's interpretability. We propose DPFact, a privacy-preserving collaborative tensor factorization method for computational phenotyping using EHR. It embeds advanced privacy-preserving mechanisms with collaborative learning. Hospitals can keep their EHR database private but also collaboratively learn meaningful clinical concepts by sharing differentially private intermediary results. Moreover, DPFact solves the heterogeneous patient population using a structured sparsity term. In our framework, each hospital decomposes its local tensors, and sends the updated intermediary results with output perturbation every several iterations to a semi-trusted server which generates the phenotypes. The evaluation on both real-world and synthetic datasets demonstrated that under strict privacy constraints, our method is more accurate and communication-efficient than state-of-the-art baseline methods.
Figures
Reference graph
Works this paper leans on
-
[1]
Bre/t_t W. Bader, Tamara G. Kolda, et al. 2017. MATLAB Tensor Toolbox Version 3.0-dev. Available online. h/t_tps://gitlab.com/tensors/tensortoolbox
work page 2017
-
[2]
Arnaud Berlioz, Arik Friedman, Mohamed Ali Kaafar, Roksana Boreli, and Shlomo Berkovsky. 2015. Applying differential privacy to matrix factoriza- tion. In Proceedings of the 9th ACM Conference on Recommender Systems . ACM, 107–114
work page 2015
-
[3]
Alex Beutel, Partha Pratim Talukdar, Abhimanu Kumar, Christos Faloutsos, Evangelos E Papalexakis, and Eric P Xing. 2014. Flexifact: Scalable /f_lexible factorization of coupled tensors on hadoop. In Proceedings of the 2014 SDM . 109–117
work page 2014
-
[4]
Mark Bun and /T_homas Steinke. 2016. Concentrated differential privacy: Simpli- /f_ications, extensions, and lower bounds. In/T_heory of Cryptography Conference. Springer, 635–658
work page 2016
-
[5]
Eric C Chi and Tamara G Kolda. 2012. On tensors, sparsity, and nonnegative factorizations. SIAM J. Matrix Anal. Appl. 33, 4 (2012), 1272–1299
work page 2012
-
[6]
Joon Hee Choi and S Vishwanathan. 2014. DFacTo: Distributed factorization of tensors. In NIPS. 1296–1304
work page 2014
-
[7]
Patrick L Combe/t_tes and Jean-Christophe Pesquet. 2011. Proximal spli/t_ting methods in signal processing. In Fixed-point algorithms for inverse problems in science and engineering. Springer, 185–212
work page 2011
-
[8]
Cynthia Dwork, Aaron Roth, et al. 2014. /T_he algorithmic foundations of differ- ential privacy. Foundations and Trends® in /T_heoretical Computer Science9, 3–4 (2014), 211–407
work page 2014
Show all 35 references
-
[9]
Cynthia Dwork and Guy N Rothblum. 2016. Concentrated differential privacy. arXiv preprint arXiv:1603.01887 (2016)
2016 arXiv
-
[10]
Cynthia Dwork, Guy N Rothblum, and Salil Vadhan. 2010. Boosting and differ- ential privacy. In 2010 IEEE 51st Annual Symposium on Foundations of Computer Science. IEEE, 51–60
2010
-
[11]
Ma/t_t Fredrikson, Somesh Jha, and /T_homas Ristenpart. 2015. Model inversion a/t_tacks that exploit con/f_idence information and basic countermeasures. InPro- ceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. ACM, 1322–1333
2015
-
[12]
Trisha Greenhalgh, Susan Hinder, Katja Stramer, Tanja Bratan, and Jill Russell
-
[13]
Yuhong Guo and Wei Xue. 2013. Probabilistic Multi-Label Classi/f_ication with Sparse Feature Learning.. In IJCAI. 1373–1379
2013
-
[14]
Briland Hitaj, Giuseppe Ateniese, and Fernando Perez-Cruz. 2017. Deep models under the GAN: information leakage from collaborative deep learning. In Pro- ceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. ACM, 603–618
2017
-
[15]
Joyce C Ho, Joydeep Ghosh, and Jimeng Sun. 2014. Marble: high-throughput phenotyping from electronic health records via sparse nonnegative tensor fac- torization. In Proceedings of the 20th ACM SIGKDD . ACM, 115–124
2014
-
[16]
Jingyu Hua, Chang Xia, and Sheng Zhong. 2015. Differentially Private Matrix Factorization.. In IJCAI. 1763–1770
2015
-
[17]
Alistair EW Johnson, Tom J Pollard, Lu Shen, H Lehman Li-wei, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. 2016. MIMIC-III, a freely accessible critical care database. Scienti/f_ic data3 (2016), 160035
2016
-
[18]
U Kang, Evangelos Papalexakis, Abhay Harpale, and Christos Faloutsos. 2012. Gigatensor: scaling tensor analysis up by 100 times-algorithms and discoveries. In Proceedings of the 18th ACM SIGKDD . ACM, 316–324
2012
-
[19]
Yejin Kim, Robert El-Kareh, Jimeng Sun, Hwanjo Yu, and Xiaoqian Jiang. 2017. Discriminative and distinct phenotyping by constrained tensor factorization. Scienti/f_ic reports7, 1 (2017), 1114
2017
-
[20]
Yejin Kim, Jimeng Sun, Hwanjo Yu, and Xiaoqian Jiang. 2017. Federated tensor factorization for computational phenotyping. In Proceedings of the 23rd ACM SIGKDD. ACM, 887–895
2017
-
[21]
Jun Liu, Shuiwang Ji, and Jieping Ye. 2009. Multi-task feature learning via efficient 𝓁2,1-norm minimization. In UAI. 339–348
2009
-
[22]
Ziqi Liu, Yu-Xiang Wang, and Alexander Smola. 2015. Fast differentially private matrix factorization. In Proceedings of the 9th ACM RecSys . 171–178
2015
-
[23]
Feiping Nie, Heng Huang, Xiao Cai, and Chris H Ding. 2010. Efficient and robust feature selection via joint l2, 1-norms minimization. In NeurIPS. 1813–1821
2010
-
[24]
Rachel L Richesson, Jimeng Sun, Jyotishman Pathak, Abel N Kho, and Joshua C Denny. 2016. Clinical phenotyping in selected national networks: demonstrating the need for high-throughput, portable, and computational methods. Arti/f_icial intelligence in medicine 71 (2016), 57–61
2016
-
[25]
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017. Membership inference a/t_tacks against machine learning models. InSecurity and Privacy (SP), 2017 IEEE Symposium on . IEEE, 3–18
2017
-
[26]
Yining Wang and Anima Anandkumar. 2016. Online and differentially-private tensor decomposition. In NeurIPS. 3531–3539
2016
-
[27]
Yichen Wang, Robert Chen, Joydeep Ghosh, Joshua C Denny, Abel Kho, You Chen, Bradley A Malin, and Jimeng Sun. 2015. Rubik: Knowledge guided tensor factorization and completion for health data analytics. In Proceedings of the 21th ACM SIGKDD. ACM, 1265–1274
2015
-
[28]
Wei-Qi Wei and Joshua C Denny. 2015. Extracting research-quality phenotypes from electronic health records to support precision medicine. Genome medicine 7, 1 (2015), 41
2015
-
[29]
Xi Wu, Fengan Li, Arun Kumar, Kamalika Chaudhuri, Somesh Jha, and Jeffrey Naughton. 2017. Bolt-on differential privacy for scalable stochastic gradient descent-based analytics. In Proceedings of the 2017 ACM International Conference on Management of Data . ACM, 1307–1322
2017
-
[30]
Xiao Xu, Shu-Xia Li, Haiqun Lin, SL Normand, Tara Lagu, Nihar Desai, Michael Duan, Eugene A Kroch, and Harlan M Krumholz. 2016. Hospital Phenotypes in the Management of Patients Admi/t_ted for Acute Myocardial Infarction.Medical care 54, 10 (2016), 929–936
2016
-
[31]
Yi Yang, Heng Tao Shen, Zhigang Ma, Zi Huang, and Xiaofang Zhou. 2011. l2, 1-norm regularized discriminative feature selection for unsupervised learning. In IJCAI, Vol. 22. 1589
2011
-
[32]
Doaa Youssef, Hadeel Abd-Elrahman, Mohamed M Shehab, Mohamed Abd- Elrheem, et al. 2015. Incidence of acute kidney injury in the neonatal intensive care unit. Saudi journal of kidney diseases and transplantation 26, 1 (2015), 67
2015
-
[33]
Lei Yu, Ling Liu, Calton Pu, Mehmet Emre Gursoy, and Stacey Truex. 2019. Differentially Private Model Publishing for Deep Learning. arXiv preprint arXiv:1904.02200 (2019)
2019 arXiv
-
[34]
turns off
Sixin Zhang, Anna E Choromanska, and Yann LeCun. 2015. Deep learning with elastic averaging SGD. In NeurIPS. 685–693. A APPENDIX /T_his section provides supplementary information. A.1 Lipschitz Constant Below is the calculation of the Lipschitz constant in objective func- tion...
2015
-
[2010]
Bmj 341 (2010), c5814
Adoption, non-adoption, and abandonment of a personal electronic health record: case study of HealthSpace. Bmj 341 (2010), c5814
2010
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.