REVIEW 4 major objections 4 minor 41 references
Personalization of Deep Learning
T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Personalized training can beat a generic model on both metrics
desk verdict Useful taxonomy and early-shaping idea, but the headline comparison is confounded because the baseline never trains on the target user's data. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying objects are the three shaping schedules and the autoencoder-based similarity measure. Shaping is a curriculum over data sources: batches are drawn entirely from either the individual's dataset or the global dataset, and the schedule determines when the individual's data dominates (early shaping), how often it appears (sample weighing), or whether it is used only for final fine-tuning (transfer learning). The similarity machinery is an autoencoder trained on all data; its 16-dimensional encoder output gives a latent representation, and Euclidean distance in that space ranks how close a global sample is to a user's sample (single-sample grouping) or how close one user's class-mean profile is to another user's (individual grouping). This lets a small personal dataset be augmented with look-alike data from the shared pool, which is what makes the augmentation 'personalized.'
What would settle it
Train the baseline on the global dataset plus the target user's own training data and evaluate on her held-out test set. If that baseline reaches or exceeds the accuracy of EarlyShape-ES-n-400 and TransferL-TL-n-300, then the reported advantage comes from exposure to the user's data, not from the curriculum schedule.
Extended reading notes
Core claim
The central claim is that personalization of a deep classifier can be achieved by curriculum shaping and by similarity-based data grouping, and that these are not merely trade-off maneuvers. Shaping changes the exposure schedule: early shaping trains only on the individual's data for a limited number of iterations before switching to the global set; sample weighing interleaves individual batches throughout training (one every n_I-th iteration); transfer learning fine-tunes a globally trained model on the individual's data. Data grouping replaces the individual's small dataset with an enlarged one, where added samples are either the most similar single samples in the global set or entire datasets of similar individuals, with similarity measured by Euclidean distance in a 16-dimensional autoencoder latent space. The paper's key experimental result is that EarlyShape with 400 initial iterations and TransferL with 300 iterations outperform the baseline on both the individual's held-out accuracy (0.983 vs 0.98 and 0.983 vs 0.98) and general accuracy (0.98 vs 0.979 and 0.979 vs 0.979), while sample weighing achieves the best personal accuracy (0.984) but significantly worse general accuracy (0.968).
Load-bearing premise
The comparison assumes that a baseline trained on the global dataset plus one individual's data and tested on a different individual is a fair control, so the gains of the shaping methods are attributed to the training schedule rather than to having seen a real user's data during training.
Editorial extensions
If this is right
- Users can get a model tuned to their own handwriting with a short curriculum phase, without retraining from scratch for everyone.
- If the two winning configurations replicate beyond this dataset, early and late shaping are both viable personalization strategies that do not force a personalization-versus-generalization sacrifice.
- The more aggressively a model is shaped toward one individual (sample weighing), the better it does on that individual and the worse on the general set, so deployment needs a tunable exposure parameter.
- Augmenting an individual's data with only the closest similar samples improves personal accuracy; adding too many or more diverse samples shifts performance back toward the general baseline.
- Reporting minimum, maximum and standard deviation of accuracy across individuals gives a practical fairness check: models with higher average accuracy also showed less spread.
Reading between the lines
- The paper's own evaluation section states that the baseline is tested on a different individual's data; that makes it an open question whether the shaping gains come from the schedule or from exposure to one real user's data, and a direct control would train the baseline on the target user's own training data.
- The same autoencoder similarity could support privacy-preserving personalization: instead of sharing raw data, only latent representations of similar samples would need to be exchanged, though the paper does not explore this.
- The class-count variation scenario was explicitly deferred, so the methods' behavior when individuals differ in how many samples they have per class remains an open testable question.
- A practical extension would be to treat the number of initial or final individual-data iterations as a tunable knob and study how personal and general accuracy trade off as that number grows, which the paper only samples at a few points.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper discusses personalization of deep learning classifiers. It defines objectives (personalized-data accuracy, global-data accuracy, fairness) and proposes two families of methods: three curriculum/shaping schedules (early shaping, sample weighing, transfer learning) and two data-grouping augmentation strategies (individual grouping and single-sample grouping) driven by an autoencoder latent-space similarity. Experiments on the NIST Special Database 19 use a VGG-style CNN and an autoencoder, measuring accuracy on an individual's test data and on a global test set, plus fairness metrics. The central empirical claim is that both curriculum learning and personalized data augmentation improve performance on an individual's data, usually at the cost of global-data performance, with two shaping configurations improving both metrics.
Significance. If the empirical claims hold, the paper would offer a structured comparison of practical personalization techniques and a useful framing of the personalization/global-performance trade-off with fairness metrics. The data-grouping portion uses a reasonable baseline (augmenting the target user's data with random global samples) and reports consistent small gains on individual accuracy. The paper also makes an honest attempt to report variance across users via standard deviations and min/max spreads. However, the shaping evaluations rest on a confounded baseline that never sees the target user's training data, the statistical reporting is too sparse to support the stated significance claims, and all conclusions come from a single dataset and architecture without released code, weakening the generality of the conclusions.
major comments (4)
- [Section IV (Evaluation, shaping baseline)] The shaping baseline is not a valid control. The text states that for shaping, the baseline was trained on the union of DG and the data of one individual D_I', while evaluation was carried out on data from another individual D_I' with D_I' != D_I; in contrast, every shaping method (EarlyShape, SampleWeigh, TransferL) is trained on the target user's own training split. The reported improvements on Acc. on DI (e.g., ES-n-400 at .983 and TL-n-300 at .983 versus the baseline at .980) could therefore be caused simply by the model having seen examples of the target user's handwriting rather than by the curriculum schedule. The footnote that otherwise the baseline would be equivalent to the sample weighing method is not convincing: a baseline trained on the union DG ∪ D_I with uniform batching is a well-defined control and is not equivalent to SW-n-2, which deliberately oversamples D_I every second batch. The shaping experiments must be rerun against a control that is trained on the same data as the shaping methods, with only the ordering/weighing changed, before any claim about the benefit of shaping can be accepted.
- [Section IV (Results and Table I)] The statistical support for the main claims is insufficient. The paper reports p-value thresholds such as '< 0.1' and '< 0.01' without stating the test used, the number of paired observations, whether corrections for multiple comparisons were applied, or the effect size. Several of the headline comparisons (for example, .983 versus .980 on Acc. on DI for shaping, or .979 versus .975 for data grouping) are small absolute differences and are only claimed significant at the p < 0.1 level, which is above the conventional 0.05 threshold. Please provide full test details, confidence intervals, and a multiple-comparison-corrected analysis, and clarify whether these comparisons are across the 100 users or across models.
- [Section III (References to Algorithm 1)] The methods are repeatedly described as 'shown in Algorithm 1' (Section III, subsections A, B, C, and the data-grouping paragraph), but Algorithm 1 is not present in the manuscript. The pseudo-code is central to specifying the training schedules, sample-weighing rule, and the two grouping procedures, so its absence is a substantial reproducibility gap. Either the algorithm must be included, or the text must fully specify these procedures without relying on the missing reference.
- [Section IV (Evaluation setup and conclusions)] The paper draws broad conclusions about personalization from experiments on a single dataset (NIST Special Database 19), a single network architecture, and one variation type (hidden factors, not class-count variation). The abstract and conclusions state general claims such as 'shaping can outperform a baseline on both data' and 'data augmentation by choosing similar samples also increases performance of an individual.' These claims need support from at least one additional dataset or task, or the claims need to be explicitly scoped to the conditions tested. In addition, the absence of released code, random seeds, and exact hyperparameter values for the 'more than 1000 trained models' makes independent verification difficult.
minor comments (4)
- [Abstract and Index Terms] The abstract contains the typo 'curriculuum learning' and the index terms include 'Artificial,Feature Shaping, Intelligence,' which appears to be an artifact of misformatted keywords; please clean these up.
- [Section III (Data grouping) versus Section IV (Evaluation)] Section III says the similarity representation is obtained from an auto-encoder 'trained on all data,' while Section IV says the auto-encoder is trained on DG (and on DI only for sample weighing). Please clarify whether any part of the target user's data, including its test split, is used to train the auto-encoder, since this matters for the data-grouping evaluation.
- [Table I] The notation in Table I (ES-n-400, SW-n-12, TL-n-300, IG-n-5, SG-n-1500, etc.) is not explained before or in the table itself; please add a table note defining n for sample weighing, early shaping, transfer learning, and grouping, and stating the sizes of the augmented datasets for individual grouping.
- [Section IV (Fairness discussion)] The statement that 'Accuracy correlates with fairness' is made largely on the basis of the summarized rows in Table I; please either report a quantitative correlation (e.g., across the 100 users) or soften the claim, since the table only shows aggregate standard deviations and min/max values for a few configurations.
Circularity Check
No significant circularity: the paper's claims are empirical and self-contained, with no derivation chain that reduces to its inputs.
full rationale
This paper is an empirical evaluation of shaping and data-grouping methods for personalizing deep learning models. It contains no mathematical derivation or formal theorem whose conclusion is equivalent to its assumptions by construction. The reported accuracies are measured on held-out test data under fixed experimental protocols. The auto-encoder used for similarity is trained on the global dataset, so the grouping results do not leak the target individual's test labels. Several self-citations appear in the related-work and references sections (e.g., [11], [20], [23], [27]), but none of these is load-bearing for the central empirical claims; they are background or prior applications. The most plausible concern raised by a skeptical reader is the baseline construction for shaping: the baseline is trained on global data plus data from one individual while being evaluated on a different individual's data, whereas the shaping methods train on the target individual's own data. However, this is an experimental-design and control concern about whether the comparison isolates the effect of curriculum scheduling; it is not a circularity in which a fitted parameter is renamed as a prediction, a result is imported from a self-citation chain, or an ansatz is smuggled in as an external theorem. Under the stated hard rules, statistical confounding and lack of a matched control do not constitute circularity. Therefore the appropriate circularity score is 0, with the control issue noted as a correctness risk rather than a circular step.
Assumptions & free parameters
free parameters (5)
- Sample weighing frequency n_I =
2 and 12 (two-thirds and one-half of batches from user data)
- Early shaping iterations n_ES =
40 and 400
- Transfer learning iterations n_TL =
10 and 300
- Data grouping size n =
1500 and 6500 samples (or 5 and 21 individuals)
- Autoencoder latent dimension =
16
assumptions (3)
- domain assumption Inputs from individual datasets and the global dataset are drawn from the same space and contain the same classes.
- domain assumption For the hidden-factors scenario, samples of a class from an individual are on average more similar to each other than samples from the global dataset.
- domain assumption The global dataset provides valuable information beyond the individual's preferred outputs.
Cite this review
Pith. "Pith review of Personalization of Deep Learning." pith.science (2026). https://pith.science/paper/HYCQ2STZ
@misc{pith2026190902803,
author = {Pith},
title = {Pith review of: Personalization of Deep Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/HYCQ2STZ}},
note = {Machine review of arXiv:1909.02803}
}
read the original abstract
We discuss training techniques, objectives and metrics toward personalization of deep learning models. In machine learning, personalization addresses the goal of a trained model to target a particular individual by optimizing one or more performance metrics, while conforming to certain constraints. To personalize, we investigate three methods of ``curriculum learning`` and two approaches for data grouping, i.e., augmenting the data of an individual by adding similar data identified with an auto-encoder. We show that both ``curriculuum learning'' and ``personalized'' data augmentation lead to improved performance on data of an individual. Mostly, this comes at the cost of reduced performance on a more general, broader dataset.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
H. Fan and M. S. Poole, “What is personalization? perspectives on the design and implementation of personalization in information sys- tems,” Journal of Organizational Computing and Electronic Commerce, vol. 16, no. 3-4, pp. 179–202, 2006
work page 2006
-
[2]
Explainable Recommendation: A Survey and New Perspectives
Y . Zhang and X. Chen, “Explainable recommendation: A survey and new perspectives,” arXiv preprint arXiv:1804.11192 , 2018
work page Pith review arXiv 2018
-
[3]
R. Sarikaya, “The technology behind personal digital assistants: An overview of the system architecture and key components,” IEEE Signal Processing Magazine, vol. 34, no. 1, 2017
work page 2017
-
[4]
Deep learning in personalization of cardiovascular stents,
Y . Lee, K. Veerubhotla, M. H. Jeong, and C. H. Lee, “Deep learning in personalization of cardiovascular stents,” Journal of cardiovascular pharmacology and therapeutics , 2019
work page 2019
-
[5]
Machine learning approaches to personalize early prediction of asthma exacerbations,
J. Finkelstein and I. cheol Jeong, “Machine learning approaches to personalize early prediction of asthma exacerbations,” Annals of the New York Academy of Sciences , vol. 1387, no. 1, p. 153, 2017
work page 2017
-
[6]
T. Golany and K. Radinsky, “Pgans: Personalized generative adver- sarial networks for ecg synthesis to improve patient-specific deep ecg classification,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2019
work page 2019
-
[7]
A day of great illumination: B.f. skinner’s discovery of shaping,
G. B. Peterson, “A day of great illumination: B.f. skinner’s discovery of shaping,” Journal of the experimental analysis of behavior , vol. 82, no. 3, pp. 317–328, 2004
work page 2004
-
[8]
Deep learning of representations for unsupervised and transfer learning,
Y . Bengio, “Deep learning of representations for unsupervised and transfer learning,” in Proceedings of ICML Workshop on Unsupervised and Transfer Learning , 2012
work page 2012
Show all 41 references
-
[9]
Privacy-preserving deep learning,
R. Shokri and V . Shmatikov, “Privacy-preserving deep learning,” in Proc. of Conf. on computer and communications security , 2015
2015
-
[10]
Deep learning with differential privacy,
M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang, “Deep learning with differential privacy,” in Proc. of SIGSAC Conf. on Computer and Communications Security , 2016
2016
-
[11]
Personalized explanation in machine learning,
J. Schneider and J. Handali, “Personalized explanation in machine learning,” European Conference on Information Systems (ECIS) , 2019
2019
-
[12]
Fairness in machine learning: Lessons from political phi- losophy,
R. Binns, “Fairness in machine learning: Lessons from political phi- losophy,” in Conference on Fairness, Accountability and Transparency, 2018, pp. 149–159
2018
-
[13]
Counterfactual fair- ness,
M. J. Kusner, J. Loftus, C. Russell, and R. Silva, “Counterfactual fair- ness,” in Advances in Neural Information Processing Systems (NIPS) , 2017
2017
-
[14]
Fairness through awareness,
C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. Zemel, “Fairness through awareness,” inProceedings of the 3rd innovations in theoretical computer science conference , 2012, pp. 214–226
2012
-
[15]
Fairness through computationally-bounded awareness,
M. Kim, O. Reingold, and G. Rothblum, “Fairness through computationally-bounded awareness,” in Advances in Neural Informa- tion Processing Systems , 2018, pp. 4842–4852
2018
-
[16]
Overcoming catastrophic forgetting in neural networks,
J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska et al., “Overcoming catastrophic forgetting in neural networks,” Proc. of the national academy of sciences , 2017
2017
-
[17]
On the importance of initialization and momentum in deep learning,
I. Sutskever, J. Martens, G. Dahl, and G. Hinton, “On the importance of initialization and momentum in deep learning,” in Int. Conf. on machine learning(ICML), 2013
2013
-
[18]
Very deep convolutional networks for large-scale image recognition,
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 , 2014
2014 arXiv
-
[19]
Wide & deep learning for recommender systems,
H.-T. Cheng, L. Koc, J. Harmsen, T. Shaked, T. Chandra, H. Aradhye, G. Anderson, G. Corrado, W. Chai, M. Ispir et al. , “Wide & deep learning for recommender systems,” in Proc. of workshop on Deep Learning for Recommender Systems , 2016
2016
-
[20]
Reconet: An interpretable neural architecture for recommender sys- tems,
F. Fusco, M. Vlachos, V . Vasileiadis, K. Wardatzky, and J. Schneider, “Reconet: An interpretable neural architecture for recommender sys- tems,” in Int. Joint Conference on Artificial Intelligence (IJCAI , 2019
2019
-
[21]
Web mining: Machine learning for web applications,
H. Chen and M. Chau, “Web mining: Machine learning for web applications,” Annual review of information science and technology , vol. 38, no. 1, pp. 289–329, 2004
2004
-
[22]
Power to the people: The role of humans in interactive machine learning,
S. Amershi, M. Cakmak, W. B. Knox, and T. Kulesza, “Power to the people: The role of humans in interactive machine learning,” AI Magazine, vol. 35, no. 4, pp. 105–120, 2014
2014
-
[23]
Human-to-ai coach: Improving human inputs to ai systems,
J. Schneider, “Human-to-ai coach: Improving human inputs to ai systems,” in Symposium on Intelligent Data Analysis (IDA) , 2020
2020
-
[24]
Per- sonalized multitask learning for predicting tomorrow’s mood, stress, and health,
S. A. Taylor, N. Jaques, E. Nosakhare, A. Sano, and R. Picard, “Per- sonalized multitask learning for predicting tomorrow’s mood, stress, and health,” IEEE Transactions on Affective Computing , 2017
2017
-
[25]
Limitations of mass customization,
P. Zipkin, “Limitations of mass customization,” MIT Sloan management review, 2001
2001
-
[26]
Mitchell, J
R. Mitchell, J. Michalski, and T. Carbonell, Machine Learning - An Artificial Intelligence Approach . Springer, 2013
2013
-
[27]
Principles of green data min- ing,
J. Schneider, M. Basalla, and S. Seidel, “Principles of green data min- ing,” in Proc. of Hawaii International Conference on System Sciences (HICSS), 2019
2019
-
[28]
Distributed representations of words and phrases and their composi- tionality,
T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed representations of words and phrases and their composi- tionality,” in Advances in neural information processing systems , 2013, pp. 3111–3119
2013
-
[29]
Small sample learning in big data era,
J. Shu, Z. Xu, and D. Meng, “Small sample learning in big data era,” arXiv preprint arXiv:1808.04572 , 2018
2018 arXiv
-
[30]
Transfer adaptation learning: A decade survey,
L. Zhang, “Transfer adaptation learning: A decade survey,” arXiv preprint, 2019. [Online]. Available: https://arxiv.org/abs/1903.04687
2019 arXiv
-
[31]
Curriculum learning,
Y . Bengio, J. Louradour, R. Collobert, and J. Weston, “Curriculum learning,” in Proc. of international conference on machine learning (ICML), 2009
2009
-
[32]
Deep multimodal learning: A survey on recent advances and trends,
D. Ramachandram and G. W. Taylor, “Deep multimodal learning: A survey on recent advances and trends,” IEEE Signal Processing Magazine, vol. 34, no. 6, pp. 96–108, 2017
2017
-
[33]
Label-free supervision of neural networks with physics and domain knowledge,
R. Stewart and S. Ermon, “Label-free supervision of neural networks with physics and domain knowledge,” in Thirty-First AAAI Conference on Artificial Intelligence , 2017
2017
-
[34]
Growing a brain: Fine- tuning by increasing model capacity,
Y .-X. Wang, D. Ramanan, and M. Hebert, “Growing a brain: Fine- tuning by increasing model capacity,” in Proc. of Conference on Computer Vision and Pattern Recognition (CVPR) , 2017
2017
-
[35]
Correcting sample selection bias by unlabeled data,
J. Huang, A. Gretton, K. Borgwardt, B. Sch ¨olkopf, and A. J. Smola, “Correcting sample selection bias by unlabeled data,” in Advances in neural information processing systems , 2007
2007
-
[36]
Personalization without user interruption: Boosting activity recognition in new subjects using un- labeled data,
R. Fallahzadeh and H. Ghasemzadeh, “Personalization without user interruption: Boosting activity recognition in new subjects using un- labeled data,” in Proceedings of the 8th International Conference on Cyber-Physical Systems, 2017, pp. 293–302
2017
-
[37]
Automated curriculum learning for neural networks,
A. Graves, M. G. Bellemare, J. Menick, R. Munos, and K. Kavukcuoglu, “Automated curriculum learning for neural networks,” in Proc. of Int. Conf. on Machine Learning (ICML) , 2017
2017
-
[38]
Learning by Asking Questions,
I. Misra, R. Girshick, R. Fergus, M. Hebert, A. Gupta, and L. van der Maaten, “Learning by Asking Questions,” in Conf. on Computer Vision and Pattern Recognition (CVPR) , 2018
2018
-
[39]
Self-paced learning with diversity,
L. Jiang, D. Meng, S.-I. Yu, Z. Lan, S. Shan, and A. Hauptmann, “Self-paced learning with diversity,” inAdvances in Neural Information Processing Systems (NIPS) , 2014
2014
-
[40]
Unsupervised domain adaptation for robust speech recognition via variational autoencoder-based data augmentation,
W.-N. Hsu, Y . Zhang, and J. Glass, “Unsupervised domain adaptation for robust speech recognition via variational autoencoder-based data augmentation,” in Automatic Speech Recognition and Understanding Workshop (ASRU), 2017
2017
-
[41]
Autoencoder-based semi- supervised curriculum learning for out-of-domain speaker verification,
S. Zheng, G. Liu, H. Suo, and Y . Lei, “Autoencoder-based semi- supervised curriculum learning for out-of-domain speaker verification,” System, vol. 3, p. 98, 2019
2019
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.