REVIEW 4 major objections 7 minor 42 references
ILASH: A Predictive Neural Architecture Search Framework for Multi-Task Applications
T0 review · 4 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read The paper proposes that layer-sharing plus a learned predictor can make multi-task neural architecture search up to 16 times more energy-efficient.
desk verdict Useful energy-efficiency results for layer-shared multi-task models, but the predictive NAS speedup rests on an unvalidated cross-dataset transfer assumption. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The search is driven by a Goodness metric, $GN = acc \times (1 - G_{th}) + \frac{lr_{index}}{lr_{total}} \times G_{th}$, which balances validation accuracy against the relative depth of the branching layer using a user-set green threshold. ILASH-Pred uses a Decision Tree regressor (Auto-ILASH) trained on logs of heuristic searches; each candidate branching layer is encoded by the kernel size, padding, and stride of that layer and its two neighbors. The regressor predicts GN for every legal branch, the best branch is chosen without training the candidate, and the final branched model is trained once on all tasks together.
What would settle it
Run ILASH-Pred on UTKFace using an Auto-ILASH model trained only on MTFL and CelebA, and compare the chosen branch points, final task accuracies, and total search energy against ILASH-Heu on UTKFace; if the predictor does not beat the heuristic in both accuracy and energy, the transfer premise is unsupported. A simpler check is to measure the Decision Tree's MAE on held-out UTKFace predictions when the model was trained without UTKFace.
Extended reading notes
Core claim
The central claim is that a hybrid layer-sharing architecture, built by a branching heuristic and then searched by a learned goodness predictor, can match the accuracy of a general multi-task NAS like AutoKeras while using 10–16 times less energy, CO2 emission, and search time, and can run at several times the frame rate on edge devices. On the Taskonomy 2D-semantic tasks, the predictive search is about 38 times faster than the paper's own heuristic search with similar final model error. The paper also claims that the final ILASH models cut inference energy by about three times compared with AutoKeras-generated models.
Load-bearing premise
The whole speedup rests on the assumption that a goodness predictor trained on heuristic search logs from some datasets will reliably rank branching choices on a different dataset; the paper never reports the predictor's error on the held-out dataset.
Editorial extensions
If this is right
- Multi-task models for edge devices could be built with a single shared trunk plus task-specific branches, shrinking model size and raising frames per second.
- Architecture search becomes cheap enough to run on modest hardware, lowering the carbon footprint per model.
- The same branching-plus-prediction scheme could be applied to other domains such as speech, video, or NLP once a heuristic search has generated a training set for the predictor.
- For 2D semantic tasks, the paper claims to be the first multi-task NAS that supports them, with roughly 38 times faster search than its own heuristic baseline.
Reading between the lines
- If the transfer assumption holds, the predictor could be reused across similar datasets without re-running the costly heuristic search, but the paper does not report held-out prediction error; reporting that error would show how far the approach generalizes.
- The layer encoding could be enriched with channel counts, receptive field, or FLOPs to make the predictor more portable, which is a natural testable extension.
- The Goodness metric collapses accuracy and layer position into one scalar; a multi-objective view might reveal that accuracy and efficiency trade off differently than GN suggests, especially when different tasks have very different difficulties.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes ILASH, a layer-sharing architecture for multi-task neural networks, and ILASH-NAS, a neural architecture search framework with two modes: ILASH-Heu, which exhaustively evaluates candidate branch points using a hand-defined Goodness metric, and ILASH-Pred, which replaces those evaluations with a machine-learned predictor (Auto-ILASH) trained on data collected by ILASH-Heu. The authors evaluate on UTKFace, MTFL, CelebA, and Taskonomy, compare search efficiency with AutoKeras, and claim up to 16x reductions in energy, time, and CO2 emission during search while keeping final model accuracy stable.
Significance. If the claims hold, the paper offers a practical, energy-conscious NAS approach for multi-task edge applications, with a useful separation between heuristic data collection and predictor-based search. The work deserves credit for measuring inference energy on four real edge devices, for attempting a leave-one-out protocol for the predictor, and for comparing against AutoKeras, which is reasonably identified as the only publicly available multi-task NAS framework. The central contribution, however, rests on two premises that are not yet established: the cross-dataset transferability of Auto-ILASH and a fair amortized accounting of the cost of generating its training data. One results table also appears to duplicate earlier rows, which raises a data-integrity concern that must be resolved before the results can be accepted.
major comments (4)
- [Section IV.A and Table III] Table III reports MAE, MSE, RMSE, and R2 on the merged ILASH Dataset using a 7:3 split. This is an in-distribution evaluation. Algorithm 3, however, applies Auto-ILASH to a dataset it has never seen: UTKFace is predicted after training on MTFL and CelebA, and each Taskonomy split is predicted after training on the other two splits. The paper never reports held-out prediction error or rank correlation on these leave-one-target-out cases. Because the entire speedup of ILASH-Pred over ILASH-Heu rests on the predictor being useful on unseen datasets, please add leave-one-target-out regression and ranking metrics and compare them against a trivial baseline such as predicting the mean GN or using lrindex/lrtotal alone. Without this, the transfer premise is unsupported.
- [Section V.B, Section V.C, and Table VI] The reported search efficiency compares ILASH-Pred runtime and energy (e.g., 1.58 hours and 0.287 kWh for Taskonomy split 1 in Table VI) against ILASH-Heu (60.2 hours and 6.34 kWh). But ILASH-Heu is not merely a baseline to be beaten; it is the data-generation step that produces the ILASH Dataset on which Auto-ILASH is trained. The 38x speedup claimed in Section V.C is therefore the cost of using the predictor after the training data already exist. Please provide an amortized comparison that includes the cost of the ILASH-Heu runs used to create the training data, or state how many target searches are needed for ILASH-Pred to break even. This is load-bearing for the headline 10-16x and 38x efficiency claims.
- [Table V] Table V appears to be a duplicate of Table IV: the Taskonomy split 1 block is identical to the UTKFace block (1.0, 0.957, 351.69, etc.), the split 2 block is identical to the MTFL block, and the split 3 block is identical to the CelebA block. If these are not actual Taskonomy measurements, the Taskonomy inference conclusions in Section V.C are unsupported. Please correct the table or provide the real measured values; if the values are correct, please explain why they coincide exactly.
- [Figure 3 and Tables IV and VI] The claim that final accuracy remains mostly stable and that ILASH-Pred is 10-16x more efficient is made from a single set of runs. No standard deviations, seed variation, or repeated-task-order statistics are reported, even though the paper itself states that task ordering affects results. Table IV reports only energy and FPS, not accuracy; accuracy appears only in Figure 3, which is not reproduced in the text. Please report means and variances over at least three seeds, and include the per-task accuracy values in a table so the accuracy-stability claim can be checked.
minor comments (7)
- [Section II] The text contains typos such as 'verage power' instead of 'average power'; please proofread the manuscript.
- [Section IV.D] The abbreviation 'ILSH Pred' appears several times instead of 'ILASH-Pred'; please make the notation consistent.
- [Equation (5)] The sentence before Equation (5) says 'We calculate the MAE using the following equation,' but the equation defines the MSE; please correct the wording.
- [Algorithm 2] The encoding loop accesses M odel.layers[idx + i] for i = -1 and i = 1, which is out of range when idx is the first or last layer; please specify boundary handling such as clamping or skipping.
- [Table IV, CelebA row] In the ILASH-Heu row for CelebA, the reported KWh-PUE value is 8.31 x 10^-8 while the CO2 emission is 0.793 x 10^-8 lbs, which is inconsistent by a factor of roughly ten; please verify the decimal point.
- [Section IV.A] The Auto-ILASH hyperparameter list for Random Forest mentions 'max features' twice with different values; please clarify the intended setting.
- [Equation (3)] The value of the Goodness weight Gth is never reported, and no sensitivity analysis is provided; since Gth controls the trade-off between accuracy and layer position, please state the value used in all experiments.
Circularity Check
No significant circularity: Auto-ILASH is a surrogate for the paper's own Goodness metric, but the final accuracy and energy claims are externally measured, and the leave-one-out protocol prevents direct target leakage.
full rationale
ILASH-Pred's Auto-ILASH predictor is trained on ILASH-Heu-generated data to approximate the paper-defined Goodness metric (Eq. 3). This is a standard surrogate-model construction rather than a circular reduction: Goodness is defined from validation accuracy and branch depth, not from the predictor itself, and the central claims are evaluated through externally measured final model accuracy, MAE, energy consumption, CO2 emission, and FPS. The leave-one-out protocol in Section IV.A means that, for each target dataset, the Auto-ILASH model is trained without that target's Goodness labels, so the reported final model results are not statistically forced by construction. The paper's self-citations ([34] and [36]) support only background assertions about NAS being resource-intensive and are not load-bearing for the main derivation. The main weaknesses are validation gaps, such as the absence of reported leave-one-target prediction error for Auto-ILASH and the exclusion of the amortized cost of generating the ILASH Dataset from the search-efficiency comparison; these are correctness and completeness concerns, not circularity. No equation reduces to its own input by construction, and no fitted parameter is renamed as an external prediction.
Assumptions & free parameters
free parameters (2)
- Goodness weight Gth
- Branching bounds ll and ul
assumptions (4)
- domain assumption Energy/CO2 are estimated by the linear model of Strubell et al. (Eqs. 1-2) with power draws assumed at 1.5x TDP.
- ad hoc to paper The Goodness metric GN (Eq. 3) is a valid surrogate for the true search objective (accuracy plus efficiency).
- ad hoc to paper The layer encoding in Algorithm 2 (kernel/pad/stride of three adjacent layers) is sufficient to predict GN on unseen tasks.
- ad hoc to paper Heuristic data from other datasets transfers to the target dataset in a leave-one-out scheme.
Cite this review
Pith. "Pith review of ILASH: A Predictive Neural Architecture Search Framework for Multi-Task Applications." pith.science (2026). https://pith.science/paper/WKZY2KDI
@misc{pith2026241202116,
author = {Pith},
title = {Pith review of: ILASH: A Predictive Neural Architecture Search Framework for Multi-Task Applications},
year = {2026},
howpublished = {\url{https://pith.science/paper/WKZY2KDI}},
note = {Machine review of arXiv:2412.02116}
}
read the original abstract
Artificial intelligence (AI) is widely used in various fields including healthcare, autonomous vehicles, robotics, traffic monitoring, and agriculture. Many modern AI applications in these fields are multi-tasking in nature (i.e. perform multiple analysis on same data) and are deployed on resource-constrained edge devices requiring the AI models to be efficient across different metrics such as power, frame rate, and size. For these specific use-cases, in this work, we propose a new paradigm of neural network architecture (ILASH) that leverages a layer sharing concept for minimizing power utilization, increasing frame rate, and reducing model size. Additionally, we propose a novel neural network architecture search framework (ILASH-NAS) for efficient construction of these neural network models for a given set of tasks and device constraints. The proposed NAS framework utilizes a data-driven intelligent approach to make the search efficient in terms of energy, time, and CO2 emission. We perform extensive evaluations of the proposed layer shared architecture paradigm (ILASH) and the ILASH-NAS framework using four open-source datasets (UTKFace, MTFL, CelebA, and Taskonomy). We compare ILASH-NAS with AutoKeras and observe significant improvement in terms of both the generated model performance and neural search efficiency with up to 16x less energy utilization, CO2 emission, and training/search time.
Figures
Reference graph
Works this paper leans on
-
[1]
Ai market will surge to near $1 trillion by 2027, bain says
S. Rai, “Ai market will surge to near $1 trillion by 2027, bain says.” Accessed: 2024-11-06
work page 2027
-
[2]
M. Weber, T. Weiss, F. Gechter, and R. Kriesten, “Approach for improved development of advanced driver assistance systems for future smart mobility concepts,” Autonomous Intelligent Systems, vol. 3, no. 1, p. 2, 2023
work page 2023
-
[3]
J. Dong, S. Chen, M. Miralinaghi, T. Chen, P. Li, and S. Labi, “Why did the ai make that decision? towards an explainable artificial intelligence (xai) for autonomous driving systems,” Transportation research part C: emerging technologies, vol. 156, p. 104358, 2023
work page 2023
-
[4]
Generative ai-empowered simulation for autonomous driving in vehicular mixed reality metaverses,
M. Xu, D. Niyato, J. Chen, H. Zhang, J. Kang, Z. Xiong, S. Mao, and Z. Han, “Generative ai-empowered simulation for autonomous driving in vehicular mixed reality metaverses,” IEEE Journal of Selected Topics in Signal Processing , 2023
work page 2023
-
[5]
P. Ponce, B. Anthony, R. Bradley, J. Maldonado-Romo, J. I. M ´endez, L. Montesinos, and A. Molina, “Developing a virtual reality and ai- based framework for advanced digital manufacturing and nearshoring opportunities in mexico,” Scientific Reports , vol. 14, no. 1, p. 11214, 2024
work page 2024
-
[6]
Federated explainable artificial intelligence (fxai): a digital manufacturing perspective,
A. Kusiak, “Federated explainable artificial intelligence (fxai): a digital manufacturing perspective,” International Journal of Production Re- search, vol. 62, no. 1-2, pp. 171–182, 2024
work page 2024
-
[7]
Tackling prediction uncertainty in machine learning for healthcare,
M. Chua, D. Kim, J. Choi, N. G. Lee, V . Deshpande, J. Schwab, M. H. Lev, R. G. Gonzalez, M. S. Gee, and S. Do, “Tackling prediction uncertainty in machine learning for healthcare,” Nature Biomedical Engineering, vol. 7, no. 6, pp. 711–718, 2023
work page 2023
-
[8]
An enhanced reversible data hiding algo- rithm using deep neural network for e-healthcare,
R. Bhardwaj and I. Tripathi, “An enhanced reversible data hiding algo- rithm using deep neural network for e-healthcare,” Journal of Ambient Intelligence and Humanized Computing , vol. 14, no. 8, pp. 10567– 10585, 2023
work page 2023
Show all 42 references
-
[9]
An intelligent heart disease prediction system based on swarm-artificial neural network,
S. Nandy, M. Adhikari, V . Balasubramanian, V . G. Menon, X. Li, and M. Zakarya, “An intelligent heart disease prediction system based on swarm-artificial neural network,” Neural Computing and Applications , vol. 35, no. 20, pp. 14723–14737, 2023
2023
-
[10]
Embracing the power of ai in retail platform operations: Considering the showrooming effect and consumer returns,
Q. Wang, X. Ji, and N. Zhao, “Embracing the power of ai in retail platform operations: Considering the showrooming effect and consumer returns,” Transportation Research Part E: Logistics and Transportation Review, vol. 182, p. 103409, 2024
2024
-
[11]
Leveraging digital technology in retailing business: Unbox- ing synergy between omnichannel retail adoption and sustainable retail performance,
M. S. Vhatkar, R. D. Raut, R. Gokhale, M. Kumar, M. Akarte, and S. Ghoshal, “Leveraging digital technology in retailing business: Unbox- ing synergy between omnichannel retail adoption and sustainable retail performance,” Journal of Retailing and Consumer Services , vol. 81, p...
2024
-
[12]
Energy and policy consid- erations for deep learning in nlp,
E. Strubell, A. Ganesh, and A. McCallum, “Energy and policy consid- erations for deep learning in nlp,” arXiv:1906.02243, 2019
1906 arXiv
-
[13]
State of IoT—spring 2023
IoT Analytics, “State of IoT—spring 2023.” https://iot-analytics.com/ product/state-of-iot-spring-2023/, 2023. Accessed: November 6, 2024
2023
-
[14]
Mitigating bias in gender, age and ethnicity classification: a multi-task convolution neural network approach,
A. Das, A. Dantcheva, and F. Bremond, “Mitigating bias in gender, age and ethnicity classification: a multi-task convolution neural network approach,” in Proceedings of the european conference on computer vision (eccv) workshops , pp. 0–0, 2018
2018
-
[15]
Facial landmark detection by deep multi-task learning,
Z. Zhang, P. Luo, C. C. Loy, and X. Tang, “Facial landmark detection by deep multi-task learning,” in Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Pro- ceedings, Part VI 13 , pp. 94–108, Springer, 2014
2014
-
[16]
Large-scale celebfaces attributes (celeba) dataset,
Z. Liu, P. Luo, X. Wang, and X. Tang, “Large-scale celebfaces attributes (celeba) dataset,” Retrieved Aug, vol. 15, 2018
2018
-
[17]
Taskonomy: Disentangling task transfer learning,
A. R. Zamir et al., “Taskonomy: Disentangling task transfer learning,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3712–3722, 2018
2018
-
[18]
Autosnn: Towards energy-efficient spiking neural networks,
B. Na, J. Mok, S. Park, D. Lee, H. Choe, and S. Yoon, “Autosnn: Towards energy-efficient spiking neural networks,” in International Conference on Machine Learning , pp. 16253–16269, PMLR, 2022
2022
-
[19]
Autonomous deep learning: A genetic dcnn designer for image classification,
B. Ma et al., “Autonomous deep learning: A genetic dcnn designer for image classification,” vol. 379, pp. 152–161, 2020
2020
-
[20]
Automatically designing cnn architectures using the genetic algorithm for image classification,
Y . Sun et al. , “Automatically designing cnn architectures using the genetic algorithm for image classification,” IEEE transactions on cy- bernetics, vol. 50, no. 9, pp. 3840–3854, 2020
2020
-
[21]
Neural architecture search with reinforcement learning,
B. Zoph and Q. V . Le, “Neural architecture search with reinforcement learning,” arXiv preprint arXiv:1611.01578 , 2016
2016 arXiv
-
[22]
Efficient architecture search by network transformation,
H. Cai, T. Chen, W. Zhang, Y . Yu, and J. Wang, “Efficient architecture search by network transformation,” in Proceedings of the AAAI Confer- ence on Artificial Intelligence , vol. 32, 2018
2018
-
[23]
Evolution of deep convolutional neural networks using cartesian genetic program- ming,
M. Suganuma, M. Kobayashi, S. Shirakawa, and T. Nagao, “Evolution of deep convolutional neural networks using cartesian genetic program- ming,” Evolutionary computation, vol. 28, no. 1, pp. 141–163, 2020
2020
-
[24]
Re- thinking architecture selection in differentiable nas,
R. Wang, M. Cheng, X. Chen, X. Tang, and C.-J. Hsieh, “Re- thinking architecture selection in differentiable nas,” arXiv preprint arXiv:2108.04392, 2021
2021 arXiv
-
[25]
Neural architecture optimization,
R. Luo et al. , “Neural architecture optimization,” Advances in neural information processing systems , vol. 31, 2018
2018
-
[26]
Progressive neural architecture search,
C. Liu et al., “Progressive neural architecture search,” in Proceedings of the European conference on computer vision (ECCV) , pp. 19–34, 2018
2018
-
[27]
Learning transferable architectures for scalable image recognition,
B. Zoph et al. , “Learning transferable architectures for scalable image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , pp. 8697–8710, 2018
2018
-
[28]
Hierarchical representations for efficient architecture search,
H. Liu, K. Simonyan, O. Vinyals, C. Fernando, and K. Kavukcuoglu, “Hierarchical representations for efficient architecture search,” arXiv preprint arXiv:1711.00436, 2017
2017 arXiv
-
[29]
Regularized evolu- tion for image classifier architecture search,
E. Real, A. Aggarwal, Y . Huang, and Q. V . Le, “Regularized evolu- tion for image classifier architecture search,” in Proceedings of AAAI conference on artificial intelligence , vol. 33, pp. 4780–4789, 2019
2019
-
[30]
Darts: Differentiable architecture search,
H. Liu, K. Simonyan, and Y . Yang, “Darts: Differentiable architecture search,” arXiv preprint arXiv:1806.09055 , 2018
2018 arXiv
-
[31]
Auto-keras: An efficient neural architecture search system,
H. Jin, Q. Song, and X. Hu, “Auto-keras: An efficient neural architecture search system,” in ACM SIGKDD international conference on knowledge discovery & data mining , pp. 1946–1956, 2019
1946
-
[32]
Autokeras: An automl library for deep learning,
H. Jin, F. Chollet, Q. Song, and X. Hu, “Autokeras: An automl library for deep learning,” Journal of Machine Learning Research , vol. 24, no. 6, pp. 1–6, 2023
2023
-
[33]
Can gpt-4 perform neural architecture search?,
M. Zheng, X. Su, S. You, F. Wang, C. Qian, C. Xu, and S. Al- banie, “Can gpt-4 perform neural architecture search?,” arXiv preprint arXiv:2304.10970, 2023
2023 arXiv
-
[34]
Lemo-nade: Multi-parameter neural architecture discovery with llms,
M. H. Rahman and P. Chakraborty, “Lemo-nade: Multi-parameter neural architecture discovery with llms,” arXiv preprint arXiv:2402.18443 , 2024
2024 arXiv
-
[35]
Seqnas: Neural architecture search for event sequence classification,
I. Udovichenko et al. , “Seqnas: Neural architecture search for event sequence classification,” IEEE Access, 2024
2024
-
[36]
Un- derstanding the innovations required for a green & secure artificial intelligence paradigm,
M. M. Rizvee, M. H. Rahman, P. Chakraborty, and S. Shomaji, “Un- derstanding the innovations required for a green & secure artificial intelligence paradigm,” in 2023 IEEE 16th Dallas Circuits and Systems Conference (DCAS), pp. 1–6, IEEE, 2023
2023
-
[37]
Energy and policy consider- ations for modern deep learning research,
E. Strubell, A. Ganesh, and A. McCallum, “Energy and policy consider- ations for modern deep learning research,” in Proceedings of the AAAI conference on artificial intelligence , vol. 34, pp. 13693–13696, 2020
2020
-
[38]
Why intel processors draw more power than expected: Tdp and turbo explained,
I. Cutress, “Why intel processors draw more power than expected: Tdp and turbo explained,” 2022
2022
-
[39]
Fast graspnext: A fast self-attention neural network architecture for multi-task learning in computer vision tasks for robotic grasping on the edge,
A. Wong et al. , “Fast graspnext: A fast self-attention neural network architecture for multi-task learning in computer vision tasks for robotic grasping on the edge,” in CVPR, pp. 2292–2296, 2023
2023
-
[40]
Knowledge distillation for multi-task learning,
W.-H. Li and H. Bilen, “Knowledge distillation for multi-task learning,” in Computer Vision–ECCV 2020 Workshops: Glasgow, UK, August 23– 28, 2020, Proceedings, Part VI 16 , pp. 163–176, Springer, 2020
2020
-
[41]
PyJoules: Python-based energy measurement library for various domains including NVIDIA GPUs
powerapi ng, “PyJoules: Python-based energy measurement library for various domains including NVIDIA GPUs.” https://github.com/ powerapi-ng/pyJoules, 2024. Accessed: 2024-01-10
2024
-
[42]
Mobilenets: Efficient convolutional neural net- works for mobile vision applications,
A. G. Howard et al. , “Mobilenets: Efficient convolutional neural net- works for mobile vision applications,” arXiv:1704.04861, 2017
2017 arXiv
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.