REVIEW 5 major objections 5 minor 46 references
Blockchain-based Crowdsourced Deep Reinforcement Learning as a Service
T0 review · 5 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read This paper proposes a blockchain-based crowdsourcing framework that treats deep reinforcement learning as a service, letting requesters hire expert workers for DRL training and for sharing pre-trained models.
desk verdict New service model worth taking seriously, but the experiments are component demos, not an end-to-end proof of efficacy. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing machinery is the QoS-driven greedy knapsack allocation executed by smart contracts. For DRL training, each worker's QoS is $\text{Exp} \times \text{Rep} \times R \times CC$, where $CC = \frac{2}{\pi} \tan^{-1}(w_1 N_{\text{CPU}})$ captures the saturating benefit of parallel CPU cores; the requester's constraints filter workers by reputation, rating, domain, CPU, RAM, and accepted GPU series. For model sharing, QoS is $\frac{\text{Exp} \times \text{Rep} \times R}{1 + S}$, where $S$ is a weighted sum of absolute differences between environment attributes of the shared model and the task. Three smart contracts, named Users Manager, Tasks Manager, and Models Manager, store worker, task, and model records on-chain and use IPFS content identifiers for model files; allocation functions sort candidates greedily at $O(N \log N)$, a complexity chosen for blockchain deployability.
What would settle it
Let workers register with inflated hardware and model-environment claims, run the greedy allocation, and audit the actual resources and model metadata; if misreporting workers are selected and then fail to train or share usable models at the promised speed, the central claim that QoS metrics pick effective workers would be falsified. A cleaner test is to compare training outcomes and wall time for identical tasks allocated with truthful versus inflated profiles, holding everything else fixed.
Extended reading notes
Core claim
On its own terms, the paper's central claim is that a consortium blockchain can autonomously manage the full lifecycle of crowdsourced DRL: user registration, task submission, QoS-based worker selection, model delivery via IPFS, feedback, and payment. The training-task QoS is the product of normalized expertise, reputation, rating, and a computational-capability score that saturates with CPU count and checks GPU and RAM thresholds; the model-sharing QoS divides the same human-factor product by one plus a weighted environment-similarity distance. The evaluation reports that more CPU cores mainly reduce wall-clock training time, GPU use increases training steps by up to 19 times, and expert models from closer environments improve convergence, while greedy recruitment beats genetic, particle-swarm, and ant-colony benchmarks on group QoS and is far faster, and smart-contract gas costs are modest.
Load-bearing premise
The framework assumes workers truthfully report their CPU cores, RAM, GPU series, and the environment attributes of their shared models, and that these self-declared values feed the QoS scores; there is no verification, benchmark, or proof-of-computation described.
Editorial extensions
If this is right
- A requester with no DRL expertise can submit a problem description and receive a trained model from a recruited expert worker, with payments and records handled automatically by smart contracts.
- Workers with more CPU cores and GPUs receive higher QoS scores, and the experiments show these resources mainly compress wall-clock training time rather than change convergence behavior.
- Model sharing with environment-similar pre-trained models speeds learning; the closer the shared model's environment attributes are to the requester's, the faster the convergence.
- Greedy recruitment yields higher group-average QoS and is orders of magnitude faster than genetic, particle-swarm, and ant-colony benchmarks, making on-chain allocation practical.
- Smart-contract gas costs are low enough to deploy on a consortium blockchain, which avoids single-point-of-failure and provides traceable, autonomous task execution.
Reading between the lines
- An unstated corollary is that the framework's integrity depends on a reputation and verification layer that the paper does not design; without one, workers can inflate their reported hardware or model attributes to win tasks.
- The environment-similarity metric assumes that a fixed, weighted set of environment attributes can represent any DRL domain, which may be difficult to generalize beyond the three tested applications.
- The framework could be extended to include on-chain quality verification of submitted models, such as benchmark reward checks, but the paper leaves that to future work.
- A testable extension would be to simulate strategic workers who misreport capabilities and measure the resulting degradation in training outcomes, which would reveal how much of the claimed efficacy relies on honest reporting.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes a consortium-blockchain-based crowdsourced Deep Reinforcement Learning as a Service (DRLaaS) framework that supports two task types: DRL training and model sharing. It defines QoS metrics for worker recruitment (Eqs. 5-8), uses a greedy optimization for worker and model allocation, describes three smart contracts (UMC, TMC, MMC) with IPFS-based model storage, and reports experiments on CPU/GPU effects, model similarity in demonstration cloning, recruitment optimization on a synthetic 600-worker dataset, and smart-contract gas costs. The central claim, stated in the abstract and conclusion, is that the framework provides effective DRL training and model sharing services and that its efficacy is proven by the experiments.
Significance. The paper addresses a real gap: making DRL accessible through crowdsourced expertise, computational resources, and pre-trained models. Its main contributions are a concrete system architecture for DRLaaS, DRL-specific worker recruitment metrics, and a smart-contract design for task/model allocation. The component-level experiments on three multi-agent DRL environments (target localization, maze cleaning, fleet coordination) usefully demonstrate that CPU parallelism and GPU availability affect DRL training throughput and that similar pre-trained models can help demonstration cloning. The gas-cost analysis gives a rough feasibility check for the smart contracts. However, the significance is currently limited by the absence of an integrated evaluation of the proposed service loop and by unverified assumptions about truthful worker reporting and the functional forms of the QoS metrics.
major comments (5)
- [Section 5] The evaluation does not exercise the proposed DRLaaS framework as an integrated service. The full loop of requester task submission, smart-contract-based allocation, actual worker training or model sharing, IPFS-based model return, payment, and rating is never run; the smart contracts appear only as gas-cost estimates in Table 5 and are not deployed or executed on a ledger. Consequently, the abstract's statement that the framework 'is tested on several DRL applications, proving its efficacy' is not supported by the presented evidence, which validates components in isolation rather than the system-level service claim.
- [Section 4.2.1 and Section 5.2.1] The validation of the computational-capability component is partly tautological. The QoS in Eq. (6) includes CCWj from Eq. (5), which is built on the number of CPU cores, and the task constraints in Section 4.2.1 include GPU availability; Figs. 6 and 7 then show that more CPU cores and the presence of a GPU speed up DRL training. This confirms that these attributes are relevant, but it does not validate the specific functional form of Eq. (5) or the multiplicative combination in Eq. (6). A simple hardware filter would likely produce the same training-speed differences, so the experiments do not show that the QoS metric selects better workers beyond what such a filter would achieve.
- [Sections 4.4-4.5, Tables 2-4] The framework's correctness depends on workers truthfully reporting their computational capabilities (CPU cores, RAM, GPU series) and model environment details, but no verification mechanism, benchmark, or proof-of-computation is described. If workers misreport these attributes, the greedy allocation in Section 4.3 selects the wrong workers and the QoS values become meaningless. The conclusion mentions future quality-control mechanisms, but the current design does not mitigate this dependency, which is load-bearing for the claim that the service is effective.
- [Section 5.2.2 and Eq. (7)] The model similarity metric is not actually validated as a ranking function. Figure 8 compares a few hand-picked expert models by their environment labels (e.g., 2A2W vs 3A3W), showing qualitatively that closer environments help; however, the paper never computes S(m,Ti) for these models, never varies the weights wn in Eq. (7), and never tests whether the top-ranked models under Eq. (7) yield better training outcomes. The model-sharing allocation process is therefore unsubstantiated.
- [Section 5.2.3, Figs. 9-10] The comparison against GA, PSO, and ACO on a synthetic 600-worker dataset provides weak evidence for the recruitment method. When the objective is the additive sum of worker QoS values, selecting a group of size N is achieved exactly by greedy top-k selection, so it is unsurprising that greedy outperforms general-purpose metaheuristics on that objective. Moreover, in Fig. 10b the 'workers' are simulated by running training on hardware with the declared attributes of selected workers; no actual worker accepts a task, trains a model, or uploads it, so the experiment does not demonstrate distributed crowdsourced training.
minor comments (5)
- [Section 4.1, Table 1] The notation for worker attributes is inconsistent: the text uses subscripts such as DomTi in Rep Wj Tr-DomTi while Table 1 defines DomTi as the task domain, and the worker tuple mixes DomWj and DomTj; please unify the notation.
- [Section 4.2.1, Eqs. (2)-(3)] The commitment rate CM and completion rate CP are undefined when the total assigned tasks or accepted tasks are zero; initial values for new workers should be specified explicitly.
- [Figure 7 caption] The caption contains a typo: the second subfigure is labeled '(a) varying the use of GPU' but should be '(b)'.
- [Section 4.2.2, Eq. (7)] The environment attributes Fn in Eq. (7) appear to be raw counts (e.g., number of agents, number of walls), so the weighted sum of absolute differences is scale-dependent; the paper should discuss normalization before summing attributes of different units or magnitudes.
- [Section 5.2.4, Table 5] The table entry 'updateInfo)' has a typo and should read 'updateInfo()'; also, the discussion would benefit from stating whether the gas costs are estimates obtained from a simulator or from a deployed contract.
Circularity Check
No significant circularity: the QoS metrics are design heuristics with independent outcome-based checks; no fitted parameter is renamed as a prediction, and self-citations are not load-bearing.
full rationale
The paper's central claim is that its blockchain-based DRLaaS framework provides effective DRL training and model-sharing services. The QoS for training tasks (Eq. 6) is a product of expertise, reputation, rating, and a computational-capability term (Eq. 5) defined as a monotone transform of the worker's CPU-core count, with GPU and RAM constraints. Section 5.2.1 then measures how CPU-core count and GPU availability affect DRL training speed and convergence. This is an empirical sanity check of the metric's premise, not a circular prediction: no constants are fitted to the validation data, and the w1 parameter in Eq. 5 is a design choice rather than an estimate obtained from Figs. 6-7. The model-similarity metric (Eq. 7) is validated independently in Fig. 8, where environment attributes are varied and transfer benefit is measured by episode length, which is not part of the metric itself. Fig. 10a does compare the proposed QoS against benchmarks on the same QoS objective, which is somewhat tautological since the proposed metric contains additional components by construction, but Fig. 10b supplies an external outcome-based check by training the selected workers' configurations and measuring episode length, an outcome not contained in the QoS formula. The smart-contract evaluation is a gas-cost complexity analysis, not a derivation of service quality. Self-citations such as [17], [34], and [38] provide environment implementations and the demonstration-cloning method, but they are not used to assert uniqueness, to forbid alternatives, or to import an unverified load-bearing premise into the recruitment derivation. The abstract's claim that the framework 'is tested on several DRL applications, proving its efficacy' is broader than the component-level evidence, but that is an evidentiary/support issue, not circularity. No load-bearing step reduces, by the paper's own equations or by self-citation, to its own inputs.
Assumptions & free parameters
free parameters (2)
- w1
- wn
assumptions (4)
- domain assumption DRL environments in the same domain share a pre-defined set of numeric attributes, and a weighted L1 distance between attribute vectors predicts transfer usefulness.
- domain assumption The worker recruitment problem can be modeled as a knapsack with independent worker QoS values, so greedy top-k selection is appropriate.
- domain assumption A worker can take a textual problem description and successfully design and train a DRL solution without any additional validation.
- domain assumption Computational capability saturates with the number of CPU cores and is adequately modeled by a scaled arctan function.
Cite this review
Pith. "Pith review of Blockchain-based Crowdsourced Deep Reinforcement Learning as a Service." pith.science (2026). https://pith.science/paper/5KS4NUD4
@misc{pith2026250116369,
author = {Pith},
title = {Pith review of: Blockchain-based Crowdsourced Deep Reinforcement Learning as a Service},
year = {2026},
howpublished = {\url{https://pith.science/paper/5KS4NUD4}},
note = {Machine review of arXiv:2501.16369}
}
read the original abstract
Deep Reinforcement Learning (DRL) has emerged as a powerful paradigm for solving complex problems. However, its full potential remains inaccessible to a broader audience due to its complexity, which requires expertise in training and designing DRL solutions, high computational capabilities, and sometimes access to pre-trained models. This necessitates the need for hassle-free services that increase the availability of DRL solutions to a variety of users. To enhance the accessibility to DRL services, this paper proposes a novel blockchain-based crowdsourced DRL as a Service (DRLaaS) framework. The framework provides DRL-related services to users, covering two types of tasks: DRL training and model sharing. Through crowdsourcing, users could benefit from the expertise and computational capabilities of workers to train DRL solutions. Model sharing could help users gain access to pre-trained models, shared by workers in return for incentives, which can help train new DRL solutions using methods in knowledge transfer. The DRLaaS framework is built on top of a Consortium Blockchain to enable traceable and autonomous execution. Smart Contracts are designed to manage worker and model allocation, which are stored using the InterPlanetary File System (IPFS) to ensure tamper-proof data distribution. The framework is tested on several DRL applications, proving its efficacy.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
M. Shurrab, R. Mizouni, S. Singh, H. Otrok, Reinforcement learning framework for uav-based target localization applications, Internet of Things (2023) 100867
work page 2023
-
[2]
G.-P. Antonio, C. Maria-Dolores, Multi-agent deep reinforcement learning to manage connected autonomous vehicles at tomorrow’s intersections, IEEE Trans- actions on Vehicular Technology 71 (2022) 7033–7043
work page 2022
-
[3]
C. Berner, et al., Dota 2 with large scale deep reinforcement learning, arXiv preprint arXiv:1912.06680 (2019)
arXiv 2019
-
[4]
Li, Deep reinforcement learning: An overview, arXiv preprint arXiv:1701.07274 (2017)
Y . Li, Deep reinforcement learning: An overview, arXiv preprint arXiv:1701.07274 (2017)
arXiv 2017
-
[5]
S. Gronauer, K. Diepold, Multi-agent deep reinforcement learning: a survey, Artificial Intelligence Review (2022) 1–49. 30
work page 2022
-
[6]
Q. Wang, W. Ma, W. Wang, B-lnn: Inference-time linear model for secure neural network inference, Information Sciences 638 (2023) 118966
work page 2023
-
[7]
Z. Pan, J. Zeng, R. Cheng, H. Yan, J. Li, Pnas: A privacy preserving framework for neural architecture search services, Information Sciences 573 (2021) 370– 381
work page 2021
-
[8]
M. Ribeiro, K. Grolinger, M. A. Capretz, Mlaas: Machine learning as a service, in: 2015 IEEE 14th international conference on machine learning and applica- tions (ICMLA), IEEE, pp. 896–902
work page 2015
Show all 46 references
-
[9]
M. Aly, K. H. Rahouma, S. M. Ramzy, Pay attention to the speech: Covid- 19 diagnosis using machine learning and crowdsourced respiratory and speech recordings, Alexandria Engineering Journal 61 (2022) 3487–3500
2022
-
[10]
Y . Dong, L. Jiang, C. Li, Improving data and model quality in crowdsourcing using co-training-based noise correction, Information Sciences 583 (2022) 174– 188
2022
-
[11]
Washington, E
P. Washington, E. Leblanc, K. Dunlap, Y . Penev, A. Kline, K. Paskov, M. W. Sun, B. Chrisman, N. Stockham, M. Varma, et al., Precision telemedicine through crowdsourced machine learning: testing variability of crowd workers for video- based autism feature recognition, Journal ...
2020
-
[12]
Zhang, L
Y . Zhang, L. Jiang, C. Li, Instance redistribution-based label integration for crowdsourcing, Information Sciences 674 (2024) 120702
2024
-
[13]
Mart ´ın-Morat´o, A
I. Mart ´ın-Morat´o, A. Mesaros, Strong labeling of sound events using crowd- sourced weak labels and annotator competence estimation, IEEE /ACM Transac- tions on Audio, Speech, and Language Processing 31 (2023) 902–914
2023
-
[14]
Elsokkary, H
N. Elsokkary, H. Otrok, S. Singh, R. Mizouni, H. Barada, M. Omar, Crowd- sourced last mile delivery: Collaborative workforce assignment, Internet of Things 22 (2023) 100692
2023
-
[15]
Alagha, S
A. Alagha, S. Singh, H. Otrok, R. Mizouni, RFLS-resilient fault-proof localiza- tion system in IoT and crowd-based sensing applications, Journal of Network and Computer Applications 170 (2020)
2020
-
[16]
Liang, W
D. Liang, W. Cao, Z. Xu, M. Wang, A novel approach of two-stage three-way co-opetition decision for crowdsourcing task allocation scheme, Information Sci- ences 559 (2021) 191–211
2021
-
[17]
Alagha, S
A. Alagha, S. Singh, R. Mizouni, J. Bentahar, H. Otrok, Target localization us- ing multi-agent deep reinforcement learning with proximal policy optimization, Future Generation Computer Systems 136 (2022) 342–357
2022
-
[18]
Nair, et al., Overcoming exploration in reinforcement learning with demon- strations, in: 2018 IEEE international conference on robotics and automation (ICRA), pp
A. Nair, et al., Overcoming exploration in reinforcement learning with demon- strations, in: 2018 IEEE international conference on robotics and automation (ICRA), pp. 6292–6299. 31
2018
-
[19]
Kadadha, S
M. Kadadha, S. Singh, R. Mizouni, H. Otrok, A context-aware blockchain-based crowdsourcing framework: Open challenges and opportunities, IEEE Access (2022)
2022
-
[20]
Xidias, P
E. Xidias, P. Zacharia, A. Nearchou, Path planning and scheduling for a fleet of autonomous vehicles, Robotica 34 (2016) 2257–2273
2016
-
[21]
Jiang, C
S. Jiang, C. Amato, Multi-agent reinforcement learning with directed exploration and selective memory reuse, in: Proceedings of the 36th annual ACM symposium on applied computing, pp. 777–784
-
[22]
S. Zhao, M. Talasila, G. Jacobson, C. Borcea, S. A. Aftab, J. F. Murray, Pack- aging and sharing machine learning models via the acumos ai open platform, in: 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), IEEE, pp. 841–846
2018
-
[23]
Kumar, R
A. Kumar, R. Tourani, M. Vij, S. Srikanteswara, Sclera: A framework for privacy- preserving mlaas at the pervasive edge, in: 2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other A ffiliated Events (PerCom Workshops), IEEE, pp. 175–180
2022
-
[24]
Singh, J
S. Singh, J. Thaliath, I. F. Siddiqui, A. Jain, S. Yoon, M. Attique, N. M. F. Qureshi, Machine learning as a service for beyond 5g networks, in: 2022 IEEE Globecom Workshops (GC Wkshps), IEEE, pp. 455–460
2022
-
[25]
Graur, D
D. Graur, D. Aymon, D. Kluser, T. Albrici, C. A. Thekkath, A. Klimovic, Cachew: Machine learning input data processing as a service, in: 2022 USENIX Annual Technical Conference (USENIX ATC 22), pp. 689–706
2022
-
[26]
Abououf, H
M. Abououf, H. Otrok, R. Mizouni, S. Singh, E. Damiani, How artificial intelli- gence and mobile crowd sourcing are inextricably intertwined, IEEE Network 35 (2020) 252–258
2020
-
[27]
Y . Ren, W. Liu, A. Liu, T. Wang, A. Li, A privacy-protected intelligent crowd- sourcing application of iot based on the reinforcement learning, Future generation computer systems 127 (2022) 56–69
2022
-
[28]
Abououf, S
M. Abououf, S. Singh, H. Otrok, R. Mizouni, E. Damiani, Machine learning in mobile crowd sourcing: A behavior-based recruitment model, ACM Transactions on Internet Technology (TOIT) 22 (2021) 1–28
2021
-
[29]
Puttinaovarat, P
S. Puttinaovarat, P. Horkaew, Flood forecasting system based on integrated big and crowdsource data by using machine learning techniques, IEEE Access 8 (2020) 5885–5905
2020
-
[30]
J. C. Chang, S. Amershi, E. Kamar, Revolt: Collaborative crowdsourcing for labeling machine learning datasets, in: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 2334–2346. 32
2017
-
[31]
R. S. Sutton, A. G. Barto, Reinforcement learning: An introduction, MIT press, 2018
2018
-
[32]
H. Sami, H. Otrok, J. Bentahar, A. Mourad, E. Damiani, Reward shaping using convolutional neural network, Information Sciences 648 (2023) 119481
2023
-
[33]
H. Sami, J. Bentahar, A. Mourad, H. Otrok, E. Damiani, Graph convolutional recurrent networks for reward shaping in reinforcement learning, Information Sciences 608 (2022) 63–80
2022
-
[34]
Alagha, R
A. Alagha, R. Mizouni, J. Bentahar, H. Otrok, S. Singh, Multi-agent deep re- inforcement learning with demonstration cloning for target localization, IEEE Internet of Things Journal (2023)
2023
-
[35]
Damani, Z
M. Damani, Z. Luo, E. Wenzel, G. Sartoretti, Primal 2: Pathfinding via reinforce- ment and imitation multi-agent learning-lifelong, IEEE Robotics and Automation Letters 6 (2021) 2666–2673
2021
-
[36]
Alagha, R
A. Alagha, R. Mizouni, S. Singh, H. Otrok, A. Ouali, SDRS: A stable data- based recruitment system in IoT crowdsensing for localization tasks, Journal of Network and Computer Applications 177 (2021) 102968
2021
-
[37]
Kadadha, H
M. Kadadha, H. Otrok, R. Mizouni, S. Singh, A. Ouali, On-chain behavior predic- tion machine learning model for blockchain-based crowdsourcing, Future Gener- ation Computer Systems 136 (2022) 170–181
2022
-
[38]
Alagha, J
A. Alagha, J. Bentahar, H. Otrok, S. Singh, R. Mizouni, Blockchain-assisted demonstration cloning for multi-agent deep reinforcement learning, IEEE Inter- net of Things Journal (2023)
2023
-
[39]
Alagha, S
A. Alagha, S. Singh, H. Otrok, R. Mizouni, Influence-and interest-based worker recruitment in crowdsourcing using online social networks, IEEE Transactions on Network and Service Management (2022)
2022
-
[40]
H. Sami, R. Mizouni, H. Otrok, S. Singh, J. Bentahar, A. Mourad, Learnchain: Transparent and cooperative reinforcement learning on blockchain, Future Gen- eration Computer Systems 150 (2024) 255–271
2024
-
[41]
Benet, Ipfs-content addressed, versioned, p2p file system, arXiv preprint arXiv:1407.3561 (2014)
J. Benet, Ipfs-content addressed, versioned, p2p file system, arXiv preprint arXiv:1407.3561 (2014)
2014 arXiv
-
[42]
Alagha, S
A. Alagha, S. Singh, R. Mizouni, A. Ouali, H. Otrok, Data-driven dynamic active node selection for event localization in IoT applications-a case study of radiation localization, IEEE Access 7 (2019) 16168–16183
2019
-
[43]
Y . Wang, Y . Gao, Y . Li, X. Tong, A worker-selection incentive mechanism for optimizing platform-centric mobile crowdsourcing systems, Computer Networks 171 (2020) 107144. 33
2020
-
[44]
Y . Wang, C. Zhao, S. Xu, Method for spatial crowdsourcing task assignment based on integrating of genetic algorithm and ant colony optimization, IEEE Access 8 (2020) 68311–68319
2020
-
[45]
Wehbi, S
O. Wehbi, S. Arisdakessian, O. A. Wahab, H. Otrok, S. Otoum, A. Mourad, M. Guizani, Fedmint: Intelligent bilateral client selection in federated learning with newcomer iot devices, IEEE Internet of Things Journal (2023)
2023
-
[46]
Chahoud, H
M. Chahoud, H. Sami, A. Mourad, S. Otoum, H. Otrok, J. Bentahar, M. Guizani, On-demand-fl: A dynamic and e fficient multi-criteria federated learning client deployment scheme, IEEE Internet of Things Journal (2023). 34
2023
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.