Neural-Network-Driven Reward Prediction as a Heuristic: Advancing Q-Learning for Mobile Robot Path Planning

Hong Liu; Kaijie Yun; Yang Liu; Yiming Ji; Zongwu Xie

REVIEW 5 major objections 5 minor 1 cited by

Neural-Network-Driven Reward Prediction as a Heuristic: Advancing Q-Learning for Mobile Robot Path Planning

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Learned path priors cut Q-learning convergence steps by 90%

desk verdict A plausible use of learned path priors to speed up tabular Q-learning, but the headline 90% claim rests on an unspecified resolution mapping and an unfair baseline formulation. read the letter →

arxiv 2412.12650 v1 pith:QEVDMBYQ submitted 2024-12-17 cs.RO cs.AIcs.LG

Yiming Ji , Kaijie Yun , Yang Liu , Zongwu Xie , Hong Liu This is my paper · ORCID

classification cs.ROcs.AIcs.LG

keywords Q-learningpathplanningmobilerobotneuralnetworkpredictionrewardshapingQ-tableinitializationconvergenceaccelerationCNN

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that a pretrained neural network's predictions about where the optimal path lies can serve as prior knowledge that makes Q-learning converge much faster in mobile-robot path planning. It introduces a dual-output network, NDR, that predicts a narrow 'guideline' distribution and a broad 'region' distribution of the optimal path. These outputs are converted into a continuous reward function and a biased Q-table initialization. On eight 50x50 grid maps, the resulting NDR-QL method converges in about a tenth of the steps of plain Q-learning and outperforms two previously improved Q-learning methods on both convergence speed and path length.

What carries the argument

The central mechanism is the dual-output NDR model, a CNN with an STDC backbone, separate start/end input channels, attention fusion modules, and a guideline decoder. Its 'guideline' output feeds Eq. (3) to define a dense reward field over the map, and its 'region' output, after an adaptive thresholding step that ensures start-end connectivity, initializes the Q-table via Eq. (6). These two injections of learned prior information carry the acceleration; the ablation study shows that each alone gives 40–90% convergence improvement and that combining them with a distance-based heuristic achieves the fastest convergence.

What would settle it

One concrete test is to run NDR-QL on a 50x50 map where the adaptive threshold procedure never produces a region mask that connects start and end; if the method then fails to beat distance-based initialization, the connectivity assumption is exposed. A second is to compare convergence when the NDR outputs are properly resampled and aligned to the 50x50 grid versus used as-is; a significant difference would indicate the current pipeline is misaligned. A third is to check whether the learned reward misleads the agent when multiple disconnected optimal corridors exist, since the guideline is trained to mark a single narrow route.

Watch

Extended reading notes

Core claim

The central claim is that neural-network-driven prior information, in the form of per-cell probability distributions of the optimal path, can replace distance-based heuristics in Q-learning and accelerate convergence substantially. The NDR model separates start and end points into their own input channels and fuses low- and high-level features to predict a guideline (a narrow distribution intended to mark the optimal route) and a region (a broad distribution that reliably links start and end). The guideline is scaled by the maximum reward to define a continuous reward at every state, and the region is thresholded into a mask that initializes the Q-table toward promising areas. In the reported experiments, NDR-QL reduces convergence steps by an average of 90% compared with baseline Q-learning, and it yields better path quality metrics than O-QL and IQL on most of the eight test maps.

Load-bearing premise

The load-bearing premise is that the NDR model's per-cell outputs, trained on 201x201 maps, remain valid reward values and mask values when the Q-learning grid is downscaled to 50x50 with no resampling or alignment step, and that the adaptive threshold search always finds a threshold that connects the start and end points.

Editorial extensions

If this is right

If the central claim holds, any neural model that can output a reliable path-region probability map could serve as a drop-in warm start for tabular Q-learning in grid-world navigation, not just the specific NDR architecture.
The reported 90% convergence reduction suggests that learned spatial priors can substantially alleviate the curse of dimensionality in discrete reinforcement-learning planning, at least on small grid maps.
The ablation shows that distance-based Q-table initialization can hurt performance on maps with dead-ends, whereas learned-region initialization consistently helps, implying that prior knowledge should encode map structure, not just geometry.
Because the guideline and region priors are computed once at the start, the method adds one forward pass of the network before training and then costs the same per episode as ordinary Q-learning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension the authors do not explore is using the same prior to warm-start deep RL agents, for example by initializing the value network or seeding the replay buffer with transitions from the region, rather than a tabular Q-table.
The reported gains are on maps downscaled from 201x201 to 50x50; an open question is whether the benefit persists at full resolution or when the map changes online, since the prior is computed from a static map.
Because the region F1 score drops from roughly 93% on seen maps to roughly 72% on unseen maps, the achievable speedup in novel environments may be lower than the reported 90% average, and a robustness study across map distributions would quantify that gap.
The adaptive threshold search assumes that binarizing the region output can always connect start and end; if a map yields no such threshold, the Q-table initialization would fall back to the distance-based term, so the method's benefit on such maps remains untested.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Readable paper, and the core idea is reasonable: take a network that predicts both a narrow 'guideline' and a broader 'region' for the optimal path, use the guideline to shape rewards and the region to bias the initial Q-table. To my knowledge this is the first time both outputs are used this way for Q-learning, and the ablation in Table IV is genuinely useful — it shows each component helps independently and that combining them works best. The architectural tweaks give modest F1 gains over RGP, which is fine.

The problems are all in the empirical support, not the idea.

The biggest one: the NDR model is trained and evaluated at 201x201, but the Q-learning grid is 50x50. I cannot find any resampling or coordinate alignment step between the network outputs and Eqs. 3 and 6. If the outputs are used at native resolution on a different grid, the reward and initialization are not well-defined, and the adaptive threshold search is also ambiguous. This is fixable with a sentence, but without it the central 90% claim is not reproducible from the text.

Second, Eq. 2 is not a distance reward: the exponent has no absolute values or squares, so rc can exceed rmax and rewards increase with distance from the target in some quadrants. This handicaps the D-CRF baseline, and since that baseline is used in the comparison and ablation, the reported advantage of the NDR reward is partly an artifact. Correct the formula or justify it.

Third, the paper compares against 'IQL' but strips it of its learning-rate schedule and action-selection policy. That is transparently stated, but then the name IQL overstates the comparison. Call it IQL-variant and note the full method would likely do better.

Fourth, all results are single runs with no error bars. Q-learning with epsilon-greedy is noisy; the 90% number needs seeds and variance. And Table II has a contradictory point: on MAP-5, NDR-QL's shortest path is 154 while plain QL gets 112. That needs an explanation or the claim about path quality needs qualification.

On circularity: yes, the heuristic is trained on ground-truth optimal paths, so the speedup is partly by construction. That's expected for a learned heuristic, but it would be honest to state the cost of building the heuristic and discuss what happens when the NDR generalizes imperfectly to new maps.

Who is this for? People working on learning-based priors for classical planners. The idea is plausible and the paper is understandable. It deserves a serious referee, but it needs a major revision before the numbers can be trusted. I'd send it to review, not desk reject.

Referee Report

5 major / 5 minor

Summary. The paper proposes NDR-QL, an extension of tabular Q-learning for mobile robot path planning that uses a trained neural network (the NDR model) to predict two types of prior information: a narrow "guideline" distribution and a broader "region" distribution. The guideline is used to define a continuous reward function (Eq. 3), and the region is used to initialize the Q-table with a binary mask (Eqs. 6-7). The NDR model itself modifies prior dual-output predictors by separating start/end channels and adding an attention fusion module. Experiments on public grid-map datasets report that NDR improves F1 scores over RGP by up to 5% on seen data and 4% on unseen data, and that NDR-QL reduces Q-learning convergence steps by about 90% on average across eight 50x50 maps, with path quality claimed to be better than O-QL, IQL, and basic QL.

Significance. If the central claims hold, the paper offers a useful practical recipe: instead of using only geometric distance heuristics, a pretrained neural predictor can supply both a shaped reward and a search-region prior for tabular Q-learning. The architecture changes to the prediction model are incremental but sensible, and the idea of testing learned heuristics inside a standard RL loop is timely. The paper also provides an ablation study separating the effects of reward shaping and Q-table initialization. However, the experimental support for the headline 90% speedup is weakened by underspecified resolution alignment, a possibly misleading baseline reward formula, an ablated IQL baseline, and single-run statistics with no convergence criterion. These issues are fixable but currently limit confidence in the quantitative claims.

major comments (5)

[IV-A, III-B, III-C] The NDR model is trained and evaluated on 201x201 maps, while the Q-learning environment is downscaled to 50x50. Equations (3) and (6) index NDR_g(xi,yi) and N_r(xi,yi) by grid cell (xi,yi), but no resampling, cropping, or coordinate-alignment procedure is described for mapping the 201x201 predictions onto the 50x50 grid. Without this step, the reward function and Q-table initialization are not well-defined, and the adaptive threshold search in Section III-C, which checks connectivity on the Q-learning grid, is likewise ambiguous. The authors must specify the exact mapping (e.g., nearest-neighbor, bilinear interpolation, or max-pooling) and justify that the per-cell semantics are preserved.
[III-B, Eq. (2)] The distance-based continuous reward function in Eq. (2) uses exp(-(xi-xd)/Gx - (yi-yd)/Gy) without absolute values or squares. When xi < xd or yi < yd, the exponent becomes positive and the reward can exceed rmax, which is not a sensible distance-based reward. The same issue appears in Eq. (4). This likely handicaps the D-CRF and IQL baselines, making the comparison against NDR-CRF unfair. The formula should be corrected to a true distance measure, e.g., exp(-(|xi-xd|/Gx + |yi-yd|/Gy)) or an L2 norm, and experiments rerun.
[IV-C, Table II] The claim that NDR-QL "yields superior path results" is contradicted by the reported shortest-distance column. For MAP-5, NDR-QL has shortest distance 154, while QL, O-QL, and IQL achieve 112, 120, and 116, respectively. NDR-QL is therefore not consistently better in path quality across the tested maps. The paper should either revise the claim to specify the metric in which NDR-QL is superior (e.g., longest-distance or convergence speed) or provide a more complete path-quality comparison that acknowledges this exception.
[IV-C, IV-D] All convergence-step results appear to come from single runs, with no error bars, no number of random seeds, and no definition of the convergence criterion. The paper states that "convergence steps" are cumulative steps until convergence, but it never defines what constitutes convergence (e.g., reaching the target for N consecutive episodes, or Q-value change below a threshold). Without this definition and without variance estimates, the reported 90% average improvement is not statistically supported. At minimum, the authors should state the convergence rule and provide multiple-seed results with mean and standard deviation.
[IV-C, IQL baseline] The IQL baseline is explicitly stripped of its improved learning-rate adjustment and enhanced action-selection policy, retaining only its distance-based Q-table initialization and continuous reward function. The paper then compares NDR-QL to this ablated IQL and counts it as an "improved Q-learning method." This is a reasonable ablation for isolating heuristic priors, but it should be labeled as IQL-without-improvements, and the comparison to the full IQL method should either be reported or clearly deferred. Otherwise, the comparative claim overstates the advantage over the original IQL.

minor comments (5)

[IV-C, Table II caption] The caption says the eight maps are illustrated in Fig. 3, Fig. 6, and Fig. 7, but Fig. 3 is a schematic visualization of reward functions, not a map sample. The reference should be corrected to the actual figures containing the maps.
[III-B, Eq. (2)] The parameters Gx and Gy are described as controlling the decrease rate, but the equation does not constrain them to be positive. A short statement that Gx, Gy > 0 and rmax > 0 would prevent ambiguity.
[III-C, Eq. (7)] The notation MASK[N_r(xi,yi)] is introduced without a formal definition of the mask operator. It is clear from context that it is a binarization, but a more explicit definition would improve readability.
[IV-D, Table IV] The table's column headers (D-C, N-C, D-Q, N-Q) are not fully self-explanatory; a one-sentence explanation in the caption of what each checkmark column denotes would help.
[III-B, Eq. (4)] The sentence "for simplicity, r0 is omitted" is confusing because Eq. (2) includes r0 while Eq. (4) does not. If r0 is omitted, the reward at the target is not rmax but r0 + rmax in the D-CRF term, so the simplification should be stated more carefully.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the NDR heuristic is a disclosed supervised prior, not a prediction derived from Q-learning.

full rationale

The paper's two central claims are (1) the NDR model predicts optimal-path guidelines and regions with higher F1 than prior models, and (2) using those predictions as a reward function and Q-table initialization accelerates Q-learning convergence. Neither claim reduces to its own input. The NDR model is trained on ground-truth path labels and validated on held-out maps, which is a standard supervised prediction task; its F1 numbers are externally comparable to RGP, NEED, and MPT. Equations (3) and (6) transparently inject the learned prior into Q-learning: the reward is scaled by the guideline output and the Q-table is biased by a mask of the predicted region. The reported 90% convergence improvement is an empirical consequence of that injection, not a logical derivation of the conclusion from itself. The paper explicitly frames this as 'heuristic information,' and it does not claim that Q-learning independently discovers the optimal path without this prior. There is no load-bearing self-citation: RGP [10] is an external baseline by different authors, and the cited prior work is used as a comparison point rather than as the justification for the main result. The reviewer's concern about the missing 201x201-to-50x50 resampling is a reproducibility and correctness issue, not a circularity issue, because it affects whether Eqs. (3) and (6) are well-defined, not whether they are equivalent to their inputs. Overall, the derivation chain is self-contained and the heuristic nature of the neural-network prior is openly disclosed.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

All numerical content of the heuristic is either a hand-set scalar (rmax, omega, thd, Gx, Gy, alpha, epsilon) or a network weight fitted to dataset labels. The central claim therefore rests on many unspecified hyperparameter values and on the validity of transferring network predictions to the downscaled RL grid.

free parameters (6)

NDR network weights = trained on path-planning dataset; not released
NDR_g and N_r are direct network outputs; every reward and Q-table value in Eq. 3 and Eq. 6 depends on these fitted weights.
rmax = not reported (target reward is 40 in Eq. 1)
Scales the continuous reward in Eq. 3 and Eq. 4; no value is given for the QL experiments.
omega = not reported
Controls the blend between NDR and distance heuristics in Eq. 4 and Eq. 6; the value is never stated.
thd initial threshold = 0.99, decremented by 0.01 until connectivity
Adaptive binarization threshold in Eq. 7; the final per-map threshold is not reported.
Gx, Gy, r0 in D-CRF baseline = not reported
Distance-based continuous reward parameters in Eq. 2 and Eq. 4; needed to reproduce the baseline and ablation comparisons.
alpha and epsilon = 0.1 and 0.2
Fixed learning rate and exploration probability; stated, but no sensitivity analysis is provided.

assumptions (4)

standard math Finite MDP Q-learning converges to the optimal policy under the chosen learning rate and epsilon-greedy exploration.
The paper relies on Q-learning convergence to claim optimal path finding, without giving a formal convergence proof.
domain assumption The dataset ground-truth guidelines and regions are valid optimal or near-optimal paths, and NDR sigmoid outputs can be treated as per-cell optimal-path probabilities.
The NDR model is trained on these labels and its outputs are used directly as rewards and masks in Eq. 3 and Eq. 6.
domain assumption NDR predictions trained on 201x201 maps transfer to the downscaled 50x50 Q-learning grid without explicit resampling or recalibration.
Section IV-A describes downscaling the map but does not describe how NDR outputs are aligned with the 50x50 grid before Eq. 3 and Eq. 6 are applied.
domain assumption For every test map, a threshold thd exists such that binarizing the region prediction connects the start and end points.
The adaptive threshold search in Section III-C assumes connectivity is reachable by decrementing from 0.99; no failure handling is described.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Neural-Network-Driven Reward Prediction as a Heuristic: Advancing Q-Learning for Mobile Robot Path Planning." pith.science (2026). https://pith.science/paper/QEVDMBYQ

@misc{pith2026241212650,
  author       = {Pith},
  title        = {Pith review of: Neural-Network-Driven Reward Prediction as a Heuristic: Advancing Q-Learning for Mobile Robot Path Planning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QEVDMBYQ}},
  note         = {Machine review of arXiv:2412.12650}
}

read the original abstract

Q-learning is a widely used reinforcement learning technique for solving path planning problems. It primarily involves the interaction between an agent and its environment, enabling the agent to learn an optimal strategy that maximizes cumulative rewards. Although many studies have reported the effectiveness of Q-learning, it still faces slow convergence issues in practical applications. To address this issue, we propose the NDR-QL method, which utilizes neural network outputs as heuristic information to accelerate the convergence process of Q-learning. Specifically, we improved the dual-output neural network model by introducing a start-end channel separation mechanism and enhancing the feature fusion process. After training, the proposed NDR model can output a narrowly focused optimal probability distribution, referred to as the guideline, and a broadly distributed suboptimal distribution, referred to as the region. Subsequently, based on the guideline prediction, we calculate the continuous reward function for the Q-learning method, and based on the region prediction, we initialize the Q-table with a bias. We conducted training, validation, and path planning simulation experiments on public datasets. The results indicate that the NDR model outperforms previous methods by up to 5\% in prediction accuracy. Furthermore, the proposed NDR-QL method improves the convergence speed of the baseline Q-learning method by 90\% and also surpasses the previously improved Q-learning methods in path quality metrics.

Figures

Figures reproduced from arXiv: 2412.12650 by the authors.

**Figure 1.** The proposed NDR-QL method. At t = 0, the pretrained NDR model is invoked to initialize each Q-table and predict the continuous reward function. fail to generate a complete path. Sampling-based methods like RRT leverage random sampling for path planning [ 4], offering robust search capabilities and rapid exploration speeds. However, they are limited by lower search accuracy and poorer path smoothness compared to o… view at source ↗

**Figure 2.** Structure of the NDR model. moves from initial to target state. The common design of the reward function is as shown in Eq. 1: R =    −10, scenario1 40, scenario2 −5, scenario3 rc, scenario4 (1) Here, scenario1 is collision with obstacles or boundary violation, scenario2 is reaching the target, scenario3 is revisiting a node, and scenario4 covers all other cases with reward rc. A key challenge in RL-based pa… view at source ↗

**Figure 3.** A schematic visualization of the reward function. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: A schematic visualization of the Q-table and a [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

**Figure 5.** Figure 5: Results from experiments in dynamic scenarios. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]

**Figure 7.** Figure 7: Computational results of the proposed NDR-QL algo [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

FlexPath: Adapting Learned Connectivity Guidance to Path Preferences
cs.CV 2026-06 unverdicted novelty 6.0 of 10

FlexPath decouples learning of task-independent feasible path priors from task-specific adaptation via imitation learning and differentiable Path Shape Objectives for image-based planning.

Reference graph

Works this paper leans on

25 extracted references · 18 canonical work pages · cited by 1 Pith paper

[1]

Path planning of autonomous mobile robot in comprehensive unknown environm ent using deep reinforcement learning,

Z. Bai, H. Pang, Z. He, B. Zhao, and T. Wang, “Path planning of autonomous mobile robot in comprehensive unknown environm ent using deep reinforcement learning,” IEEE Internet of Things Journal , 2024

work page 2024
[2]

Re search on path planning with the integration of adaptive a-star alg orithm and improved dynamic window approach,

T. Liao, F. Chen, Y . Wu, H. Zeng, S. Ouyang, and J. Guan, “Re search on path planning with the integration of adaptive a-star alg orithm and improved dynamic window approach,” Electronics, vol. 13, no. 2, p. 455, 2024

work page 2024
[3]

Agv path planning based o n improved a-star algorithm,

D. Zhang, C. Chen, and G. Zhang, “Agv path planning based o n improved a-star algorithm,” in 2024 IEEE 7th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), vol. 7. IEEE, 2024, pp. 1590–1595

work page 2024
[4]

Intelligent path planning by an improved rrt algorithm wit h dual grid map,

R. Zhang, H. Guo, D. Andriukaitis, Y . Li, G. Kr´ olczyk, an d Z. Li, “Intelligent path planning by an improved rrt algorithm wit h dual grid map,” Alexandria Engineering Journal , vol. 88, pp. 91–104, 2024

work page 2024
[5]

Path planning of modular robots on various terrain s using q- learning versus optimization algorithms,

S. Haghzad Klidbary, S. Bagheri Shouraki, and S. Sheikhp our Kourab- baslou, “Path planning of modular robots on various terrain s using q- learning versus optimization algorithms,” Intelligent Service Robotics , vol. 10, pp. 121–136, 2017

work page 2017
[6]

C. J. C. H. Watkins, Learning from delayed rewards . King’s College, Cambridge United Kingdom, 1989

work page 1989
[7]

S. J. Prince, Understanding deep learning . MIT press, 2023

2023
[8]

Simu lation of sequential data: An enhanced reinforcement learning appro ach,

M. V anhulsel, D. Janssens, G. Wets, and K. V anhoof, “Simu lation of sequential data: An enhanced reinforcement learning appro ach,” Expert Systems with Applications , vol. 36, no. 4, pp. 8032–8039, 2009

work page 2009

Show all 25 references

[9]

Path planning for mobile robot based on improved reinforcement learning algorithm,

X. Xu and J. Y uan, “Path planning for mobile robot based on improved reinforcement learning algorithm,” Journal of Chinese Inertial Technol- ogy, vol. 27, no. 3, pp. 314–320, 2019

2019
[10]

Efﬁciency improve ment to neural-network-driven optimal path planning via region an d guideline prediction,

Y . Huang, C.-T. Tsao, and H.-H. Lee, “Efﬁciency improve ment to neural-network-driven optimal path planning via region an d guideline prediction,” IEEE Robotics and Automation Letters , 2024

2024
[11]

Lightweight ne ural path planning,

J. Li, S. Wang, Z. Chen, Z. Kan, and J. Y u, “Lightweight ne ural path planning,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2023, pp. 6713–6718

2023
[12]

Neural rrt*: Learning-based optimal path planning,

J. Wang, W. Chi, C. Li, C. Wang, and M. Q.-H. Meng, “Neural rrt*: Learning-based optimal path planning,” IEEE Transactions on Automation Science and Engineering , vol. 17, no. 4, pp. 1748–1758, 2020

2020
[13]

Motion planning transformers: A motion planning fram ework for mobile robots,

J. J. Johnson, U. S. Kalra, A. Bhatia, L. Li, A. H. Qureshi , and M. C. Yip, “Motion planning transformers: A motion planning fram ework for mobile robots,” arXiv preprint arXiv:2106.02791 , 2021

2021 arXiv
[14]

Solving the optimal pa th planning of a mobile robot using improved q-learning,

E. S. Low, P . Ong, and K. C. Cheah, “Solving the optimal pa th planning of a mobile robot using improved q-learning,” Robotics and Autonomous Systems, vol. 115, pp. 143–161, 2019

2019
[15]

Optimal path planning approa ch based on q-learning algorithm for mobile robots,

A. Maoudj and A. Hentout, “Optimal path planning approa ch based on q-learning algorithm for mobile robots,” Applied Soft Computing , vol. 97, p. 106796, 2020

2020
[16]

A modiﬁed q-learning pat h planning approach using distortion concept and optimizati on in dynamic environment for autonomous mobile robot,

E. S. Low, P . Ong, and C. Y . Low, “A modiﬁed q-learning pat h planning approach using distortion concept and optimizati on in dynamic environment for autonomous mobile robot,” Computers & Industrial Engineering, vol. 181, p. 109338, 2023

2023
[17]

An opt imized q-learning algorithm for mobile robot local path planning,

Q. Zhou, Y . Lian, J. Wu, M. Zhu, H. Wang, and J. Cao, “An opt imized q-learning algorithm for mobile robot local path planning, ” Knowledge- Based Systems , vol. 286, p. 111400, 2024

2024
[18]

Robot path planning via neural-network-driven prediction,

J. Wang, J. Liu, W. Chen, W. Chi, and M. Q.-H. Meng, “Robot path planning via neural-network-driven prediction,” IEEE transactions on artiﬁcial intelligence , vol. 3, no. 3, pp. 451–460, 2021

2021
[19]

Re- thinking bisenet for real-time semantic segmentation,

M. Fan, S. Lai, J. Huang, X. Wei, Z. Chai, J. Luo, and X. Wei , “Re- thinking bisenet for real-time semantic segmentation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern reco gnition, 2021, pp. 9716–9725

2021
[20]

Deep residual learni ng for image recognition,

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learni ng for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778

2016
[21]

Pp-liteseg: A superior real-time semantic segmentation model,

J. Peng, Y . Liu, S. Tang, Y . Hao, L. Chu, G. Chen, Z. Wu, Z. C hen, Z. Y u, Y . Du, et al. , “Pp-liteseg: A superior real-time semantic segmentation model,” arXiv preprint arXiv:2204.02681 , 2022

2022 arXiv
[22]

Pyramid scene parsing network,

H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, “Pyramid scene parsing network,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 2881–2890

2017
[23]

U-net: Convol utional networks for biomedical image segmentation,

O. Ronneberger, P . Fischer, and T. Brox, “U-net: Convol utional networks for biomedical image segmentation,” in Medical image computing and computer-assisted intervention–MICCAI 2015: 18th intern ational con- ference, Munich, Germany, October 5-9, 2015, proceedings, part III ...

2015
[24]

A novel e nergy management strategy based on dual reward function q-learni ng for fuel cell hybrid electric vehicle,

Y . Zhang, R. Ma, D. Zhao, Y . Huangfu, and W. Liu, “A novel e nergy management strategy based on dual reward function q-learni ng for fuel cell hybrid electric vehicle,” IEEE Transactions on Industrial Electronics, vol. 69, no. 2, pp. 1537–1547, 2021. 8

2021
[25]

Modiﬁed q-learn ing with distance metric and virtual target on path planning of mobil e robot,

E. S. Low, P . Ong, C. Y . Low, and R. Omar, “Modiﬁed q-learn ing with distance metric and virtual target on path planning of mobil e robot,” Expert Systems with Applications , vol. 199, p. 117191, 2022

2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Path planning of autonomous mobile robot in comprehensive unknown environm ent using deep reinforcement learning,

Z. Bai, H. Pang, Z. He, B. Zhao, and T. Wang, “Path planning of autonomous mobile robot in comprehensive unknown environm ent using deep reinforcement learning,” IEEE Internet of Things Journal , 2024

work page 2024

[2] [2]

Re search on path planning with the integration of adaptive a-star alg orithm and improved dynamic window approach,

T. Liao, F. Chen, Y . Wu, H. Zeng, S. Ouyang, and J. Guan, “Re search on path planning with the integration of adaptive a-star alg orithm and improved dynamic window approach,” Electronics, vol. 13, no. 2, p. 455, 2024

work page 2024

[3] [3]

Agv path planning based o n improved a-star algorithm,

D. Zhang, C. Chen, and G. Zhang, “Agv path planning based o n improved a-star algorithm,” in 2024 IEEE 7th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), vol. 7. IEEE, 2024, pp. 1590–1595

work page 2024

[4] [4]

Intelligent path planning by an improved rrt algorithm wit h dual grid map,

R. Zhang, H. Guo, D. Andriukaitis, Y . Li, G. Kr´ olczyk, an d Z. Li, “Intelligent path planning by an improved rrt algorithm wit h dual grid map,” Alexandria Engineering Journal , vol. 88, pp. 91–104, 2024

work page 2024

[5] [5]

Path planning of modular robots on various terrain s using q- learning versus optimization algorithms,

S. Haghzad Klidbary, S. Bagheri Shouraki, and S. Sheikhp our Kourab- baslou, “Path planning of modular robots on various terrain s using q- learning versus optimization algorithms,” Intelligent Service Robotics , vol. 10, pp. 121–136, 2017

work page 2017

[6] [6]

C. J. C. H. Watkins, Learning from delayed rewards . King’s College, Cambridge United Kingdom, 1989

work page 1989

[7] [7]

S. J. Prince, Understanding deep learning . MIT press, 2023

2023

[8] [8]

Simu lation of sequential data: An enhanced reinforcement learning appro ach,

M. V anhulsel, D. Janssens, G. Wets, and K. V anhoof, “Simu lation of sequential data: An enhanced reinforcement learning appro ach,” Expert Systems with Applications , vol. 36, no. 4, pp. 8032–8039, 2009

work page 2009