PICTS: A Novel Deep Reinforcement Learning Approach for Dynamic P-I Control in Scanning Probe Microscopy

Huajun Liu; Kaiyang Zeng; Qibin Zeng; Shuming Wei; Wanheng Lu; Ziwei Wei

REVIEW 3 major objections 5 minor 38 references

PICTS: A Novel Deep Reinforcement Learning Approach for Dynamic P-I Control in Scanning Probe Microscopy

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A deep-reinforcement-learning agent that retunes a microscope's feedback loop in real time reduces deflection errors by 26–90% on difficult samples.

desk verdict A genuinely new hardware integration of DRL into a commercial SPM, but the headline 26–90% error reduction is confounded by a hardware change and needs a same-hardware fixed-gain control arm before it can be believed. read the letter →

arxiv 2502.07326 v1 pith:I5XHX6FR submitted 2025-02-11 cond-mat.mtrl-sci cs.LGphysics.app-ph

Ziwei Wei , Shuming Wei , Qibin Zeng , Wanheng Lu , Huajun Liu , Kaiyang Zeng This is my paper · ORCID

classification cond-mat.mtrl-scics.LGphysics.app-ph

keywords scanningprobemicroscopydeepreinforcementlearningproportional-integralcontroladaptivefield-programmablegatearraydeflectionerrorSoftActor-Critictopographyimaging

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Scanning probe microscopy's height-control loop normally runs with fixed proportional-integral (P-I) gains, and those fixed gains produce artefacts on sharp edges, soft multiphase structures, and large topographical steps. The paper presents a system, PICTS, that replaces the fixed gains with a deep-reinforcement-learning agent that adjusts the P and I gains at every scan point based on the current error, control output, actuator history, and current gains. On a calibration grating, a biphasic PS-LDPE film, and exfoliated HOPG, the adaptive controller reduces mean deflection error by 26% to 90% relative to the commercial fixed-gain controller, with error distributions more concentrated near zero. The paper argues this makes SPM control practical for difficult samples without a high-performance computer.

What carries the argument

The load-bearing mechanism is a three-cycle parallel control-and-training architecture. A field-programmable gate array (FPGA) runs the high-speed P-I control loop at 128 kHz, computes the error signal, and detects overshoots and oscillations; a host computer runs a Soft Actor-Critic agent that at 512 Hz reads the current state and outputs small increments to the P and I gains; and a training thread at 1 Hz continually updates the agent's policy. The reward function combines a squared-error accuracy term, penalties for overshoot and oscillation derived from the FPGA-based detector, and a penalty for losing probe-sample contact, which together steer the agent toward gain combinations that keep the probe precisely tracking. The incremental action space is designed to produce smooth gain changes, one adjustment per scan point.

What would settle it

Run a fixed-gain P-I controller on the same FPGA board with the same gains the commercial controller used, and compare deflection errors; if the fixed-gain FPGA loop alone already matches the DRL controller's low errors, then the DRL adaptation is not the driver of the reported improvement.

Watch

Extended reading notes

Core claim

The central claim is that a deep reinforcement learning controller that dynamically adjusts the proportional and integral gains of the SPM height loop outperforms a commercial fixed-gain P-I controller, reducing deflection errors by 26% to 90% depending on the sample, with fewer outliers and error values clustered near zero. The agent infers surface context from the error signal, control output, and actuator history, and changes the gains accordingly: it raises P on downward slopes to prevent the probe from disengaging, lowers I when transitioning onto soft material to avoid oscillations, and reduces gains after a feature to prevent overcompensation. These behaviours are demonstrated on a calibration grating with sharp edges, a soft biphasic PS-LDPE polymer, and HOPG with 4–6 micrometre height variations, with additional aerogel and carbon-fibre-reinforced polymer tests reported in the supplementary material.

Load-bearing premise

The claim rests on the assumption that the error reduction comes from the learned gain adaptation and not from the different hardware (custom FPGA loop vs. commercial controller) used in the two compared arms.

Editorial extensions

If this is right

If the reported error reductions hold, SPM users could scan sharp-edged or soft, multiphase samples without manually retuning P and I gains for each region, and would see improved trace-retrace consistency.
The system's FPGA-plus-CPU split means the adaptive control loop could be added to existing commercial SPM hardware without a high-performance computer, lowering the barrier for labs.
The largest reductions—up to 90%—occur on the most challenging surfaces (HOPG with large height variations), so the method is most valuable where conventional fixed-gain control breaks down.
Because training runs concurrently with scanning, the agent's policy continues to adapt over time, which should also absorb slow drifts such as tip wear or thermal drift during long scans.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural next experiment would be to run the learned gain schedule on the commercial controller's own hardware, or a fixed-gain P-I loop on the FPGA, to separate the benefit of the learned adaptation from the benefit of the FPGA implementation itself.
The same three-cycle architecture could be applied to other SPM feedback loops—such as amplitude control in tapping mode or the lateral force loop—or to other probe-based instruments that rely on fast P-I control.
The online-training design suggests a path to controllers that co-adapt with tip degradation: as the probe wears, the agent's policy could keep adjusting indefinitely, though the paper only demonstrates stable operation over a four-hour window.
A testable prediction is that on samples with extremely high topographical slope, the adaptive controller's deflection-error advantage over fixed-gain control should grow monotonically with slope magnitude, because the agent explicitly changes gains on downward slopes to prevent disengagement.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Zoe, here is my read on Wei et al. The genuinely new thing is the system: they put a SAC agent in the loop on a commercial MFP-3D, with FPGA-based PI control at 128 kHz, host-side decision at 512 Hz, and online training at 1 Hz. That integration—DRL tuning P-I gains on real hardware, not in simulation—is new as far as the cited literature goes. The architecture is thoughtful, and the qualitative images on grating, PS-LDPE, and HOPG show a real working system. I believe the scans happened and the system does what they say.

The soft spot is exactly the one the stress-test flags. The DRL arm uses a custom FPGA loop with its own ADC/DAC, low-pass filter, and 128 kHz rate; the baseline is the MFP-3D internal controller. That is two variables changed at once: gain adaptation and control hardware. Without a fixed-gain run on the same FPGA path, the 26–90% error reduction cannot be attributed specifically to the learned P-I adjustment. A faster loop and cleaner analog path could easily account for a large chunk of it. That weakens the central causal claim, though not the weaker claim that the whole PICTS system outperforms the commercial controller.

The second issue is that Fig. 5 reports mean error reductions with no error bars, no repeated-scan statistics, and no per-scan scatter. Given the qualitative images show strong effects, I suspect the effect is real, but the magnitude—90% vs 26%—is not pinned down. Third, there is no code, no data, and the SI is “obtained upon requirements”; explicit reward weights, SAC hyperparameters, and thresholds for the overshoot/oscillation detector are all missing. For an ML-for-control paper, that makes reproducibility weak. Minor: references 21 and 22 are duplicated.

Who this is for: the SPM instrumentation community and people working on RL for real-time control. It deserves a serious referee, but the referee should ask for a same-hardware fixed-gain control experiment, repeated-scan error bars, and full hyperparameters/data. I would engage with it, but I would not cite the headline improvement numbers until the confounding control is done.

Reading group: maybe—good for discussing experimental design confounds in RL-for-control work.

Recommendation: send to peer review with major revision, not desk reject.

Referee Report

3 major / 5 minor

Summary. The paper presents PICTS, a parallel control and training system that integrates a soft actor-critic deep reinforcement learning agent with an FPGA-based proportional-integral controller for scanning probe microscopy. The agent adjusts P-I gains in real time using a reward function that penalizes deflection error, overshoot, oscillation, and probe-sample disengagement. The authors compare the DRL-based controller with the commercial MFP-3D controller with fixed P-I gains on a calibration grating, PS-LDPE, HOPG, aerogel, and CFRP samples. They report qualitative improvements in topography and deflection images and quantitative reductions in mean deflection error of 26% to 90% across conditions, with error histograms more concentrated near zero. The paper claims that PICTS balances real-time control and computational load by offloading control to an FPGA and training to a host computer.

Significance. If the claimed improvements are robust, this is a practically significant demonstration of DRL-based adaptive control in commercial SPM hardware, addressing a real limitation of fixed-gain P-I controllers on sharp edges, soft multiphase materials, and large topographical variations. The main strengths are the physical integration with an MFP-3D system, the explicit three-cycle architecture, the use of a standard SAC algorithm, and the evaluation across multiple challenging samples with quantitative error histograms. The paper is also careful to scan with the commercial controller first to avoid attributing sample damage to the DRL controller. However, the central quantitative claim is currently undercut by a hardware confound and by the absence of repeated-scan statistics, so the significance is contingent on additional control experiments.

major comments (3)

[Section 2.1.1 and Section 4.1.1] The comparison is confounded by hardware differences between the two control arms. The DRL path uses a custom FPGA-based PI loop (cRIO-9064 with NI-9775/9263 and a low-pass filter) at 128 kHz, while the baseline uses the MFP-3D internal commercial controller with fixed P-I gains. The two arms therefore differ in both the gain-adaptation strategy and the control hardware/signal chain. No fixed-gain PI control run on the same FPGA path is reported, so the observed 26% to 90% deflection-error reduction cannot be attributed specifically to DRL gain adaptation; it may reflect the faster loop rate, analogue filtering, or different ADC/DAC path. The authors should add a fixed-gain control condition on the same FPGA hardware, or explicitly reframe the claim as a system-level comparison of PICTS with the commercial controller and temper the DRL-specific causal language in the abstract and Section 2.2.2.
[Section 2.2.2 and Figure 5] The quantitative claim lacks uncertainty quantification and repeated-scan statistics. Figure 5B reports one mean deflection-error value per condition with no error bars, no number of repeated scans, and no per-line or per-image spread; the histograms in Figure 5A similarly do not report counts, bin widths, or vertical-axis units. The 26% to 90% improvement range is therefore not statistically grounded. Please provide repeated-scan statistics (at least n, mean, and standard deviation or confidence interval) for each condition, a formal comparison between controllers, and a clear description of how the deflection-error signal was sampled and aggregated.
[Section 2.1.3 and Section 2.2.2] The primary evaluation metric, mean deflection error, is also the main term optimized by the reward function. The reported improvement is thus partly the optimized objective rather than an independent validation of imaging quality or stability. This does not invalidate the measured engineering result, but the paper should acknowledge the overlap and report at least one independent metric, such as trace-retrace correlation, image sharpness, edge fidelity, or contact-loss rate, to support the broader claims of improved stability and scanning precision.

minor comments (5)

[Section 4.2.2] The statement that hyperparameters were determined by grid search in a simulated environment is not accompanied by any description of the simulator or a table of the chosen hyperparameter values; providing these details is necessary for reproducibility.
[Reference list] Reference 22 (Arulkumaran et al.) appears twice, and the reference numbering after that entry is offset; please correct the numbering and remove the duplicate.
[Section 4.1.1] The sentence 'Analogue Input-Output Communication: This operates via TCP over Ethernet' is unclear for analogue signals; please clarify the actual signal path, the role of TCP/IP, and the reported 100 MB/s speed and 1000 μs delay.
[Supplementary Information] The SI is described as 'available upon requirements'; for a methods-heavy paper with additional samples and analyses, the SI should be provided with the manuscript.
[Section 2.2.1(i)] The fixed-gain baseline is described only as integral gain values of 10 and 20; please specify the corresponding proportional gain and any other relevant controller settings so the baseline is reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the DRL advantage is an empirical measurement, not a derivation that reduces to its inputs.

full rationale

I found no load-bearing circular steps. The paper's central claim is an empirical comparison: a DRL-tuned PI controller (PICTS) is measured against a commercial fixed-gain controller on deflection-error statistics (Section 2.2.2, Fig. 5). The reported 26-90% reductions are measured system outputs obtained after online RL training, not quantities that are equal to the reward by construction. Although the reward function (Section 2.1.3) includes a squared error term and the evaluation metric is mean deflection error, this is an objective/evaluation alignment, not circularity: no equation defines the measured error as the reward, and the observed improvement depends on the actual probe-sample dynamics. The self-citations (Refs. 3 and 19) are background material on SPM characterization and machine-learning PFM and are not used to justify the DRL control claim. No uniqueness theorem from the authors' prior work is invoked, and no fitted parameter is relabeled as a prediction. The main validity risk is experimental design, namely that the DRL arm uses a different FPGA-based PI loop and no fixed-gain FPGA control arm; that is a confound/correctness issue, not a circularity. Honest non-finding.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim is empirical and does not rest on equations fitted to data, but the learned policy is defined by unspecified reward weights, thresholds, and hyperparameters, and the comparison lacks a hardware-matched baseline.

free parameters (5)

Reward function component weights = not reported
The reward combines squared-error, overshoot, oscillation, and disengagement penalties (Section 2.1.3); the relative weights are not given, so the learned policy is only defined up to these unspecified choices.
Overshoot and oscillation detector thresholds = not reported
The OOD applies a detection threshold, zero-crossing detection, and an oscillation ratio (Section 2.1.3); these thresholds shape the reward and are not specified.
SAC hyperparameters (learning rates, hidden sizes, target entropy) = chosen by grid search in simulation, not reported
Section 4.2.2 states hyperparameters were determined through grid search in a simulated environment but does not report the values, making the trained policy under-specified.
Prioritized replay exponents alpha and beta = 0.5 and 0.5
Stated in Section 4.2.2; these chosen values affect training stability and are free choices.
Initial P-I gain values for DRL runs = not reported
The agent starts from some initial gain setting per sample; this starting point is not stated and can strongly affect early-scan performance.

assumptions (4)

domain assumption The SPM height-control problem is a Markov decision process with the chosen state vector (error, control output, historical positions, current P-I gains).
Section 2.1.2 defines the state and action space; if relevant dynamics are not captured by these four components, the learned policy is incomplete.
domain assumption Adjusting P-I gains once per scan point at 512 Hz is sufficient to track the features that cause the reported errors.
Section 4.1.2 sets Cycle 2 to 512 Hz based on scan resolution; this assumes the gain update rate is adequate for the tested topographies.
domain assumption Continuous on-sample SAC training is safe and does not damage the tip or sample.
Training runs in parallel with real scans on physical samples (Section 4.2.2); no safety layer or abort criteria are described.
domain assumption Deflection error is a valid proxy for image quality and tracking accuracy.
Deflection error is used throughout as the measured outcome; it is standard in SPM but is not equivalent to direct morphological fidelity.

how reviews work

0 comments

Cite this review

Pith. "Pith review of PICTS: A Novel Deep Reinforcement Learning Approach for Dynamic P-I Control in Scanning Probe Microscopy." pith.science (2026). https://pith.science/paper/I5XHX6FR

@misc{pith2026250207326,
  author       = {Pith},
  title        = {Pith review of: PICTS: A Novel Deep Reinforcement Learning Approach for Dynamic P-I Control in Scanning Probe Microscopy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I5XHX6FR}},
  note         = {Machine review of arXiv:2502.07326}
}

read the original abstract

We have developed a Parallel Integrated Control and Training System, leveraging the deep reinforcement learning to dynamically adjust the control strategies in real time for scanning probe microscopy techniques.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 34 canonical work pages

[1]

Department of Mechanical Engineering, National University of Singapore, 9 Engineering Drive 1, Singapore, 117576

work page
[2]

Institute of Materials Research and Engineering, Agencies of Science, Technology and Research (A*STAR), 2 Fusionopolis Way, Innovis, #08-03, Singapore, 138634

work page
[3]

Department of Electrical and Computer Engineering, National University of Singapore, 4 Engineering Drive 3, Singapore 117583

work page
[4]

377 Linquan Street, Suzhou Industrial Park, Suzhou, Jiangsu Province, China, 215123

NUS Research Institute (NUSRI), No. 377 Linquan Street, Suzhou Industrial Park, Suzhou, Jiangsu Province, China, 215123. + Equal contributions * Corresponding author (mpezk@nus.edu.sg ) Abstract Scanning Probe M icroscopy (SPM) often faces great challenges when dealing with nonlinear and time -varying behaviours, especially during scanning the samples wit...

work page
[5]

observers

Introduction Scanning Probe Microscopy (SPM) stands as a cornerstone technology in nanoscale science, offering unparalleled capabilities for imaging and manipulating materials with atomic-level precision . 1–4 Its versatility has made it indispensable across multidisciplinary fields such as materials science,3 physics,5 biology,1,2 and chemistry.6 SPM tec...

work page
[6]

Host Computer

Results The following sections provide a detailed analysis of how the PICTS achieves superior control performance (Section 2.1). Section 2.2 presents experimental results from the commercial SPM system with the new controller designed in this work, demonstrating that the PICTS delivers improved measurement accuracy compared with that from the commercial S...

work page
[7]

Conclusion Controlling mechanisms in Scanning Probe Microscopy (SPM) face significant challenges when operating in highly nonlinear and dynamic environments. Complex sample surfaces and various probe -sample interactions necessitate experts to tune controllers and achieve high- quality images, making it diffic ult to distinguish morphological features fro...

work page
[8]

Methods 4.1 Technical settings 16 In implementing the DRL control strategy for the SPM system, we create a lightweight system that can operate efficiently on the host computer, equipped with a standard CPU. The system architecture is designed to ensure efficient coordination between the FPGA, host computer, and the commercial controller, leveraging high- ...

work page

Show all 38 references

[9]

& Laxminarayana, K

Jalili, N. & Laxminarayana, K. A review of atomic force microscopy imaging systems: application to molecular metrology and biological sciences. Mechatronics 14, 907–945 (2004)

2004
[10]

Dufrêne, Y. F. et al. Imaging modes of atomic force microscopy for application in molecular and cell biology. Nat. Nanotechnol. 12, 295–307 (2017)

2017
[11]

& Zeng, K

Li, T. & Zeng, K. Probing of Local Multifield Coupling Phenomena of Advanced Materials by Scanning Probe Microscopy Techniques. Adv. Mater. 30, 1803064 (2018)

2018
[12]

Bian, K. et al. Scanning probe microscopy. Nat. Rev. Methods Primer 1, 36 (2021)

2021
[13]

F., Lutz, C

Crommie, M. F., Lutz, C. P. & Eigler, D. M. Confinement of electrons to quantum corrals on a metal surface. Science 262, 218–221 (1993)

1993
[14]

& Meyer, G

Gross, L., Mohn, F., Moll, N., Liljeroth, P. & Meyer, G. The Chemical Structure of a Molecule Resolved by Atomic Force Microscopy. Science 325, 1110–1114 (2009)

2009
[15]

Reifenberger, R. G. Fundamentals Of Atomic Force Microscopy - Part I: Foundations. (World Scientific, 2015)

2015
[16]

& Moheimani, S

Mahmood, I. & Moheimani, S. Making a commercial atomic force microscope more accurate and faster using positive position feedback control. Rev. Sci. Instrum. 80, 063705 (2009)

2009
[17]

Stark, R. W. Bistability, higher harmonics, and chaos in AFM. Mater. Today 13, 24–32 (2010)

2010
[18]

& Bhushan, B

Avila, A. & Bhushan, B. Electrical Measurement Techniques in Atomic Force Microscopy. Crit. Rev. Solid State Mater. Sci. 35, 38–51 (2010)

2010
[19]

Melitz, W., Shen, J., Kummel, A. C. & Lee, S. Kelvin probe force microscopy and its application. Surf. Sci. Rep. 66, 1–27 (2011)

2011
[20]

Jesse, S., Baddorf, A. P. & Kalinin, S. V. Switching spectroscopy piezoresponse force microscopy of ferroelectric materials. Appl. Phys. Lett. 88, 062908 (2006). 20

2006
[21]

V., Rar, A

Kalinin, S. V., Rar, A. & Jesse, S. A Decade of Piezoresponse Force Microscopy: Progress, Challenges and Opportunities. Preprint at http://arxiv.org/abs/cond- mat/0509009 (2005)

2005
[22]

& Otterlo, M

Wiering, M. & Otterlo, M. van. Reinforcement Learning: State-of-the-Art. (Springer, Heidelberg New York, 2012)

2012
[23]

Kalinin, S. V. et al. Automated and Autonomous Experiment in Electron and Scanning Probe Microscopy. Preprint at http://arxiv.org/abs/2103.12165 (2021)

2021 arXiv
[24]

Liu, Y. et al. Machine Learning-Based Reward-Driven Tuning of Scanning Probe Microscopy: Towards Fully Automated Microscopy. Preprint at https://doi.org/10.48550/ARXIV.2408.04055 (2024)

2024 doi arXiv
[25]

Liu, Y. et al. General Resolution Enhancement Method in Atomic Force Microscopy (AFM) Using Deep Learning. Preprint at http://arxiv.org/abs/1809.03704 (2018)

2018 arXiv
[26]

& Gonzalez -Martinez, J

Sotres, J., Boyd, H. & Gonzalez -Martinez, J. F. Enabling autonomous scanning probe microscopy imaging of single molecules with deep learning. Nanoscale 13, 9193–9203 (2021)

2021
[27]

& Zeng, K

Liu, Y., Yu, B., Liu, Z., Beck, D. & Zeng, K. High- Speed Piezoresponse Force Microscopy and Machine Learning Approaches for Dynamic Domain Growth in Ferroelectric Materials. ACS Appl. Mater. Interfaces 12, 9944–9952 (2020)

2020
[28]

& Krull, C

Krull, A., Hirsch, P., Rother, C., Schiffrin, A. & Krull, C. Artificial -intelligence- driven scanning probe microscopy. Commun. Phys. 3, 1–8 (2020)

2020
[29]

Ma, E. Y. Scanning probe microscopy based on reinforcement learning. CS229 project final report.22. Arulkumaran, K., Deisenroth, M. P., Brundage, M. & Bharath, A. A. Deep Reinforcement Learning: A Brief Survey. IEEE Signal Process. Mag. 34, 26–38 (2017)

2017
[30]

P., Brundage, M

Arulkumaran, K., Deisenroth, M. P., Brundage, M. & Bharath, A. A. Deep Reinforcement Learning: A Brief Survey. IEEE Signal Process. Mag. 34, 26–38 (2017)

2017
[31]

Leinen, P. et al. Autonomous robotic nanofabrication with reinforcement learning. Sci. Adv. 6, eabb6987 (2020)

2020
[32]

W., Deng, N., Tutsch, R

Degenhardt, J., Bounaim, M. W., Deng, N., Tutsch, R. & Dai, G. A New Kind of Atomic Force Microscopy Scan Control Enabled by Artificial Intelligence: Concept for Achieving Tip and Sample Safety Through Asymmetric Control. Nanomanufacturing Metrol. 7, 11 (2024)

2024
[33]

Lv, H., Xu, H., Wang, L. & Li, H. Scanning control of atomic force microscope based on deep reinforcement learning. in 2021 International Conference on Artificial Intelligence and Electromechanical Automation (AIEA) 19–22 (2021). doi:10.1109/AIEA53260.2021.00012

2021
[34]

Haarnoja, T. et al. Soft Actor -Critic Algorithms and Applications. Preprint at http://arxiv.org/abs/1812.05905 (2019)

2019 arXiv
[35]

& Levine, S

Haarnoja, T., Zhou, A., Abbeel, P. & Levine, S. Soft Actor -Critic: Off -Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. in Proceedings of the 35th International Conference on Machine Learning 1861–1870 (PMLR, 2018). 21

2018
[36]

& Luo, J

Tan, X., Guo, D. & Luo, J. Dynamic friction energy dissipation and enhanced contrast in high frequency bimodal atomic force microscopy. Friction 10, 748–761 (2022)

2022
[37]

S., Goosen, J

Keyvani, A., Sadeghian, H., Tamer, M. S., Goosen, J. F. L. & Van Keulen, F. Minimizing tip -sample forces and enhancing sensitivity in atomic force microscopy with dynamically compliant cantilevers. J. Appl. Phys. 121, 244505 (2017)

2017
[38]

& Silver, D

Schaul, T., Quan, J., Antonoglou, I. & Silver, D. Prioritized Experience Replay. (2015). Preprint at https://doi.org/10.48550/arXiv.1511.05952 (2016)

2015 doi arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.

[1] [1]

Department of Mechanical Engineering, National University of Singapore, 9 Engineering Drive 1, Singapore, 117576

work page

[2] [2]

Institute of Materials Research and Engineering, Agencies of Science, Technology and Research (A*STAR), 2 Fusionopolis Way, Innovis, #08-03, Singapore, 138634

work page

[3] [3]

Department of Electrical and Computer Engineering, National University of Singapore, 4 Engineering Drive 3, Singapore 117583

work page

[4] [4]

377 Linquan Street, Suzhou Industrial Park, Suzhou, Jiangsu Province, China, 215123

NUS Research Institute (NUSRI), No. 377 Linquan Street, Suzhou Industrial Park, Suzhou, Jiangsu Province, China, 215123. + Equal contributions * Corresponding author (mpezk@nus.edu.sg ) Abstract Scanning Probe M icroscopy (SPM) often faces great challenges when dealing with nonlinear and time -varying behaviours, especially during scanning the samples wit...

work page

[5] [5]

observers

Introduction Scanning Probe Microscopy (SPM) stands as a cornerstone technology in nanoscale science, offering unparalleled capabilities for imaging and manipulating materials with atomic-level precision . 1–4 Its versatility has made it indispensable across multidisciplinary fields such as materials science,3 physics,5 biology,1,2 and chemistry.6 SPM tec...

work page

[6] [6]

Host Computer

Results The following sections provide a detailed analysis of how the PICTS achieves superior control performance (Section 2.1). Section 2.2 presents experimental results from the commercial SPM system with the new controller designed in this work, demonstrating that the PICTS delivers improved measurement accuracy compared with that from the commercial S...

work page

[7] [7]

Conclusion Controlling mechanisms in Scanning Probe Microscopy (SPM) face significant challenges when operating in highly nonlinear and dynamic environments. Complex sample surfaces and various probe -sample interactions necessitate experts to tune controllers and achieve high- quality images, making it diffic ult to distinguish morphological features fro...

work page

[8] [8]

Methods 4.1 Technical settings 16 In implementing the DRL control strategy for the SPM system, we create a lightweight system that can operate efficiently on the host computer, equipped with a standard CPU. The system architecture is designed to ensure efficient coordination between the FPGA, host computer, and the commercial controller, leveraging high- ...

work page