From Rules to Rewards: Reinforcement Learning for Interest Rate Adjustment in DeFi Lending

Claudio Tessone; Florian Groetschla; Hanxiao Qu; Krzysztof Gogol

REVIEW 3 major objections 5 minor 29 references

From Rules to Rewards: Reinforcement Learning for Interest Rate Adjustment in DeFi Lending

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that an offline RL agent, TD3-BC, trained on historical Aave data, can adjust lending rates more responsively than rule-based curves, improving capital efficiency, LP returns, and stress resilience.

desk verdict A useful empirical comparison of offline RL algorithms for Aave rate setting, but the 'outperforms' claim is invalid because the evaluation is an open-loop replay with no environment. read the letter →

arxiv 2506.00505 v1 pith:DWCJQ3OR submitted 2025-05-31 cs.LG

Hanxiao Qu , Krzysztof Gogol , Florian Groetschla , Claudio Tessone This is my paper

classification cs.LG

keywords offlinereinforcementlearningDeFilendinginterestrateoptimizationAaveTD3-BCcapitalefficiencyliquidationriskautomatedgovernance

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether an offline reinforcement-learning agent—one trained on a fixed historical dataset rather than through live trial and error—can replace the fixed rule-based interest-rate curves used by DeFi lending protocols like Aave. Using historical data from Aave V2 and V3 for the WETH and WBTC pools, the paper trains three agents (conservative Q-learning, behavior cloning, and TD3 with behavior cloning) and reports that TD3-BC learns rate-adjustment policies that outpace Aave's threshold-triggered logic. If the claim holds, lending protocols could automate interest-rate governance, reacting to utilization shifts and market stress in near real time instead of waiting for community votes. The evaluation compares the learned policy's rate trajectories with Aave's historical decisions across normal and crisis periods, including the May 2021 crash, the FTX collapse, and the March 2023 USDC depeg.

What carries the argument

The load-bearing machinery is the offline RL formulation of interest-rate setting as a Markov decision process. The state collects liquidity, debt, interest-rate, and risk indicators for a pool; the action is a relative change to the liquidity rate and the variable borrow rate; the reward penalizes deviation from an optimal utilization rate, balances borrowing cost against depositor yield, and penalizes abrupt rate jumps. TD3-BC (twin delayed deep deterministic policy gradient with behavior cloning) trains a deterministic policy with twin critics, delayed actor updates, and a behavior-cloning regularization term that keeps the policy close to historical Aave actions while still optimizing the reward. That cloning term is what lets the agent stay in-distribution and avoids the instability seen with conservative Q-learning.

What would settle it

Run the learned policy in a simulated lending environment where the agent's rate choices actually determine the following day's state, then compare realized utilization, liquidation rates, and LP yields against Aave's rule-based policy over the same stress windows; if the realized trajectories diverge from the paper's reported trajectories, the counterfactual assumption fails.

Watch

Extended reading notes

Core claim

The central discovery is that a hybrid offline RL algorithm, TD3-BC, produces interest-rate policies that are more responsive and more profitable for liquidity providers than Aave's existing utilization-kink rule, while remaining stable in stress periods. The learned policy adjusts both the variable borrow rate and the supply (liquidity) rate in small, frequent increments instead of waiting for utilization thresholds, and it reacts faster than Aave during historical stress events. The paper reports modestly higher rate volatility (0.45%–1.57% in normal periods, up to 84.49% in stress windows), which it interprets as finer risk pricing rather than instability, along with higher median liquidity rates and fewer near-zero-yield episodes for LPs. The conclusion is that offline RL can serve as automated, real-time governance for DeFi lending parameters.

Load-bearing premise

The evaluation assumes that the historical data for the following day would also be what happened if the learned policy had chosen the rates, so the reported trajectories treat the agent's choices as if they caused the observed outcomes even though no simulator or live test checks that.

Editorial extensions

If this is right

A deployed TD3-BC controller could update borrowing and supply rates more frequently and in smaller steps, removing the delay imposed by threshold-triggered rate updates and community governance votes.
During stress episodes such as the USDC depeg or FTX collapse, the learned policy would raise borrow rates faster than Aave's rules, potentially reducing bad-debt accumulation and curbing over-leveraging.
Liquidity providers would receive higher median yields and fewer near-zero-return periods, which could improve capital retention in lending pools.
The higher rate volatility observed under stress would represent deliberate risk pricing rather than instability, if the paper's interpretation is accepted.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper: the reported trajectories assume the historical next states would not change under the learned policy's actions; a simulator or live test where the agent's rates actually drive the next state is needed before deployment.
Beyond the paper: the same state-action-reward recipe could be applied to other Aave assets, such as stablecoins, and to other lending protocols that use utilization-kink interest-rate curves.
Beyond the paper: the reward weights that balance borrower cost against LP yield are a policy choice, so the claimed superiority of TD3-BC is relative to the chosen weights and could shift under different protocol objectives.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know. First, the paper does a clean job of framing Aave's interest-rate parameter adjustment as an offline RL problem, comparing CQL, BC, and TD3-BC on WETH and WBTC pools from V2 and V3. Second, the central claim that TD3-BC outperforms Aave's rule-based rates is not supported by the evidence as presented.

The genuine merits: the state/action/reward definitions are reasonable, the data span includes real stress periods, and the comparison of three offline algorithms on the same protocol is a useful empirical exercise. The paper is also well written for a practitioner audience.

The soft spot is a big one. The evaluation in Section 5 is an open-loop replay. The learned policy is applied to historical states, but the next states in the offline dataset were generated by Aave's actual actions, not by the policy's actions. So the reported rate paths, volatility numbers, and liquidity-rate distributions are not the outcomes a deployed policy would produce. This is not a minor technicality. An RL rate-setter exists precisely because interest-rate choices are supposed to change utilization, borrowing, and liquidity. The paper never models that feedback. The stress test is correct: the counterfactual is invalid.

There's also a reward/evaluation circularity. The reward penalizes utilization deviation, rate volatility, and rewards the supply rate. The main metrics are exactly those quantities. Showing that TD3-BC moves those same metrics relative to Aave is in part just reporting reward components, not independent performance. And the paper reports no error bars or multiple seeds, gives no reward coefficient values, and releases no code or data. These are all fixable, but they are not an incidental list.

Who is this for? Researchers in DeFi protocol optimization or offline RL applications in finance. The idea is worth discussing, but the current evaluation misreads what offline data can tell you.

Recommendation: I would not send this to peer review in its current form. The authors need to either build a closed-loop simulator that models how rate changes affect utilization and liquidity, or clearly recast the results as an open-loop sensitivity analysis without claiming counterfactual superiority. Once that is done, it could become a solid empirical paper. As is, the load-bearing claim turns on an invalid counterfactual.

Referee Report

3 major / 5 minor

Summary. The paper applies offline reinforcement learning to learn interest-rate adjustment policies for Aave lending pools, comparing Conservative Q-Learning (CQL), Behavior Cloning (BC), and TD3 with Behavior Cloning (TD3-BC) on historical WBTC and WETH data from Aave V2 and V3. The central claim is that TD3-BC outperforms Aave's rule-based interest-rate logic in rate responsiveness, liquidity-provider profitability, and stress-period resilience. The contributions are an MDP formulation with a hand-designed reward, a data-processing pipeline, and a comparative evaluation of three offline RL methods. The evaluation compares rate trajectories, volatility statistics, liquidity-rate distributions, and stress-event reactions between Aave's historical policy and the learned TD3-BC policy.

Significance. If the central claim were supported, this work would be a useful step toward automated, real-time interest-rate governance in DeFi lending, an area that has seen growing interest from both academia (e.g., Auto.gov) and industry (e.g., Chaos Labs). The paper's strengths include a clear MDP formulation, a detailed description of the data-preprocessing pipeline, and a transparent comparison of three offline RL methods. However, the evaluation methodology is open-loop, the metrics are partly circular with the reward function, and the stress events lie inside the training period, so the expected significance is not realized in the current version. No code or dataset is provided, limiting reproducibility.

major comments (3)

[§4 (Eq. 17), §5] The evaluation in Section 5 is an open-loop replay, not a closed-loop evaluation. The offline dataset stores transitions (S_t, A_t, R_t, S_{t+1}) where S_{t+1} is the state historically observed after Aave's actual action A_t. When the learned TD3-BC policy selects a different action A'_t ≠ A_t, the paper still uses the historical S_{t+1} and the reward computed from it. This implicitly assumes that the interest-rate action has no effect on utilization, borrowing, or liquidity — precisely the effect that an RL rate-setting policy is intended to influence. Consequently, the rate trajectories in Figure 1, the volatility statistics in Tables 9–10, the distributions in Figures 2–3, and the stress responses in Table 3 are not counterfactual outcomes of the learned policy. To support the claim of superior performance, the authors need either a simulation environment that models borrower and depositor responses to rate changes, or a valid off-policy evaluation method (e.g., importance sampling) that accounts for the discrepancy between the policy's actions and the historical actions.
[§5, Eq. (16)] The evaluation metrics are not independent of the reward function being optimized. Equation (16) rewards the supply rate (βλ S_t) and penalizes interest-rate changes via γ((Δb_t)^2+(Δr_t)^2). Section 5 then evaluates the policy on the standard deviation of rate changes (Tables 9–10) and on the distribution of liquidity rates (Figure 3), which are direct transformations of the policy's own outputs under the open-loop replay. The reported 'higher median liquidity rates' and 'reduced near-zero yield outcomes' are therefore by construction a comparison between the learned action sequence and Aave's historical action sequence, not a comparison of realized market outcomes. The claim that TD3-BC 'outperforms' Aave is circular: the model is rewarded to increase the supply rate and to adjust rates, and then is measured on its supply rate and rate-adjustment volatility. An evaluation on downstream protocol objectives — realized utilization, bad debt, liquidation shortfalls, or LP returns net of market effects — would be needed to break the circularity.
[§5 (Table 3), §4.1] The stress-event evaluation is in-sample, not a test of generalization. The dataset spans March 18, 2021 to February 25, 2025 (Section 4.1), and the paper does not specify a temporal train/test split. The FTX collapse (November 2022), the USDC depeg (March 2023), and the ETH crash (August 2024) all fall inside the training period. The claim that the policy 'without being explicitly trained to detect stress' nonetheless exhibits robust stress responses is therefore not demonstrated: the policy could simply reproduce or interpolate rate patterns seen during training. A stress-resilience claim requires evaluation on a hold-out period after training, or on a simulated stress scenario not present in the training data.

minor comments (5)

[§3, Eq. (12)] The notation is confusing: A_t is initially defined as {Δr_t, Δb_t}, then Eq. (12) writes 'A_t = A_t − A_{t−1}' while the text says 'we use relative changes.' Please clarify whether the action is the rate level or the change in rate, and use distinct symbols for the two.
[References, [6] and [7]] References [6] and [7] appear to cite the same work (Fujimoto et al., 'A Minimalist Approach to Offline Reinforcement Learning' at NeurIPS 2021) with different author lists and venues. This duplication should be corrected.
[§2.2, §4.4, Eq. (16)] The same hyperparameter is denoted inconsistently: Eq. (9) uses λ for the behavior-cloning weight, Eq. (23) uses α, and Eq. (16) also uses α for the utilization penalty. Please use a consistent notation for hyperparameters.
[Tables 9–10] The relative percentage changes in Tables 9–10 are reported without confidence intervals or significance tests; given the small differences in some rows (e.g., +0.45% in V2-WETH liquidity rate), the claim of consistent differences would be strengthened by statistical testing or by reporting raw distributions.
[§4.1, §4.4] The paper does not report the number of training samples, the train/validation split, or the hyperparameter values for α, β, λ, γ, and the behavior-cloning weight (λ_TD3BC), which limits reproducibility.

Circularity Check

2 steps flagged · score 6.0 of 10

The headline 'superior performance' of TD3-BC rests on a self-referential reward metric (LP profitability is the supply-rate term the reward directly maximizes) and on an open-loop replay that reuses historical next states, so the claimed counterfactual outcomes reduce to the paper's own inputs.

self definitional [Section 5, 'LP Profitability (Liquidity Rate Distribution)'; Reward Eq. (16)]
"LP Profitability (Liquidity Rate Distribution): ... we analyze the full distribution of liquidity rates generated under the Aave and TD3-BC policies ... The TD3-BC policy produces a liquidity rate structure that is both more rewarding and more robust to utilization fluctuations."

The action space defines Δrt as 'Change in the liquidity rate (supply interest rate)' (Eq. 11), so the 'liquidity rates generated under TD3-BC' are cumulative sums of the policy's own action outputs, not realized market yields. The reward (Eq. 16) explicitly contains +βλSt, a positive term rewarding this same supply rate. Reporting that TD3-BC gives higher median liquidity rates is therefore a restatement that the policy moved the quantity its objective was designed to increase. Actual LP profitability in Aave is Rd = Rb × U × (1 − Reserve Factor) (Eq. 3), which depends on the endogenously determined utilization U; with no environment or simulator, the policy's chosen rate path does not determine LP earnings.
other [Section 3, dataset tuple (Eq. 17); Section 5, trajectory evaluation]
"Once the state, action, reward, and next_state components are defined, the dataset is structured into state-action-reward-next_state tuples: (St, At, Rt, St+1) ... This section provides a structured interpretation of interest rate trajectories produced by Aave's rule-based policy and the TD3-BC agent."

No transition model P(s'|s,a) is specified in Section 4; S_{t+1} in each tuple is the historical state observed after Aave's actual action, not the state that would follow the RL agent's chosen Δrt and Δbt. Section 5 then presents rate series, volatility tables, and stress-event responses as if they were the policy's counterfactual outcomes. Because the historical next states are reused regardless of the policy's action, the 'trajectories' are open-loop replays: the metric is a function of the policy's outputs at historical states, not of the policy's effect on utilization, borrowing, or liquidity.

full rationale

The paper contains no external benchmark or simulator. Its strongest claims—that TD3-BC balances utilization, capital stability, and risk and outperforms rule-based logic—are supported by metrics that are either components of the training reward or open-loop replays of historical states. The LP-profitability result is the clearest circular step: the reward explicitly rewards the supply rate, and the evaluation measures the supply-rate distribution generated by the policy's own actions, so the 'prediction' of higher LP returns is a check that the policy optimized its own objective rather than an external outcome. The stress-period and volatility claims additionally rely on reusing historical S_{t+1} tuples, so the policy's rate adjustments are never allowed to alter the future states they are supposed to influence. There is no significant load-bearing self-citation chain; the authors' prior SoK is used only as background. The open-loop evaluation and the reward-metric overlap are distinct flaws, but together they mean the central performance prediction reduces to the paper's own definitions, warranting a partial-circularity score of 6 rather than a higher score that would require a fully self-citation-forced derivation.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on multiple free coefficients in the reward (α, β, λ, γ, U*, and the TD3-BC regularization weight) whose values are not reported. The evaluation also implicitly assumes that historical states can stand in for a real or simulated lending environment, which is a domain assumption that is not validated.

free parameters (6)

α (utilization deviation penalty)
Eq. 16; not reported. Determines how strongly the policy enforces the U* target and therefore shapes rate actions.
β (borrow cost penalty coefficient)
Eq. 16; not reported. Balances borrower cost reduction against lender return in the reward.
λ (borrow/supply balance scaling)
Eq. 16; not reported. Scales the supply-rate reward relative to the borrow-rate penalty in the reward.
γ (rate stability penalty coefficient)
Eq. 16; not reported. Penalizes squared rate changes and directly suppresses the kind of volatility the paper later measures.
U* (optimal utilization target)
Eqs. 1 and 13; not reported for WETH/WBTC V2/V3. The quadratic utilization penalty assumes a known optimal utilization rate.
λ_TD3BC (behavior cloning regularization coefficient)
Eq. 9; not reported. Controls the trade-off between the TD3 objective and the behavior cloning objective in TD3-BC training.

assumptions (5)

domain assumption The AaveScan historical dataset provides a complete and representative record of lending-pool states and rate actions.
Used in Section 4.1 as the sole source for state-action-reward-next_state tuples; sampling or data errors would propagate through training and evaluation.
domain assumption The specified MDP (state S_t, action A_t, reward R_t, next state S_{t+1}) captures the lending protocol's true dynamics.
Section 3, Eqs. 10-17. The historical tuples are treated as valid transitions, but the next state is not conditioned on the RL action during evaluation.
ad hoc to paper The hand-designed reward (Eq. 16) represents protocol objectives of capital efficiency, borrower cost, and rate stability.
Section 3; the coefficients α, β, λ, γ are author-defined and are also the basis of the evaluation metrics.
domain assumption Evaluating the learned policy on historical states, without a closed-loop environment, yields meaningful counterfactual performance.
Implicit throughout Section 5; rate trajectories are generated from historical states while the paper does not model how users or utilization respond to RL rate changes.
standard math Standard offline RL assumptions (sufficient dataset coverage, no severe distribution shift) hold for the Aave dataset.
Required for CQL and TD3-BC value estimates to be reliable; not tested in the paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of From Rules to Rewards: Reinforcement Learning for Interest Rate Adjustment in DeFi Lending." pith.science (2026). https://pith.science/paper/DWCJQ3OR

@misc{pith2026250600505,
  author       = {Pith},
  title        = {Pith review of: From Rules to Rewards: Reinforcement Learning for Interest Rate Adjustment in DeFi Lending},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DWCJQ3OR}},
  note         = {Machine review of arXiv:2506.00505}
}

read the original abstract

Decentralized Finance (DeFi) lending enables permissionless borrowing via smart contracts. However, it faces challenges in optimizing interest rates, mitigating bad debt, and improving capital efficiency. Rule-based interest-rate models struggle to adapt to dynamic market conditions, leading to inefficiencies. This work applies Offline Reinforcement Learning (RL) to optimize interest rate adjustments in DeFi lending protocols. Using historical data from Aave protocol, we evaluate three RL approaches: Conservative Q-Learning (CQL), Behavior Cloning (BC), and TD3 with Behavior Cloning (TD3-BC). TD3-BC demonstrates superior performance in balancing utilization, capital stability, and risk, outperforming existing models. It adapts effectively to historical stress events like the May 2021 crash and the March 2023 USDC depeg, showcasing potential for automated, real-time governance.

Figures

Figures reproduced from arXiv: 2506.00505 by the authors.

**Figure 2.** Distribution of RL Borrow Rate Adjustment Magnitudes (Log Scale) [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗

**Figure 3.** Liquidity Rate Distribution by Protocol and Asset (Log Scale) [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗

Figures from the paper (11 more)

**Figure 4.** Figure 4: Interest Rate Trends Over Time The study of interest rate fluctuations across Aave V2 and V3 presents a compelling narrative of market adaptation, liquidity shocks, and strategic governance interventions within decentralized finance (DeFi). Our empirical observations…

**Figure 5.** Figure 5: Comparison of critic loss for WETH using CQL [PITH_FULL_IMAGE:figures/full_fig_p029_5.png]

**Figure 6.** Figure 6: Comparison of critic loss for WBTC using CQL [PITH_FULL_IMAGE:figures/full_fig_p029_6.png]

**Figure 7.** Figure 7: Comparison of actor loss for WETH using CQL [PITH_FULL_IMAGE:figures/full_fig_p030_7.png]

**Figure 8.** Figure 8: Comparison of actor loss for WBTC using CQL [PITH_FULL_IMAGE:figures/full_fig_p031_8.png]

**Figure 9.** Figure 9: Comparison of actor loss for WETH using BC [PITH_FULL_IMAGE:figures/full_fig_p031_9.png]

**Figure 10.** Figure 10: Comparison of actor loss for WBTC using BC [PITH_FULL_IMAGE:figures/full_fig_p032_10.png]

**Figure 11.** Figure 11: Comparison of critic loss for WETH using TD3BC [PITH_FULL_IMAGE:figures/full_fig_p033_11.png]

**Figure 12.** Figure 12: Comparison of critic loss for WBTC using TD3BC [PITH_FULL_IMAGE:figures/full_fig_p033_12.png]

**Figure 13.** Figure 13: Comparison of actor loss for WETH using TD3BC [PITH_FULL_IMAGE:figures/full_fig_p034_13.png]

**Figure 14.** Figure 14: Comparison of actor loss for WBTC using TD3BC [PITH_FULL_IMAGE:figures/full_fig_p034_14.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

29 extracted references · 29 canonical work pages

[1]

BIS Quarterly Review (2021)

Aramonte, S., Huang, W., Schrimpf, A.: Defi risks and the decentralisation illusion. BIS Quarterly Review (2021)

work page 2021
[2]

FSB Report (2023)

Board, F.S.: The financial stability risks of decentralised finance. FSB Report (2023)

work page 2023
[3]

Crypto, U.: Unchained crypto news (2025),https://www.unchainedcrypto.com, accessed: 2025-03-09 Reinforcement Learning in DeFi Lending 17

work page 2025
[4]

DeFi Llama: Total Value Locked All Chains (2024), https://defillama.com/ chains, accessed on February 7, 2025

work page 2024
[5]

Forum, A.G.: Aave governance forum (2025),https://governance.aave.com, ac- cessed: 2025-03-09

work page 2025
[6]

Advances in Neural Information Processing Systems (NeurIPS) (2021)

Fujimoto, S., Gu, S.S.: A minimalist approach to offline reinforcement learning. Advances in Neural Information Processing Systems (NeurIPS) (2021)

work page 2021
[7]

In: NeurIPS (2021)

Fujimoto, S., Gu, D., Henderson, D.: A Minimalist Approach to Offline Reinforce- ment Learning. In: NeurIPS (2021)

work page 2021
[8]

International Conference on Machine Learning (ICML) (2018)

Fujimoto, S., Hoof, H.v., Meger, D.: Addressing function approximation error in actor-critic methods. International Conference on Machine Learning (ICML) (2018)

work page 2018

Show all 29 references

[9]

Gogol, K., Killer, C., Schlosser, M., Bocek, T., Stiller, B., Tessone, C.: Sok: De- centralized finance (defi) – fundamentals, taxonomy and risks (2024)

2024
[10]

Gudgeon, L., Werner, S.M., Perez, D., Knottenbelt, W.J.: Defi protocols for loan- able funds: Interest rates, liquidity and market efficiency (2020)

2020
[11]

NeurIPS (2020)

Kumar, A., Zhou, A., Tucker, G., Levine, S.: Conservative q-learning for offline reinforcement learning. NeurIPS (2020)

2020
[12]

arXiv preprint arXiv:2003.00042 (2020)

Kumar, A., Fu, J., Soh, M., Levine, S.: Implicit under-parameterization inhibits data-efficient deep reinforcement learning. arXiv preprint arXiv:2003.00042 (2020)

2020 arXiv
[13]

venus.io/t/chaos-labs-risk-parameter-updates-08-28-2023/3720 , accessed: 2024-05-19

Labs, C.: Risk parameter updates for venus protocol (2023),https://community. venus.io/t/chaos-labs-risk-parameter-updates-08-28-2023/3720 , accessed: 2024-05-19

2023
[14]

Labs, C.: Introducing edge: The next generation oracle protocol (2024)

2024
[15]

Master’s thesis, ETH Zürich, Zurich, Switzerland (May 2023),https:// pub.tik.ee.ethz.ch/students/2022-HS/MA-2022-28.pdf, supervisors: Ye Wang, Benjamin Estermann; Prof

Plüss, J.D.: Exploring Trading on Decentralized Exchanges using Reinforcement Learning. Master’s thesis, ETH Zürich, Zurich, Switzerland (May 2023),https:// pub.tik.ee.ethz.ch/students/2022-HS/MA-2022-28.pdf, supervisors: Ye Wang, Benjamin Estermann; Prof. Dr. Roger Wattenhofer

2023
[16]

Schär, F.: Decentralized finance: On blockchain- and smart contract-based financial markets (2020)

2020
[17]

MIT Press, 2nd edn

Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, 2nd edn. (2018)

2018
[18]

Interna- tional Joint Conference on Artificial Intelligence (IJCAI) (2018)

Torabi, F., Warnell, G., Stone, P.: Behavioral cloning from observation. Interna- tional Joint Conference on Artificial Intelligence (IJCAI) (2018)

2018
[19]

Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI) (2018)

Torabi, F., Warnell, G., Stone, P.: Behavior cloning from observation. Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI) (2018)

2018
[20]

In: Proceedings of the 4th ACM Conference on Advances in Financial Technologies

Werner, S., Perez, D., Gudgeon, L., Klages-Mundt, A., Harz, D., Knottenbelt, W.: Sok: Decentralized finance (defi). In: Proceedings of the 4th ACM Conference on Advances in Financial Technologies. pp. 30–46 (2022)

2022
[21]

arXiv preprint arXiv:2311.17715 (2023)

Xiong, X., Wang, Z., Cui, T., Knottenbelt, W., Huth, M.: Market misconduct in de- centralized finance (defi): Analysis, regulatory challenges and policy implications. arXiv preprint arXiv:2311.17715 (2023)

2023 arXiv
[22]

Xu, H., Brini, A.: Improving defi accessibility through efficient liquidity provision- ing with deep reinforcement learning (2025)

2025
[23]

Xu, J., Perez, D., Feng, Y., Livshits, B.: Auto.gov: Learning-based on-chain gov- ernance for defi (2023)

2023
[24]

Zhou, L., Xiong, X., Ernstberger, J., Chaliasos, S., Wang, Z., Wang, Y., Qin, K., Wattenhofer, R., Song, D., Gervais, A.: Sok: Decentralized finance (defi) attacks. arXiv preprint arXiv:2208.13035 (2022) 18 Hanxiao Qu, Krzysztof Gogol, Florian Grötschla, Claudio Tessone A Rela...

2022 arXiv
[25]

Timestamp Processing: The _block_timestamp field is converted from UNIX time to a human-readable date format
[26]

Each value is scaled down using the appropriate decimal conversion: totalLiquidity = totalLiquidity 10decimals

Normalization of On-Chain Values: Most values in the dataset are stored as large integers due to Ethereum’s decimal precision (1018). Each value is scaled down using the appropriate decimal conversion: totalLiquidity = totalLiquidity 10decimals
[27]

The APY is computed as: AP Y= 1 + rateAPR 365 365 − 1

Computing Interest Rate APY:Aave’s rates are stored as APRs (An- nual Percentage Rates). The APY is computed as: AP Y= 1 + rateAPR 365 365 − 1
[28]

Risk Metric Computation: – Loan-to-Value Ratio (LTV) is calculated as: LT V= Total Debt Total Liquidity as Collateral+ 10−6 – Liquidation Risk: isAtRisk = ( 1, LT V >reserveLiquidationThreshold 0, otherwise
[29]

BGD: Full Deprecation of Stable Rate Borrowing

Interest Rate Momentum and Volatility: – Momentum is computed using a rolling average: Momentumt = 1 n tX i=t−n Ri – Volatility is calculated as: Volatilityt = vuut 1 n tX i=t−n (Ri − ¯R)2 Reinforcement Learning in DeFi Lending 23 D Exploratory Data Analysis (EDA) Interest Rat...

2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

BIS Quarterly Review (2021)

Aramonte, S., Huang, W., Schrimpf, A.: Defi risks and the decentralisation illusion. BIS Quarterly Review (2021)

work page 2021

[2] [2]

FSB Report (2023)

Board, F.S.: The financial stability risks of decentralised finance. FSB Report (2023)

work page 2023

[3] [3]

Crypto, U.: Unchained crypto news (2025),https://www.unchainedcrypto.com, accessed: 2025-03-09 Reinforcement Learning in DeFi Lending 17

work page 2025

[4] [4]

DeFi Llama: Total Value Locked All Chains (2024), https://defillama.com/ chains, accessed on February 7, 2025

work page 2024

[5] [5]

Forum, A.G.: Aave governance forum (2025),https://governance.aave.com, ac- cessed: 2025-03-09

work page 2025

[6] [6]

Advances in Neural Information Processing Systems (NeurIPS) (2021)

Fujimoto, S., Gu, S.S.: A minimalist approach to offline reinforcement learning. Advances in Neural Information Processing Systems (NeurIPS) (2021)

work page 2021

[7] [7]

In: NeurIPS (2021)

Fujimoto, S., Gu, D., Henderson, D.: A Minimalist Approach to Offline Reinforce- ment Learning. In: NeurIPS (2021)

work page 2021

[8] [8]

International Conference on Machine Learning (ICML) (2018)

Fujimoto, S., Hoof, H.v., Meger, D.: Addressing function approximation error in actor-critic methods. International Conference on Machine Learning (ICML) (2018)

work page 2018