Financial Decision Making using Reinforcement Learning with Dirichlet Priors and Quantum-Inspired Genetic Optimization

Debjit Dhar; Prasun Nandy; Rik Das

REVIEW 4 major objections 5 minor 33 references

Financial Decision Making using Reinforcement Learning with Dirichlet Priors and Quantum-Inspired Genetic Optimization

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Quantum-inspired RL matches Apple's budget splits to 0.999 cosine

desk verdict The headline cosine similarity is an artifact of the reward function, which trains the policy to match next-quarter actual allocations and then evaluates against the same values; the paper does not demonstrate predictive or prescriptive budgeting value. read the letter →

arxiv 2509.00095 v1 pith:6EL3OTD7 submitted 2025-08-27 cs.LG cs.NE

Prasun Nandy , Debjit Dhar , Rik Das This is my paper · ORCID

classification cs.LGcs.NE

keywords budgetallocationreinforcementlearningDirichletpriorsTD3geneticalgorithmquantum-inspiredoptimizationfinancialdecisionmakingBayesianupdating

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a hybrid budget-allocation model—deep reinforcement learning with a Dirichlet prior over spending categories and a genetic optimizer that mutates policy weights with quantum-style rotations—can reproduce a large company's actual R&D versus SG&A splits on data it has not seen. Working from Apple's quarterly financials (2009–2025), the agent learns a two-way split that balances net income against three penalties: closeness to the observed next-quarter split, smoothness from one quarter to the next, and disagreement with a Dirichlet prior. The best configuration reaches cosine similarity 0.9990 and KL divergence 0.0023 against the real allocations, improving from 0.8832 without the genetic layer and 0.9813 with genetics but no quantum-style mutation. The value of the claim, if true, is that budget decisions can be cast as learnable, interpretable, uncertainty-aware policies rather than static rules.

What carries the argument

The load-bearing object is the reward function in Eq. (5): $r_t = -\|a_t - \hat{a}_t\|_1 - \lambda_1 \|a_t - a_{t-1}\|_2 - \lambda_2 D_{\mathrm{KL}}(\mathrm{Dir}(\alpha_t) \| \mathrm{Dir}(\alpha_{\mathrm{prior}}))$. The first term pushes the agent to imitate the actual next-quarter allocation split, the second discourages abrupt changes, and the third keeps the agent's Dirichlet belief, updated as $\alpha_t = \alpha_{t-1} + \hat{a}_t \cdot c$, close to its prior. The trained actor's weights are then treated as genes, and a rotation gate $R(\Delta\theta)$ applied to each qubit-encoded gene supplies the mutation that lets the search escape local optima.

What would settle it

Run the same training and evaluation while replacing the reward target with a forecast of next-quarter allocations made from data available at time $t$, or compare the trained policy against a trivial baseline that outputs the previous quarter's split; if the trivial baseline also scores cosine similarity near 0.9990, or if removing the lookahead target destroys the score, then the result is an artifact of the reward design rather than evidence of adaptive budgeting.

Watch

Extended reading notes

Core claim

The paper's central claim is that combining three components in one pipeline—TD3 deep RL, a Dirichlet belief state updated from observed allocation ratios, and a genetic algorithm whose mutation step rotates qubit-encoded gene amplitudes by a parameterized angle—produces budget allocations that track Apple's historical R&D/SG&A split almost exactly on held-out quarters. The reported test-set numbers for the full model are MAE 0.0229, RMSE 0.0283, cosine similarity 0.9990, and KL divergence 0.0023. Each ablation step degrades the fit, with RL plus Dirichlet priors alone at 0.8832 and the non-quantum genetic version at 0.9813, so the evolutionary fine-tuning stages are credited with most of the final accuracy.

Load-bearing premise

The whole evaluation rests on treating Apple's historical split between R&D and SG&A as the ground truth for good budgeting, and on making that split visible to the agent through the reward term; if matching past allocations is not the right objective, the reported 0.9990 similarity is a measure of imitation, not of decision quality.

Editorial extensions

If this is right

Adding the genetic and quantum-style refinement layers raises test-set cosine similarity from 0.8832 for RL plus Dirichlet priors to 0.9990, so the paper's central finding is that the evolutionary fine-tuning stage carries much of the performance.
Because the Dirichlet posterior is updated from observed splits and penalized against a prior, the policy retains an interpretable belief about plausible allocations rather than a single point estimate.
The ablation ordering implies the non-quantum genetic stage alone accounts for a large jump to 0.9813, with the quantum-style mutation supplying the remaining refinement.
The authors' stated extensions—adding macroeconomic indicators, constraints, real-hardware quantum tests, and multi-agent versions—outline a route from this two-category allocation problem to fuller budgeting systems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The headline cosine similarity is computed against the same historical allocation ratios that the reward's first term pushes the agent to imitate, so the metric should be read as a fidelity-to-past-behavior score, not as evidence that the policy improves future profitability.
Because the target $\hat{a}_t$ is the actual next-quarter split, the reward contains lookahead information; a deployable version would have to substitute a forecast, and the reported 0.9990 fit would almost certainly degrade under that change.
A natural stress test, not run in the paper, is to compare against a no-learning baseline that simply repeats the most recent observed split; if that baseline scores near 0.9990 cosine similarity, the learned policy's apparent success is largely the persistence of Apple's allocation ratios.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: the central result does not survive contact with the reward function. Eq. (5) penalizes -||a_t - a_hat_t||_1, where a_hat_t is the actual next-quarter allocation, and Table I evaluates the policy against exactly that same a_hat_t sequence. So the 0.9990 cosine similarity is a fit-to-label number, not evidence of predictive or prescriptive validity. The stress-test note holds up: the last training transition uses the first test quarter's actuals, so the chronological split leaks; and a persistence baseline that repeats the previous quarter's actual ratio would likely score near 0.9990 because Apple's R&D/SG&A split is smooth. The profit objective described in the text is absent from Eq. (5) — the equation only contains the L1 accuracy term, an L2 smoothness term, and the Dirichlet KL penalty. That is a load-bearing omission, not a typo.

What is new and worth credit: the specific combination of TD3 with a Dirichlet belief update plus quantum-inspired genetic mutation applied to a corporate budget split appears not to have been published before. The methodology is described clearly enough to follow, the libraries are standard, and the GA hyperparameters are reported (10 generations, population 5, elite 0.4, mutation 0.1). The Dirichlet KL penalty is a reasonable regularization idea, even if the prior [5.0, 3.0] is set arbitrarily.

Soft spots, in proportion: no code or data are provided; no seed variance is reported; the best seed is chosen post hoc (seed=60); TD3 hyperparameters are unreported. Minor but telling: Figure 2's caption says "RL + SPC + Quantum-Genetic Algorithm" while the text says RL+DP+Genetic+Quantum, likely a leftover from an earlier draft. Also reference [20] appears to be a convex optimization survey, not the convergence analysis of Bayesian RL that the sentence implies. The conclusion overclaims "guaranteed returns."

Who this is for: someone curious about how NOT to evaluate a financial RL agent, or a reader wanting a checklist of evaluation pitfalls. As a contribution, it does not deserve a serious referee slot yet. The right next step is to rewrite the reward to actually include profit, hold out a proper test set with no t+1 leakage, compare against persistence and simple baselines, and report distribution over seeds. If the authors do that, the quantum-genetic part might become interesting; right now it is decorative. My recommendation: desk reject.

Referee Report

4 major / 5 minor

Summary. The paper proposes a hybrid framework for quarterly budget allocation between R&D and SG&A, combining TD3 reinforcement learning with Dirichlet-based Bayesian belief updates and a genetic algorithm whose mutation operator is inspired by quantum rotation gates. The agent is trained on Apple Inc. quarterly data (2009–2025) with an 80/20 chronological split, and the authors report high alignment with actual allocations on the held-out test set (cosine similarity 0.9990, KL divergence 0.0023). The abstract claims that the model maximizes profitability while adhering to historical spending patterns, and the paper concludes that the framework can deliver 'adaptive, forward-looking policies' for corporate budgeting.

Significance. If the reported results were valid, the paper would offer a novel integration of Bayesian RL and quantum-inspired evolutionary search for a practical financial allocation problem, and the use of real Apple data is a useful feature. The paper also documents a complete, modular pipeline with explicit hyperparameters, which aids reproducibility. However, the central empirical claim is not currently supported: the reward function in Eq. (5) trains the policy to reproduce next-quarter actual allocations, and the evaluation in Table I measures agreement with the same target, making the headline numbers a measure of training fit rather than predictive or prescriptive validity. The absence of a profitability term in the reward, the lack of error bars, and post hoc seed selection further undermine the significance claims.

major comments (4)

[III-B, Eq. (5)] The reward function in Eq. (5) does not contain the profitability term that the text repeatedly describes. Section III-B states that the reward integrates profitability and allocation accuracy, and the introduction says the agent learns to 'maximize net income while staying close to historically realistic ratios.' However, Eq. (5) is rt = -||a_t - a_hat_t||_1 - lambda1 * ||a_t - a_{t-1}||_2 - lambda2 * KL(Dir(alpha_t)||Dir(alpha_prior)), with no net-income or profit term. Consequently, the model is not trained to maximize profitability at all; it is trained solely to match the next-quarter empirical allocation a_hat_t and to maintain smoothness and belief coherence. The abstract's claim that the framework 'maximizes profitability' is therefore unsupported by the stated objective.
[III-A Eq. (4) and IV Table I] The evaluation metric is the same target used in the training reward, making the reported results circular. Eq. (4) defines a_hat_t as the actual next-quarter allocation computed from R&D_{t+1} and SG&A_{t+1}, and the first reward term in Eq. (5) penalizes ||a_t - a_hat_t||_1. Table I then reports cosine similarity and KL divergence between the model's outputs and this same a_hat_t sequence on the held-out set. A cosine similarity of 0.9990 and KL divergence of 0.0023 therefore quantify how well the optimization reproduced the labels it was trained to imitate, not whether the model discovered profitable or generalizable budgeting rules. A persistence baseline that repeats the previous quarter's observed allocation would likely score similarly because Apple's R&D/SG&A ratios are slowly varying; no such baseline is reported.
[III-A, IV] The chronological split does not fully prevent data leakage because the reward for the last training transition uses test data. Eq. (4) defines a_hat_t using R&D_{t+1} and SG&A_{t+1}, and Section IV states an 80/20 chronological split. Thus, the reward for the final training timestep at t = 0.8T is computed from the first quarter of the held-out test period (t+1 = 0.8T+1). This leaks test information into training and invalidates the claim that the evaluation is on 'unseen fiscal data.' The leakage also affects the Dirichlet update in Eq. (6), which uses the same a_hat_t.
[IV, Table I] Table I reports only single-run values and explicitly states that the best scores were obtained by setting seed=60, which indicates post hoc seed selection. No error bars, standard deviations, or multiple-seed averages are provided, so it is impossible to assess the statistical significance of the improvements from the genetic and quantum-inspired components. The comparison in Table I is also incomplete: it does not include a persistence baseline, a random policy, or a standard GA without quantum mutation, so the marginal contributions of the Dirichlet prior and the quantum-inspired operator cannot be separated from variance or from the GA's general exploration.

minor comments (5)

[III-D, IV] The quantum-inspired mutation is described qualitatively as 'non-local' and 'probabilistic,' but no experiment isolates its effect. The paper should compare the quantum mutation against a Gaussian noise mutation with the same budget and also report the distribution of fitness across GA generations with error bars, as Figure 3 currently shows a single trajectory.
[Figure 2 caption] The caption refers to 'RL + SPC + Quantum-Genetic Algorithm,' but the configuration in Table I and text is 'RL+DP+Genetic+Quantum' or 'RL+DP+Genetic+Quantum.' The acronym SPC is never defined.
[V, Conclusion] The final paragraph states that the framework enables firms to 'generate sustained and guaranteed returns over time.' Given the evaluation limitations and the absence of any counterfactual profitability analysis, 'guaranteed returns' is an overclaim and should be removed.
[Financial Disclosure] The financial disclosure says only that 'the authors declare the financial support received for this work' without identifying the funding source; if there is a sponsor, it should be named.
[III-C, Eq. (9)] The TD3 update equations are standard, but the notation for the target value y in Eq. (9b) omits the target-network parameter dependence; it should be written as y = r + gamma * min(Q1'(s', pi'(s')), Q2'(s', pi'(s'))) to be precise.

Circularity Check

2 steps flagged · score 6.0 of 10

The reported 0.9990 cosine similarity is measured against the same actual allocations used as the training reward (Eq. 5 via Eq. 4), so it largely quantifies fit to the label rather than independent predictive or prescriptive validity.

fitted input called prediction [Abstract; Eqs. (4)-(5); Table I (Sections III-B and IV)]
"Equation (4) computes ˆat = [R&D_{t+1}/(R&D_{t+1}+SG&A_{t+1}), SG&A_{t+1}/(R&D_{t+1}+SG&A_{t+1})], the empirical budget allocation at timestep t, normalized to form a probability distribution over categories. This serves as ground-truth feedback for policy evaluation. rt = −∥at− ˆat∥1−λ1∥at−at−1∥2 −λ2·DKL (Dir(αt)∥ Dir(αprior)) (5). Table I: RL+DP+Genetic+Quantum 0.0229 0.0283 0.9990 0.0023."

The first reward term in Eq. (5) trains the policy to output the actual next-quarter allocation a_hat_t defined in Eq. (4). Table I then evaluates the policy by comparing its outputs to exactly this same a_hat_t sequence using MAE, RMSE, cosine similarity, and KL divergence. The reported 0.9990/0.0023 therefore measure how well the optimization fit the label it was rewarded to reproduce, not whether the model discovered profitable or generalizable budgeting rules. The text promises profit maximization, but Eq. (5) contains only accuracy, smoothness, and Dirichlet terms, so this alignment is the entire quantitative basis for the central claim.
other [Section III, 'Dataset and Preprocessing'; Eq. (4)]
"A chronological split ensures that the temporal structure of the financial data is preserved for training and evaluation, thus avoiding data leakage."

The claim of avoiding data leakage is contradicted by the paper's own Eq. (4): a_hat_t is computed from R&D_{t+1} and SG&A_{t+1}, the next quarter's actual figures. Under an 80/20 chronological split, the reward for the last training transition uses the first quarter of the held-out test set, so a portion of the 'unseen fiscal data' enters training. The test-set boundary is therefore not clean, and the out-of-sample alignment is partly an in-sample fit.

full rationale

The central reported result is high alignment with actual Apple allocations on a held-out split. That alignment is the same quantity the policy is explicitly rewarded to match: Eq. (5) minimizes ||a_t − a_hat_t||_1, and Table I scores cosine/KL against the same a_hat_t. This makes the headline number a fit-to-label measure rather than independent evidence for the RL/quantum-genetic framework. The boundary leakage from Eq. (4)'s t+1 target compounds the issue by putting the first test quarter's actuals into the final training reward. I do not score 8-10 because most of the split is genuinely out-of-sample, the policy must still generalize across quarters, and the absence of a persistence baseline is a benchmarking gap rather than a definitional identity. No load-bearing self-citation or imported uniqueness theorem appears; citations [4] and [10] are ordinary prior-work references. The main defect is thus the reward/evaluation identity and the leakage, which partially reduce the central prediction to its own training target.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

Every reported alignment number depends on the reward/evaluation symmetry: the L1 accuracy term in Eq. (5) is computed against the same actual allocation used for cosine similarity and KL divergence in Table I. The model is therefore a supervised imitator of historical ratios. The remaining assumptions are standard library behavior, the accuracy of the Macrotrends dataset, and the choice of several unreported hyperparameters. No independently falsifiable entity is introduced.

free parameters (6)

lambda_1 (temporal smoothness weight)
Regularization weight in Eq. (5) for ||a_t - a_{t-1}||_2; value not reported.
lambda_2 (Dirichlet KL weight)
Weight on DKL term in Eq. (5); value not reported.
c (Dirichlet update scaling factor)
Controls effective sample size in Eq. (6); value not reported.
alpha_prior = [5.0, 3.0]
Hand-chosen prior concentration parameters in Eq. (8); not fitted to data but chosen by authors.
GA hyperparameters = population_size=5, generations=10, elite_fraction=0.4, mutation_rate=0.1
Set by hand in Section III-D; no sensitivity analysis.
seed = 60
Table I caption says best scores obtained by setting seed=60, indicating the seed was chosen to maximize results.

assumptions (5)

domain assumption The Stable-Baselines3 and PennyLane libraries correctly implement TD3 and quantum rotation operations.
Section III-C and III-D rely on library correctness without verification.
domain assumption Macrotrends financial data for Apple is accurate, quarterly, and complete from 2009 to 2025.
Reference [9] is the sole data source; no cleaning or validation beyond dropping missing values is described.
standard math The Dirichlet conjugate update alpha_t = alpha_{t-1} + a_hat_t * c correctly models evolving budget beliefs.
Dirichlet-multinomial conjugacy is standard, but the choice of c and the interpretation of a_hat as counts is an assumption.
ad hoc to paper Using the actual next-quarter allocation as the reward target is an appropriate learning signal for budget optimization.
This assumption is what makes the evaluation metric align with the training objective; it is not justified by any theory of optimal budgeting.
domain assumption The state transition in the environment is the real chronological sequence of Apple's financials, independently of the agent's actions.
The environment is non-interactive, so the RL formulation is effectively a supervised time-series labeling task.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Financial Decision Making using Reinforcement Learning with Dirichlet Priors and Quantum-Inspired Genetic Optimization." pith.science (2026). https://pith.science/paper/6EL3OTD7

@misc{pith2026250900095,
  author       = {Pith},
  title        = {Pith review of: Financial Decision Making using Reinforcement Learning with Dirichlet Priors and Quantum-Inspired Genetic Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6EL3OTD7}},
  note         = {Machine review of arXiv:2509.00095}
}

read the original abstract

Traditional budget allocation models struggle with the stochastic and nonlinear nature of real-world financial data. This study proposes a hybrid reinforcement learning (RL) framework for dynamic budget allocation, enhanced with Dirichlet-inspired stochasticity and quantum mutation-based genetic optimization. Using Apple Inc. quarterly financial data (2009 to 2025), the RL agent learns to allocate budgets between Research and Development and Selling, General and Administrative to maximize profitability while adhering to historical spending patterns, with L2 penalties discouraging unrealistic deviations. A Dirichlet distribution governs state evolution to simulate shifting financial contexts. To escape local minima and improve generalization, the trained policy is refined using genetic algorithms with quantum mutation via parameterized qubit rotation circuits. Generation-wise rewards and penalties are logged to visualize convergence and policy behavior. On unseen fiscal data, the model achieves high alignment with actual allocations (cosine similarity 0.9990, KL divergence 0.0023), demonstrating the promise of combining deep RL, stochastic modeling, and quantum-inspired heuristics for adaptive enterprise budgeting.

Figures

Figures reproduced from arXiv: 2509.00095 by the authors.

**Figure 1.** Methodology Pipeline • Belief Coherence (DKL): Penalizes excessive divergence from learned priors, enforcing epistemic consistency. Unlike deterministic SPC rule-based penalization, this formulation allows the agent to operate under epistemic uncertainty and to evolve its belief distribution based on empirical feedback through Dirichlet posterior updates. This Bayesian approach is better suited to real-world finan… view at source ↗

**Figure 2.** Allocation predictions using RL + SPC + Quantum-Genetic Algorithm [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

**Figure 3.** GA Optimization [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Quantum Mutations The custom Gym environment simulates economic decisionmaking under uncertainty, incorporating Bayesian updating of budget priorities and a reward function that penalizes deviation from historical allocations via L2 loss. This dual-objective reward—…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 19 canonical work pages

[20]

Convex optimization: Algorithms and complexity,

M. Ghavamzadeh, S. Mannor, J. Pineau, and A. Tamar, “Convex optimization: Algorithms and complexity,” Foundations and Trends® in Machine Learning , vol. 8, no. 5–6, p. 359–483, 2015. [Online]. Available: http://dx.doi.org/10.1561/2200000049

doi:10.1561/2200000049 2015
[1]

A novel application of statistical process control charts in financial market surveillance with the idea of profile monitoring,

A. Yeganeh and S. C. Shongwe, “A novel application of statistical process control charts in financial market surveillance with the idea of profile monitoring,” PLOS ONE , vol. 18, no. 7, pp. 1–26, 07 2023. [Online]. Available: https://doi.org/10.1371/journal.pone.0288627

work page doi:10.1371/journal.pone.0288627 2023
[2]

The role of the business model in capturing value from innovation: evidence from xerox corporation’s technology spin-off companies,

H. Chesbrough and R. S. Rosenbloom, “The role of the business model in capturing value from innovation: evidence from xerox corporation’s technology spin-off companies,” Industrial and Corporate Change, vol. 11, no. 3, pp. 529–555, 06 2002. [Online]. Available: https://doi.org/10.1093/icc/11.3.529

work page doi:10.1093/icc/11.3.529 2002
[3]

A budget-adaptive allocation rule for optimal computing budget allocation,

Z. Cao, H. Wang, E. P. Chew, H. Li, and K. C. Tan, “A budget-adaptive allocation rule for optimal computing budget allocation,” European Journal of Operational Research , vol. 325, no. 2, pp. 247–260, 2025. [Online]. Available: https://www.sciencedirect.com/science/article/pii/ S0377221725002656

work page 2025
[4]

A dual-model solution for business reasoning with large language mod- els,

S. Trivedi, S. Roy, R. Das, D. Mukherjee, S. Mahapatra, and P. Nandy, “A dual-model solution for business reasoning with large language mod- els,” in 2024 IEEE Pune Section International Conference (PuneCon) , 2024, pp. 1–6

work page 2024
[5]

Introduction to reinforcement learning,

M. Ghasemi and D. Ebrahimi, “Introduction to reinforcement learning,”

work page
[6]

Actor-critic reinforcement learning for control with stability guarantee,

M. Han, L. Zhang, J. Wang, and W. Pan, “Actor-critic reinforcement learning for control with stability guarantee,” 2020. [Online]. Available: https://arxiv.org/abs/2004.14288

arXiv 2020
[7]

Reinforcement learning: An introduction,

R. Sutton and A. Barto, “Reinforcement learning: An introduction,” IEEE Transactions on Neural Networks , vol. 9, no. 5, pp. 1054–1054, 1998

1998

Show all 33 references

[8]

A review of reinforcement learning in financial applications,

Y . Bai, Y . Gao, R. Wan, S. Zhang, and R. Song, “A review of reinforcement learning in financial applications,” 2024. [Online]. Available: https://arxiv.org/abs/2411.12746

2024 arXiv
[9]

Apple income statement 2010–2024 — aapl,

M. LLC, “Apple income statement 2010–2024 — aapl,” https://www. macrotrends.net/stocks/charts/AAPL/apple/income-statement?freq=A, 2024, accessed: July 22, 2025

2010
[10]

Harnessing the power of quantum feature representation for lever- aging fused descriptor definition in breast cancer diagnosis,

R. Das, N. Dutta, D. Mukherjee, S. Mahapatra, I. Mitra, and P. Nandy, “Harnessing the power of quantum feature representation for lever- aging fused descriptor definition in breast cancer diagnosis,” in 2024 International Conference on Electrical, Communication and Computer En...

2024
[11]

Playing atari with deep reinforcement learning,

V . Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing atari with deep reinforcement learning,” 2013. [Online]. Available: https://arxiv.org/abs/1312.5602

2013 arXiv
[12]

Optimizing automated trading systems with deep reinforcement learning,

M. Tran, D. Pham-Hi, and M. Bui, “Optimizing automated trading systems with deep reinforcement learning,” Algorithms, vol. 16, no. 1,
[13]

Causal inference on investment constraints and non-stationarity in dynamic portfolio optimization through reinforcement learning,

Y . Nakayama and T. Sawaki, “Causal inference on investment constraints and non-stationarity in dynamic portfolio optimization through reinforcement learning,” 2023. [Online]. Available: https: //arxiv.org/abs/2311.04946

2023 arXiv
[14]

A deep reinforcement learning framework for dynamic portfolio optimization: Evidence from china’s stock market,

G. Huang, X. Zhou, and Q. Song, “A deep reinforcement learning framework for dynamic portfolio optimization: Evidence from china’s stock market,” 2025. [Online]. Available: https://arxiv.org/abs/2412. 18563

2025
[15]

A systematic approach to portfolio optimization: A comparative study of reinforcement learning agents, market signals, and investment horizons,

F. Espiga-Fern ´andez, A. Garc ´ıa-S´anchez, and J. Ordieres-Mer ´e, “A systematic approach to portfolio optimization: A comparative study of reinforcement learning agents, market signals, and investment horizons,” Algorithms, vol. 17, no. 12, 2024. [Online]. Available: https:...

2024
[16]

Deep reinforcement learning for portfolio selection,

Y . Jiang, J. Olmo, and M. Atwi, “Deep reinforcement learning for portfolio selection,” Global Finance Journal , vol. 62, p. 101016, 2024. [Online]. Available: https://www.sciencedirect.com/science/article/pii/ S1044028324000887

2024
[17]

Vlassis, M

N. Vlassis, M. Ghavamzadeh, S. Mannor, and P. Poupart, Bayesian Reinforcement Learning . Berlin, Heidelberg: Springer Berlin Heidelberg, 2012, pp. 359–386. [Online]. Available: https://doi.org/10.1007/978-3-642-27645-3 11

2012 doi
[18]

Bayesian reinforcement learning: A basic overview,

P. Kang, P. N. Tobler, and P. Dayan, “Bayesian reinforcement learning: A basic overview,” Neurobiology of Learning and Memory , vol. 211, p. 107924, 2024. [Online]. Available: https://www.sciencedirect.com/ science/article/pii/S1074742724000352

2024
[19]

Generalized bayesian deep reinforcement learning,

S. S. Roy, R. G. Everitt, C. P. Robert, and R. Dutta, “Generalized bayesian deep reinforcement learning,” 2025. [Online]. Available: https://arxiv.org/abs/2412.11743

2025 arXiv
[21]

Genetic multi-armed bandits: a reinforcement learning approach for discrete optimization via simulation,

D. Preil and M. Krapp, “Genetic multi-armed bandits: a reinforcement learning approach for discrete optimization via simulation,” 2023. [Online]. Available: https://arxiv.org/abs/2302.07695

2023 arXiv
[22]

Using genetic algorithms to find technical trading rules,

F. Allen and R. Karjalainen, “Using genetic algorithms to find technical trading rules,” Journal of Financial Economics , vol. 51, no. 2, pp. 245–271, 1999. [Online]. Available: https://www.sciencedirect.com/ science/article/pii/S0304405X9800052X

1999
[23]

Quantum-inspired genetic algorithm,

A. Narayanan and M. Moore, “Quantum-inspired genetic algorithm,” in Proceedings of the IEEE International Conference on Evolutionary Computation (ICEC), 1996, pp. 61–66

1996
[24]

Hybrid optimization search-based ensem- ble model for portfolio optimization and return prediction in business investment,

M. Naik and A. Albuquerque, “Hybrid optimization search-based ensem- ble model for portfolio optimization and return prediction in business investment,” Progress in Artificial Intelligence , vol. 11, 08 2022

2022
[25]

Openai gym,

G. Brockman, V . Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, “Openai gym,” 2016. [Online]. Available: https://arxiv.org/abs/1606.01540

2016 arXiv
[26]

Addressing function approximation error in actor-critic methods,

S. Fujimoto, H. van Hoof, and D. Meger, “Addressing function approximation error in actor-critic methods,” 2018. [Online]. Available: https://arxiv.org/abs/1802.09477

2018 arXiv
[27]

Understanding the difficulty of training deep feedforward neural networks,

X. Glorot and Y . Bengio, “Understanding the difficulty of training deep feedforward neural networks,” in Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics , ser. Proceedings of Machine Learning Research, Y . W. Teh and M. Titteri...

2010
[28]

Pennylane: Automatic differentiation of hybrid quantum-classical computations,

V . B. et al, “Pennylane: Automatic differentiation of hybrid quantum-classical computations,” 2022. [Online]. Available: https: //arxiv.org/abs/1811.04968

2022 arXiv
[29]

Pytorch: An imperative style, high-performance deep learning library,

A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. K ¨opf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “Pytorch: An imperative style, high-...

2019 arXiv
[30]

Stable-baselines3: Reliable reinforcement learning implementations,

A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dormann, “Stable-baselines3: Reliable reinforcement learning implementations,” Journal of Machine Learning Research , vol. 22, no. 268, pp. 1–8, 2021. [Online]. Available: http://jmlr.org/papers/v22/ 20-1364.html

2021
[31]

Notes on kullback-leibler divergence and likelihood,

J. Shlens, “Notes on kullback-leibler divergence and likelihood,” CoRR, vol. abs/1404.2000, 2014. [Online]. Available: http://arxiv.org/abs/1404. 2000

2000 arXiv
[2023]

Available: https://www.mdpi.com/1999-4893/16/1/23

[Online]. Available: https://www.mdpi.com/1999-4893/16/1/23

1999
[2024]

Available: https://arxiv.org/abs/2408.07712

[Online]. Available: https://arxiv.org/abs/2408.07712

arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [20]

Convex optimization: Algorithms and complexity,

M. Ghavamzadeh, S. Mannor, J. Pineau, and A. Tamar, “Convex optimization: Algorithms and complexity,” Foundations and Trends® in Machine Learning , vol. 8, no. 5–6, p. 359–483, 2015. [Online]. Available: http://dx.doi.org/10.1561/2200000049

doi:10.1561/2200000049 2015

[2] [1]

A novel application of statistical process control charts in financial market surveillance with the idea of profile monitoring,

A. Yeganeh and S. C. Shongwe, “A novel application of statistical process control charts in financial market surveillance with the idea of profile monitoring,” PLOS ONE , vol. 18, no. 7, pp. 1–26, 07 2023. [Online]. Available: https://doi.org/10.1371/journal.pone.0288627

work page doi:10.1371/journal.pone.0288627 2023

[3] [2]

The role of the business model in capturing value from innovation: evidence from xerox corporation’s technology spin-off companies,

H. Chesbrough and R. S. Rosenbloom, “The role of the business model in capturing value from innovation: evidence from xerox corporation’s technology spin-off companies,” Industrial and Corporate Change, vol. 11, no. 3, pp. 529–555, 06 2002. [Online]. Available: https://doi.org/10.1093/icc/11.3.529

work page doi:10.1093/icc/11.3.529 2002

[4] [3]

A budget-adaptive allocation rule for optimal computing budget allocation,

Z. Cao, H. Wang, E. P. Chew, H. Li, and K. C. Tan, “A budget-adaptive allocation rule for optimal computing budget allocation,” European Journal of Operational Research , vol. 325, no. 2, pp. 247–260, 2025. [Online]. Available: https://www.sciencedirect.com/science/article/pii/ S0377221725002656

work page 2025

[5] [4]

A dual-model solution for business reasoning with large language mod- els,

S. Trivedi, S. Roy, R. Das, D. Mukherjee, S. Mahapatra, and P. Nandy, “A dual-model solution for business reasoning with large language mod- els,” in 2024 IEEE Pune Section International Conference (PuneCon) , 2024, pp. 1–6

work page 2024

[6] [5]

Introduction to reinforcement learning,

M. Ghasemi and D. Ebrahimi, “Introduction to reinforcement learning,”

work page

[7] [6]

Actor-critic reinforcement learning for control with stability guarantee,

M. Han, L. Zhang, J. Wang, and W. Pan, “Actor-critic reinforcement learning for control with stability guarantee,” 2020. [Online]. Available: https://arxiv.org/abs/2004.14288

arXiv 2020

[8] [7]

Reinforcement learning: An introduction,

R. Sutton and A. Barto, “Reinforcement learning: An introduction,” IEEE Transactions on Neural Networks , vol. 9, no. 5, pp. 1054–1054, 1998

1998