Using machine learning to inform harvest control rule design in complex fishery settings

Carl Boettiger; Carl J. Walters; Christopher L. Cahill; Felipe Montealegre-Mora

REVIEW 3 major objections 6 minor 6 references

Using machine learning to inform harvest control rule design in complex fishery settings

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Mean fish weight helps harvest rules only when the goal is trophy-sized catch, not yield or stable harvests.

desk verdict Useful RL-for-fisheries case study, but the headline negative result about mean-weight observations is only as strong as the PPO convergence diagnostics, which are currently thin. read the letter →

arxiv 2412.12400 v2 pith:2YF7SCXH submitted 2024-12-16 q-bio.PE cs.LGq-bio.QM

Felipe Montealegre-Mora , Carl Boettiger , Carl J. Walters , Christopher L. Cahill This is my paper · ORCID

classification q-bio.PEcs.LGq-bio.QM

keywords harvestcontrolrulesreinforcementlearningspasmodicrecruitmentwalleyefisherypartialobservabilitymeanweightobservationtrophyfishingutilityage-structuredpopulationmodel

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether extra information—mean fish weight on top of stock biomass—improves harvest control rules in a simulated age-structured walleye fishery with spasmodic recruitment and observation error. The authors find that the value of that extra observation depends entirely on the management objective. For yield maximization and for a risk-averse utility that penalizes variable catches, a rule using only biomass performs as well as any rule using mean weight. For a trophy-fishing objective that values only large fish, the mean-weight observation lets a reinforcement-learned rule wait for strong cohorts to reach trophy size and then harvest in pulses, gaining about 30% more utility than constant-exploitation or precautionary rules. If correct, this means monitoring priorities and policy flexibility should be matched to what managers actually value.

What carries the argument

The central object is a partially observed Markov decision process (POMDP): a 20-age-class Beverton-Holt walleye model whose true state is the biomass per age class, but an agent sees only noisy observations of survey-vulnerable biomass $B^{survey}$ and mean fish weight $\bar{W}^{survey}$, both with multiplicative Gaussian error. Recruitment is spasmodic by construction, with a 2.5% probability per year of a pulse 10–30 times average. The mechanism that carries the argument is the neural-network harvest control rule trained by proximal policy optimization (PPO), which can express nonlinear, multi-observation feedback rules without a pre-specified functional form. In the trophy scenario the network learns a bang-bang pulse policy: harvest rate near zero while the dominant cohort is small, then a spike of $U \approx 0.75$ when mean weight and biomass are high. Mean weight is the informative signal because a recruitment pulse produces a characteristic dip in mean weight followed by a rise as the cohort ages into survey vulnerability and trophy size.

What would settle it

Re-run the HARA scenario with the same 2RL architecture but with an order of magnitude more training steps and a moving-average mean-weight observation; if that policy then exceeds the constant-U policy's utility, the claim that mean weight has no value for risk-averse management would be overturned.

Watch

Extended reading notes

Core claim

In a 20-age-class stochastic model of an Alberta walleye fishery calibrated to measured spasmodic recruitment (a 2.5% yearly chance of a pulse 10–30 times average recruitment), the paper compares five harvest control rules under three utilities: total yield, HARA risk-averse utility ($U^{\gamma}$ with $\gamma=0.6$), and trophy utility that counts only fish older than age 10. Policies were a constant exploitation rate $U_{MSY}$, a conventional precautionary rule (cPP), an optimized precautionary rule (oPP), a neural-network rule using only survey biomass (1RL), and a neural-network rule using biomass plus mean weight (2RL). The central finding is that the 2RL rule's extra observation does not help under yield or HARA utility—all optimized policies perform nearly identically, with only the conventional precautionary rule clearly worse under HARA—but it delivers a clear gain under trophy utility (mean 126.90 vs 96.44 for $U_{MSY}$ and 92.73 for 1RL). The learned trophy policy is a pulse-harvesting strategy: it avoids fishing while a large cohort is young and mean weight is low, then applies a short high-exploitation pulse around years 8–15 after the recruitment event, when that cohort has reached trophy size. The paper interprets this as evidence that age-structure information becomes decision-relevant precisely when the utility function is size-dependent, and that flexible policy search is most valuable in that regime.

Load-bearing premise

The load-bearing premise is that the two-observation reinforcement-learned policies were trained long enough and stably enough that their performance reflects the true information value of the mean-weight observation, rather than an artifact of unfinished optimization.

Editorial extensions

If this is right

Managers targeting yield or stable catches can keep single-observation, biomass-based harvest control rules without sacrificing performance, even in spasmodically recruiting age-structured fisheries.
In size-dependent fisheries, a flexible policy that waits for strong cohorts and harvests in pulses can beat constant-exploitation and precautionary rules by roughly 30% in the simulated setting.
Monitoring programs that already measure mean fish weight should treat it as optional for yield and risk-averse objectives, but as potentially valuable for trophy or size-selective objectives.
Reinforcement-learned rules can produce non-intuitive feedback shapes, such as exploitation decreasing with biomass at high biomass, that would be hard to specify a priori in management strategy evaluation.
The near-tie among all optimized policies for yield and HARA utility suggests that, in this model, many different policy shapes are equivalent, giving managers freedom to pursue other objectives without sacrificing much utility.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension: the value of the mean-weight observation should increase as the trophy size threshold rises, so re-running the comparison with thresholds between ages 5 and 15 would map where the advantage appears.
The paper's own suggestion that longer training would push the 2RL HARA policy toward $U_{MSY}$ implies that the negative result could be an optimization artifact; a much longer training run is the direct experiment to settle that.
A regulator-friendly approximation to the learned trophy policy would be a hand-coded pulse rule: skip or sharply cut fishing for a fixed window after a detected large recruitment event, then reopen at high exploitation; if such a rule captures most of the 2RL gain, complex neural policies may not be needed in practice.
If pulsed trophy harvesting were adopted across many lakes at once, regional synchrony in recruitment pulses could make aggregate catch more variable over time than the single-lake model suggests, a risk the paper does not model.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a genuinely useful applied RL paper for fisheries, but the claim that mean weight only helps for trophy utility is only as good as the PPO training diagnostics, and those are thin.

What's new: most prior RL-for-fisheries work uses simple models or single objectives. Here they set up a partially observed age-structured walleye model with spasmodic recruitment, compare three utility functions (yield, HARA risk-averse, trophy), and systematically ask whether adding a mean-weight observation improves the policy. That comparison is new. The methods are solid: the population model is reasonable, the observation error is explicit, and they optimize standard reference-point policies with Bayesian optimization and RL policies with PPO. Code is on GitHub. The trophy result is the real contribution: 2RL gets about 30% more trophy utility than everything else, and the pulse-fishing explanation is convincing and connects to classic theory (Walters 1969). That part I believe.

Soft spots: the negative result for yield and HARA is fragile. Table 2 shows 2RL yield (249.47) is not better than 1RL (250.25), and 2RL HARA (391.27) is actually worse than 1RL (402.43) and oPP (401.73). The authors themselves note the 2RL HARA policy is noisy and that a moving average, larger networks, or longer training might make it converge to UMSY. That is exactly the failure mode that would explain why mean weight appears useless: the 2RL policy may simply be under-optimized. There are no learning curves, no multiple seeds, no best-of-seed selection, and only one PPO run per policy. So the claim 'mean weight is not useful for yield/HARA' is not yet established. At best it's a provisional negative result that needs convergence diagnostics and equivalence testing (or at least paired comparisons with proper uncertainty). The abstract also overstates the gains from optimization; Table 2 shows large gains only in the trophy column, not for yield or HARA.

Also, the cPP is derived from the same model's U* and B*, so it's not an independent benchmark. That's minor and standard in MSE, but worth noting.

Who it's for: people working on RL for natural resource management, and fisheries scientists interested in whether additional observations are worth collecting. The paper is a good case study and a cautionary example of how RL can discover non-intuitive policies (pulse fishing) but also how easy it is to over-interpret negative results without proper training diagnostics.

Recommendation: deserves peer review, but should be revised to either add PPO training diagnostics (multiple seeds, learning curves) or soften the negative claims. If the authors can show the 2RL policies are converged, the paper becomes much stronger. If not, the honest conclusion is 'we didn't find evidence that mean weight helps for yield/HARA in this model,' not 'mean weight is not useful.'

Referee Report

3 major / 6 minor

Summary. This paper uses a simulated age-structured walleye fishery with spasmodic recruitment and partial observability to compare harvest control rules: a constant exploitation rate (U*), a precautionary policy calibrated from the model's optimized U* and B*, an optimized precautionary rule, and neural-network policies trained with PPO using either biomass alone or biomass plus mean weight. Policies are evaluated under yield-maximizing, HARA risk-averse, and trophy-fishing utilities. The central empirical claims are that an additional mean-weight observation improves harvest control rule performance only for the size-dependent trophy utility, where the two-observation RL policy attains roughly 30% more utility than the other policies (Table 2: 2RL 126.90 vs UMSY 96.44), and that mean weight is not useful for yield or HARA utility.

Significance. If the claims hold, the paper makes a useful contribution by showing, in a realistic POMDP-style fishery simulator, that the value of an additional state observation depends strongly on the management objective. It also reproduces a non-intuitive pulse-harvest strategy for trophy objectives, with a clear mechanism shown in Figures 8 and 9. Strengths include a well-specified age-structured simulation model, an MSE-style evaluation protocol, and companion open-source code. The positive trophy result is large and internally consistent. However, the negative result on mean weight for yield and HARA is currently fragile because it rests on the assumption that the PPO-trained 2RL policies are near-optimal; the manuscript itself provides evidence that the HARA 2RL policy is not converged.

major comments (3)

[Section 2.6 and Section 3, Table 2] The central negative result, that the mean-weight observation is not useful for yield or HARA utility, is not established because the two-observation PPO policies may be under-optimized. Only one PPO run per policy configuration is reported in Section 2.6, with no learning curves or seed variation, and Section 3 concedes that the HARA 2RL policy is noisy and would plausibly converge to UMSY with longer training or smoothing. In Table 2, HARA 2RL (391.27 ± 21.51) is worse than 1RL (402.43 ± 21.14) and oPP (401.73 ± 20.93), and yield 2RL (249.47 ± 23.86) is no better than 1RL (250.25 ± 22.82); these patterns are exactly what one would expect from an under-trained larger network rather than from the uninformative value of mean weight. Please provide convergence diagnostics such as learning curves, multiple seeds, or best-of-seed selection, or a formal near-optimality test, before concluding that mean weight is not useful for these objectives.
[Section 3, Table 2] The claim that 'nearly all policies obtain essentially equal amounts of utility' rests on overlapping standard deviations, which is not a statistical equivalence test. The phrase 'within a standard deviation' in the table caption is not sufficient evidence that yield and HARA performances are equal or near-optimal. Report paired differences with confidence intervals, using the shared stochastic sequence design already applied in Figures 5-9, or use an equivalence test with a pre-specified margin.
[Abstract and Section 1] The statement that 'considerable gains can be achieved' by optimizing harvest control rule parameters overstates the results in Table 2: the large gain is confined to the trophy-utility 2RL policy (126.90 vs 88.51-96.44), while for yield and HARA the optimized policies are close to or worse than UMSY (e.g., yield 2RL 249.47 vs UMSY 237.28; HARA 2RL 391.27 vs UMSY 400.58). Please qualify the abstract and introductory framing to distinguish the strong trophy result from the null yield/HARA result.

minor comments (6)

[Section 1] Typo: 'systes' should be 'systems'.
[Section 5] Typo: 'velieve' should be 'believe'.
[Equation (23)] The productivity variable π is defined but never used subsequently; either use it or remove the definition.
[Equation (22)] The notation 'argmaxF' is unclear; the subscript or function name should be defined explicitly.
[Section 2.5] The 'conventional precautionary policy' (cPP) is not an independent DFO default: its parameters are derived from this paper's optimized U* and B* via Eq. (17). The text should state this explicitly wherever cPP is described as the recommended government of Canada rule.
[References] The reference list has inconsistent formatting, including missing parentheses around years and incomplete journal details, and several in-text citations such as 'Sutton, Barto, and others 1998' should follow the journal's style.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the optimization-and-evaluation loop is a closed simulation (MSE-style), not a fitted input renamed as a prediction; self-citations are background model sources, not load-bearing.

full rationale

The paper's central comparison—1RL vs 2RL across yield, HARA, and trophy utilities—is generated by optimizing policies against the simulator and then evaluating those same policies on the simulator. Because both quantities are simulation outputs rather than empirical quantities being predicted, this is standard management strategy evaluation and does not reduce to a fitted parameter renamed as a result. The closest candidate for a circular construction is the conventional precautionary policy (cPP): its parameters are fixed from the optimized constant-U policy via X1=0.4B*, X2=0.8B*, Y2=U*, with B*=yield(U*)/U* (Sec. 2.4, Eqs. 16-17). But the paper does not present cPP as an independent external benchmark or as a test of mean-weight information; it is a standard fixed-form comparator, and the central mean-weight conclusions are drawn from 1RL/2RL and oPP comparisons, not from cPP. The use of Cahill et al. (2022) for model structure and parameter values is a citation to prior peer-reviewed empirical modeling work by overlapping authors, but it supplies the simulated environment rather than the result, so it is not a self-citation chain forcing the conclusion. The concern that PPO may be under-trained for 2RL (larger network, 6 million steps, one run) is a robustness/convergence risk affecting the strength of the negative yield/HARA findings, not a circularity: an under-optimized policy is not a policy whose outcome is equal to its input by definition. No equation in the paper defines an input observation as the output utility, and no fitted quantity is relabeled as a prediction.

Assumptions & free parameters 9 free parameters · 8 assumptions · 0 invented entities

The central results are achieved entirely through simulation. The findings therefore rest on the realism of the population dynamics model (recruitment, growth, vulnerability, mortality), on the chosen observation noise and utility functions, and on the assumption that the numerical optimizers (Bayesian optimization and PPO) found near-optimal policies. Most model parameters are inherited from Cahill et al. 2022 or set by hand; none are estimated from new data in this paper.

free parameters (9)

U* constant exploitation rate = 0.132 (yield), 0.129 (HARA), 0.084 (trophy)
Optimized by Bayesian optimization to maximize average utility; used as a baseline policy and to set cPP parameters.
oPP parameters (X1, X2, Y2) = Yield: 0.422, 2.132, 0.633; HARA: 0.301, 0.734, 0.179; Trophy: 0.0, 5.751, 0.323
Optimized rectilinear precautionary rule parameters for each utility scenario (Table 1).
Neural network weights (1RL and 2RL) = Not reported; thousands of parameters
Optimized via PPO over 6 million time-steps; these weights define the RL harvest control rules.
p_big = 0.025
Chosen by hand probability of a large recruitment pulse in eq. (5); controls the frequency of spasmodic recruitment.
sigma recruitment (lognormal) = 0.4
Chosen by hand variability of normal recruitment deviations in eq. (5).
uniform recruitment pulse range = [10, 30]
Chosen by hand magnitude of large year classes in eq. (5).
observation error sigma (eB, eW) = 0.1
Multiplicative Gaussian noise on biomass and mean weight observations, eq. (11).
HARA risk aversion gamma = 0.6
Chosen by hand to set risk aversion in eq. (13).
Trophy fishing age threshold a_thr = 10
Chosen by hand; fish younger than this contribute no utility in eq. (14).

assumptions (8)

domain assumption Beverton-Holt stock-recruitment relationship
Section 2.1 eq. (1); recruitment is a deterministic function of SSB with multiplicative noise.
domain assumption Von Bertalanffy growth and survey vulnerability-at-age
Section 2.2 eq. (8); length-at-age and vulnerability schedules determine what the survey observes.
domain assumption Spasmodic recruitment mixture distribution
Eq. (5); large pulses occur with probability 2.5% and are 10-30 times average recruitment.
domain assumption Multiplicative Gaussian observation error
Eq. (11); policies observe noisy biomass and mean weight, making the problem a POMDP.
domain assumption Utility functions as defined (yield, HARA, trophy)
Section 2.3; all results are conditional on these three objectives.
ad hoc to paper Optimizers (Bayesian optimization, PPO) converge near optimality
Sections 2.5-2.6; there are no convergence guarantees for PPO on this POMDP, and weak convergence could affect the mean-weight conclusion.
domain assumption Simulation horizon and episode counts are sufficient
Section 2.1 and Appendix B; 1000-year episodes and 250 evaluation episodes are used to estimate average utility.
domain assumption Model parameters from Cahill et al. 2022 are representative of Alberta walleye
Section 2; most biological parameters are inherited from prior work without new estimation.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Using machine learning to inform harvest control rule design in complex fishery settings." pith.science (2026). https://pith.science/paper/2YF7SCXH

@misc{pith2026241212400,
  author       = {Pith},
  title        = {Pith review of: Using machine learning to inform harvest control rule design in complex fishery settings},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2YF7SCXH}},
  note         = {Machine review of arXiv:2412.12400}
}

read the original abstract

In fishery science, harvest management of size-structured stochastic populations is a long-standing and difficult problem. Rectilinear precautionary policies based on biomass and harvesting reference points have now become a standard approach to this problem. While these standard feedback policies are adapted from analytical or dynamic programming solutions assuming relatively simple ecological dynamics, they are often applied to more complicated ecological settings in the real world. In this paper we explore the problem of designing harvest control rules for partially observed, age-structured, spasmodic fish populations using tools from reinforcement learning (RL) and Bayesian optimization. Our focus is on the case of Walleye fisheries in Alberta, Canada, whose highly variable recruitment dynamics have perplexed managers and ecologists. We optimized and evaluated policies using several complementary performance metrics. The main questions we addressed were: 1. How do standard policies based on reference points perform relative to numerically optimized policies? 2. Can an observation of mean fish weight, in addition to stock biomass, aid policy decisions?

Figures

Figures reproduced from arXiv: 2412.12400 by the authors.

**Figure 2.** Weight, survey vulnerability, and harvest vulnerability at-age. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Reward distributions for each of the five optimized policies in each of the three scenarios. These distributions [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

Figures from the paper (6 more)

**Figure 4.** Figure 4: Optimized policy functions for each of the scenarios. Top row: policies which only use the vulnerable biomass [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]

**Figure 5.** Figure 5: First 400 years for an episode simulated with each of the HCRs optimized for [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

**Figure 6.** Figure 6: First 400 years for an episode simulated with each of the HCRs optimized for [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

**Figure 7.** Figure 7: First 400 years for an episode simulated with each of the HCRs optimized for [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

**Figure 8.** Figure 8: A zoom into part of the episode for the trophy utility scenario. We display large recruitment years as vertical [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]

**Figure 9.** Figure 9: Policy interactions with our dynamical model in the timesteps following a large year class. For each optimized [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

6 extracted references · 4 canonical work pages

[1]

Magnuson-Stevens Fishery Conservation and Management Act

ACT, AN (1996). “Magnuson-Stevens Fishery Conservation and Management Act”. In:Public Law 94, p

work page 1996
[3]

Optimal fishery policy: An equilibrium solution with irreversible investment

Athena Scientific. Botsford, Louis W and Thomas C Wainwright (1985). “Optimal fishery policy: An equilibrium solution with irreversible investment”. In: Journal of mathematical biology 21, pp. 317–327. Caddy, JF and JA Gulland (1983). “Historical patterns of fish stocks”. In:Marine policy 7.4, pp. 267–278. 17 Cahill, Christopher L et al. (2022). “Unveilin...

work page 1985
[6]

Analysis of the eastern Pacific yellowfin tuna fishery based on multiple management objectives

MIT press Cambridge. Vaca-Rodríguez, Juan Guillermo and Roberto Ramón Enríquez-Andrade (2006). “Analysis of the eastern Pacific yellowfin tuna fishery based on multiple management objectives”. In: Ecological Modelling 191.2, pp. 275–290. Walters, Carl and Steven JD Martell (2002). “Stock assessment needs for sustainable fisheries management”. In:Bulletin ...

work page 2006
[7]

A fishery decisionmaking framework incorporating the precautionary approach

DFO, A. (2009). “A fishery decisionmaking framework incorporating the precautionary approach”. In. Ditria, Ellen M et al. (2022). “Artificial intelligence and automated monitoring for assisting conservation of marine ecosystems: A perspective”. In: Frontiers in Marine Science 9, p. 918104. Edwards, Charles TT and Dorothy J Dankel (2016). Management scienc...

arXiv 2009
[95]

Uncertain measurements of renewable resources: approximations, harvesting policies and value of accuracy

Moxnes, Erling (2003). “Uncertain measurements of renewable resources: approximations, harvesting policies and value of accuracy”. In: Journal of environmental economics and management 45.1, pp. 85–108. Murphy, Brian R, David W Willis, and others (1996).Fisheries techniques. Citeseer. Myers, Ransom A (1998). “When do environment–recruitment correlations w...

arXiv 2003
[265]

Reinforcement learning and optimal control

Bertsekas, Dimitri (2019). Reinforcement learning and optimal control. V ol

work page 2019

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Magnuson-Stevens Fishery Conservation and Management Act

ACT, AN (1996). “Magnuson-Stevens Fishery Conservation and Management Act”. In:Public Law 94, p

work page 1996

[2] [3]

Optimal fishery policy: An equilibrium solution with irreversible investment

Athena Scientific. Botsford, Louis W and Thomas C Wainwright (1985). “Optimal fishery policy: An equilibrium solution with irreversible investment”. In: Journal of mathematical biology 21, pp. 317–327. Caddy, JF and JA Gulland (1983). “Historical patterns of fish stocks”. In:Marine policy 7.4, pp. 267–278. 17 Cahill, Christopher L et al. (2022). “Unveilin...

work page 1985

[3] [6]

Analysis of the eastern Pacific yellowfin tuna fishery based on multiple management objectives

MIT press Cambridge. Vaca-Rodríguez, Juan Guillermo and Roberto Ramón Enríquez-Andrade (2006). “Analysis of the eastern Pacific yellowfin tuna fishery based on multiple management objectives”. In: Ecological Modelling 191.2, pp. 275–290. Walters, Carl and Steven JD Martell (2002). “Stock assessment needs for sustainable fisheries management”. In:Bulletin ...

work page 2006

[4] [7]

A fishery decisionmaking framework incorporating the precautionary approach

DFO, A. (2009). “A fishery decisionmaking framework incorporating the precautionary approach”. In. Ditria, Ellen M et al. (2022). “Artificial intelligence and automated monitoring for assisting conservation of marine ecosystems: A perspective”. In: Frontiers in Marine Science 9, p. 918104. Edwards, Charles TT and Dorothy J Dankel (2016). Management scienc...

arXiv 2009

[5] [95]

Uncertain measurements of renewable resources: approximations, harvesting policies and value of accuracy

Moxnes, Erling (2003). “Uncertain measurements of renewable resources: approximations, harvesting policies and value of accuracy”. In: Journal of environmental economics and management 45.1, pp. 85–108. Murphy, Brian R, David W Willis, and others (1996).Fisheries techniques. Citeseer. Myers, Ransom A (1998). “When do environment–recruitment correlations w...

arXiv 2003

[6] [265]

Reinforcement learning and optimal control

Bertsekas, Dimitri (2019). Reinforcement learning and optimal control. V ol

work page 2019