Replacement Policy of Systems with Dependent Components via Integration of Dynamic Programming and Simulated Annealing

Abdolhamid Eshragh-Jahromi; Hamed Badakhsh; Mohammad Pirhooshyaran

arxiv: 1907.08689 · v2 · pith:VU26B6WQnew · submitted 2019-07-19 · 📡 eess.SY · cs.SY· math.DS

Replacement Policy of Systems with Dependent Components via Integration of Dynamic Programming and Simulated Annealing

Hamed Badakhsh , Mohammad Pirhooshyaran , Abdolhamid Eshragh-Jahromi This is my paper

Pith reviewed 2026-05-24 18:52 UTC · model grok-4.3

classification 📡 eess.SY cs.SYmath.DS

keywords replacement policydependent componentsdynamic programmingsimulated annealingdeterioration limitsmaintenance optimizationmulti-component systems

0 comments

The pith

Dynamic programming combined with simulated annealing produces superior replacement policies for dependent components using only past data.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a replacement policy for multi-component systems where deterioration in one part accelerates others. Dynamic programming determines the limit for the second part as a function of the first part's rate, while simulated annealing estimates those rates from replacement times recorded under the old fixed-limit policy. This integrated method is shown in two examples to outperform the standard special limit replacement approach. Readers interested in maintenance of interacting systems would see value in policies that adapt limits without requiring new sensors or direct rate measurements.

Core claim

For every deterioration rate of part 1, dynamic programming yields a deterioration limit for part 2 after which either part 2 alone or both parts are replaced. Deterioration rates are recovered by simulated annealing applied to historical replacement times. The resulting policy demonstrates significant superiority over the special limit replacement method in the two presented examples.

What carries the argument

Dynamic programming for conditional deterioration limits on part 2 given part 1's rate, integrated with simulated annealing to estimate deterioration rates from past replacement times.

If this is right

The new policy can decide to replace one part or both depending on the rates and limits.
Only replacement time data from the prior policy is needed as input.
The method improves upon fixed deterioration limits by accounting for dependence between parts.
Performance gains are demonstrated through two numerical examples.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If the deterioration process is Markovian as assumed, the policy could be updated online as new replacements occur.
Extending the dynamic program to three or more parts would require addressing the curse of dimensionality in the state space.
Similar estimation of hidden rates from event times might apply to other maintenance problems with limited observation.

Load-bearing premise

Simulated annealing can extract the underlying deterioration rates accurately enough from the replacement times produced by the previous policy to enable an improved strategy.

What would settle it

A simulation or field test where the proposed policy leads to higher long-run average cost or more unplanned failures than the special limit method would disprove the claimed superiority.

read the original abstract

In a dependent multi-component system, increasing the deterioration of a part leads to the increased deterioration rate of other parts as well. In these systems, a deterioration limit is usually pre-determined for each part and the considered part is replaced while reaching this limit. In this paper, replacement conditions of these parts were examined according to the replacement times in the past. Using dynamic programming, for every deterioration rate of part 1, there is a deterioration limit for part 2, after which either part 2 or both parts should be replaced. The only available system data are the replacement time of the parts in the past according to the replacement policy at the time of reaching deterioration limit. Therefore, simulated annealing optimization method was used for estimating deterioration rates. Finally, two examples were presented for comparing the proposed method with the special limit replacement method, which showed the significance superiority of the former.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

This applies DP to set coupled replacement thresholds and SA to back out rates from old-policy data, but the examples leave the recovery step untested.

read the letter

The core move is to treat the two parts as coupled through deterioration rates, use dynamic programming to pick the best limit on part 2 once part 1's rate is known, and run simulated annealing on historical replacement times to recover those rates. That framing is straightforward and matches the engineering setting they describe. The two numerical examples are presented as evidence that the resulting policy beats the old fixed-limit rule. Credit for laying out the dependence explicitly and turning the policy choice into a solvable DP problem. The soft spot is the data loop. Replacement times come from the prior policy, SA fits the rates to those times, and the new policy is then scored inside the fitted model. The abstract gives no recovery error, no identifiability check, and no sensitivity of the final cost to rate error, so it is not clear whether the reported gains survive realistic estimation noise. Without those checks the superiority claim rests on an assumption rather than a demonstrated recovery. This is the kind of paper that belongs in a reliability or maintenance-scheduling venue. Readers who already work with DP or SA on replacement problems could extract the modeling trick and try it on their own data. It is worth sending to referees so the examples can be examined in full and the estimation step can be stress-tested; the work is coherent enough on its own terms to justify that step.

Referee Report

2 major / 2 minor

Summary. The paper proposes an integrated dynamic programming and simulated annealing approach for replacement policies in dependent multi-component systems. Dynamic programming determines deterioration limits for part 2 conditional on part 1's rate; simulated annealing estimates the underlying deterioration rates from historical replacement times generated under a prior fixed-limit policy. The resulting policy is claimed to be significantly superior to the special limit replacement method on two numerical examples.

Significance. If the simulated annealing recovery of deterioration rates is accurate and the superiority is robust, the method offers a practical way to improve maintenance decisions using only historical replacement data in interdependent systems. The integration of DP for policy optimization with SA for parameter estimation from limited data is a reasonable technical contribution, though its value hinges on validation of the rate-recovery step.

major comments (2)

[Abstract and numerical examples] Abstract and numerical examples section: the central claim of significant superiority rests on simulated annealing recovering the true deterioration rates from replacement times generated under the old fixed-limit policy, yet no recovery error, identifiability analysis, or sensitivity of the final policy to rate-estimation error is reported. This is load-bearing because the DP-derived limits are computed from the estimated rates.
[Abstract] Abstract: the evaluation of the new policy uses data derived from the fitted rates, creating a circularity risk; the manuscript provides no separate validation (e.g., forward simulation under known rates or hold-out replacement times) to confirm that the estimated rates produce an improved policy rather than an artifact of the fitting process.

minor comments (2)

[Abstract] Abstract contains the phrasing 'significance superiority' which should be corrected to 'significant superiority'.
[Abstract] The abstract states that two examples 'showed the significance superiority' but supplies no quantitative metrics, cost values, or description of the comparison protocol; these details belong in the main text even if space-constrained in the abstract.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments highlighting the need for explicit validation of the simulated annealing rate recovery and safeguards against circular evaluation. We address each major comment below and commit to revisions that strengthen the numerical validation without altering the core contribution.

read point-by-point responses

Referee: [Abstract and numerical examples] Abstract and numerical examples section: the central claim of significant superiority rests on simulated annealing recovering the true deterioration rates from replacement times generated under the old fixed-limit policy, yet no recovery error, identifiability analysis, or sensitivity of the final policy to rate-estimation error is reported. This is load-bearing because the DP-derived limits are computed from the estimated rates.

Authors: We agree that the absence of reported recovery error, identifiability checks, and sensitivity analysis leaves the central claim under-supported. The manuscript presents only the final policy performance under the estimated rates. In revision we will add (i) the achieved estimation errors for the two examples, (ii) a brief discussion of identifiability conditions under the replacement-time observation model, and (iii) a sensitivity study that perturbs the estimated rates and recomputes both the DP policy and the resulting cost improvement. These additions directly address the load-bearing role of the rate estimates. revision: partial
Referee: [Abstract] Abstract: the evaluation of the new policy uses data derived from the fitted rates, creating a circularity risk; the manuscript provides no separate validation (e.g., forward simulation under known rates or hold-out replacement times) to confirm that the estimated rates produce an improved policy rather than an artifact of the fitting process.

Authors: The evaluation is performed under the model whose parameters were recovered from the same historical replacement times that were generated by the old policy; this is the realistic setting in which only replacement data are available. Nevertheless, the referee correctly notes the lack of out-of-sample confirmation. In the revised manuscript we will include a separate forward-simulation experiment: data are generated from known ground-truth rates under the old policy, the SA-DP procedure is applied, and the new policy is then evaluated on fresh trajectories drawn from the same known rates. This provides an independent check that the recovered rates yield a genuinely superior policy. revision: yes

Circularity Check

1 steps flagged

Deterioration rates fitted via SA to old-policy replacement times; new DP policy superiority shown on the fitted model

specific steps

fitted input called prediction [Abstract]
"The only available system data are the replacement time of the parts in the past according to the replacement policy at the time of reaching deterioration limit. Therefore, simulated annealing optimization method was used for estimating deterioration rates. Finally, two examples were presented for comparing the proposed method with the special limit replacement method, which showed the significance superiority of the former."

Replacement times were produced by the old policy; SA recovers rates from those times; DP then produces a new policy whose superiority is asserted in examples that operate under the recovered rates. The superiority metric is therefore computed inside the fitted model rather than on fresh data generated independently of the fit.

full rationale

The paper's central result rests on estimating deterioration rates from replacement times generated exclusively under the prior fixed-limit policy, then deriving an improved policy via DP under those rates and demonstrating superiority in examples that use the same fitted rates. This matches the fitted-input-called-prediction pattern: the claimed performance gain is evaluated inside the model recovered from the baseline data rather than on independent hold-out or external benchmarks. No equations reduce by algebraic identity, but the evaluation loop is closed by construction on the fitted parameters.

Axiom & Free-Parameter Ledger

1 free parameters · 2 axioms · 0 invented entities

Based on abstract only; the central claim rests on the ability of simulated annealing to recover deterioration rates from policy-generated data and on the validity of the dynamic-programming optimality equations for the two-component Markov decision process.

free parameters (1)

deterioration rates of each part
Unknown rates are estimated by simulated annealing from past replacement times; these fitted values drive the dynamic-programming policy.

axioms (2)

domain assumption The system can be modeled as a Markov decision process with deterioration states and replacement actions.
Dynamic programming is applied directly to deterioration rates without stating or proving the Markov property or transition structure.
ad hoc to paper Simulated annealing recovers the true deterioration rates from replacement times generated under the prior policy.
The abstract treats this recovery as feasible without providing convergence guarantees or identifiability conditions.

pith-pipeline@v0.9.0 · 5701 in / 1411 out tokens · 20065 ms · 2026-05-24T18:52:58.010675+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

23 extracted references · 23 canonical work pages

[1]

Introduction The parts of a system may be dependent or independent. If they are independent, they can be separately planned for; however, in reality, parts are inter-dependent in a system and there are few systems in which the parts are independent from each other in all terms. In this study, two issues of deteriorating parts and dependency of parts were ...

work page
[2]

proceeding without replacement

Investigating effect of dependency on optimal policy of replacement 2.1 Modeling In this section, dynamic programming was used in order to , determining optimal policy , and investigating effects of dependency. Since programming horizon is infinite, two average reward and discount factor methods can be used; in this study, discount factor method was appli...

work page
[3]

optimization-simulation

Obtaining deterioration rates using previous failure data 7 Here our aim is to evaluate failure rate modes of system parts by employing replaceme nt data available from the past. A proposed model is presented in next subsection and then the selection criteria for appropriate algorithm is discussed. 3.1 Modeling As mentioned in the introduction , the only ...

work page
[4]

Example 1 Assume that data of failure time of the parts in a two-component system (such as two involved gears) are as in Table 1 and failure limit of each part is 0.9 mm

Numerical example In this section, two examples are given to examine the efficiency of the method. Example 1 Assume that data of failure time of the parts in a two-component system (such as two involved gears) are as in Table 1 and failure limit of each part is 0.9 mm. Other data are as follows: Replacement cost of part 1: 100 Replacement cost of part 2: ...

work page
[5]

Dynamic programming was used f or modeling and optimiz ing the problem and the optimal policy was obtained

Conclusion In this study, optimal replacement time of deteriorating parts which depends on each other and has a definite deterioration rate was examined and attempts were made to present a practical method for solving this issue. Dynamic programming was used f or modeling and optimiz ing the problem and the optimal policy was obtained. However, in reality...

work page
[6]

A survey of maintenance policies of deteriorating systems

Wang H. A survey of maintenance policies of deteriorating systems. European journal of operational research. 2002;139:469-89

work page 2002
[7]

A sequential condition ‐based repair/replacement policy with non‐periodic inspections for a system subject to continuous wear

Castanier B, Bérenguer C, Grall A. A sequential condition ‐based repair/replacement policy with non‐periodic inspections for a system subject to continuous wear. Applied stochastic models in business and industry. 2003;19:327-47

work page 2003
[8]

Optimal continuous -wear limit replacement under periodic inspections

Park KS. Optimal continuous -wear limit replacement under periodic inspections. Reliabil ity, IEEE Transactions on. 1988;37:97-102

work page 1988
[9]

A survey of the application of gamma processes in maintenance

Van Noortwijk J. A survey of the application of gamma processes in maintenance. Reliability Engineering & System Safety. 2009;94:2-21

work page 2009
[10]

A gamma wear process

Abdel-Hameed M. A gamma wear process. Reliability, IEEE Transactions on. 1975;24:152-3

work page 1975
[11]

Finite -time maintenance cost analysis of engineering systems affected by stochastic degradation

Pandey M, Cheng T, van der Weide J. Finite -time maintenance cost analysis of engineering systems affected by stochastic degradation. Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability. 2011;225:241-50

work page 2011
[12]

The probability distribution of maintenance cost of a system affected by the gamma process of degradation: Finite time solution

Cheng T, Pandey MD, van der Weide JA. The probability distribution of maintenance cost of a system affected by the gamma process of degradation: Finite time solution. Reliability Engineering & System Safety. 2012;108:65-76

work page 2012
[13]

An accurate analysis of maintenance cost of structures experiencing stochastic degradation

Cheng T, Pa ndey M. An accurate analysis of maintenance cost of structures experiencing stochastic degradation. Structure and Infrastructure Engineering. 2012;8:329-39

work page 2012
[14]

Reliability for systems of degrading components with distinct component shock sets

Song S, Coit DW, Feng Q. Reliability for systems of degrading components with distinct component shock sets. Reliability Engineering & System Safety. 2014;132:115-24

work page 2014
[15]

Maintenance grouping strategy for multi -component systems with dynamic contexts

Vu HC, Do P, Barros A, Bérenguer C. Maintenance grouping strategy for multi -component systems with dynamic contexts. Reliability Engineering & System Safety. 2014;132:233-49

work page 2014
[16]

A random walk model for a multi-component deteriorating system

Stadje W, Zuckerman D. A random walk model for a multi-component deteriorating system. Operations Research Letters. 2001;29:199-205

work page 2001
[17]

Optimal periodic replacement policy for a two-unit system with failure rate interaction

Lai M-T, Chen Y-C. Optimal periodic replacement policy for a two-unit system with failure rate interaction. The inter national journal of advanced manufacturing technology. 2006;29:367 - 71. 25

work page 2006
[18]

Optimal condition -based maintenance decisions for systems with dependent stochastic degradation of components

Hong H, Zhou W, Zhang S, Ye W. Optimal condition -based maintenance decisions for systems with dependent stochastic degradation of components. Reliability Engineering & System Safety. 2014;121:276-88

work page 2014
[19]

Integrating noncyclical preventive maintenance scheduling and production planning for multi -state systems

Fitouhi M-C, Nourelfath M. Integrating noncyclical preventive maintenance scheduling and production planning for multi -state systems. Reliability Engineering & System Safety. 2014;121:175-86

work page 2014
[20]

Metaheuristics: from design to implementation: John Wiley & Sons; 2009

Talbi E-G. Metaheuristics: from design to implementation: John Wiley & Sons; 2009

work page 2009
[21]

Fitness landscape analysis and memetic algorithms for the quadratic assignment problem

Merz P, Freisleben B. Fitness landscape analysis and memetic algorithms for the quadratic assignment problem. Evolutionary Computation, IEEE Transactions on. 2000;4:337-52

work page 2000
[22]

Correlated and uncorrelated fitness landscapes and how to tell the difference

Weinberger E. Correlated and uncorrelated fitness landscapes and how to tell the difference. Biological cybernetics. 1990;63:325-36

work page 1990
[23]

to proceed without replacement A(i,j)

Dao CD, Zuo MJ, Pandey M. Selective maintenance for multi -state series–parallel systems under economic dependence. Reliability Engineering & System Safety. 2014;121:240-9. 26 Appendix A: Proving ascending nature of cost relative to deterioration rate For the purpose of proving, induction was used. According to the definition of the model, it is clear tha...

work page 2014

[1] [1]

Introduction The parts of a system may be dependent or independent. If they are independent, they can be separately planned for; however, in reality, parts are inter-dependent in a system and there are few systems in which the parts are independent from each other in all terms. In this study, two issues of deteriorating parts and dependency of parts were ...

work page

[2] [2]

proceeding without replacement

Investigating effect of dependency on optimal policy of replacement 2.1 Modeling In this section, dynamic programming was used in order to , determining optimal policy , and investigating effects of dependency. Since programming horizon is infinite, two average reward and discount factor methods can be used; in this study, discount factor method was appli...

work page

[3] [3]

optimization-simulation

Obtaining deterioration rates using previous failure data 7 Here our aim is to evaluate failure rate modes of system parts by employing replaceme nt data available from the past. A proposed model is presented in next subsection and then the selection criteria for appropriate algorithm is discussed. 3.1 Modeling As mentioned in the introduction , the only ...

work page

[4] [4]

Example 1 Assume that data of failure time of the parts in a two-component system (such as two involved gears) are as in Table 1 and failure limit of each part is 0.9 mm

Numerical example In this section, two examples are given to examine the efficiency of the method. Example 1 Assume that data of failure time of the parts in a two-component system (such as two involved gears) are as in Table 1 and failure limit of each part is 0.9 mm. Other data are as follows: Replacement cost of part 1: 100 Replacement cost of part 2: ...

work page

[5] [5]

Dynamic programming was used f or modeling and optimiz ing the problem and the optimal policy was obtained

Conclusion In this study, optimal replacement time of deteriorating parts which depends on each other and has a definite deterioration rate was examined and attempts were made to present a practical method for solving this issue. Dynamic programming was used f or modeling and optimiz ing the problem and the optimal policy was obtained. However, in reality...

work page

[6] [6]

A survey of maintenance policies of deteriorating systems

Wang H. A survey of maintenance policies of deteriorating systems. European journal of operational research. 2002;139:469-89

work page 2002

[7] [7]

A sequential condition ‐based repair/replacement policy with non‐periodic inspections for a system subject to continuous wear

Castanier B, Bérenguer C, Grall A. A sequential condition ‐based repair/replacement policy with non‐periodic inspections for a system subject to continuous wear. Applied stochastic models in business and industry. 2003;19:327-47

work page 2003

[8] [8]

Optimal continuous -wear limit replacement under periodic inspections

Park KS. Optimal continuous -wear limit replacement under periodic inspections. Reliabil ity, IEEE Transactions on. 1988;37:97-102

work page 1988

[9] [9]

A survey of the application of gamma processes in maintenance

Van Noortwijk J. A survey of the application of gamma processes in maintenance. Reliability Engineering & System Safety. 2009;94:2-21

work page 2009

[10] [10]

A gamma wear process

Abdel-Hameed M. A gamma wear process. Reliability, IEEE Transactions on. 1975;24:152-3

work page 1975

[11] [11]

Finite -time maintenance cost analysis of engineering systems affected by stochastic degradation

Pandey M, Cheng T, van der Weide J. Finite -time maintenance cost analysis of engineering systems affected by stochastic degradation. Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability. 2011;225:241-50

work page 2011

[12] [12]

The probability distribution of maintenance cost of a system affected by the gamma process of degradation: Finite time solution

Cheng T, Pandey MD, van der Weide JA. The probability distribution of maintenance cost of a system affected by the gamma process of degradation: Finite time solution. Reliability Engineering & System Safety. 2012;108:65-76

work page 2012

[13] [13]

An accurate analysis of maintenance cost of structures experiencing stochastic degradation

Cheng T, Pa ndey M. An accurate analysis of maintenance cost of structures experiencing stochastic degradation. Structure and Infrastructure Engineering. 2012;8:329-39

work page 2012

[14] [14]

Reliability for systems of degrading components with distinct component shock sets

Song S, Coit DW, Feng Q. Reliability for systems of degrading components with distinct component shock sets. Reliability Engineering & System Safety. 2014;132:115-24

work page 2014

[15] [15]

Maintenance grouping strategy for multi -component systems with dynamic contexts

Vu HC, Do P, Barros A, Bérenguer C. Maintenance grouping strategy for multi -component systems with dynamic contexts. Reliability Engineering & System Safety. 2014;132:233-49

work page 2014

[16] [16]

A random walk model for a multi-component deteriorating system

Stadje W, Zuckerman D. A random walk model for a multi-component deteriorating system. Operations Research Letters. 2001;29:199-205

work page 2001

[17] [17]

Optimal periodic replacement policy for a two-unit system with failure rate interaction

Lai M-T, Chen Y-C. Optimal periodic replacement policy for a two-unit system with failure rate interaction. The inter national journal of advanced manufacturing technology. 2006;29:367 - 71. 25

work page 2006

[18] [18]

Optimal condition -based maintenance decisions for systems with dependent stochastic degradation of components

Hong H, Zhou W, Zhang S, Ye W. Optimal condition -based maintenance decisions for systems with dependent stochastic degradation of components. Reliability Engineering & System Safety. 2014;121:276-88

work page 2014

[19] [19]

Integrating noncyclical preventive maintenance scheduling and production planning for multi -state systems

Fitouhi M-C, Nourelfath M. Integrating noncyclical preventive maintenance scheduling and production planning for multi -state systems. Reliability Engineering & System Safety. 2014;121:175-86

work page 2014

[20] [20]

Metaheuristics: from design to implementation: John Wiley & Sons; 2009

Talbi E-G. Metaheuristics: from design to implementation: John Wiley & Sons; 2009

work page 2009

[21] [21]

Fitness landscape analysis and memetic algorithms for the quadratic assignment problem

Merz P, Freisleben B. Fitness landscape analysis and memetic algorithms for the quadratic assignment problem. Evolutionary Computation, IEEE Transactions on. 2000;4:337-52

work page 2000

[22] [22]

Correlated and uncorrelated fitness landscapes and how to tell the difference

Weinberger E. Correlated and uncorrelated fitness landscapes and how to tell the difference. Biological cybernetics. 1990;63:325-36

work page 1990

[23] [23]

to proceed without replacement A(i,j)

Dao CD, Zuo MJ, Pandey M. Selective maintenance for multi -state series–parallel systems under economic dependence. Reliability Engineering & System Safety. 2014;121:240-9. 26 Appendix A: Proving ascending nature of cost relative to deterioration rate For the purpose of proving, induction was used. According to the definition of the model, it is clear tha...

work page 2014