Taking total expectation and setting ak :=E[V ∞ ε (ek)] gives the scalar recursion ak+1 ≤β 2 ε ak +α 2C2 ε Wmax + (4(1 +γ)B Q)2

Hence, it follows from Equation (56) that E[V ∞ ε (ek+1)| F M k ]≤β 2 ε V ∞ ε (ek) +α 2Cε(Cε −1) Wmax + (4(1 +γ)B Q)2 +α 2Cε Wmax + (4(1 +γ)B Q)2 =β 2 ε V ∞ ε (ek) +α 2C2 ε Wmax +