MathComputational mathOptimization

Pitfalls of gradient descent

Theory

Good steps: Armijo rule

Report a typo

Choose the step sizes that guarantee that the Armijo rule holds true for the function

f(x,y,z)=x2y−xz+y3z.f(x,y,z)=x^2y - xz + y^3z.Consider xk=[2,−1,−4]\bold{x_k} = [2,-1,-4] and c1=0.1c_1 = 0.1.

Tip: For the Armijo rule to hold true, the following must be true: f(xk−γk⋅∇f(xk))≤f(xk)−c1⋅γk⋅D∇f(xk)f(xk)f(\bold{x}_k - \gamma_k \cdot \nabla f(\bold{x}_k)) \leq f(\bold{x}_k) - c_1 \cdot \gamma_k \cdot D_{\nabla f(\bold{x}_k)}f(\bold{x}_k)

Select one or more options from the list
___

Create a free account to access the full topic