Optimization formula — definition, units and applicable conditions

found 4formulas · currently displayed 4Articles Explanation · Unit · Applicable conditions

Optimization

gradient descent update

x_{k+1}=x_k-\eta_k\nabla f(x_k)

updates parameters along the local steepest descent direction of the objective function.

Optimization

x_{k+1}=x_k-\frac{f(x_k)}{f'(x_k)}

uses local tangent zeros to approximate the roots of nonlinear equations.

Optimization

\int_a^b f(x)dx\approx h\left[\frac{f(a)+f(b)}2+\sum_{i=1}^{n-1}f(a+ih)\right]

Divide the integration interval into small trapezoids to approximate the area.

Optimization

\nabla f(x)+\lambda\nabla g(x)=0,\quad g(x)=0

finds the stationary point of the objective function on the equality constrained surface.