$f(x) = frac12||F(x)||^2$, consider $x^k+1 = x^k-lambda_k(J_f(x^k))^-1F(x_k)$, prove using Armijo: $f(x^k+1)/f(x^k)le 1-lambda_k$

Clash Royale CLAN TAG#URR8PPP
up vote
0
down vote
favorite
Let $f(x) = frac12||F(x)||^2$, where
$F:mathbbR^ntomathbbR^n, Fin C^1$. Consider the iterative
method defined by $x^k+1 = x^k-lambda_k(J_f(x^k))^-1F(x_k)$.
Suppose $Jf(x)$ is non singular for all $x$. Prove that in the Armijo
condition using $alpha=0.5$ we have $$f(x^k+1)/f(x^k)le
1-lambda_k$$
So, the Armijo condition is the following:
$$f(x+lambda d) le f(x) + alphalambda d^tnabla f(x)$$
for $alpha=0.5$:
$$f(x+lambda d) le f(x) + 0.5lambda d^tnabla f(x)$$
$$f(x) =frac12||F(x)||^2implies f(x_1,cdots,x_n) = (frac12F_1^2(x_1,cdots,x_n) + cdots + frac12F_n^2(x_1,cdots,x_n))impliesfracpartial fpartial x_i = sum_j2F_jfracpartial F_jpartial x_i implies nabla f = J_f beginbmatrix
F_1 \
cdots \
F_n
endbmatrix = J_f F(x)$$
If we look at $x^k+1 = x^k-lambda_k(J_f(x^k))^-1F(x_k)$ we see that the direction is $d = -(J_f(x^k))^-1F(x_k)$.
Since $x^k+1+ lambda_k(J_f(x^k))^-1F(x_k) = x^k$, we have in armijo:
$$f(x^k+1+ lambda_k(J_f(x^k))^-1F(x_k)) = f(x^k) le f(x^k+1)- 0.5lambda_k[(J_f(x^k))^-1F(x_k)]^tnabla f(x_k)$$
I don't know what to do from here. What should the inverse of the jacobian do?
UPDATE:
$$f(x^k+1+ lambda_k(J_f(x^k))^-1F(x_k)) = f(x^k) le f(x^k+1)- 0.5lambda_k[(J_f(x^k))^-1F(x_k)]^tnabla f(x_k) = \ f(x^k+1)- 0.5lambda_kF(x_k)^t(J_f(x^k))^-1nabla f(x_k) = f(x^k+1)- 0.5lambda_kF(x_k)^t(J_f(x^k))^-1J_f(x_k)F(x_k) = \ f(x^k+1)- 0.5lambda_k||F(x_k)||^2$$
so
$$ f(x^k) le f(x^k+1)- 0.5lambda_k||F(x_k)||^2 = f(x^k+1)- 0.5lambda_k2f(x_k) = f(x^k+1)-lambda f(x_k)implies \f(x_k) + lambda f(x_k)le f(x_k+1)implies 1-lambda le fracf(x_k+1)f(x_k)$$
which is almost what I want. Can you spot the error?
multivariable-calculus derivatives optimization
add a comment |Â
up vote
0
down vote
favorite
Let $f(x) = frac12||F(x)||^2$, where
$F:mathbbR^ntomathbbR^n, Fin C^1$. Consider the iterative
method defined by $x^k+1 = x^k-lambda_k(J_f(x^k))^-1F(x_k)$.
Suppose $Jf(x)$ is non singular for all $x$. Prove that in the Armijo
condition using $alpha=0.5$ we have $$f(x^k+1)/f(x^k)le
1-lambda_k$$
So, the Armijo condition is the following:
$$f(x+lambda d) le f(x) + alphalambda d^tnabla f(x)$$
for $alpha=0.5$:
$$f(x+lambda d) le f(x) + 0.5lambda d^tnabla f(x)$$
$$f(x) =frac12||F(x)||^2implies f(x_1,cdots,x_n) = (frac12F_1^2(x_1,cdots,x_n) + cdots + frac12F_n^2(x_1,cdots,x_n))impliesfracpartial fpartial x_i = sum_j2F_jfracpartial F_jpartial x_i implies nabla f = J_f beginbmatrix
F_1 \
cdots \
F_n
endbmatrix = J_f F(x)$$
If we look at $x^k+1 = x^k-lambda_k(J_f(x^k))^-1F(x_k)$ we see that the direction is $d = -(J_f(x^k))^-1F(x_k)$.
Since $x^k+1+ lambda_k(J_f(x^k))^-1F(x_k) = x^k$, we have in armijo:
$$f(x^k+1+ lambda_k(J_f(x^k))^-1F(x_k)) = f(x^k) le f(x^k+1)- 0.5lambda_k[(J_f(x^k))^-1F(x_k)]^tnabla f(x_k)$$
I don't know what to do from here. What should the inverse of the jacobian do?
UPDATE:
$$f(x^k+1+ lambda_k(J_f(x^k))^-1F(x_k)) = f(x^k) le f(x^k+1)- 0.5lambda_k[(J_f(x^k))^-1F(x_k)]^tnabla f(x_k) = \ f(x^k+1)- 0.5lambda_kF(x_k)^t(J_f(x^k))^-1nabla f(x_k) = f(x^k+1)- 0.5lambda_kF(x_k)^t(J_f(x^k))^-1J_f(x_k)F(x_k) = \ f(x^k+1)- 0.5lambda_k||F(x_k)||^2$$
so
$$ f(x^k) le f(x^k+1)- 0.5lambda_k||F(x_k)||^2 = f(x^k+1)- 0.5lambda_k2f(x_k) = f(x^k+1)-lambda f(x_k)implies \f(x_k) + lambda f(x_k)le f(x_k+1)implies 1-lambda le fracf(x_k+1)f(x_k)$$
which is almost what I want. Can you spot the error?
multivariable-calculus derivatives optimization
add a comment |Â
up vote
0
down vote
favorite
up vote
0
down vote
favorite
Let $f(x) = frac12||F(x)||^2$, where
$F:mathbbR^ntomathbbR^n, Fin C^1$. Consider the iterative
method defined by $x^k+1 = x^k-lambda_k(J_f(x^k))^-1F(x_k)$.
Suppose $Jf(x)$ is non singular for all $x$. Prove that in the Armijo
condition using $alpha=0.5$ we have $$f(x^k+1)/f(x^k)le
1-lambda_k$$
So, the Armijo condition is the following:
$$f(x+lambda d) le f(x) + alphalambda d^tnabla f(x)$$
for $alpha=0.5$:
$$f(x+lambda d) le f(x) + 0.5lambda d^tnabla f(x)$$
$$f(x) =frac12||F(x)||^2implies f(x_1,cdots,x_n) = (frac12F_1^2(x_1,cdots,x_n) + cdots + frac12F_n^2(x_1,cdots,x_n))impliesfracpartial fpartial x_i = sum_j2F_jfracpartial F_jpartial x_i implies nabla f = J_f beginbmatrix
F_1 \
cdots \
F_n
endbmatrix = J_f F(x)$$
If we look at $x^k+1 = x^k-lambda_k(J_f(x^k))^-1F(x_k)$ we see that the direction is $d = -(J_f(x^k))^-1F(x_k)$.
Since $x^k+1+ lambda_k(J_f(x^k))^-1F(x_k) = x^k$, we have in armijo:
$$f(x^k+1+ lambda_k(J_f(x^k))^-1F(x_k)) = f(x^k) le f(x^k+1)- 0.5lambda_k[(J_f(x^k))^-1F(x_k)]^tnabla f(x_k)$$
I don't know what to do from here. What should the inverse of the jacobian do?
UPDATE:
$$f(x^k+1+ lambda_k(J_f(x^k))^-1F(x_k)) = f(x^k) le f(x^k+1)- 0.5lambda_k[(J_f(x^k))^-1F(x_k)]^tnabla f(x_k) = \ f(x^k+1)- 0.5lambda_kF(x_k)^t(J_f(x^k))^-1nabla f(x_k) = f(x^k+1)- 0.5lambda_kF(x_k)^t(J_f(x^k))^-1J_f(x_k)F(x_k) = \ f(x^k+1)- 0.5lambda_k||F(x_k)||^2$$
so
$$ f(x^k) le f(x^k+1)- 0.5lambda_k||F(x_k)||^2 = f(x^k+1)- 0.5lambda_k2f(x_k) = f(x^k+1)-lambda f(x_k)implies \f(x_k) + lambda f(x_k)le f(x_k+1)implies 1-lambda le fracf(x_k+1)f(x_k)$$
which is almost what I want. Can you spot the error?
multivariable-calculus derivatives optimization
Let $f(x) = frac12||F(x)||^2$, where
$F:mathbbR^ntomathbbR^n, Fin C^1$. Consider the iterative
method defined by $x^k+1 = x^k-lambda_k(J_f(x^k))^-1F(x_k)$.
Suppose $Jf(x)$ is non singular for all $x$. Prove that in the Armijo
condition using $alpha=0.5$ we have $$f(x^k+1)/f(x^k)le
1-lambda_k$$
So, the Armijo condition is the following:
$$f(x+lambda d) le f(x) + alphalambda d^tnabla f(x)$$
for $alpha=0.5$:
$$f(x+lambda d) le f(x) + 0.5lambda d^tnabla f(x)$$
$$f(x) =frac12||F(x)||^2implies f(x_1,cdots,x_n) = (frac12F_1^2(x_1,cdots,x_n) + cdots + frac12F_n^2(x_1,cdots,x_n))impliesfracpartial fpartial x_i = sum_j2F_jfracpartial F_jpartial x_i implies nabla f = J_f beginbmatrix
F_1 \
cdots \
F_n
endbmatrix = J_f F(x)$$
If we look at $x^k+1 = x^k-lambda_k(J_f(x^k))^-1F(x_k)$ we see that the direction is $d = -(J_f(x^k))^-1F(x_k)$.
Since $x^k+1+ lambda_k(J_f(x^k))^-1F(x_k) = x^k$, we have in armijo:
$$f(x^k+1+ lambda_k(J_f(x^k))^-1F(x_k)) = f(x^k) le f(x^k+1)- 0.5lambda_k[(J_f(x^k))^-1F(x_k)]^tnabla f(x_k)$$
I don't know what to do from here. What should the inverse of the jacobian do?
UPDATE:
$$f(x^k+1+ lambda_k(J_f(x^k))^-1F(x_k)) = f(x^k) le f(x^k+1)- 0.5lambda_k[(J_f(x^k))^-1F(x_k)]^tnabla f(x_k) = \ f(x^k+1)- 0.5lambda_kF(x_k)^t(J_f(x^k))^-1nabla f(x_k) = f(x^k+1)- 0.5lambda_kF(x_k)^t(J_f(x^k))^-1J_f(x_k)F(x_k) = \ f(x^k+1)- 0.5lambda_k||F(x_k)||^2$$
so
$$ f(x^k) le f(x^k+1)- 0.5lambda_k||F(x_k)||^2 = f(x^k+1)- 0.5lambda_k2f(x_k) = f(x^k+1)-lambda f(x_k)implies \f(x_k) + lambda f(x_k)le f(x_k+1)implies 1-lambda le fracf(x_k+1)f(x_k)$$
which is almost what I want. Can you spot the error?
multivariable-calculus derivatives optimization
edited Aug 26 at 23:52
asked Aug 26 at 19:49
Guerlando OCs
30321244
30321244
add a comment |Â
add a comment |Â
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2895443%2ffx-frac12fx2-consider-xk1-xk-lambda-kj-fxk-1%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password