$f(x) = frac12||F(x)||^2$, consider $x^k+1 = x^k-lambda_k(J_f(x^k))^-1F(x_k)$, prove using Armijo: $f(x^k+1)/f(x^k)le 1-lambda_k$

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
0
down vote

favorite













Let $f(x) = frac12||F(x)||^2$, where
$F:mathbbR^ntomathbbR^n, Fin C^1$. Consider the iterative
method defined by $x^k+1 = x^k-lambda_k(J_f(x^k))^-1F(x_k)$.
Suppose $Jf(x)$ is non singular for all $x$. Prove that in the Armijo
condition using $alpha=0.5$ we have $$f(x^k+1)/f(x^k)le
1-lambda_k$$




So, the Armijo condition is the following:



$$f(x+lambda d) le f(x) + alphalambda d^tnabla f(x)$$



for $alpha=0.5$:



$$f(x+lambda d) le f(x) + 0.5lambda d^tnabla f(x)$$



$$f(x) =frac12||F(x)||^2implies f(x_1,cdots,x_n) = (frac12F_1^2(x_1,cdots,x_n) + cdots + frac12F_n^2(x_1,cdots,x_n))impliesfracpartial fpartial x_i = sum_j2F_jfracpartial F_jpartial x_i implies nabla f = J_f beginbmatrix
F_1 \
cdots \
F_n
endbmatrix = J_f F(x)$$



If we look at $x^k+1 = x^k-lambda_k(J_f(x^k))^-1F(x_k)$ we see that the direction is $d = -(J_f(x^k))^-1F(x_k)$.



Since $x^k+1+ lambda_k(J_f(x^k))^-1F(x_k) = x^k$, we have in armijo:



$$f(x^k+1+ lambda_k(J_f(x^k))^-1F(x_k)) = f(x^k) le f(x^k+1)- 0.5lambda_k[(J_f(x^k))^-1F(x_k)]^tnabla f(x_k)$$



I don't know what to do from here. What should the inverse of the jacobian do?



UPDATE:



$$f(x^k+1+ lambda_k(J_f(x^k))^-1F(x_k)) = f(x^k) le f(x^k+1)- 0.5lambda_k[(J_f(x^k))^-1F(x_k)]^tnabla f(x_k) = \ f(x^k+1)- 0.5lambda_kF(x_k)^t(J_f(x^k))^-1nabla f(x_k) = f(x^k+1)- 0.5lambda_kF(x_k)^t(J_f(x^k))^-1J_f(x_k)F(x_k) = \ f(x^k+1)- 0.5lambda_k||F(x_k)||^2$$



so



$$ f(x^k) le f(x^k+1)- 0.5lambda_k||F(x_k)||^2 = f(x^k+1)- 0.5lambda_k2f(x_k) = f(x^k+1)-lambda f(x_k)implies \f(x_k) + lambda f(x_k)le f(x_k+1)implies 1-lambda le fracf(x_k+1)f(x_k)$$



which is almost what I want. Can you spot the error?







share|cite|improve this question


























    up vote
    0
    down vote

    favorite













    Let $f(x) = frac12||F(x)||^2$, where
    $F:mathbbR^ntomathbbR^n, Fin C^1$. Consider the iterative
    method defined by $x^k+1 = x^k-lambda_k(J_f(x^k))^-1F(x_k)$.
    Suppose $Jf(x)$ is non singular for all $x$. Prove that in the Armijo
    condition using $alpha=0.5$ we have $$f(x^k+1)/f(x^k)le
    1-lambda_k$$




    So, the Armijo condition is the following:



    $$f(x+lambda d) le f(x) + alphalambda d^tnabla f(x)$$



    for $alpha=0.5$:



    $$f(x+lambda d) le f(x) + 0.5lambda d^tnabla f(x)$$



    $$f(x) =frac12||F(x)||^2implies f(x_1,cdots,x_n) = (frac12F_1^2(x_1,cdots,x_n) + cdots + frac12F_n^2(x_1,cdots,x_n))impliesfracpartial fpartial x_i = sum_j2F_jfracpartial F_jpartial x_i implies nabla f = J_f beginbmatrix
    F_1 \
    cdots \
    F_n
    endbmatrix = J_f F(x)$$



    If we look at $x^k+1 = x^k-lambda_k(J_f(x^k))^-1F(x_k)$ we see that the direction is $d = -(J_f(x^k))^-1F(x_k)$.



    Since $x^k+1+ lambda_k(J_f(x^k))^-1F(x_k) = x^k$, we have in armijo:



    $$f(x^k+1+ lambda_k(J_f(x^k))^-1F(x_k)) = f(x^k) le f(x^k+1)- 0.5lambda_k[(J_f(x^k))^-1F(x_k)]^tnabla f(x_k)$$



    I don't know what to do from here. What should the inverse of the jacobian do?



    UPDATE:



    $$f(x^k+1+ lambda_k(J_f(x^k))^-1F(x_k)) = f(x^k) le f(x^k+1)- 0.5lambda_k[(J_f(x^k))^-1F(x_k)]^tnabla f(x_k) = \ f(x^k+1)- 0.5lambda_kF(x_k)^t(J_f(x^k))^-1nabla f(x_k) = f(x^k+1)- 0.5lambda_kF(x_k)^t(J_f(x^k))^-1J_f(x_k)F(x_k) = \ f(x^k+1)- 0.5lambda_k||F(x_k)||^2$$



    so



    $$ f(x^k) le f(x^k+1)- 0.5lambda_k||F(x_k)||^2 = f(x^k+1)- 0.5lambda_k2f(x_k) = f(x^k+1)-lambda f(x_k)implies \f(x_k) + lambda f(x_k)le f(x_k+1)implies 1-lambda le fracf(x_k+1)f(x_k)$$



    which is almost what I want. Can you spot the error?







    share|cite|improve this question
























      up vote
      0
      down vote

      favorite









      up vote
      0
      down vote

      favorite












      Let $f(x) = frac12||F(x)||^2$, where
      $F:mathbbR^ntomathbbR^n, Fin C^1$. Consider the iterative
      method defined by $x^k+1 = x^k-lambda_k(J_f(x^k))^-1F(x_k)$.
      Suppose $Jf(x)$ is non singular for all $x$. Prove that in the Armijo
      condition using $alpha=0.5$ we have $$f(x^k+1)/f(x^k)le
      1-lambda_k$$




      So, the Armijo condition is the following:



      $$f(x+lambda d) le f(x) + alphalambda d^tnabla f(x)$$



      for $alpha=0.5$:



      $$f(x+lambda d) le f(x) + 0.5lambda d^tnabla f(x)$$



      $$f(x) =frac12||F(x)||^2implies f(x_1,cdots,x_n) = (frac12F_1^2(x_1,cdots,x_n) + cdots + frac12F_n^2(x_1,cdots,x_n))impliesfracpartial fpartial x_i = sum_j2F_jfracpartial F_jpartial x_i implies nabla f = J_f beginbmatrix
      F_1 \
      cdots \
      F_n
      endbmatrix = J_f F(x)$$



      If we look at $x^k+1 = x^k-lambda_k(J_f(x^k))^-1F(x_k)$ we see that the direction is $d = -(J_f(x^k))^-1F(x_k)$.



      Since $x^k+1+ lambda_k(J_f(x^k))^-1F(x_k) = x^k$, we have in armijo:



      $$f(x^k+1+ lambda_k(J_f(x^k))^-1F(x_k)) = f(x^k) le f(x^k+1)- 0.5lambda_k[(J_f(x^k))^-1F(x_k)]^tnabla f(x_k)$$



      I don't know what to do from here. What should the inverse of the jacobian do?



      UPDATE:



      $$f(x^k+1+ lambda_k(J_f(x^k))^-1F(x_k)) = f(x^k) le f(x^k+1)- 0.5lambda_k[(J_f(x^k))^-1F(x_k)]^tnabla f(x_k) = \ f(x^k+1)- 0.5lambda_kF(x_k)^t(J_f(x^k))^-1nabla f(x_k) = f(x^k+1)- 0.5lambda_kF(x_k)^t(J_f(x^k))^-1J_f(x_k)F(x_k) = \ f(x^k+1)- 0.5lambda_k||F(x_k)||^2$$



      so



      $$ f(x^k) le f(x^k+1)- 0.5lambda_k||F(x_k)||^2 = f(x^k+1)- 0.5lambda_k2f(x_k) = f(x^k+1)-lambda f(x_k)implies \f(x_k) + lambda f(x_k)le f(x_k+1)implies 1-lambda le fracf(x_k+1)f(x_k)$$



      which is almost what I want. Can you spot the error?







      share|cite|improve this question















      Let $f(x) = frac12||F(x)||^2$, where
      $F:mathbbR^ntomathbbR^n, Fin C^1$. Consider the iterative
      method defined by $x^k+1 = x^k-lambda_k(J_f(x^k))^-1F(x_k)$.
      Suppose $Jf(x)$ is non singular for all $x$. Prove that in the Armijo
      condition using $alpha=0.5$ we have $$f(x^k+1)/f(x^k)le
      1-lambda_k$$




      So, the Armijo condition is the following:



      $$f(x+lambda d) le f(x) + alphalambda d^tnabla f(x)$$



      for $alpha=0.5$:



      $$f(x+lambda d) le f(x) + 0.5lambda d^tnabla f(x)$$



      $$f(x) =frac12||F(x)||^2implies f(x_1,cdots,x_n) = (frac12F_1^2(x_1,cdots,x_n) + cdots + frac12F_n^2(x_1,cdots,x_n))impliesfracpartial fpartial x_i = sum_j2F_jfracpartial F_jpartial x_i implies nabla f = J_f beginbmatrix
      F_1 \
      cdots \
      F_n
      endbmatrix = J_f F(x)$$



      If we look at $x^k+1 = x^k-lambda_k(J_f(x^k))^-1F(x_k)$ we see that the direction is $d = -(J_f(x^k))^-1F(x_k)$.



      Since $x^k+1+ lambda_k(J_f(x^k))^-1F(x_k) = x^k$, we have in armijo:



      $$f(x^k+1+ lambda_k(J_f(x^k))^-1F(x_k)) = f(x^k) le f(x^k+1)- 0.5lambda_k[(J_f(x^k))^-1F(x_k)]^tnabla f(x_k)$$



      I don't know what to do from here. What should the inverse of the jacobian do?



      UPDATE:



      $$f(x^k+1+ lambda_k(J_f(x^k))^-1F(x_k)) = f(x^k) le f(x^k+1)- 0.5lambda_k[(J_f(x^k))^-1F(x_k)]^tnabla f(x_k) = \ f(x^k+1)- 0.5lambda_kF(x_k)^t(J_f(x^k))^-1nabla f(x_k) = f(x^k+1)- 0.5lambda_kF(x_k)^t(J_f(x^k))^-1J_f(x_k)F(x_k) = \ f(x^k+1)- 0.5lambda_k||F(x_k)||^2$$



      so



      $$ f(x^k) le f(x^k+1)- 0.5lambda_k||F(x_k)||^2 = f(x^k+1)- 0.5lambda_k2f(x_k) = f(x^k+1)-lambda f(x_k)implies \f(x_k) + lambda f(x_k)le f(x_k+1)implies 1-lambda le fracf(x_k+1)f(x_k)$$



      which is almost what I want. Can you spot the error?









      share|cite|improve this question













      share|cite|improve this question




      share|cite|improve this question








      edited Aug 26 at 23:52

























      asked Aug 26 at 19:49









      Guerlando OCs

      30321244




      30321244

























          active

          oldest

          votes











          Your Answer




          StackExchange.ifUsing("editor", function ()
          return StackExchange.using("mathjaxEditing", function ()
          StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
          StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
          );
          );
          , "mathjax-editing");

          StackExchange.ready(function()
          var channelOptions =
          tags: "".split(" "),
          id: "69"
          ;
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function()
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled)
          StackExchange.using("snippets", function()
          createEditor();
          );

          else
          createEditor();

          );

          function createEditor()
          StackExchange.prepareEditor(
          heartbeatType: 'answer',
          convertImagesToLinks: true,
          noModals: false,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          noCode: true, onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          );



          );













           

          draft saved


          draft discarded


















          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2895443%2ffx-frac12fx2-consider-xk1-xk-lambda-kj-fxk-1%23new-answer', 'question_page');

          );

          Post as a guest



































          active

          oldest

          votes













          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes















           

          draft saved


          draft discarded















































           


          draft saved


          draft discarded














          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2895443%2ffx-frac12fx2-consider-xk1-xk-lambda-kj-fxk-1%23new-answer', 'question_page');

          );

          Post as a guest













































































          這個網誌中的熱門文章

          tkz-euclide: tkzDrawCircle[R] not working

          How to combine Bézier curves to a surface?

          1st Magritte Awards