Vtyjkyuk

I just read about projected gradient descent but I did not see the intuition to use Projected one instead of normal gradient descent. Would you tell me the reason and preferable situations of projected gradient descent? What does that projection contribute?

At a basic level, projected gradient descent is just a more general method for solving a more general problem.

Gradient descent minimizes a function by moving in the negative gradient direction at each step. There is no constraint on the variable.
$$
textProblem 1: min_x f(x)
$$
$$
x_k+1 = x_k - t_k nabla f(x_k)
$$

On the other hand, projected gradient descent minimizes a function subject to a constraint. At each step we move in the direction of the negative gradient, and then "project" onto the feasible set.

$$
textProblem 2: min_x f(x) text subject to x in C
$$

$$
y_k+1 = x_k - t_k nabla f(x_k)\
x_k+1 = textarg min_x in C |y_k+1-x|
$$

The previous answer is perfect. I would like to add one thing, imagine you have to optimize a convex function but with a non convex constraint, that time gradient descent will help but with every iteration we will make sure that my solution does not go out of the constraint domain so in every iteration I will project my solution into the constraint set until the convergence reached.In that way this method is extremely useful.