What is the difference between the projection onto the column space and projection onto row space?

Clash Royale CLAN TAG#URR8PPP
up vote
1
down vote
favorite
If I see a question that asks "find the projection a vector $b$ onto a matrix $A$" I would either solve by using $A^TAhat x =A^Tb$ and then the projection would equal $Ahat x$,
and if the matrix $A$ was orthogonal then I would use $proj_bA = fracb cdot q_1q_1 cdot q_1q_1 + ... + fracb cdot q_kq_k cdot q_kq_k$ where $q_k$ represents the $k^th$ vector in matrix $A$.
My question is what if a question says find the projection of some vector $b$ onto the column/row space of matrix $A$?
What does this mean and what would I need to do differently to calculate it?
linear-algebra
add a comment |Â
up vote
1
down vote
favorite
If I see a question that asks "find the projection a vector $b$ onto a matrix $A$" I would either solve by using $A^TAhat x =A^Tb$ and then the projection would equal $Ahat x$,
and if the matrix $A$ was orthogonal then I would use $proj_bA = fracb cdot q_1q_1 cdot q_1q_1 + ... + fracb cdot q_kq_k cdot q_kq_k$ where $q_k$ represents the $k^th$ vector in matrix $A$.
My question is what if a question says find the projection of some vector $b$ onto the column/row space of matrix $A$?
What does this mean and what would I need to do differently to calculate it?
linear-algebra
1
Projecting onto the row-space of $A$ is the same as projecting onto the column-space of $A^T$.
â Omnomnomnom
May 6 '16 at 20:28
1
Solving $A^TA hat x = A^Tb$ is "projective $b$ onto the column-space of the matrix $A$". Strictly speaking, it is non-sense to say that you're "projecting a vector onto a matrix".
â Omnomnomnom
May 6 '16 at 20:29
@Omnomnomnom so if I have to project onto the row space I would use $AA^That x=Ab$?
â idknuttin
May 6 '16 at 20:38
1
Yep.$$
â Omnomnomnom
May 6 '16 at 20:52
add a comment |Â
up vote
1
down vote
favorite
up vote
1
down vote
favorite
If I see a question that asks "find the projection a vector $b$ onto a matrix $A$" I would either solve by using $A^TAhat x =A^Tb$ and then the projection would equal $Ahat x$,
and if the matrix $A$ was orthogonal then I would use $proj_bA = fracb cdot q_1q_1 cdot q_1q_1 + ... + fracb cdot q_kq_k cdot q_kq_k$ where $q_k$ represents the $k^th$ vector in matrix $A$.
My question is what if a question says find the projection of some vector $b$ onto the column/row space of matrix $A$?
What does this mean and what would I need to do differently to calculate it?
linear-algebra
If I see a question that asks "find the projection a vector $b$ onto a matrix $A$" I would either solve by using $A^TAhat x =A^Tb$ and then the projection would equal $Ahat x$,
and if the matrix $A$ was orthogonal then I would use $proj_bA = fracb cdot q_1q_1 cdot q_1q_1 + ... + fracb cdot q_kq_k cdot q_kq_k$ where $q_k$ represents the $k^th$ vector in matrix $A$.
My question is what if a question says find the projection of some vector $b$ onto the column/row space of matrix $A$?
What does this mean and what would I need to do differently to calculate it?
linear-algebra
asked May 6 '16 at 19:52
idknuttin
1,182825
1,182825
1
Projecting onto the row-space of $A$ is the same as projecting onto the column-space of $A^T$.
â Omnomnomnom
May 6 '16 at 20:28
1
Solving $A^TA hat x = A^Tb$ is "projective $b$ onto the column-space of the matrix $A$". Strictly speaking, it is non-sense to say that you're "projecting a vector onto a matrix".
â Omnomnomnom
May 6 '16 at 20:29
@Omnomnomnom so if I have to project onto the row space I would use $AA^That x=Ab$?
â idknuttin
May 6 '16 at 20:38
1
Yep.$$
â Omnomnomnom
May 6 '16 at 20:52
add a comment |Â
1
Projecting onto the row-space of $A$ is the same as projecting onto the column-space of $A^T$.
â Omnomnomnom
May 6 '16 at 20:28
1
Solving $A^TA hat x = A^Tb$ is "projective $b$ onto the column-space of the matrix $A$". Strictly speaking, it is non-sense to say that you're "projecting a vector onto a matrix".
â Omnomnomnom
May 6 '16 at 20:29
@Omnomnomnom so if I have to project onto the row space I would use $AA^That x=Ab$?
â idknuttin
May 6 '16 at 20:38
1
Yep.$$
â Omnomnomnom
May 6 '16 at 20:52
1
1
Projecting onto the row-space of $A$ is the same as projecting onto the column-space of $A^T$.
â Omnomnomnom
May 6 '16 at 20:28
Projecting onto the row-space of $A$ is the same as projecting onto the column-space of $A^T$.
â Omnomnomnom
May 6 '16 at 20:28
1
1
Solving $A^TA hat x = A^Tb$ is "projective $b$ onto the column-space of the matrix $A$". Strictly speaking, it is non-sense to say that you're "projecting a vector onto a matrix".
â Omnomnomnom
May 6 '16 at 20:29
Solving $A^TA hat x = A^Tb$ is "projective $b$ onto the column-space of the matrix $A$". Strictly speaking, it is non-sense to say that you're "projecting a vector onto a matrix".
â Omnomnomnom
May 6 '16 at 20:29
@Omnomnomnom so if I have to project onto the row space I would use $AA^That x=Ab$?
â idknuttin
May 6 '16 at 20:38
@Omnomnomnom so if I have to project onto the row space I would use $AA^That x=Ab$?
â idknuttin
May 6 '16 at 20:38
1
1
Yep.$$
â Omnomnomnom
May 6 '16 at 20:52
Yep.$$
â Omnomnomnom
May 6 '16 at 20:52
add a comment |Â
2 Answers
2
active
oldest
votes
up vote
1
down vote
accepted
if the columns of matrix $A$ are linearly independent,
the projection of a vector, $b$, onto the column space of A can be computed
as
$$P=A(A^TA)^-1A^T$$
From here.
Wiki seems to say the same. It also says here that
The column space of $A$ is equal to the row space of $A^T$.
I'm guessing that
if the rows of matrix $A$ are linearly independent,
the projection of a vector, $b$, onto the row space of A can be computed
as
$$P=A^T(AA^T)^-1A$$
add a comment |Â
up vote
0
down vote
You didn't say whether the entries of your matrix are real or complex. The matrices $Aleft(A^TAright)^-1A^T$ and $A^Tleft(AA^Tright)^-1A$ work fine for a real matrix $A$ whose columns or rows are linearly independent. But if $A$ has any complex entries, then more generally the orthogonal projection onto the column and row spaces are$$P_c=Aleft(A^*Aright)^-1A^*$$ and $$P_r=A^*left(AA^*right)^-1Ambox,$$ respectively, where $A^*$ denotes the conjugate transpose of $A$, that is,$$A^*=overlineA^Tmbox.$$
As with real matrices, the formula for $P_c$ requires that the columns of $A$ be linearly independent, while the formula for $P_r$ requires that the rows of $A$ be linearly independent. Furthermore, in general one left-multiplies by $P_c$ but right-multiplies by $P_r$. That is, given vector $b$, we have
$$mboxproj_mboxcolAb=P_cb$$
but
$$mboxproj_mboxrowAb=bP_rmbox.$$
We write $b$ as a column vector when projecting onto the column space, but as a row vector when projecting onto the row space.
For example, take
$$A=left(beginarrayccc1 & i & 1+i\1 & i & 1-iendarrayright)mbox.$$
The rows of $A$ are linearly independent, so
$$P_r=A^*left(AA^*right)^-1A=left(beginarrayccc0.5 & 0.5i & 0\-0.5i & 0.5 & 0\0 & 0 & 1endarrayright)mbox.$$
Now take $b=left(beginarrayccc1 & i & 1+iendarrayright)$. Notice that $b$ is simply the first row of $A$. So we should have $mboxproj_mboxrowAb=b$. Right-multiplying $b$ by $P_r$ gives
$$left(beginarrayccc1 & i & 1+iendarrayright)left(beginarrayccc0.5 & 0.5i & 0\-0.5i & 0.5 & 0\0 & 0 & 1endarrayright)=left(beginarrayccc1 & i & 1+iendarrayright)$$
as expected. However, writing $b$ as a column vector and left-multiplying by $P_r$ gives an incorrect result:
$$left(beginarrayccc0.5 & 0.5i & 0\-0.5i & 0.5 & 0\0 & 0 & 1endarrayright)left(beginarrayc1\i\1+iendarrayright)=left(beginarrayc0\0\1+iendarrayright)mbox.$$
This example is meant to illustrate that one must be more careful with nonreal orthogonal projection matrices. In contrast, if $P_r$ is a real matrix, then both $bP_r$ and $P_rb$ give the correct result.
add a comment |Â
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
accepted
if the columns of matrix $A$ are linearly independent,
the projection of a vector, $b$, onto the column space of A can be computed
as
$$P=A(A^TA)^-1A^T$$
From here.
Wiki seems to say the same. It also says here that
The column space of $A$ is equal to the row space of $A^T$.
I'm guessing that
if the rows of matrix $A$ are linearly independent,
the projection of a vector, $b$, onto the row space of A can be computed
as
$$P=A^T(AA^T)^-1A$$
add a comment |Â
up vote
1
down vote
accepted
if the columns of matrix $A$ are linearly independent,
the projection of a vector, $b$, onto the column space of A can be computed
as
$$P=A(A^TA)^-1A^T$$
From here.
Wiki seems to say the same. It also says here that
The column space of $A$ is equal to the row space of $A^T$.
I'm guessing that
if the rows of matrix $A$ are linearly independent,
the projection of a vector, $b$, onto the row space of A can be computed
as
$$P=A^T(AA^T)^-1A$$
add a comment |Â
up vote
1
down vote
accepted
up vote
1
down vote
accepted
if the columns of matrix $A$ are linearly independent,
the projection of a vector, $b$, onto the column space of A can be computed
as
$$P=A(A^TA)^-1A^T$$
From here.
Wiki seems to say the same. It also says here that
The column space of $A$ is equal to the row space of $A^T$.
I'm guessing that
if the rows of matrix $A$ are linearly independent,
the projection of a vector, $b$, onto the row space of A can be computed
as
$$P=A^T(AA^T)^-1A$$
if the columns of matrix $A$ are linearly independent,
the projection of a vector, $b$, onto the column space of A can be computed
as
$$P=A(A^TA)^-1A^T$$
From here.
Wiki seems to say the same. It also says here that
The column space of $A$ is equal to the row space of $A^T$.
I'm guessing that
if the rows of matrix $A$ are linearly independent,
the projection of a vector, $b$, onto the row space of A can be computed
as
$$P=A^T(AA^T)^-1A$$
answered May 7 '16 at 9:10
BCLC
6,69622073
6,69622073
add a comment |Â
add a comment |Â
up vote
0
down vote
You didn't say whether the entries of your matrix are real or complex. The matrices $Aleft(A^TAright)^-1A^T$ and $A^Tleft(AA^Tright)^-1A$ work fine for a real matrix $A$ whose columns or rows are linearly independent. But if $A$ has any complex entries, then more generally the orthogonal projection onto the column and row spaces are$$P_c=Aleft(A^*Aright)^-1A^*$$ and $$P_r=A^*left(AA^*right)^-1Ambox,$$ respectively, where $A^*$ denotes the conjugate transpose of $A$, that is,$$A^*=overlineA^Tmbox.$$
As with real matrices, the formula for $P_c$ requires that the columns of $A$ be linearly independent, while the formula for $P_r$ requires that the rows of $A$ be linearly independent. Furthermore, in general one left-multiplies by $P_c$ but right-multiplies by $P_r$. That is, given vector $b$, we have
$$mboxproj_mboxcolAb=P_cb$$
but
$$mboxproj_mboxrowAb=bP_rmbox.$$
We write $b$ as a column vector when projecting onto the column space, but as a row vector when projecting onto the row space.
For example, take
$$A=left(beginarrayccc1 & i & 1+i\1 & i & 1-iendarrayright)mbox.$$
The rows of $A$ are linearly independent, so
$$P_r=A^*left(AA^*right)^-1A=left(beginarrayccc0.5 & 0.5i & 0\-0.5i & 0.5 & 0\0 & 0 & 1endarrayright)mbox.$$
Now take $b=left(beginarrayccc1 & i & 1+iendarrayright)$. Notice that $b$ is simply the first row of $A$. So we should have $mboxproj_mboxrowAb=b$. Right-multiplying $b$ by $P_r$ gives
$$left(beginarrayccc1 & i & 1+iendarrayright)left(beginarrayccc0.5 & 0.5i & 0\-0.5i & 0.5 & 0\0 & 0 & 1endarrayright)=left(beginarrayccc1 & i & 1+iendarrayright)$$
as expected. However, writing $b$ as a column vector and left-multiplying by $P_r$ gives an incorrect result:
$$left(beginarrayccc0.5 & 0.5i & 0\-0.5i & 0.5 & 0\0 & 0 & 1endarrayright)left(beginarrayc1\i\1+iendarrayright)=left(beginarrayc0\0\1+iendarrayright)mbox.$$
This example is meant to illustrate that one must be more careful with nonreal orthogonal projection matrices. In contrast, if $P_r$ is a real matrix, then both $bP_r$ and $P_rb$ give the correct result.
add a comment |Â
up vote
0
down vote
You didn't say whether the entries of your matrix are real or complex. The matrices $Aleft(A^TAright)^-1A^T$ and $A^Tleft(AA^Tright)^-1A$ work fine for a real matrix $A$ whose columns or rows are linearly independent. But if $A$ has any complex entries, then more generally the orthogonal projection onto the column and row spaces are$$P_c=Aleft(A^*Aright)^-1A^*$$ and $$P_r=A^*left(AA^*right)^-1Ambox,$$ respectively, where $A^*$ denotes the conjugate transpose of $A$, that is,$$A^*=overlineA^Tmbox.$$
As with real matrices, the formula for $P_c$ requires that the columns of $A$ be linearly independent, while the formula for $P_r$ requires that the rows of $A$ be linearly independent. Furthermore, in general one left-multiplies by $P_c$ but right-multiplies by $P_r$. That is, given vector $b$, we have
$$mboxproj_mboxcolAb=P_cb$$
but
$$mboxproj_mboxrowAb=bP_rmbox.$$
We write $b$ as a column vector when projecting onto the column space, but as a row vector when projecting onto the row space.
For example, take
$$A=left(beginarrayccc1 & i & 1+i\1 & i & 1-iendarrayright)mbox.$$
The rows of $A$ are linearly independent, so
$$P_r=A^*left(AA^*right)^-1A=left(beginarrayccc0.5 & 0.5i & 0\-0.5i & 0.5 & 0\0 & 0 & 1endarrayright)mbox.$$
Now take $b=left(beginarrayccc1 & i & 1+iendarrayright)$. Notice that $b$ is simply the first row of $A$. So we should have $mboxproj_mboxrowAb=b$. Right-multiplying $b$ by $P_r$ gives
$$left(beginarrayccc1 & i & 1+iendarrayright)left(beginarrayccc0.5 & 0.5i & 0\-0.5i & 0.5 & 0\0 & 0 & 1endarrayright)=left(beginarrayccc1 & i & 1+iendarrayright)$$
as expected. However, writing $b$ as a column vector and left-multiplying by $P_r$ gives an incorrect result:
$$left(beginarrayccc0.5 & 0.5i & 0\-0.5i & 0.5 & 0\0 & 0 & 1endarrayright)left(beginarrayc1\i\1+iendarrayright)=left(beginarrayc0\0\1+iendarrayright)mbox.$$
This example is meant to illustrate that one must be more careful with nonreal orthogonal projection matrices. In contrast, if $P_r$ is a real matrix, then both $bP_r$ and $P_rb$ give the correct result.
add a comment |Â
up vote
0
down vote
up vote
0
down vote
You didn't say whether the entries of your matrix are real or complex. The matrices $Aleft(A^TAright)^-1A^T$ and $A^Tleft(AA^Tright)^-1A$ work fine for a real matrix $A$ whose columns or rows are linearly independent. But if $A$ has any complex entries, then more generally the orthogonal projection onto the column and row spaces are$$P_c=Aleft(A^*Aright)^-1A^*$$ and $$P_r=A^*left(AA^*right)^-1Ambox,$$ respectively, where $A^*$ denotes the conjugate transpose of $A$, that is,$$A^*=overlineA^Tmbox.$$
As with real matrices, the formula for $P_c$ requires that the columns of $A$ be linearly independent, while the formula for $P_r$ requires that the rows of $A$ be linearly independent. Furthermore, in general one left-multiplies by $P_c$ but right-multiplies by $P_r$. That is, given vector $b$, we have
$$mboxproj_mboxcolAb=P_cb$$
but
$$mboxproj_mboxrowAb=bP_rmbox.$$
We write $b$ as a column vector when projecting onto the column space, but as a row vector when projecting onto the row space.
For example, take
$$A=left(beginarrayccc1 & i & 1+i\1 & i & 1-iendarrayright)mbox.$$
The rows of $A$ are linearly independent, so
$$P_r=A^*left(AA^*right)^-1A=left(beginarrayccc0.5 & 0.5i & 0\-0.5i & 0.5 & 0\0 & 0 & 1endarrayright)mbox.$$
Now take $b=left(beginarrayccc1 & i & 1+iendarrayright)$. Notice that $b$ is simply the first row of $A$. So we should have $mboxproj_mboxrowAb=b$. Right-multiplying $b$ by $P_r$ gives
$$left(beginarrayccc1 & i & 1+iendarrayright)left(beginarrayccc0.5 & 0.5i & 0\-0.5i & 0.5 & 0\0 & 0 & 1endarrayright)=left(beginarrayccc1 & i & 1+iendarrayright)$$
as expected. However, writing $b$ as a column vector and left-multiplying by $P_r$ gives an incorrect result:
$$left(beginarrayccc0.5 & 0.5i & 0\-0.5i & 0.5 & 0\0 & 0 & 1endarrayright)left(beginarrayc1\i\1+iendarrayright)=left(beginarrayc0\0\1+iendarrayright)mbox.$$
This example is meant to illustrate that one must be more careful with nonreal orthogonal projection matrices. In contrast, if $P_r$ is a real matrix, then both $bP_r$ and $P_rb$ give the correct result.
You didn't say whether the entries of your matrix are real or complex. The matrices $Aleft(A^TAright)^-1A^T$ and $A^Tleft(AA^Tright)^-1A$ work fine for a real matrix $A$ whose columns or rows are linearly independent. But if $A$ has any complex entries, then more generally the orthogonal projection onto the column and row spaces are$$P_c=Aleft(A^*Aright)^-1A^*$$ and $$P_r=A^*left(AA^*right)^-1Ambox,$$ respectively, where $A^*$ denotes the conjugate transpose of $A$, that is,$$A^*=overlineA^Tmbox.$$
As with real matrices, the formula for $P_c$ requires that the columns of $A$ be linearly independent, while the formula for $P_r$ requires that the rows of $A$ be linearly independent. Furthermore, in general one left-multiplies by $P_c$ but right-multiplies by $P_r$. That is, given vector $b$, we have
$$mboxproj_mboxcolAb=P_cb$$
but
$$mboxproj_mboxrowAb=bP_rmbox.$$
We write $b$ as a column vector when projecting onto the column space, but as a row vector when projecting onto the row space.
For example, take
$$A=left(beginarrayccc1 & i & 1+i\1 & i & 1-iendarrayright)mbox.$$
The rows of $A$ are linearly independent, so
$$P_r=A^*left(AA^*right)^-1A=left(beginarrayccc0.5 & 0.5i & 0\-0.5i & 0.5 & 0\0 & 0 & 1endarrayright)mbox.$$
Now take $b=left(beginarrayccc1 & i & 1+iendarrayright)$. Notice that $b$ is simply the first row of $A$. So we should have $mboxproj_mboxrowAb=b$. Right-multiplying $b$ by $P_r$ gives
$$left(beginarrayccc1 & i & 1+iendarrayright)left(beginarrayccc0.5 & 0.5i & 0\-0.5i & 0.5 & 0\0 & 0 & 1endarrayright)=left(beginarrayccc1 & i & 1+iendarrayright)$$
as expected. However, writing $b$ as a column vector and left-multiplying by $P_r$ gives an incorrect result:
$$left(beginarrayccc0.5 & 0.5i & 0\-0.5i & 0.5 & 0\0 & 0 & 1endarrayright)left(beginarrayc1\i\1+iendarrayright)=left(beginarrayc0\0\1+iendarrayright)mbox.$$
This example is meant to illustrate that one must be more careful with nonreal orthogonal projection matrices. In contrast, if $P_r$ is a real matrix, then both $bP_r$ and $P_rb$ give the correct result.
edited Aug 20 at 7:17
answered Aug 20 at 6:34
Marc Nye
564
564
add a comment |Â
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f1774595%2fwhat-is-the-difference-between-the-projection-onto-the-column-space-and-projecti%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
1
Projecting onto the row-space of $A$ is the same as projecting onto the column-space of $A^T$.
â Omnomnomnom
May 6 '16 at 20:28
1
Solving $A^TA hat x = A^Tb$ is "projective $b$ onto the column-space of the matrix $A$". Strictly speaking, it is non-sense to say that you're "projecting a vector onto a matrix".
â Omnomnomnom
May 6 '16 at 20:29
@Omnomnomnom so if I have to project onto the row space I would use $AA^That x=Ab$?
â idknuttin
May 6 '16 at 20:38
1
Yep.$$
â Omnomnomnom
May 6 '16 at 20:52