Derivative of a matrix using Kronecker product
Clash Royale CLAN TAG#URR8PPP
up vote
1
down vote
favorite
I have this Equation (1) and I want derivative in respect to $underline s $:
$$diag(underline s)times C times diag(underline s)$$
that $underline s $ is a $n times 1$ vector and $C$ is an $n times n$ matrix. If I open up the Equation, I can find derivative but I would like to find it with Kronecker product. I tried and reach to this:
$$ ( E_j^T otimes E_i^T )vec(underline s otimes I + I otimes underline s) $$
that $ E_j $ is all zeros matrix except (j,j) element that is one. actually, I replaced derivative and vec operator because for its linearity. anyone can help me to devectorizing this or Find derivative in another way? Is it correct to displacement of vec and derivation?
thanks.
derivatives matrix-calculus kronecker-product
asked Sep 10 at 8:38
Alireza Pourafzal
85
add a comment |Â
up vote
1
down vote
favorite
I have this Equation (1) and I want derivative in respect to $underline s $:
$$diag(underline s)times C times diag(underline s)$$
that $underline s $ is a $n times 1$ vector and $C$ is an $n times n$ matrix. If I open up the Equation, I can find derivative but I would like to find it with Kronecker product. I tried and reach to this:
$$ ( E_j^T otimes E_i^T )vec(underline s otimes I + I otimes underline s) $$
that $ E_j $ is all zeros matrix except (j,j) element that is one. actually, I replaced derivative and vec operator because for its linearity. anyone can help me to devectorizing this or Find derivative in another way? Is it correct to displacement of vec and derivation?
thanks.
derivatives matrix-calculus kronecker-product
asked Sep 10 at 8:38
Alireza Pourafzal
85
You might have better luck with elementwise/Hadamard $odot$ products rather than Kronecker $otimes$ products. $$eqalignF&=rm Diag(s)cdot Ccdotrm Diag(s)=Codot ss^Tcr dF&=Codot(s,ds^T+ds,s^T)$$
– greg
Sep 10 at 16:49
add a comment |Â
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I have this Equation (1) and I want derivative in respect to $underline s $:
$$diag(underline s)times C times diag(underline s)$$
that $underline s $ is a $n times 1$ vector and $C$ is an $n times n$ matrix. If I open up the Equation, I can find derivative but I would like to find it with Kronecker product. I tried and reach to this:
$$ ( E_j^T otimes E_i^T )vec(underline s otimes I + I otimes underline s) $$
that $ E_j $ is all zeros matrix except (j,j) element that is one. actually, I replaced derivative and vec operator because for its linearity. anyone can help me to devectorizing this or Find derivative in another way? Is it correct to displacement of vec and derivation?
thanks.
derivatives matrix-calculus kronecker-product
asked Sep 10 at 8:38
Alireza Pourafzal
85
I have this Equation (1) and I want derivative in respect to $underline s $:
$$diag(underline s)times C times diag(underline s)$$
that $underline s $ is a $n times 1$ vector and $C$ is an $n times n$ matrix. If I open up the Equation, I can find derivative but I would like to find it with Kronecker product. I tried and reach to this:
$$ ( E_j^T otimes E_i^T )vec(underline s otimes I + I otimes underline s) $$
that $ E_j $ is all zeros matrix except (j,j) element that is one. actually, I replaced derivative and vec operator because for its linearity. anyone can help me to devectorizing this or Find derivative in another way? Is it correct to displacement of vec and derivation?
thanks.
derivatives matrix-calculus kronecker-product
derivatives matrix-calculus kronecker-product
asked Sep 10 at 8:38
Alireza Pourafzal
85
asked Sep 10 at 8:38
Alireza Pourafzal
85
edited Sep 10 at 11:28
asked Sep 10 at 8:38
Alireza Pourafzal
85
asked Sep 10 at 8:38
Alireza Pourafzal
85
asked Sep 10 at 8:38
Alireza Pourafzal
85
85
You might have better luck with elementwise/Hadamard $odot$ products rather than Kronecker $otimes$ products. $$eqalignF&=rm Diag(s)cdot Ccdotrm Diag(s)=Codot ss^Tcr dF&=Codot(s,ds^T+ds,s^T)$$
– greg
Sep 10 at 16:49
add a comment |Â
You might have better luck with elementwise/Hadamard $odot$ products rather than Kronecker $otimes$ products. $$eqalignF&=rm Diag(s)cdot Ccdotrm Diag(s)=Codot ss^Tcr dF&=Codot(s,ds^T+ds,s^T)$$
– greg
Sep 10 at 16:49
You might have better luck with elementwise/Hadamard $odot$ products rather than Kronecker $otimes$ products. $$eqalignF&=rm Diag(s)cdot Ccdotrm Diag(s)=Codot ss^Tcr dF&=Codot(s,ds^T+ds,s^T)$$
– greg
Sep 10 at 16:49
You might have better luck with elementwise/Hadamard $odot$ products rather than Kronecker $otimes$ products. $$eqalignF&=rm Diag(s)cdot Ccdotrm Diag(s)=Codot ss^Tcr dF&=Codot(s,ds^T+ds,s^T)$$
– greg
Sep 10 at 16:49
add a comment |Â
1 Answer
1
active
oldest
votes
up vote
3
down vote
accepted
To combine your approach with the one in my comment...
First write the function and its differential using a Hadamard product
$$eqalign
F &= rm Diag(s),C,rm Diag(s) = Codot ss^T cr
dF &= Codot ds,s^T + Codot s,ds^T cr
$$
Define the diagonal matrix $$D_c=rm Diagbig(rm vec(C)big)$$
and use it to vectorize the differential
$$eqalign
df
&= D_c,rm vec(ds,s^T) + D_c,rm vec(s,ds^T) cr
&= D_c,(sotimes I),ds + D_c,(Iotimes s),ds cr
&= D_c,big(sotimes I + Iotimes sbig),dscr
$$
Yielding the gradient of the vectorized function as
$$fracpartial fpartial s = D_c,big(sotimes I + Iotimes sbig)$$
answered Sep 10 at 20:19
greg
6,1781717
add a comment |Â
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
3
down vote
accepted
To combine your approach with the one in my comment...
First write the function and its differential using a Hadamard product
$$eqalign
F &= rm Diag(s),C,rm Diag(s) = Codot ss^T cr
dF &= Codot ds,s^T + Codot s,ds^T cr
$$
Define the diagonal matrix $$D_c=rm Diagbig(rm vec(C)big)$$
and use it to vectorize the differential
$$eqalign
df
&= D_c,rm vec(ds,s^T) + D_c,rm vec(s,ds^T) cr
&= D_c,(sotimes I),ds + D_c,(Iotimes s),ds cr
&= D_c,big(sotimes I + Iotimes sbig),dscr
$$
Yielding the gradient of the vectorized function as
$$fracpartial fpartial s = D_c,big(sotimes I + Iotimes sbig)$$
answered Sep 10 at 20:19
greg
6,1781717
add a comment |Â
up vote
3
down vote
accepted
To combine your approach with the one in my comment...
First write the function and its differential using a Hadamard product
$$eqalign
F &= rm Diag(s),C,rm Diag(s) = Codot ss^T cr
dF &= Codot ds,s^T + Codot s,ds^T cr
$$
Define the diagonal matrix $$D_c=rm Diagbig(rm vec(C)big)$$
and use it to vectorize the differential
$$eqalign
df
&= D_c,rm vec(ds,s^T) + D_c,rm vec(s,ds^T) cr
&= D_c,(sotimes I),ds + D_c,(Iotimes s),ds cr
&= D_c,big(sotimes I + Iotimes sbig),dscr
$$
Yielding the gradient of the vectorized function as
$$fracpartial fpartial s = D_c,big(sotimes I + Iotimes sbig)$$
answered Sep 10 at 20:19
greg
6,1781717
add a comment |Â
up vote
3
down vote
accepted
up vote
3
down vote
accepted
To combine your approach with the one in my comment...
First write the function and its differential using a Hadamard product
$$eqalign
F &= rm Diag(s),C,rm Diag(s) = Codot ss^T cr
dF &= Codot ds,s^T + Codot s,ds^T cr
$$
Define the diagonal matrix $$D_c=rm Diagbig(rm vec(C)big)$$
and use it to vectorize the differential
$$eqalign
df
&= D_c,rm vec(ds,s^T) + D_c,rm vec(s,ds^T) cr
&= D_c,(sotimes I),ds + D_c,(Iotimes s),ds cr
&= D_c,big(sotimes I + Iotimes sbig),dscr
$$
Yielding the gradient of the vectorized function as
$$fracpartial fpartial s = D_c,big(sotimes I + Iotimes sbig)$$
answered Sep 10 at 20:19
greg
6,1781717
To combine your approach with the one in my comment...
First write the function and its differential using a Hadamard product
$$eqalign
F &= rm Diag(s),C,rm Diag(s) = Codot ss^T cr
dF &= Codot ds,s^T + Codot s,ds^T cr
$$
Define the diagonal matrix $$D_c=rm Diagbig(rm vec(C)big)$$
and use it to vectorize the differential
$$eqalign
df
&= D_c,rm vec(ds,s^T) + D_c,rm vec(s,ds^T) cr
&= D_c,(sotimes I),ds + D_c,(Iotimes s),ds cr
&= D_c,big(sotimes I + Iotimes sbig),dscr
$$
Yielding the gradient of the vectorized function as
$$fracpartial fpartial s = D_c,big(sotimes I + Iotimes sbig)$$
answered Sep 10 at 20:19
greg
6,1781717
edited Sep 10 at 20:52
answered Sep 10 at 20:19
greg
6,1781717
answered Sep 10 at 20:19
greg
6,1781717
answered Sep 10 at 20:19
greg
6,1781717
6,1781717
add a comment |Â
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2911682%2fderivative-of-a-matrix-using-kronecker-product%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
You might have better luck with elementwise/Hadamard $odot$ products rather than Kronecker $otimes$ products. $$eqalignF&=rm Diag(s)cdot Ccdotrm Diag(s)=Codot ss^Tcr dF&=Codot(s,ds^T+ds,s^T)$$
– greg
Sep 10 at 16:49