Is the mixture of Exponential family distributions an Exponential family distribution too?
Clash Royale CLAN TAG#URR8PPP
up vote
1
down vote
favorite
Consider we have a mixture of multinomials or in a broader sense, a mixture of $f$s where $f$ is an distribution of exponential family type and the membership components are known with the sum of 1. Is the new distribution an exponential family too?
probability
add a comment |Â
up vote
1
down vote
favorite
Consider we have a mixture of multinomials or in a broader sense, a mixture of $f$s where $f$ is an distribution of exponential family type and the membership components are known with the sum of 1. Is the new distribution an exponential family too?
probability
add a comment |Â
up vote
1
down vote
favorite
up vote
1
down vote
favorite
Consider we have a mixture of multinomials or in a broader sense, a mixture of $f$s where $f$ is an distribution of exponential family type and the membership components are known with the sum of 1. Is the new distribution an exponential family too?
probability
Consider we have a mixture of multinomials or in a broader sense, a mixture of $f$s where $f$ is an distribution of exponential family type and the membership components are known with the sum of 1. Is the new distribution an exponential family too?
probability
asked Apr 29 '16 at 14:35
CoderInNetwork
539
539
add a comment |Â
add a comment |Â
2 Answers
2
active
oldest
votes
up vote
1
down vote
In general the answer is "no".
The mixture distribution
$$
alpha h_1(x)exp^theta_1^T phi_1(x) - A_1(theta_1) + (1-alpha) h_2(x)exp^theta_2^T phi_2(x) - A_2(theta_2)
$$
generally can't be factored into something like
$$
h_3(x) exp^theta_3^T phi_3(x) - A_3(theta_3)
$$
even if the two components come from the same exponential family (so $h_1=h_2, phi_1=phi_2, A_1=A_2$). A classic example is a mixture of Gaussians.
Although the product of exponential families is an (unnormalized)
exponential family, the mixture of exponential families is not an
exponential family.
--page 6 of Statistical exponential families: A digest with flash cards
add a comment |Â
up vote
-1
down vote
In my viewpoint, this depends on the nature of the mixed distributions. Without loss of generality, consider a probability distribution with a single rate parameter $theta$. The latter probability distribution belongs to the exponential family if its PDF can be expressed in the form
$$
f(x | theta) = e^eta(theta)T(x)-A(theta)+B(x)
$$
Now, consider an exponential distribution with rate parameter $theta$ and a gamma distribution with known shape parameter $2$ and rate parameter $theta$ with corresponding PDFs
$$
f(x | theta) = theta e^-theta x equiv f(x | theta) = e^-theta x + log theta + 0
$$
and
$$
f(x | theta) = theta^2 x e^-theta x equiv f(x | theta) = e^-theta x + 2 log theta + x
$$
respectively. By mixing these distribution using the weights $theta(1 + theta)^-1$ and $(1 + theta)^-1$, respectively, we will get the PDF of the Lindley distribution that is given by
$$
f(x | theta) = theta^2 (1 + theta)^-1 (1 + x) e^-theta x equiv f(x | theta) = e^-theta x + log leftlbracetheta^2 (1 + theta)rightrbrace + log(1 + x)
$$
My personal verdict is that there might be "a few" special cases when a mixture of $f$s, where $f$ is a distribution of the exponential family, will yield a distribution that belongs to the exponential family.
thanks @FALAM. actually your example is not an contraction to my claim. I don't ask about mixing a gamma with an exponential but for example mixing two gammas or two exponentials. (all of the component measures are from a same distribution). BTW your analysis is interesting although not fit to the question exactly.
â CoderInNetwork
May 8 '16 at 7:37
Thanks for your reply @CoderInNetwork. Actually, the exponential distribution is a gamma distribution with shape parameter equals to 1. Hence, the Lindley distribution is in fact the probability distribution of a mixture of two gamma random variables with similar rate parameters and different shape parameters.
â FALAM
May 21 '16 at 14:18
add a comment |Â
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
In general the answer is "no".
The mixture distribution
$$
alpha h_1(x)exp^theta_1^T phi_1(x) - A_1(theta_1) + (1-alpha) h_2(x)exp^theta_2^T phi_2(x) - A_2(theta_2)
$$
generally can't be factored into something like
$$
h_3(x) exp^theta_3^T phi_3(x) - A_3(theta_3)
$$
even if the two components come from the same exponential family (so $h_1=h_2, phi_1=phi_2, A_1=A_2$). A classic example is a mixture of Gaussians.
Although the product of exponential families is an (unnormalized)
exponential family, the mixture of exponential families is not an
exponential family.
--page 6 of Statistical exponential families: A digest with flash cards
add a comment |Â
up vote
1
down vote
In general the answer is "no".
The mixture distribution
$$
alpha h_1(x)exp^theta_1^T phi_1(x) - A_1(theta_1) + (1-alpha) h_2(x)exp^theta_2^T phi_2(x) - A_2(theta_2)
$$
generally can't be factored into something like
$$
h_3(x) exp^theta_3^T phi_3(x) - A_3(theta_3)
$$
even if the two components come from the same exponential family (so $h_1=h_2, phi_1=phi_2, A_1=A_2$). A classic example is a mixture of Gaussians.
Although the product of exponential families is an (unnormalized)
exponential family, the mixture of exponential families is not an
exponential family.
--page 6 of Statistical exponential families: A digest with flash cards
add a comment |Â
up vote
1
down vote
up vote
1
down vote
In general the answer is "no".
The mixture distribution
$$
alpha h_1(x)exp^theta_1^T phi_1(x) - A_1(theta_1) + (1-alpha) h_2(x)exp^theta_2^T phi_2(x) - A_2(theta_2)
$$
generally can't be factored into something like
$$
h_3(x) exp^theta_3^T phi_3(x) - A_3(theta_3)
$$
even if the two components come from the same exponential family (so $h_1=h_2, phi_1=phi_2, A_1=A_2$). A classic example is a mixture of Gaussians.
Although the product of exponential families is an (unnormalized)
exponential family, the mixture of exponential families is not an
exponential family.
--page 6 of Statistical exponential families: A digest with flash cards
In general the answer is "no".
The mixture distribution
$$
alpha h_1(x)exp^theta_1^T phi_1(x) - A_1(theta_1) + (1-alpha) h_2(x)exp^theta_2^T phi_2(x) - A_2(theta_2)
$$
generally can't be factored into something like
$$
h_3(x) exp^theta_3^T phi_3(x) - A_3(theta_3)
$$
even if the two components come from the same exponential family (so $h_1=h_2, phi_1=phi_2, A_1=A_2$). A classic example is a mixture of Gaussians.
Although the product of exponential families is an (unnormalized)
exponential family, the mixture of exponential families is not an
exponential family.
--page 6 of Statistical exponential families: A digest with flash cards
edited Aug 18 at 15:10
answered Jan 26 '17 at 17:11
Yibo Yang
470316
470316
add a comment |Â
add a comment |Â
up vote
-1
down vote
In my viewpoint, this depends on the nature of the mixed distributions. Without loss of generality, consider a probability distribution with a single rate parameter $theta$. The latter probability distribution belongs to the exponential family if its PDF can be expressed in the form
$$
f(x | theta) = e^eta(theta)T(x)-A(theta)+B(x)
$$
Now, consider an exponential distribution with rate parameter $theta$ and a gamma distribution with known shape parameter $2$ and rate parameter $theta$ with corresponding PDFs
$$
f(x | theta) = theta e^-theta x equiv f(x | theta) = e^-theta x + log theta + 0
$$
and
$$
f(x | theta) = theta^2 x e^-theta x equiv f(x | theta) = e^-theta x + 2 log theta + x
$$
respectively. By mixing these distribution using the weights $theta(1 + theta)^-1$ and $(1 + theta)^-1$, respectively, we will get the PDF of the Lindley distribution that is given by
$$
f(x | theta) = theta^2 (1 + theta)^-1 (1 + x) e^-theta x equiv f(x | theta) = e^-theta x + log leftlbracetheta^2 (1 + theta)rightrbrace + log(1 + x)
$$
My personal verdict is that there might be "a few" special cases when a mixture of $f$s, where $f$ is a distribution of the exponential family, will yield a distribution that belongs to the exponential family.
thanks @FALAM. actually your example is not an contraction to my claim. I don't ask about mixing a gamma with an exponential but for example mixing two gammas or two exponentials. (all of the component measures are from a same distribution). BTW your analysis is interesting although not fit to the question exactly.
â CoderInNetwork
May 8 '16 at 7:37
Thanks for your reply @CoderInNetwork. Actually, the exponential distribution is a gamma distribution with shape parameter equals to 1. Hence, the Lindley distribution is in fact the probability distribution of a mixture of two gamma random variables with similar rate parameters and different shape parameters.
â FALAM
May 21 '16 at 14:18
add a comment |Â
up vote
-1
down vote
In my viewpoint, this depends on the nature of the mixed distributions. Without loss of generality, consider a probability distribution with a single rate parameter $theta$. The latter probability distribution belongs to the exponential family if its PDF can be expressed in the form
$$
f(x | theta) = e^eta(theta)T(x)-A(theta)+B(x)
$$
Now, consider an exponential distribution with rate parameter $theta$ and a gamma distribution with known shape parameter $2$ and rate parameter $theta$ with corresponding PDFs
$$
f(x | theta) = theta e^-theta x equiv f(x | theta) = e^-theta x + log theta + 0
$$
and
$$
f(x | theta) = theta^2 x e^-theta x equiv f(x | theta) = e^-theta x + 2 log theta + x
$$
respectively. By mixing these distribution using the weights $theta(1 + theta)^-1$ and $(1 + theta)^-1$, respectively, we will get the PDF of the Lindley distribution that is given by
$$
f(x | theta) = theta^2 (1 + theta)^-1 (1 + x) e^-theta x equiv f(x | theta) = e^-theta x + log leftlbracetheta^2 (1 + theta)rightrbrace + log(1 + x)
$$
My personal verdict is that there might be "a few" special cases when a mixture of $f$s, where $f$ is a distribution of the exponential family, will yield a distribution that belongs to the exponential family.
thanks @FALAM. actually your example is not an contraction to my claim. I don't ask about mixing a gamma with an exponential but for example mixing two gammas or two exponentials. (all of the component measures are from a same distribution). BTW your analysis is interesting although not fit to the question exactly.
â CoderInNetwork
May 8 '16 at 7:37
Thanks for your reply @CoderInNetwork. Actually, the exponential distribution is a gamma distribution with shape parameter equals to 1. Hence, the Lindley distribution is in fact the probability distribution of a mixture of two gamma random variables with similar rate parameters and different shape parameters.
â FALAM
May 21 '16 at 14:18
add a comment |Â
up vote
-1
down vote
up vote
-1
down vote
In my viewpoint, this depends on the nature of the mixed distributions. Without loss of generality, consider a probability distribution with a single rate parameter $theta$. The latter probability distribution belongs to the exponential family if its PDF can be expressed in the form
$$
f(x | theta) = e^eta(theta)T(x)-A(theta)+B(x)
$$
Now, consider an exponential distribution with rate parameter $theta$ and a gamma distribution with known shape parameter $2$ and rate parameter $theta$ with corresponding PDFs
$$
f(x | theta) = theta e^-theta x equiv f(x | theta) = e^-theta x + log theta + 0
$$
and
$$
f(x | theta) = theta^2 x e^-theta x equiv f(x | theta) = e^-theta x + 2 log theta + x
$$
respectively. By mixing these distribution using the weights $theta(1 + theta)^-1$ and $(1 + theta)^-1$, respectively, we will get the PDF of the Lindley distribution that is given by
$$
f(x | theta) = theta^2 (1 + theta)^-1 (1 + x) e^-theta x equiv f(x | theta) = e^-theta x + log leftlbracetheta^2 (1 + theta)rightrbrace + log(1 + x)
$$
My personal verdict is that there might be "a few" special cases when a mixture of $f$s, where $f$ is a distribution of the exponential family, will yield a distribution that belongs to the exponential family.
In my viewpoint, this depends on the nature of the mixed distributions. Without loss of generality, consider a probability distribution with a single rate parameter $theta$. The latter probability distribution belongs to the exponential family if its PDF can be expressed in the form
$$
f(x | theta) = e^eta(theta)T(x)-A(theta)+B(x)
$$
Now, consider an exponential distribution with rate parameter $theta$ and a gamma distribution with known shape parameter $2$ and rate parameter $theta$ with corresponding PDFs
$$
f(x | theta) = theta e^-theta x equiv f(x | theta) = e^-theta x + log theta + 0
$$
and
$$
f(x | theta) = theta^2 x e^-theta x equiv f(x | theta) = e^-theta x + 2 log theta + x
$$
respectively. By mixing these distribution using the weights $theta(1 + theta)^-1$ and $(1 + theta)^-1$, respectively, we will get the PDF of the Lindley distribution that is given by
$$
f(x | theta) = theta^2 (1 + theta)^-1 (1 + x) e^-theta x equiv f(x | theta) = e^-theta x + log leftlbracetheta^2 (1 + theta)rightrbrace + log(1 + x)
$$
My personal verdict is that there might be "a few" special cases when a mixture of $f$s, where $f$ is a distribution of the exponential family, will yield a distribution that belongs to the exponential family.
answered May 2 '16 at 1:25
FALAM
30916
30916
thanks @FALAM. actually your example is not an contraction to my claim. I don't ask about mixing a gamma with an exponential but for example mixing two gammas or two exponentials. (all of the component measures are from a same distribution). BTW your analysis is interesting although not fit to the question exactly.
â CoderInNetwork
May 8 '16 at 7:37
Thanks for your reply @CoderInNetwork. Actually, the exponential distribution is a gamma distribution with shape parameter equals to 1. Hence, the Lindley distribution is in fact the probability distribution of a mixture of two gamma random variables with similar rate parameters and different shape parameters.
â FALAM
May 21 '16 at 14:18
add a comment |Â
thanks @FALAM. actually your example is not an contraction to my claim. I don't ask about mixing a gamma with an exponential but for example mixing two gammas or two exponentials. (all of the component measures are from a same distribution). BTW your analysis is interesting although not fit to the question exactly.
â CoderInNetwork
May 8 '16 at 7:37
Thanks for your reply @CoderInNetwork. Actually, the exponential distribution is a gamma distribution with shape parameter equals to 1. Hence, the Lindley distribution is in fact the probability distribution of a mixture of two gamma random variables with similar rate parameters and different shape parameters.
â FALAM
May 21 '16 at 14:18
thanks @FALAM. actually your example is not an contraction to my claim. I don't ask about mixing a gamma with an exponential but for example mixing two gammas or two exponentials. (all of the component measures are from a same distribution). BTW your analysis is interesting although not fit to the question exactly.
â CoderInNetwork
May 8 '16 at 7:37
thanks @FALAM. actually your example is not an contraction to my claim. I don't ask about mixing a gamma with an exponential but for example mixing two gammas or two exponentials. (all of the component measures are from a same distribution). BTW your analysis is interesting although not fit to the question exactly.
â CoderInNetwork
May 8 '16 at 7:37
Thanks for your reply @CoderInNetwork. Actually, the exponential distribution is a gamma distribution with shape parameter equals to 1. Hence, the Lindley distribution is in fact the probability distribution of a mixture of two gamma random variables with similar rate parameters and different shape parameters.
â FALAM
May 21 '16 at 14:18
Thanks for your reply @CoderInNetwork. Actually, the exponential distribution is a gamma distribution with shape parameter equals to 1. Hence, the Lindley distribution is in fact the probability distribution of a mixture of two gamma random variables with similar rate parameters and different shape parameters.
â FALAM
May 21 '16 at 14:18
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f1764166%2fis-the-mixture-of-exponential-family-distributions-an-exponential-family-distrib%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password