Is the mixture of Exponential family distributions an Exponential family distribution too?

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
1
down vote

favorite












Consider we have a mixture of multinomials or in a broader sense, a mixture of $f$s where $f$ is an distribution of exponential family type and the membership components are known with the sum of 1. Is the new distribution an exponential family too?







share|cite|improve this question
























    up vote
    1
    down vote

    favorite












    Consider we have a mixture of multinomials or in a broader sense, a mixture of $f$s where $f$ is an distribution of exponential family type and the membership components are known with the sum of 1. Is the new distribution an exponential family too?







    share|cite|improve this question






















      up vote
      1
      down vote

      favorite









      up vote
      1
      down vote

      favorite











      Consider we have a mixture of multinomials or in a broader sense, a mixture of $f$s where $f$ is an distribution of exponential family type and the membership components are known with the sum of 1. Is the new distribution an exponential family too?







      share|cite|improve this question












      Consider we have a mixture of multinomials or in a broader sense, a mixture of $f$s where $f$ is an distribution of exponential family type and the membership components are known with the sum of 1. Is the new distribution an exponential family too?









      share|cite|improve this question











      share|cite|improve this question




      share|cite|improve this question










      asked Apr 29 '16 at 14:35









      CoderInNetwork

      539




      539




















          2 Answers
          2






          active

          oldest

          votes

















          up vote
          1
          down vote













          In general the answer is "no".



          The mixture distribution
          $$
          alpha h_1(x)exp^theta_1^T phi_1(x) - A_1(theta_1) + (1-alpha) h_2(x)exp^theta_2^T phi_2(x) - A_2(theta_2)
          $$
          generally can't be factored into something like
          $$
          h_3(x) exp^theta_3^T phi_3(x) - A_3(theta_3)
          $$
          even if the two components come from the same exponential family (so $h_1=h_2, phi_1=phi_2, A_1=A_2$). A classic example is a mixture of Gaussians.




          Although the product of exponential families is an (unnormalized)
          exponential family, the mixture of exponential families is not an
          exponential family.




          --page 6 of Statistical exponential families: A digest with flash cards






          share|cite|improve this answer





























            up vote
            -1
            down vote













            In my viewpoint, this depends on the nature of the mixed distributions. Without loss of generality, consider a probability distribution with a single rate parameter $theta$. The latter probability distribution belongs to the exponential family if its PDF can be expressed in the form
            $$
            f(x | theta) = e^eta(theta)T(x)-A(theta)+B(x)
            $$
            Now, consider an exponential distribution with rate parameter $theta$ and a gamma distribution with known shape parameter $2$ and rate parameter $theta$ with corresponding PDFs
            $$
            f(x | theta) = theta e^-theta x equiv f(x | theta) = e^-theta x + log theta + 0
            $$
            and
            $$
            f(x | theta) = theta^2 x e^-theta x equiv f(x | theta) = e^-theta x + 2 log theta + x
            $$
            respectively. By mixing these distribution using the weights $theta(1 + theta)^-1$ and $(1 + theta)^-1$, respectively, we will get the PDF of the Lindley distribution that is given by
            $$
            f(x | theta) = theta^2 (1 + theta)^-1 (1 + x) e^-theta x equiv f(x | theta) = e^-theta x + log leftlbracetheta^2 (1 + theta)rightrbrace + log(1 + x)
            $$



            My personal verdict is that there might be "a few" special cases when a mixture of $f$s, where $f$ is a distribution of the exponential family, will yield a distribution that belongs to the exponential family.






            share|cite|improve this answer




















            • thanks @FALAM. actually your example is not an contraction to my claim. I don't ask about mixing a gamma with an exponential but for example mixing two gammas or two exponentials. (all of the component measures are from a same distribution). BTW your analysis is interesting although not fit to the question exactly.
              – CoderInNetwork
              May 8 '16 at 7:37










            • Thanks for your reply @CoderInNetwork. Actually, the exponential distribution is a gamma distribution with shape parameter equals to 1. Hence, the Lindley distribution is in fact the probability distribution of a mixture of two gamma random variables with similar rate parameters and different shape parameters.
              – FALAM
              May 21 '16 at 14:18










            Your Answer




            StackExchange.ifUsing("editor", function ()
            return StackExchange.using("mathjaxEditing", function ()
            StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
            StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
            );
            );
            , "mathjax-editing");

            StackExchange.ready(function()
            var channelOptions =
            tags: "".split(" "),
            id: "69"
            ;
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function()
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled)
            StackExchange.using("snippets", function()
            createEditor();
            );

            else
            createEditor();

            );

            function createEditor()
            StackExchange.prepareEditor(
            heartbeatType: 'answer',
            convertImagesToLinks: true,
            noModals: false,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            noCode: true, onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            );



            );








             

            draft saved


            draft discarded


















            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f1764166%2fis-the-mixture-of-exponential-family-distributions-an-exponential-family-distrib%23new-answer', 'question_page');

            );

            Post as a guest






























            2 Answers
            2






            active

            oldest

            votes








            2 Answers
            2






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes








            up vote
            1
            down vote













            In general the answer is "no".



            The mixture distribution
            $$
            alpha h_1(x)exp^theta_1^T phi_1(x) - A_1(theta_1) + (1-alpha) h_2(x)exp^theta_2^T phi_2(x) - A_2(theta_2)
            $$
            generally can't be factored into something like
            $$
            h_3(x) exp^theta_3^T phi_3(x) - A_3(theta_3)
            $$
            even if the two components come from the same exponential family (so $h_1=h_2, phi_1=phi_2, A_1=A_2$). A classic example is a mixture of Gaussians.




            Although the product of exponential families is an (unnormalized)
            exponential family, the mixture of exponential families is not an
            exponential family.




            --page 6 of Statistical exponential families: A digest with flash cards






            share|cite|improve this answer


























              up vote
              1
              down vote













              In general the answer is "no".



              The mixture distribution
              $$
              alpha h_1(x)exp^theta_1^T phi_1(x) - A_1(theta_1) + (1-alpha) h_2(x)exp^theta_2^T phi_2(x) - A_2(theta_2)
              $$
              generally can't be factored into something like
              $$
              h_3(x) exp^theta_3^T phi_3(x) - A_3(theta_3)
              $$
              even if the two components come from the same exponential family (so $h_1=h_2, phi_1=phi_2, A_1=A_2$). A classic example is a mixture of Gaussians.




              Although the product of exponential families is an (unnormalized)
              exponential family, the mixture of exponential families is not an
              exponential family.




              --page 6 of Statistical exponential families: A digest with flash cards






              share|cite|improve this answer
























                up vote
                1
                down vote










                up vote
                1
                down vote









                In general the answer is "no".



                The mixture distribution
                $$
                alpha h_1(x)exp^theta_1^T phi_1(x) - A_1(theta_1) + (1-alpha) h_2(x)exp^theta_2^T phi_2(x) - A_2(theta_2)
                $$
                generally can't be factored into something like
                $$
                h_3(x) exp^theta_3^T phi_3(x) - A_3(theta_3)
                $$
                even if the two components come from the same exponential family (so $h_1=h_2, phi_1=phi_2, A_1=A_2$). A classic example is a mixture of Gaussians.




                Although the product of exponential families is an (unnormalized)
                exponential family, the mixture of exponential families is not an
                exponential family.




                --page 6 of Statistical exponential families: A digest with flash cards






                share|cite|improve this answer














                In general the answer is "no".



                The mixture distribution
                $$
                alpha h_1(x)exp^theta_1^T phi_1(x) - A_1(theta_1) + (1-alpha) h_2(x)exp^theta_2^T phi_2(x) - A_2(theta_2)
                $$
                generally can't be factored into something like
                $$
                h_3(x) exp^theta_3^T phi_3(x) - A_3(theta_3)
                $$
                even if the two components come from the same exponential family (so $h_1=h_2, phi_1=phi_2, A_1=A_2$). A classic example is a mixture of Gaussians.




                Although the product of exponential families is an (unnormalized)
                exponential family, the mixture of exponential families is not an
                exponential family.




                --page 6 of Statistical exponential families: A digest with flash cards







                share|cite|improve this answer














                share|cite|improve this answer



                share|cite|improve this answer








                edited Aug 18 at 15:10

























                answered Jan 26 '17 at 17:11









                Yibo Yang

                470316




                470316




















                    up vote
                    -1
                    down vote













                    In my viewpoint, this depends on the nature of the mixed distributions. Without loss of generality, consider a probability distribution with a single rate parameter $theta$. The latter probability distribution belongs to the exponential family if its PDF can be expressed in the form
                    $$
                    f(x | theta) = e^eta(theta)T(x)-A(theta)+B(x)
                    $$
                    Now, consider an exponential distribution with rate parameter $theta$ and a gamma distribution with known shape parameter $2$ and rate parameter $theta$ with corresponding PDFs
                    $$
                    f(x | theta) = theta e^-theta x equiv f(x | theta) = e^-theta x + log theta + 0
                    $$
                    and
                    $$
                    f(x | theta) = theta^2 x e^-theta x equiv f(x | theta) = e^-theta x + 2 log theta + x
                    $$
                    respectively. By mixing these distribution using the weights $theta(1 + theta)^-1$ and $(1 + theta)^-1$, respectively, we will get the PDF of the Lindley distribution that is given by
                    $$
                    f(x | theta) = theta^2 (1 + theta)^-1 (1 + x) e^-theta x equiv f(x | theta) = e^-theta x + log leftlbracetheta^2 (1 + theta)rightrbrace + log(1 + x)
                    $$



                    My personal verdict is that there might be "a few" special cases when a mixture of $f$s, where $f$ is a distribution of the exponential family, will yield a distribution that belongs to the exponential family.






                    share|cite|improve this answer




















                    • thanks @FALAM. actually your example is not an contraction to my claim. I don't ask about mixing a gamma with an exponential but for example mixing two gammas or two exponentials. (all of the component measures are from a same distribution). BTW your analysis is interesting although not fit to the question exactly.
                      – CoderInNetwork
                      May 8 '16 at 7:37










                    • Thanks for your reply @CoderInNetwork. Actually, the exponential distribution is a gamma distribution with shape parameter equals to 1. Hence, the Lindley distribution is in fact the probability distribution of a mixture of two gamma random variables with similar rate parameters and different shape parameters.
                      – FALAM
                      May 21 '16 at 14:18














                    up vote
                    -1
                    down vote













                    In my viewpoint, this depends on the nature of the mixed distributions. Without loss of generality, consider a probability distribution with a single rate parameter $theta$. The latter probability distribution belongs to the exponential family if its PDF can be expressed in the form
                    $$
                    f(x | theta) = e^eta(theta)T(x)-A(theta)+B(x)
                    $$
                    Now, consider an exponential distribution with rate parameter $theta$ and a gamma distribution with known shape parameter $2$ and rate parameter $theta$ with corresponding PDFs
                    $$
                    f(x | theta) = theta e^-theta x equiv f(x | theta) = e^-theta x + log theta + 0
                    $$
                    and
                    $$
                    f(x | theta) = theta^2 x e^-theta x equiv f(x | theta) = e^-theta x + 2 log theta + x
                    $$
                    respectively. By mixing these distribution using the weights $theta(1 + theta)^-1$ and $(1 + theta)^-1$, respectively, we will get the PDF of the Lindley distribution that is given by
                    $$
                    f(x | theta) = theta^2 (1 + theta)^-1 (1 + x) e^-theta x equiv f(x | theta) = e^-theta x + log leftlbracetheta^2 (1 + theta)rightrbrace + log(1 + x)
                    $$



                    My personal verdict is that there might be "a few" special cases when a mixture of $f$s, where $f$ is a distribution of the exponential family, will yield a distribution that belongs to the exponential family.






                    share|cite|improve this answer




















                    • thanks @FALAM. actually your example is not an contraction to my claim. I don't ask about mixing a gamma with an exponential but for example mixing two gammas or two exponentials. (all of the component measures are from a same distribution). BTW your analysis is interesting although not fit to the question exactly.
                      – CoderInNetwork
                      May 8 '16 at 7:37










                    • Thanks for your reply @CoderInNetwork. Actually, the exponential distribution is a gamma distribution with shape parameter equals to 1. Hence, the Lindley distribution is in fact the probability distribution of a mixture of two gamma random variables with similar rate parameters and different shape parameters.
                      – FALAM
                      May 21 '16 at 14:18












                    up vote
                    -1
                    down vote










                    up vote
                    -1
                    down vote









                    In my viewpoint, this depends on the nature of the mixed distributions. Without loss of generality, consider a probability distribution with a single rate parameter $theta$. The latter probability distribution belongs to the exponential family if its PDF can be expressed in the form
                    $$
                    f(x | theta) = e^eta(theta)T(x)-A(theta)+B(x)
                    $$
                    Now, consider an exponential distribution with rate parameter $theta$ and a gamma distribution with known shape parameter $2$ and rate parameter $theta$ with corresponding PDFs
                    $$
                    f(x | theta) = theta e^-theta x equiv f(x | theta) = e^-theta x + log theta + 0
                    $$
                    and
                    $$
                    f(x | theta) = theta^2 x e^-theta x equiv f(x | theta) = e^-theta x + 2 log theta + x
                    $$
                    respectively. By mixing these distribution using the weights $theta(1 + theta)^-1$ and $(1 + theta)^-1$, respectively, we will get the PDF of the Lindley distribution that is given by
                    $$
                    f(x | theta) = theta^2 (1 + theta)^-1 (1 + x) e^-theta x equiv f(x | theta) = e^-theta x + log leftlbracetheta^2 (1 + theta)rightrbrace + log(1 + x)
                    $$



                    My personal verdict is that there might be "a few" special cases when a mixture of $f$s, where $f$ is a distribution of the exponential family, will yield a distribution that belongs to the exponential family.






                    share|cite|improve this answer












                    In my viewpoint, this depends on the nature of the mixed distributions. Without loss of generality, consider a probability distribution with a single rate parameter $theta$. The latter probability distribution belongs to the exponential family if its PDF can be expressed in the form
                    $$
                    f(x | theta) = e^eta(theta)T(x)-A(theta)+B(x)
                    $$
                    Now, consider an exponential distribution with rate parameter $theta$ and a gamma distribution with known shape parameter $2$ and rate parameter $theta$ with corresponding PDFs
                    $$
                    f(x | theta) = theta e^-theta x equiv f(x | theta) = e^-theta x + log theta + 0
                    $$
                    and
                    $$
                    f(x | theta) = theta^2 x e^-theta x equiv f(x | theta) = e^-theta x + 2 log theta + x
                    $$
                    respectively. By mixing these distribution using the weights $theta(1 + theta)^-1$ and $(1 + theta)^-1$, respectively, we will get the PDF of the Lindley distribution that is given by
                    $$
                    f(x | theta) = theta^2 (1 + theta)^-1 (1 + x) e^-theta x equiv f(x | theta) = e^-theta x + log leftlbracetheta^2 (1 + theta)rightrbrace + log(1 + x)
                    $$



                    My personal verdict is that there might be "a few" special cases when a mixture of $f$s, where $f$ is a distribution of the exponential family, will yield a distribution that belongs to the exponential family.







                    share|cite|improve this answer












                    share|cite|improve this answer



                    share|cite|improve this answer










                    answered May 2 '16 at 1:25









                    FALAM

                    30916




                    30916











                    • thanks @FALAM. actually your example is not an contraction to my claim. I don't ask about mixing a gamma with an exponential but for example mixing two gammas or two exponentials. (all of the component measures are from a same distribution). BTW your analysis is interesting although not fit to the question exactly.
                      – CoderInNetwork
                      May 8 '16 at 7:37










                    • Thanks for your reply @CoderInNetwork. Actually, the exponential distribution is a gamma distribution with shape parameter equals to 1. Hence, the Lindley distribution is in fact the probability distribution of a mixture of two gamma random variables with similar rate parameters and different shape parameters.
                      – FALAM
                      May 21 '16 at 14:18
















                    • thanks @FALAM. actually your example is not an contraction to my claim. I don't ask about mixing a gamma with an exponential but for example mixing two gammas or two exponentials. (all of the component measures are from a same distribution). BTW your analysis is interesting although not fit to the question exactly.
                      – CoderInNetwork
                      May 8 '16 at 7:37










                    • Thanks for your reply @CoderInNetwork. Actually, the exponential distribution is a gamma distribution with shape parameter equals to 1. Hence, the Lindley distribution is in fact the probability distribution of a mixture of two gamma random variables with similar rate parameters and different shape parameters.
                      – FALAM
                      May 21 '16 at 14:18















                    thanks @FALAM. actually your example is not an contraction to my claim. I don't ask about mixing a gamma with an exponential but for example mixing two gammas or two exponentials. (all of the component measures are from a same distribution). BTW your analysis is interesting although not fit to the question exactly.
                    – CoderInNetwork
                    May 8 '16 at 7:37




                    thanks @FALAM. actually your example is not an contraction to my claim. I don't ask about mixing a gamma with an exponential but for example mixing two gammas or two exponentials. (all of the component measures are from a same distribution). BTW your analysis is interesting although not fit to the question exactly.
                    – CoderInNetwork
                    May 8 '16 at 7:37












                    Thanks for your reply @CoderInNetwork. Actually, the exponential distribution is a gamma distribution with shape parameter equals to 1. Hence, the Lindley distribution is in fact the probability distribution of a mixture of two gamma random variables with similar rate parameters and different shape parameters.
                    – FALAM
                    May 21 '16 at 14:18




                    Thanks for your reply @CoderInNetwork. Actually, the exponential distribution is a gamma distribution with shape parameter equals to 1. Hence, the Lindley distribution is in fact the probability distribution of a mixture of two gamma random variables with similar rate parameters and different shape parameters.
                    – FALAM
                    May 21 '16 at 14:18












                     

                    draft saved


                    draft discarded


























                     


                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function ()
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f1764166%2fis-the-mixture-of-exponential-family-distributions-an-exponential-family-distrib%23new-answer', 'question_page');

                    );

                    Post as a guest













































































                    這個網誌中的熱門文章

                    How to combine Bézier curves to a surface?

                    Mutual Information Always Non-negative

                    Why am i infinitely getting the same tweet with the Twitter Search API?