Is the mixture of Exponential family distributions an Exponential family distribution too?

Multi tool use
Multi tool use

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
1
down vote

favorite












Consider we have a mixture of multinomials or in a broader sense, a mixture of $f$s where $f$ is an distribution of exponential family type and the membership components are known with the sum of 1. Is the new distribution an exponential family too?







share|cite|improve this question
























    up vote
    1
    down vote

    favorite












    Consider we have a mixture of multinomials or in a broader sense, a mixture of $f$s where $f$ is an distribution of exponential family type and the membership components are known with the sum of 1. Is the new distribution an exponential family too?







    share|cite|improve this question






















      up vote
      1
      down vote

      favorite









      up vote
      1
      down vote

      favorite











      Consider we have a mixture of multinomials or in a broader sense, a mixture of $f$s where $f$ is an distribution of exponential family type and the membership components are known with the sum of 1. Is the new distribution an exponential family too?







      share|cite|improve this question












      Consider we have a mixture of multinomials or in a broader sense, a mixture of $f$s where $f$ is an distribution of exponential family type and the membership components are known with the sum of 1. Is the new distribution an exponential family too?









      share|cite|improve this question











      share|cite|improve this question




      share|cite|improve this question










      asked Apr 29 '16 at 14:35









      CoderInNetwork

      539




      539




















          2 Answers
          2






          active

          oldest

          votes

















          up vote
          1
          down vote













          In general the answer is "no".



          The mixture distribution
          $$
          alpha h_1(x)exp^theta_1^T phi_1(x) - A_1(theta_1) + (1-alpha) h_2(x)exp^theta_2^T phi_2(x) - A_2(theta_2)
          $$
          generally can't be factored into something like
          $$
          h_3(x) exp^theta_3^T phi_3(x) - A_3(theta_3)
          $$
          even if the two components come from the same exponential family (so $h_1=h_2, phi_1=phi_2, A_1=A_2$). A classic example is a mixture of Gaussians.




          Although the product of exponential families is an (unnormalized)
          exponential family, the mixture of exponential families is not an
          exponential family.




          --page 6 of Statistical exponential families: A digest with flash cards






          share|cite|improve this answer





























            up vote
            -1
            down vote













            In my viewpoint, this depends on the nature of the mixed distributions. Without loss of generality, consider a probability distribution with a single rate parameter $theta$. The latter probability distribution belongs to the exponential family if its PDF can be expressed in the form
            $$
            f(x | theta) = e^eta(theta)T(x)-A(theta)+B(x)
            $$
            Now, consider an exponential distribution with rate parameter $theta$ and a gamma distribution with known shape parameter $2$ and rate parameter $theta$ with corresponding PDFs
            $$
            f(x | theta) = theta e^-theta x equiv f(x | theta) = e^-theta x + log theta + 0
            $$
            and
            $$
            f(x | theta) = theta^2 x e^-theta x equiv f(x | theta) = e^-theta x + 2 log theta + x
            $$
            respectively. By mixing these distribution using the weights $theta(1 + theta)^-1$ and $(1 + theta)^-1$, respectively, we will get the PDF of the Lindley distribution that is given by
            $$
            f(x | theta) = theta^2 (1 + theta)^-1 (1 + x) e^-theta x equiv f(x | theta) = e^-theta x + log leftlbracetheta^2 (1 + theta)rightrbrace + log(1 + x)
            $$



            My personal verdict is that there might be "a few" special cases when a mixture of $f$s, where $f$ is a distribution of the exponential family, will yield a distribution that belongs to the exponential family.






            share|cite|improve this answer




















            • thanks @FALAM. actually your example is not an contraction to my claim. I don't ask about mixing a gamma with an exponential but for example mixing two gammas or two exponentials. (all of the component measures are from a same distribution). BTW your analysis is interesting although not fit to the question exactly.
              – CoderInNetwork
              May 8 '16 at 7:37










            • Thanks for your reply @CoderInNetwork. Actually, the exponential distribution is a gamma distribution with shape parameter equals to 1. Hence, the Lindley distribution is in fact the probability distribution of a mixture of two gamma random variables with similar rate parameters and different shape parameters.
              – FALAM
              May 21 '16 at 14:18










            Your Answer




            StackExchange.ifUsing("editor", function ()
            return StackExchange.using("mathjaxEditing", function ()
            StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
            StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
            );
            );
            , "mathjax-editing");

            StackExchange.ready(function()
            var channelOptions =
            tags: "".split(" "),
            id: "69"
            ;
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function()
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled)
            StackExchange.using("snippets", function()
            createEditor();
            );

            else
            createEditor();

            );

            function createEditor()
            StackExchange.prepareEditor(
            heartbeatType: 'answer',
            convertImagesToLinks: true,
            noModals: false,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            noCode: true, onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            );



            );








             

            draft saved


            draft discarded


















            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f1764166%2fis-the-mixture-of-exponential-family-distributions-an-exponential-family-distrib%23new-answer', 'question_page');

            );

            Post as a guest






























            2 Answers
            2






            active

            oldest

            votes








            2 Answers
            2






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes








            up vote
            1
            down vote













            In general the answer is "no".



            The mixture distribution
            $$
            alpha h_1(x)exp^theta_1^T phi_1(x) - A_1(theta_1) + (1-alpha) h_2(x)exp^theta_2^T phi_2(x) - A_2(theta_2)
            $$
            generally can't be factored into something like
            $$
            h_3(x) exp^theta_3^T phi_3(x) - A_3(theta_3)
            $$
            even if the two components come from the same exponential family (so $h_1=h_2, phi_1=phi_2, A_1=A_2$). A classic example is a mixture of Gaussians.




            Although the product of exponential families is an (unnormalized)
            exponential family, the mixture of exponential families is not an
            exponential family.




            --page 6 of Statistical exponential families: A digest with flash cards






            share|cite|improve this answer


























              up vote
              1
              down vote













              In general the answer is "no".



              The mixture distribution
              $$
              alpha h_1(x)exp^theta_1^T phi_1(x) - A_1(theta_1) + (1-alpha) h_2(x)exp^theta_2^T phi_2(x) - A_2(theta_2)
              $$
              generally can't be factored into something like
              $$
              h_3(x) exp^theta_3^T phi_3(x) - A_3(theta_3)
              $$
              even if the two components come from the same exponential family (so $h_1=h_2, phi_1=phi_2, A_1=A_2$). A classic example is a mixture of Gaussians.




              Although the product of exponential families is an (unnormalized)
              exponential family, the mixture of exponential families is not an
              exponential family.




              --page 6 of Statistical exponential families: A digest with flash cards






              share|cite|improve this answer
























                up vote
                1
                down vote










                up vote
                1
                down vote









                In general the answer is "no".



                The mixture distribution
                $$
                alpha h_1(x)exp^theta_1^T phi_1(x) - A_1(theta_1) + (1-alpha) h_2(x)exp^theta_2^T phi_2(x) - A_2(theta_2)
                $$
                generally can't be factored into something like
                $$
                h_3(x) exp^theta_3^T phi_3(x) - A_3(theta_3)
                $$
                even if the two components come from the same exponential family (so $h_1=h_2, phi_1=phi_2, A_1=A_2$). A classic example is a mixture of Gaussians.




                Although the product of exponential families is an (unnormalized)
                exponential family, the mixture of exponential families is not an
                exponential family.




                --page 6 of Statistical exponential families: A digest with flash cards






                share|cite|improve this answer














                In general the answer is "no".



                The mixture distribution
                $$
                alpha h_1(x)exp^theta_1^T phi_1(x) - A_1(theta_1) + (1-alpha) h_2(x)exp^theta_2^T phi_2(x) - A_2(theta_2)
                $$
                generally can't be factored into something like
                $$
                h_3(x) exp^theta_3^T phi_3(x) - A_3(theta_3)
                $$
                even if the two components come from the same exponential family (so $h_1=h_2, phi_1=phi_2, A_1=A_2$). A classic example is a mixture of Gaussians.




                Although the product of exponential families is an (unnormalized)
                exponential family, the mixture of exponential families is not an
                exponential family.




                --page 6 of Statistical exponential families: A digest with flash cards







                share|cite|improve this answer














                share|cite|improve this answer



                share|cite|improve this answer








                edited Aug 18 at 15:10

























                answered Jan 26 '17 at 17:11









                Yibo Yang

                470316




                470316




















                    up vote
                    -1
                    down vote













                    In my viewpoint, this depends on the nature of the mixed distributions. Without loss of generality, consider a probability distribution with a single rate parameter $theta$. The latter probability distribution belongs to the exponential family if its PDF can be expressed in the form
                    $$
                    f(x | theta) = e^eta(theta)T(x)-A(theta)+B(x)
                    $$
                    Now, consider an exponential distribution with rate parameter $theta$ and a gamma distribution with known shape parameter $2$ and rate parameter $theta$ with corresponding PDFs
                    $$
                    f(x | theta) = theta e^-theta x equiv f(x | theta) = e^-theta x + log theta + 0
                    $$
                    and
                    $$
                    f(x | theta) = theta^2 x e^-theta x equiv f(x | theta) = e^-theta x + 2 log theta + x
                    $$
                    respectively. By mixing these distribution using the weights $theta(1 + theta)^-1$ and $(1 + theta)^-1$, respectively, we will get the PDF of the Lindley distribution that is given by
                    $$
                    f(x | theta) = theta^2 (1 + theta)^-1 (1 + x) e^-theta x equiv f(x | theta) = e^-theta x + log leftlbracetheta^2 (1 + theta)rightrbrace + log(1 + x)
                    $$



                    My personal verdict is that there might be "a few" special cases when a mixture of $f$s, where $f$ is a distribution of the exponential family, will yield a distribution that belongs to the exponential family.






                    share|cite|improve this answer




















                    • thanks @FALAM. actually your example is not an contraction to my claim. I don't ask about mixing a gamma with an exponential but for example mixing two gammas or two exponentials. (all of the component measures are from a same distribution). BTW your analysis is interesting although not fit to the question exactly.
                      – CoderInNetwork
                      May 8 '16 at 7:37










                    • Thanks for your reply @CoderInNetwork. Actually, the exponential distribution is a gamma distribution with shape parameter equals to 1. Hence, the Lindley distribution is in fact the probability distribution of a mixture of two gamma random variables with similar rate parameters and different shape parameters.
                      – FALAM
                      May 21 '16 at 14:18














                    up vote
                    -1
                    down vote













                    In my viewpoint, this depends on the nature of the mixed distributions. Without loss of generality, consider a probability distribution with a single rate parameter $theta$. The latter probability distribution belongs to the exponential family if its PDF can be expressed in the form
                    $$
                    f(x | theta) = e^eta(theta)T(x)-A(theta)+B(x)
                    $$
                    Now, consider an exponential distribution with rate parameter $theta$ and a gamma distribution with known shape parameter $2$ and rate parameter $theta$ with corresponding PDFs
                    $$
                    f(x | theta) = theta e^-theta x equiv f(x | theta) = e^-theta x + log theta + 0
                    $$
                    and
                    $$
                    f(x | theta) = theta^2 x e^-theta x equiv f(x | theta) = e^-theta x + 2 log theta + x
                    $$
                    respectively. By mixing these distribution using the weights $theta(1 + theta)^-1$ and $(1 + theta)^-1$, respectively, we will get the PDF of the Lindley distribution that is given by
                    $$
                    f(x | theta) = theta^2 (1 + theta)^-1 (1 + x) e^-theta x equiv f(x | theta) = e^-theta x + log leftlbracetheta^2 (1 + theta)rightrbrace + log(1 + x)
                    $$



                    My personal verdict is that there might be "a few" special cases when a mixture of $f$s, where $f$ is a distribution of the exponential family, will yield a distribution that belongs to the exponential family.






                    share|cite|improve this answer




















                    • thanks @FALAM. actually your example is not an contraction to my claim. I don't ask about mixing a gamma with an exponential but for example mixing two gammas or two exponentials. (all of the component measures are from a same distribution). BTW your analysis is interesting although not fit to the question exactly.
                      – CoderInNetwork
                      May 8 '16 at 7:37










                    • Thanks for your reply @CoderInNetwork. Actually, the exponential distribution is a gamma distribution with shape parameter equals to 1. Hence, the Lindley distribution is in fact the probability distribution of a mixture of two gamma random variables with similar rate parameters and different shape parameters.
                      – FALAM
                      May 21 '16 at 14:18












                    up vote
                    -1
                    down vote










                    up vote
                    -1
                    down vote









                    In my viewpoint, this depends on the nature of the mixed distributions. Without loss of generality, consider a probability distribution with a single rate parameter $theta$. The latter probability distribution belongs to the exponential family if its PDF can be expressed in the form
                    $$
                    f(x | theta) = e^eta(theta)T(x)-A(theta)+B(x)
                    $$
                    Now, consider an exponential distribution with rate parameter $theta$ and a gamma distribution with known shape parameter $2$ and rate parameter $theta$ with corresponding PDFs
                    $$
                    f(x | theta) = theta e^-theta x equiv f(x | theta) = e^-theta x + log theta + 0
                    $$
                    and
                    $$
                    f(x | theta) = theta^2 x e^-theta x equiv f(x | theta) = e^-theta x + 2 log theta + x
                    $$
                    respectively. By mixing these distribution using the weights $theta(1 + theta)^-1$ and $(1 + theta)^-1$, respectively, we will get the PDF of the Lindley distribution that is given by
                    $$
                    f(x | theta) = theta^2 (1 + theta)^-1 (1 + x) e^-theta x equiv f(x | theta) = e^-theta x + log leftlbracetheta^2 (1 + theta)rightrbrace + log(1 + x)
                    $$



                    My personal verdict is that there might be "a few" special cases when a mixture of $f$s, where $f$ is a distribution of the exponential family, will yield a distribution that belongs to the exponential family.






                    share|cite|improve this answer












                    In my viewpoint, this depends on the nature of the mixed distributions. Without loss of generality, consider a probability distribution with a single rate parameter $theta$. The latter probability distribution belongs to the exponential family if its PDF can be expressed in the form
                    $$
                    f(x | theta) = e^eta(theta)T(x)-A(theta)+B(x)
                    $$
                    Now, consider an exponential distribution with rate parameter $theta$ and a gamma distribution with known shape parameter $2$ and rate parameter $theta$ with corresponding PDFs
                    $$
                    f(x | theta) = theta e^-theta x equiv f(x | theta) = e^-theta x + log theta + 0
                    $$
                    and
                    $$
                    f(x | theta) = theta^2 x e^-theta x equiv f(x | theta) = e^-theta x + 2 log theta + x
                    $$
                    respectively. By mixing these distribution using the weights $theta(1 + theta)^-1$ and $(1 + theta)^-1$, respectively, we will get the PDF of the Lindley distribution that is given by
                    $$
                    f(x | theta) = theta^2 (1 + theta)^-1 (1 + x) e^-theta x equiv f(x | theta) = e^-theta x + log leftlbracetheta^2 (1 + theta)rightrbrace + log(1 + x)
                    $$



                    My personal verdict is that there might be "a few" special cases when a mixture of $f$s, where $f$ is a distribution of the exponential family, will yield a distribution that belongs to the exponential family.







                    share|cite|improve this answer












                    share|cite|improve this answer



                    share|cite|improve this answer










                    answered May 2 '16 at 1:25









                    FALAM

                    30916




                    30916











                    • thanks @FALAM. actually your example is not an contraction to my claim. I don't ask about mixing a gamma with an exponential but for example mixing two gammas or two exponentials. (all of the component measures are from a same distribution). BTW your analysis is interesting although not fit to the question exactly.
                      – CoderInNetwork
                      May 8 '16 at 7:37










                    • Thanks for your reply @CoderInNetwork. Actually, the exponential distribution is a gamma distribution with shape parameter equals to 1. Hence, the Lindley distribution is in fact the probability distribution of a mixture of two gamma random variables with similar rate parameters and different shape parameters.
                      – FALAM
                      May 21 '16 at 14:18
















                    • thanks @FALAM. actually your example is not an contraction to my claim. I don't ask about mixing a gamma with an exponential but for example mixing two gammas or two exponentials. (all of the component measures are from a same distribution). BTW your analysis is interesting although not fit to the question exactly.
                      – CoderInNetwork
                      May 8 '16 at 7:37










                    • Thanks for your reply @CoderInNetwork. Actually, the exponential distribution is a gamma distribution with shape parameter equals to 1. Hence, the Lindley distribution is in fact the probability distribution of a mixture of two gamma random variables with similar rate parameters and different shape parameters.
                      – FALAM
                      May 21 '16 at 14:18















                    thanks @FALAM. actually your example is not an contraction to my claim. I don't ask about mixing a gamma with an exponential but for example mixing two gammas or two exponentials. (all of the component measures are from a same distribution). BTW your analysis is interesting although not fit to the question exactly.
                    – CoderInNetwork
                    May 8 '16 at 7:37




                    thanks @FALAM. actually your example is not an contraction to my claim. I don't ask about mixing a gamma with an exponential but for example mixing two gammas or two exponentials. (all of the component measures are from a same distribution). BTW your analysis is interesting although not fit to the question exactly.
                    – CoderInNetwork
                    May 8 '16 at 7:37












                    Thanks for your reply @CoderInNetwork. Actually, the exponential distribution is a gamma distribution with shape parameter equals to 1. Hence, the Lindley distribution is in fact the probability distribution of a mixture of two gamma random variables with similar rate parameters and different shape parameters.
                    – FALAM
                    May 21 '16 at 14:18




                    Thanks for your reply @CoderInNetwork. Actually, the exponential distribution is a gamma distribution with shape parameter equals to 1. Hence, the Lindley distribution is in fact the probability distribution of a mixture of two gamma random variables with similar rate parameters and different shape parameters.
                    – FALAM
                    May 21 '16 at 14:18












                     

                    draft saved


                    draft discarded


























                     


                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function ()
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f1764166%2fis-the-mixture-of-exponential-family-distributions-an-exponential-family-distrib%23new-answer', 'question_page');

                    );

                    Post as a guest













































































                    GNZkcDlGrCNSB4ir yopx1wl9,vnK ka0Rc1ScE1LC,ZTrkaHamqSHgn7j,NzCQWL s,K2runpC6PVVIcIdXegQsJbEfKZuM iV8hp
                    CbPUSmgqDd,Pu29yG3shvBnttwd1awERavmc7unlk6bkQ,csx8VhVnRC

                    這個網誌中的熱門文章

                    How to combine Bézier curves to a surface?

                    Propositional logic and tautologies

                    Distribution of Stopped Wiener Process with Stochastic Volatility