Stationary distribution of DTMC with infinite state space
Clash Royale CLAN TAG#URR8PPP
up vote
0
down vote
favorite
I am solving the stationary distirbution of a Discrete time Markov Chain with infinite state space. The state space is $pi^H_0,pi^H_1,...,pi^L_0,pi^L_1,...$. The transition matrix has structure. I eventually reduce the system linear equations into the following two series. But feel no clue about how to start to solve them....
beginalign*
&begincases
&pi^H_0=api^H_1\
&pi^H_1=api^H_0+api^H_2+cpi^L_0\
&pi^H_n=bpi^H_n-1+api^H_n+1,~forall~ngeq 2
endcases~~
begincases
&pi^L_0=cpi^L_1\
&pi^L_1=bpi^H_0+dpi^L_0+cpi^L_2\
&pi^L_n=dpi^L_n-1+cpi^L_n+1,~forall~ngeq 2
endcases\
&sum_n=0^infty(pi^H_n+pi^L_n)=1\
&a+b=1,~c+d=1,~a,b,c,d~mboxare all positive constant pi^H_n_n=0^infty~mboxand pi^L_n_n=0^infty~mboxare all non-negative variables
endalign*
markov-chains markov-process
asked Sep 9 at 8:30
KevinKim
7613
add a comment |Â
up vote
0
down vote
favorite
I am solving the stationary distirbution of a Discrete time Markov Chain with infinite state space. The state space is $pi^H_0,pi^H_1,...,pi^L_0,pi^L_1,...$. The transition matrix has structure. I eventually reduce the system linear equations into the following two series. But feel no clue about how to start to solve them....
beginalign*
&begincases
&pi^H_0=api^H_1\
&pi^H_1=api^H_0+api^H_2+cpi^L_0\
&pi^H_n=bpi^H_n-1+api^H_n+1,~forall~ngeq 2
endcases~~
begincases
&pi^L_0=cpi^L_1\
&pi^L_1=bpi^H_0+dpi^L_0+cpi^L_2\
&pi^L_n=dpi^L_n-1+cpi^L_n+1,~forall~ngeq 2
endcases\
&sum_n=0^infty(pi^H_n+pi^L_n)=1\
&a+b=1,~c+d=1,~a,b,c,d~mboxare all positive constant pi^H_n_n=0^infty~mboxand pi^L_n_n=0^infty~mboxare all non-negative variables
endalign*
markov-chains markov-process
asked Sep 9 at 8:30
KevinKim
7613
"The state space is $pi^H_0,pi^H_1,... ,pi^L_0,pi^L_1, ...$." You might mean that the state space is $$H,Ltimes0,1,2,ldots$$
– Did
Sep 9 at 9:41
@Did Yes, sorry for my bad notation
– KevinKim
Sep 9 at 10:12
add a comment |Â
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I am solving the stationary distirbution of a Discrete time Markov Chain with infinite state space. The state space is $pi^H_0,pi^H_1,...,pi^L_0,pi^L_1,...$. The transition matrix has structure. I eventually reduce the system linear equations into the following two series. But feel no clue about how to start to solve them....
beginalign*
&begincases
&pi^H_0=api^H_1\
&pi^H_1=api^H_0+api^H_2+cpi^L_0\
&pi^H_n=bpi^H_n-1+api^H_n+1,~forall~ngeq 2
endcases~~
begincases
&pi^L_0=cpi^L_1\
&pi^L_1=bpi^H_0+dpi^L_0+cpi^L_2\
&pi^L_n=dpi^L_n-1+cpi^L_n+1,~forall~ngeq 2
endcases\
&sum_n=0^infty(pi^H_n+pi^L_n)=1\
&a+b=1,~c+d=1,~a,b,c,d~mboxare all positive constant pi^H_n_n=0^infty~mboxand pi^L_n_n=0^infty~mboxare all non-negative variables
endalign*
markov-chains markov-process
asked Sep 9 at 8:30
KevinKim
7613
I am solving the stationary distirbution of a Discrete time Markov Chain with infinite state space. The state space is $pi^H_0,pi^H_1,...,pi^L_0,pi^L_1,...$. The transition matrix has structure. I eventually reduce the system linear equations into the following two series. But feel no clue about how to start to solve them....
beginalign*
&begincases
&pi^H_0=api^H_1\
&pi^H_1=api^H_0+api^H_2+cpi^L_0\
&pi^H_n=bpi^H_n-1+api^H_n+1,~forall~ngeq 2
endcases~~
begincases
&pi^L_0=cpi^L_1\
&pi^L_1=bpi^H_0+dpi^L_0+cpi^L_2\
&pi^L_n=dpi^L_n-1+cpi^L_n+1,~forall~ngeq 2
endcases\
&sum_n=0^infty(pi^H_n+pi^L_n)=1\
&a+b=1,~c+d=1,~a,b,c,d~mboxare all positive constant pi^H_n_n=0^infty~mboxand pi^L_n_n=0^infty~mboxare all non-negative variables
endalign*
markov-chains markov-process
markov-chains markov-process
asked Sep 9 at 8:30
KevinKim
7613
asked Sep 9 at 8:30
KevinKim
7613
asked Sep 9 at 8:30
KevinKim
7613
asked Sep 9 at 8:30
KevinKim
7613
asked Sep 9 at 8:30
KevinKim
7613
7613
"The state space is $pi^H_0,pi^H_1,... ,pi^L_0,pi^L_1, ...$." You might mean that the state space is $$H,Ltimes0,1,2,ldots$$
– Did
Sep 9 at 9:41
@Did Yes, sorry for my bad notation
– KevinKim
Sep 9 at 10:12
add a comment |Â
"The state space is $pi^H_0,pi^H_1,... ,pi^L_0,pi^L_1, ...$." You might mean that the state space is $$H,Ltimes0,1,2,ldots$$
– Did
Sep 9 at 9:41
@Did Yes, sorry for my bad notation
– KevinKim
Sep 9 at 10:12
"The state space is $pi^H_0,pi^H_1,... ,pi^L_0,pi^L_1, ...$." You might mean that the state space is $$H,Ltimes0,1,2,ldots$$
– Did
Sep 9 at 9:41
"The state space is $pi^H_0,pi^H_1,... ,pi^L_0,pi^L_1, ...$." You might mean that the state space is $$H,Ltimes0,1,2,ldots$$
– Did
Sep 9 at 9:41
@Did Yes, sorry for my bad notation
– KevinKim
Sep 9 at 10:12
@Did Yes, sorry for my bad notation
– KevinKim
Sep 9 at 10:12
add a comment |Â
1 Answer
1
active
oldest
votes
up vote
1
down vote
The recurrences in the third lines aren't coupled, so they can be solved with the standard ansatz $pi_k^H=lambda^k$:
$$
lambda^n=blambda^n-1+alambda^n+1;,
$$
which yields the characteristic equation
$$
alambda^2-lambda+b=0
$$
(and likewise for $pi^L$). This yields two linearly independent solutions each, for a total of $4$ free coefficients. The first two lines allow you to express the values for $n=1,2$ in terms of the initial values for $n=0$, yielding one condition each. That leaves two degrees of freedom, and normalization reduces this to one. This might be eliminated if one of the characteristic values has magnitude greater than $1$ and thus isn't suitable for representing probabilities.
answered Sep 9 at 8:55
joriki
169k10181337
add a comment |Â
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
The recurrences in the third lines aren't coupled, so they can be solved with the standard ansatz $pi_k^H=lambda^k$:
$$
lambda^n=blambda^n-1+alambda^n+1;,
$$
which yields the characteristic equation
$$
alambda^2-lambda+b=0
$$
(and likewise for $pi^L$). This yields two linearly independent solutions each, for a total of $4$ free coefficients. The first two lines allow you to express the values for $n=1,2$ in terms of the initial values for $n=0$, yielding one condition each. That leaves two degrees of freedom, and normalization reduces this to one. This might be eliminated if one of the characteristic values has magnitude greater than $1$ and thus isn't suitable for representing probabilities.
answered Sep 9 at 8:55
joriki
169k10181337
add a comment |Â
up vote
1
down vote
The recurrences in the third lines aren't coupled, so they can be solved with the standard ansatz $pi_k^H=lambda^k$:
$$
lambda^n=blambda^n-1+alambda^n+1;,
$$
which yields the characteristic equation
$$
alambda^2-lambda+b=0
$$
(and likewise for $pi^L$). This yields two linearly independent solutions each, for a total of $4$ free coefficients. The first two lines allow you to express the values for $n=1,2$ in terms of the initial values for $n=0$, yielding one condition each. That leaves two degrees of freedom, and normalization reduces this to one. This might be eliminated if one of the characteristic values has magnitude greater than $1$ and thus isn't suitable for representing probabilities.
answered Sep 9 at 8:55
joriki
169k10181337
add a comment |Â
up vote
1
down vote
up vote
1
down vote
The recurrences in the third lines aren't coupled, so they can be solved with the standard ansatz $pi_k^H=lambda^k$:
$$
lambda^n=blambda^n-1+alambda^n+1;,
$$
which yields the characteristic equation
$$
alambda^2-lambda+b=0
$$
(and likewise for $pi^L$). This yields two linearly independent solutions each, for a total of $4$ free coefficients. The first two lines allow you to express the values for $n=1,2$ in terms of the initial values for $n=0$, yielding one condition each. That leaves two degrees of freedom, and normalization reduces this to one. This might be eliminated if one of the characteristic values has magnitude greater than $1$ and thus isn't suitable for representing probabilities.
answered Sep 9 at 8:55
joriki
169k10181337
The recurrences in the third lines aren't coupled, so they can be solved with the standard ansatz $pi_k^H=lambda^k$:
$$
lambda^n=blambda^n-1+alambda^n+1;,
$$
which yields the characteristic equation
$$
alambda^2-lambda+b=0
$$
(and likewise for $pi^L$). This yields two linearly independent solutions each, for a total of $4$ free coefficients. The first two lines allow you to express the values for $n=1,2$ in terms of the initial values for $n=0$, yielding one condition each. That leaves two degrees of freedom, and normalization reduces this to one. This might be eliminated if one of the characteristic values has magnitude greater than $1$ and thus isn't suitable for representing probabilities.
answered Sep 9 at 8:55
joriki
169k10181337
answered Sep 9 at 8:55
joriki
169k10181337
answered Sep 9 at 8:55
joriki
169k10181337
answered Sep 9 at 8:55
joriki
169k10181337
169k10181337
add a comment |Â
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2910546%2fstationary-distribution-of-dtmc-with-infinite-state-space%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
"The state space is $pi^H_0,pi^H_1,... ,pi^L_0,pi^L_1, ...$." You might mean that the state space is $$H,Ltimes0,1,2,ldots$$
– Did
Sep 9 at 9:41
@Did Yes, sorry for my bad notation
– KevinKim
Sep 9 at 10:12