A discrete time Markov process is a random variable $X:\Omega\to S^\infty$ on a filtered probability space $(\Omega, \mathcal{F}, \{\mathcal{F}_n\}_{n=1}^\infty, P)$such that $P(X_{j+1}\in H | \mathcal{F}_n)=P(X_{j+1} \in H | \sigma(X_j))$ where $S$ is a countable standard Borel topological space with the Borel $\sigma$-algebra.
We may define a discrete-time Markov process via a Markov transition matrix (kernel function) $K \in \mathbb{R}^{|S|\times |S|}$. That is,
$$K(a,b)=K_{a,b} =P(X_{j+1} =a | X_{j+1}=b) \quad \forall a,b \in S.$$
From this Markov transition matrix, we may obtain a Markov kernel that is, a map
$\kappa:S\times \mathcal{A}\to \mathbb{R}$ such that
- For every $x \in S, \kappa(x, \cdot)$ is a probability measure.
- For every $A \in \mathscr{A}, \kappa(\cdot, A)$ is measurable.
for $B\subset S, a\in S$, $\kappa(B,a)=\sum_{b\in B}K_{a,b}$.
The kernel function defines a linear operator $T:L^1(S)\to L^1(S)$ as
$$Tf(x) = \int_SK(x,y)f(y)\nu(dy)=\sum_{y\in S}K(x,y)f(y)$$
where $\nu$ is the counting measure.
This Markov kernel is the kernel of an integral operator. What is the integral operator doing to these functions in terms of a stochastic process? I only know of operators in the context of "the structure presesrving mappings between Banach Spaces" and it is suprising that we can characterize a markov process by kernel which in turn defines an operator. To me, it seems like the operator is the interesting structure, and we just access it using the kernel, but I'm not sure what its use is.
Is there maybe some interesting way of viewing this in terms of the theory of distributions or functional analysis?
Best Answer
To summarize my comments as an answer: If we let $X(t) \in S$ be the state of the DTMC then your $T(\cdot)$ operator takes a potential PMF for $X(t)$ as the input (at some time $t$) and gives output equal to the resulting PMF for $X(t+1)$. It is the same as the equation $$ P[X(t+1)=j]=\sum_{i \in S} P[X(t)=i]P_{ij} \quad \forall j \in S$$ where $P_{ij} = K(j,i)$ is the transition probability from $i$ to $j$.
For CTMC: If you had a CTMC $X(t) \in S$ then you could look at the embedded DTMC $\{X(t_k)\}_{k=0}^{\infty}$ where $t_0=0$ and $t_k$ is the time of the $k$th transition.
Alternatively, you could define a function $$K(a,b,\tau) = P[X(t+\tau)=a|X(t)=b] \quad \forall a,b\in S, \tau>0$$ Then your $T(\cdot)$ can be modified to $T_{\tau}(\cdot)$ to specify how much time $\tau>0$ you are considering.