Typically when we measure information/Shannon entropy (differential entropy), it is a snapshot value at a specific point in time, so there is no talk about whether it will grow or decrease. A random variable just possesses that amount of entropy like the number of ears on a dog.
If we reconcile Shannon entropy with time entropy, which instead dynamically grows larger as $t\rightarrow \infty$, from the perspective that we as humans only experience time going forwards not backwards,
in what situations would we find informational disorder/transmission (information entropy) increase or decrease, given that Shannon entropy is commonly viewed as a non-dynamic snapshot value as if it were just a random variable's intrinsic property that doesn't change?
Best Answer
Like you said, entropy is a property of a random variable's distribution. So, if you want a time-variant entropy, you need to have some time-variance in your distribution, too.
One way to look at this is repeated experiments. For example, the entropy of a coin flip doesn't change over time, but you can look at the entropy of a sequence of coin-flips, which is a growing binary sequence, that changes over time. The entropy of this sequence changes over time as the sequence grows. You actually end up with very interesting phenomena in this case, a concentration of measure around what is called a typical set that contains equally likely outcomes (see asymptotic equipartition property if you're curious) and the probability of all outcomes outside this set vanishes.
Another way to have a time-variant entropy is having a distribution that evolves with time. You can define a family of random variables that is parametrized by the time index. As a simple example, let $X_t \sim \text{Uniform}(\{1/k\}_{k=1}^t)$. So, the set of values $X_t$ can take grows over time and hence entropy keeps increasing.
However, note that, the entropy doesn't always have to increase over time. You can conversely define a family, where the uncertainty decreases over time. For example, $X_t \sim \text{Uniform}([0,1/t])$ gives you $H(X_t) = \log (1/t)$.