It's useful to consider finite-state Markov chains with states $\{ 1, \ldots, N \}$. Such a Markov chain is defined by its transitions matrix $P = (P_{ij})_{i,j=1}^N$. We require that $0 \leq P_{ij} \leq 1$ for each $i, j = 1, \ldots, N$ and that $\sum_{j=1}^N P_{ij} = 1$. Thus, we can think of $P_{ij}$ as the probability of jumping from state $i$ to state $j$. We initialize the Markov chain in a state $X_0$ and let $X_n$ be the state at time $n$ (so $X_n$ is a random variable in $\{ 1, \ldots, N \}$).
A natural requirement is that the Markov chain be irreducible, which essentially means that we can get from any state to any other state with positive probability.
A finite-state Markov chain is said to be ergodic if it is irreducible and has an additional property called aperiodicity. The ergodic theorem for Markov chains says (roughly) that an ergodic Markov chain approaches its "stationary distribution" (see the previous link) as time $n \to \infty$.
Now in the case of physical systems, an additional assumption is usually that the system be reversible. It turns out that the stationary distribution of a finite-state irreducible reversible Markov chain is the uniform distribution, which assigns equal probability $1/N$ to each of the possible states.
Putting all this together, we see that a finite-state reversible ergodic Markov chain converges to the uniform distribution (i.e. reaches an equilibrium as time goes to infinity in which all states are equally likely).
The notion of ergodic dynamical system you asked about is a vast generalization of this idea.
Your question is intuitive, so I will try to answer through one very intuitive example.
Example:
Dynamics:
You have some money, say 100$, and we're playing a coin game with a fair coin. Each time the coin lands heads (H) we take your money and multiply them by 1.5 and every time the coin lands tails (T) we take your money and multiply them by 0.6.
Averages:
Let W(t) denote your wealth at time t, and let this process run for a finite time and take the average over your wealth for that time. This is the finite time-average:
$\left\langle W(t)\right\rangle _{T} = \frac{1}{T}\sum_{t=0}^{T}W(t) $
In contrast, assume that we have N of these processes that we let run until t and then take the average over N. This is the finite ensemble-average of the observable N wealth processes.
$\left\langle W(t)\right\rangle _{N} = \frac{1}{N}\sum_{i=1}^{N}W_i(t)$
Where i denotes the ith of N processes and the average is taken at time t. Letting $T\rightarrow\infty$ and $N\rightarrow\infty$ you get the ensemble average (or expectation operator) and the time average.
Now that we have introduced the relevant dynamics (the coin game) and the averages, lets restate your definition of ergodicity - namely that the time average equals the ensemble average.
1) You are correct in asserting that each of the N coin tossing sequences have a time-average. However, in the limit, all of those N time averages are equal exactly because they are governed by the same dynamic. And they will all converge to the absorbing boundry (zero) with probability one. If you find that unintuitive, try to simulate a bunch of those processes, and plot their distribution. What you will get is a log-normal with diverging moments. E.g. there will be one-in-a-million who's wealth grows exponentially. The bulk of the ensemble will be close to zero.
2) Is the above wealth generating process ergodic? No. The ensemble average will predict infinite positive growth$^*$ while the time-average will converge to zero.$^{**}$ This is commonly known as the St. Petersburg paradox. Unrelated to your question, but both interesting and important: It is possible to create an ergodic observable using the logarithm which solves the St. Petersburg paradox.
Hope you can use this. If you want to see the formal proofs and simulations, I can recommend:
$^{*}$ Ensemble average: $\frac12\times0.6+\frac12\times1.5=1.05$ a number larger than one, reflecting positive growth of the ensemble.
$^{**}$ Time average: $x(t)=r_1^{n_1}r_2^{n_2}$ where $r_1$ and $r_2$ are the two rates and $n_1$ and $n_2$ are the frequences that the wealth process gets subjected to the rates. The limit of $x(t)$ is then for $t\rightarrow\infty$ $x(t)^{1/t} = (r_1r_2)^{1/2}$ or $\sqrt(0.9)\approx0.95$ e.g. a number less than one, which is decay in the long time limit.
Video
Article
Best Answer
This is simply a quite careless definition of ergodicity!
Here "complete" is used a loose way to mean that you can determine all that there is to determine about some statistics in question.
Surely, the author is talking about a specific statistics and indeed in the same page we have $\hat \mu_N$ and as written "where the sample mean operation can be significantly simplified".
Why is it quite careless then?
Because this is not how it works: we are not going to get the same value for all integers $N$, as the author implies, but can only approximate by taking $N$ large and we know that the approximation converges to the same value for any realization, that is, any choice of particular values (samples).
In more mathematical terms, this is indeed (a failed) try at simplifying the big picture of ergodic theory in which ergodicity is equivalent to the equality of time and space averages.