$$
\sum_{i=1}^{\infty}\left[\sum_{j=i}^{\infty}f(i,j)\right]=\sum_{j=1}^{\infty}\left[\sum_{i=1}^{j}f(i,j)\right]
$$
I have trouble understanding the change of order of this summation. I know that
$$
1≤i≤j≤\infty
$$
however, I can't seem to make sense of this even with manually comparing each sum of both double summations.
In the left side, we always have the sums: $$
S_1=f(1,1)+\dots f(1,\infty)
$$
$$
S_2=f(2,2)+\dots f(2,\infty)
$$
$$
\vdots
$$
so the total sum
$$
S=\sum S_n
$$
looks like summing each element of an $N\text{x}N$ matrix as $N\to\infty$
In the right side, we have the sums:
$$
S_1=f(1,1)
$$
$$
S_2=f(1,2) + f(2,2)
$$
$$
S_3=f(1,3)+f(2,3) + f(3,3)
$$
$$
\vdots
$$
The total sum $S$ looks like summing the top-right to bottom-left entries of a "pyramid" structure, which looks like the earlier matrix but deformed and slanted. However, I do not want to just assume that this is the case since I am working with infinite sums.
Is there a better intuition on how changing order of summations work so that I may extend it to more difficult scenarios particularly in the proof of the convolution property of Z transforms.
Best Answer
Intuition:
The change of summation work much like you are trying to loop over some $N\times N$ matrix (in your case you take $N\to\infty$). You can go by looking at each rows and within each columns or change it and go by looking at columns and each rows.
In your example lets say $i$ is the row and $j$ is the column index.
The right hand side:
$\sum_{i=1}^{\infty}$ means that you are looking through all row and $\sum_{j=i}^{\infty}$ means that you are looking for elements below the diagonal(row is already set by $i$ so the column identifies the element $(i, j)$).
The left hand side:
$\sum_{j=1}^{\infty}$ means that you are looking through all columns and $\sum_{i=1}^{j}$ means again that you are (again) looking for elements below the diagonal(now the column is already set by $j$ so the row identifies the element $(i, j)$).
You could prove that it works for any $N\times N$ matrix by induction.