Given the theorem for an ordinary series, I need to prove the same for an arbitrary double series. I've seen a very similar question, but I haven't worked with double integrals in real analysis so far and I have to stick to the theorem: Proving double sums are interchangable.
This is a theorem $№\;28$ in the Analysis book written by Svetozar Kurepa and the proof I have to apply in the case of double series:
Instead of $(a_n)_n$ I have $(a_{mn})_{mn}$
Let $(a_n)_n$ be a complex number sequence:
$I.$
$\text{series}\;\sum|a_n|\;\text{converges}\implies \text{series} \sum a_n\;\text{converges}\;\land\;\underbrace{\Big|\sum_1^{\infty}a_n\Big|\leq\;\sum_1^{\infty}|a_n|}_{\text{triangle inequality}}$
$II.$
$\text{series}\;\sum|a_n|\;\text{converges}\;\land \exists\; \text{bijection}\; \sigma:\mathbb N\rightarrow \mathbb N\implies \text{series}\; \sum a_{\sigma{(n)}}\;\text{converges}$$
$$\land\;\sum_1^{\infty}a_{\sigma{(n)}}=\sum_1^{\infty}a_n$
Proof of $I.$:
Let $(a_n)_n$ be a real number series and let's define sequences with positive terms:
$$a_n^+=\begin{cases} \displaystyle
\ a_n,\;a_n\geq 0 \\
\ 0,\;\;a_n<0
\end{cases}\;\;\;\;$$
$$a_n^{-}=\begin{cases} \displaystyle
\ 0\;\;a_n\geq 0 \\
\ -a_n\;a_n<0
\end{cases}$$
$$a_n=a_n^+-a_n^-,\;\forall n\in \mathbb N$$
$$\text{series} \sum |a_n|\;\text{converges}\;\land\;a_n^+\leq |a_n|\;\land\;a_n^-\leq |a_n|,\;\forall n\in \mathbb N\implies \sum a_n^+\;\& \sum a_n^-\;\text{converge}$$
Let $A^+$ & $A^-$ be their sums.
By the theorem for convergence of linear combinations of $2$ series:
$$A^+-A^-=\sum_1^{\infty} a_n^+-\sum_1^{\infty} a_n^-=\sum_1^{\infty} (a_n^+ -a_n^-)=\sum_1^{\infty} a_n\leftarrow \text{converges}$$
Proof of $II.$:
Let $a_n=\alpha_n+i\beta_n\;(\alpha_n,\beta_n\in \mathbb R)$
$$|\alpha_n|\leq\sqrt{\alpha_n^2+\beta_n^2}=|a_n|\implies \sum |\alpha_n|\leq \sum |a_n|\implies \text{series}\;\sum |\alpha_n|\;\text{converges}$$
$$\text{series}\;\sum |\alpha_n|\;\text{converges}\implies\text{series}\sum \alpha_n\;\text{converges}$$
Analogously:
$$|\beta_n|\leq|a_n|\implies \text{series}\;\sum |\beta_n|\;\text{converges}$$
$$\text{series}\; \sum \alpha_n\;\&\;\sum \beta_n\;\text{converge}\;\implies \text{series}\;\sum (\alpha_n +\beta_n)=\sum a_n\;\text{converges}$$
Furthermore:
$$\sum_1^{\infty} a_n=\sum_1^{\infty} \alpha_n+i\sum_1^{\infty} \beta_n=\sum_1^{\infty}\alpha_{\sigma{(n)}} +i\sum_1^{\infty} \beta_{\sigma{(n)}}=\sum_1^{\infty} (\alpha_{\sigma{(n)}}+i\beta_{\sigma{(n)}})=\sum_1^{\infty} a_{\sigma{(n)}}$$
Best Answer
Although I am not exactly a specialist in either topology or analysis, this topic that you address I hold quite dear to my heart so may I walk you through a rather general topological approach towards the aim of deriving a far more general form of the result you mention, obtainable by far more general and efficient methods than mere series summation. Please do bear with me, as this is likely to become quite an expound.
$$\prod_{i \in I}B=B^I$$
This is nothing else than the set of all families of elements of $B$ indexed by $I$. For arbitrary set $A$ we write $$\mathscr{P}(A)=\{X|\ X \subseteq A\}$$
for its powerset (set of all subsets of $A$). Therefore, combining the two notations introduced, saying that $M \in \mathscr{P}(A)^I$ means that $M$ is a family of subsets of $A$ indexed by $I$, so for every $i \in I$ we will have $M_i \subseteq A$.
On a set $A$ a binary relation $R$ will simply be a subset $R \subseteq A \times A$; we say that $x$ and $y$ are related by $R$ (are in the relation $R$) when $(x, y) \in R$ and we express this alternatively as $xRy$. We remark a distinguished relation on $A$, called the diagonal of $A$ and defined by $\Delta_A=\{(x,x)\}_{x \in A}$; in plain terms it is the relation of equality on $A$.
For arbitrary relation $R$ we define its inverse (or dual or reciprocal) $R^{-1}$ by the property $xR^{-1}y \Leftrightarrow yRx$.
An ordered set will be an ordered pair $(A, R)$ such that $R$ is an order relation on $A$; we will express the relation $xRy$ as $x \leqslant_R y$, since it is quite suggestive to employ the $\leqslant$ symbol for orders. For arbitrary set $A$ let us write $\mathscr{Ord}(A)$ for the set of all order relations on $A$.
Given a family $A$ of sets indexed by $I$ and a family of orders $$R \in \prod_{i \in I} \mathscr{Ord}(A_i)$$
which means that for each index $i \in I$ the relation $R_i$ is an order on $A_i$, we can define a natural order relation on the cartesian product
$$\prod_{i \in I} A_i$$
structure which will be called the direct product of the orders $R_i$ with $i \in I$, denoted by $S$ and defined such that
$$x \leqslant_S y \Leftrightarrow (\forall i)(i \in I \Rightarrow x_i \leqslant_{R_i} y_i)$$
This definition amounts to saying that two families $x, y \in \prod_{i \in I} A_i$ are comparable through $S$ when they are comparable componentwise, through $R_i$ at each index $i$. The ordered set $$(\prod_{i \in I} A_i, S)$$
is called the direct product of the family of ordered sets $(A_i, R_i)_{i \in I}$.
A map $f: A \to B$ between two ordered sets $(A, R)$ and $(B, S)$ is called increasing or isotonic when $$(\forall x, y)(x \leqslant_R y \Rightarrow f(x) \leqslant_S f(y))$$
and it is called an isomorphism of ordered sets (order isomorphism for short) if it is isotonic and invertible such that its inverse also is isotonic.
By an upward directed set we mean an ordered set $(A, R)$ such that for any $x, y \in A$ there will exist an element $z \in A$ with the property $x, y \leqslant_R z$.
A typical example of such objects can be found in the instance of the set of all finite subsets of a given set $A$, collection which we are going to denote by $\mathscr{F}(A)=\{X \subseteq A|\ |X|< \aleph_0\}$ and which we will order by inclusion: since for any finite subsets $X, Y \subseteq A$ the union $X \cup Y \subseteq A$ is also finite and obviously $X, Y \subseteq X \cup Y$, the property of upward-directedness is indeed satisfied. As an ordered set, $\mathscr{F}(A)$ will have a maximum if and only if $A$ is finite (in which case the maximum is $A$ itself).
Before concluding let us make the observation that a direct product of upward directed (ordered) sets is itself an upward directed set.
Next, let us recall the notion of generalized sequence of elements of a set: given set $A$, index set $I$ and order relation $R$ such that $(I, R)$ is upward directed, for any family of elements $x \in A^I$ we call $(x, R)$ a generalized sequence of elements of $A$ (with respect to order $R$, if we are to be very pedantic; it is the specific choice of an order that determines the ''sequence'' of elements in the family $x$). Although an abuse of language, it is quite customary however to refer to a generalized sequence $(x, R)$ as merely the sequence $x$, omitting any mention of the underlying order.
Given a topological space $(X, \mathscr{T})$ (I hope you were at least introduced to the notion), an index set $I$ upward directed by order $R$ and family $x \in X^I$ we say that the generalized sequence $(x, R)$ converges to element $t \in X$ (the syntagm ''sequence $(x, R)$ has $t$ as a limit" is also acceptable) if:
which we express symbolically as
$$x_i \xrightarrow[R]{i \in I} t\ (\mathscr{T})$$
or more simply
$$x \xrightarrow[R]\ t\ (\mathscr{T})$$
this being the most general notation to be used in the case of not necessarily Hausdorff spaces (when limits might not be unique whenever they exist). For reasons of rigour we make explicit in brackets the topology with respect to which the convergence takes place; however, when there is no risk of ambiguity (and thus in the vast majority of cases) we shall omit any reference to the background topology.
In the particular case of a Hausdorff space, we also adopt notation
$$\mathrm{lim}_{i \in I\\ R}x_i=t$$
or in a simplified version
$$\mathrm{lim}_{R}x=t$$
We shall say a generalized sequence converges (or is convergent) in $X$ if there exists a $t \in X$ such that $x$ converges to $t$. Let us notice that if the ordered set $(I, R)$ has a maximum $\alpha$ then any generalized sequence $x$ indexed by it will be convergent, having $x_{\alpha}$ as a limit.
This section we conclude with a couple of remarks that will play a definite role later on:
$$\mathrm{lim}_{j \in J\\ S} x_{\varphi(j)}=\mathrm{lim}_{i \in I\\ R}x_i$$
(the change of variable for generalized sequences).
$$t \in \prod_{j \in J} X_j$$
a family $(I_j, R_j)_{j \in J}$ of upward directed sets and a family of families (!) $$u \in \prod_{j \in J} X_j^{I_j}$$
(explicitly, for each $j \in J$ the object $u_j$ is itself a family of points in $X_j$ indexed by $I_j$) such that for each $j \in J$ we have
$$u_j \xrightarrow[R_j]\ t_j\ (\mathscr{T}_j)$$
If we denote by $(Y, \mathscr{S})$ the direct product of the family of topological spaces $(X_j, \mathscr{T}_j)$ (so in particular $Y=\prod_{j \in J} X_j$), $(K, S)$ the direct product of the family of ordered sets $(I_j, R_j)$ (so that $K=\prod_{j \in J}I_j$) and if we introduce the family $$y \in Y^K\\ y_k=((u_j)_{k_j})_{j \in J}$$
(family which is itself informally speaking the direct product of the family of families $u$), then we have that $(y, S)$ is a generalized sequence (informally, the direct product of the family of generalized sequences $(u_j, R_j)_{j \in J}$) and that
$$y \xrightarrow[S]\ t\ (\mathscr{S})$$
(compatibility of direct products with convergence of generalized sequences).
where we implicitly understand the upward directed order on $\mathscr{F}(I)$ to be inclusion (cnf. paragraph 1) and we shall not mention it explicitly.
With this in place we say that the family $x$ is summable in $M$ if the associated generalized sequence of partial sums $s$ converges in $M$; any limit of $s$ will be called a sum of family $x$. If the topology considered is Hausdorff, then a summable family will have only one sum, and we agree to write:
$$\sum_{i \in I}x_i=\mathrm{lim}_{F \in \mathscr{F}(I)} \sum_{i \in F}x_i$$
Written explicitly, the claim that $t$ is a sum of family $x$ means that:
$$\sum_{i \in H}x_i \in V$$
Let us end this paragraph with a couple of important remarks:
a) If the index set $I$ is finite, then by some of the remarks of paragraphs 2 and 4, any family $x$ indexed by $I$ will be summable having $\sum_{i \in I}x_i$ as a sum! Therefore, in a Hausdorff commutative topological monoid the sum of a finite family understood in the ''topological sense'' we just introduced in this section is nothing else than its algebraic sum in the ordinary algebraic sense.
b) Given an arbitrary family of objects $x$ indexed by $I$ and arbitrary subset $J$ allow me to denote by $x_{|J}=(x_i)_{i \in J}$ the restriction of $x$ to $J$; With this specification, if $x \in M^I$ and $J, J' \subseteq I$ are disjoint ($J \cap J'=\varnothing$) and such that both restrictions $x_{|J}, x_{|J'}$ are summable of sums say $s$ and respectively $t$, then the ''joint'' restriction $x_{|J \cup J'}$ is also summable, having $s+t$ as a sum; this observation easily extends to a finite collection of subsets of $I$ that are pairwise disjoint and such that the restriction of $x$ to each of them is summable.
c) Consider a permutation $\sigma \in \Sigma(I)$ of the set $I$ and the ''permutated'' family $y=(x_{\sigma(i)})_{i \in I}$. It is not difficult to show that if $x$ is summable of sum $s$ then $y$ is also summable admitting $s$ as a sum.
d) Given a sequence $x \in M^{\mathbb{N}}$, if $x$ is summable as a family with sum $s$ then the series $x$ (of general term $x_n, n \in \mathbb{N}$) also converges to sum $s$. Symbolically, if
$$\sum_{n \in H} x_n \xrightarrow{H \in \mathscr{F}(\mathbb{N})}\ s$$
then we also have
$$\sum_{k=0}^n x_k \xrightarrow {n \to \infty}\ s$$
In the case of a Hausdorff topology, if sequence $x$ is summable then
$$\sum_{n \in \mathbb{N}}x_n=\sum_{n=0}^{\infty}x_n$$
This result is also valid for double (in general multiple) sequences/series and it is this observation which makes the entire theory I am expounding on here applicable to your particular case of summating series!
We are finally prepared to give a very general formulation of the result known as:
$$\bigcup_{k \in K}J_k=I$$ $$(\forall k,l)(k,l \in K \wedge k \neq l \Rightarrow J_k \cap J_l = \varnothing)$$
$$\sum_{i \in I}x_i=\sum_{k \in K}y_k$$
Written in developed fashion the relation above becomes:
$$\sum_{i \in I} x_i=\sum_{k \in K} \sum_{i \in J_k}x_i$$
which is indeed a relation of general associativity. I am not going to prove this very general statement however, as it relates to abstract notions of uniform structures (and I do not think it yet helpful for your understanding to take matters to such a level of abstraction), and will instead concentrate on a more familiar and particular version of it. Do however write back if you would like to learn more about the general setting.
As the situation stands right now, my feeble compiler is no longer able to efficiently handle further re-edits of this answer, so I see myself compelled to continue the intended presentation in a separate answer. I apologise for the highly unconventional approach, but I ask all those interested to bear with me.