In response to Geoffrey Trang's comment on my previous answer, I will add a sharper answer as community wiki.
The following proof can be found in O.A.S. Karamzadeh's note, The Prime Avoidance Lemma Revisited, as Theorem B. Here I simplify slightly and assume the ring is commutative.
Let $P_1, \ldots, P_n$ be ideals of a ring $R$ at most one of which is not prime. Let $I$ be an additive subset of $R$ having the structure of an ideal with respect to each $P_i$, except perhaps one of the prime $P_i$. If $T$ is a subset of $R$ such that $I + T \subseteq \bigcup P_i$, then there exists a $t \in T$ such that $(I, t) \subseteq P_i$ for some $i$.
Proof:
We proceed by induction on $n$, with the case $n=1$ being trivial. We can assume that $P_i \nsubseteq P_j$ for any $i,j$. For the inductive step, we first order the $P_i$ so that $P_1$ is prime and $I$ has ideal structure with respect to the remaining $P_i$. The remaining $P_i$ will still have at most $1$ non-prime $P_i$ amongst them. We separate two cases: (1) $I + T \subseteq \bigcup_{i=2}^n P_i$, and applying the inductive hypothesis we are immediately done (2) $I + T \nsubseteq \bigcup_{i=2}^n P_i$, and thus there exist $x \in I, t \in T$ such that $x + t \in P_1 \setminus \bigcup_{i=2}^n P_i$. In this case we claim that $(I, t) \subseteq P_1$.
For convenience set $J = \bigcap_{i=2}^n P_i$. Note that $J \nsubseteq P_1$ by primeness of $P_1$ and the assumption that $P_i \nsubseteq P_j$ for all $i,j$. It thus suffices to show that $I \cap J \subseteq P_1$ because that would force $I \subseteq P_1$, and consequently force $t \in P_1$ (note that we needed $I$ to have ideal structure w.r.t. $J$ in order to say that $IJ \subseteq I \cap J$). Thus let $y \in I \cap J$. Since $x + t$ isn't in any $P_i$ for $i \geq 2$ and $y \in P_i$ for all $i \geq 2$, we deduce $x + t + y \notin \bigcup_{i=2}^n P_i$, and hence $x + t + y \in P_1$. Since $x + t$ is also in $P_1$, we deduce that $y \in P_1$. This shows that indeed $I \cap J \subseteq P_1$, and completes the proof. $\square$
Finally I'll give a couple of simple examples that demonstrate why the assumptions on the algebraic structure of $I$ and on the number of prime ideals are crucial in the above statement.
That $I$ needs to have ideal structure with respect to (all but a particular one of) the $P_i$
Consider the ring $R = F_2[x]$ and the coset $x + F_2$. This coset is formed from a bonafide subring $F_2 \subset F_2[x]$, and consists of the two non-unit elements $x, x+1$, so of course we can cover it by proper (prime) ideals of $R$ (take $(x), (x+1))$. However, the coset $x + F_2$ generates $R$ as an ideal, and so cannot be contained in any proper prime ideal.
That at most one of the $P_i$ can generally be non-prime
Consider the ring $R = F_2[x,y]/(x^2, y^2)$. Consider the coset $x + (y)$. It consists of the elements $x, x + yx, x+y, x + yx + y$, and can be covered by the non-prime ideals $(x), (x + y)$. However, $y$ is not contained in either of these ideals, so of course $(x,y)$ isn't either.
Best Answer
Let $A=\mathbb {Z}[X,Y], a=(2,X,Y), a_{1}=(2,X^2,Y), a_{2}=(2,X,Y^2), a_{3}=(2,X+Y,X^2,Y^2,XY)$
Then $a$ is contained in the union of $a_1,a_2,a_3$ (you can check it by computing in $A/(2,X^2,Y^2,XY)$ , which is a ring with $4$ elements) but not contained in any $a_i$.
Source of the counter example: https://fr.wikipedia.org/wiki/Lemme_d%27%C3%A9vitement_des_id%C3%A9aux_premiers