The fact that the preimage of an open is open IS NOT equivalent to the fact that $f$ maps open sets to open sets. Such maps are called open maps. There exist continuous maps that are not open, open maps that are not continuous, open continuous maps and maps which are neither continuous nor open!
To find continuous maps from $X$ to $Y$ in your case amounts exactly to find all the possible preimages of $A$. Since $Y$ has only two elements, the preimage of $A$ determines uniquely the map. So there are exactly 5 continuous maps $X\to Y$. Two of them are the constant ones, the third is the one mapping $1$ to $A$ (and everything else to $B$), the fourth is the one mapping $3$ and $4$ to $A$ and the fifth maps $1$,$3$ and $4$ to $A$.
You don't have to care about $\varnothing$, because its preimage under any map is again $\varnothing$ which has to be open by definition of topology. In the same way, you don't have to care about the preimage of $Y$, which is always $X$.
$f^{-1}[Y]$ is the pre-image of a set under $f$, it's not about an inverse map. I prefer the notion with brackets over the one with braces for this notion.
If $f: X \to Y$ is any function and $B \subseteq Y$ then $f^{-1}[B] = \{x \in X \mid f(x) \in B\}$ is just the set of points of the domain that map into $B$. So if $B=Y$, by definition of a function from $X$ to $Y$, all $f(x) \in Y$ and so $f^{-1}[Y]=X$ follows.
So if $f$ is constant with value $c$, $f^{-1}[B]=X$ if $c \in B$ and $\emptyset$ otherwise. So always open in $X$ regardless of what topology $X$ or $Y$ have.
As an afterhought: suppose that $X$ has the trivial topology $\{\emptyset,X\}$, and if $f$ is not constant so there are $x,x'$ in $X$ such that $f(x) \neq f(x')$. If in $Y$ there is an open set $O$ that contains just one of the points $\{f(x), f(x')\}$ then $f$ cannot be continuous as $f^{-1}[O]$ is non-empty (it contains $x$ or $x'$) and also not equal to $X$ (it misses $x$ or $x'$; the other one), so $f^{-1}[O]$ cannot be open. But e.g. $X=\{1,2,3\}$ in the trivial topology to $Y=\{1,2,3\}$ in the topology $\{\emptyset, \{2,3\}, Y\}$ has a non-constant continuous function (defined by $f(1)=2, f(2)=3, f(3)=2$, which has $f^{-1}[\emptyset] = \emptyset, f^{-1}[Y] =X, f^{-1}[\{2,3\}] = X$ so is continuous)).
So it's not true that a trivial topology only has constant maps to non-trivial $Y$, it will depend on the topology of $Y$.
In fact, let $I(2)$ be the space $\{0,1\}$ in the indiscrete/ trivial topology.
$Y$ is $T_0$ iff every continuous $f: I(2) \to Y$ is constant
Proof: If $Y$ is $T_0$ and $f$ is a function from $I(2)$ to $Y$ that is non-constant then $f(0) \neq f(1)$ and so by $T_0$-ness there is an open set $O$ in $Y$ such that $\{f(0),f(1)\} \cap O$ has one member, say $f(0)$ WLOG. Then $f^{-1}[O]=\{0\}$ is not open in $I(2)$. So $f$ is not continuous.
If we assume the mapping property, we can show that $Y$ is $T_0$: suppose $Y$ is not $T_0$. Then there are two points $y_1\neq y_2$ in $Y$ such that for every open set $O$ in $Y$ we have $y_1 \in O \land y_2 \in O$ or $y_1 \notin O \land y_2 \notin O$; we cannot separate them by an open set. But that implies that the map $f$ sending $0$ to $y_1$ and $1$ to $y_2$ is continuous, as $f^{-1}[O]=\emptyset$ or $=I(2)$ depending on which is the case for $O$. And $f$ is non-constant, so that would contradict the mapping property. So $Y$ is $T_0$.
So in topological category theory we could say that the $T_0$ spaces are the $I(2)$-co-connected spaces in $\mathbf{Top}$, or some such term. Connected spaces are $D(2)$-connected (where $D(2)=\{0,1\}$ in the discrete topology), as is classical (all continuous functions into $D(2)$ are constant).
Best Answer
Your argument is wrong. Note that $f^{-1}(U)$ cannot be all of $X$, as we have some $y\in Y\setminus U$ and since $f$ is onto we have some $x\in X$ such that $f(x)=y$, so $x\notin f^{-1}(U)$.