Definition 1: A function $f:X\to Y$ for topological spaces $X$ and $Y$ is continuous if the preimage $f^{-1}(U)\subset X$ is open in $X$ for each $U\subset Y$ open in $Y$.
Definition 2: A function $f:X\to Y$ for topological spaces $X$ and $Y$ is open if a set $Z\subset X$ being open in $X$ implies that $f(Z)\subset Y$ is open in $Y$.
Definition 3: A function between topological spaces is a homeomorphism $\iff $ it is a continuous bijection and the inverse function $f^{-1}$ is also continuous.
Definition 4: Let $f:X\to Y$ be a function between sets. The preimage of a subset $U\subset Y$ is the collection of $x\in X$ such that $f(x)\in U$, notated $f^{-1}(U)$. Concretely,
$$f^{-1}(U)=\{x\in X\mid f(x)\in U\}.$$
Theorem: The function $f:X\to Y$ for topological spaces $X$ and $Y$ is an open, continuous bijection $\iff$ $f$ is a homeomorphism.
Proof Attempt: Let $f:X\to Y$ be an open, continuous bijection. Our goal is to show that $f$ is a homeomorphism; in particular, we need to show that the inverse function $f^{-1}:Y\to X$ is continuous. To do this, we need to show that for each set $Q\subset X$ open in $X$ the preimage $(f^{-1})^{-1}(Q)\subset Y$ is open in $Y$. Let $Z\subset X$ be open in $X$. Consider the following manipulation of the definition of preimage:
$$(f^{-1})^{-1}(Z)=\{y\in Y \mid f^{-1}(y)\in Z\}=\{y\in Y \mid y\in f(Z)\}=f(Z)\subset Y.$$
Since $f$ is open and $Z\subset X$ is open in $X$, $(f^{-1})^{-1}(Z)=f(Z)\subset Y$ is open in $Y$. Thus, $f^{-1}$ is continuous, and in particular $f$ is a homeomorphism.
Now let $f$ be a homeomorphism. Then $f$ is a continuous bijection with a continuous inverse $f^{-1}$. We aim to show that $f$ is open. Let $Z\subset X$ be open in $X$. Since $f^{-1}$ is continuous, the preimage $(f^{-1})^{-1}(Z)=f(Z)\subset Y$ is open in $Y$. Thus $f$ is open.$\;\;\;\boxed{}$
Notes: Does this proof make sense? Is it valid given the definitions? The equality $\{y\in Y \mid y\in f(Z)\}=f(Z)$ was very tricky for me to come to understand, and I'm only partly sure it is true.
Response to Answerers: Thank you for your time and expertise! I think I have a more economical grasp of the concept now. My newfound conception is that for a bijection $f$, $f$ being open is equivalent to $f^{-1}$ being continuous. This should be illustrated through the following diagram, where for notational ease I've relabeled $f^{-1}=g$. The key for me has been the understanding that the implications in the definitions of openness (of $f$) and continuity (of $g$) are the same:
Best Answer
You're overdoing the notation bit. The only observation of note is that for a bijection (!) being open is the same as the inverse being continuous.
To keep notation clean use $g: Y \to X$ for the unique inverse of $f$ (so that $f(x)=y$ iff $g(y)=x$ etc.) and note that indeed for any $O \subseteq X$ we have $$y \in g^{-1}[O] \iff g(y) \in O \iff \exists x \in O: g(y)=x \iff \exists x \in O: f(x)=y \iff y \in f[O]$$ hence
$$g^{-1}[O]=f[O]\tag{1}$$
from which it immediately follows that $f$ open implies $g$ continuous (taking $O$ to be any open set) or $f$ closed implies $g$ continuous (taking $O$ to be closed etc.) too, and vice versa.