The first two characterisations (using $f^{-1}$) are derivable from each other : in any space $X$ we have the complements duality between interior and closure
$$\operatorname{int}(A)=X\setminus \overline{X\setminus A}$$
and $$\overline{A}= X\setminus \operatorname{int}(X\setminus A)$$
and moreover, $f^{-1}$ plays nice with complements, i.e.
$$f^{-1}[Y\setminus A]=X\setminus f^{-1}[A]$$
and complements reverse inclusions.
But we don't have the complement property for forward images (we do for injective functions, but generally $f[X\setminus A]\neq f[X]\setminus f[A]$ etc.)
The property that $f[\operatorname{int}(A)] \subseteq \operatorname{int}(f[A])$ characterises open maps, as can be easily seen. But the reverse
$$f[\operatorname{int}(A)] \supseteq \operatorname{int}(f[A])$$
does not characterise continuity, as can be seen from the examples here, (both implications with continuity fail) which I won't repeat here.
Here it is:
|
For all $A \subset X$ resp. $B \subset Y$ |
is equivalent to |
| (0) |
$B$ closed $\Rightarrow f^{-1}(B)$ closed |
$f$ continuous |
| (1) |
$f(\operatorname{int} A) \subset \operatorname{int} f(A)$ |
$f$ open |
| (2) |
$f(\operatorname{int} A) \supset \operatorname{int} f(A)$ |
- |
| (3) |
$f^{-1}(\operatorname{int} B) \subset \operatorname{int} f^{-1}(B)$ |
$f$ continuous |
| (4) |
$f^{-1}(\operatorname{int} B) \supset \operatorname{int} f^{-1}(B)$ |
$f$ open |
| (5) |
$f(\operatorname{cl} A) \subset \operatorname{cl} f(A)$ |
$f$ continuous |
| (6) |
$f(\operatorname{cl} A) \supset \operatorname{cl} f(A)$ . |
$f$ closed |
| (7) |
$f^{-1}(\operatorname{cl} B) \subset \operatorname{cl} f^{-1}(B)$ |
$f$ open |
| (8) |
$f^{-1}(\operatorname{cl} B) \supset \operatorname{cl} f^{-1}(B)$. |
$f$ continuous |
Let us first collect some well-known facts about images and preimages of functions $f : X \to Y$ between sets $X,Y$.
$f^{-1}$ is compatible with all standard set-theoretic operations. That is, $f^{-1}(B \cup B') = f^{-1}(B) \cup f^{-1}(B')$, $f^{-1}(B \cap B') = f^{-1}(B) \cap f^{-1}(B')$, $f^{-1}(Y \setminus B) = X \setminus f^{-1}(B)$. The latter is also written as $f^{-1}(\complement B) = \complement f^{-1}(B)$, where $\complement$ denotes the complement.
$f$ is compatible with union. That is, $f(A \cup A') = f(A) \cup f(A')$.
$A \subset f^{-1}(f(A))$. Equality holds if $f$ is injective.
$f(f^{-1}(B)) \subset B$. Equality holds if and only $B \subset f(X)$.
$A \subset f^{-1}(B)$ if and only if $f(A) \subset B$.
$A \cap f^{-1}(B) = \emptyset$ if and only if $f(A) \cap B = \emptyset$.
Moreover, we shall need the following well-known relation between the operators $\operatorname{int}$ and $\operatorname{cl}$ in a topological space $Z$.
- $\operatorname{cl} M = \complement \operatorname{int} \complement M$ and $\operatorname{int} M = \complement \operatorname{cl} \complement M$ for all $M \subset Z$.
- $f$ continuous $\Leftrightarrow$ (0):
A subset $M \subset Z$ is open if and only if $\complement M$ is closed. The equivalence follows from $f^{-1}(B) = f^{-1} (\complement \complement B) = \complement f^{-1} (\complement B)$.
- $f$ continuous $\Rightarrow$ (5):
$f^{-1}(\operatorname{cl}f(A))$ is closed and contains $A$. Hence $\operatorname{cl}{A} \subset f^{-1}(\operatorname{cl}{f(A)})$ which implies $f(\operatorname{cl}{A}) \subset f(f^{-1}(\operatorname{cl}{f(A)})) \subset \operatorname{cl}{f(A)}$.
We have $f(\operatorname{cl}{f^{-1}(B)}) \subset \operatorname{cl}{f(f^{-1}(B)))} \subset \operatorname{cl}{B}$, hence $\operatorname{cl}{f^{-1}(B)} \subset f^{-1}(\operatorname{cl}{B})$.
We have $\operatorname{cl}{f^{-1}(\complement B)} \subset f^{-1}(\operatorname{cl}{\complement B})$. Therefore
$$f^{-1}(\operatorname{int} B) = f^{-1}(\complement\operatorname{cl}{\complement B}) = \complement f^{-1}(\operatorname{cl}{\complement B}) \subset \complement \operatorname{cl}{f^{-1}(\complement B)}
= \complement \operatorname{cl}{\complement f^{-1}(B)} = \operatorname{int} f^{-1}(B) .$$
- (3) $\Rightarrow$ $f$ continuous:
For any open $V \subset Y$ we have $f^{-1}(V) = f^{-1}(\operatorname{int} V) \subset \operatorname{int} f^{-1}(V)$, hence $f^{-1}(V) = \operatorname{int} f^{-1}(V)$ which is open.
- $f$ open $\Rightarrow$ (1):
We have $f(\operatorname{int} A) \subset f(A)$. Since $f(\operatorname{int} A)$ is open, we conclude $f(\operatorname{int} A) \subset \operatorname{int} f(A)$.
- (1) $\Rightarrow$ $f$ open:
For any open $U \subset X$ we have $f(U) = f(\operatorname{int} U) \subset \operatorname{int} f(U)$, hence $f(U) = \operatorname{int} f(U)$ which is open.
- $f$ open $\Rightarrow$ (7):
Let $x \in f^{-1}(\operatorname{cl}{B})$. Then $f(x) \in \operatorname{cl}{B}$. Let $U$ be an open neigborhood of $x$ in $X$. Then $f(U)$ is an open neigborhood of $f(x)$ in $Y$, hence $f(U) \cap B \ne \emptyset$. This implies $U \cap f^{-1}(B) \ne \emptyset$. We conclude $x \in \operatorname{cl}{f^{-1}(B)}$.
- (7) $\Rightarrow$ $f$ open:
Let $U \subset X$ be open. Since $f^{-1}(\operatorname{cl}{\complement f(U)}) \subset \operatorname{cl}{f^{-1}(\complement f(U))} = \operatorname{cl}{\complement f^{-1}(f(U))} \subset \operatorname{cl}{\complement U} = \complement U$, we get $f^{-1}( \operatorname{cl}{\complement f(U)}) \cap U = \emptyset$ which is equivalent to $ \operatorname{cl}{\complement f(U)} \cap f(U) = \emptyset$. This shows $\operatorname{cl}{\complement f(U)} \subset \complement f(U)$. Therefore $\operatorname{cl}{\complement f(U)} = \complement f(U)$, i.e $\complement f(U)$ is closed and $f(U)$ is open.
We have $f^{-1}(\operatorname{cl}{\complement B}) \subset \operatorname{cl}{f^{-1}(\complement B)}$. Therefore
$$\operatorname{int} f^{-1}(B) = \complement \operatorname{cl}{\complement f^{-1}(B)} = \complement \operatorname{cl}{f^{-1}(\complement B)} \supset \complement f^{-1}(\operatorname{cl}{\complement B})
= \complement f^{-1}(\complement \operatorname{int} B) = \complement \complement f^{-1}( \operatorname{int} B) \\= f^{-1}( \operatorname{int} B) .$$
We have $f^{-1}(\operatorname{int}{\complement B}) \supset \operatorname{int}{f^{-1}(\complement B)}$. Therefore
$$\operatorname{cl} f^{-1}(B) = \complement \operatorname{int}{\complement f^{-1}(B)} = \complement \operatorname{int}{f^{-1}(\complement B)} \subset \complement f^{-1}(\operatorname{int}{\complement B})
= \complement f^{-1}(\complement \operatorname{cl} B) = \complement \complement f^{-1}( \operatorname{cl} B) \\= f^{-1}( \operatorname{cl} B) .$$
- $f$ closed $\Rightarrow$ (6):
We have $f(A) \subset f(\operatorname{cl}{A})$ which is closed. Hence $\operatorname{cl}{f(A)} \subset f(\operatorname{cl}{A})$.
- (6) $\Rightarrow$ $f$ closed:
For any closed $C \subset X$ we have $f(C) = f(\operatorname{cl}{C}) \supset \operatorname{cl}{f(C)}$, hence $f(C) = \operatorname{cl}{f(C)}$ which is closed.
- Property (2) on itself does not have a meaning.
Example 1. $f : \mathbb{R} \to \mathbb{R}^2, f(x) =
\begin{cases}
(x,0) & x \le 0 \\
(x,1) & x > 0
\end{cases}
$ satisfies (2) because $\operatorname{int} f(A) = \emptyset$ for all $A$. $f$ is not continuous, not open and not closed.
Note that $f$ is injective, not surjective and $f(\mathbb{R})$ is not open in $\mathbb{R}^2$.
Example 2. $f: I \times I \to I, f(s,t) = s$ is continuous, open, closed. It does not satisfy (2) because for the diagonal $\Delta \subset I \times I$ we have $f(\operatorname{int}\Delta)) = f(\emptyset) = \emptyset$, but $\operatorname{int} f(\Delta) = \operatorname{int} I = I$.
Note that $f$ is surjective and not injective.
- However, there is an interesting special case in which (2) implies continuity.
Theorem. Let $f(X)$ be open in $Y$. If (2) is satisfied, then $f$ is continuous.
Proof. Let $V \subset Y$ be open. We have to show that $f^{-1}(V)$ is open in $X$. Let $V' = V \cap f(X)$ which is again open in $Y$. We have $f^{-1}(V) = f^{-1}(V')$. Thus we may assume that $V \subset f(X)$. This is essential for the proof because $f(f^{-1}(M)) = M$ if and only if $M \subset f(X)$.
It is a bit tricky to find a set to which we apply (2).
To prove that $f^{-1}(V)$ is open we show that the difference $D = f^{-1}(V) \setminus \operatorname{int} f^{-1}(V)$ is empty. Let $E = D \cup f^{-1}(V \setminus f(D))$. We have $E \subset f^{-1}(V)$, thus $\operatorname{int} E \subset \operatorname{int} f^{-1}(V)$, in particular $\operatorname{int} E \cap D = \emptyset$. We conclude
$$\operatorname{int} E \subset f^{-1}(V \setminus f(D)) .$$
Noting $f(D) \subset V$ we get
$$f(E) = f(D) \cup f(f^{-1}(V \setminus f(D))) = f(D) \cup( V \setminus f(D)) = V .$$
But by (2) we get
$$V = \operatorname{int} V = \operatorname{int} f(E) \subset f(\operatorname{int} E) \subset f(f^{-1}(V \setminus f(D))) = V \setminus f(D)$$
which is only possible when $f(D) = \emptyset$, i.e. $D = \emptyset$.
Remark. The assumption ''$f(X)$ open in $Y$'' can be relaxed a little. In fact, it suffices to assume that for any open $V \subset Y$, $f(X) \subset V$ or $V' = V \cap f(X)$ is open in $Y$.
Corollary 1. Let $f$ be surjective. If (2) is satisfied, then $f$ is continuous.
Corollary 2. Let $f$ be open. If (2) is satisfied, then $f$ is continuous.
The converse of the above theorem is false (see example 2). But again there is an interesting special case.
Lemma. Let $f$ be injective. If $f$ is continuous, then (2) is satisfied.
Proof. By (3) we have
$$f^{-1}(\operatorname{int} f(A)) \subset \operatorname{int} f^{-1}(f(A)) = \operatorname{int} A$$
which implies
$$\operatorname{int} f(A) \subset f(\operatorname{int} A) .$$
Best Answer
So you want to show $f: X \to Y$ is an open map iff $$\forall B \subseteq Y: f^{-1}[\partial B] \subseteq \partial f^{-1}[B]\tag{1}$$
where $\partial A$ denotes the boundary of a set $A$ in a space (also denoted by $\operatorname{Bd}(A)$ or $\operatorname{Fr}(A)$ in other texts).
So suppose $f$ is open and $B \subseteq Y$, and let $x \in f^{-1}[\partial B]$ (so $f(x) \in \partial B$), and we want to show that $x \in \partial f^{-1}[B]$, so let $O$ be an open neighbourhood of $x$. We need to show that $O$ intersects both $f^{-1}[B]$ and its complement. The set $f[O]$ is then open (as $f$ is open) and contains $f(x) \in \partial B$ so $f[O]$ intersects $B$ and $Y\setminus B$. This by definition means that for some $x' \in O$, $f(x') \in B$ and for some $x'' \in O$ we have $f(x'') \notin B$, and then $x' \in O \cap f^{-1}[B] \neq \emptyset$ and $x'' \in O \cap X\setminus f^{-1}[B] \neq \emptyset$ and as $O \ni x$ was arbitrary we've shown that $x \in \partial f^{-1}[B]$, and $(1)$ has been shown. Note that the proof really "writes itself": at each stage it's clear what the goal is and where to use the assumption of openness.
Now assume $(1)$ holds, and let $O$ be open in $X$. Note that $B:=f[O]$ is open in $Y$ iff $\partial B \cap B = \emptyset$ (we're looking for a connection between boundaries and openness to be able to use the assumption $(1)$). So suppose that intersection is not empty, so there is some $x \in O$ such that $f(x) \in f[O]$ and $f(x) \in \partial f[O]$. As $x \in f^{-1}[\partial B]$ by $(1)$ we have that $x \in \partial f^{-1}[B]$ and as $x \in O$, $O$ must intersect $f^{-1}[B]$ and also its complement, but the latter is nonsense, as $O \subseteq f^{-1}[B]$ by definition! So no point in $B \cap \partial B$ exists after all, and so $B$ is open (e.g. because $\partial B^\complement = \partial B \subseteq B^\complement$ so $B^\complement$ is closed).
This shows the equivalence.