There is a problem with your proof: The function $f(z)^n/z$ is not holomorphic in $\Omega$ if $0 \in \Omega$, and it is not bounded on the boundary of $V \cup \Omega$ if $0 \in \Gamma$.
If $0 \in \Bbb C \setminus \overline \Omega$ then $\delta = \operatorname{dist}(0, \Gamma) > 0$, and your approach works with the estimate
$$
\left|\frac{f(z_0)^n}{z_0}\right| \leq \max\left(\frac{B^n}{r}, \frac{M^n}{\delta}\right) \, .
$$
If there is any other point $z_1$ in $\Bbb C \setminus \overline \Omega$ then the same idea works by considering $f(z)^n/(z-z_1)$ instead.
But all this fails if $\overline \Omega = \Bbb C$, i.e. if there is no open disk in the complement of the domain. In that case one has to replace $1/z$ by a different function, see below.
Also note that the statement is wrong if the boundary of $\Omega$ is empty, i.e. if $\Omega = \Bbb C$. Then the condition “$|f|\le M$ on $\Gamma$” holds for arbitrary values of $M$ (as a “vacuous truth”), but that does not imply that $|f| \le M$ in $\Omega$.
A correct formulation of the statement would be:
Let $\Omega \subset \Bbb C$ be a domain with non-empty boundary $\Gamma$, and $f: \Omega \cup \Gamma \to \Bbb C $ a continuous function which is holomorphic in $\Omega$. Suppose there are real constants $M$ and $B$ such that
- $|f(z)| \le M$ for all $z \in \Gamma$, and
- $|f(z)| \le B$ for all $z \in \Omega$.
Then $|f(z)| \le M$ for all $z \in \Omega$.
(The following proof is taken from Bak J., Newman D.J. (2010) Maximum-Modulus Theorems for Unbounded Domains. In: Complex Analysis. Undergraduate Texts in Mathematics. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-7288-0_15 .)
We may assume that $f$ is not constant, because the statement is trivially true for constant functions. (Why?)
Fix $z_0 \in \Bbb \Omega$ and a positive integer $n$. We choose any $z_1 \in \Omega$ with $f(z_1) \ne f(z_0)$ and define the function $g$ on $\Omega \cup \Gamma$ as
$$
g(z) = \begin{cases}
\frac{f(z)-f(z_1)}{z-z_1} & \text{ if } z \ne z_1 \, , \\
f'(z_1) & \text{ if } z = z_1 \, .
\end{cases}
$$
$g$ replaces the function $1/z$ from your proof. It has the following properties:
- $g$ is holomorphic in $\Omega$ and continuous on $\Omega \cup \Gamma$.
- $\lim_{z \to \infty} g(z) = 0$ (because $f$ is bounded).
- $g$ is bounded on $\Omega \cup \Gamma$.
We set $K = \sup \{z \in \Omega \cup \Gamma : |g(z)| \}$ and choose $r> 0$ so large such that
- $|z_0| < r$, and
- $|g(z)| < K M^n/B^n$ for all $z \in \Omega$ with $|z| = r$.
Now we can apply the maximum modulus theorem to $h(z) = f(z)^n g(z)$ on $\Omega \cap B_r(0)$. On boundary points $z$ with $|z| = r$ we have
$$
|f(z)^n g(z)| \le B^n |g(z)| \le M^n K
$$
and on boundary points $z \in \Gamma$
$$
|f(z)^n g(z)| \le M^n K
$$
holds as well. It follows that
$$
|f(z_0)| \le M \left( \frac{K}{|g(z_0)|}\right)^{1/n} \, ,
$$
and for $n \to \infty$ we conclude that $|f(z_0)| \le M$.
Question to the reader: Where does the proof use that the boundary of the domain is not empty?
Best Answer
Yes, in b., there should be the $L$ you mention. It however doesn't change the result, because following the hint we get $$|f(a)|\leqslant M,$$ hence $M$ is also the maximum for all the points which are enclosed by $C$.
Maximum modulus principle is true for domains (that is, connected open sets).