I will denote by $C(\omega)=\{0,1,2,\dots\}\cup\{\omega\}$ the one-point compactification of the discrete space on the countable set $\{0,1,2,\dots\}$.
I.e. all points different from $\omega$ are isolated and neighborhoods of $\omega$ are precisely complements of finite subsets of $\{0,1,2,\dots\}$.
(This space is homeomorphic to the space $\{0\}\cup\{\frac1{n+1}; n=0,1,2,\dots\}$ with the topology inherited from real line. So this is simply a convergent sequence.)
Now I take $M=\{0,1\}\times C(\omega)$, where $\{0,1\}$ has the discrete topology. (I.e., $M$ is the topological sum of two copies of $C(\omega)$.) And I choose $p=(0,\omega)$.
It is easy to see that $M$ is a compact metric space.
(E.g. it is homeomorphic to $\{0,1\}\times(\{0\}\cup\{\frac1{n+1}; n=0,1,2,\dots\})$ with the topology induced by the usual Euclidean metric on $\mathbb R^2$.)
Note that every subset of $\{0\}\times C(\omega)$ is open in $M\setminus\{p\}$.
Let us define a map $f \colon M\setminus\{p\} \to M\setminus\{p\}$ by putting
$$f(0,2n)=(1,2n)\\
f(0,2n+1)=(0,n)\\
f(1,n)=(1,2n+1)\\
f(1,\omega)=(1,\omega).$$
This map is bijective and continuous, but the extension $\overline f\colon M\to M$ with $\overline f(0,\omega)=(0,\omega)$ is not continuous, since the sequence $x_n=(0,2n)$ converges to $(0,\omega)$ in $M$ but $\overline f(x_n)=(1,2n)$ converges to $(1,\omega)\ne \overline f(0,\omega)$.
Here's my attempt to sketch the above map (the two big arrows indicate the direction in which the sequences $C(\omega)\times\{0\}$ and $C(\omega)\times\{1\}$ converge):
It is perhaps worth mentioning that $f^{-1}$ is not continuous. The sequence $(1,2n)$ converges to $(1,\omega)$ but $f^{-1}(1,2n)=(0,2n)$ does not converge.
So this does not contradict Theorem 29.1 from Munkres, which was mentioned in comments.
Theorem 29.1. Let $X$ be a space. Then $X$ is locally compact Hausdorff if and only
if there exists a space $Y$ satisfying the following conditions:
- $X$ is a subspace of $Y$.
- The set $Y-X$ consists of a single point.
- $Y$ is a compact Hausdorff space.
If $Y$ and $Y'$ are two spaces satisfying these conditions, then there is a homeomorphism
of $Y$ with $Y'$ that equals the identity map on $x$.
The OP indicated that he was originally thinking about the case that $M=S^n$. Since $S^n$ is the one-point compactification of $\mathbb R^n$, from Theorem 29.1 and from the fact that every continuous bijection $\mathbb R^n\to\mathbb R^n$ is a homeomorphism (which was shown in this question) we get that in the case $M=S^n$ the result from the question is true.
A simpler example is just the identity from a discrete $\mathbb{R}$ to the usual $\mathbb{R}$, both are locally compact metrisable, any map from a discrete space is continuous. The compact neighbourhoods in the first space are only the finite ones, so there we do have that there is a local homeomorphism (the finite sets are homeomorphic in both spaces), but they're not neighbourhoods in the image.
Likewise, for your example, the compact neighbourhoods $[0,r]$ of $0$ in $[0,2\pi)$ do have compact homeomorphic images in $S^1$ but these are not neighbourhoods of $f(0)$ in $S^1$ any more.
To go from something like a local homeomorphism to global one, we need a stronger condition.
Something like: for every $x \in X$ and every compact neighbourhood $C$ of $x$ in $X$, we must have that $f[C]$ is a (necessarily homeomorphic) neighbourhood of $f(x)$ as well. And as we saw, this is by no means garantueed from just being continuous and bijective.
Best Answer
Let $(X,d_X)$ and $(Y,d_Y)$ be two metric spaces, with $X$ compact, and let $f:X\to Y$ be a continuous bijection.
Definition: The function $f$ is continuous if for every set $O \subset Y$ which is open in $Y$, $f^{-1}(O)$ is open in $X$.
Lemma: The function $f$ is continuous if and only if for every set $C \subset Y$ which is closed in $Y$, $f^{-1}(C)$ is closed in $X$.
Lemma: $\left(f^{-1}\right)^{-1}=f$
Since it is already given that $f$ is continuous and bijective, we only have to proof that $f^{-1}:Y \to X$ is continuous. By the first lemma it is enough to prove that, for every closed $C \subset X$, $\left(f^{-1}\right)^{-1}(C)$ is closed in $Y$. By the second lemma, we have that $\left(f^{-1}\right)^{-1}(C)=f(C)$. Hence we should prove that $f(C)$ is closed. This follows from the fact that $X$ is compact, so $C$ is also compact (closed subset of a compact space). It then follows that $f(C)$ is compact and hence closed.
This proves that $f$ is a homeomorphism.