Here's another example.
Consider $\mathbb{R}P^3$ in the model of a 3-ball with antipodal boundary points identified.
Consider the $G = \mathbb{Z}/2\mathbb{Z}$ action on $\mathbb{R}P^3$ given by the antipodal map sending $(x,y,z)$ to $-(x,y,z)$.
I claim that $\mathbb{R}P^3/G$ is homeomorphic to a cone on $\mathbb{R}P^2$. To see this, use coordinates on $C\mathbb{R}P^2$ given by $([x,y,z],t)$ where we think of $(x,y,z)\in\mathbb{R}^3$ and we're collapsing $\mathbb{R}P^2\times\{0\}$ to a point. Now,map $(x,y,z)$ in $\mathbb{R}P^3$ to $\big([x,y,z], (x^2+y^2+z^2)\big)$ (and map the origin to the cone point). This is clearly continuous away from the origin. It's not too hard to see that it's continuous at the origin as well.
It's also not hard to see that this descends to a bijective map from $\mathbb{R}P^3/G$ to $C\mathbb{R}P^2$, which is therefore a continuous bijection between compact Hausdorff spaces, so is itself a homeomorphism.
Finally, notice that $C\mathbb{R}P^2$ is not a topological manifold (with or without boundary) because of the cone point $p$. A neighborhood $U$ of the cone point $p$ has, by excision, $H_k(U, U-p)\cong H_k(C\mathbb{R}P^2, C\mathbb{R}P^2-p) = H_{k-1}(\mathbb{R}P^2)$, which means $p$ can be neither a manifold point nor a manifold-with-boundary boundary point.
If $G$ is a topological group acting on a topological space $M$, the usual definition is that the action is proper if the map $G\times M\to M\times M$ defined by $(g,x)\mapsto (g\cdot x,x)$ is a proper map, which means that the preimage of every compact set is compact. This definition works both in the topological category and in the smooth category.
For sufficiently nice spaces, there are other characterizations that are often useful. For example:
- If $M$ is Hausdorff, then properness is equivalent to the condition you described.
- If $M$ and $G$ are Hausdorff and first-countable, then properness is equivalent to the following condition: If $(x_i)$ is a sequence in $M$ and $(g_i)$ is a sequence in $G$ such that both $(x_i)$ and $(g_i\cdot x_i)$ converge, then a subsequence of $(g_i)$ converges.
Actions of compact groups (on Hausdorff spaces) are always proper, so properness really has meaning only for actions of noncompact groups. In this case, properness has the intuitive meaning that "most" of $G$ (i.e., all but a compact subset) moves compact sets of $M$ far away from themselves.
You can find more information on group actions in the topological category in my Introduction to Topological Manifolds (2nd ed.), and in the smooth category in Introduction to Smooth Manifolds (2nd ed.), among other places.
Best Answer
Using the Iwasawa decomposition, any matrix in the special linear group $SL(2,\mathbb{R})$ can be uniquely represented as $M=KAN$, where $k$ belongs to the subgroup $K=SO(2)$, $A\in\left\{\begin{pmatrix}\lambda&0\\0&1/\lambda\end{pmatrix},\ \lambda>0\right\}$ and $N\in\left\{\begin{pmatrix}1&a\\0&1\end{pmatrix},\ a\in\mathbb{R}\right\}$.
An alternative definition of a proper action of a Lie group $G$ on a manifold $M$ is as follows: for any convergent sequences $(x_n)$ and $(g_n\cdot x_n)$ (in $M$), we can extract a convergent subsequence $(g_{\varphi(n)})$ in $G$.
Since $SO(2)$ is a compact group, we can assume that $(z_n)$ and $(A_nN_n\cdot z_n)$ are convergent sequences. Here, $z_n=x_n+iy_n$ where $(x_n)$ and $(y_n)$ are convergent sequences in $\mathbb{R}$ and $]0,+\infty[$ respectively. The convergence of $$A_nN_n\cdot z_n=\begin{pmatrix}\lambda_n&0\\0&1/\lambda_n\end{pmatrix}\begin{pmatrix}1&a_n\\0&1\end{pmatrix}\cdot z_n=\lambda_n^2(z_n+a_n)=\lambda_n^2(x_n+a_n)+i\lambda_n^2y_n$$ implies that $(\lambda_n)$ is a convergent sequence in $(0,+\infty)$, and therefore, $a_n=(x_n+a_n)-x_n$ is also a convergent sequence.