This is not unlike Francois' answer.
First, let's look at the naive description of the order of a zero/pole of a differential $\omega$ on $C$ at some point $p$: write $\omega = f(z)\;dz$ in terms of some local coordinate $z$ near $p$, and then define $\operatorname{ord}_p(\omega) = \operatorname{ord}_p(f)$. Explicitly, this says two things:
- Every differential is proportional to the differential of a local coordinate, locally;
- The differential of a local coordinate is regular where it is defined.
In order for that to make sense, you have to check that for any other local coordinate $w$, the ratio (derivative) $dw/dz$ is regular at $p$. Of course, some precise algebraic computation is necessary, but intuitively, this is just the statement that both $z$ and $w$ have "slope 1" at $p$, so are equal to order one.
The fact that you have to choose a local coordinate is what is troubling you (it also troubles me); it comes about because there is no impartial basis for comparison, like there is with rational functions, which you can just compare to the function 1. The way around this, which also frees you from coordinate choices, is to talk about the entire sheaf of differentials rather than individual differentials.
Let's define $\Omega_C$ to be the sheaf of regular (Kähler) differentials on $C$, as defined in any basic algebraic geometry book. You give me $\omega$, a rational section of $\Omega_C$, or in other words, an element of $\Omega_C(U)$ for some open set $U$, and we want to find its divisor. Here is how we restate the first part of the above computations:
- The fact that $\omega$ can be expressed locally in terms of rational functions means that $\Omega_C$ is a line bundle (which is is; the trivializations are choices of local coordinate).
What about the second part? Let's continue: a section of $\Omega_C(U)$ is the same thing as a map $\phi \colon \mathcal{O}_C|_U \to \Omega_C|_U$, sending the rational function 1 to the differential $\omega$. Suppose for the sake of argument that $\omega$ had only zeros but no poles; then around a point $p \in C \setminus U$, it would look like $z^n\;dz$, choosing a local coordinate, and therefore, the image of $\phi$ would look like the ideal $(z^n) \subset \mathcal{O}_{C,p}$. More intrinsically, the cokernel of $\phi$ would have length $n$ at $p$. Thus, the convention that $dz$ is regular means that:
- When $\phi$ is an inclusion of sheaves, the order of vanishing of $\omega$ at $p$ is the length of $\operatorname{coker}(\phi)$ at $p$.
Of course, $\omega$ has poles, since you said $C$ is proper. Thus, $\phi$ does not even extend to an inclusion of sheaves. However, we want to think of a pole of something as being like a zero of the inverse, and we know how to find zeros. Suppose that we extend $\phi$ as much as possible, so that its zeros lie in $U$, form the divisor of zeros:
$$D_Z = \operatorname{div}(\omega)_Z = \sum_p \ell(\operatorname{coker}(\phi)|_p)p$$
and replace $\phi$ by its induced map $\phi \colon \mathcal{O}_C(-D_Z)|_{U \setminus D_Z} \to \Omega_C|_{U \setminus D_Z}$. Then this new $\phi$ is an isomorphism, and we can invert it; the poles of $\phi$ are by definition the zeros of $\phi^{-1}$. The divisor of poles $D_P = \operatorname{div}(\omega)_P$, defined as the divisor of zeros of $\phi^{-1}$, is disjoint from $D_Z$, because the new $\phi$ already is an isomorphism on $D_Z$, so that, after twisting by $-D_P$, $\phi^{-1}$ extends to all of $C$ (you should convince yourself, by playing with DVR's, that it really does). Then
$$\operatorname{div}(\omega) = \operatorname{div}(\omega)_Z - \operatorname{div}(\omega)_P$$
is the canonically-defined divisor of $\omega$. The short definition of this divisor is therefore:
- $\operatorname{div}(\omega)$ is the unique divisor $D$ such that the induced map $\phi \colon \mathcal{O}_C(-D)|_U \to \Omega_C|_U$ extends to an isomorphism of line bundles.
This is what the divisor corresponding to a line bundle usually means. Note that none of this is particular to differential forms, but allows you to define the zeros and poles of any rational section of any line bundle.
Best Answer
Yes, the formula holds, even when the action is not free. Here is the principle of a proof for the case of a tame action (which covers the characteristic $0$ case). Denote by $\pi:X\to Y=X/G$ the quotient morphism.
First consider the exact sequence of $G$-sheaves on $X$
$$ 0 \to \pi^* \Omega^1_{Y/k} \to \Omega^1_{X/k} \to \Omega^1_{X/Y} \to 0$$
Twisting by $(\Omega^1_{X/k})^{\vee}$ gives
$$ 0 \to \pi^* \Omega^1_{Y/k} \otimes (\Omega^1_{X/k})^{\vee} \to \mathcal O_X \to \Omega^1_{X/Y} \otimes (\Omega^1_{X/k})^{\vee} \to 0$$
In other words, $\pi^* \Omega^1_{Y/k} \otimes (\Omega^1_{X/k})^{\vee}$ identifies with the ideal sheaf of the ramification locus which is, since the action is tame, $\mathcal O_X (-\sum_{x\in X} (e_x-1) x)$, where $e_x$ is the ramification index at $x$. So we get an isomorphism of $G$-equivariant invertible sheaves
$$ \Omega^1_{X/k} \simeq \pi^* \Omega^1_{Y/k} \otimes \mathcal O_X (\sum_{x\in X} (e_x-1) x)$$
Now consider any $G$-invariant divisor $D$ on $X$. By a local analysis it is easy to convince oneself that
$$\pi_*^G (\mathcal O_X (D) )\simeq \mathcal O_Y(\left[\frac{\pi_* D}{\# G}\right])$$
where $\pi_*^G$ is the functor push-froward and take the invariants, $[\delta]$ is the integral part of the divisor with rational coefficients $\delta = \frac{\pi_* D}{\# G}$, taken coefficient by coefficient.
If $K_X$ (resp. $K_Y$) is the canonical divisor of $X$ (resp. of $Y$) then we have
$$K_X = \pi^* K_Y +\sum_{x\in X} (e_x-1) x $$
so
$$\left[\frac{\pi_* K_X}{\# G}\right] = K_Y +\sum_{y\in Y} [1-\frac{1}{e_y}]y $$
hence $[\frac{\pi_* K_X}{\# G}]= K_Y$ and finally $\pi_*^G (\Omega_{X/k}^1) \simeq \Omega_{Y/k}^1$. Taking global sections on $Y$ gives finally :
$$H^0(X,\Omega_{X/k}^1)^G= H^0(Y,\Omega_{Y/k}^1)$$