Taken from Wikipedia (here), a scoring rule has the following definition
Let $\Omega$ be a sample space, and $\mathcal{A}$ is a $\sigma$-algebra of subsets of $\Omega$. Let $\mathcal{P}$ be a convex class of probability measures on $(\Omega, \mathcal{A})$. A scoring rule is a function $S: \mathcal{P} \times \Omega \to \overline{\mathbb{R}}$, where $\overline{\mathbb{R}} := \mathbb{R} \cup \{\pm \infty\}$, such that the integral of $S$ on $\Omega$ exists.
What do they mean by "convex class of probability measures"? Do they mean that the probability measures are convex? If a probability measure are (uniquely) identified by the distribution function, does that measure belong to the class $\mathcal{P}$?
All in all, I am just not sure what "convex class of probability measures" is.
Best Answer
To say that a set $\mathcal{P}$ of probability measures on $(\Omega, \mathcal{A})$ is convex (more standard than saying `a convex class') just means that for all $P, Q \in \mathcal{P}$, and all $\alpha \in [0,1]$, the probability measure $\alpha P + (1-\alpha)Q$ is a member of $\mathcal{P}$.
Recall that $\alpha P + (1-\alpha)Q$ is defined pointwise: for any $A \in \mathcal{A}$, $$(\alpha P + (1-\alpha)Q)(A) := \alpha P(A) + (1-\alpha)Q(A)$$
A bit more generally, convex sets are usually taken to be convex subsets of some ambient vector space, so that convexity is spelled out in terms of the vector space operations.
In this case, the vector space could be the vector space of signed measures on $(\Omega, \mathcal{A})$ with addition and scalar multiplication defined pointwise. So spelling things out, $\mathcal{P}$ is a convex class of probability measures if it contains only probability measures and is a convex subset of that vector space.