I have been trying to make an expression in terms of a “Dirac Delta function'' rigorous, but I failed miserably. Please help, if you can. References are welcome.
The motivation for this comes from calculating densities $\rho$ in a measure space $M$, if one is given `samples' $\alpha(\omega)$ indexed by $\omega$ in another measure space $\Omega$. This has applications in statistical physics, so perhaps people have done this already.
For starters, if $\left(\Omega, \mathcal A \right)$ is a measurable space and $\omega \in \Omega$, then the Dirac measure is
$$ \delta_\omega \colon \,
\mathcal A \mapsto \mathbb R \, , \quad A \to \delta_\omega\left(A \right) :=
\begin{cases}
1 \, , \, \omega \in A \\
0 \, , \,\omega \notin A
\end{cases} \, .
$$
I believe that the problem at hand requires this concept, and not the Dirac distribution.
Now let $(M,\mathcal A, \mu)$, $(\Omega,\mathcal B, \nu)$ be $\sigma$-finite measure spaces and let
$\alpha \colon \Omega \to \mathcal M$ be a measurable function. Then I want that for any $\theta \in M$ and any sequence of measurable sets $\left(N_n (\theta)\right)_{n \in \mathbb N}$ containing $\theta$, having finite measure and with
$$\lim_{n \to \infty} \mu \left(N_n (\theta)\right) = 0$$
that
$$\theta \mapsto \rho \left( \theta \right) =
\lim_{n \to \infty} \frac{\nu \left( \alpha^{-1}
\left(N_n (\theta)\right)\right)}{\mu \left(N_n (\theta)\right)}$$
is well-defined and measurable.
This is where the Dirac measure comes in: I'm quite sure that for another (arbitrary?) sequence $\left( A_n \right)_{n \in \mathbb N}$ of
measurable sets in $\Omega$ with finite measure and with
$$\lim_{n \to \infty} \nu \left( A_n^C \cap \Omega \right) = 0$$
(i.e. `$A_n$ converges to $\Omega$ in measure') that
$$\rho \left( \theta\right) \overset{!}{=} \lim_{n \to \infty}
\frac{\int_{A_n} d \nu \negthinspace \left( \omega\right) \, \, \delta\left( \theta – \alpha_{\omega}\right)}{\nu \left( A_n\right)}$$
The problem is 1) to turn this into a mathematically sensible expression in terms of the Dirac measure, and 2) to show equality with the above one. Note that neither $M$ nor $\Omega$ can be assumed to be finite measure spaces.
EDIT: Alright, let me clarify things a bit.
1) The above description is a formalization of the following problem: Say I decided to place an uncountable number of points in $M= \mathbb{R}^n$, but I decided to label them in a rather inconvenient manner. So for each label $\omega \in \Omega$ I have a point $\alpha_\omega \in M$. Now the problem is that, say, suddenly I don't care about the labels any more, but rather I want to know how these points are distributed in space with respect to some volume measure (e.g. the Lebesgue measure $\mu$). That is, I want a density.
So I want to integrate over my label space (each label has equal weight, so I have a measure $\nu$ on it) and only count those points in which I'm evaluating the density at. The above expressions are what I came up with.
2) The last equation is purely symbolic. The point is to "count" only those points in $\Omega$ for which $\theta = \alpha_\omega$. So it's not really useful to consider it as the Dirac delta of a function. The sequence of sets is only there, because $\nu(\Omega)$ may be infinite.
3) Yes, the first "definition" is probably problematic. For instance, what if all $\alpha_\omega$ are equal? We want to get a possibly singular measure out… Almost everywhere existence is perfect. Radon-Nikodym only works for $\Omega = M$, which is too restrictive.
Best Answer
I believe the following answers your question, even though it might not be immediately apparent to you.
Let $(\Omega,\mathcal{B},\nu)$ and $(M,\mathcal{A},\mu)$ be measure spaces (let's say $\Omega$ is actually a probability space) and let $\alpha:\Omega\rightarrow M$ be a measurable function. We can define a new probability measure on $M$, called the law or distribution of $\alpha$, as follows: the measure of any set $A\in\mathcal{A}$ is now defined to be the probability (according to $\nu$) that $\alpha(\omega)\in A$, in other words $$ \mathbb{P} (A) := \nu(\{\omega \in \Omega \vert \alpha(\omega) \in A\}).$$ This is well defined precisely because we assumed that $\alpha$ is measurable. Another way of writing this is to say that $\mathbb{P}(A) := \nu( \alpha^{-1} (A))$; yet another way of writing the same thing is $\mathbb{P} := \nu \circ \alpha^{-1}$. Another name for this measure is the pushforward of $\nu$ onto M with respect to $\alpha$, or just the pushforward to be short. (I'm saying all of this because the probability literature uses all of this interchangeably so you may as well know that ahead of time.)
Now let me attempt to convince you that the law of $\alpha$ is the thing that you actually want, rather than something to do with Dirac deltas.
Firstly, you say that your motivation is that you want to "place uncountably many points" in $M$ according to $\alpha$. Well, suppose that due to measuring precision, we cannot actually check "how many" of these points land at any specific $\theta\in M$ - rather, we have to look at a region of the space $A$ which has finite measure, and see what "fraction" of the points have ended up in $A$. But this is precisely $\nu(\{\omega \in \Omega \vert \alpha(\omega) \in A\})$.
More mathematically, it is good to develop the perspective, in probability, of not thinking in a totally "pointwise" fashion all the time. For one, lots of things are defined not pointwise but pointwise almost surely, so trying to stipulate "how many points land at a specific other point" is not likely to be well-defined. However, there is a special case where specifying a function "up to measure zero" actually gives you a unique pointwise function, namely if you also assume that the function has to be continuous. Likewise, there is a continuity assumption that allows you to think of the law of $\alpha$ as a pointwise function rather than a measure, namely the assumption that the law of $\alpha$ is absolutely continuous with respect to $\mu$, the underlying measure on $M$ - in this case you can express the law of $\alpha$ in terms of a distribution function, like so: $\mathbb{P}(A)= \int_A \rho(\theta) d\mu(\theta)$, for some measurable function $\rho$. However, it is emphatically not always the case that the law of $\alpha$ is absolutely continuous, even in physically realistic scenarios. Trying to think of the law of $\alpha$ off the bat in terms of a distribution function like $\rho$ is baking in an extraneous assumption to your problem, like assuming for no particular reason that the trajectory of a particle is differentiable.
Now, what the notion of "distribution" (in the sense of: Schwartz distribution, Dirac delta distribution) is doing is it relaxes our notion of function so that we can think of the law of $\alpha$ in terms of some such $\rho$, if we really insist. Even in this case, however, it is not always the case that a Schwartz distribution can be thought of as an "uncountable sum of deltas" as you seem to be trying to do. Therefore, it really is best practice to treat the law of $\alpha$ as the "primitive" object of study for your type of problem.
I hope that helps.
Addendum: now, you may ask, how do you actually compute the law of $\alpha$ in a reasonable way? The answer is that for some measurable functions, you can't, but the distribution function $\rho$ perspective also suffers from this problem. In reality, what you do is sample from $\alpha$ a (large) finite number of times, and hope that this procedure will allow you to approximate the law of $\alpha$ if you sample enough points (and you're not very unlucky). There are some general results in statistics concerning when/if this actually works; the keywords you should look for are that the sample gives you an empirical measure which converges in distribution to the law of $\alpha$.