This is possibly a duplicate of this question of mine, however, here I ask for clarification regarding an estimation that is done when calculating the expectation of the kernel density estimator (KDE) using little-o. The conditions on the KDE stated here are inspired by an exercise in an undergraduate textbook.
Background
Suppose $x_1, …, x_n$ are independent and identically distributed observations of a random variable $X$ with unknown distribution function $F$ and probability density function $f\in C^m$, for some $m>1$ fixed. Let $k\in C^{m+1}$ be a given fixed function such that
\begin{align}
k&\geq 0, \\
\mathrm{supp} (k)&=[-1,1], \\
\int_{\mathbb{R}} k(u)\mathrm{d}u&=1, \\
\int_{\mathbb{R}} k(u)u^l\mathrm{d}u&=0 \ \text{for all} \ 1\leq l<m \ \text{and}\\
\int_{\mathbb{R}} k(u)u^m\mathrm{d}u&<\infty .
\end{align}
Define the KDE $f_n$ of $f$ by $$f_n(t)=\frac{1}{n}\sum_{i=1}^n \frac{1}{h}k\left(\frac{t-x_i}{h}\right),$$
where $h=h(n)$ is the bandwidth. What is the expectation of $f_n$, i.e. $\mathbb{E}[f_n(t)]$?.
By linearity of the expectation, identical distribution of $x_1,…,x_n$, the law of the unconscious statistician and the change of variables $u=(t-x)/h$,
\begin{align}
\mathbb{E}[f_n(t)]&=\frac{1}{n}\sum_{i=1}^n \mathbb{E}\left[\frac{1}{h}k\left(\frac{t-x_i}{h}\right)\right]\\
&=\mathbb{E}\left[\frac{1}{h}k\left(\frac{t-x}{h}\right)\right]\\
&=\int_{\mathbb{R}}\frac{1}{h}k\left(\frac{t-x}{h}\right)f(x)\mathrm{d}x\\
&=\int_{\mathbb{R}}\frac{1}{h}k(u)f(t-hu)h\mathrm{d}u\\
&=\int_{\mathbb{R}}k(u)f(t-hu)\mathrm{d}u. \tag{1}
\end{align}
From $f\in C^m$, it follows that $$f(t-hu)=\sum_{l=0}^m \frac{f^{(l)}(t)}{l!} (-hu)^l+o((hu)^m). \tag{2}$$
$o(g(y))$ is a set (or a function) such that $f(y)\in o(g(y))$ (or $f(y)=o(g(y))$) satisfies $\lim_{y\to y_0} f(y)/g(y)=0$ for $y_0$ denoting a real number, a complex number or $\pm \infty$. In $(2)$, $y_0=0$. From $(1)$, $(2)$ and linearity of integration,
\begin{align}
\mathbb{E}[f_n(t)]&=\int_{\mathbb{R}}k(u)\left(\sum_{l=0}^m \frac{f^{(l)}(t)}{l!} (-hu)^l+o((hu)^m)\right)\mathrm{d}u \\
&=\sum_{l=0}^m\int_{\mathbb{R}}k(u)\frac{f^{(l)}(t)(-hu)^l}{l!}\mathrm{d}u+\int_{\mathbb{R}}k(u)o((hu)^m)\mathrm{d}u. \tag{3}
\end{align}
From the given conditions on $k$, the $l=0$ term reads
$$\int_{\mathbb{R}} k(u)f(t)\mathrm{d}u=f(t)\int_{\mathbb{R}} k(u) \mathrm{d}u=f(t).$$
The $1\leq l<m$ terms are
$$\int_{\mathbb{R}} k(u)\frac{f^{(l)}(t)}{l!} (-hu)^l\mathrm{d}u=\frac{f^{(l)}(t)(-h)^l}{l!}\int_{\mathbb{R}} k(u)u^l\mathrm{d}u=0.$$
Finally, the $l=m$ term is $$ \frac{f^{(m)}(t)(-h)^m}{m!}\int_{\mathbb{R}} k(u)u^m\mathrm{d}u<\infty.$$
From the definition of $o(g(y))$ given above, $o((hu)^m)$ denotes a function (the remainder) of $h$ and $u$ that for small $hu$, i.e. $hu\to 0$, approaches $0$ faster than $(hu)^m$. The remainder appears under the integral sign of an improper integral, which is the limit of a definite integral. For finite $u$, $hu\to 0$ means $h\to 0$. Thus the remainder is not only in $o((hu)^m)$ but also non-uniformly in $o(h^m)$, that is, for the remainder it holds that $o((hu)^m)=u^mo(h^m)=o(h^m)$.
Question
In the answer to the above linked question it is claimed, with slight modification, that
\begin{equation}
\int_\mathbb{R} k(u) o((hu)^m)\mathrm{d}u = \int_\mathbb{R} k(u) o(h^m)\mathrm{d}u =o(h^m)\int_\mathbb{R} k(u) u^m\mathrm{d}u =o(h^m) \tag{4},
\end{equation}
but if the remainder is non-uniformly in $o(h^m)$, then the last two equalities in $(4)$ may not hold. The following example shows how a similar reasoning may fail.
For each positive $a$ and $x$ near $0$,
\begin{equation}
g(x,a)=\frac{x^2}{a^2+x^2}\in o\!\left(x^{3/2}\right). %\ \text{for} \ x \ \text{near} \ 0.
\end{equation}
Define
\begin{equation}
f(x)=\int_0^1g(x,a)\,\mathrm{d}a.
\end{equation}
Is $f(x)\in o\!\left(x^{3/2}\right)$? It is tempting to reason as in $(4)$;
\begin{equation}
\int_0^1g(x,a)\,\mathrm{d}a=\int_0^1o\!\left(x^{3/2}\right)\mathrm{d}a=o\!\left(x^{3/2}\right).
\end{equation}
However, $\lim_{x\to0}f(x)/x=\pi/2$, which means that $f(x)\not\in o\!\left(x\right)\supseteq o\!\left(x^{3/2}\right)$.
So, for some $g(h,u)\in o(1)$1,
$$\int_\mathbb{R} k(u) o((hu)^m)\mathrm{d}u= h^m \int_\mathbb{R} k(u) u^m g(h,u)\mathrm{d}u,$$
but without knowing how $g(h,u)$ behaves away from zero, it seems like no further estimates can be done. Is it possible to calculate the expectation of the KDE using little-o?
Footnotes:
- The notation $g(h,u)$ implies the notation $g(hu)$. Unlike $g(hu)$, $g(h,u)$ includes those functions in $o(1)$ where $h$ and $u$ not only appear as $hu$.
Best Answer
Here is a suggested solution:
The integration occurs over $[-1,1]$ due to $\mathrm{supp}(k)=[-1,1]$ and $f$ being a probability density function, i.e. it integrates to $1$ over $\mathbb{R}$ and is thus bounded.
Instead of
write,
Regarding the integral with the remainder, note that $o((hu)^m)$ does not specify the remainder; it only specifies that the remainder converges to $0$ faster than $(hu)^m$ as $hu\to 0$. $h$ is the free variable while $u$ is the bounded variable; $hu\to 0$ means $h\to 0$. Thus the remainder satisfies $o((hu)^m)=u^mo(h^m)=o(h^m)$. If the remainder is only a function of $h$, the estimation of the integral is straightforward. If it also depends on $u$, then the integral can be estimated if the remainder, viewed as a sequence of functions, converges uniformly to $0$1. Then \begin{equation} \int_{[-1,1]} k(u) o((hu)^m)\mathrm{d}u =\int_{[-1,1]} k(u) o(h^m) \mathrm{d}u=o(h^m). \end{equation}
Footnotes: