What you write is more or less true. First note that it is not enough to assume that $f(x,t)$ is differentiable at $x_0$ (for (almost) all $t$), we need that there is a neighorhood $(x_0 - \varepsilon, x_0 + \varepsilon) =:I$ such that $f(\cdot, t)$ is differentiable in $x$ for all $x \in I$ and (almost) all $t \in A$, where the "exceptional null set" must not depend on $x$.
The assumption $|\partial f/\partial x (x,t)| \leq g(x)$ only makes sense in this setting, otherwise we could only require $x_0$ instead of $x$ and the proof also needs this assumption (see below).
Now, let $h_n \to 0$ with $h_n \neq 0$ for all $n$ and w.l.o.g. with $x_0 + h_n \in I$ for all $n$. Let
$$
F(x) := \int_A f(x,t) \, dt \text{ for } x \in I,\\
G(x) := \int_A \frac{\partial f}{\partial x}(x,t) \, dt \text{ for } x \in I.
$$
Here, the integral defining $F$ is interpreted as an improper Riemann integral (it is possible that this integral does not exist as a Lebesgue integral), but the integral defining $G$ is interpreted as a Lebesgue integral. If $t \mapsto \frac{\partial f}{\partial x}$ is Riemann integrable over every compact subinterval of $A$ (for example, if it is continuous in $t$), then it can also be interpreted as an improper Riemann integral, at least if we assume that the dominating function $g$ is (improper) Riemann integrable.
Observe that the assumptions guarantee that $F,G$ are well-defined. Measurability of $t \mapsto \frac{\partial f}{\partial x}(x,t)$ for each $x \in I$ is implied by the pointwise convergence $$\frac{\partial f}{\partial x}(x,t) = \lim_n \frac{f(x+h_n, t) - f(x,t)}{h_n},
$$
which will also be employed below.
We now have
\begin{eqnarray*}
\left|\frac{F\left(x_{0}+h_{n}\right)-F\left(x_{0}\right)}{h_{n}}-G\left(x_{0}\right)\right| & = & \left|\int_{A}\frac{f\left(x_{0}+h_{n},t\right)-f\left(x_{0},t\right)}{h_{n}}-\frac{\partial f}{\partial x}\left(x_{0},t\right)\,{\rm d}t\right|\\
& \leq & \int_{A}\left|\frac{f\left(x_{0}+h_{n},t\right)-f\left(x_{0},t\right)}{h_{n}}-\frac{\partial f}{\partial x}\left(x_{0},t\right)\right|\,{\rm d}t.
\end{eqnarray*}
Note that the integrands are measurable and that for (almost) all $t \in A$, the mean value theorem yields some $\xi_{n,t} \in I$ (even between $x_0$ and $x_0 +h_n$) such that
$$
\left|\frac{f\left(x_{0}+h_{n},t\right)-f\left(x_{0},t\right)}{h_{n}}\right|=\left|\frac{\partial f}{\partial x}\left(\xi_{n,t},t\right)\right|\leq g\left(t\right).
$$
This shows that the integrands in the integrals are Lebesgue-integrable and dominated by an integrable function. As the integrand in the last integral converges pointwise to $0$ for $n \to \infty$, the dominated convergence theorem yields
$$
\left|\frac{F\left(x_{0}+h_{n}\right)-F\left(x_{0}\right)}{h_{n}}-G\left(x_{0}\right)\right| \to 0,
$$
so that $F$ is differentiable (in $x_0$) with the expected derivative.
Okay, after writing an answer for a long time I retract my comment: I don't think it is possible to give a canonical (in a sense which I'll explain soon) useful meaning to an "improper integral" in a general setting. I'll leave the point where the my answer broke as a reference (the previous answer can be seen in the end of the post).
Firstly, one important observation (in everything that follows, "integrable" means with respect to the Lebesgue sense): if $f: \mathbb{R}^n \to \mathbb{R}$ is integrable, then it is improperly integrable and to the same value (also, it doesn't depend on how you go to infinity). More precisely, let $A_i$ be any increasing sequence of sets such that $\bigcup A_i=\mathbb{R}^n$. Then - if $f$ is integrable - we have
$$\lim \int_{A_i}f=\int_{\mathbb{R}^n}f .$$
This is a direct consequence of the dominated convergence theorem. The problem is when $f$ is not integrable (this is exactly like the contrast absolutely convergent/conditionally convergent. Even more so as we shall see.)
The case in $\mathbb{R}$ already shows that there is a huge issue: the "way" you take the limit matters (c.f. Cauchy principal value). Therefore, what makes sense to define is the following: having chosen an increasing sequence of sets $A_n$ such that $\bigcup A_n=X$ on a measure space $X$, let
$$\int_X^{\operatorname{imp}(A_n)}f:=\lim \int_{A_n}f.$$
This depends on the choice of $A_n$, as it should. It is in this sense that I said that the definition is not "canonical". It coincides with the Lebesgue integral of $f$ if it is integrable (again due to the dominated convergence theorem). Note however that the above definition cannot even define the improper integral $\int_{-\infty}^\infty f$ (in its usual definition $\int_a^\infty f+\int_{-\infty}^bf$) properly.
One important observation is that the way the sequence is conceived is important, since we are taking a limit. This is in strong relation to the way a series can conditionally converge and not absolutely converge (indeed, a series is just an integral on a countable set). Note however that there is the general concept of a summable family (c.f. here) which if inspected closely may resemble the definition I assembled below: in fact, it is precisely the definition below in the case where we take the discrete topology (the compact sets on the discrete topology are the finite sets) and the counting measure. The concept below seems to enjoy the same good property as the improper integral: it coincides (in $\mathbb{R}$, and I believe that also in any $\sigma$-compact space) with the integral if the function is integrable. However, it does not generalize the concept of improper integral as you wanted, since they don't coincide even in $\mathbb{R}$. It is "another" integral.
Consider a locally compact Hausdorff space $X$ and a regular measure $\mu$ on $X$. Considering $\leq$ the inclusion on the set $\mathcal{K}$ of compact subsets of $X$, we have that $\mathcal{K}$ is directed, since finite union of compact sets is compact. Hence, given a real function $f: X \to \mathbb{R}$, we have a net $\lambda: \mathcal{K} \to \mathbb{R}$ given by
$$\lambda_K:=\int_K f \, d\mu.$$
Then, define
$$\int_X^{\operatorname{imp}} f:=\lim \lambda. $$
This coincides with the improper Lebesgue integral in $\mathbb{R}$ (!!no!!): for example, Let $\int_{[0,\infty)}^{\operatorname{impR}}$ denote the improper Lebesgue integral. Suppose $\int_{[0,\infty)}^{\operatorname{impR}} f=L$. Let $\varepsilon>0$. There exists $A>0$ such that if $x>A$ then $|L-\int_{[0,x)} f|<\varepsilon/4$. Note that this implies that $\left|\int_{[x,y]} f\right| <\varepsilon/2$ for every $x,y>A$, since
$$\left|\int_{[x,y]} f\right| = \left|\int_{[0,y]}f-\int_{[0,x]}f\right| = \left|L-\int_{[0,x]} f + \int_{[0,y]} f-L\right| < \varepsilon/2. $$ Take $K=[0,A+1]$. Given any $K'>K$, then $K' \subset [0,B]$ for some $B>A+1.$ It follows that
$$\left|L-\int_{K'} f\right|=\left|L-\int_Kf+\int_K f-\int_{K'} f\right| \leq \varepsilon/2+\left|\int_{K'-K}f\right|$$ Then, if $K'> K$.... the argument broke here. This won't work. Just take a non-integrable function and consider $K'$ a compact set big enough adjoining only portions where $f$ is positive. So, the notions don't coincide necessarily. However, it coincides if $f$ is integrable.
Best Answer
I want to give an example that came up within my master thesis: integration of internal functions over convex domains in non-standard analysis.
The set of hyperreals ${^*\mathbb R}$ is an ordered field extending the reals which contains infinitesimal ($|x|<\tfrac 1n$ for all $n\in\mathbb N$) and unlimted numbers ($|x|>n$ for all $n\in\mathbb N$). An internal function ${^*f:{}^*\mathbb R \to {}^*\mathbb R}$ arises as a extension of a sequence $f_n$ of ordinary function to unlimited numbers. As an example
$$f_\epsilon(x) = \frac{1}{\sqrt{2\pi}\epsilon}\exp(-\frac{1}{2}\frac{x^2}{\epsilon^2})$$
can be extended to infinitesimal $\epsilon>0$ whereupon it acts like a the dirac delta, in the sense that for any compactly supported, smooth test function $\varphi$,
$$ \int f_\epsilon(x) \varphi(x) dx \simeq \varphi(0)$$
where $\simeq$ means that the quantities only differ by an infinitesimal amount. This type of integral is easier to rigorize because we integrate over a compact domain. However one would like to integrate over more general convex domains in $^*\mathbb R$.
There are some obvious problems. For example what should $\int_{\mathbb I} 1 dx$ be? Here $\mathbb I$ is the set of infinitesimals. There is no largest infinitesimal, yet at the same time the integral is 'obviously' bounded by any positive non-infinitesimal number.
A way out of this dilemma is, instead of letting the integral be a map onto $^*\mathbb R$, to consider a sort of fuzzy integral that maps a function onto an external number $\alpha \in \mathbb E$. An external number is nothing but a pair $(\alpha_0,\mathcal N)$, where $\alpha_0\in{^*\mathbb R}$ is the value of the integral and $\mathcal N\subset {}^*\mathbb R$ is sort of it's uncertainty. In the above case we'd have $\alpha_0 = 0$ and $\mathcal N = \mathbb I$.
Here the set $\mathcal N$ is always a so called neutrix, that is a convex additive subgroup of ${^*\mathbb R}$. These sets can have very peculiar properties. This theory of integration is called external integration and was developed by I. van den Berg and Fouad Koudjeti.
Neutrices, external numbers, and external calculus (unfortunately paywalled)
Some more background on external numbers can be found here
One of the the issues with this theory is however that it is rather complicated and only can handle non-negative functions. I believe that it should be possible to develop a 'neat' nonstandard integration theory that addresses these issues.
Finally I want to give a nice example of how one can intuitively work with nonstandard integration which is also from the cited book but translated into a more approachable language.
We use a nonstandard version of Laplace's method used to prove Stirling's formula $n!\sim \sqrt{2\pi n} (\frac ne)^n$: By the gamma function representation, $n! = \int_0^\infty t^n e^{-t}dx$. Instead of $n$, we insert an unlimited hypernatural number $N$. Note that the integrand has a unique global maximum at $t=N$. Doing a coordinate transformation $x=t/N-1$ we obtain
$$N! = \int_{-1}^\infty e^{-N(x+1)}N^{N+1} (x+1)^N dx = N^{N+1}e^{-N} \int_{-1}^\infty e^{-N(x-\log(1+x))}dx $$
Here note that the integrand is infinitesimal whenever $x-\log(1+x)$ is not infinitesimal itself. In fact the only non-negligible contribution can occur when $-N(x-\log(1+x)) \sim -\tfrac 12 Nx^2 +O(x^3)$ is not infinitesimal, which is the case when $x\in \frac{1}{\sqrt N} \mathbb L$, where $\mathbb L$ is the set of limited numbers. ($|x|<n$ for some $n\in\mathbb N$). Hence we can restrict the integral to this set. By doing so we make a slight error in the form of $N! = (1+\epsilon)\cdot\ldots$ with $\epsilon$ infinitesimal.
In fact an equivalence relation $\asymp$ is given on the hyperreals by $x\asymp y \iff x \in y(1+\epsilon)$ for some infinitesimal $\epsilon$.
$$ N! \asymp N^{N+1} e^{-N} \int_{\frac{1}{\sqrt N }\mathbb L} e^{-N(x-\log(1+x))}dx $$
Since this set is contained within the set of infinitesimals, we can replace $x-\log(1+x)$ by its first order Taylor approximation and only do a small error again:
$$ N! \asymp N^{N+1} e^{-N} \int_{\frac{1}{\sqrt N }\mathbb L} e^{-\tfrac 12 Nx^2}dx $$
Now we can to a retransformation $x = \frac{1}{\sqrt N}y$ leading to
$$ N! \asymp N^{N} e^{-N} \sqrt N \int_{\mathbb L} e^{-\tfrac 12 y^2}dy = \sqrt{2\pi N} N^N e^{-N} $$
which proves Stirlings formula as we have shown that for all unlimited $N$ there exists an infinitesimal $\epsilon$ such that
$$ \frac{N!}{\sqrt{2\pi N} (N/e)^N} = 1 + \epsilon$$