Note that in general you cannot simply replace arbitrary quantities by infinitesimally equivalent ones. For example, $\lim_{x \to 0} \frac{\sin(x)-x}{x^3}$ is not zero, which is what you would get if you replaced $\sin(x)$ by $x$ in it. In most cases what you are doing when you replace infinitesimally equivalent quantities is multiplying the final result by $1$, writing $1$ as a limit of a ratio of infinitesimally equivalent quantities, and then dragging this limit inside your original one. So for example in the first problem in the image, the steps look like:
$$\lim_{x \to 0} \frac{\ln(1+4x)}{\sin(3x)}=\lim_{x \to 0} \frac{\ln(1+4x)}{\sin(3x)} \lim_{x \to 0} \frac{4x}{\ln(1+4x)} \lim_{x \to 0} \frac{\sin(3x)}{3x}=\lim_{x \to 0} \frac{4x}{3x}=4/3.$$
I am assuming that the discussion is about time-homogeneous Markov chains: in any case, let me consider that case, since it is easier and the general case does not really add much to the discussion.
Let $$p_{i,j}(h) = \Pr(X(t+h)=j \mid X(t)=i).$$
Since we have assumed that the chain is time-homogeneous, this quantity does not depend on $t$. This is the transition probability of going from state $i$ to state $j$ after time $h$. Note that trivially $p_{i,j}(0) = \delta_{i,j}$.
The transition rate $q_{i,j}$ can be defined as the derivative at zero of this function: $q_{i,j} := \frac{d}{dh} p_{i,j}(h)|_{h=0}$. Note that one of the equivalent ways of defining the derivative at zero is using the $o(h)$ notation:
$q_{i,j}$ is the only constant you can multiply $h$ by which makes the following true
$$ p_{i,j}(h) = p_{i,j}(0) + q_{i,j}h + o(h).$$
One has to be careful now in that $q_{i,j}$ is expressing the transition rate. In general the function $p_{i,j}(h)$ will not be linear, so it is not true that $q_{i,j} h$ is the transition probability after time $h$, which as we said is given by $p_{i,j}(h)$. The reason is that during that interval of time, if we had transitioned from $i$ to $j$ we could as well have transitioned away to some other state, or we could have transitioned first to some third state and then to $j$, so $p_{i,j}(h)$ could be smaller or larger than $q_{i,j}h$.
What it is true is that, if we know all the transition rates $q_{i,j}$ for all $i$ and $j$, we can recover $p_{i,j}(h)$. If we now denote by $Q=(q_{i,j})_{i,j}$ the matrix of transition rates, and by $P(h) = (p_{i,j}(h))_{i,j}$ the time-dependent matrix of transition probabilities, then this is the solution of the first-order differential equation
$$ P'(h) = QP(h) $$
with initial condition $P(0) = (\delta_{i,j})_{i,j}$ is the identity matrix.
Best Answer
You ask "why Cauchy's definition of infinitesimal, along with his 'basic approach' was superseded?"
The answer is that Cantor, Dedekind, Weierstrass and others developed a foundation for analysis to deal with certain difficulties related to Fourier series, uniform continuity, and uniform convergence. This development resulted in a formalisation that was a decisive moment in the history of analysis and was a great accomplishment. This is all well-known and rivers of ink have been spilt on the subject.
Yet the great accomplishment masked a significant failure that is less frequently spoken about. Namely, these 19th century giants failed to formalize an aspect of the procedures of calculus and analysis that was ubiquitous until and including Cauchy, namely the notion of infinitesimal. Instead, they provided infinitesimal-free paraphrases for the traditional definitions. For example, Cauchy's lucid definition of continuity of $y=f(x)$ ("infinitesimal change in $x$ always leads to infinitesimal change in $y$") got replaced by the familiar jargon ("for every epsilon there exists a delta such that, if $|x-c|$ is less then delta, then $|f(x)-f(c)|$, etc.").
Not only did they fail to formalize it but, unable to do so, some of them became convinced that there was something wrong with the notion of infinitesimal itself, and from this jumped to the conclusion that infinitesimals must be inconsistent or self-contradictory. Cantor went as far as publishing an article claiming to "prove" that infinitesimals were inconsistent. In correspondence Cantor referred to infinitesimals as "paper numbers", "cholera bacillus of mathematics", and even "abomination"; the details can be found in
Dauben, Joseph Warren. Georg Cantor. His mathematics and philosophy of the infinite. Princeton University Press, Princeton, NJ, 1990
and
Ehrlich, Philip. The rise of non-Archimedean mathematics and the roots of a misconception. I. The emergence of non-Archimedean systems of magnitudes. Arch. Hist. Exact Sci. 60 (2006), no. 1, 1–121.
A solid set-theoretic formalisation for infinitesimals did not emerge until around 1960 and by then Weierstrassian paraphrases were solidly in place, making it difficult to overcome institutional inertia.
Cauchy's idea of representing an infinitesimal by a sequence tending to zero is basically valid, but needs some polishing. Cauchy's infinitesimal specifically is dealt with in a number of articles that you can find here.