I have been reading some non-standard analysis from Keisler's book and I think it is logically consistent till now but there are criticisms against it and why isn't non-standard analysis accepted more widely, the whole book is almost similar to the concept of limit. I have read Apostol's books earlier . Then why aren't we taught only the infinitesimal calculus only and what is special about the $\epsilon – \delta$ approach ?
[Math] Infinitesimal calculus
calculusinfinitesimalsnonstandard-analysis
Best Answer
As far as teaching the calculus is concerned, infinitesimals are useful in explaining concepts such as derivative, integral, and even limit. That's why Kathleen Sullivan's controlled study of infinitesimal and epsilontic methodologies in the 1970s revealed that students taught using infinitesimals possess better conceptual understanding of the fundamental concepts of the calculus; see here and here. (A PDF copy can be found here, through an related page Calculus with Infinitesimals that the OP may be interested in.)
Once the students have mastered the key concepts, one can explain the epsilon, delta definitions in an accessible way (the students already understand what the definition is trying to tell us). But one can't do away with $(\epsilon, \delta$)-type definitions altogether. For example, Keisler's proof of the ratio test on page 524 exploits the $\epsilon, N$ definition.
So we still need these definitions, even in the context of teaching infinitesimal calculus. Furthermore, they are needed when developing the foundations of analysis, for example to define Cauchy sequences or rationals, etc.
As far as your subsidiary question "why isn't non-standard analysis accepted more widely", this is a larger problem that should perhaps be treated in a separate question if you are still interested in the issue.