1)Definition: An inertial frame of reference is a frame of reference where Newton's first law applies (uniform motion if without external force).
Now if we have other frame of references that are moving relative to this inertial frame with
uniform relative velocities, then all the others are also called inertial frame of references.
2)Transformation between inertial reference frames:In Newtonian mechanics, the laws of physics are invariant under Galilean transformation. While in special relativity, the laws of physics are invariant under Lorentz transformation. The latter reduces to the former in classical limit.
What is the question that Mach tried to address in his principle?
Mach's principle isn't as clear as people suggest, but IMHO what it tries to address is inertia. Resistance to change-in-motion.
I mean, we know how to detect the inertial and non-inertial frames (by Newton’s law).
I "root for relativity", but I have to say this: an inertial frame isn't some actual thing that has an objective existence. The universe exists, you exist, the Earth exists. But an inertial reference frame is little more than a steady state of motion, and a non-inertial frame is little more than a changing state of motion.
Once this is understood we also see that due to the acceleration of a non-inertial frame pseudo forces appear.
Yes, fire your boosters and you're pressed back into your seat. Because of your inertia, and because of your changing state of motion. But in truth your seat is pushing into your back.
Since there is no privileged inertial frame the acceleration of a non-inertial frame is quite unique i.e., one and the same with respect to every inertial frame. Right?
There is a privileged frame of sorts, which is the CMB rest frame, see this question. It isn't an absolute frame in the strict sense, but you can use it to gauge your motion with respect to the universe, and the universe is as absolute as it gets.
So what extra does the Mach’s principle try to answer? I’m a little confused.
Like CuriousOne said, you're right to be confused. Because Mach's principle is contradicted by E=mc². Inertia doesn't depend on distant rotating stars, it depends on local physics here and now. A photon has energy E=hf and momentum p=hf/c. These are two measures of resistance to change-in-motion for a wave travelling linearly through space at c. You divide by c to go from one to the other. Then remember the wave nature of matter: when you trap that wave in a mirror-box, it increases the inertia of the system. Because mass is a measure of energy-content, like Einstein said, and you divide by c again to say how much mass there is. But all it really is, is resistance to change-in-motion for a wave going round and round at c. Open the box, and it's a radiating body that loses mass. That radiation "conveys inertia between the emitting and absorbing bodies". Catch it in another mirror-box, and you increase the mass of that system. Having said all that, check out this article where Mark Hadley says large-scale rotation could be the cause of CP violation. It isn't quite Mach's principle as we normally understand it, but it relates to what's in the Wikipedia article, and IMHO is very interesting.
Best Answer
As you say, there's a perfectly sensible operational definition of an inertial frame: it's one in which free particles move with constant velocity. Even in general relativity, it makes sense to talk about inertial frames, but only locally. To be precise, an inertial frame is well-defined only in an infinitesimal neighborhood of a spacetime point, although in practice it's a sensible approximation to extend such a frame to a finite neighborhood, as long as the size is small compared to any length scales associated with spacetime curvature.
The fact that there are inertial frames is essentially an axiom of general relativity. The theory is based on the idea that spacetime has a certain geometric structure, which allows for the existence of geodesics, along which free particles travel. Within a sufficiently small neighborhood the geodesics near a given point "look" to a good approximation like what you'd get in an inertial frame.
So there's not really a good answer to the question of why inertial frames exist: it's just part of the assumed framework of the theory. But that's not quite what you asked. You asked if there's a reason why a given frame S is inertial and a different frame S' isn't. It sort of depends on what you think would count as a reason. For a given spacetime geometry, the geodesics are well-specified (as solutions to a certain differential equation, or as curves that have certain geometrical properties). The inertial frames are the frames that make the geodesics look like straight lines. It's all terribly mathematically well-defined and self-consistent, but it may not have the intuitive feel of a "reason why."
You mention the possibility that the reason is "all the other stuff in the universe." As you may know, this idea has a noble pedigree: it goes by the name of Mach's principle. Einstein was apparently quite enamored of Mach's principle when he was coming up with general relativity, and he would probably have been very happy if the theory had the property that the inertial frames were determined by all the other matter in the Universe. But general relativity's relationship with Mach's principle is complicated and problematic, to say the least. For instance, good old flat Minkowski spacetime is a perfectly valid solution to the equations of general relativity. That solution has well-defined inertial frames, even though there is no matter around to "cause" them.