The dominant hypothesis regarding the formation of the Moon is that a Mars-sized object collided with the proto-Earth 4.5 billion years ago. The Earth is rotating now because of that collision 4.5 billion years ago.
As the linked question shows, angular momentum is a conserved quantity. Just as something has to happen to make a moving object change its linear momentum, something has to happen to make a rotating object change its angular momentum. That "something" is called force in the case of linear momentum, torque in the case of angular momentum.
External torques do act on the Earth. Tidal forces transfer angular momentum from the Earth's rotation to the Moon's orbit. The Moon formed fairly close to the Earth shortly after that giant impact 4.5 billion years ago, and a day was probably only four to six hours long back then. By a billion years ago, the Moon had retreated significantly and the Earth had slowed down so that a day was 18 to 21 hours long. The Earth has continued slowing down, and will continue to do so.
If those external torques didn't exist we would still have a fast spinning Earth.
Whether the sun "rises" in the east depends on your position on earth, and the time of the year. In northern latitudes, during the summer, the sun rises significantly North of East, and in the winter it rises in the South. For example, today's sunrise/sunset directions in Umeå Sweden, look like this (source: www.suncalc.net)
The yellow line shows the direction of sunrise, the orange line the current direction of the sun, and the red line the direction at sunset.
As you can see, the sun never gets close to being in the East...
The reason for this is the fact that the earth's axis of rotation is not perpendicular to its plane of rotation about the sun (the ecliptic). The same mechanism that causes summer and winter in the higher latitudes gives rise to this changing direction.
But as for the fundamental question: the rotation of the earth about its axis is much faster than the rotation of the earth about the sun - so the rotation of the earth is dominating the direction of the sunrise. Now if the earth stopped rotating altogether, the sunrise would be in the "Westerly direction", since the direction of both rotations (seen from say the North Star) is in the counterclockwise direction. It would just rise and set only once a year... But the year is 365.24 days, and while that is so, the sun will rise "mostly in the east".
A small addendum: because the earth's orbit is elliptical, its angular speed relative to the sun changes a little bit with the seasons. This is enough to make a sundial "off" by up to 15 minutes, depending on the time of year. This is captured in the "equation of time" and shown, for example, in this graph (from http://upload.wikimedia.org/wikipedia/commons/0/02/Tijdvereffening-equation_of_time-en.jpg):
And just for your amusement - on October 25th, the sun briefly rises in the South on Svalbard (Spitsbergen) before disappearing for the winter...
Best Answer
Because the rotation of the earth is very smooth and doesn't change, the centripetal acceleration we feel is very nearly constant. This means that the (small) centrifugal force from the rotation gets added to gravity to make up the "background force" we don't notice.
Earthquakes are not at all smooth and the accelerations involved are large and change direction a lot. This makes it easy to feel them.
Vi Hart has a good explanation here.