"Suppose two objects collide and combine into a single object, will the total relativistic momentum and relativistic mass stay the same?"
The answer is "yes", or rather their sums over the system of bodies will stay the same, but I would counsel you to stop using the term relativistic mass. It's going out of use for a number of good reasons that I won't bore you with now.
$m \gamma c^2$ in which m is the body's mass (formerly called 'rest mass') represents the sum of the body's internal energy, $mc^2$ and its kinetic energy ($\gamma m - m)c^2$. So for a closed system its sum over the bodies of the system is conserved in collisions, elastic or inelastic. Think of $\gamma m$ as the body's total energy, expressed in mass units.
The beauty of it is that a body's total energy (divided by the mere constant, c) and the three components of its momentum $(m \gamma u_x, m \gamma u_y, m \gamma u_z)$ make up a 4-component vector (or 4-vector): $(m \gamma c, m \gamma u_x, m \gamma u_y, m \gamma u_z)$. So for a closed system, despite collisions elastic or inelastic, the vector sum of these vectors is conserved, that is the sums of each component separately. One conserved 4-vector deals with conservation of energy and conservation of momentum.
Note also that the modulus of the 4-vector, defined as $\sqrt{(m \gamma c)^2 - (m \gamma u_x)^2 - (m \gamma u_y)^2 - (m \gamma u_z)^2}$, is simply $mc$, the body's mass multiplied by the mere constant, c.
$m$ is a constant for the body (provided we don't tamper with the body, e.g. by changing its internal energy!) and doesn't vary from frame to frame. It is a Lorentz invariant. [Beware: the sum of the masses (rest masses) of bodies in a system has no obvious significance; it is certainly not the mass (rest mass) of the system!]
I've gone on longer than I should have done. It's all so wonderful. A classic introduction to Special Relativity, first rate on concepts, is Spacetime Physics by Taylor and Wheeler.
If net force and velocity are both along the $x$-axis, $F_y,F_z,v_y,v_z$ are 0 and there remains
$$
F = F_x = \partial_t \frac{m v_x}{\sqrt{1 - v^2/c^2}}~,
$$
where also $v = v_x$. This means
$$
F = \partial_t \frac{m v}{\sqrt{1 - v^2/c^2}} = \partial_t \frac{m}{\sqrt{\frac 1 {v^2} - \frac 1 {c^2}}} \overset{\text{chain rule}}= - \frac{m}{2\sqrt{\frac 1 {v^2} - \frac 1 {c^2}}^3} \partial_t \left( \frac{1}{v^2} - \frac{1}{c^2} \right) \overset{\text{again chain rule}}= - \frac{m}{2\sqrt{\frac 1 {v^2} - \frac 1 {c^2}}^3} \left( \frac{-2}{v^3} \right) \underbrace{\partial_t v}_{=a}~.
$$
Multiplying the $v^3$ back into the root and canceling out the signs and the twos yields the result from the textbook. The trick here is to initially multiply both the denominater and the numerator with $1/v$, which leaves the fraction unchanged but reduces the number of occurances of $v$.
Best Answer
A proper derivation of the formula will require knowledge of Noether's theorem which states that for every symmetry of an action, there exists a quantity that is conserved. In particular, momentum is the conserved quantity corresponding to spatial translations.
The dynamics of particle in special relativity is described by the action $$ L = - m \sqrt{1 - {\dot x}^2 }~. $$ This action is invariant under $x(t) \to x(t) + \epsilon$. The corresponding conserved quantity is then $$ p = \frac{1}{ \epsilon } \frac{ \partial L }{ \partial{\dot x } } \delta x = \frac{ m {\dot x} }{ \sqrt{ 1 - {\dot x}^2 } }~. $$
A similar derivation maybe done in Newtonian mechanics where the action is $$ L = \frac{1}{2} m {\dot x}^2 ~. $$ Then, the momentum is $$ p = \frac{1}{ \epsilon } \frac{ \partial L }{ \partial{\dot x } } \delta x = m {\dot x} ~. $$ which is the usual definition of momentum.