Using your definition of "falling," heavier objects do fall faster, and here's one way to justify it: consider the situation in the frame of reference of the center of mass of the two-body system (CM of the Earth and whatever you're dropping on it, for example). Each object exerts a force on the other of
$$F = \frac{G m_1 m_2}{r^2}$$
where $r = x_2 - x_1$ (assuming $x_2 > x_1$) is the separation distance. So for object 1, you have
$$\frac{G m_1 m_2}{r^2} = m_1\ddot{x}_1$$
and for object 2,
$$\frac{G m_1 m_2}{r^2} = -m_2\ddot{x}_2$$
Since object 2 is to the right, it gets pulled to the left, in the negative direction. Canceling common factors and adding these up, you get
$$\frac{G(m_1 + m_2)}{r^2} = -\ddot{r}$$
So it's clear that when the total mass is larger, the magnitude of the acceleration is larger, meaning that it will take less time for the objects to come together. If you want to see this mathematically, multiply both sides of the equation by $\dot{r}\mathrm{d}t$ to get
$$\frac{G(m_1 + m_2)}{r^2}\mathrm{d}r = -\dot{r}\mathrm{d}\dot{r}$$
and integrate,
$$G(m_1 + m_2)\left(\frac{1}{r} - \frac{1}{r_i}\right) = \frac{\dot{r}^2 - \dot{r}_i^2}{2}$$
Assuming $\dot{r}_i = 0$ (the objects start from relative rest), you can rearrange this to
$$\sqrt{2G(m_1 + m_2)}\ \mathrm{d}t = -\sqrt{\frac{r_i r}{r_i - r}}\mathrm{d}r$$
where I've chosen the negative square root because $\dot{r} < 0$, and integrate it again to find
$$t = \frac{1}{\sqrt{2G(m_1 + m_2)}}\biggl(\sqrt{r_i r_f(r_i - r_f)} + r_i^{3/2}\cos^{-1}\sqrt{\frac{r_f}{r_i}}\biggr)$$
where $r_f$ is the final center-to-center separation distance. Notice that $t$ is inversely proportional to the total mass, so larger mass translates into a lower collision time.
In the case of something like the Earth and a bowling ball, one of the masses is much larger, $m_1 \gg m_2$. So you can approximate the mass dependence of $t$ using a Taylor series,
$$\frac{1}{\sqrt{2G(m_1 + m_2)}} = \frac{1}{\sqrt{2Gm_1}}\biggl(1 - \frac{1}{2}\frac{m_2}{m_1} + \cdots\biggr)$$
The leading term is completely independent of $m_2$ (mass of the bowling ball or whatever), and this is why we can say, to a leading order approximation, that all objects fall at the same rate on the Earth's surface. For typical objects that might be dropped, the first correction term has a magnitude of a few kilograms divided by the mass of the Earth, which works out to $10^{-24}$. So the inaccuracy introduced by ignoring the motion of the Earth is roughly one part in a trillion trillion, far beyond the sensitivity of any measuring device that exists (or can even be imagined) today.
I hope this doesn't confuse you, but in one sense, yes, heavier bodies do fall faster than light ones, even in a vacuum. Previous answers are correct in pointing out that if you double the mass of the falling object, the attraction between it and the earth doubles, but since it is twice as massive its acceleration is unchanged. This, however, is true in the frame of reference of the center of mass of the combined bodies. It is also true that the earth is attracted to the falling body, and with twice the mass (of the falling body), the earth's acceleration is twice as large. Therefore, in the earth's frame of reference, a heavy body will fall faster than a light one.
Granted, for any practical experiment I don't see how you'd measure a difference that small, but in principle it is there.
Best Answer
The main force to take into account is air resistance, which increases with the square of velocity.
When the balls are first dropped the air resistance increases. For the lighter ball there will come a point at which the increasing resistance due to its increasing speed exactly counteracts the force of gravity, so the ball will no longer accelerate but will continue to drop at a fixed speed.
For the heavier ball the force due to gravity is greater, so the ball must reach a higher speed before air resistance matches its weight. If the heavier ball was 100 times heavier, say, then its terminal speed would be ten times that of the lighter ball.