Using your definition of "falling," heavier objects do fall faster, and here's one way to justify it: consider the situation in the frame of reference of the center of mass of the two-body system (CM of the Earth and whatever you're dropping on it, for example). Each object exerts a force on the other of
$$F = \frac{G m_1 m_2}{r^2}$$
where $r = x_2 - x_1$ (assuming $x_2 > x_1$) is the separation distance. So for object 1, you have
$$\frac{G m_1 m_2}{r^2} = m_1\ddot{x}_1$$
and for object 2,
$$\frac{G m_1 m_2}{r^2} = -m_2\ddot{x}_2$$
Since object 2 is to the right, it gets pulled to the left, in the negative direction. Canceling common factors and adding these up, you get
$$\frac{G(m_1 + m_2)}{r^2} = -\ddot{r}$$
So it's clear that when the total mass is larger, the magnitude of the acceleration is larger, meaning that it will take less time for the objects to come together. If you want to see this mathematically, multiply both sides of the equation by $\dot{r}\mathrm{d}t$ to get
$$\frac{G(m_1 + m_2)}{r^2}\mathrm{d}r = -\dot{r}\mathrm{d}\dot{r}$$
and integrate,
$$G(m_1 + m_2)\left(\frac{1}{r} - \frac{1}{r_i}\right) = \frac{\dot{r}^2 - \dot{r}_i^2}{2}$$
Assuming $\dot{r}_i = 0$ (the objects start from relative rest), you can rearrange this to
$$\sqrt{2G(m_1 + m_2)}\ \mathrm{d}t = -\sqrt{\frac{r_i r}{r_i - r}}\mathrm{d}r$$
where I've chosen the negative square root because $\dot{r} < 0$, and integrate it again to find
$$t = \frac{1}{\sqrt{2G(m_1 + m_2)}}\biggl(\sqrt{r_i r_f(r_i - r_f)} + r_i^{3/2}\cos^{-1}\sqrt{\frac{r_f}{r_i}}\biggr)$$
where $r_f$ is the final center-to-center separation distance. Notice that $t$ is inversely proportional to the total mass, so larger mass translates into a lower collision time.
In the case of something like the Earth and a bowling ball, one of the masses is much larger, $m_1 \gg m_2$. So you can approximate the mass dependence of $t$ using a Taylor series,
$$\frac{1}{\sqrt{2G(m_1 + m_2)}} = \frac{1}{\sqrt{2Gm_1}}\biggl(1 - \frac{1}{2}\frac{m_2}{m_1} + \cdots\biggr)$$
The leading term is completely independent of $m_2$ (mass of the bowling ball or whatever), and this is why we can say, to a leading order approximation, that all objects fall at the same rate on the Earth's surface. For typical objects that might be dropped, the first correction term has a magnitude of a few kilograms divided by the mass of the Earth, which works out to $10^{-24}$. So the inaccuracy introduced by ignoring the motion of the Earth is roughly one part in a trillion trillion, far beyond the sensitivity of any measuring device that exists (or can even be imagined) today.
I hope this doesn't confuse you, but in one sense, yes, heavier bodies do fall faster than light ones, even in a vacuum. Previous answers are correct in pointing out that if you double the mass of the falling object, the attraction between it and the earth doubles, but since it is twice as massive its acceleration is unchanged. This, however, is true in the frame of reference of the center of mass of the combined bodies. It is also true that the earth is attracted to the falling body, and with twice the mass (of the falling body), the earth's acceleration is twice as large. Therefore, in the earth's frame of reference, a heavy body will fall faster than a light one.
Granted, for any practical experiment I don't see how you'd measure a difference that small, but in principle it is there.
Best Answer
No, not in reality, in air. In a vacuum, say, on the surface of the moon (as demonstrated here), they fall at the same rate.
If the two objects have the same shape, the drag force on the each object, as a function of speed $v$, is the same.
The total force accelerating the object downwards is the difference between the force of gravity and the drag force:
$$F_{net} = mg - f_d(v)$$
The acceleration of each object is thus
$$a = \frac{F_{net}}{m} = g - \frac{f_d(v)}{m}$$
Note that in the absence of drag, the acceleration is $g$. With drag, however, the acceleration, at a given speed, is reduced by
$$\frac{f_d(v)}{m}$$
For the much more massive lead feather, this term is much smaller than for the duck's feather.