why does the voltmeter not form a closed loop with the circuit and hence cause energy to be dissipated by internal resistance
It does, but the voltmeter probably has an impedance of 10,000,000 ohms - as most multimeters do - so the current flowing is negligibly small (0.6 microAmps) and therefore the voltage drop due to battery internal resistance is also negligible.
4 x Alkaline AA has resistance maybe 4 x 300 milliohms. At 0.6 microAmps that's a voltage drop of 0.72 microvolts (if I did my sums right - please check)
Without Voltmeter
See the circuit given below. Before inserting the voltmeter, the current through the circuit is $ E /(R_1 +R_2)$ where $E$ is the EMF of the battery. So the potential difference across $R_2$ is
$$V_0=E\frac{R_2}{R_1+R_2}$$
Image source
After Voltmeter
Let ther resistance of voltmeter be $R_V$. Then the equivalent resistance of the circuit will be
$$R_{\text{eq}} =R_1 + \frac{R_2 R_V}{R_2+R_V}$$
Therefore the current through the circuit will be
$$I_{\text{total}}= \frac{E}{R_{\text{eq}}}$$
And the current through $R_2$ will be,
$$I_2 = I_{\text{total}} \frac{R_V}{R_2 + R_V}$$
Therefore the potential difference across $R_2$ will be
$$V=I_2 R_2 = I_{\text{total}} \frac{R_V R_2}{R_2 + R_V}= \frac{E}{R_{\text{eq}}}\frac{R_V R_2}{R_2 + R_V} =\frac{E}{ R_1 + \frac{R_2 R_V}{R_2+R_V}}\frac{R_V R_2}{R_2 + R_V} $$
which simplifies to
$$V= E \frac{R_2 R_V}{R_1 R_2 +R_2 R_V + R_V R_1}$$
Clearly this is different from the original result without the voltmeter($V_0$). Also when $R_V \rightarrow \infty$, $V\rightarrow V_0$ which is expected. At all finite values of $R_V$, $V<V_0$
Intuition
When you connect a voltmeter in parallel, the equivalent resistance of the circuit decreases and the total current increases. But at the same time the current also gets divided across $R_2$ and $R_V$ which results in lower current through $R_2$. The latter effect dominates and a lower potential difference is obtained across $R_2$. So we try to minimize these two effects by increasing $R_V$. This is clearly evident from the mathematics of this scenario.
Best Answer
If you have a battery with no internal resistance, or with an internal resistance that is negligibly small compared to your resistor $R$ (and the voltmeter resistance), then yes a relatively low resistance of your voltmeter won't change the measured voltage.
You are correct that the voltmeter can affect the voltage across $R$ only when there are multiple resistances in the circuit because if there are no other resistances the voltage drop across the (non-existant) other resistances has to be zero.