I apparently cannot post images, so I apologize but you'll have to open this link in a new window to see my atrocious diagrams :) Diagrams -> http://i.imgur.com/Lxfu1e2.png
EDIT: Here are the diagrams, sorry about my lack of artistic skills, haha.
Voltage is an electrical potential difference, which is essentially a force caused by electrons wanting to evenly distribute in a material since the electrons repel each other. Olin Lathrop is using the common analogy between fluid and electric systems, which is why he calls it a "pressure", because pressure is a type of fluid force, but for this example it might be easier to understand if I keep it in electrical terms.
So the batteries try to maintain a voltage (potential difference) of 1.5 V across the terminals. So in the first part of the diagram I threw together, node 'V0' is going to be my reference node which due to the ground symbol means it will be 0 V. So looking at node 'V1' we know that the battery will try to maintain a 1.5 V potential difference, and we know its negative terminal is 0 V, so the positive terminal, or node 'V1', must be 1.5 V. Now, looking at the second battery, the negative terminal is connected to node 'V1', and if the battery is to maintain its 1.5 V difference across terminals, we can add 1.5 V (battery) to the 1.5 V (at node 'V1') to get the voltage at node 'V2', which turns out to be 3 V, as stated in Olin Lathrop's answer.
So why do the voltages add? Batteries are a container of voltage or electrical potential difference, and the potential force is created by a electron rich side separated from an electron deficit side. The electrons want to disperse evenly through the battery, but cannot pass through the middle, and thus must take the long way around to get to the other side. So with two batteries, you can see that there are actually two (basically) equal forces, two sets of 1.5 V difference.
Now to get to the heart of your question, basically why can't the middle two parts simply intermingle at node 'V1'? I'll use 3 different cases to help explain We also need to keep in mind that positive charges aren't really there, so the last diagram is a little more accurate still, where there are simply electrons and a lack of electrons (the electrical potential difference).
Case 1: Assume that the batteries are connected via node 'V1', but not from 'V0' to 'V2'. If we try to imagine moving electrons from the left battery to the right battery via node 'V1', we are actually pushing electrons CLOSER TOGETHER. We know that electrons repel each other so they don't like to do that. If we magically could though, we would be increasing the electrical potential difference, or voltage, in the right battery and decreasing it in the left. (But the total system would maintain the same total voltage, since we did nothing to the system except move electrons around.) So instead of the electrons forcing their way closer to more electrons, they just sit there and do nothing.
Case 2: The outer terminals, nodes 'V0' and 'V2', are connected but the inner two are not connected. If we push electrons from 'V0' through the circuit to node 'V2', we end up in the same situation as case 1, trying to push electrons into more electrons! This is why electrons don't want to flow if there is not a closed loop, because they'll just end up running into electrons that aren't moving somewhere else.
Case 3: Both sets of terminals are connected, (if there's a closed loop for the electrons to flow through). At the instant that we make a loop for the electrons to travel through, a few things are happening with these forces simultaneously. Remember those electrons that want to flow through node 'V1' but can't because there are too many electrons at node 'V0'? And now, remember the electrons that want to flow from node 'V0' through the circuit to node 'V2' but can't because there are too many electrons at node 'V1'? Well we get a situation where they can work together to solve each other's problems! Imagine we pull an electron from node 'V0', to node 'V2', and as that electron gets close to 'V2', we move another electron at 'V1' from the left battery to the electron depleted area in the right battery. If we do them simultaneously, there is an electron coming and going from both places that were previously causing an electron road-block! So as electrons flow from 'V0' to 'V2' there will simultaneously be electrons flowing through node 'V1'. And now we have two sets of electrons flowing with two 1.5 V potential differences in a row, which is why we get the 3 V potential difference!
Reality Check: There are some differences with reality though. For example, as batteries are used, they will decrease in voltage. Since the electrons are trying to evenly distribute, the electrical potential difference will decrease as well. Also, the barriers between the electron rich and electron deficit areas of the battery aren't perfect. All batteries have a self-discharge rate, which is effectively the electrons making it from one side of the battery to the other, it's just generally so small in comparison to the amount of electrons that we can actually use, that we ignore it. Internal resistance is also mentioned above, another slight deviation from the theoretical operation of batteries. But you can learn about all that stuff as you feel ready.
I hope this helps, and if I made any mistakes, please let me know! :)
There exists a similar question here.
This description of a voltaic battery solves the conundrum which the question raises:
If one connects the + and - of the same battery with no resistance, one shorts and discharges it anomalously. There should always be a resistance ( the lamp in the drawing) on the same battery.It is clear in the diagram that a chemical path exists, a current of ions that closes the circuit and current flows.
A chemical circuit has to be closed for current to flow. Two different batteries have two different chemical flows . Electrons that might leave the anode to go to the cathode of the other battery cannot close the chemical circuit of the parent cell because the ions have no reason to move as no charge has gone to the copper cathode of the original cell. At the same time the second cell is at the same fix , as no chemical circuit is closed there too, no charge has left the zinc anode, no motion of ions possible.
Best Answer
The short answer is that you need a complete circuit for a battery to work. However, I find a longer answer to be very useful to help with understanding.
If you have a circuit, you are solving a problem in what we call "electrodynamics." Electrodynamics is studying what happens when electrons are moving. In a circuit, electrons are constantly circling around the loop (powered by the Electromotive Force (EMF) of the chemical reactions in the battery). These moving electrons can do work, such as lighting an LED (which decreases the energy in the electrons and emits that energy as photons of light).
If you have no movement of electrons, you have an "eletrostatics" problem. We see electrostatics all the time, like when you scuff your feet on the carpet and then get shocked on a doorknob. Electrostatics is interested in what things look like when the electrons finish moving. Without anything like a chemical reaction to keep the electrons moving, they do eventually settle into a lowest energy arrangement.
The setup you describe sits in the fuzzy region between the two. If you let it sit, it has to become an electrostatics problem, because there's no loop for the electrons to flow around in. They have to eventually find a lowest energy setup. However, your intuition is correct that the electrons are going to have to flow a bit in order to arrive at this arrangement.
We're going to have to look at what happens when you hook all of these elements together. But first, a warning: the electrostatics side of this problem is going to be rather boring. Typical electrostatics problems involve high voltages. 200V to 20kV are not uncommon in electrostatics. Our little batteries can pump out 1.5V, so expect the final electrostatics solution to this problem to be rather uninteresting. It's the transients that are interesting.
Before we hooked everything together, the chemical reactions in each battery have pushed more electrons towards the negative side of the battery and fewer towards the positive side (there's more positive ions on the positive side). This chemical reaction will continue to push the electrons like this, but as it does, we start to build up a negative charge at the negative side (and a positive charge at the positive side). The electrons bundled up on the negative side start to repel, and this repelling actually fights against the chemical reaction, making it less and less likely to occur. Eventually the whole system reaches an equilibrium, at the point where the voltage difference between positive and negative sides is 1.5V (for normal AA batteries).
Now you bring the system together. Your intuition is actually going to be correct at first. We've got a bunch of electrons at the bottom of the bottom battery, and a bunch of positive ions at the top of the top battery, and a wire between them. Electrons will indeed flow from bottom to top, and they'll do some work as they pass through the LED, generating some light. However, we're going to find this is a VERY small amount of light, because this process happens really fast.
Now that there are fewer electrons at the bottom of the bottom battery, the charge difference between positive and negative sides of the battery is smaller, so the chemical reaction isn't inhibited as much. It starts reacting, pumping more electrons across to the negative side. The same occurs in the top battery, because there's not as many positive charges at the top of the top battery.
But after a very short time, this process slows. Soon we have so many positive ions at the top of the bottom battery, and so many electrons trapped at the bottom of the top battery that, even with the wire connecting them, there's still enough electrostatic charge within each battery to prevent the chemical reaction from going forward any further.
So now we arrive at the electrostatics part of the problem. There's 3V of potential between the positive terminal of the bottom battery (chock full of positive ions) and the negative terminal of the bottom battery (full of electrons). There's enough charge there such that neither battery's chemical equations "want" to move forward. Thus, there's no electrons flowing, no electrons doing work, so the LED stays unlit.
This process happens fast. It's far faster than you can see. We can add some elements like capacitors to slow it down, but with these batteries, you'll never actually notice this effect occurs. However, if you work with static sensitive parts, you'll learn this process can occur in an instant and burn your parts. A measly 3V of static potential like we have here won't damage anything. However, if you aren't careful, things like wool sweaters can easily build up very large static charges, on the order of thousands of volts. If you are charged up like this and touch the circuit, it's exactly like your batteries here, only on a much larger scale. Many more electrons rush out of your body, seeking to equalize these potentials. All of them do "work," which in the case of static sensitive parts can include things like burning through precious oxide layers and rendering them inoperable!