You're right. If $P(H) = 0.2$, and you're using zero-one loss (that is, you need to guess an actual outcome as opposed to a probability or something, and furthermore, getting heads when you guessed tails is equally as bad as getting tails when you guessed heads), you should guess tails every time.
People often mistakenly think that the answer is to guess tails on a randomly selected 80% of trials and heads on the remainder. This strategy is called "probability matching" and has been studied extensively in behavioral decision-making. See, for example,
West, R. F., & Stanovich, K. E. (2003). Is probability matching smart? Associations between probabilistic choices and cognitive ability. Memory & Cognition, 31, 243–251. doi:10.3758/BF03194383
Imagine that you are in Paris in 1654 and you and your friend are observing a gambling game based on sequential rolling of a six sided dice. Now, gambling is highly illegal and busts by the gendarme are quite frequent, and to be caught at a table with stacks of livre is to almost surely guarantee a lengthy stint in the Chateau d'If.
To get around this you and your friend have a gentleman's agreement on a bet made between two of you prior to the last die roll. He agrees to pay you five livre if you observe two sixes in the next five rolls of dice, and you agree to pay him the same amount if two ones are rolled, with no other action if these combinations do not come up.
Now, the last die roll is a six so you are on the edge of your seat, figuratively. At this moment, heavily armed guardsmen burst into the den and arrests everyone at the table, and the crowd disperses.
Your friend believes that the bet made between the two of you is now invalidated. However, you believe that he should pay you some amount as one six had already been rolled. What is a fair way of settling this dispute between the two of you?
(This is my interpretation of the origins of the expected value as presented in here and discussed in greater detail here)
Let's answer this question of fair value in a non rigorous way. The amount your friend should pay you can be calculated in the following manner. Consider all possible rolls of four dice. Some sets of rolls (namely those containing at least one six) will result in your friend paying out the agreed amount. However, on other sets (namely, those not containing a single six) will result in you receiving no money. How do you balance the possibility of these two types of rolls happening? Simple, average out the amount you would have been paid over ALL possible rolls.
However, your friend, (quite unlikely), can still win his bet! You have to consider the number of times two ones will be rolled in the remaining four dice, and average out the amount you will pay him over the number of all possible rolls of four dice. This is the fair amount you should pay your friend for his bet. Thus the amount you end up getting is the amount your friend should pay you, minus what you should pay your friend.
This is why we call it the "expected value". It is the average amount you expect to receive if you are able to simulate an event happening in multiple simultaneous universes.
Best Answer
There are several different measures of "location" or "central tendency". Expected value is the most popular one, but there are others -- median, mode, geometric mean, etc.
While all measures of central tendency are, in some ways, similar, it is important to remember that they actually measure different things. Here are the interpretations.
There is confusion because, with symmetric distributions, which are common, mean, median, and mode are numerically equal. So people just think of the mean and not the median or mode. But, depending on the problem, what you are actually interested in could be the mode, even if it is numerically equal to the mean.
Now, let's look at your specific questions.