You can use the binomial distribution instead with parameters $n=10$ and $p=10\%$. Specifically, the number of misses $Y$ in the first $n=10$ shots is binomially distributed with parameters $n=10$ and $p$ as above, i.e. $p=10\%$. Then the probability that the second miss comes no later than the $10$-th shot is equal to the probability that there are at least two misses in the first $10$ shots. Hence you want to calculate \begin{align}P(Y\ge 2)&=1-P(Y<2)\\[0.2cm]&=1-P(Y=0)-P(Y=1)\\[0.2cm]&=1-\dbinom{10}{0}(0.10)^0(1-0.10)^{10}-\dbinom{10}{1}(0.10)^1(1-0.10)^{10-1}\end{align}
The answer to the first question is wrong.
To see this, suppose that the probability of success at each shot is $0.5$. That means that you can work out the probability without any calculation: just count among the $2\times 2\times 2\times 2\times 2=32$ possible results.
The probability of getting at least 3 successes is $\frac {16}{32}$.
The probability of getting precisely 3 successes is $\frac{10}{32}$.
Neither of these figures matches what the method quoted for the first example would give, which is $0.5^3\times 0.5^2=\frac 1 {32}$.
I have changed the probability to $0.5$ to show that the method given produces wrong results, in a case where it easily to see without calculation what the right result should be. A method which fails for $p=0.5$ will also fail for $p=0.8$ - its failure will be less easy to see, that’s all.
Since the method given is nonsensical, no wonder you are confused!
The second example is simple to understand if you are not confused by the first one. There is no third drawing. The probability of success on two drawings is simply the square of the probability of success on one.
EDIT: As drhab points out, the answer that is being given in the first example is to the question “What is the probability of three successes followed by two failures?” His solution of the question as you stated it is correct and he should make it into a proper answer instead of a comment.
Best Answer
The question states that the first miss attempt happens on or later. The presence of the "or later" signifies that there will be at least 12 makes, which is $0.9^{12}$.
Another way of thinking about it is through summing all the probabilities that the first miss occurs after the first 13 shots. This can be modeled by a geometric variable as denoted below: $$ \sum_{i=13}^\infty (0.9)^{i-1}(1-0.9) $$
Both of these methods are correct and yield the same answer its just how you think about it.