Your calculation represents the probability of getting the red ball ten times in a row with replacement: this is a totally different problem. It should be easy to see that it is false by how small it is: the probability of getting the red in just the first pick is one out of twenty, so your chances of getting the red ball in the process of taking ten balls out should be even greater.
Here's how the reasoning with $X$ could have gone: the chance the $n$-th ball came up red is the probability the other $n-1$ before it came up nonred and then you got the red ball out of the remaining ones (the latter colored red in the calculations below). The probability the first pick was nonred is $19/20$, the probability the second pick was nonred is $18/20$, and so on, so we get the pattern
- $P(X=1)=\color{Red}{\frac{1}{20}}=1/20$
- $P(X=2)=\frac{19}{20}\times\color{Red}{\frac{1}{19}}=1/20$
- $P(X=3)=\frac{19}{20}\frac{18}{19}\times\color{Red}{\frac{1}{18}}=1/20$
- $P(X=4)=\frac{19}{20}\frac{18}{19}\frac{17}{18}\times\color{Red}{\frac{1}{17}}=1/20$
- $\qquad\qquad\cdots\cdots\cdots\cdots\cdots\cdots$
- $P(X=10)=\frac{19}{20}\frac{18}{19}\cdots\frac{11}{12}\times\color{Red}{\frac{1}{11}}=1/20$
Hence $P(X=1)+P(X=2)+\cdots+P(X=10)=\frac{1}{20}+\cdots\frac{1}{20}=10(\frac{1}{20})=1/2.$
Alternatively, we could have reasoned with symmetry as follows: every way of picking twenty balls out of twenty balls and keeping track of what order they come out will be like putting the balls in some order. The probability a specific ball will end up in a specific position is equal to the same probability for any other specific ball, say $p$. Since at least one ball must be in the position we have that the sum of each ball's probability being there equals one, or $20p=1$, hence $p=1/20$. Sum this over the first ten positions for the red ball and you get $10(1/20)=1/2$ as the probability the red ball is in the first ten positions. Since it makes no difference whether we actually take out the last ten balls or keep them in the box, this must be our desired value.
Two quicker ways could have been the following:
Symmetry: For every way of taking 10 balls out of 20 and leaving the other 10 in the box, there is exactly one way of taking those other 10 out of the box and leaving the original 10 in, hence by symmetry the probability of choosing the red ball equals the probability of not choosing the red ball. Thus the probability is $1/2$.
Counting: The number of ways of getting a red ball in a choice of 10 out of 20 is equal to the number of ways the other 9 non-red balls can be chosen out of the total 19 non-red balls: in other words, $19\choose9$. And the total number of ways of picking 10 balls out of 20 is $20\choose10$. So the probability of getting the red ball is $$\frac{{19\choose9}}{{20\choose10}}=\frac{19!}{9!10!} \frac{10!10!}{20!}=\frac{10}{20}=\frac{1}{2}.$$
Having these two techniques in your probability toolkit is important because problems that can be solved easily with them are ubiquitous in combinatorial situations. However it is also important that you don't split a probability problem into a large number of smaller cases (originally, we split ours up into 10 different cases) when doing so isn't actually necessary - this saves a lot of energy.
$1.$ With replacement: There are three ways this can happen, red, red, red; blue, blue, blue; and green, green, green.
The probability of red, red, red is $\left(\dfrac{5}{19}\right)^3$. Find similar expressions for the other two colurs, and add up.
$2.$ Without Replacement: The probability of red, red, red is $\left(\dfrac{5}{19}\right)\left(\dfrac{4}{18}\right)\left(\dfrac{3}{17}\right)$. Find similar expressions for the other two colours, and add up.
Remark: We could also do the problem by a counting argument. Let's look at the first problem. Put labels on the balls to make them distinct. Then for with replacement, there are $19^3$ strings of length $3$ of our objects. (The three objects in the string are not necessarily distinct.) All of these $19^3$ strings are equally likely.
There are $5^3$ all red strings, $11^3$ all blue, and $8^3$ all green. This gives probability $\dfrac{5^3+6^3+8^3}{19^3}$.
One can do a similar calculation for the without replacement case.
For without replacement, there is a third approach. We can choose $3$ objects from $19$ in $\dbinom{19}{3}$ ways. And we can choose $3$ of the same colour in $\dbinom{5}{3}+\dbinom{6}{3}+\dbinom{8}{3}$ ways.
Best Answer
The answer to (a) is clearly $\dfrac{7}{20}$. This is because every one of the $20$ balls is equally likely to be the third ball drawn.
For (b), let $R$ be the event that the third ball drawn is red. Let $B$ be the event the first two balls drawn are blue. We want the conditional probability $\Pr(B|R)$. By the usual expression for conditional probability, we have $$\Pr(B|R)=\frac{\Pr(B\cap R)}{\Pr(R)}.$$
We already know $\Pr(R)$, so all we need is $\Pr(B\cap R)$. This is not hard to calculate: $$\Pr(B\cap R)=\frac{13}{20}\cdot\frac{12}{19}\cdot \frac{7}{18}.$$