Technically speaking, there's only one option, which is to keep stepping forward. Mathematical games, as far as I understand it, don't include not playing as an option. There's no way to optimize or formulate any strategy, because the other player in the game has perfect play as their only move, and always wins with perfect play.
So I guess the optimal strategy is to keep playing until you lose all your money, since it's the only strategy.
I would look at the definition of a game at this Wikipedia article
With regard to how much money you can win by drawing the blue ball, assuming you earn $10$ dollars every time you draw the blue ball, the scenario without replacement has only two possible outcomes:
$0$ blue balls, probability $P(X=0) = 0.7,$ gain $0.$
$1$ blue ball, probability $P(X=1) = 0.3,$ gain $10.$
The expected value of your winnings is
$$ E(10X) = 0(0.7) + 10(0.3) = 0 + 3 = 3.$$
(Note that $\frac3{10}\times 10 = 0.3 \times 10 \neq 0.3.$ Be careful what you write!)
The scenario with replacement has these possible outcomes:
$0$ blue balls, probability $P(Y=0) = \left(\frac9{10}\right)^3 = 0.729,$ gain $0.$
$1$ blue ball, probability
$P(Y=1) = \binom31 \left(\frac1{10}\right) \left(\frac9{10}\right)^2 = 0.243,$
gain $10.$
$2$ blue balls, probability
$P(Y=2) = \binom32 \left(\frac1{10}\right)^2 \left(\frac9{10}\right) = 0.027,$
gain $20.$
$3$ blue balls, probability $P(Y=3) = \left(\frac1{10}\right)^3 = 0.001,$
gain $30.$
Note that if we just care about whether we draw blue ball at least once, we can find the probability whether that happens either by adding up all the cases where it does happen,
$$P(Y=1) + P(Y=2) + P(Y=3) = 0.243+0.027+0.001 = 0.271,$$
or by taking $1$ and subtracting the probability that it does not happen,
$$1 - P(Y=0) = 1 - 0.729 = 0.271.$$
But when you gain $10$ for each time you draw the blue ball, the expected value of your winnings is
\begin{align}
E(10Y) &= 0(P(0)) + 10(P(1)) + 20(P(2)) + 30(P(3)) \\
&= 0 + 10(0.243) + 20(0.027) + 30(0.001) \\
&= 0 + 2.43 + 0.54 + 0.03 \\
&= 3.
\end{align}
In fact the expected number of blue balls drawn is exactly the same with or without replacement,
and the expected payment (receiving $10$ dollars each time blue is drawn) also is exactly the same in each case.
Most people would interpret "the probability of getting blue" in each scenario as the probability of getting blue at least once.
This probability is less in the scenario without replacement
(or to put it another way, the probability of not even once drawing blue is greater),
but once you start counting the number of times blue is drawn (and paying $10$ dollars each time)
the chance of getting a double or triple payout makes up for the increased chance of getting no payout.
When you ask about a probability, you are asking about something that either happens or does not happen, two possible outcomes.
It is possible to set up an expected value that also deals with only two possible outcomes, but expected value very often deals with more than two possible outcomes (in your example with replacement, four possible outcomes), and in those cases it tends to give answers different than you would get by looking only at two possibilities.
Best Answer
Assume that we have $a$ white balls and $b$ blacks balls. We can choose between two things: to play or not to play. In the first case our profit is $0$. Now assume that we choose to play. With probability $\frac{a}{a + b}$ we gain one dollar and we are left with $a - 1$ balls and $b$ black balls. Next, with probability $\frac{b}{a + b}$ we lose one dollar and we are left with $a$ white balls and $b - 1$ black balls.
This gives the following reccurence formula for $P_{a,b}$, the best expected profit we can get for $a$ white balls and $b$ black balls: $$P_{a,b} = \max\left\{0, \frac{a}{a+b}\left(1 + P_{a-1,b}\right) + \frac{b}{a + b} \left(-1 + P_{a,b - 1}\right)\right\}$$
Now, if we have $a$ white balls and no black balls, then obviously the best we can do is to win $a$ dollars. On the other hand, if there is no white balls, then we should not play at all. I.e., $P_{a,0} = a, P_{0, b} = 0$.
Using these formulas, we obtain:
$$P_{0,0} = P_{0,1} = P_{0,2}=P_{0,3} = 0,$$ $$P_{1,0} = 1, P_{2,0} = 2,$$ $$P_{1,1} = \max\left\{0, \frac{1}{2}(1 + P_{0,1}) + \frac{1}{2}(-1 + P_{1,0})\right\} = \frac{1}{2}, $$ $$P_{2,1} = \max\left\{0, \frac{2}{3}(1 + P_{1,1}) + \frac{1}{3}(-1 + P_{2,0})\right\} = \frac{4}{3},$$ $$P_{1,2} = \max\left\{0, \frac{1}{3}(1 + P_{0,2}) + \frac{2}{3}(-1 + P_{1,1})\right\} = 0,$$ $$P_{1,3} = \max\left\{0, \frac{1}{4}(1 + P_{0,3}) + \frac{3}{4}(-1 + P_{1,2})\right\} = 0,$$ $$P_{2,2} = \max\left\{0, \frac{1}{2}(1 + P_{1,2}) + \frac{1}{2}(-1 + P_{2,1})\right\} = \frac{2}{3},$$ $$P_{2,3} = \max\left\{0, \frac{2}{5}(1 + P_{1,3}) + \frac{3}{5}(-1 + P_{2,2})\right\} = \frac{1}{5},$$
i.e., on average we can gain $1/5$ dollars in the inital game. And the strategy is as follows. Look at the numbers above. Assume that we are left with $a$ white balls and $b$ black balls. If $P_{a,b} > 0$, draw a ball, otherwise stop.