First assume that ZFC is indeed consistent. If every statement were either provably true or provably false or provably independent, then there would be a decision procedure that sorted all statements into these three groups -- just enumerate all valid proofs until one whose conclusion is one of the three forms were found.
We could use this to decide the halting problem. For any $n$ consider the statement "Turing machine $T_n$ halts". There are then three cases.
$ ZFC \vdash Con(ZFC) \Rightarrow T_n\text{ halts}$
In this case, it must be true that $T_n$ halts.
$ ZFC \vdash Con(ZFC) \Rightarrow \neg(T_n\text{ halts})$
In this case it must be false that $T_n$ halts.
$ ZFC \vdash Con(ZFC) \Rightarrow{}$ "$T_n$ halts" is independent of ZFC.
In this case there must be a model of ZFC in which $T_n$ does not halt and so has an infinitely long computation. However, in every model of ZFC, the model's $\omega$ must contain a surjective image of the meta-$\omega$. (And similarly for whatever objects we use to represent Turing tapes). Therefore the infinite computation in the model corresponds to an infinite computation at the meta-level. Therefore $T_n$ really does not halt.
In conclusion, if ZFC is consistent, then your hypothesis leads to a solution to the halting problem, which we know is impossible. Thus by contradiction your hypothesis must be false if ZFC is consistent. On the other hand if ZFC is not consistent, then your hypothesis is still false, because then there are no unprovable statements at all.
The issue here is how complicated is each statement, when formulated as a claim about the natural numbers (the Riemann hypothesis can be made into such statement).
For the purpose of this discussion we work in the natural numbers, with $+,\cdot$ and the successor function, and Peano Axioms will be our base theory; so by "true" and "false" we mean in the natural numbers and "provable" means from Peano Arithmetic.
We will say that a statement is "simple" if in order to verify it, you absolutely know that you do not have to check all the natural numbers. (The technical term here is "bounded" or "$\Delta_0$".)
For example, "There is a prime number small than $x$" is a simple statement, since verifying whether or not some $n$ is prime requires only checking its divisibility by numbers less than $n$. So we only need to check numbers below $x$ in order to verify this.
On the other hand, "There is a Fermat prime larger than $x$" is not a simple statement, since possibly this is false but only checking all the numbers above $x$ will tell us the truth of this statement.
The trick is that a simple statement is true if and only if it is provable. This requires some work, but it is not incredibly difficult to show. Alas, most interesting questions cannot be formulated as simple statements. Luckily for us, this "provable if and only if true" can be pushed a bit more.
We say that a statement is "relatively simple" if it has the form "There exists some value for $x$ such that plugging it in will turn the statement simple". (The technical term is $\Sigma_1$ statement.)
Looking back, the statement about the existence of a Fermat prime above $x$ is such statement. Because if $n>x$ is a Fermat prime, then the statement "$n$ is larger than $x$ and a Fermat prime" is now simple.
Using a neat little trick we can show that a relatively simple statement is also true if and only if it is provable.
And now comes the nice part. The Riemann hypothesis can be formulated as the negation of a relatively simple statement. So if the Riemann hypothesis was false, its negation was provable, so Riemann hypothesis would be refutable. This means that if you cannot disprove the Riemann hypothesis, it has to be true. The same can also be said on the Goldbach conjecture.
So both of these might turn to be independent, in the sense that they cannot be proved from Peano Arithmetic, but if you show that they are at least consistent, then you immediately obtain that they are true. And this would give us a proof of these statements from a stronger theory (e.g. set theory).
You could also ask the same about the twin prime conjecture. But this conjecture is no longer a relatively simple statement nor the negation of one. So the above does not hold for, and it is feasible that the conjecture is consistent, but false in the natural numbers.
Best Answer
The phenomenon that a statement is true if it is undecidable occurs for statements that can be disproved by counterexamples. A statement of the form “$A(x)$ for all $x\in X$”, where $A(x)$ is some statement about $x$ and $X$ is some infinite set, can be disproved by showing that it is false for a single $x\in X$. By contrast, it cannot be proved by proving that it is true in individual cases, since there are infinitely many cases to consider.
If it is false, there is a counterexample, and the counterexample can be used to disprove it, so it cannot be undecidable; whereas if it is true, there are infinitely many examples of it being true, but they cannot be used to prove it, so it may still be undecidable. Thus, since it is compatible with undecidability that it is true but incompatible that it is false, undecidability implies that it is true.
The Riemann hypothesis is a statement of this form, namely that the Riemann zeta function is non-zero for all complex numbers that are neither negative even integers nor have real part $\frac12$.
Your example is not of this form. You have showed that it is compatible with the assumptions that the statement be true or false (which is different from it being “both provably true and false”), and thus that it is undecidable. But it doesn’t follow in this case that it is true, since in this case the statement being false does not provide a way to disprove it.