At least in your example about Khovanov's TQFT for $\mathfrak{sl}_3$ link homology, part of the answer is that that theory also allows 'singular surfaces', locally modelled on a book with 3 pages. Now you can take a "punctured torus with a disk stuck in its throat" (see my paper, for example), as a singular 2-manifold with boundary that circle, and you'll see that this gives the missing element of $A$.
I'm not sure, though, what the general story is here.
Wikipedia's description of the Casson invariant gives the first important reason to study it. As an invariant that comes from the $\text{SU}(2)$ representation variety of $\pi_1(M)$, it reveals in particular that $\pi_1(M)$ is non-zero. At the time, before Perelman's proof of the Poincaré conjecture and geometrization, there was a lot of mystery about potential counterexamples to the Poincaré conjecture. For instance, one speculation was that the so-called $\mu$ invariant could reveal a counterexample. Since the Casson invariant lifts the $\mu$ invariant, and since it proves that $\pi_1(M)$ is non-trivial when it is non-zero, it is one way to see that the $\mu$ invariant can never certify a counterexample to the Poincaré conjecture. (Of course, no we know that there are no counterexamples.)
A second fundamental reason to study the Casson invariant is that it is the only finite-type invariant of homology spheres of degree 1. Many interesting 3-manifold invariants are finite-type, or (conjecturally) carry the same information as a sequence of finite-type invariants. This is known more rigorously at the level of knots; for instance, the derivatives of the Alexander polynomial, the Jones polynomial, and many other polynomials at $1$ are all finite-type invariants. At the level of knots, the second derivative of the Alexander polynomial, $\Delta''_K(1)$, is known to be the only non-trivial finite-type invariant of degree 2, and there is nothing in degree 1. So it means that this invariant appears over and over again as part of the information of many other invariants; there are many different definitions of the same $\Delta''_K(1)$. The same thing should happen to the Casson invariant, and indeed there are already two very different-looking types of definitions: (1) Casson's definition; (2) either the first LMO invariant or the first configuration-space integral invariant.
A third fundamental reason is that Casson invariant has an important categorification, Floer homology, which is the objects in the theory whose morphisms come from Donaldson theory. One wrinkle of this construction is that it is only a categorification of one of the definitions of Casson's invariant, Casson's definition. If Casson's invariant has many definitions, then it might (for all I know) have many different categorifications.
If your question is meant in the narrow sense of what topology you can prove with the Casson invariant, then you can definitely prove some things but only (so far) a limited amount. However, if you are interested in quantum topological invariants in their own right, and not just as a tool for pre-quantum topology problems, then the Casson invariant is important because it is a highly non-trivial invariant that you encounter early and often.
Best Answer
The basic problem here is the naming rather than the object itself. The name TCFT as far as I understand indicates that it's a topological field theory, whose origin is in conformal field theory. Namely there is a construction (a "topological twist") starting from an $N=2$ supersymmetric conformal field theory in two dimensions, that produces a topological field theory (in fact two, the A- and B-twists). This is a special case of a general theory of topological twists of SUSY quantum field theories, which is by far the predominant source of topological quantum field theories as far as I know, including many of the most interesting ones coming from SUSY gauge theories in four dimensions (these are sometimes called "TFTs of Witten type" as opposed to the very rare Schwarz type, like Chern-Simons theory, which come with a manifestly topological formulation).
Now when we say "TFT" here it is at a more refined chain level than the classical Atiyah-Segal axiomatic definition --- a synonym for TCFT in the sense of say Costello's beautiful paper on the subject is differential graded TFT. This means roughly that the theory is topological on a derived level -- its outputs are topologically invariant up to coherent homotopies. (This is the kind of refined topological invariance one always gets out of twisting SUSY field theory.)
What makes this very confusing initially is it seems conformal structures on a Riemann surface are playing an essential role: TCFT is defined in terms of chains on moduli spaces of complex structures. However this is a red herring (unless you are interested in the CFT origin of the construction) -- the moduli of complex structures is just playing the role of a nice model for the classifying space $BDiff(\Sigma)$ of topological surfaces, and everything can be said purely topologically (as it is in Hopkins-Lurie's work on the Cobordism Hypothesis). So really we are just defining a TFT on the chain level, in families (ie universally over moduli of topological surfaces). (This is a perspective I learned from Segal and Teleman and Freed and Costello, see Lurie's manuscript on TFTs for the contemporary perspective.)