This is a great question I wish I understood the answer to better.
I know two vague answers, one based on derived algebraic geometry and one based on string theory.
The first answer, that Costello explained to me and I most likely misrepeat,
is the following. The B-model on a CY X as an extended TFT can be defined in terms of
DAG: we consider the worldsheet $\Sigma$ as merely a topological space or simplicial set (this is a reflection of the lack of instanton corrections in the B-model), and consider the mapping space $X^\Sigma$ in the DAG sense. For example for $\Sigma=S^1$ this is the derived loop space (odd tangent bundle) of $X$.. In this language it's very easy to say what the theory assigns to 0- and 1-manifolds: to a point we assign coherent sheaves on $X$, to a 1-manifold cobordism we assign the functor given by push-pull of sheaves between obvious maps of mapping spaces (see e.g. the last section here). For example for $S^1$ we recover Hochschild homology of $X$. Now for 2-manifold bordisms we want to define natural operations by push-pull of functions, but for that we need a measure -- and the claim is the Calabi-Yau structure (together with the appropriate DAG version of Grothendieck-Serre duality, which Kevin said Lurie provides) gives exactly this integration...
Anyway that gives a tentative answer to your question: the B-model assigns to a surface $\Sigma$ the "volume" of the mapping space $X^\Sigma$, defined in terms of the CY form.
More concretely, you chop up $\Sigma$ into pieces, and use the natural operations on Hochschild homology, such as trace pairing and identification with Hochschild cohomology (and hence pair-of-pants multiplication).. of course this last sentence is just saying "use the Frobenius algebra structure on what you assigned to the circle" so doesn't really address your question - the key is to interpret the volume of $X^\Sigma$ correctly.
The second answer from string theory says that while genus 0 defines a Frobenius manifold you shouldn't consider other genera individually, but as a generating series -- i.e. the genus is paired with the (topological) string coupling constant, and together defines a single object, the topological string partition function, which you should try to interpret rather than term by term. (This is also the topic of Costello's paper on the partition function). BTW for genus one there is a concrete answer in terms of Ray-Singer torsion, but I don't think that extends obviously to higher genus.
As to how to interpret it, that's the topic of the famous BCOV paper - i.e. the Kodaira-Spencer theory of gravity. For one thing, the partition function is determined recursively by the holomorphic anomaly equation, though I don't understand that as "explaining" the higher genus contributions. But in any case there's a Chern-Simons type theory quantizing the deformation theory of the Calabi-Yau, built out of the Kodaira-Spencer dgla in a simple looking way, and that's what the B-model is calculating.
A very inspiring POV on this is due to Witten, who interprets the entire partition function as the wave function in a standard geometric quantization picture for the middle cohomology of the CY (or more suggestively, of the moduli of CYs). This is also behind the Givental quantization formalism for the higher genus A-model, where the issue is not defining the invariants
but finding a way to calculate them.
Anyway I don't know a totally satisfactory mathematical formalism for the meaning of this partition function (and have tried to get it from many people), so would love to hear any thoughts. But the strong message from physics is that we should try to interpret this entire partition function - in particular it is this function which appears in a million different guises under various dualities (eg in gauge theory, as solution to quantum integrable systems, etc etc...)
The statement that $HF^{\ast}(X,X)$ is isomorphic to $QH^\ast(X)$ is a version of the Piunikhin-Salamon-Schwarz (PSS) isomorphism (proved, under certain assumptions, in McDuff-Salamon's book "J-holomorphic curves in symplectic topology"). PSS is a canonical ring isomorphism from $QH^{\ast}(X)$ to the Hamiltonian Floer cohomology of $X$, and the latter can be compared straightforwardly to the Lagrangian Floer cohomology of the diagonal.
Now to Hochschild cohomology of the Fukaya category $F(X)$. There's a geometrically-defined map $QH^{\ast}(X) \to HH^{\ast}(F(X))$, due to Seidel in a slightly different setting (see his "Fukaya categories and deformations"), inspired by the slightly vague but prescient remarks of Kontsevich from 1994. One could define this map without too much trouble, say, for monotone manifolds. It's constructed via moduli spaces of pseudo-holomorphic polygons subject to Lagrangian boundary conditions, with an incidence condition of an interior marked point with chosen cycles in $X$. The question is whether this is an isomorphism.
This statement is open, and will probably not be proven true in the near future, for a simple reason: $QH^*(X)$ is non-trivial, while we have no general construction of Floer-theoretically essential Lagrangians.
There are two positive things I can say. One is that Kontsevich's heuristics, which involve interpreting $HH^{\ast}$ as deformations of the identity functor, now have a natural setting in the quilted Floer theory of Mau-Wehrheim-Woodward (in progress). This says that the Fukaya category $F(X\times X)$ naturally embeds into the $A_\infty$-category of $A_\infty$-endofunctors of $F(X)$.
The other is that for Weinstein manifolds (a class of exact symplectic manifolds with contact type boundary), there seems to be an analogous map from the symplectic cohomology $SH^{\ast}(X)$ (a version of Hamiltonian Floer cohomology on the conical completion of $X$) to $HH^{\ast}$ of the wrapped Fukaya category, which involves non-compact Lagrangians. (Edit August 2010: I was careless about homology versus cohomology. I should have said that $HH_{\ast}$ maps to $SH^{\ast}$.) Proving that this is an isomorphism is more feasible because one may be able to prove that Weinstein manifolds admit Lefschetz fibrations. The Lefschetz thimbles are then objects in the wrapped Fukaya category.
One might then proceed as follows. The thimbles for a Lefschetz fibration should generate the triangulated envelope of the wrapped category (maybe I should split-close here; not sure) - this would be an enhancement of results from Seidel's book. Consequently, one should be able to compute $HH_{\ast}$ just in terms of $HH_{\ast}$ for the full subcategory generated by the thimbles. The latter should be related to $SH^{\ast}$ by ideas closely related to those in Seidel's paper "Symplectic homology as Hochschild homology".
What could be simpler?
ADDED: Kevin asks for evidence for or against $QH^{\ast}\to HH^{\ast}$ being an isomorphism. I don't know any evidence contra. Verifying it for a given $X$ would presumably go in two steps: (i) identify generators for the (triangulated envelope of) $F(X)$, and (ii) show that the map from $QH^{\ast}$ to $HH^{\ast}$ for the full subcategory that they generate is an isomorphism. There's been lots of progress on (i), less on (ii), though the case of toric Fanos has been studied by Fukaya-Oh-Ohta-Ono, and in this case mirror symmetry makes predictions for (i) which I expect will soon be proved. In simply connected disc-cotangent bundles, the zero-section generates, and both $HH_{\ast}$ for the compact Fukaya category and $SH^{\ast}$ are isomorphic to loop-space homology, but I don't think it's known that the resulting isomorphism is Seidel's.
Added August 2010: Abouzaid (1001.4593) has made major progress in this area.
Best Answer
At first sight, the Fukaya category has obvious cyclic symmetry, because the $A_\infty$ structure maps count points in spaces of rigid pseudo-holomorphic polygons subject to Lagrangian boundary conditions, and these spaces depend only on the cyclic order of the Lagrangians. This indeed proves that the cohomological Fukaya category, in which the hom-spaces are Floer cohomology spaces, is cyclically symmetric.
The trouble comes when these Lagrangians don't intersect one another transversely - for instance, the same Lagrangian occurs more than once - because then the morphism spaces and structure maps invoke Hamiltonian perturbations which need not be cyclically symmetric. The problem which Fukaya has solved over the reals (see Matthew Ballard's answer) is, I presume, to find a way to make these perturbations cyclically symmetric whilst also achieving the necessary coherence between them, as well as transversality for compactified moduli spaces of inhomogeneous pseudo-holomorphic polygons (or worse, their abstract perturbations). These are the things which actually define the $A_\infty$-structure.
FOOO worked extremely hard to get geometrically-meaningful units in their Fukaya endomorphism algebras, where other authors are content to define units by tweaking the $A_\infty$-structure algebraically. My hope would be that algebra will also give a cheaper approach to cyclic symmetry, particularly since I'm told that for Costello's theorem to hold, one only needs "derived" cyclic symmetry.
By the way, let's be clear that Costello's theorem, suggestive as it may be, is not about GW invariants! It's about theories over $M_{g,n}$, not over $\overline{M}_{g,n}$.