I have a proposal for what Chern-Simons should assign to a point:
The $\otimes$-category of (certain) representations of $\widetilde{\Omega G}$.
Here, $\Omega G$ is the based loop group, and the tilde indicates that one should take the central extension inherited from the level $k$ central extensions of $LG$.
Another way of phrasing the proposal, that also works when $G$ is not connected (Dijkgraaf-Witten theory being the special cases thereof when $G$ is finite) is to say that it's the category of (certain) vector bundles over the moduli space of $G$-bundles over $[0,1]$ trivialized at $\{0,1\}$.
The precise definition is spelled out at minute 50 of the following video: http://youtu.be/2imygWqTET8
(and if you're going to watch it, I recommend watching from the beginning)
Added later: Here is a set of notes written by Qiaochu of a talk that I gave on the subject.
You can understand this philosophy as a generalization of Noether's theorem. Let me only state Noether's theorem in the quantum case because it's actually easier to understand there than in the classical case (for me, anyway).
As setup, we have a Hilbert space of states $V$ and a state vector $\psi \in V$ which evolves according to a Hamiltonian $H$, meaning (in natural units, so $\hbar = 1$) that
$$\psi(t) = e^{iHt} \psi.$$
Suppose in addition that we have a one-parameter family $g(t)$ of symmetries of our quantum system, meaning that $g(t)$ commutes with the Hamiltonian: $g(t) H = H g(t)$. In particular, $g(t)$ is a one-parameter family of unitary maps, and so by Stone's theorem $g(t)$ must have the form
$$g(t) = e^{iAt}$$
for some self-adjoint operator $A$ (which in physically relevant examples is often unbounded). (In the finite-dimensional case this is just saying that the Lie algebra of the unitary group is the Lie algebra of skew-adjoint matrices.) Noether's theorem is the observation that this means $A$ must also commute with $H$, which means that it is an observable which is conserved under time evolution in the sense that
$$e^{iHt} A e^{-iHt} = A$$
(time evolution of $A$ looks like conjugation in the Heisenberg picture). This general observation reproduces many of the familiar conserved quantities in physics. To give two examples:
- If $g(t)$ is translation in a space direction, $A$ is momentum in that direction. For example, if $g(t)$ is translation in the $x$ direction, $A = i \frac{\partial}{\partial x}$.
- If $g(t)$ is rotation around an axis, $A$ is angular momentum around that axis.
Because $A$ is a conserved quantity, it's natural to break up $V$ into eigenspaces of $A$ (corresponding to states where $A$ has a definite value), and the reason is that time evolution preserves all of these eigenspaces. This means that the statement "$\psi$ belongs to such-and-such eigenspace" is physically meaningful, e.g. the statement that $\psi$ has a fixed momentum.
The connection to representation theory comes from thinking of $g(t)$ as a representation of $\mathbb{R}$, so that the eigenspaces of $A$ are the isotypic components of this representation. Irreducible representations correspond to eigenvectors, which are, as above, states where $A$ has a definite and fixed value.
Now many physical systems come with a noncommutative group of symmetries, so it's natural to generalize $g(t)$ to an action of a nonabelian Lie group $G$, for example $SO(3)$, which we again posit to commute with $H$. What we might call the generalized Noether theorem is the observation that this implies that time evolution preserves the decomposition of $V$ into isotypic components of this representation, so it's again physically meaningful to say things like "$\psi$ belongs to the isotypic component corresponding to such-and-such irreducible representation" (in physics language, "$\psi$ transforms under...") because such statements are preserved by time evolution. This is the beginning of Wigner's classification (although that classification is relativistic whereas this story I've been telling is decidedly not so some tweaks need to be made). So you can think of the irrep a state belongs to as a "generalized conserved quantity."
(The reason we want to consider irreps is that they give more precise information while continuing to be physically meaningful. I could talk about e.g. particles whose momentum lies in a certain range instead of talking about particles with particular values of their momentum, but the latter is more precise so I do that first.)
The relationship to the groups $U(1), SU(2), SU(3)$ appearing in the standard model requires a bit more elaboration, because these groups don't act by physical symmetries (like the Poincare group) but by gauge symmetries. But that's a story that's a bit outside my competence to describe the physical relevance of. I can tell you that the $U(1)$ factor corresponds to charge conservation.
I should mention that I asked exactly this question awhile ago, and after thinking about the answer I got I wrote this blog post about a toy model of quantum mechanics on a finite graph that you might find helpful.
Best Answer
Strictly speaking, this answer is not about the 3d TQFT which you mention in your question, but rather a 2d version of Dijkgraaf-Witten theory (described in Section 2 of Freed-Hopkins-Lurie-Teleman).
To every finite group $G$, there is a 2d TQFT $Z_G$ which assigns to a closed orientable surface $\Sigma$ the following sum over isomorphism classes of $G$ local systems $P \to \Sigma$: $$ Z_G(\Sigma) = \sum 1/|Aut(P)| = |Hom(\pi_1(\Sigma),G)|/|G|. $$ In the framework of TQFT as a symmetric monoidal functor $Bord \to Vect$, this assigns to a circle the space of class functions on $G$ (which is a commutative Frobenius algebra under convolution). We can extend further and define $Z_G$ on a point to be the category of representations of $G$ (or alternatively, the group algebra of $G$, depending on your set-up).
Analysing this TQFT on surfaces allows you to recover interesting group-theoretic identities. For example, by cutting up the surface $\Sigma$ into pairs of pants, recovers the following formula (probably first due to Frobenius): $$ Z_G(\Sigma) = \sum_{V\in \widehat{G}} \left(\frac{\dim V}{|G|}\right)^{\chi(\Sigma)}. $$ There are similar formulas involving the other entries in the character table for $G$, by considering surfaces with boundary (which can be thought of as counting $G$ local systems on a closed surface with singularities).
These formulas were used by Hausel and Rodriguez-Villegas to compute data about the Hodge numbers of character varieties in their paper Mixed Hodge Polynomials of Character Varieties.
The recent work of Ben-Zvi and Nadler Character Theory of a Complex Group is in some sense a categorified analogue of this TQFT, but where the finite group is replaced by a complex reductive group (as explained in the introduction). In ongoing work of myself with the authors, we are trying to understand what this structure says about the cohomology of character varieties.
At the risk of over advertising my own work, let me also mention this paper: Spin Hurwitz Numbers and TQFT, which describes an analogue of the Dijkgraaf-Witten TQFT for surfaces with spin structure.
There are probably many other references which I will try to add later...