I asked this question a few weeks ago and was dissatisfied with most of the answers I found on the internet, so I eventually managed to procure a copy of Griffiths' excellent text on elementary particles (really, all of his texts are excellent) which includes a section exactly answering my question with what I was looking for. I decided then to answer it myself, in case some other curious person reads this and wants to know.
This is just a very cursory explanation, intended to answer my own question to my own satisfaction.
Griffiths starts by introducing what are basically three copies of EM charge called color charge, and proposes these to be three-element column vectors:
$$c_{red} = \left(\begin{array}{c}1\\0\\0\end{array}\right), c_{blue} = \left(\begin{array}{c}0\\1\\0\end{array}\right), c_{green}=\left(\begin{array}{c}0\\0\\1\end{array}\right).$$
These could in principle could take any vector value whatsoever, except for effects of symmetry in the theory and color confinement.
To figure out how these vector charges interact, we turn to the Gell-Mann $\lambda$-matrices, which are to $SU(3)$ what the Pauli matrices are to $SU(2)$. These are listed by Griffiths, but writing matrices would be a pain; you can look them up on Wikipedia.
Griffiths then takes Feynman scattering amplitudes in lowest order for the chromodynamic interaction, and from these develops potentials for various interactions.
For quark-anti-quark, he has
$$V_{q\bar{q}}(r) = -f\frac{\alpha_s\hbar c}{r}.$$
This is a long-range force in principle, but it is made short-range due to confinement. It takes the same form as the Coulomb potential. The important thing here is the $f$, which Griffiths calls the "color factor". This color factor is like $q_1q_2$ in electrostatics or $\mathbf{p}_1\circ\mathbf{p}_2$ for dipole-dipole forces, and will depend on the color state of the interacting particles in question. It is calculated by
$$f = \frac{1}{4} (c_3^\dagger\lambda^\alpha c_1)(c_2^\dagger\lambda^\alpha c_4),$$
where summation is implied over $\alpha$. Here $c_1$ is charge of incoming quark, $c_3$ charge of outgoing quark, and $c_2,c_4$ charges of incoming and outgoing antiquark.
As an example, Griffiths calculates the interaction between red and anti-blue.
$$c_1=c_3=\left(\begin{array}{c}1\\0\\0\end{array}\right), c_2=c_4=\left(\begin{array}{c}0\\1\\0\end{array}\right).$$
Hence
$$f = \frac{1}{4}\left[(1,0,0)\lambda^\alpha\left(\begin{array}{c}1\\0\\0\end{array}\right) \right]\left[ (0,1,0)\lambda^\alpha \left(\begin{array}{c}0\\1\\0\end{array}\right)\right] = \frac{1}{4}\lambda^\alpha_{11}\lambda^\alpha_{22}.$$
That is, it involves a sum over products of the 1st diagonal element and 2nd diagonal element o each of the Gell-Mann matrices. By looking at their form, the only matrices with both these elements non-zero are the ones labeled by Griffiths $\lambda^3$ and $\lambda^8$. These lead to
$$f = \frac{1}{4}[(1)(-1)+(1/\sqrt{3})(1/\sqrt{3})] = -\frac{1}{6},$$
$$V_{r\bar{b}} = \frac{1}{6}\frac{\alpha_s \hbar c}{r},$$
which is evidently a repulsive force. Griffiths also calculates other interactions. For instance, quark-antiquark singlet interactions, $(1/\sqrt{3})(r\bar{r}+b\bar{b}+g\bar{g})$, which have color factor $f=\frac{4}{3}$ and thus are attractive, explaining confinement of quarks to color-singlet states and the lack of observation for colored states. He also calculates quark-quark interactions, which have a slightly different potential,
$$V_{qq}=f\frac{\alpha_s \hbar c}{r}.$$
As an example, he calculates red-red interaction; it has factor 1/3, hence is repulsive.
There is a lot of this in this very wonderful book, but that's enough to satisfy my curiosity of what color charge is and how it works. Hopefully it is helpful to anyone else. Of course, this was highly simplified for the sake of my own simplified brain and no doubt infuriating to pedants in the field, but if you would like a better explanation and understanding, this was all taken from Chapter 8.4 of Introduction to Elementary Particles by David Griffiths, published by Wiley-VCH, Second Revised Edition -- just to cite sources.
Best Answer
Color charge is the representation of the SU(3) gauge group. The representation theory of SU(3) is described below:
The basic representation is called the "3" or the fundamental, or defining, representation. It is a triplet of complex numbers $V^i$, which transform under a 3 by 3 SU(3) matrix by getting multiplied by the matrix. The value of "i" is sometimes called "red","green","blue", so that a quark which is all in the $V^1$ direction is red, etc. This is reasonable, because every fundamental representation vector is a linear combination with complex coefficient of red-green-blue.
Hermann Weyl proved that every other representation occurs somewhere among the tensors: $V^{i_1,i_2 .. i_n}_{j_1,j_2,...,j_m}$ where the upper indices transform by multiplying the index by the SU(3) matrix, and lower indices by multiplying by the conjugate matrix, which is also the inverse. Addition of representations is just like addition of angular momentum in quantum mechanics, by taking tensor products.
Warm-up: Quick Representations SU(2)
For SU(2), the invariant tensors are $\delta^i_j$, $\epsilon_{ij}$, and $\epsilon^{ij}$, which are trace and two-dimensional volume. You can raise and lower indices using $\epsilon$ tensors, so every tensor representation is equivalent to one with all the indices down. Any three antisymmetric indices are necessarily zero, and any two antisymmetric indices can be deleted by contracting them with the appropriate epsilon tensor. So there are only clumps of symmetric indices in a representation.
The irreducible representations are exhausted by the fully symmetric tensors with all indices down:
$$ T_{i_1, i_2 , .... i_n}$$
Because when you multiply two of these, you get a tensor
$$ T_{i_1, i_2,....,i_n;j_1, j_2, ... j_n'}$$
with symmetry on permuting the first n indices and the last n' indices. I will write this as (n,n'). By contracting the $\epsilon$ tensor on one of the i's and one of the j's (they all give the same result because they are symmetric), you extract an (n-1,n'-1) representation from this. The remainder is fully symmetric on n+n' indices, because you have removed the antisymmetric part. The result is the decomposition
$$(n,n') = (n+n') + (n-1,n'-1)$$
So that, recursively, the tensor product of (n) and (n') decomposes into
$$(n+n'), (n+n'-2), (n+n'-4), ... (1) or (0)$$
where the last term is 1 if n+n' is odd, or 1 if n+n' is even. If you recognize that the n-index totally symmetric tensor with two possible values for each index has exactly n+1 different components, you realize that the (n) representation is just the spin (n/2) representation, and the decomposition above is the familiar Clebsch-Gordon series for addition of quantum angular momentum.
The tensorial method is never taught for some reason, but it is the quickest way to do Clebsch-Gordon decompositions in real life.
Back to SU(3)
SU(3) transformations preserve inner products, and 3-dimensional complex volumes, so there are three basic invariant tensors, $\delta^i_j$, $\epsilon_{ijk}$, and $\epsilon^{ijk}$. The $\epsilon$ tensors allow you to take the antisymmetric part on any two upper indices and turn it into a lower index, or the anyisymmetric part of two lower indices and turn it into an upper index.
The irreducible representations of SU(3) are all tensors
$$ T^{i_1,i_2,....,i_n}_{j_1,j_2,...,j_m}$$
which are fully symmetric on the upper indices, and fully symmetric on the lower indices. To see this, call this representation (n;m), and tensor two such representations to produce
$$(n,n';m,m')$$
Which means n totally symmetric upper indices followed by n' totally symmetric upper indices, and m totally symmetric lower indices followed by m' totally symmetric lower indices.
Then, acting the epsilon tensor between the n and n' clump produces
$$(n,n'; 1,m,m')$$
leaving behind $(n+n';m,m')$, since it takes away the antisymmetric part. The recursive rule is as ollows
$$(n_1,...,n_k; m_1,,...,m_k) \rightarrow $$ $$ (n_1+n_2,n_3,...,n_k;m_1,...,m_k) \oplus (n_1-1,n_2-1,n_3,...n_k; 1, m_1, ...,m_k)$$
$$(n_1,...,n_k; m_1,....,m_k) \rightarrow$$ $$ (n1,...,n_k; m_1+m_2,m_3,...,m_k) \oplus (1,n_1,...,n_k;m_1-1,m_2-1,...,m_k)$$
These rules correspond to acting with the two epsilon tensors, and they terminate on terms of the form (n;m) in a finite number of steps, because either thing on the right hand side either has a smaller number of clumps, or a smaller sum of indices. Decomposing the traces out of (n;m) gives all the irreducible representations.
Removing the traces
After you reduce the tensors to (n;m), you get rid of all the trace parts, by subtracting $\delta^i_j$ times an (n;m) tensor. This turns every (n,m) of the previous section into a series of trace-reduced (n-k;m-k) parts which go from k=0 to k=min(m,n). These tensors are the true irreducible representations.
The color charge is defined as the representation of SU(3) of the colored object. The representation is indexed by (n,m). The color-charge of an instance of the actual object is an n-list of rgb-rgb-..rgb values, where the order doesn't matter, and an m-list of cmy-cmy...-cmy colors, where the order doesn't matter. These are the basic colors, and you superpose them with arbitrary complex numbers, but imposing the trace condition, which is a little hard to state in RGB language--- it says that the sum of (r,LIST;c,LIST) + (g,LIST;m,LIST') + (b,LIST;y,LIST') is zero for any colors in LIST and LIST'.
To add two color charges, you use the procedure above for tensor products. The sum of two color charges is a complicated mixture of color-charges, given by decomposing the tensor representation.
These rules are relatively complicated, so be thankful that the only fundamental color representations in the world are quark fundamental triplets, and gluon one-up-index, one-down-index traceless tensors, and that all hadrons are singlets.