I have come across tensor products many times in physics, namely for matrices, vector-space elements, Hilbert-space elements (quantum states), and representations of groups and algebras. However, the abstract meaning of the tensor product still alludes me. Although I can understand its context in physics, I cannot see that its physical description and its abstract definitions, for example those given on Wikipedia, are equivalent. Could someone provide me with a geometrical, as well as an algebraic, interpretation of the tensor product and also explain why it is required?
Linear Algebra – Physical Meaning of the Tensor Product
abstract-algebralinear algebrarepresentation-theorytensor-products
Related Solutions
Yes, at least for finite dimensional vector spaces. First, I'll show you for two vector spaces $H_1$ and $H_2$ and you can proceed by induction. Use the fact that any linear operator can be represented as a sum of tensor product of linear operators, and if $H$ is Hermitian then $\frac{H + H^\dagger}{2} = H$. That is, $$H = \frac{\sum_{i = 1}^m A_i \otimes B_i + A_i^\dagger \otimes B_i^\dagger}{2}$$ for linear operators $A_i :H_1 \to H_1$ and $B_i:H_2 \to H_2$. Now $$ H = \sum_{i = 1}^m \left(\frac{A_i + A_i^\dagger}{2}\right) \otimes \left(\frac{B_i + B_i^\dagger}{2}\right) - \sum_{i = 1}^m \left(\frac{A_i - A_i^\dagger}{2i}\right) \otimes \left(\frac{B_i - B_i^\dagger}{2i}\right),$$ and each term is a tensor product of Hermitian operators.
Well, this may not qualify as "geometric intuition for the tensor product", but I can offer some insight into the tensor product of line bundles.
A line bundle is a very simple thing -- all that you can "do" with a line is flip it over, which means that in some basic sense, the Möbius strip is the only really nontrivial line bundle. If you want to understand a line bundle, all you need to understand is where the Möbius strips are.
More precisely, if $X$ is a line bundle over a base space $B$, and $C$ is a closed curve in $B$, then the preimage of $C$ in $X$ is a line bundle over a circle, and is therefore either a cylinder or a Möbius strip. Thus, a line bundle defines a function $$ \varphi\colon \;\pi_1(B)\; \to \;\{-1,+1\} $$ where $\varphi$ maps a loop to $-1$ if its preimage is a Möbius strip, and maps a loop to $+1$ if its preimage is a cylinder.
It's not too hard to see that $\varphi$ is actually a homomorphism, where $\{-1,+1\}$ forms a group under multiplication. This homomorphism completely determines the line bundle, and there are no restrictions on the function $\varphi$ beyond the fact that it must be a homomorphism. This makes it easy to classify line bundles on a given space.
Now, if $\varphi$ and $\psi$ are the homomorphisms corresponding to two line bundles, then the tensor product of the bundles corresponds to the algebraic product of $\varphi$ and $\psi$, i.e. the homomorphism $\varphi\psi$ defined by $$ (\varphi\psi)(\alpha) \;=\; \varphi(\alpha)\,\psi(\alpha). $$ Thus, the tensor product of two bundles only "flips" the line along the curve $C$ if exactly one of $\varphi$ and $\psi$ flip the line (since $-1\times+1 = -1$).
In the example you give involving the torus, one of the pullbacks flips the line as you go around in the longitudinal direction, and the other flips the line as you around in the meridional direction:
Therefore, the tensor product will flip the line when you go around in either direction:
So this gives a geometric picture of the tensor product in this case.
Incidentally, it turns out that the following things are all really the same:
Line bundles over a space $B$
Homomorphisms from $\pi_1(X)$ to $\mathbb{Z}/2$.
Elements of $H^1(B,\mathbb{Z}/2)$.
In particular, every line bundle corresponds to an element of $H^1(B,\mathbb{Z}/2)$. This is called the Stiefel-Whitney class for the line bundle, and is a simple example of a characteristic class.
Edit: As Martin Brandenburg points out, the above classification of line bundles does not work for arbitrary spaces $B$, but does work in the case where $B$ is a CW complex.
Best Answer
The concept of tensor products of vector spaces is required for the description of General Relativity. For example, the metric tensor $ \mathbf{g} $ of a pseudo-Riemannian manifold $ (M,\mathbf{g}) $ can be viewed as a ‘disjoint collection’ of tensor products. In terms of a local-coordinate system $ (x^{1},\ldots,x^{n}) $, we can express $ \mathbf{g} $ as $ g_{ij} \cdot d{x^{i}} \otimes d{x^{j}} $, where $ d{x^{i}} $ and $ d{x^{j}} $ are differential $ 1 $-forms and $ g_{ij} $ is just a scalar coefficient. At each $ p \in M $, what $ \mathbf{g} $ does is to take two tangent vectors at $ p $ as input and produce a scalar in $ \mathbb{R} $ as output, all in a linear fashion. In other words, at each point $ p $, we can view $ \mathbf{g} = g_{ij} \cdot d{x^{i}} \otimes d{x^{j}} $ as a bilinear mapping from $ {T_{p}}(M) \times {T_{p}}(M) $ to $ \mathbb{R} $, where $ {T_{p}}(M) $ denotes the tangent space at $ p $.
The metric tensor is the very object that encodes geometrical information about the manifold $ M $. All other useful quantities are constructed from it, such as the Riemann curvature tensor and the Ricci curvature tensor (of course, we still need something called an ‘affine connection’ in order to define these tensors). In General Relativity, the metric tensor encodes the curvature of spacetime, which can and does affect the performance of high-precision instrumentation such as the Global Positioning System (GPS).
For vector spaces over a field $ \mathbb{K} $, tensor products are usually defined in terms of multilinear mappings. You may have seen the more abstract definition using a system of generators, but it can be shown that these two definitions are the same. From the categorical point of view, both constructions satisfy the same universal property (this property is explicated in the Wikipedia article on tensor products), so they must be isomorphic in the category of $ \mathbb{K} $-vector spaces.
You need to understand that tensor products are algebraic objects, while tensor fields are geometrical objects. Even more fundamental than the concept of a tensor field is that of a tensor bundle. You can think of a tensor bundle as a collection of tensor products attached to individual points of a manifold in a consistent manner (by ‘consistent’, I mean that the axioms of a vector bundle must be satisfied). How we relate the tensor product attached to a point to the tensor product attached to another point is what the subject of differential geometry is all about. On the other hand, what we do with the tensor product attached to a fixed single point is pure algebra.
Any discussion of spacetime on a global scale requires General Relativity, which takes into account the curvature of spacetime. However, in a small neighborhood of any fixed point in spacetime, one always has what is called a ‘local inertial frame’. In such a frame, gravitational effects can be transformed away, in which case, General Relativity simply reduces to Special Relativity, which some mathematicians view as nothing more than just advanced linear algebra. This illustration reinforces the idea that ‘global is geometry but local is algebra’.