In this question, I am primarily referring to notations and diagrams used in The Penrose Transform by Baston and Eastwood.
Given a Lie algebra $\mathfrak{g}$ and a parabolic subalgebra $\mathfrak{p}$, in Baston and Eastwood, a notation for the (standard) parabolic subalgebra is given following the rule: draw a Dynkin diagram for $\mathfrak{g}$, and then for every simple root in algebra but not in the parabolic subalgebra, replace the corresponding dot with a cross $\times$.
I believe I understand how to view these parabolic subalgebras in the fundamental representation of the larger algebra. As an example, if our algebra was $\mathfrak{sl}(3,\mathbf{C})$ and the parabolic subalgebra was the Borel subalgebra, the subalgebra would contain the two simple roots, their product, and the diagonal (corresponding to the Cartan subalgebra). Because the simple roots (in the Cartan-Weyl basis) have 1's in the (1,2) and (2,3) slots in the matrices, respectively, these fill out the upper triangular matrix. This would correspond to the Dynkin diagram for $\mathfrak{sl}(3,\mathbf{C})$ except both dots are replaced by $\times$.
Similarly, one could consider the parabolic subalgebra as just that containing only one of the simple roots, in which case your subalgebra would be the matrices with either 2 zeroes along the left or along the bottom, depending on which simple root you excluded. (See pages 15-16 of Baston-Eastwood.) These would correspond to the same Dynkin diagram, except only one dot is replaced with $\times$.
The above makes sense to me: if we can represent an algebra as a matrix algebra, the subalgebra can be represented as matrix subalgebra, particularly with zeroes in some slots.
What I don't understand is that a similar notation is used to represent certain weight vectors (that correspond to certain representations) and those representations are then also said to be equal to some subspace of matrices in the fundamental representation of say, $\mathfrak{sl}(3,\mathbf{C})$.
As an example: let $\mathfrak{g} = \mathfrak{sl}(3,\mathbf{C})$ and let $\mathfrak{p} = \mathfrak{b}$, the Borel subalgebra. Then, consider the highest weight vectors (for $\mathfrak{p}$) given by
$$\lambda_1 = 2 \lambda_\alpha – \lambda_\beta$$
and
$$\lambda_2 = – \lambda_\alpha + 2 \lambda_\beta$$
where $\lambda_{\alpha_i}$ is the fundamental weight associated to the simple root $\alpha_i$. In the book (specifically, page 82), the direct sum of the representations associated to these highest-weight vectors can also be described as the set of matrices
$$\begin{pmatrix}
0 & 0 & 0\\
\ast & 0 & 0 \\
0 & \ast & 0
\end{pmatrix}\,. $$
I do not understand the correspondence that Baston and Eastwood are making here. It seems like some strange mixture of the description above (how Dynkin diagrams of parabolic subalgebras can describe matrix subalgebras of the larger algebra) and how highest weight vectors can correspond to representations.
My hope is that someone might be able to elucidate where this relationship comes from.
Best Answer
Firstly, yes (conjugacy classes of) parabolic subalgebras are in 1-to-1 correspondence with subsets of the simple roots as you say and we denote these with crossed nodes on our Dynkin (or Satake) diagrams.
Secondly, irreducible representations are in correspondence with dominant integral weights (i.e. their highest weights). In other words these are weights which are a linear combinations of the fundamental weights $\omega_i$ (with coefficients $a_i$ that are nonnegative integers) which are natural paired with the simple roots. Specifically, they are the dual basis to the simple coroots. Thus we can specify an irreducible representation again by a decorated Dynkin diagram where we put $a_i$ over the corresponding node.
Baston and Eastwood go on to look at how the parabolic subalgebra acts on representations of $\mathfrak{g}$. Now there are a few nice interactions of this notation. If we take a given parabolic subalgebra and choose a representation $V$ which has $0$'s on the uncrossed nodes and non-zero integers on the crossed nodes, the parabolic subalgebra induces a filtration (or flag) on the representation which terminates in a 1-dimensional subspace (i.e. a line). Now looking at the conjugacy class of this parabolic we get a manifold $G/P$ and with each element we associate a line in $V$. In fact this is a realisation of $G/P$ as a projective variety.
More generally, for an arbitrary representation $V$ we just get a flag in $V$. Hence we call $G/P$ a flag manifold (or variety).
The smallest subspace in the flag is an irreducible representation of $\mathfrak{p}$ which they denote with a Dynkin diagram with crossed nodes and numbers. [Edit: this only accounts for representations where all the numbers are positive integers, I will have a think about this more carefully and come back to fix it]. They then consider the space $\mathfrak{g}/\mathfrak{p}$ which can be identified with the tangent space $T_{\mathfrak{p}}G/P$. If your conception of $\mathfrak{p}$ is as the upper triangular matrices (so it is Borel) then you could realise this as the strictly lower triangular ones. The filtration induced on $\mathfrak{g}$ looks like: $$\mathfrak{g} = \mathfrak{p}^{(2)} \geq \mathfrak{p}^{(1)} \geq \mathfrak{p}^{(0)} \geq \mathfrak{p}^{(-1)} \geq \mathfrak{p}^{(-2)} \geq \{0\}$$
Where $$ \mathfrak{p}^{(1)} = \begin{pmatrix} *&*&*\\ *&*&* \\0&*&* \end{pmatrix}, \mathfrak{p}^{(0)} = \mathfrak{p} = \begin{pmatrix} *&*&*\\ 0&*&* \\0&0&* \end{pmatrix}, \mathfrak{p}^{(-1)} = \begin{pmatrix} 0&*&*\\ 0&0&* \\0&0&0 \end{pmatrix}, \mathfrak{p}^{(-2)} = \begin{pmatrix} 0&0&*\\ 0&0&0 \\0&0&0 \end{pmatrix}$$ Now look at $\mathfrak{p}^{(1)}/ \mathfrak{p} \leq \mathfrak{g}/\mathfrak{p}$ which can be identified with $\begin{pmatrix} 0&0&0\\ *&0&0 \\0&*&0 \end{pmatrix}$ and ask how does $\mathfrak{p}$ act on it. The nilradical $\mathfrak{p}^{(-1)}$ acts trivially so this is really just an action by the Levi subalgebra which in this case is exactly the Cartan subalgebra. Comparing this action with the irreducible $\mathfrak{p}$ representations we defined a minute ago and we see these are the ones defined by the given choices of $\mathfrak{g}$ representation.
Edit: I've taken a longer look at this and I'm still not sure exactly how to find the irreducible reps for weights that are only dominant for $\mathfrak{p}$ and not $\mathfrak{g}$. Even using example 3.2.2 has left me confused. They say there that our Dynkin diagram (two crosses) with labelling $(p,q)$ corresponds to the 1-dimensional representation $L^p \otimes (H/L)^{p+q}$ or $(H/L)^p \otimes (V/H)^{p+q}$ where I imagine powers mean tensor powers, $L^{-1}$ means $L^*$ and $\{0\} \leq L \leq H \leq V = \mathbb{C}^3$ is our flag. However, the example we are looking at is $(L^* \otimes H/L) \oplus ((H/L)^* \otimes V/H)$ both of which correspond to $(-1,2)$ apparently. $(2,-1)$ seems then to correspond to $L^2 \otimes H/L$ or $(H/L)^2 \otimes V/H$ which do not fit into $\mathfrak{g} \leq V^* \otimes V$. This all seems to contradict the story you identify later in the book so either I've seriously misunderstood something here or there's a mistake in one of these parts.
The one thing I have gleaned from the examples is that we can think of the numbers over the crosses can be thought of as taking tensor powers of the "factors" of our representations.
Edit 2: Okay I've thought about this even more. What is happening in this case should go as follows. Since we have picked a Borel subalgebra, every weight (in any representation) is fixed by the Levi since it is just a Cartan subalgebra. Thus the representation specified by $(p,q)$ over 2 crossed nodes is just the weight space for the weight $p\omega_1 + q\omega_2$ thought of as a 1-dimensional representation over the Levi. You may pick your favourite representation containing that weight for this to live inside. In your scenario we note that those two weights are actually roots so we can think of them inside the adjoint representation. I still think there is something wrong with example 3.2.2 or at least my understanding of it.
For the general case of a partial flag manifold I believe the story would go instead: now that the Levi is no longer a Cartan subalgebra it can generate bigger irreducible representations. In order to avoid over counting we need to find the highest weight for these and that leads back to the idea that the uncrossed nodes must have positive numbers attached. Again, the representation could be found inside some $\mathfrak{g}$ representation containing that weight (we may need to choose one that has a 1-dimensional weight space for that weight). Then the representation is simply that generated by the action of the Levi on this chosen weight space.