It is true that both automorphisms $\alpha$ and $\beta$ stabilise the Cartan subalgebra $\mathfrak{h}$ consisting of the diagonal matrices, and both stabilise the root basis
$B:= (\alpha_i : diag(c_1, ..., c_n) \mapsto c_i-c_{i+1})_{i=1,..., n}$.
But only $\alpha$ stabilises the whole épinglage $(\mathfrak{g}, \mathfrak{h}, B, (E_{i+1,i})_{\alpha_i})$ which additionally contains fixed basis vectors of the root spaces of the $\alpha_i$, here $E_{i+1, i}$. An automorphism of $\mathfrak{g}$ which stabilises all these data is then uniquely determined by its induced action on the Dynkin diagram: that is the content of Corollaire 1 to the Bourbaki proposition whose proof (but not assertion) I doubt in MSE/764696. Further, if you fix any other épinglage and take the Lie algebra automorphism $\alpha'$, corresponding to the same Dynkin diagram automorphism, with respect to that épinglage, then $\alpha = \gamma \circ \alpha' \circ \gamma^{-1}$ for a $\gamma \in Aut_0(\mathfrak{g})$, so in particular $\alpha$ and $\alpha'$ have the same invariant subalgebra. I do not know if this is what Fulton and Harris had in mind, but I think it gives a reasonable perspective.
I do not know if among the automorphisms that induce a certain automorphism of the Dynkin diagram, the ones that stabilise an épinglage have the largest invariant algebras. It sounds reasonable.
Theorem: For any Lie group $G$, the assignment $H\mapsto\text{Lie}(H)$ is a bijection between (equivalence classes of) embeddings (not necessarily closed) $H\hookrightarrow G$ of connected Lie groups $H$ into $G$ and Lie subalgebras ${\mathfrak h}\subseteq {\mathfrak g}:=\text{Lie}(G)$.
As an example of a non-closed embedding, consider the abelian Lie group ${\mathbb T}^2$ with Lie algebra ${\mathbb R}^2$. Any line through the origin with irrational slope defines a Lie subalgebra of ${\mathbb R}^2$ corresponding to a non-closed embedding ${\mathbb R}\hookrightarrow {\mathbb T}^2$ densely winding ${\mathbb R}$ around ${\mathbb T}^2$.
In your context, you encounter two instances of this:
The (closed) connected subgroup $\text{Aut}^{\circ}({\mathfrak g}) \leq \text{GL}({\mathfrak g})$ corresponds to the Lie subalgebra $\text{Lie}(\text{Aut}({\mathfrak g}))\cong \text{der}({\mathfrak g})$ of $\text{Lie}(\text{GL}({\mathfrak g}))\cong {\mathfrak g}{\mathfrak l}({\mathfrak g})$.
There is a unique Lie group embedding into $\text{Aut}^{\circ}({\mathfrak g})$ corresponding to the subalgebra $\text{ad}({\mathfrak g})\subseteq\text{der}({\mathfrak g})$ of $\text{Lie}(\text{Aut}({\mathfrak g}))$, and the domain of that embedding is the group the book is talking about. Note, however, that this group is not closed in general. See e.g. Exercise 18.1.2 in Hilbert-Neeb's "Structure and Geometry of Lie Groups".
Best Answer
What do you mean by “the brackets must be given by the change of basis formula”? If $\mathfrak{g}$ is a Lie algebra and you pick a basis $B=\{x_1,\ldots, x_n\}$ of $\mathfrak{g}$, then you can write $[x_i,x_j] = \sum_{k=1}^n c_{ij}^k x_k$, and the scalars $(c_{ij}^k)_{i,j,k \in \{1,2,\ldots,n\}}$ (called the “structure constants” with respect to the basis $B$) completely determine the Lie bracket.
If $L$ is an invertible linear map and $L(x_j) = \sum_{i=1}^n l_{ij}x_i$, we can write out the conditions on the matrix of $L$ given by the requirement that $[L(x),L(y)] = L([x,y])$ explicitly in terms of the structure constants:
$$[L(x_i),L(x_j)] = \left[\sum_{p=1}^n l_{pi}x_p,\sum_{q=1}^n l_{qj} x_q\right] = \sum_{p,q,r=1}^n l_{pi}l_{qj}c_{pq}^r x_r$$
On the other hand,
$$L([x_i,x_j]) = L\left(\sum_{k=1}^n c_{ij}^kx_k\right)= \sum_{k,r=1}^n l_{rk}c_{ij}^kx_r$$
Thus $L$ is an automorphism of $\mathfrak{g}$ precisely when
$$\sum_{k=1}^n l_{rk}c_{ij}^k = \sum_{p,q=1}^n l_{pi}l_{qj}c_{pq}^r.$$
This is a system of $n^3$ degree 2 equations in the entries of the matrix $(l_{ij})$.
Hopefully this convinces you that you the condition that $L([x,y]) = [L(x),L(y)]$ for all $x,y \in \mathfrak{g}$ is not automatically satisfied by any element of $\mathrm{GL}(\mathfrak{g})$, but to be completely concrete, take $\mathfrak{g} = \mathfrak{s}_2$ the two-dimensional non-abelian Lie algebra. This has basis $\{x_1,x_2\}$ and $[x_1,x_2]=x_2$ (and all other brackets of basis elements $0$ by the alternating property) so that the structure constants are zero except for $c_{12}^2$. Then if $L\colon \mathfrak{s}_2 \to \mathfrak{s}_2$ is a linear map with matrix $\left(\begin{array}{cc} a & b \\ c & d\end{array}\right)$ we see that the entries of the matrix must satisfy
$$r=1: \quad l_{12} =0$$ $$r=2: \quad l_{22} = l_{11}l_{22}$$
Since $L \in \mathrm{GL}(\mathfrak{g})$, it follows that the matrix of $L$ has the form $\left(\begin{array}{cc} 1 & 0 \\ l_{21} & l_{22} \end{array}\right)$ (where $l_{22} \neq 0$ since we require $L$ to be invertible).