Dear Mikhail,
If I understand correctly, your Lemma 2 is implied by
SGA 3 Exposé 22, Théorème 5.4.7.
Everything is on a general base S (that you may take as your algebraically closed field). The kind of subgroup you want is called "de type (R)" (see Définition 5.2.1) and a subset of R that corresponds to such a group is also called "de type (R)". Now the theorem above exactly says that when a subset of R is closed, it is "de type (R)" which exactly means that there is a corresponding connected subgroup of G. By the way, Théorème 5.4.7 does not assume the subset to be symmetric, and you get things like Borel subgroups if you take only "half" of the roots. In the symmetric case, the group is reductive by Proposition 5.10.1.
Hope this helps.
Baptiste Calmès
Nothing is "better-suited to using the classical language"; if you cannot express things clearly via schemes then think harder about it until you can. Also, any connected reductive group over a field has a unique quasi-split inner form. (See Proposition 7.2.12 in the article Reductive Group Schemes in the Proceedings of the 2011 Luminy Summer School on SGA3 for a proof of the same result more generally over any semi-local base scheme.)
If $k_s/k$ is a separable closure and $H$ is the $k$-split form of $G$, with a split maximal $k$-torus $S$ and Borel $k$-subgroup $B\supset S$, there is a ${\rm{Gal}}(k_s/k)$-equivariant exact sequence of groups $$1 \rightarrow H^{\rm{ad}}(k_s)\to {\rm{Aut}}_{k_s}(H_{k_s})\stackrel{\pi}{\to} {\rm{Aut}}(R,\Delta)\to 1$$ where $(R,\Delta)$ is the based root datum for $(H,B,S)$. A pinning $\{u_{\alpha}\}$ identifies ${\rm{Aut}}(R,\Delta)$ with ${\rm{Aut}}_k(H,S,B,\{u_{\alpha}\})$ and thereby defines a "forgetful" homomorphic section
$$\sigma:{\rm{Aut}}(R,\Delta) \rightarrow {\rm{Aut}}_k(H) \subset
{\rm{Aut}}_{k_s}(H_{k_s})$$
to $\pi$.
Consider the class $[G] \in {\rm{H}}^1(k_s/k, {\rm{Aut}}_{k_s}(H_s))$ corresponding to $G$. The formalism of non-abelian Galois cohomology as in section 5 of Chapter I of Serre's book Galois Cohomology shows that the set of isomorphism classes of inner forms of $G$ (i.e., the image of ${\rm{H}}^1(k_s/k, G^{\rm{ad}}(k_s))$ in ${\rm{H}}^1(k_s/k, {\rm{Aut}}_{k_s}(G_{k_s})$) is identified with the ${\rm{H}}^1(\pi)$-fiber through $[G]$.
In particular, ${\rm{H}}^1(\sigma)({\rm{H}}^1(\pi)([G]))$ is a class in the same fiber as $[G]$, so for the existence of a quasi-split inner form of $G$ it suffices to show that all classes in the image of ${\rm{H}}^1(\sigma)$ are quasi-split. But $\sigma$ is defined via the identification of ${\rm{Aut}}(R,\Delta)$
with ${\rm{Aut}}_k(H,S,B, \{u_{\alpha}\})$ and so factors through the subgroup ${\rm{Aut}}_{k_s}(H_{k_s},B_{k_s}) \subset {\rm{Aut}}_{k_s}(H_{k_s})$. Thus, any class in the image of ${\rm{H}}^1(\sigma)$ corresponds to the isomorphism class of a $k$-group $H'$ obtained through ${\rm{Gal}}(k_s/k)$-twisting of $H$ preserving its Borel $k$-subgroup $B$, so by design $H'$ admits a Borel $k$-subgroup $B'$ and thus $H'$ is quasi-split over $k$.
The remaining claim (not posed in the question, but mentioned at the start of this answer and very important in practice) is that the quasi-split inner form is unique. That is, if $G_1$ and $G_2$ are quasi-split inner forms of $G$ then they are $k$-isomorphic. More specifically, if $G_1$ is the quasi-split inner form made via the above construction and $G_2$ is any quasi-split inner form then $G_2 \simeq G_1$. This lies deeper in the sense that it rests on a more closer study of the preceding construction.
First, it is an instructive exercise to prove in a clean way that $G_2$ is necessarily an inner form of $G_1$, so we can rename $G_1$ as $G$ to reduce to showing that if $G$ admits a Borel $k$-subgroup $B$ with $(G,B)$ built from the split $k$-form via the above procedure resting on $\sigma$ then any quasi-split inner form $G'$ of $G$ is $k$-isomorphic to $G$. For a given choice of $k_s$-isomorphism $f:G'_{k_s} \simeq G_{k_s}$ corresponding to inner-twisting cocycles realizing $G'$ from $G$ via Galois descent, if we postcompose with a $G(k_s)$-conjugation we get a cohomologous 1-cocycle that has the same "inner-twisting" property. Thus, it is harmless to arrange that $f(B'_{k_s}) = B_{k_s}$, so then the inner-twisting is valued in the $B_{k_s}$-stabilizer subgroup of $G^{\rm{ad}}(k_s)$. But this stabilizer is exactly $B^{\rm{ad}}(k_s)$ for $B^{\rm{ad}} = B/Z_G$ so it suffices to prove
$${\rm{H}}^1(k,B^{\rm{ad}}) = 1.$$
In this way we can replace $G$ with $G^{\rm{ad}}$ without ruining any of our running hypotheses on $G$ but gaining that we may now assume $G$ is of adjoint type.
The $k$-unipotent radical of any parabolic $k$-subgroup of a connected reductive group is always $k$-split and so has vanishing ${\rm{H}}^1$. Thus, since $B = T \ltimes U$ for a maximal $k$-torus $T$ of $B$ and
$U = \mathscr{R}_{u,k}(B)$, so $B/U \simeq T$, it suffices to show ${\rm{H}}^1(k, T) = 1$ when $G$ is quasi-split of adjoint type and made from its split form via the procedure resting on $\sigma$. For this we finally have to make a serious observation that uses that $G$ is of adjoint type and is constructed from its split form via $\sigma$: the $k$-torus $T$ is "induced" (i.e., $T \simeq {\rm{R}}_{k'/k}({\rm{GL}}_1)$ for a finite etale $k$-algebra $k'$), from which it is immediate via Hilbert 90 that the desired ${\rm{H}}^1$-vanishing holds.
Why is $T$ induced? By inspection of how $\sigma$ is made, $T$ is build from Galois-twisting of a split "adjoint torus" $S = {\rm{GL}}_1^{\Delta}$ through Galois action on ${\rm{X}}(S_{k_s}) = \mathbf{Z}^{\Delta}$ via a permutation action on $\Delta$. The Galois-orbits on $\Delta$ then make explicit that the associated $k$-form $T$ of $S$ is an induced torus. (Explicitly, if we pick a point in each Galois-orbit on $\Delta$ then the stabilizer in ${\rm{Gal}}(k_s/k)$ of each such base point is an open subgroup corresponding to a finite separable extension $k'_i/k$, and one shows $T \simeq \prod_i {\rm{R}}_{k'_i/k}({\rm{GL}}_1)$.)
Best Answer
Split redictive groups are classified by a combinatorial structure called root datum. If your $G$ is split, the outer automorphism group is the constant group scheme of automorphisms of this structure preserving simple roots. If $G$ is semisimple simply connected or adjoint it is indeed just the automorphism group of the Dynkin diagram, but in the other semisimple cases it can be a proper subgroup of the latter; for general reductive groups it is finitely generated but not always finite. If $G$ is not split, you can get a twisted form of this constant scheme. SGA 3 Exp. XXIV may help.