$\newcommand{\Hom}{\mathrm{Hom}}$$\newcommand{\Z}{\mathbb{Z}}$$\newcommand{\Spec}{\mathrm{Spec}}$$\newcommand{\ov}[1]{\overline{#1}}$$\newcommand{\bb}[1]{\mathbb{#1}}$
When I write $\overline{F}$ below I mean the separable closure of $F$.
So, by definition $X^\ast(T):=\mathrm{Hom}_{\ov{F}-\mathrm{grps}}(T_{\ov{F}},\mathbb{G}_{m,\ov{F}})$. The action of $\sigma\in\Gamma_F$ (I'll use $\Gamma_F$ for the absolute Galois group) is defined as follows:
$$\sigma\cdot f:= \sigma_{\bb{G}_{m,\ov{F}}}\circ f\circ \sigma_{T_{\ov{F}}}^{-1}$$
where for any $F$-scheme $X$ we define $\sigma_{X_{\ov{F}}}$ to be the natural map $X_{\ov{F}}\to X_{\ov{F}}$ which, when one writes $X_{\ov{F}}:= X\times_{\Spec(F)}\Spec(\ov{F})$, is trivial on the $X$-component and is $\Spec(\sigma^{-1})$.
So, let's do an example very explicitly. Let's set $F$ to be any field and $E/F$ to be a degree $2$ Galois extension. Then, we can form the group $U(1)_{E/F}$, or just $U(1)$, defined as the kernel of the natural map $\mathrm{Res}_{E/F}\bb{G}_{m,E}\to \bb{G}_{m,F}$ given by the norm. This is a one-dimensional torus. If we write $E=F(\sqrt{d})$ where $d\in F$, then as an affine scheme we can write $U(1)\cong \Spec(F[a,b]/(a^2-db^2-1))$ where Hopf algebra structure is the map
$$F[a,b]/(a^2-db^2-1)\to F[a_1,b_1,a_2,b_2]/(a_1^2-db_1^2-1,a_2^2-db_2^2-1)$$
(where we have made the identification of $F[a,b]/(a^2-db^2-1)\otimes_F F[a,b]/(a^2-db^2-1)$ with $F[a_1,b_1,a_2,b_2]/(a_1^2-db_1^2-1,a_2^2-db_2^2-1)$) given by
$$a\mapsto a_1a_2+db_1b_2,\quad b\mapsto ab_2+ba_2$$
Let us calculate its character lattice. Well, we have a natural isomorphism $\varphi:U(1)_{\ov{F}}\to \bb{G}_{m,\ov{F}}$ given on coordinate rings (using $t$ for the paramter of $\bb{G}_{m,F}$) by
$$\ov{F}[t]\to \ov{F}[a,b]/(a^2-db^2-1):t\mapsto a+b\sqrt{d}$$
with inverse
$$\Spec(\varphi^{-1}):\ov{F}[a,b]/(a^2-db^2-1)\to \ov{F}[t]$$
given by
$$a\mapsto \frac{1}{2}(t+t^{-1}),\quad b\mapsto \frac{1}{2\sqrt{d}}(t-t^{-1})$$
It's then not hard to see that $\Hom_{\ov{F}-\mathrm{grps}}(U(1)_{\ov{F}},\bb{G}_{m,\ov{F}})=\Z\cdot\varphi\cong \Z$. Let's now see how $\Gamma_F$ acts on $\Z\cdot \varphi$. Of course, it suffices to specify how it acts on $\varphi$. Now, by definition we have that $\sigma\cdot \varphi$ is the map $ \sigma_{\bb{G}_{m,\ov{F}}}\circ \varphi\circ \sigma_{T_{\ov{F}}}^{-1}$. Let's see how this acts on coordinate rings. Namely, this map corresponds to a map
$$\ov{F}[t]\to \ov{F}[a,b]/(a^2-db^2-1)$$
which can be thought of as factorized as
\begin{equation}
$$\ov{F}[t]\xrightarrow{\sigma^{-1}}\ov{F}[t]\xrightarrow{\Spec(\varphi)}\ov{F}[a,b]/(a^2-db^2-1)\xrightarrow{\sigma}\ov{F}[a,b]/(a^2-db^2-1)$$
and we'll call these arrows 1, 2, and 3 (labeled left to right). Well, let's see where $t$ goes. In the first map it maps to $t$ again. Under the second map it maps to $a+b\sqrt{d}$ and under the last map it maps to $a+b\sigma(\sqrt{d})$.
In particular, note that $\Gamma_F$ acts through the quotient $\mathrm{Gal}(E/F)$ and if we denote by $\sigma$ the non-trivial element of $\mathrm{Gal}(E/F)$ this satisfies that $\sigma\cdot \varphi$ sends $t$ to $a-b\sqrt{d}$. Note though that this is just $(a+b\sqrt{d})^{-1}$ and so it's not hard to see that $\sigma\cdot \varphi=-\varphi$ (our groups are multiplicatively written so it might be more natural to write $\varphi^{-1}$ but this could be confused with the inverse of $\varphi$ so I wrote it additively).
In other words, $X^\ast(U(1))$ is the $\Gamma_F$-module $\mathbb{Z}$ where the $\Gamma_F$ structure is the composition
$$\Gamma_F\twoheadrightarrow \mathrm{Gal}(E/F)\to \mathrm{Aut}(\mathbb{Z})\cong \mathbb{Z}/2\mathbb{Z}$$
where the arrow $\mathrm{Gal}(E/F)\to\mathbb{Z}/2\mathbb{Z}$ is the obvious isomorphism.
To see the relationship between $\sigma_{T_{\ov{F}}}$ and $T(\ov{F})$ for a torus $T$ over $F$ note that $T(\ov{F})$ is nothing other than $\mathrm{Hom}_F(\mathrm{Spec}(\ov{F}),T)$. But, this is the same as $\Hom_{\ov{F}}(\Spec(\ov{F}),T_{\ov{F}})$. By the above we then know how $\Gamma_F$ should act on $f\in\Hom_{\ov{F}}(\Spec(\ov{F}),T_{\ov{F}})$:
$$\sigma\cdot f:= \sigma_{T_{\ov{F}}}\circ f\circ \sigma_{\Spec(F)_{\ov{F}}}^{-1}$$
Let's do two examples to see that this is what you should get:
- If $T$ is $\mathbb{G}_{m,F}$ then $T(\ov{F})=\ov{F}^\times$ and for $\alpha\in T(\ov{F})$ you expect that $\sigma_{T_{\ov{F}}}(\alpha)=\sigma(\alpha)$. Well, note that in scheme world this $\alpha\in\ov{F}$ corresponds to the map $\Spec(\ov{F})\to T_{\ov{F}}$ that corresponds to $\ov{F}$-algebra map $\ov{F}[t,t^{-1}]\to \ov{F}$ that takes $t$ to $\alpha$. Note then that $\Spec(\sigma\cdot f)=\Spec(\sigma_{\Spec(F)_{\ov{F}}}^{-1})\circ \Spec(f)\circ\Spec(\sigma_{T_{\ov{F}}})$ takes (using the sequence of maps on the right hand side) $t$ iteratively to $t$, then $\alpha$, then finally $\sigma(\alpha)$. So you get what you expect.
- If $T=U(1)_{E/F}$ then $T(\ov{F})=\{(\alpha,\beta)\in\ov{F}^2:\alpha^2-d\beta^2=1\}$. You expect that $\sigma$ acts on $(a,b)$ as $(\sigma(a),\sigma(b))$. To see this, note that $(\alpha,\beta)$ corresponds to the $\ov{F}$-algebra map $\ov{F}[a,b]/(a^2-db^2-1)\to \ov{F}$ taking $a$ to $\alpha$ and $b$ to $\beta$. Note then that if you consider $\Spec(\sigma\cdot f)=\Spec(\sigma_{\Spec(F)_{\ov{F}}}^{-1})\circ \Spec(f)\circ\Spec(\sigma_{T_{\ov{F}}})$ this takes $a$ to $\sigma(\alpha)$ and $b$ to $\sigma(\beta)$, just as you'd expect.
Choose $y \in L$ with $\sigma(y)=-y$.
Let's concretely look at the two examples case 1: $H = \pmatrix{0&0&0&1\\0&0&1&0\\0&1&0&0\\1&0&0&0}$ and case 2: $H = \pmatrix{1&0&0&0\\0&1&0&0\\0&0&1&0\\0&0&0&1}$.
Part 1: Seeing the standard Galois action $\alpha \mapsto \sigma \cdot \alpha$ on roots
Method 1: With the Lie algebra
Case 1: The Lie algebra of $G(k)$ is
$$\mathfrak g = \{ X \in M_4(L): Tr(X)=0 \text{ and } XH + H X^* =0\} = \{X \in M_4(L): Tr(X) = 0 \text{ and } x_{ij} = -\sigma(x_{5-j,5-i})\}$$
$$= \{ \pmatrix{a+by&c+dy&e+fy&gy\\h+iy&j-by&ky&-e+fy\\l+my&ny&-j-by&-c+dy\\oy&-l+my&-h+iy&-a+by} : a, ..., o \in k \}.$$
Surely its scalar extension $\mathfrak g_K$ is (isomorphic to) the split Lie algebra $\mathfrak{sl}_4(L) = \{X \in M_4(L): Tr(X)=0\}$, but let us look closely how Galois operates on that; namely, it does not just act on the entries of the matrix (when written as above), but rather as
$$\sigma_{\mathfrak g_L} := \sigma \otimes id_{\mathfrak g} \qquad \qquad (1)$$
on $\mathfrak g_L:= L \otimes_k \mathfrak g$. That is, e.g. the matrix
$$\pmatrix{0&1&0&0\\0&0&0&0\\0&0&0&0\\0&0&0&0} \in \mathfrak g_L$$ "correctly viewed" as an element of the tensor product $L \otimes_k \mathfrak g$ is
$$=\frac12 \otimes \underbrace{\pmatrix{0&1&0&0\\0&0&0&0\\0&0&0&-1\\0&0&0&0}}_{\in \mathfrak g} + \frac{1}{2y} \otimes \underbrace{\pmatrix{0&y&0&0\\0&0&0&0\\0&0&0&y\\0&0&0&0}}_{\in \mathfrak g}$$
so that $$\sigma(\pmatrix{0&1&0&0\\0&0&0&0\\0&0&0&0\\0&0&0&0})=
\frac12 \otimes \pmatrix{0&1&0&0\\0&0&0&0\\0&0&0&-1\\0&0&0&0} + \frac{1}{2\color{red}{\cdot(-y)}} \otimes \pmatrix{0&y&0&0\\0&0&0&0\\0&0&0&y\\0&0&0&0} = \pmatrix{0&0&0&0\\0&0&0&0\\0&0&0&-1\\0&0&0&0} $$
from which one gets $\sigma (\pmatrix{0&x&0&0\\0&0&0&0\\0&0&0&0\\0&0&0&0}) = \pmatrix{0&0&0&0\\0&0&0&0\\0&0&0&-\sigma(x)\\0&0&0&0}$ for all $x \in L$. Likewise, for example,
$$ \sigma(\underbrace{\pmatrix{1&0&0&0\\0&-1&0&0\\0&0&0&0\\0&0&0&0}}_{=:H_{\alpha_1}})= \underbrace{\pmatrix{0&0&0&0\\0&0&0&0\\0&0&1&0\\0&0&0&-1}}_{=:H_{\alpha_3}})$$
whereas $\pmatrix{0&0&0&0\\0&1&0&0\\0&0&-1&0\\0&0&0&0}) \in \mathfrak g$ and hence is actually fixed by $\sigma$.
Now in this case, indeed the diagonal matrices in $\mathfrak g$ form a maximal toral subalgebra (a.k.a. Cartan subalgebra) which contain a maximal split toral subalgebra (namely, the two-dimensional $\mathfrak s := \{ \pmatrix{a&0&0&0\\0&j&0&0\\0&0&-j&0\\0&0&0&-a} : a,j \in k \}$), so we can use the diagonal as above to see how Galois acts on the roots. Turns out that $\sigma$ maps the root space $\mathfrak g_{L, \alpha_1}$ onto the root space $\mathfrak g_{L, \alpha_3}$ (I call $\alpha_i = \eta_i - \eta_{i-1}$ the standard roots of the diagonal in $\mathfrak{sl}_4(L)$); and, accordingly, interchanges the coroots $H_{\alpha_1}$ and $H_{\alpha_3}$; in short, now there are many ways to define the action of Galois on the roots via $\sigma \cdot \alpha_1 := \alpha_3$ and $\sigma \cdot \alpha_2 =\alpha_2$. The actual technical formula to do it is
$$\sigma \cdot \alpha := \sigma \circ \alpha \circ \sigma_{\mathfrak g_L}^{-1}$$
where the first $\sigma$ is just the normal Galois on $L$, but the second (inner) $\sigma_{\mathfrak g_L}$ is defined via $(1)$ and in particular is not just Galois action on entries, so that this normal Galois action $\alpha \mapsto \sigma \cdot \alpha$ is not trivial.
Case 2: Now the Lie algebra of $G(k)$ is
$$\mathfrak g = \{ X \in M_4(L): Tr(X)=0 \text{ and } X = - X^* \} = \{X \in M_4(L): Tr(X) = 0 \text{ and } x_{ij} = -\sigma(x_{j,i})\}$$
$$= \{ \pmatrix{ay&b+cy&d+ey&f+gy\\-b+cy&hy&i+jy&k+ly\\-d+ey&-i+jy&my&n+oy\\-f+gy&-k+ly&-n+oy&(-a-h-m)y} : a, ..., o \in k \}.$$
Again its scalar extension $\mathfrak g_K$ is (isomorphic to) the split Lie algebra $\mathfrak{sl}_4(L)$, but this time Galois operates on that via
$$\sigma_{\mathfrak g_L} : X \mapsto -X^* (= -\sigma(X)^{tr}).$$
This seems to immediately give us an action $\sigma \cdot \alpha_i = -\alpha_i$ ($i=1,2,3$) on the roots, however we have to be very careful here. The problem is that although again the diagonal matrices in $\mathfrak g$ define a maximal toral subalgebra (CSA) $\mathfrak t$, and certainly $\mathfrak t_L$ is a split CSA in $\mathfrak g_L$, this $\mathfrak t$ does not necessarily contain a maximal split toral subalgebra of $\mathfrak g$ (e.g. if $k$ is a $p$-adic field, one can show there are non-zero ad-diagonalisable elements in $\mathfrak g$, but obviously none is in $\mathfrak t$): and we need that to set up the proper Galois action on the roots.
But if we assume that $k=\mathbb R$, $y=i$ (imaginary unit), then one can show that $\mathfrak g$ has no non-zero split toral subalgebra at all, and we are good to go, and indeed $\sigma$ operates on the entire root system as $-id$.
Method 2: With tori and algebraic geometry
(References: Galois action on character group and Galois action on the character group of a torus, but I find these a bit misleading because they focus on cases where the Galois action is just the one on points / matrix entries. Actually Definition of the unitary group via a cocycle on $\operatorname{GL}_n$ shows better what's going on here.)
Case 1: Our group has a torus with $T(L) = \{ \pmatrix{x_1&0&0&0\\0&x_2&0&0\\0&0&x_3&0\\0&0&0&x_4} : x_i \in L, x_1 x_2 x_3 x_4 =1\}$ which is not split in $G(k)$, rather we get
$$T(k) = \{ \pmatrix{a&0&0&0\\0&b&0&0\\0&0&\sigma(b)^{-1}&0\\0&0&0&\sigma(a)^{-1}} : a,b \in L, ab \in k\}.$$
Accordingly, the Galois action on $T(L)$ is again not given by $\sigma$ acting on entries, but rather via
$$\sigma_{T(L)} : \pmatrix{x_1&0&0&0\\0&x_2&0&0\\0&0&x_3&0\\0&0&0&x_4} \mapsto \pmatrix{\sigma(x_4)^{-1}&0&0&0\\0&\sigma(x_3)^{-1}&0&0\\0&0&\sigma(x_2)^{-1}&0\\0&0&0&\sigma(x_1)^{-1}}.$$
Again we define the $\sigma$-action on a character $\chi : T(L) \rightarrow L^\times$ of the torus via
$$\sigma \cdot \chi := \sigma \circ \chi \circ \sigma_{T(L)}^{-1}$$
and again we can see that rather than being trivial, e.g. the character $\chi_1 : \pmatrix{x_1&0&0&0\\0&x_2&0&0\\0&0&x_3&0\\0&0&0&x_4} \mapsto \dfrac{x_1}{x_2}$ (i.e. the root $\alpha_1$ written multiplicatively) gets sent to
$$ \sigma \cdot \chi_1 : \pmatrix{x_1&0&0&0\\0&x_2&0&0\\0&0&x_3&0\\0&0&0&x_4} \mapsto \dfrac{x_3}{x_4}$$
i.e. the root $\alpha_3$ written multiplicatively, etc.
Case 2: (But again assuming that the maximal $k$-split torus of $G$ is trivial, e.g in the case $k=\mathbb R$, but not for $p$-adic fields $k$.) Again we can choose $T(L) = \{ \pmatrix{x_1&0&0&0\\0&x_2&0&0\\0&0&x_3&0\\0&0&0&x_4} : x_i \in L, x_1 x_2 x_3 x_4 =1\}$ but this time in $G(k)$ we get
$$T(k) = \{ \pmatrix{a&0&0&0\\0&b&0&0\\0&0&c&0\\0&0&0&d} : a,b,c,d \in L, abcd =1, a^{-1} = \sigma(a), b^{-1}=\sigma(b), c^{-1}=\sigma(c)\}.$$
Accordingly, the Galois action on $T(L)$ is again not given by $\sigma$ acting on entries, but rather via
$$\sigma_{T(L)} : \pmatrix{x_1&0&0&0\\0&x_2&0&0\\0&0&x_3&0\\0&0&0&x_4} \mapsto \pmatrix{\sigma(x_1)^{-1}&0&0&0\\0&\sigma(x_2)^{-1}&0&0\\0&0&\sigma(x_3)^{-1}&0\\0&0&0&\sigma(x_4)^{-1}}.$$
Again we define the $\sigma$-action on a character $\chi : T(L) \rightarrow L^\times$ of the torus via
$$\sigma \cdot \chi := \sigma \circ \chi \circ \sigma_{T(L)}^{-1}$$
and again we can see that rather than being trivial, e.g. the character $\chi_1 : \pmatrix{x_1&0&0&0\\0&x_2&0&0\\0&0&x_3&0\\0&0&0&x_4} \mapsto \dfrac{x_1}{x_2}$ (i.e. the root $\alpha_1$ written multiplicatively) gets sent to
$$ \sigma \cdot \chi_1 : \pmatrix{x_1&0&0&0\\0&x_2&0&0\\0&0&x_3&0\\0&0&0&x_4} \mapsto \dfrac{x_2}{x_1}$$
i.e. the root $\color{red}{-}\alpha_1$ written multiplicatively, etc.
Part 2: The $*$-action
(References: 6.2 in Borel-Tits, 2.3 in Tits' Classification article in the Boulder Proceedings, ...)
Now this can be done in various ways, using maximal parabolic subgroups and whatnot, but here is a way of doing it just on the root level:
Say we are given a root system $R$ in an Euclidean space $V$, and an action of a finite Galois group $\Gamma$ via automorphisms of $R$.
Set $V_0 := \{v \in V: \sum_{\sigma \in \Gamma} \sigma \cdot v =0 \}$. (To make things work, here it is crucial that in the above one had chosen a maximal Torus $T$ which contains a maximal $k$-split torus $S$: Because then and only then will $V/V_0$ be well-behaved enough and realise the $k$-relative roots as quotient ("folding") of the original root system $R$.)
A $\Gamma$-linear order on $R$ (or $V$) is a choice of a positive half-space in $V$ such that for all $v \in V \setminus V_0$, we have $v$ positive $\implies \sigma \cdot v$ positive for all $\sigma \in \Gamma$. One can show that such $\Gamma$-linear orders always exist.
A $\Gamma$-basis for our root system $R$ is a set of simple roots among the ones positive for a $\Gamma$-linear order. I.e. a basis consisting of roots which are positive for a $\Gamma$-linear order.
Examples: In case 1, there are eight possible $\Gamma$-bases: $\{\alpha_1, \alpha_2, \alpha_3\}$ and its negative, $\{\alpha_1, -\alpha_1-\alpha_2-\alpha_3, \alpha_3\}$ and its negative, $\{\alpha_1+\alpha_2, -\alpha_2, \alpha_2+\alpha_3\}$ and its negative, and a fourth one I leave to you. Whereas e.g. $\{\alpha_2, \alpha_3, -\alpha_1-\alpha_2-\alpha_3\}$ is a basis, but not a $\Gamma$-basis of $R$.
In case 2, every basis of the root system is a $\Gamma$-basis (because $V=V_0$, the extra condition for positive systems to be $\Gamma$-linear is empty).
Now the $*$-action is defined as follows: For any basis $\Delta$ of the root system, $\sigma \cdot \Delta$ is again a basis. As is well-known, the Weyl group operates simply transitively on the set of bases of the root system; further, let's take a $\Gamma$-basis for $\Delta$. Then there is a Weyl group element $w_{\sigma}$ (actually, a unique one) such that $w_{\sigma}(\sigma \cdot \Delta) = \Delta$. Then we define the $*$-action as
$$\sigma * \alpha = w_{\sigma} (\sigma \cdot \alpha).$$
So to say, the $*$-action is the normal Galois action but "bent back" from a $\Gamma$-basis into itself via the Weyl group.
Now in our examples, in case 1, the normal action already stabilises each $\Gamma$-basis, and so the $*$-action is just the normal action. In case 2, the normal action actually sends each basis to its own negative, so as $w_\sigma$ we have to take the longest element of the Weyl group, which again gives the non-trivial automorphism of the Dynkin diagram.
In both cases, the $*$-action of the non-trivial element of the Galois group is given by $\alpha_1 \mapsto \alpha_3, \alpha_2 \mapsto \alpha_2$.
Now you will notice that a lot of this seems like overkill. Indeed, in both cases, it seems as if one does not need $\Gamma$-bases at all! Indeed, one can readily tell whether the "normal" Galois action acts through the Weyl group or not, and if not, then it necessarily gives the non-trivial diagram automorphism.
I am sure though that in more complicated examples (different root systems but more importantly, more intricate Galois groups!) this will make a difference. And then it is a crucial fact remarked in the above sources that the $*$-action, unlike the normal action, does not depend on many choices, in particular of the involved tori.
Best Answer
For any algebraic variety $Z$ defined over $K$, the absolute Galois group $G$ of $K$ acts canonically on $Z_{\overline{K}}$ through its action on $\overline{K}$. If you prefer, it acts on the points $Z(\overline{K})$ through its action on the coordinates.
For any $\sigma\in G$, let us denote by $\sigma_Z$ the automorphism of $Z_{\overline{K}}$ defined by the action of $\sigma$. Then $G$ acts canonically on $X^*(T)$ by
$$ \sigma * \chi = \sigma_{\mathbb G_m} \circ \chi \circ (\sigma_{T})^{-1}, \quad \sigma\in G, \chi\in X^*(T).$$ You see that $\sigma *\chi=\chi$ if and only if $\chi$ is already defined over $K$.
For example, if $T=\mathbb G_m^d$ is a split torus, then all characters of $T$ are defined over $K$ and $X^*(T)=\mathbb Z^d$ with the trivial Galois action. Conversely, if $X^*(T)$ has trivial Galois action, then all characters of $T$ are defined over $K$. If we choose a basis of $X^*(T)$, they define an isomorphism $T_{\overline{K}}\to (\mathbb G_m)^d_{\overline{K}}$ which is invariant by Galois, hence is defined over $K$. This implies that $T$ is split over $K$.