Starting with some background information from Wikipedia, we have that under time reversal the position is unchanged while the momentum changes sign.
In quantum mechanics we can express the action of time reversal on these operators as $\Theta\,\mathbf{x}\,\Theta^\dagger = \mathbf{x}$ and $\Theta\,\mathbf{p}\,\Theta^\dagger = -\mathbf{p}$. It is worth mentioning here that the time reversal operator, $\Theta$, is anti-unitary, which allows it to be expressed as $\Theta = UK$ where $U$ is unitary and $K$ is the complex conjugation operator.
As for the creation/annihilation operators used in second quantization the sign changes under $\Theta$ would suggest a transformation of $a_r \rightarrow a_r$ and $a_k \rightarrow a_{-k}$. If you are worried about the fact that $k$ represents a crystal momentum and not a true momentum you can just take the position transformation, which is perhaps more trustworthy, and use $a_k = \sum_r \, a_r \,\mathrm{exp}[ -\mathrm{i} k\cdot r]$ to verify $a_k \rightarrow a_{-k}$ directly.
Using these transformations you should be able to verify that the tight-binding Hamiltonian is invariant under time reversal in position and momentum space for a lattice with or without a basis. Keep in mind that you would generally take the complex conjugate of the coefficients in $H$, however in your case $t$ and $\epsilon_k$ are both real. Its important to remember though, mostly to make sure $H$ stays hermitian.
As far as your comment about $\sigma_y$, this is only necessary if you include spin. Spin changes sign under time reversal so $\Theta\,\mathbf{S}\,\Theta^\dagger = -\mathbf{S}$. In this case, we can formally write $\Theta = \mathrm{exp}[-i \pi J_y]\,K$, which is probably the relation you are alluding to.
According to J.J. Sakurai's Modern Quantum Mechanics one possible convention for the time-reversed angular momentum states is $\Theta | j,m\rangle = (-1)^m |j,-m\rangle$. This suggests that with spin indices the creation/annihilation operators transform like $a_{r,m} \rightarrow (-1)^m\,a_{r,-m}$ and $a_{k,m} \rightarrow (-1)^m \, a_{-k,-m}$ under time reversal. From what I understand, most spin Hamiltonians will be invariant under this transformation. An example when this is not the case would be in the presence of an external magnetic field which couples to the spins through a $\mathbf{S}\cdot \mathrm{B}-$like term.
It is interesting how even in the absence of an external field the groundstate of spin Hamiltonians can still sponanteously break the time reversal symmetry present in $H$, but rather than discuss this myself I will direct you to this very well written answer.
A time reversal operator is an anti-unitary operator, which can be expressed as:
$\mathcal{T}=UK$
where $K$ denotes complex conjugate and $U$ is a unitary operator. In case of spinless particles, $U$ is chosen to be Identity. Thus $\mathcal{T}=K$.
If the system has $\mathcal{T}$-reversal symmetry:
$$
KH\psi=HK\psi
$$
which leads to:
$$
H^*\psi^*=H\psi^*
$$
meaning that the Hamiltonian must be real symmetric.
Best Answer
You are right!
If you consider whatever is producing the $\vec{B}$ field as part of the system then the whole thing is symmetric under time reversal. It is only when the $\vec{B}$ field is considered as something external (on which the time-reversal operator does not act) that the symmetry is broken.
I don't think a 'universal' time-reversal operation is any more (or less) physical than one which acts on only a certain subsystem (can you make time run backwards?).
In practice, what time-reversal symmetry means is something like this. (I will give a classical description for ease of intuition, a quantum version can be formulated along similar lines.) Suppose I have an interacting system of $N$ particles which I prepare in state $(\vec{q}_1(0),\vec{p}_1(0))$, and after time $t$ is found to be in state $(\vec{q}_1(t),\vec{p}_1(t))$. I now prepare a second system with identical particles in state $(\vec{q}_2(0),\vec{p}_2(0))=(\vec{q}_1(t),-\vec{p}_1(t))$ (i.e. with the same positions and opposite velocities of the final state of system 1). If the interaction is time-reversal symmetric then after running the second experiment for time $t$ it is found that $(\vec{q}_2(t),\vec{p}_2(t))=(\vec{q}_1(0),-\vec{p}_1(0))$, so that the dynamics of the second experiment are exactly the reverse of those of the first experiment.
Now, if I know that the interaction is electromagnetic, and there are no external forces, then I know the results must obey time-reversal symmetry (as the laws of electrodynamics have this symmetry). However, if I choose to impose an external magnetic field on the first experiment then when I perform the second experiment I have the choice of whether to reverse this field or not.
Statements 1 and 2 seem equally meaningful (and equally 'physical') to me.