Let $G$ be a semisimple algebraic group over an algebraically closed field $k$. In the case that $k$ has characteristic 0, there has been intensive study of the BGG category O of representations of the enveloping algebra of the Lie algebra of $G$ (which is also, in this case, the hyperalgebra of $G$). When $k$ is a field of positive characteristic, though, the enveloping algebra and the hyperalgebra are different, and there has been a lot of study of representations of the enveloping algebra. On the other hand, it seems that there has been much less study of representations of the hyperalgebra, perhaps because it is more complicated (for example, it is not finitely generated).
So let's now assume that the characteristic of $k$ is positive. A seminal paper of Haboush, "Central Differential Operators of Split Semisimple Groups Over Fields of Positive Characteristic," set up the foundations of category O for hyperalgebras in this setting. More precisely, let $U$ denote the hyperalgebra of $G$. In that paper Haboush defined Verma modules for $U$ (which are defined analogously to the characteristic 0 case) and proved that many of the properties one would expect, including relationships with simple modules and certain "integrality" properties, hold. However, this is just the beginning; there are many questions one could ask about whether or not category O in positive characteristic behaves like category O in characteristic 0. E.g, what is the structure of the blocks in this category? What about the structure of projective generators? Are there translation functors? If so, how do they behave? I.e., how much of the (huge) characteristic 0 story carries over to positive characteristic? Haboush's paper is from 1980, and I haven't been able to find any papers that carry on the study started in the paper — does anyone know of any?
(As a sidebar, I would note that the hyperalgebra/enveloping algebra dichotomy in positive characteristic is mirrored in the study of quantum groups at roots of 1: the De Concini-Kac algebra is the analog of the enveloping algebra, and the Lusztig algebra is the analog of the hyperalgebra).
Best Answer
Maybe I can answer the original question more directly, leaving aside the interesting recent geometric work discussed further in later posts like the Feb 10 one by Chuck: analogues of Beilinson-Bernstein localization on flag varieties and consequences for algebraic groups (Bezrukavnikov, Mirkovic, Rumynin).
The 1979 conference paper by Haboush may be hard to access and also hard to read in detail, but it raises some interesting questions especially about centers of certain hyperalgebras. I tried to give an overview in Math Reviews: MR582073 (82a:20049) 20G05 (14L40 17B40) Haboush,W. J., Central differential operators on split semisimple groups over fields of positive characteristic. Séminaire d’Algèbre Paul Dubreil et Marie-Paule Malliavin, 32ème année (Paris, 1979), pp. 35–85, Lecture Notes in Math., 795, Springer, Berlin, 1980.
The hyperalgebra here is the Hopf algebra dual of the algebra of regular functions on a simply connected semisimple algebraic group $G$ over an algebraically closed field of characteristic $p$, later treated in considerable depth by Jantzen in his 1987 Academic Press book Representations of Algebraic Groups (revised edition, AMS, 2003). After the paper by Haboush, for example, Donkin finished the determination of all blocks of the hyperalgebra.
While the irreducible (rational) representations are all finite dimensional and have dominant integral highest weights (Chevalley), the module category involves locally finite modules such as the infinite dimensional injective hulls (but no projective covers). The role of the finite Weyl group is now played by an affine Weyl group relative to $p$ (of Langlands dual type) with translations by $p$ times the root lattice. In fact, higher powers of $p$ make life even more complicated.
The older work of Curtis-Steinberg reduces the study of irreducibles to the finitely many "restricted" ones for the Lie algebra $\mathfrak{g}$. For these and other small enough weights, Lusztig's 1979-80 conjectures provide the best hope for an analogue of Kazhdan-Lusztig conjectures when $p>h$ (the Coxeter number). The recent work applies for $p$ big enough": Andersen-Jantzen-Soergel, BMR, Fiebig.
Anyway, the hyperalgebra involves rational representations of $G$ including restricted representations of $\mathfrak{g}$, while the usual enveloping algebra of the Lie algebra involves all its representations. But the irreducible ones are finite dimensional. I surveyed what was known then in a 1998 AMS Bulletin paper. Lusztig's 1997-1999 conjectures promised more insight into the non-restricted irreducibles and are now proved for large enough $p$ in a preprint by Bezrukavnikov-Mirkovic. This and their earlier work with Rumynin use a version of "differential operators" on a flag variety starting with the usual rather than divided-power (hyperalgebra) version of the universal enveloping algebra of $\mathfrak{g}$.
To make a very long story shorter, Haboush was mainly looking for the center of the hyperalgebra (still an elusive beast unlike the classical enveloping algebra center, due to the influence of all powers of $p$). His weaker version of Verma modules may or may not lead further. But there is no likely analogue of the BGG category for the hyperalgebra in any case. That category depended too strongly on finiteness conditions and well-behaved central characters.
ADDED: It is a long story, but my current viewpoint is that the characteristic $p$ theory for both $G$ and $\mathfrak{g}$ (intersecting in the crucial zone of restricted representations of $\mathfrak{g}$) is essentially finite dimensional and requires deep geometry to resolve. True, the injective hulls of the simple $G$-modules with a highest weight are naturally defined and infinite dimensional (though locally finite), but the hope is that they will all be direct limits of finite dimensional injective hulls for (the hyperalgebras of) Frobenius kernels relative to powers of $p$. Shown so far for $p \geq 2h-2$ (Ballard, Jantzen, Donkin). In particular, the universal highest weight property of Verma modules in the BGG category (and others) is mostly replaced in characteristic $p$ by Weyl modules (a simple consequence of Kempf vanishing observed by me and codified by Jantzen). Then the problems begin, as Lusztig's conjectures have shown. The Lie algebra case gets into other interesting territory for non-restricted modules.