One of the reasons why the classical theory of binary quadratic forms
is hardly known anymore is that it is roughly equivalent to the theory
of ideals in quadratic orders. There is a well known correspondence
which sends the $SL_2({\mathbb Z})$-equivalence class of a form
$$ (A,B,C) = Ax^2 + Bxy + Cy^2 $$
with discriminant
$$ \Delta = B^2 – 4AC $$
to the equivalence class of the ideal
$$ \Big(A, \frac{B – \sqrt{\Delta}}2\Big). $$
One then checks that this gives a group isomorphism between the
primitive forms with discriminant $\Delta$ and the class group
of ideals coprime to the conductor of the order with discriminant $\Delta$.
If we go from forms with integral coefficients to forms with
coefficients in a polynomial ring $k[T]$ for some field $k$,
then everything transforms nicely as long as $k$ has
characteristic $\ne 2$. In characteristic $2$, there is a well
known theory of function fields (instead of $Y^2 = f(T)$,
consider equations $Y^2 + Y h(T) = f(T)$), but I have no idea
what the right objects on the forms side are. There should be
decent objects, since, after all, calculations in the Jacobian
of hyperelliptic curves are performed essentially by composition
and reduction of forms. But I don't thinkg that $Ax^2 + Bxy + Cy^2$
will work; for example, these guys all have square discriminant.
Question: which objects (forms or something else?) correspond to
ideals in quadratic function fields with characteristic $2$?
Added: Thank you for the (three at this point) excellent answers.
I am now almost convinced that binary forms over $F_2[T]$ actually
do work; the reason why I thought they would not was not so much the
square discriminant but the fact that the action of SL$_2$ fixed the
middle coefficient $B$ of $(A,B,C)$, so I didn't get much of a
reduction. But now I see that perhaps this is not so bad after all,
and that some pair $(B,?)$ will be the analog of the discriminant in
the characteristic 2 case.
I knew about Kneser's work on composition, but was convinced (by a
lengthy article of Towber in Adv. Math.) that classical Gauss composition
of forms over PID's, which I was interested in, is quite far away from
composition of quadratic spaces over general rings. Special thanks to
Torsten for showing how to go from quadratic spaces back to forms!
Best Answer
Starting, generally, with a commutative ring $R$ and a rank $2$ projective module $P$ given with a quadratic form $\varphi$ we can form its Clifford algebra $C(P)$. Its even part $S:=C^+(P)$ is then a commutative $R$ algebra of rank $2$. If (which we may assume locally) $P=Re_1+Re_2$ we have that $S$ has basis $1,e_1e_2=:h$ and $h^2=e_1e_2e_1e_2=e_1(\langle e_1,e_2\rangle-e_1e_2)e_2=\langle e_1,e_2\rangle h-e_1^2e_2^2=\langle e_1,e_2\rangle h-\varphi(e_1)\varphi(e_2)$, where $\langle-,-\rangle$ is the bilinear form associated to $\varphi$, giving an explicit quadratic algebra over $R$. Furthermore, $C^-(P)=P$ and it thus becomes an $C^+(P)$-module, explicitly $h\cdot e_1=e_1e_2e_1=\langle e_1,e_2\rangle e_1-e_1^2e_2=\langle e_1,e_2\rangle e_1-\varphi(e_1)e_2$. Furthermore, putting $L:=S/R$ we get an isomorphism $\gamma\colon \Lambda^2_RP \to L$ given by $u\land v \mapsto \overline{uv}$ (note that $u^2=\varphi(u)$ which maps to zero in $L$). Note for future use that, putting $[u,v]:=\gamma(u\land v)$, we also have $[[u,v]u,u]=\varphi(u)[u,v]$, where the left hand side is well-defined as $[([u,v]+r)u,u]=[[u,v]u,u]+r[u,u]=[[u,v]u,u]$. We have therefore constructed from $(P,\varphi)$ a triple $(S,P,\gamma)$, where $S$ is a quadratic (i.e., projective of rank $2$) $R$-algebra, $P$ an $S$-module projective of rank $2$ as $R$-module and an isomorphism $\gamma\colon \Lambda^2_RP \to S/R$ of $R$-modules. Conversely, given such a triple $(S,P,\gamma)$ there is a unique quadratic form $\varphi$ on $P$ such that $[[u,v]u,u]=\varphi(u)[u,v]$ (where again the left hand side is well-defined). It is easily verified that these constructions are inverse to each other.
In case $R=\mathbb Z$ I hope this gives the usual association between forms and modules over orders in quadratic number fields (but I admit shamelessly that I haven't checked it). When $R=k[t]$ I again hope this gives what we want.
In terms of algebraic group schemes and torsors we have the following situation. If the quadratic form is perfect, we have that $S$ is an \'etale covering and hence corresponds to an element of $H^1(\mathrm{Spec}R,\mathbb Z/2)$. It is the image under $O_2 \to \mathbb Z/2$ of the torsor in $H^1(\mathrm{Spec}R,O_2)$ corresponding to $\varphi$. As $\mathbb Z/2$ also acts as an automorphism group of $SO_2$ (by conjugation in $O_2$) we can use the torsor in $H^1(\mathrm{Spec}K,\mathbb Z/2)$ to twist $SO_2$ to get $SO(\varphi)$, the connected component of $O(\varphi)$. Torsors over this group corresponds to isomorphism classes of pairs consisting of a rank $2$ quadratic $R$-forms $\phi$ and an $R$-isomorphism $C^+(\phi)\simeq S$. We have an alternative description of $SO(\varphi)$: We can consider the units $T:=S^\ast$ of $S$ as an algebraic group and have a (surjective) norm map $T \to \mathbb G_m$ and then $SO(\varphi)$ is the kernel of this norm map. Using the exact sequence $1\to SO(\varphi)\to T\to\mathbb G_m\to1$ we see that an $SO(\varphi)$-torsor is given by a projective $S$-module $P$ of rank $1$ together with the choice of an isomorphism $\Lambda^2_RP\simeq R$.
[Added] Overcoming some of my laziness I did the calculation in the integral case ($R=\mathbb Z$ although the only thing we use is that $2$ is invertible): Start with the quadratic form $Cx^2+Bxy+Ay^2$ (the switch between $A$ and $C$ is to make my definition come out the same as the formula of the question). We then have $h^2=Bh-AC$ so that $h=\frac{B-\sqrt{\Delta}}{2}$. We have $h\cdot e_1=Be_1-Ce_2$ and $h\cdot e_2=Ae_1$. This is just an abstract module over $S$ but we can make it a fractional ideal by mapping $e_2$ to $A$, then $e_1$ maps to $h$ so that the fractional ideal is indeed $(A,\frac{B-\sqrt{\Delta}}{2})$.
[Added later] One could say that the association of the ideal $(A,h)$ to the quadratic form $Cx^2+Bxy+Ay^2$ over the ring $R[h]/(h^2-Bh+AC)$ is an answer to the question which workds in all classical cases ($R=\mathbb Z$ and
$R=k[T]$). The reason that this looks simpler than the traditional ($R=\mathbb Z$) answer is that we let the presentation of the quadratic order depend on the quadratic form itself. Usually we have fixed the quadratic order and want to
consider all forms with this fixed quadratic order as associated order. This
means that we should fix some normal form for the order and then express the
ideal in terms of this normal. When $R=\mathbb Z$ orders are in bijection with their discriminants $\Delta$ which are actual integers (as the discriminant is well-determined modulo squares of units). A normal form for the order is
$\mathbb Z+\mathbb Z\sqrt{\Delta}$ or $\mathbb Z+\mathbb Z(\sqrt{\Delta}+1)/2$ depending on whether $\Delta$ is even or odd. A curious feature of the form
$(A,\frac{B-\sqrt{\Delta}}{2})$ of the ideal is that the distinction between the odd
and even case is not apparent. However, the crucial thing is that it expresses a
$\mathbb Z$-basis for the ideal in terms of the canonical form of the quadratic
order.
The case of $R=k[T]$ for $k$ a field of odd characteristic is somewhat deceptive as the discriminant $\Delta=B^2-4AC$ is not quite an invariant of the quadratic order as there are units different from $1$ that are squares (except when $k=\mathbb Z/3$!). Hence the formula $(A,\frac{B-\sqrt{\Delta}}{2})$ is not quite of the same
nature as for the $\mathbb Z$ case as there is a very slight dependence of $\Delta$
on the form (and not just on the order). We could fix that by choosing coset
representatives for the squares as a subgroup of $k^\ast$ and then the formula for
the ideal would would take the form $(A,\frac{B-\lambda\sqrt{\Delta}}{2})$ where $\Delta$ now has been normalised so as to have its top degree coefficient to be a coset
representative.
The case when $k$ has characteristic $2$ is more complicated. We get and order of the form $k[T][h]/(h^2+gh+f)$ but the question of a normal form is trickier. The discriminant of the order (in the sense of the discriminant of the trace form) is equal to $g^2$ and as all elements of $k$ are squares we can normalise things so that $g$ is monic. However, the order is not determined by its discriminant. This can be seen already in the unramified case when $g=1$ when a normal form for $f$ is that it contain no monomials of even degree and the constant term is one of a chosen set of coset representives for the subgroup $\{\lambda^2+\lambda\}$ of $k$. We can decide to rather arbitrarily fix a generator $H$ for every order with $H^2=GH+F$ where one sensible first normalisation is that $G$ be monic (for which we have to assume that $k$ is perfect). Then we have that $h=H+a$, with $a\in k[T]$, and the ideal would be $(A,H+a)$.
There is a particular (arguably canonical) choice of $H$: We assume $G$ is monic and then can write uniquely $G=G_1G_2\cdots G_n$ with $G_i$ monic and $G_{i+1}|G_i$. The normal form is then that $F$ have the form $F=G_1F_1+G_1^2G_2F_2+\cdots+G_1^2G_2^2\cdots G_nF_n+G^2F'$ where $\deg F_i<\deg G_i$ and $F'$ contain no square monomials and its constant term belongs to a chosen set of coset representatives for $\{\lambda^2+\lambda\}$ in $k$.
One further comment relating to the classical formulas. When passing from a fractional ideal to a quadratic form one classically divides by the norm of the ideal (as is done in KConrad's reply). This means that the constructed form is primitive, i.e., the ideal generated by its values is the unit ideal. Hence if one starts with a quadratic form, passes to the ideal and then back to a quadratic form one does not end up with the same form if the form is not primitive. Rather the end form is the "primitivisation" where one has divided the form by a generator for its ideal of values. This of course only makes sense if the base ring is a PID. Even for a general Dedekind ring if one wants to work with primitive forms one has to accept quadratic forms that take values in general rank $1$ modules (i.e., fractional ideals).
The approach above makes another choice. It deals only with $R$-valued forms but accepts non-primitive ones. This would seem to lead to a contradiction as the classical construction leads to a bijection between modules and primitive forms and the above leads to a bijection between modules and arbitrary forms. There is no contradiction however (phew!) as the above construction leads to smaller orders than the classical one in the non-primitive case.
Classically what one really works with (when $R$ is a PID) are lattices in $L$ (the fraction field of $S$), where a lattice $M$ is a finitely generated $R$-submodule of $K$ containing a $K$-basis ($K$ the fraction field of $R$) for $L$. The order one associates to $M$ is the subring of $L$ stabilising $M$. When the quadratic form $\varphi$ is non-primitive $C^+(P)$ is not equal to this canonically associated order but is a proper suborder by it. Dividing by a generator for the ideal of values gives us a primitive form for which $C^+(P)$ is equal to the canonical order.
Finally there is a particular miracle that occurs for lattices in quadratic extensions (of the fraction field of a Dedekind ring $R$), $M$ as a module over its stabilisation order is projective. This is why classes of primitive forms with fixed order are in bijection with the class group of the order.