You can't represent such a transform by a $2 \times 2$ matrix, since such a matrix represents a linear mapping of the two-dimensional plane (or an affine mapping of the one-dimensional line), and will thus always map $(0,0)$ to $(0,0)$.
So you'll need to use a $3 \times 3$ matrix, since you need to represent affine mappings. If you represent the point $[x,y]$ as the vector $(x,y,1)^T$, then the matrix $$
T_{u,v} = \begin{pmatrix}
1 & 0 & u \\
0 & 1 & v \\
0 & 0 & 1
\end{pmatrix}
$$
represents the translation in the direction $[u,v]$, i.e. $$
T_{u,v} [x,y] = T_{u,v}(x,y,1)^T = (x+u,y+v,1) = [x+u,y+v] \text{.}
$$
To find the representation of a linear mapping as a $3 \times 3$ matrix, simply take the $2\times 2$ matrix that represents the mapping in euclidean two-dimensional space, and embedd it into a $3 \times 3$ matrix like this $$
\begin{pmatrix}
M & 0 \\
0 & 1
\end{pmatrix} \text{.}
$$
For example, you'd represent a rotation around the origin (which is a linear mapping) as $$
R_\varphi = \begin{pmatrix}
\cos\varphi & -\sin\varphi & 0 \\
\sin\varphi & \cos\varphi & 0 \\
0 & 0 & 1
\end{pmatrix} \text{.}
$$
You can use normal matrix multiplication to combine the matrix representation of affice mappings - for example, to rotate around the point $[u,v]$, compute the product matrix $$
T_{u,v} R_\varphi T_{-u,-v} \text{,}
$$
which simply says "shift the center of the rotation to the origin, rotate around the origin and shift back".
First, subtract the coordinates of A from those of all three vertices,
giving the translation (assuming translation is done before rotation --
they don't commute).
This gives vertices (0,0,0), (bx,by,bz) and (cx,cy,cz) as vertices.
Define the vector
u = (bx,by,bz)
pointing from A to B, and normalise it to the unit vector
U = ( 1 /|u|) u = ( 1 / √( bx^2 + by^2 + bz^2 ) ) (bx,by,bz)
Take the cross product
w = u × (cx,cy,cz),
which is at right angles to the triangle.
Normalise it to give the unit vector
W = ( 1 /|w|) w .
Find the cross-product
V = U ×W ,
automatically a unit vector.
In coordinates corresponding to the basis {U, V, W }, the triangle lies as you want. The rotation matrix that carries the usual (1,0,0) to U, the usual (0,1,0) to V, and the usual (,0,0,1) to W has U, V and W as its columns. You want the other direction, so take the inverse — which for a rotation matrix is just the transpose.
This is not "the transformation for the above case", because you did not specify whether C should lie on the y>0 half plane.
Best Answer
A simple way to do this would be to canonicalize both triangles and then concatenate the transform that canonicalizes the "origins" with the inverse of the one that canonicalizes the "destinies".
For instance, a) apply a translation to move the point in the first pair to the origin; b) apply a rotation about an axis through the origin to move the point in the second pair onto the $z$-axis; c) apply a rotation about the $z$-axis to move the point in the third pair into the $y$-$z$-plane with non-negative $y$ coordinate. That moves all congruent triangles with identically numbered points into the same position.