I did the maths, and it turns out there actually is a right answer, if both players are playing to maximise their chance of winning (and both players know that). $A$ should reroll on a 7 or lower, and $B$ should reroll on a 9 or lower.
We can calculate $P(X \ge Y)$ by considering separately the four cases where the 12-sided and 20-sided dice are rolled once or twice each. Assuming $m \ge k$, we have
\begin{align}
P(X\ge Y|k,m)&=
\frac{km\sum_{i=1}^{12} i}{12^2\times 20^2}
+\frac{m\sum_{i=k+1}^{12} i}{12\times 20^2}
+ \frac{k\sum_{i=m+1}^{12} (i-m)}{12^2 \times 20}
+ \frac{\sum_{i=m+1}^{12} (i-m)}{12 \times 20} \\
&=\frac{78km}{12^2\times 20^2}
+ \frac{78m-\frac{k(k+1)}{2}m}{12\times 20^2}
+ \frac{k\frac{(12-m)(13-m)}{2}}{12^2\times 20}
+ \frac{\frac{(12-m)(13-m)}{2}}{12\times 20}.
\end{align}
Taking $m<k$ is clearly non-optimal for $B$, but you can handle it by replacing the $\sum_{i=m+1}^{12}(i-m)$ in the last term with $\sum_{i=k+1}^{12}(i-m)$, i.e. subtracting $\frac{\frac{(k-m)(k-m+1)}{2}}{12\times 20}$ from the final result.
Here is a table of the probability (multiplied by $12^2\times 20^2$ for clarity) that $A$ will win, for each $k$ (columns) and $m$ (rows).
I've highlighted the minimum probability in each column in yellow, and the maximum in each row in green. If $A$ chooses $k$ as 1-7 or as 11, then $B$ should choose $m$ as 9, to maximise their probability of winning. But if $A$ chooses $k$ as 8-10, then $B$ should choose $m=10$. Similarly, $A$ should choose $k=7$ if they expect $B$ to choose $m$ between 5 and 7 or between 9 and 10 (actually, if they know $B$ will choose $m=10$, it's an equally good idea for $A$ to choose $k=6$ or $k=7$, but if there's even a tiny chance $B$ might choose $m=9$ then $A$ should probably choose $k=7$). Thus, if both players are playing rationally, they will settle on $A$ playing $k=7$ and $B$ playing $m=9$. In that case, $A$ wins with probability $\frac{12594}{57600}=\frac{2099}{9600}$, and $B$ wins with probability $\frac{7501}{9600}$.
If one of the players chooses something else, it will decrease their odds of winning even if the other player keeps the same strategy. But the other player might be able to improve their odds even further by adjusting to their opponent's mistake. For example, if $A$ chooses $k=8$ while $B$ chooses $m=9$, then $A$'s probability of winning goes down to $\frac{12552}{57600}$, but $B$ can capitalise on $A$'s aggression by choosing $m=10$, and decrease $A$'s odds of winning further to $\frac{12480}{57600}$.
Best Answer
You win when your roll is strictly greater than either of the other player's two rolls. If you roll $x$, then the probability of this happening is $(x-1)^2/36$; and the probability of you rolling any particular $x\in\{1,2,3,4,5,6\}$ is just $1/6$. So the probability that you win is $$ \frac{0^2+1^2+2^2+3^2+4^2+5^2}{216}=\frac{55}{216} \approx 25.5\%. $$ So you're right, it's not fair... if you change the rules so you win in the case of a tie, then your win probability becomes $42.1\%$, which is an improvement.