To avoid confusion with limits, note that
$$\int_0^{\pi} \dfrac{dx}{a+b \cos(x)} = \int_{\pi}^{2\pi} \dfrac{dx}{a+b \cos(x)}$$
Hence, we have
$$I = \int_0^{2\pi} \dfrac{dx}{a+b \cos(x)} = 2\int_0^{\pi} \dfrac{dx}{a+b \cos(x)}$$
Now use your substitution $t = \tan(x/2)$ and note that $t$ goes from $0$ to $\infty$ as $x$ goes from $0$ to $\pi$.
EDIT
An easier way if you are familiar with a little bit of complex analysis is as follows.
Let $z = e^{ix}$. Then we have that $dz = iz dx$. Hence,
\begin{align}
I & = \int_0^{2 \pi} \dfrac{dx}{a+b \cos(x)} = \oint_{\vert z \vert = 1} \dfrac{dz}{iz \left( a + \dfrac{b}2 \left(z+ \dfrac1z \right)\right)}\\
& = \dfrac1i \oint_{\vert z \vert = 1} \dfrac{dz}{az + \dfrac{b}2 \left(z^2+1 \right)} = \dfrac2{ib} \oint_{\vert z \vert = 1}\dfrac{dz}{z^2 + \dfrac{2a}b z + 1}\\
& = \dfrac2{ib} \oint_{\vert z \vert = 1} \dfrac{dz}{\left(z + \dfrac{a}b \right)^2 + 1 - \dfrac{a^2}{b^2}} = \dfrac2{ib} \oint_{\vert z \vert = 1} \dfrac{dz}{\left(z + r + \sqrt{r^2-1} \right)\left(z + r - \sqrt{r^2-1} \right)}
\end{align}
where $r = \dfrac{a}b \in \mathbb{R}$.
First note that if $\vert r \vert \leq 1$, then $\left \vert r \pm \sqrt{r^2-1} \right \vert = 1$. Hence, the integral doesn't exist.
If $r > 1$, then $ \left \vert r + \sqrt{r^2 - 1} \right \vert > 1$ and $\left \vert r - \sqrt{r^2-1} \right \vert < 1$. Hence, by Cauchy integral formula, we have that
$$I = \dfrac{2}{ib} 2 \pi i \dfrac1{2 \sqrt{r^2-1}} = \dfrac{2 \pi \, \text{sign}(a)}{\sqrt{a^2 - b^2}}$$
If $r < -1$, then $\left \vert r - \sqrt{r^2 - 1} \right \vert > 1$ and $\left \vert r + \sqrt{r^2-1} \right \vert < 1$. Hence, by Cauchy integral formula, we have that
$$I = \dfrac{2}{ib} 2 \pi i \dfrac{-1}{2 \sqrt{r^2-1}} = \dfrac{2 \pi \, \text{sign}(a)}{\sqrt{a^2 - b^2}}$$
Hence, to summarize,
$$I = \begin{cases} \dfrac{2 \pi \, \text{sign}(a)}{\sqrt{a^2 - b^2}} & \text{if } \vert a \vert > \vert b\\ \text{Does not exists} & \text{if } \vert a \vert \leq \vert b\end{cases}$$
Here is a solution using real analysis. First, denote the two integrals as
\begin{align*}
J & \equiv\int\limits_0^{\pi}\frac {x\left(\sin\frac x2\color{red}-\cos\frac x2\right)}{\sqrt{\sin x}}\,\mathrm dx\\K & \equiv\int\limits_0^{\pi}\frac {x\left(\sin\frac x2\color{red}+\cos\frac x2\right)}{\sqrt{\sin x}}\,\mathrm dx
\end{align*}
And from your post, recall that $K=\pi^2$. Adding the two integrals together removes the $\cos\frac x2$ factor inside the integrand, leaving only
\begin{align*}
J+K & =2\int\limits_0^{\pi}\frac {x\sin\frac x2}{\sqrt{\sin x}}\,\mathrm dx\\ & =\sqrt 2\int\limits_0^{\pi}x\sqrt{\tan\frac x2}\,\mathrm dx\\ & =4\sqrt{2}\int\limits_0^{+\infty}\frac {\sqrt t}{1+t^2}\arctan t\,\mathrm dt
\end{align*}
Where a double angle identity was utilized in the second equation and the half-angle tangent substitution in the third equation. The last integral has been evaluated here before using Complex Analysis, Feynman's Trick, etc. Here is an alternative approach using double integrals. First, enforce the substitution $x=\sqrt t$ so that the integral becomes
$$\int\limits_0^{+\infty}\frac {\sqrt t}{1+t^2}\arctan t\,\mathrm dt=2\int\limits_0^{+\infty}\frac {x^2}{1+x^4}\arctan x^2\,\mathrm dx$$
Next, use the identity
$$\arctan x^2=\int\limits_0^1\frac {x^2}{1+x^4 y^2}\,\mathrm dy$$
Swapping the order of integration and using partial fraction decomposition, then we get
\begin{align*}
2\int\limits_0^{+\infty}\,\int\limits_0^1\frac {x^4}{(1+x^4)(1+x^4y^2)}\,\mathrm dy\,\mathrm dx & =2\int\limits_0^1\frac 1{y^2-1}\int\limits_0^{+\infty}\frac 1{1+x^4}-\frac 1{1+x^4y^2}\,\mathrm dx\,\mathrm dy\\ & =\frac {\pi}{\sqrt 2}\int\limits_0^1\frac 1{y^2-1}\left(1-\frac 1{\sqrt y}\right)\,\mathrm dy\\ & =\frac {\pi^2}{4\sqrt 2}+\frac {\pi\log 4}{4\sqrt 2}
\end{align*}
To recap, we have the equation
$$J+\pi^2=4\sqrt{2}\int\limits_0^{+\infty}\frac {\sqrt t}{1+t^2}\arctan t\,\mathrm dt=\pi^2+\pi\log 4$$
Subtracting a $\pi^2$ from both sides, then
$$\int\limits_0^{\pi}\frac {x\left(\sin\frac x2-\cos\frac x2\right)}{\sqrt{\sin x}}\,\mathrm dx\color{blue}{=\pi\log 4}$$
Best Answer
You can find the solution below, and afterwards an explanation as of why your $u$ substitution did not work.
An alternative way is to write
$${\int_{0}^{2\pi}\frac{1}{1+\cos^2(x)} dx=4\int_{0}^{\frac{\pi}{2}}\frac{1}{1+\cos^2(x)}dx}$$
$${=4\int_{0}^{\frac{\pi}{2}}\frac{\sin^2(x) + \cos^2(x)}{1+\cos^2(x)}dx}$$
Now, dividing the top and bottom by ${\cos^2(x)}$ gives
$${4\int_{0}^{\frac{\pi}{2}}\frac{\tan^2(x) + 1}{\sec^2(x) + 1} dx}$$
Now using some more trig identities, we get
$${=4\int_{0}^{\frac{\pi}{2}}\frac{\sec^2(x)}{\tan^2(x) + 2}dx}$$
$${=4\int_{0}^{\infty}\frac{1}{2+u^2}du}$$
Now, solving that integral
$${\int_{0}^{\infty}\frac{1}{2+u^2}du=\frac{1}{2}\int_{0}^{\infty}\frac{1}{1+\left(\frac{u}{\sqrt{2}}\right)^2}du}$$
Now do ${k=\frac{u}{\sqrt{2}}}$:
$${=\frac{1}{\sqrt{2}}\int_{0}^{\infty}\frac{1}{1+k^2}dk=\frac{\pi}{2\sqrt{2}}}$$
So putting the whole thing together:
$${=4\times \frac{\pi}{2\sqrt{2}}=\sqrt{2}\pi}$$
Which is the correct answer :)
EDIT: After some reconsideration, I don't believe the issue in your original ${u}$ substitution was really to do with injectivity at all. In fact injectivity is not a strict requirement for $u$ substitution. If we redo all the steps we just did, but don't change the domain of integration, you end up with
$${\int_{0}^{2\pi}\frac{1}{1+\cos^2(x)}dx=\int_{0}^{2\pi}\frac{\sec^2(x)}{\tan^2(x) + 2}dx}$$
You may be tempted to once again do ${u=\tan(x)}$ (which is essentially what you did I believe?) but one requirement for $u$ substitution that is very clearly needed is for $u$ to be continuous on the domain of integration (it needs to be differentiable, and so clearly continuity is a requirement!). ${u=\tan(x)}$ certainly is not continuous on ${(0,2\pi)}$, and so you really end up with an improper integral of sorts. Now, we could split up the integral instead, just as we would with any old improper integral:
$${\int_{0}^{2\pi}\frac{\sec^2(x)}{\tan^2(x) + 2}dx=\int_{0}^{\frac{\pi}{2}}\frac{\sec^2(x)}{\tan^2(x) + 2}dx + \int_{\frac{\pi}{2}}^{\frac{3\pi}{2}}\frac{\sec^2(x)}{\tan^2(x) + 2}dx + \int_{\frac{3\pi}{2}}^{2\pi}\frac{\sec^2(x)}{\tan^2(x) + 2}dx}$$
And now it is legitimate to do ${u=\tan(x)}$, since ${\tan(x)}$ will be differentiable and continuous on these domains (well, technically not at the endpoints - but you take a limit, as per the definition of how we deal with improper integrals). Indeed you will get the answer of ${\sqrt{2}\pi}$ if you evaluate this expression.
So if this was the issue, why did people jump to injectivity / bijectivity? Well there are some instances where this is (indirectly) the problem. An example:
$${\int_{0}^{2\pi}xdx}$$
Clearly the answer to this is ${2\pi^2}$. Now do the substitution ${u=\sin(x)}$ - our endpoints become ${\int_{0}^{0}}$... does this mean the integral is $0$? NO! Recall $u$ substitution only says that
$${\int_{a}^{b}f(\phi(x))\phi'(x)dx = \int_{\phi(a)}^{\phi(b)}f(u)du}$$
If you actually attempt to write ${\int_{0}^{2\pi}xdx}$ to match the form of the lefthand side so we can legit utilise $u$ substitution you will end up using some nasty ${\arcsin}$ rubbish - but the key point to take away is that the ${\arcsin}$ function only gives back principle values. ${\arcsin(\sin(x))}$ does not necessarily equal to ${x}$ for all ${x \in \mathbb{R}}$!. So in actuality you end up having a piecewise function to represent ${x}$, so you are forced to split the integral up. So in this case injectivity is in fact a sort of "requirement" indirectly (unless you split up the integral).
I hope this helped explain a bit better :)