Because $X\mapsto X^2$ has two semi-inverses, the Jacobian transformation is:$$\begin{align}f_{X^2}(y)&=\left|\frac{d(\surd y)}{dy}\right|f_{X}(\surd y)+\left|\frac{d(-\surd y)}{dy}\right|f_{X}(-\surd y)\\&=\frac{\mathbf 1_{a\leq\surd y\leq b}+\mathbf 1_{-b\leq\surd y\leq-a}}{2(b-a)\surd y}\cdot\mathbf 1_{0<y}\end{align}$$So for example, should we take $X\sim\mathcal U(-1,2)$ then $$\begin{align}f_{\small X^2}(y) & =(6\surd y)^{-1}(\mathbf 1_{0<\surd y\leq 2}+\mathbf 1_{0<\surd y\leq 1})\\&=\dfrac{\mathbf 1_{0<y\leq 1}}{3\surd y}+\frac{\mathbf 1_{1<y\leq 4}}{6\surd y}\end{align}$$
Welcome to the concept of folding.
When $a<0<b$ the support for the pdf of $X$ is $[a,0]\cup(0,b]$.
Now under the transformation $X\mapsto X^2$, these two disjoint parts of the support will map to overlapping domains: as $[a,0]\mapsto[0,a^2]$ and $(0,b]\mapsto(0,b^2]$ . So they fold together: $$[a,0]\cup(0,b]\mapsto[0,\min(a^2,b^2)]\cup(\min(a^2,b^2),\max(a^2,b^2)]$$
So where $a<0<b$ we obtain:
$$\begin{align}f_{X^2}(y)&=\frac{\mathbf 1_{0\leq y\leq\min(a^2,b^2)}}{(b-a)\surd y}+\frac{\mathbf 1_{\min(a^2,b^2)\leq y\leq\max(a^2,b^2)}}{2(b-a)\surd y}\end{align}$$
In the two cases where $0$ is not included inside the support $[a,b]$, there will be no such folding, and the support maps bijectively, $[a,b]\mapsto [\min(a^2,b^2),\max(a^2,b^2)]$, and we simply obtain:
$$\begin{align}f_{X^2}(y)&=\frac{\mathbf 1_{\min(a^2,b^2)\leq y\leq\max(a^2,b^2)}}{2(b-a)\surd y}\end{align}$$
your first instinct is quite correct.
$$f_X(x)=(1-a)\cdot\mathbb{1}_{\left(0;0.5 \right)\cup(0.5;1)}(x)+a\delta(x-0.5)$$
now your impulse contributes exactly for a proportion $a$
Remember that this is NOT a pdf because your F is not absolutely continuous
Best Answer
Are you aware of the filtering/sifting property of Dirac Delta function? The filtering property says that $\forall\epsilon,\gamma>0$ the value of$$\int_{a-\epsilon}^{a+\gamma} h(x)\delta(a-x)dx=\lim_{x\to a}h(a)$$ Take $g(x)=1/x$ as $m$, then you have$$f_Y(y)=\int_1^\infty\frac1{m^2}f_X(1/m)\delta(y-m)~dm$$
For $y<1,$ $\delta(y-m)=0$ so $f_Y(y)=0,y<1$. For $y>1$, we have $1/y\in(0,1)$ and so$$f_Y(y)=\lim_{m\to y}\frac{f_X(1/m)}{m^2}=\lim_{1/m=k\to1/y}k^2f_X(k)=\frac{f_X(1/y)}{y^2}$$since $k^2f_X(k)$ is continuous in $(0,1)$. Finally you get$$f_Y(y)=\begin{cases}0,&y\le1\\\frac1{y^2}\left(\frac12+\frac1y\right),&y>1\end{cases}$$