As explained several times on the site, writing down correctly the joint density (1) and then, applying universal identities (2), saves any kind of head-scratching and ensures that one finds the correct answer (pretty boring, eh...).
(1) In the present case, the first step is to be aware that every two-dimensional density is always defined on the whole plane $\mathbb R^2$. In practice, this means that one should avoid every kind of auxiliary condition written on the side that one does not know how to deal with afterwards, and instead, write down the joint density as a function $$f_{X,Y}:\mathbb R^2\to\mathbb R.$$ Here, for every $(x,y)$ in $\mathbb R^2$, $$f_{X,Y}(x,y)=24xy\mathbf 1_{0\lt x\lt1}\mathbf 1_{0\lt y\lt1}\mathbf 1_{0\lt x+y\lt1}.$$
(2) The second step is to use the fact that, for every joint density $f_{X,Y}$, the density $f_X$ is a function $$f_X:\mathbb R\to\mathbb R,$$ such that, for every $x$ in $\mathbb R$, $$f_X(x)=\int_\mathbb Rf_{X,Y}(x,y)\mathrm dy.$$ No fancy complicated interval of integration, just a plain integral on the whole real line--and it works, always (I told you, boring...).
All this is automatic, hence we can put our brain in position ON only now, and try to find a simple expression for $$f_X(x)=\int_\mathbb R24xy\mathbf 1_{0\lt x\lt 1}\mathbf 1_{0\lt y\lt1}\mathbf 1_{0\lt x+y\lt1}\mathrm dy.$$ Factoring everything independent on $y$ yields $$f_X(x)=24x\mathbf 1_{0\lt x\lt1}\int_\mathbb Ry\mathbf 1_{0\lt y\lt1}\mathbf 1_{0\lt x+y\lt1}\mathrm dy.$$ Note that, for every $0\lt x\lt1$ (since for other values of $x$, one does not care), $$\int_\mathbb Ry\mathbf 1_{0\lt y\lt1}\mathbf 1_{0\lt x+y\lt1}\mathrm dy=\int_0^{1-x}y\mathrm dy=\tfrac12(1-x)^2,$$ hence, finally, the density $f_X$ is defined, for every $x$ in $\mathbb R$, by the identity $$f_X(x)=\tfrac12(1-x)^2\cdot24x\mathbf 1_{0\lt x\lt1}=12x(1-x)^2\mathbf 1_{0\lt x\lt1}.$$
(Oh, and I forgot: to make a détour by the CDF when one is given a joint PDF and one looks for a marginal PDF, is like passing by Mexico and Vancouver to go from New York to Boston. It works, but.)
Here is a hint for your second case: for every $(x,y)$ in $\mathbb R^2$, $$\mathbf 1_{-y\lt x \lt y}\mathbf 1_{y\gt0}=\mathbf 1_{y\gt|x|}.$$
You are right that this looks like a bivariate normal distribution. Therefore let us try to express the given density in such a way. The density of a bivariate normal distribution is given by $g(z)= \frac{1}{2\pi \sqrt{\text{det} \Sigma}} e^{-\frac{1}{2}(z - \mu)^T \Sigma^{-1} (z-\mu)}, z\in \mathbb{R}^2 $, where $\Sigma \in \mathbb{R}^{2 \times 2}$ is the covariance matrix and $\mu \in \mathbb{R}^2$ is the expectation. The fact that the term in the exponent of your given density function does not contain any terms that are not depending on $x$ or $y$ suggests $\mu =0$. Now easy calculations give that
\begin{align}
\Sigma^{-1}= \left( \begin{matrix} 2& -1 \\ -1 & 1 \end{matrix} \right)
\end{align}
fulfills
\begin{align}
\left( \begin{matrix} x &y \end{matrix} \right) \Sigma^{-1} \left( \begin{matrix} x \\y \end{matrix} \right) = x^2 + (x-y)^2.
\end{align}
Now inverting gives $\Sigma= \left( \begin{matrix} 1& 1 \\ 1 & 2 \end{matrix} \right) $ and $\text{det}\Sigma=1$. Therefore the given density function is the density of a bivarite normal distribution with covariance matrix $\Sigma $ as above and zero expectation. It is well known that for a multivariate normal distribution $Z \sim N(\mu,\Sigma)$ the random variable $c^TZ$ has the distribution $N(c^T\mu, c^T \Sigma c)$ and thus in your case $X \sim N(0,1)$ and $Y \sim N(0,2)$.
Best Answer
Marginal pdf of $X$ is $$h(x)=\int_{-\sqrt{1-x^2}}^{\sqrt{1-x^2}}f(x,y)dy=c\int_{-\sqrt{1-x^2}}^{\sqrt{1-x^2}}\sqrt{1-x^2-y^2}dy$$
Let $\sqrt{1-x^2}=a$ then it becomes $$c\int_{-a}^{a}\sqrt{a^2-y^2}dy=2c\big(\frac{y}{2}\sqrt{a^2-y^2}+\frac{a^2}{2}\sin^{-1}\frac{y}{a}\big)|_{0}^a=2c(\pi a^2/4)=3a^2/4=3(1-x^2)/4$$
You can compute cdf as $$P(X\leq x)=H(x)=\int_{-1}^xh(t)dt$$