The case when $A$ and $B$ are independent you can do in your head. For the general dependent case, instead of finding $f_{XY}(x,y)$, you can compute $E[X|Y=y]$ by integrating $f_{AB}(a,b)$ over the line segments:
$$\{(a, y) : a\in [y,1]\}\cup \{(y,b) : b \in [y,1]\}$$
So:
$$ E[X|Y=y] = \frac{\int_{y}^1 af_{AB}(a,y)da + \int_{y}^1 bf_{AB}(y,b)dy}{\int_{y}^1f_{AB}(a,y)da + \int_{y}^1f_{AB}(y,b)dy} $$
If $A$ and $B$ are independent then $f_{AB}(a,b)=1$ for all $a,b \in [0,1]$ and the above integrals give $E[X|Y=y]=(1+y)/2$.
Of course, a more intuitive way in the independent case is to just observe that, given the min is $y$, the max is uniformly distributed over $[y,1]$, so its mean is the midpoint $(1+y)/2$.
Assuming you mean $\lambda$ is the rate parameter here (i.e. Exponential with mean $1/\lambda$).
First of all, recheck your density of $S$. The correct density as mentioned in comments is $$f_S(s)=\lambda^2se^{-\lambda s}\mathbf1_{s>0}$$
It is easy to verify that the density of $T$ is $$f_T(t)=\mathbf1_{0<t<1}$$
You can find the joint distribution function of $(S,T)$ as follows:
For $s>0$ and $0<t<1$,
\begin{align}
P(S\le s,T\le t)&=P\left(X+Y\le s,\frac{X}{X+Y}\le t\right)
\\&=\iint_D f_{X,Y}(x,y)\,dx\,dy\quad\quad,\text{ where }D=\{(x,y):x+y\le s,x/(x+y)\le t\}
\\&=\lambda^2\iint_D e^{-\lambda(x+y)}\mathbf1_{x,y>0}\,dx\,dy
\end{align}
Change variables $(x,y)\to(u,v)$ such that $$u=x+y\quad,\quad v=\frac{x}{x+y}$$
This implies $$x=uv\quad,\quad y=u(1-v)$$
Clearly, $$x,y>0\implies u>0\,,\,0<v<1$$
And $$dx\,dy=u\,du\,dv$$
So again for $s>0\,,\,0<t<1$,
\begin{align}
P(S\le s,T\le t)&=\lambda^2\iint_R ue^{-\lambda u}\mathbf1_{u>0,0<v<1}\,du\,dv\qquad,\text{ where }R=\{(u,v):u\le s,v\le t\}
\\&=\left(\int_0^s \lambda^2 ue^{-\lambda u}\,du\right) \left(\int_0^t \,dv\right)
\\\\&=P(S\le s)\,P(T\le t)
\end{align}
This proves the independence of $S$ and $T$, with $S$ a Gamma variable and $T$ a $U(0,1)$ variable.
And from the joint distribution function, it is readily seen (without differentiating) that the joint density of $(S,T)$ is $$f_{S,T}(s,t)=\lambda^2 se^{-\lambda s}\mathbf1_{s>0,0<t<1}$$
Note that the change of variables isn't really necessary if you are comfortable with the first form of the double integral.
Best Answer
You seem to be confused on a couple of points so I will try to address them in order the order you asked;
Hope that helps!