Ingo, when you consider the couple, you may put one of the "spouses" at the origin, so his torque is $P\times d_0$ for $d_0=0$, so his torque vanishes. Meanwhile, she is located at a nonzero $d$ so her contribution is $P\times d$ and nonzero. Because his torque is zero, it doesn't matter whether you add him or not.
The only difference between the whole couple (including the husband at the origin) and the separate wife at nonzero $d$ is that the total force (not moment) of the whole couple is zero, while the wife separately acts both by torque and by a nonzero ordinary force as well.
Hint : Friction opposes tendency to move. Tension is produced if the string is stretched $very$ slightly. So, increase friction to maximum and then tension will act if necessary.
Ironically, you are thinking absolutely right. Give yourself a cookie.
From part $a$, we know that the blocks will be at rest at all angles below that.
You are also right as at very small angles there is no need of tension and we can ignore it to solve for, again, an angle condition. You have done excellent work. Congrats.
Now we come to the middle angles. Oh... they drive you insane, don't they?
Let's start. We can start our analysis from 2 blocks, 1 will give a contradiction and other will give a result, but I will start with the one giving contradiction. This will help you.
All angles are in degrees :
$\theta=35 $
Lets start by analysing Block A (No racism intended)
Gravity is trying to pull it down : $5*10*\sin(35)N=28.67N$
Friction comes to the rescue(up) : $50*\cos(35)N=16.38N$ // read my hint to know why friction is put max here
As it is at rest, $T=12.29N$
Now Block B is also at rest,
Weight = $114.71N$
max f= $81.92N$
$16.38+114.71=12.29+f$
$f=118.8N$
OOPS, it exceed max value. So, Lets start by analysing Block B. (I love alliteration)
Gravity trying : $114.71N$
Friction comes to the rescue(up) : $81.92N$
You can take from here I guess. calculate tension. Note that you have to revise your calculation for tension again as reaction friction force will be provided by A. Better assume it $f$ from starting FBD of B.
This will yield the correct answer. Friction will be less than max value for upper block. In most cases, You should start analysing with heavier block(my experience). Hope your doubts are cleared.
Best Answer
The first derivation is correct, but only if you mean to take the difference between the two vectors, $\mathbf{F_1} - \mathbf{F_2}$; the figure would then show $\mathbf{F_D}$ running from the tip of one vector to the tip of the other, across the parallelogram. This is the Law of Cosines, which refers to the angle enclosed by the two sides of the triangle: $$F_D^2=\Vert \mathbf{F_2} - \mathbf{F_1}\Vert^2=\Vert\mathbf{F_1}\Vert^2+\Vert\mathbf{F_2}\Vert^2-2\mathbf{F_1}\mathbf{F_2}=F_1^2+F_2^2-2F_1F_2\cos(\alpha )$$
The second derivation obtains the correct result, but is flawed. Here we follow the figure in order to obtain the longer diagonal of the parallelogram by moving $\mathbf{F_1}$ to the tip of $\mathbf{F_2}$, the enclosed angle becomes the supplement, $(\pi - \alpha )$; and the vector $\mathbf{F_1}$ should be pointing away from the new vertex.
Now when we evaluate the law of cosines we get $$F_R^2=\Vert-\mathbf{F_2} + \mathbf{F_1}\Vert^2=\Vert\mathbf{F_1}\Vert^2+\Vert\mathbf{F_2}\Vert^2+2\mathbf{F_1}(-\mathbf{F_2})=F_1^2+F_2^2-2F_1F_2\cos(\pi - \alpha )$$
But $cos(\pi - \alpha ) = -cos(\alpha )$, which recovers the second formula, from the first: $$F_R^2=\Vert\mathbf{F_2}-\mathbf{F_1}\Vert^2=\Vert\mathbf{F_1}\Vert^2+\Vert\mathbf{F_2}\Vert^2-2\mathbf{F_1}\mathbf{F_2}=F_1^2+F_2^2+2F_1F_2\cos\alpha$$
You can also obtain the second formula from Euclid's Law of Parallelograms, which states that the the sum of the squares of the four sides is equal to the sum of the squares of the two diagonals. Subtract the first formula, the Law of Cosines, which gives the square of the length of the short diagonal, from the sum of the squares of the four sides, $$ F_R^2 + F_D^2=(2\Vert\mathbf{F_1}\Vert^2+2\Vert\mathbf{F_2}\Vert^2)$$ and you are left with the second formula, which gives the length squared of the long diagonal of the parallelogram.