The aim of our answer is to show analytically that what OP is striving for is viable only in a very special case of the described scenario. A set of equations to determine heights and widths of the photos for this case is given.
For brevity, we use simple letters $h$, $w$, $s$, and $r$ to denote heights, widths, scale factors, and aspect ratios of the geometrical entities we are dealing with. They are subscripted with capital letters $\bf{A}$, $\bf{B}$, $\bf{C}$, and $\bf{F}$ to show that they belong to photo-A, -B, -C, and frame respectively. The lateral gap between photos is denoted by $x$.
We wish to scale down (or up) the three photos encased in a frame, say Frame-0, (see $\mathrm{Fig.\space 1}$), so that they fit neatly as shown in $\mathrm{Fig.\space 2}$ inside a new frame, say Frame-1 while observing following constraints.
$\qquad 1.\space \text{The lateral gap between the photos is the same (i.e. $x$) for both scenarios.}$
$\qquad 2.\space \text{Scaling of photos must be distortionless (i.e. their aspect ratios should be preserved)}.$
Theoretically, the gap between photos and the scale factors can have any positive real values greater than zero. But in practice, we do not let scale factors to be equal to 1, since that means no scaling.
We hope to derive a relationship between the aspects ratios of the two frames as a function of scale factor of photo-A, (i.e. $s_A$), and gap between the photos, (i.e. $x$). To do this we use the three equations given in OP’s problem statement and several of ours.
$\qquad\qquad\text{Equations for the Frame-0:}$
$$h_\mathrm{F0} =h_\mathrm{C} = h_\mathrm{A} + x + h_\mathrm{B} \tag{1}$$
$$w_\mathrm{B} = w_\mathrm{A} \tag{2}$$
$$w_\mathrm{F0} = w_\mathrm{A} + x + w_\mathrm{C} \tag{3}$$
$\qquad\qquad\text{Equations for the Frame-1:}$
$$h_\mathrm{F1} =s_\mathrm{C} h_\mathrm{C} = s_\mathrm{A} h_\mathrm{A} + x + s_\mathrm{B} h_\mathrm{B} \tag{4}$$
$$ s_\mathrm{B} w_\mathrm{B} = s_\mathrm{A} w_\mathrm{A} \tag{5}$$
$$w_\mathrm{F1} = s_\mathrm{A} w_\mathrm{A} + x + s_\mathrm{C} w_\mathrm{C} \tag{6}$$
Equations (2) and (5) give us
$$s_\mathrm{B} = s_\mathrm{A}. \tag{7}$$
Using (1), (2), and (5), we can show that
$$h_\mathrm{F1} = s_\mathrm{A} h_\mathrm{F0}+ \left( 1 - s_\mathrm{A}\right) x \qquad\text{and} \tag{8}$$
$$s_\mathrm{C} = s_\mathrm{A} + \left( 1 - s_\mathrm{A}\right)\frac{x}{ h_\mathrm{F0}} .\qquad\qquad \tag{9}$$
Replacing the last term on the right hand side of (6) using (3) and (9) yields the following expression for $w_\rm{F1}$,
$$w_\mathrm{F1} = s_\mathrm{A} w_\mathrm{A} + x + \bigg\{ s_\mathrm{A} + \left(1- s_\mathrm{A}\right) \frac{x}{ h_\mathrm{F0}} \bigg\} \left(w_\mathrm{F0}- w_\mathrm{A} -x\right), $$
which can be expanded and simplified to get,
$$w_\mathrm{F1} = s_\mathrm{A} w_\mathrm{A} +\left(1- s_\mathrm{A}\right)\left(h_\mathrm{F0} + w_\mathrm{F0}- w_\mathrm{A} -x \right) \frac{x}{ h_\mathrm{F0}}. \tag{10}$$
Now, we can write down an expression for the aspect ratio, i.e. $\frac{h_{F1}}{w_{F1}}$ of the frame-1 in terms of the aspect ratio, i.e. $\frac{h_{F0}}{w_{F0}}$ of the frame-0 using (8) and (10).
$$\frac{h_{F1}}{w_{F1}} = \frac{ s_\mathrm{A} h_\mathrm{F0}+ \left( 1 - s_\mathrm{A}\right) x}{ s_\mathrm{A} w_\mathrm{A} + x + \Big\{ s_\mathrm{A} + \left(1- s_\mathrm{A}\right) \frac{x}{ h_\mathrm{F0}} \Big\} \left(w_\mathrm{F0}- w_\mathrm{A} -x\right)} $$
When simplified, this reduces to
$$r_\mathrm{F1}= \cfrac{h_\mathrm{F1}}{w_\mathrm{F1}} = \cfrac{r_\mathrm{F0}}{1+\cfrac{x\left(1 - s_\mathrm{A}\right)\left(h_\mathrm{F0}- w_\mathrm{A}-x\right)}{\Bigl\{ s_\mathrm{A} h_\mathrm{F0}+x\left(1 - s_\mathrm{A}\right)\Big\} w_\mathrm{F0}}}. \tag{11}$$
If we manage to maintain the aspect ratio of the frame while scaling, we have $r_\mathrm{F1}= r_\mathrm{F0}$. For this to happen, we must have,
$$x\left(1 - s_\mathrm{A}\right)\left(h_\mathrm{F0}- w_\mathrm{A}-x\right)=0. \tag{12}$$
This has three diffrent solutions, each one is corresponding to a special cases of the scaling scenario, i.e.
$$x = 0,\qquad\quad\tag{13}$$
$$s_\mathrm{A} = 1, \quad\text{and} \tag{14}$$
$$x= h_\mathrm{F0}- w_\mathrm{A}. \tag{15}$$
We have already stated at the beginning of this answer that we are not interested in the first two cases. Therefore, there remains only one case, namely $ x= h_\mathrm{F0}- w_\mathrm{A}$, which gives us exactly what OP want. However, whether this case can be realized in practice at will or not is another story. Please note that (15) implies $h_\mathrm{F0} \gt w_\mathrm{A}$.
We can give a test that will show whether a frame-0 with its three photos can be successfully scaled to obtain a frame-1 with all aspect ratios intact. As shown in $\mathrm{Fig.\space 3}$, draw the two diagonals of the rectangle-A and rectangle-B to intersect each other. If this point of intersection lands anywhere on the left vertical side of the rectangle-C, you can perform the scaling while preserving the aspect ratios of all entities.
The following three equations can be used to calculate the scale factor of each of the photos, using which the width and height of the individual photo can be determined.
$$s_\mathrm{A} = s_\mathrm{B} = \frac{h_\mathrm{F1} - x}{h_\mathrm{F0} - x} \tag{16}$$
$$s_\mathrm{C}=\frac{ h_\mathrm{F1} }{ h_\mathrm{F0} }\tag{17}$$
$\underline{\text{Conclusion}}$:
Unless the photos and the frame of the original set satisfy the geometrical constrain (15), i.e.
$$x = h_\mathrm{F0} - w_\mathrm{A},\quad\text{where}\quad h_\mathrm{F0} \gt w_\mathrm{A},$$
they cannot be scaled while preserving their respective aspect ratios to arrange them inside a new frame, the aspect ratio of which is equal to that of the original frame.
Best Answer
Let $h_1, h_2$ be the heights of the rectangles and $w_1, w_2$ be their widths. Let $w_T$ be the total desired width. We seek scaling constants $c_1, c_2$ such that when the dimensions of the rectangles are multiplied by $c_1$ and $c_2$, respectively, their heights $c_1 h_1$ and $c_2 h_2$ are equal; moreover, their total width $c_1 w_1 + c_2 w_2$ must equal the total desired width $w_T$. Hence we seek the solution to the system of equations $$\begin{align} c_1 w_1 + c_2 w_2 &= w_T, \\ c_1 h_1 &= c_2 h_2. \end{align}$$ The second equation implies $$c_2 = \frac{c_1 h_1}{h_2},$$ so substituting this into the first equation and solving for $c_1$ yields $$c_1 = \frac{w_T}{w_1 + \frac{h_1}{h_2} w_2} = \frac{h_2 w_T}{w_1 h_2 + h_1 w_2}.$$ This in turn implies $$c_2 = \frac{h_2 w_T}{w_1 h_2 + h_1 w_2} \cdot \frac{h_1}{h_2} = \frac{h_1 w_T}{w_1 h_2 + h_1 w_2}.$$ These are the desired scaling factors.
Let us put this result into practice. Suppose the first rectangle has height $h_1 = 5$ and width $w_1 = 3$, and the second rectangle has height $h_2 = 4$ and width $w_2 = 7$. We want the total width to be $w_T = 9$. Then the required scaling factors are $$c_1 = \frac{4(9)}{3(4) + 5(7)} = \frac{36}{47} \approx 0.765957, \quad c_2 = \frac{5(9)}{47} = \frac{45}{47} \approx 0.957447.$$ (Notice the denominators of $c_1$ and $c_2$ are the same.) The resulting scaled rectangles have common height $$c_1 h_1 = c_2 h_2 = \frac{180}{47} \approx 3.82979,$$ and widths $$c_1 w_1 = \frac{108}{47} \approx 2.29787, \quad c_2 w_2 = \frac{315}{47} \approx 6.70213.$$ Their total width is $$c_1 w_1 + c_2 w_2 = \frac{108 + 315}{47} = \frac{423}{47} = 9 = w_T.$$