If strong duality does not hold, then we have no reason to believe there must exist Lagrange multipliers such that jointly they satisfy the KKT conditions. Here is an counter-example
${\bf counter-example 1}$
If one drops the convexity condition on objective function, then strong duality could fails even with relative interior condition.
The counter-example is the same as the following one.
${\bf counter-example 2}$
For non-convex problem where strong duality does not hold, primal-dual optimal pairs may not satisfy KKT condition.
Consider the optimization problem
\begin{align}
\operatorname{minimize} & \quad e^{-x_1x_2} \\
\text{subject to} & \quad x_1\le 0.
\end{align}
The domain for the problem is $D = \{ (x_1,x_2) \ge 0 \}$. The problem is not convex by calculating the Hessian matrix. Clearly, any $x_1 = 0, x_2 \in\mathbb R_+$ is a primal optimal solution with primal optimal value $1$ .
The Lagrangian is
$$
L(x_1,x_2,\lambda) = e^{-x_1x_2} + \lambda x_1.
$$
The dual function is
\begin{align}
G(\lambda) &= \inf L(x_1,x_2,\lambda) =
\begin{cases}
0& \lambda\ge 0\\
-\infty& \lambda < 0
\end{cases}
\end{align}
Thus, $\lambda \geq 0$ is dual optimal solution with dual optimal value $0$, so dual gap is $1$, strong duality fails. As for the KKT conditions, remember the domain is $D = \{ (x_1,x_2) \ge 0 \}$
\begin{align*}
&\lambda-x_2e^{-x_1x_2}=0\\
&x_1\le 0\\
&\lambda\ge 0\\
&\lambda x_1=0\\
\end{align*}
Pick any primal-dual pair satisfying $x_1 = 0, x_2\ge 0, \lambda\ge0, \lambda\ne x_2$, the KKT conditions fail.
${\bf counter-example 3}$
For a non-convex problem, even strong duality holds, solutions for KKT conditions may not be primal-dual optimal solution.
Consider the optimization problem on $\mathbb R$
\begin{align}
\operatorname{minimize} & \quad x^3 \\
\text{subject to} & -x^3-1\le 0.
\end{align}
The objective function is not convex by calculating the Hessian matrix. Clearly, $x=-1$ is the unique primal optimal solution with primal optimal value $-1$.
The Lagrangian is
$$
L(x,\lambda) = x^3 - \lambda (x^3+1).
$$
The dual function is
\begin{align}
G(\lambda) &= \inf L(x,\lambda) =
\begin{cases}
-1& \lambda=1\\
-\infty& otherwise
\end{cases}
\end{align}
Thus, $\lambda = 1 $ is dual optimal solution with dual optimal value $-1$, so dual gap is $0$, strong duality holds. While the KKT conditions are
\begin{align*}
&3x^2(1-\lambda)=0\\
&-x^3-1\le 0\\
&\lambda\ge 0\\
&\lambda (-x^3-1)=0\\
\end{align*}
Solutions for KKT conditions are $x=-1, \lambda=1$ or $x=0,\lambda=0$. Notice that $x=0,\lambda=0$ satisfies KKT conditions but has nothing to do with primal-dual optimal solutions.
The discussion indicates for non-convex problem, KKT conditions may be neither necessary nor sufficient conditions for primal-dual optimal solutions.
${\bf counter-example4}$
For a convex problem, even strong duality holds, there could be no solution for the KKT condition, thus no solution for Lagrangian multipliers.
Consider the optimization problem on domain $\mathbb R$
\begin{align}
\operatorname{minimize} & \quad x \\
\text{subject to} & \quad x^2\le 0.
\end{align}
Obviously, the problem is convex with unique primal optimal solution $x=0$ and optimal value $0$; Feasible set is $\{0\}$, therefore Slater's condition fails.
The Lagrangian is
$$
L(x,\lambda) = x + \lambda x^2.
$$
The dual function is
\begin{align}
G(\lambda) &= \inf L(x,\lambda) =
\begin{cases}
-\infty& \lambda\le 0\\
-\frac{1}{4\lambda} &\lambda >0
\end{cases}
\end{align}
Thus, dual optimal value is $0$, so dual gap is $0$, strong duality holds. However, there are no solution for dual optimal solution because the optimal value is attained as $\lambda\rightarrow \infty$. As for the KKT conditions
\begin{align*}
&1+2\lambda x=0\\
&x^2\le 0\\
&\lambda\ge 0\\
&\lambda x^2=0\\
\end{align*}
No solution for KKT conditions.
This is the convex problem where the dual problem has no feasible solution and KKT conditions have no solution but the primal problem is simple to solve.
${\bf counter-example 5}$
For a differentiable convex problem, there could be no solution for the KKT conditions, even if the primal-dual pair exists. In that case, the strong duality fails.
Consider the optimization problem on domain $D:=\{(x,y):y>0\}$
\begin{align}
\operatorname{minimize} & \quad e^{-x} \\
\text{subject to} & \quad \frac{x^2}{y}\le 0.
\end{align}
The problem can be proved to be convex with primal optimal solution $x=0, y>0$ and optimal value $1$; Feasible set is $\{(0,y): y > 0\}$, therefore Slater's condition fails.
The Lagrangian is
$$
L(x,y,\lambda) = e^{-x} + \lambda \frac{x^2}{y}.
$$
After some careful calculation, the dual function is
\begin{align}
G(\lambda) &= \inf L(x,y,\lambda) =
\begin{cases}
0& \lambda\ge 0\\
-\infty &\lambda <0
\end{cases}
\end{align}
Thus, dual optimal value is $0$, so dual gap is $1$, strong duality fails. We can pick primal-dual pair to be $x=0, y=1, \lambda=2$. As for the KKT conditions
\begin{align*}
&-e^{-x}+\frac{2\lambda x}{y}=0\\
&\frac{x^2}{y}\le0\\
&\lambda\ge 0\\
&\lambda \frac{x^2}{y}=0\\
\end{align*}
with $y>0$, thus no solution for KKT conditions.
This counter-example warns us that we have to be careful about the strong duality condition even for differentiable convex problems.
Best Answer
Since $R=KQ$, you can focus on solving for $Q$. So your problem looks like \begin{align} \min_{Q}||J-KQ||_F^2 \end{align} (squaring of the objective won't change the solution, convince yourself). Now, we have $$vec(J-KQ)=vec(J)-(I\otimes K)vec(Q)$$ where $vec(.)$ operator stacks up the columns of argument matrix and $\otimes$ is the kronecker product. Again $$||J-KQ||_F^2=||Ax-b||_2^2$$ where I define $A=I\otimes K$ and $b=vec(J)$ and $x=vec(Q)$. Thus \begin{align} \min_{Q}||J-KQ||_F^2 \,\,\,\,\,\equiv\,\,\,\,\,\min_{x}||Ax-b||_2^2 \end{align} Now try solving the last problem which is related to the solution of a linear system of equations and least squares.
EDIT
Expand $||Ax-b||_2^2=(Ax-b)^T(Ax-b)$. Now try to derive the KKT conditions using vector differentiation. A good place to start would be the wiki article. If KKT conditions was your only requirement, you could have done that earlier itself. Note that $$||J-KQ||_F^2=\mbox{trace}\left\{(J-KQ)^T(J-KQ)\right\}$$. Expand the multiplication insider the trace and then use the rules of matrix differentiation. Go through the same wiki article on how to do that. I seriously urge you put some effort into this. It will be really helpful.