I read a paper which want to solve two body problem with Second Order Runge-Kutta (the paper want to find optimum weight of Runge-Kutta with ANN). Here it is the two body problem.
\begin{align*}
x"=\frac{-x}{r^3}\\
y" = \frac{-y}{r^3}\\
r= \sqrt{x^2+y^2}
\end{align*}
First the author change two body problem into ordinary ODE system with 4 equation like this (I can understand)
$v=[v_1, v_2 , v_3 , v_4]=[x, y , x', y']$
$f(t,v)=v'(t)=[v'_1,v'_2,v'_3.v'_4]=[v_3,v_4,\frac{-v_1}{r^3},\frac{-v_2}{r^3}]$
And then the paper wants to find the best Runge-Kutta weights for two-body problems with ANN. ANN is not my focus here but I am confused why the Runge-Kutta solution become like this (I only write 2 from 4 equation here)
\begin{align*}
x_{n+1}&=x_n+h(b_1k_{1x}+b_2k_{2x})\\
k_{1x}&=x'_n\\
k_{2x}&=x'_n+ha_{21}k_{1x}=x'_n(1+ha_{21})
\end{align*}
\begin{align*}
y_{n+1}&=y_n+h(b_1k_{1y}+b_2k_{2y})\\
k_{1y}&=y'_n\\
k_{2y}&=y'_n+ha_{21}k_{1y}=y'_n(1+ha_{21})
\end{align*}
$b_1$,$b_2$ and $a_{21}$ are parameter from Second Order Rungge Kutta and h is step length.
I'm confused why $k_{2x}$ and $k_{2y}$ become like that. Could anyone explain? I attached the paper. Thank you.
Constructing Runge-Kutta Methods with the Use of Artificial Neural Networks
Best Answer
It seems that in the desctription of the layers the naming of the slope variables is overall inconsistent. In the initial description of the method for the system and its vector valued rhs function everything is correct. The graphical representation of the network also appears to be correct.
In a sensible notation convention, $k_{1x}$ is the slope that modifies $x_n$ and $k_{1x'}$ the slope that modifies $x_n'$. Writing it down component wise one should get \begin{align} k_{1x}&=x_n',& k_{2x}&=x_n'+ha_{21}k_{1x'},\\ k_{1y}&=y_n',& k_{2y}&=y_n'+ha_{21}k_{1y'},\\ k_{1x'}&=f_{x'}(x_n,y_n),& k_{2x'}&=f_{x'}(x_n+ha_{21}k_{1x},y_n+ha_{21}k_{1y}),\\ k_{1y'}&=f_{y'}(x_n,y_n),& k_{2y'}&=f_{y'}(x_n+ha_{21}k_{1x},y_n+ha_{21}k_{1y}),\\ \hline x_{n+1}&=x_n+h(b_1k_{1x}+b_2k_{2x}),\\ y_{n+1}&=y_n+h(b_1k_{1y}+b_2k_{2y}),\\ x'_{n+1}&=x'_n+h(b_1k_{1x'}+b_2k_{2x'}),\\ y'_{n+1}&=y'_n+h(b_1k_{1y'}+b_2k_{2y'}),\\ \end{align}
It seems somewhat strange that this is classified as a neural network problem. What I see is a computational graph with algorithmic derivatives in the backwards or gradient mode for a standard optimization problem. Of course, in general NN training is some kind of optimization by gradient descent or better (conjugate gradients, BFGS,...), the distinction of NN from standard optimization was for me that NN has a very large set of parameters that are in in themselves organized a layered network structure but are mostly unrelated, a black box, to the problem to be solved via NN. Thus the neural network is capable to specialize to a large family of functions, the learning procedure selects one that is close to the given data set.
However, here there is essentially only one free parameter $c_2=a_{21}$. If one wanted, one could plot the graph of the cumulative step errors for the training set in a graph over $c_2$ and determine the minimum of this scalar optimization problem. It would be more interesting to apply this approach to a 4-stage explicit method with order-3 conditions. Also use training data from elliptical orbits and a variety of radii to get a generally applicable result.
Further, due to the rotational symmetry of the system, inclusive error function, and the test data, it would be sufficient to consider only the steps with $(x_0,y_0)=(1,0)$ over the different values of $h$. Apply a weight according to the step size, for instance scale the step error with $h^{-3}$.
For circular orbits and several step sizes this gives the following graph close to the minimum $c_2\approx\frac29$