What is the difference between rate of convergence and order of convergence? Have they any relationship to each other? For example could i have two sequences with the same rates of convergence but different orders of convergence, and vice versa?
[Math] Relationship between rate of convergence and order of convergence
analysisnumerical methodssequences-and-series
Related Solutions
Following the relevant Wikipedia article, note that the rate of convergence is the number $\mu\in (0,1)$ such that
$$\lim_{k \to \infty} \frac{|x_{k+1} - L|}{|x_k - L|} = \mu $$
provided such a $\mu$ exists and $\{x_k\}$ converges to L. If this holds, $\{x_k\}$ is said to converge linearly to L. Note that the rate of convergence only exists if $\{x_k\}$ converges linearly to L!
But what if the above limit exists and equals $0$? Then $\{x_k\}$ is said to converge superlinearly.
Order of convergence is an additional definition use to distinguish between sequences that converge superlinearly. Such a sequence $\{x_k\}$ has order of convergence q if
$$\lim_{k \to \infty} \frac{|x_{k+1} - L|}{|x_k - L|^q} = C\:,\textrm{ for }\:q>1$$
Given that $C$ is a positive constant, and $\{x_k\}$ converges to L. This is the equation you have included in your question. $q$ need not be an integer, as in the case of the secant method, where q is in fact the golden ratio $\approx 1.618$.
The best intuitive explanation that I can give is that rate of convergence and order of convergence are two numbers used to describe the speed of different kinds of convergence. A sequence has either a rate of convergence (if the convergence is linear) or an order of convergence (if the convergence is superlinear), and not both. The higher the rate/order, the faster the convergence.
The sequence you provide converges via the first limit to $1$ (work it out!), so it has neither a rate of convergence nor an order of convergence.
Ian made two excellent points in comments:
In the absence of nice smoothness properties in the equation, nice convergence properties of a numerical method can fail.
Your numerical method is actually an accurate solver for a Poisson problem where the right side is the indicator function of a small square whose side length has order $h$. I think the convergence rate of the solution to this problem to the solution of the original problem may already be only second order, in which case refining the method can't improve anything.
I will elaborate on the second point. The discrete values of $\rho$ you feed into any finite difference method do not represent pointwise values of $\rho$, but rather its averages on scale $h$. So, feeding in $\rho=2$ at $(0.5,0.5)$ (which I assume is one of the grid points) corresponds to prescribing the Laplacian of $2$ on the square with vertices $(0.5\pm h/2, 0.5\pm h/2)$. The analytic solution to the latter problem is of order $h^2$. Indeed, it is given by the integral of Green's function $G$: $$ u(x,y) = \int_{0.5-h/2}^{0.5+h/2}\int_{0.5-h/2}^{0.5+h/2} 2G(x,y;x',y')\,dx'\,dy' $$ which, at a point $(x,y)\ne (0.5,0.5)$, is asymptotic to $h^2 G(x,y,0.5,0.5)$.
At $(x,y)=(0.5,0.5)$ the analytic solution is actually slightly larger: $h^2\log(1/h)$, but this is probably not apparent in the numerical results.
Best Answer
The order of convergence is one of the primary ways to estimate the actual rate of convergence, the speed at which the errors go to zero. Typically the order of convergence measures the asymptotic behavior of convergence, often up to constants. For example, Newton's method is said to have quadratic convergence, so the method has order 2. However, the true rate of convergence depends on the problem, the initial value taken, etc, and is typically impossible to quantify exactly. The order simply estimates this rate in terms of polynomial behavior, typically.
The order of convergence doesn't tell you everything. A numerical integration scheme with step size $h$ could have cubic order of convergence, so the errors go as $O(h^3)$, but the true error could be $100000h^3 + \ldots$, which would mean that for many practical problems the rate of convergence is actually quite slow.