brownian motioncorrelationcross correlationpearson-r
I could not find anywhere, how to calculate correlation between two arrays. Say I do have Array1 with X and Y values and also Array2 with X and Y values. I tried to do some calculation and inserting two same arrays and expecting the correlation to be 1, but the result was something like -0.16…
I would be glad for any suggestion. Btw. I've found this Correlation coefficient between two arrays of 2D points? but still didnt find anything what could help me.
This is true that $SS_{tot}$ will change ... but you forgot the fact that the regression sum of of squares will change as well. So let's consider the simple regression model and denote the Correlation Coefficient as $r_{xy}^2=\dfrac{S_{xy}^2}{S_{xx}S_{yy}}$, where I used the sub-index $xy$ to emphasize the fact that $x$ is the independent variable and $y$ is the dependent variable. Obviously, $r_{xy}^2$ is unchanged if you swap $x$ with $y$. We can easily show that $SSR_{xy}=S_{yy}(R_{xy}^2)$, where $SSR_{xy}$ is the regression sum of of squares and $S_{yy}$ is the total sum of squares where $x$ is independent and $y$ is dependent variable. Therefore: $$R_{xy}^2=\dfrac{SSR_{xy}}{S_{yy}}=\dfrac{S_{yy}-SSE_{xy}}{S_{yy}},$$ where $SSE_{xy}$ is the corresponding residual sum of of squares where $x$ is independent and $y$ is dependent variable. Note that in this case, we have $SSE_{xy}=b^2_{xy}S_{xx}$ with $b=\dfrac{S_{xy}}{S_{xx}}$ (See e.g. Eq. (34)-(41) here.) Therefore: $$R_{xy}^2=\dfrac{S_{yy}-\dfrac{S^2_{xy}}{S^2_{xx}}.S_{xx}}{S_{yy}}=\dfrac{S_{yy}S_{xx}-S^2_{xy}}{S_{xx}.S_{yy}}.$$ Clearly above equation is symmetric with respect to $x$ and $y$. In other words: $$R_{xy}^2=R_{yx}^2.$$ To summarize when you change $x$ with $y$ in the simple regression model, both numerator and denominator of $R_{xy}^2=\dfrac{SSR_{xy}}{S_{yy}}$ will change in a way that $R_{xy}^2=R_{yx}^2.$
Trend can mess up your correlation matrix for sure. For instance, here's the code in MATLAB that demonstrates it.
x=randn(100,2);
corr(x)
subplot(2,1,1)
ans =
1.0000 0.1099
0.1099 1.0000
plot(x)
y=x+repmat((1:100)',1,2);
corr(y)
ans =
1.0000 0.9991
0.9991 1.0000
subplot(2,1,2)
plot(y)
All I did was to generate random noise with or without deterministic trend.
This effect is even more evident on crosscorrelation plots for these two cases:
>> subplot(2,1,1)
>> crosscorr(x(:,1),x(:,2))
>> subplot(2,1,2)
>> crosscorr(y(:,1),y(:,2))
Depending on your situation you might be interested in the trend or the noise. If you're interested in the noise, then the correlation matrix in the second case is meaningless, it's a spurious correlation. In this case differencing will help as shown below:
>> corr(diff(x))
ans =
1.0000 0.0479
0.0479 1.0000
You see how we got back the reasonable correlation, no correlation in this case. Differencing doesn't work in every case, of course.
Best Answer
An array with 2 dimensions is a matrix. Since your arrays have 2 dimensions, you are looking to find the correlation between these two matrices.
Here is a question with answers in R: "Correlation between matrices in R"
Here is a question with answers in MATLAB: "Matlab correlation between two matrices"