I need to test random numbers generators in 1, 2 and 3 dimensions. Criteria of test is that generated numbers are from uniform distribution. I am doing this using Matlab. I'm told to use Chi-2 test statistic $$\chi^2=\sum^B_{i=1}\frac{(expected(i)-observed(i))^2}{expected(i)}$$
where $B$ is number of bins, $expected(i)=\frac{N}{B}$ for all $i$ is expected number of elements in $i_{th}$ bin and $N$ is number of generated numbers.
What I have done is that I generated
- $N\times 1$ matrix of random numbers for $1D$
- $N\times 2$ matrix of random numbers for $2D$
- $N\times 3$ matrix of random numbers for $3D$
Then, I set a number of bins (B) for each component of matrix elements and I created zero matrices of sizes
- $1\times B$ for $1D$
- $B\times B$ for $2D$
- $B\times B\times B$ for $3D$
and, using loops, I counted how many generated numbers are in which bin and those were, actually, values $observed(i)$.
At this point I am able to calculate $\chi^2$ test statistic for all 3 dimensions.
I know how to read $\chi^2$ table but I don't know how to check significance using Matlab. I'm also not sure about degrees of freedom, but I think it's $B-1$ for $1D$, $BB-1$ for $2D$ and $BBB-1$ for $3D$ since we don't have to estimate any parameter for uniform distribution. I would appreciate some help. Thanks!
Best Answer
Just to expend on it and give the all the code.
Setting parameters
Sample N uniformly distributed values between a and b. And in the second line add some bais to make it not uniform if you want to test the code.
Using
chi2gof
As described here, with
chi2gof
, you can't use the 'cdf of the hypothesized distribution' and need to specified the bins, the edges and the expected values.Using
chi2cdf
With this function you need to supply the chi-squared test statistic, $\displaystyle \chi ^{2}$ which can be computed with the function
histogramm
:Note, that if you don't use the edges to compute the number of value per bins,
histogramm
will choose them from the lower value to the highest and therefore the final score will be different than withchi2gof
Quick Theory recall
Just to recall how to interpret the final value, few definition:
x
are coming from a uniform distribution.x
is not a coming from a uniform distribution ? "y
in a uniform distribution, the error (or Pearson's cumulative test) will be equal of higher than the actual observationx
(=worst case).p
is lower than a significant level $\alpha$. That is, for small value ofp
, we can safely say thatx
is not coming from a uniform distribution.