MATLAB: Statistical Significance using ttest2

statisticsttest2

I'm curious how to determine whether two datasets are statistically similar from a two-sample t-test (ttest2). I understand that the result gives the decision to reject (1) or accept (0) the null hypothesis, but how does the null hypothesis decision give statistical significance?

Here's an example: If I have two thermometers (A and B) that record temperature at the same location (but at different sampling frequencies) for one year, I would get two time series that should look similar but have a different number of data points. Using h = ttest2(A,B,'Vartype','unequal') h would be either 0 or 1.

Given the null hypothesis of: "the data in vectors A and B comes from independent random samples from normal distributions with equal means and equal but unknown variances"(from https://www.mathworks.com/help/stats/ttest2.html ), wouldn't rejection of this hypothesis mean that the datasets are NOT correlated with statistical significance? If so, does 0 represent high correlation in this case?

Thanks for your help.

Best Answer

The paired t-test says nothing about correlation. (The 'unequal' flag is about the variances.)

Consider:

t = linspace(0, 10*pi);
v1 = sin(t);
v2 = 2*rand(size(t))-1;
[h,p] = ttest2(v1, v2)
h =
     0
p =
     0.6819

It does indicate if the means are different (rejecting the null hypothesis) at a specific significance.

EDIT — Added example.

Related Solutions

MATLAB: Kstest and hypothesis rejecting

When in doubt, use help()

help kstest
 kstest Single sample Kolmogorov-Smirnov goodness-of-fit hypothesis test.
    H = kstest(X) performs a Kolmogorov-Smirnov (K-S) test to determine if
    a random sample X could have come from a standard normal distribution,
    N(0,1). H indicates the result of the hypothesis test:
       H = 0 => Do not reject the null hypothesis at the 5% significance

       level. 
       H = 1 => Reject the null hypothesis at the 5% significance
       level.

The null hypothesis depends on your inputs. Again, from the documentation using doc(ktest):

Test the null hypothesis that the data comes from a normal distribution with 
a mean of 75 and a standard deviation of 10. Use these parameters to center 
and scale each element of the data vector since, by default, kstest tests for 
a standard normal distribution.
x = (test1-75)/10;
h = kstest(x)

MATLAB: Problem in statistical test

The terminology can be confusing. From the Wikipedia article Null hypothesis (Basic definitions):

"The statement being tested in a test of statistical significance is called the null hypothesis. The test of significance is designed to assess the strength of the evidence against the null hypothesis. Usually, the null hypothesis is a statement of 'no effect' or 'no difference'."

and from the ttest2 documentation:

‘If h = 0, this indicates a failure to reject the null hypothesis at the Alpha significance level.’

So the hypothesis that there is no difference (null hypothesis) is accepted (i.e. not rejected).

There is no difference between the two groups at the chosen significance level, and assuming that they conform to the conditions of the t-test.

At least that is the way I have always read it.

.

Related Question