I was wondering how $p$-values change with sample size or is their any relation between the two. To my knowledge, a $p$-value denotes the probability of finding observed or more extreme results than the null hypothesis claims (typically no difference). Based on the following example, let your null hypothesis be that there exists no difference in the amount of heads and tails you flip on a fair coin, that is you flip the same exact amount and your alternative be that a difference does exist. You flip a fair coin $n = 10$ times and get $7$ heads and $3$ tails, which suggests a relatively low $p$-value. But as you flip this coin more times (say $n = 100$) and now get $45$ heads and $55$ tails, your $p$-value increases – which results in you being more likely to fail to reject the null over the alternative hypothesis.
Thus, does increasing sample size, increase $p$-values in general?
Best Answer
Well, the p-value can be seen as a random variable, so as you get more data, calculate the p-value anew, the value will most probably change. But I take your question to be if the distribution of the p-value changes.
Your question was not completely clear, but I take the question to be if the distribution will change under the null hypothesis. By the definition of the p-value, then, under the null hypothesis its distribution is uniform on $(0,1)$. That does not depend on the sample size $n$. So, under the null hypothesis the answer is NO. The p-value (distribution) do not depend on $n$.
For the alternative the answer is different. If the alternative is true, we expect that with more data we will get more evidence against the null, so the p-value will be (stochastically) smaller. In that case, the p-value distribution will depend on $n$.
But there might be counterexamples, for instance, if you are using a bad (not consistent or not using the data effectively) hypothesis test, or if your data is not really bearing on your hypothesis. But for the majority of reasonable situations, the conclusion will hold.
For the future, such questions are better asked at https://stats.stackexchange.com/, where you would have got good answers must faster than here.