IMO (as not-a-logician or formally trained statistician per se), one shouldn't take any of this language too seriously. Even rejecting a null when p < .001 doesn't make the null false without a doubt. What's the harm in "accepting" the alternative hypothesis in a similarly provisional sense then? It strikes me as a safer interpretation than "accepting the null" in the opposite scenario (i.e., a large, insignificant p), because the alternative hypothesis is so much less specific. E.g., given $\alpha=.05$, if p = .06, there's still a 94% chance that future studies would find an effect that's at least as different from the null*, so accepting the null isn't a smart bet even if one cannot reject the null. Conversely, if p = .04, one can reject the null, which I've always understood to imply favoring the alternative. Why not "accepting"? The only reason I can see is the fact that one could be wrong, but the same applies when rejecting.
The alternative isn't a particularly strong claim, because as you say, it covers the whole "space". To reject your null, one must find a reliable effect on either side of the null such that the confidence interval doesn't include the null. Given such a confidence interval (CI), the alternative hypothesis is true of it: all values within are unequal to the null. The alternative hypothesis is also true of values outside the CI but more different from the null than the most extremely different value within the CI (e.g., if $\rm CI_{95\%}=[.6,.8]$, it wouldn't even be a problem for the alternative hypothesis if $\mathbb P(\rm head)=.9$). If you can get a CI like that, then again, what's not to accept about it, let alone the alternative hypothesis?
There might be some argument of which I'm unaware, but I doubt I'd be persuaded. Pragmatically, it might be wise not to write that you're accepting the alternative if there are reviewers involved, because success with them (as with people in general) often depends on not defying expectations in unwelcome ways. There's not much at stake anyway if you're not taking "accept" or "reject" too strictly as the final truth of the matter. I think that's the more important mistake to avoid in any case.
It's also important to remember that the null can be useful even if it's probably untrue. In the first example I mentioned where p = .06, failing to reject the null isn't the same as betting that it's true, but it's basically the same as judging it scientifically useful. Rejecting it is basically the same as judging the alternative to be more useful. That seems close enough to "acceptance" to me, especially since it isn't much of a hypothesis to accept.
BTW, this is another argument for focusing on CIs: if you can reject the null using Neyman–Pearson-style reasoning, then it doesn't matter how much smaller than $\alpha$ the p is for the sake of rejecting the null. It may matter by Fisher's reasoning, but if you can reject the null at a level of $\alpha$ that works for you, then it might be more useful to carry that $\alpha$ forward in a CI instead of just rejecting the null more confidently than you need to (a sort of statistical "overkill"). If you have a comfortable error rate $\alpha$ in advance, try using that error rate to describe what you think the effect size could be within a $\rm CI_{(1-\alpha)}$. This is probably more useful than accepting a more vague alternative hypothesis for most purposes.
* Another important point about the interpretation of this example p value is that it represents this chance for the scenario in which it is given that the null is true. If the null is untrue as evidence would seem to suggest in this case (albeit not persuasively enough for conventional scientific standards), then that chance is even greater. In other words, even if the null is true (but one doesn't know this), it wouldn't be wise to bet so in this case, and the bet is even worse if it's untrue!
If you do a two-tailed test and computation gives you $p=0.03$, then $p<0.05$. The result is significant. If you do a one-tailed test, you will get a different result, depending on which tail you investigate. It will be either a lot larger or only half as big.
$\alpha=0.05$ is the usual convention, no matter whether you test one- ode two-tailed. You don't halve that (except maybe in Bonferroni-correction, which is not the topic here). Thus yes, sometimes a one-tailed test will give you a significant result where the two-tailed does not. However, this is not how things work: You have to always determine upfront, whether you consider a one- or a two-tailed test appropriate as you have to determine your $\alpha$-level upfront. Then you calculate the $p$-value for that test and there are no more degrees of freedom how to test or what to compare the $p$-value to. If you determine on the sidedness of your test depending on whether you like the result, this is not good scientific practice.
That being said, there is hardly ever a situation where it is appropriate to test one-tailed. In far most circumstances it would be worth communicating a significant result in both directions. If you test one-tailed, some of your audience will consider this a trick to hack your $p$-value into being as small as possible.
Best Answer
ISTR there is a form of hypothesis testing where the null hypothesis is the thing you want to assert to be true. IIRC this is based on statistical power, which is the probability [in a frequentist sense] that the null hypothesis will be rejected when it is false. Therefore if the p-value is above the significance level, but the test has high statistical power, then we would expect the null to be rejected if it were false as the test has high power, so the fact that it doesn't suggests it isn't, simple! ;o)
I'll see if I can remember what it is called and look it up, until then caveat lector!
Update: I think what I had in mind is "accept support" hypothesis testing, rather than "reject support" testing, see e.g. here.
Another (hopefully) illustrative update:
Climate skeptics often claim that there has been no global warming since 1998, often citing a BBC interview with Prof. Phil Jones of the Climatic Research Unit at UEA (where I also work). Prof. Jones was asked:
Q: Do you agree that from 1995 to the present there has been no statistically-significant global warming
and answered:
A: Yes, but only just. I also calculated the trend for the period 1995 to 2009. This trend (0.12C per decade) is positive, but not significant at the 95% significance level. The positive trend is quite close to the significance level. Achieving statistical significance in scientific terms is much more likely for longer periods, and much less likely for shorter periods.
The test Jones is using here is the standard reject-support type hypothesis test, where the null hypothesis is the opposite of which he would assert to be true
H0: The rate of warming since 1998 is zero. H1: The rate of warming since 1998 is greater than zero.
Over the period concerned, the likelihood of the observations under the null hypothesis p > 0.05, which is why Prof. Jones correctly said that there had not been statistically significant warming since 1998.
However, for a skeptic to use this test to support their view that there were no global warming would not be a good idea as they are arguing FOR the null hypothesis, and reject-support hypothesis testing is biased in favour of the null hypothesis. We start off by assuming that H0 is true and only proceed with H1 if H0 is inconsistent with the observations.
What a climate skeptic should do is to perform an accept-support test, so we fix a significance level and then see if we have sufficient observations for the power of the test to be sufficient to be confident of rejecting the null hypothesis if it were actually false. Sadly computing statistical power is rather tricky (which is presumably why reject-support testing is more popular). It turns out that in this case, the test doesn't have sufficient statistical power. Combining the two hypothesis tests we find that the observations don't rule out the possibility that it hasn't warmed, nor do they rule out the possibility that it has continued to rise at the original rate (which is easily seen by looking at the confidence interval for the trend without all this hassle).
Note that Prof. Jones suggests that the likelihood of being able to find statistically significant warming depends on the length of the timescale on which you look, which suggests that he does understand the idea of the power of a test.
Hopefully this example illustrates that you can take H0 to be the thing that you want to be true, but it is so much more complicated that it is worth avoiding if you can. It is also a nice example of how the general public doesn't really understand statistical significance.