Vtyjkyuk

In hypothesis testing, alternative hypothesis doesn't have to be the opposite of null hypothesis. For example, for $H_0: mu=0$, $H_a$ is allowed to be $mu>1$, or $mu=1$. My question: Why is this allowed? What if in reality, $mu=-1$ or $mu=2$, in which case if one applies, say, likelihood ratio test, one may (wrongly) conclude that $H_0$ is accepted, or $H_0$ is rejected and hence $H_a$ is accepted?

What about this proposal: $H_a$ should always be the opposite of $H_0$? That is, $H_a: H_0$ is not true. This way, we are effectively testing only a single hypothesis $H_0$, rejecting it if the p-value is below a predefined significance level, and not have to test two hypotheses at the same time that can be both wrong.

What you've identified is one of the fundamental flaws with this approach to hypothesis testing: namely, that the statistical tests you are doing do not assess the validity of the statement you are actually interested in assessing the truth of.

In this form of hypothesis testing, $H_a$ is never accepted, you can only ever reject $H_0$. This is widely misunderstood and misrepresented by users of statistical testing.

$H_a$ is, properly the complement of $H_0$ in the sample space of the distribution under the null hypothesis. One-sided tests, should therefore properly have $H_0: mu ge c$ (for some number $c$), with $H_a: mu < c$ (or vice versa: $H_0: mu le c$, with $H_a: mu > c$), for precisely the reason you allude to: if the null hypothesis in a one-sided test is specified as $H_0: mu = 0$, then a one-sided alternative hypothesis cannot express the complement of $H_0$. I (and others) therefore disagree with those who use the confusing nomenclature you describe.

See my answer here for a similar question and issue.

This points to one of the few serious problems with the conventional statistics through null hypothesis significance testing (NHST). A much more meaningful approach in this case is to totally abandon NHST, and adopt the Bayesian framework. If you have some prior information available, just incorporate it into your model through prior distribution. Unfortunately most statistics consumers are simply too indoctrinated, obsessed and entrenched with the old school of thinking. See more discussion here.

Put properly, we don't actually test if an alternative hypothesis is true. It is often described that way, but as far as basic statistics goes, that is incorrect.

We actually test whether there is, or is not, enough evidence to accept some "new"/"novel"/"not-default" hypothesis H. We do this by

1. Taking into account what we know (Bayesian style if appropriate);


2. Choosing a test we think is applicable to the data and hypothesis we are probing, and


3. Stipulating a point which will be deemed "significant".

The significance level

This last item, the "significamce level", is often a source of confusion. What we actually say is, "If the hypothesis is wrong, then how exceptional would our results be?" So, suppose we set a significance level of 0.1% (P=0.001), what we are saying is:

"If our hypothesis is wrong, we just got a 1 in 1000 result by pure chance. That's so unlikely that we conclude the hypothesis is probably correct."

So you can "draw the line" where you like - for some research such as particle physics, you'd want 2 separate (independent) experiments both with a significance level of 1 in some millions, before concluding the hypothesis is probably correct. For a rigged dice game, a 1 in 3 level might be enough to persuade you not to play that game :)

But either way it is crucial to pick the level beforehand, otherwise you're probably just make a self serving statement using 'whatever level you like".