$A*B$ can be significant in all of these scenarios. Consider $A \in \{-1, 0, 1\}$ and $B \in \{-1, 1\}$ where the underlying model is $E[Y|A,B] = A*B$. In a roughly balanced situation, with (roughly) equal sample sizes for each combination of $A \times B$, neither $A$ nor $B$ will be significant (except for the $\alpha$ fraction of the time when a true null hypothesis is rejected), but the interaction term certainly will be! Here's a numeric example:
A <- rep(c(-1,0,1), 100)
B <- rep(c(-1,1), 150)
X <- A*B
Y <- X + rnorm(300)
> summary(lm(Y~A+B+A*B))
Call:
lm(formula = Y ~ A + B + A * B)
Residuals:
Min 1Q Median 3Q Max
-3.03520 -0.59349 -0.03184 0.62857 2.49359
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -0.02083 0.05668 -0.367 0.714
A -0.03797 0.06942 -0.547 0.585
B 0.05867 0.05668 1.035 0.301
A:B 0.90789 0.06942 13.078 <2e-16 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 0.9818 on 296 degrees of freedom
Multiple R-squared: 0.3681, Adjusted R-squared: 0.3617
F-statistic: 57.47 on 3 and 296 DF, p-value: < 2.2e-16
Or, more simply:
> cor(A,Y)
[1] -0.02527534
> cor(B,Y)
[1] 0.04782935
> cor(A*B,Y)
[1] 0.6042723
It should be intuitively clear that if we can construct an example where $A$ and $B$ are both insignificant, yet the interaction is significant, we can do so for either of your other two cases.
As for likely... One could argue that in real life, apart from physics and a few other disciplines, pretty much all interaction terms are very likely to be nonzero (albeit perhaps very small), and "significance" in its statistical sense is merely a function of sample size.
Insignificant at the 10% level means that the 90%-confidence interval overlaps with zero. A significant difference at the 1% level means that the (larger!) 99% confidence intervals do not overlap. This should not be possible.
The F-test does not test the hypothesis whether two coefficients are of different size. This test tells you that large firms are not small firms. What you are probably looking for is the Wald test. Maybe that’s the problem?
Best Answer
F test is the first thing to do in regression after identifying the features and if it fails then it means that none of the predictors/features for the given degrees of freedom is significant, so stop there. Now, I get this question a lot that if we have any single significant predictor present in the model then why bother an $F$ test. The answer here is that suppose we have $100$ predictors and we do not do an $F$ test and just rely on $t$ test to see if any predictor is significant using p-value $\le 0.05$ and then proceed with those which are significant, then there is a serious problem with this approach which is, that by chance we will have $5$ out of $100$ predictors which won't be significant but will have low p value due to chance alone. This is the reason an $F$ test is important and the first criteria to fit a model.