I want to start out by giving two examples:
1) Graham's problem is to decide whether a given edge-coloring (with two colors) of the complete graph on vertices $\lbrace-1,+1\rbrace^n$ contains a planar $K_4$ colored with just one color. Graham's result is that such a $K_4$ exists provided $n$ is large enough, larger than some integer $N$.
Recently, I learned that the integer $N$ determined by Graham's problem is known to lie between $13$ and $F^{7}(12)$, which is called Graham's number, a number which beats any imagination and according to Wikipedia is practically incomprehensible. Some consider it to be the largest number which was ever used in a serious mathematical argument.
One way of looking at this is the following: Graham's problem of deciding whether for given $n$ and given coloring such a planar $K_4$ exists takes constant time. Indeed, if $n$ is small you have to look, if $n$ is large you are done. However, this takes polynomial time (check all four-tuples of vertices) for all practical purposes (assuming that $N$ is close to its upper bound).
I am sure someone can now cook up an algorithm which needs exponential time for all practical purposes but constant or polynomial time in general.
2) I also learned that the best proof of Szemerédi's regularity lemma yields a bound on a certain integer $n(\varepsilon)$ which is the $\log(1/\varepsilon^5)$́-iteration of the exponential function applied to $1$. This bound seems ridiculous in the sense that it does not even allow for interesting applications of this result (say with $\varepsilon=10^{-6}$) to networks like the internet, neural networks or even anything practically thinkable. At this point, this is only an upper bound, but Timothy Gowers showed that $\log(1/\varepsilon)$-iterations are necessary.
Again, it seems that one could cook up reasonable algorithmic problems which have solutions which are polynomial time but practically useless. Maybe one can do better in concrete cases, but this then needs additional input.
Coming closer to the question, what if finally $P=NP$ holds, but the proof involves something like the existence of a solution to Graham's problem or an application of Szemerédi's regularity lemma, so that finally the bounds of the polynomial time algorithm are for some reason so poor that nobody even wants to construct it explicitly. Maybe the bounds are exponential for all practical purposes, but still polynomial.
I often heard the argument that once a polynomial solution for a reasonable problem is found, further research has also produced practicable polynomial time algorithms. At least for Graham's problem this seems to fail miserably so far.
Question: Is there any theoretical evidence for this?
Now, maybe a bit provocative:
Question: Why do we think that $P\neq NP$ is necessarily important?
I know that $P$ vs. $NP$ is important for theoretical and conceptional reasons, but what if finally $P=NP$ holds but no effective proof can be found. I guess this wouldn't change much.
EDIT: Just to be clear: I do not dismiss complexity theory at all and I can appreciate theoretical results, even if they are of no practical use.
Best Answer
The $P \ne NP$ problem is the best way we know to formulate the belief (which was expressed even before the problem was formally stated) that certain specific algorithmic problems (such as finding a Hamiltonian cycle in a graph) requires exponential number of steps as a function of their description. The formulation is based on the important notion of a nondeterministic algorithm. The conjecture that P is not equal to NP is the basis of a mathematical theory of complexity which is remarkably beautiful. It is likely that a proof for the conjecture will lead to better understanding of the limitations of computation which will go beyond the conjecture itself. (Unfortunately we are very far from such a proof.) A counterexample (which is not expected) may lead to a major change of our reality and not just our understanding of it. (I suppose this is one of the main reasons in believing the conjecture.) Your concern is that maybe, because of the asymptotic nature of the conjecture, we can question its real relevance. Namely problems we regard as intractable may still be even if P=NP because of huge constants in the polynomial involved, or perhaps even if NP not equal P there can be efficient algorithm to relevant cases of intractable problems. These are serious concerns that are often raised and sometimes there are efforts to study them scientifically.
Overall, it looks that these asymptotic concerns are secondary compared to the problem itself. Moreover, there are many examples that the asymptotic behavior and practical behavior are similar beyond what the theory dictates (but a few examples also in the other direction). There are other related concerns regarding the relevance for the $P \ne NP$ problem. Is the worst case analysis relevant to practical problems? Is the hardness apply when we are interested in approximate solution rather than in exact solution? Both these questions (like the asymptotic issue) can be regarded as secondary in importance compared to the $NP \ne P$ problem itself but otherwise very important. Indeed both hardness of approximation and average case analysis are very central research topics.
(A side remark, there is something unnatural about the way Graham's numbers are used in the question. You are dealing with a decision problem where the answer is always yes when n is sufficiently large. And there is a simple polynomial time algorithm as a function of n to decide the answer for a every given n. So the relevance of this particular example (which is interesting) to the question asked about NP complete problems (which is also interesting) is not so strong.)
Finally, regarding relevance. Andreas raised the interesting possibility that NP=P but the constant involved are so huge that the polynomial algorithm for solving NP complete problems is utterly not practical. A related interesting possibility is that there even exists a practical polynomial time algorithm for NP complete problems but that finding this algorithm itself is computationally intractable. As I said, both these possibilities are considered unplausible. Even if true they will not harm the relevance of the NP=P problem in making the central distinction between intractable and tractable problems. In order to make these concerns interesting one should come up with a theoretical framework to study them, and then study them fruitfully. (As I mentioned above, this was done for related concerns regarding approximation and regarding average case behavior.)