It is proved in [Kaminker, J., Proc. Amer. Math. Soc. 41 (1973), 305–308] that the tangent sphere bundle of a closed smooth H-manifold is (unstably) fibre homotopy trivial. On other other hand, surgery theory allows to construct H-manifolds with non-trivial rational Pontrjagin classes, see e.g. [Victor Belfi, Pacific J. Math. vol 36, Number 3 (1971), 615-621] but this is a standard surgery-theoretic argument.
Finally, in [Milnor-Spanier, Pacific J. Math. vol 10, Number 2 (1960), 585-590] it is shown that the tangent bundle to $S^n$ is fiber homotopy trivial if and only if $n=1,3,7$,
which are exactly the parallelizable spheres (by Bott-Milnor).
Thus the above is a complete answer to your question. This phenomenon you ask for never occus for spheres but occurs for lots of H-manifolds.
The definition that Martin mentions comes close to the definition of a tangent vector which I learnt as an undergraduate.
"Definition: A geometric tangent vector is an equivalence class of germs of smooth maps $\mathbb{R} \to M$ at $0$, where two germs are equivalent
if their first order jets at $0$ agree."
More generally, we could say that for any test manifold $U$, a smooth map $U \to TM$ is an equivalence class of germs of maps $U \times \mathbb{R} \to M$
around $U \times \{0\}$, where two germs are equivalent if their first order jets in $\mathbb{R}$-direction are equal.
To make sense out of the equivalence relation, we need, horribile dictu, a small computation in local coordinates with the chain rule.
It takes some work to show that the functor $U \mapsto \{ \text{germs of maps} U \times \mathbb{R} \to M\}$ is representable by a vector bundle $TM$. Likewise, one needs some local computations and arguments with charts for that.
The problem of course is that there does not exist a manifold $\ast[\epsilon]$ such that maps $\ast[\epsilon] \to M$ correspond to tangent
vectors of $M$. So the functor $M \mapsto TM$ does not have an adjoint. It might be a good idea to use supermanifolds instead. Remeber that there is
a $(0,1)$-dimensional supermanifold with function algebra $\mathbb{R} [\epsilon]/ \epsilon^2$. However, I am not expert enough to tell you something more
specific about that.
I am deeply convinced that it is indeed necessary to invoke coordinates in some form and that the ''nasty'' computations with the chain rule and the fundamental theorem of calculus are absolutely crucial. If we wish to define a structure in which we can talk about derivatives of maps between manifolds, the most basic properties of the derivative of maps between euclidean spaces ought to play a role. One can of course use mathematical high-tech weaponry of all sorts to hide the local charts carefully. I will not try to do so. Anyway, if you wish to create the tangent bundle not as an object in a functor category, but as a manifold, you need to check that it is a manifold, i.e. you need charts at some point.
The construction of the tangent bundle immediately generalizes to a construction of the frame bundle $Fr(M)$ instead. Because it is so funny, we immediately define the higher frame bundles $Fr^k(M)$.
Definition: a $k$-frame is an equivalence class of germs of local diffeomorphisms $\mathbb{R}^n \to M$, where two germs are equivalent if their $k$-th order
jets coincide. More generally, for a manifold $U$, let $Fr^k M (U)$ be the set of germs of smooth maps $U \times \mathbb{R}^n \to M$, which are diffeomorphic in $\mathbb{R}^n $-direction.
Equivalence is defined by equality of $k$-jets."
Again, the chain rule is needed to justify the definition of the equivalence relation. Now we define the jet groups $J^k (n)$. Consider the power series ring $R:=\mathbb{R}[[x_1,\ldots ,x_n]]$ and let $G$ be the group of ring automorphisms. By the action on Taylor expansions, we get a homomorphism $Diff(\mathbb{R};0) \to G$. Let $I \subset R$ be the unique maximal ideal. Clearly, the group $G$ preserves the filtration $R \supset I \supset I^2 \supset \ldots$. Hence it acts on the finite-dimensional vector spaces $R / I^{k+1}$. The $k$th jet group $J^k (n)$ is the quotient of $G$ by the kernel of its
action on $R / I^{k+1}$. This is easily seen to be a linear algebraic group. There are obvious maps $J^{k+1} (n)\to J^k (n)$ and an equally obvious isomorphism
$J^1 (n) \cong GL_n (\mathbb{R})$. The extensions have nilpotent kernel and they are split (take derivatives of polynomial diffeomorphisms),
but the splitting is not natural (it is natural with respect to linear maps $\mathbb{R}^n \to \mathbb{R}^n$, but not more generally).
It is easy to see that $U \mapsto Fr^k (M)(U)$ is a torsor over the group $map (U; J^k (n))$ (compose with germs of diffeomorphisms).
The functor $U \mapsto Fr^k (M)(U)$ is representable by a $J^k (n)$-principal bundle. This is done in charts and then by gluing. Again, the functor $Fr^k (M)$
certainly does not have an adjoint.
The second link that Martin gave alludes to an axiomatic characterization of the tangent bundle (and the higher frame bundles as well).
It does not quite fit to the functor-of-points ideology, but I think still very useful, because it axiomatizes the gluing and hence I say some words about it as well.
Let $C_n$ be the category of smooth manifolds of dimension $n$ as objects and local diffeomorphisms as morphisms.
A natural fibre bundle on $C_n$ is the following set of data: For each $M \in \operatorname{Ob} (C_n)$, there is a smooth fibre bundle
$F_M \to M$ and for each map $f:M \to N$ in $C_n$, there is a bundle map $F_M \to F_N$; plus some obvious functoriality
properties. There is also a canonical notion of a morphism of natural fibre bundles. Let $F(0)$ be the fibre of
$F_{\mathbb{R}^n} \to \mathbb{R}^n$ at $0$. There is an action of $Diff(\mathbb{R}^n;0)$ (diffeomorphisms fixing the origin) on $F(0)$.
Here is an axiomatic characterization of the tangent bundle:
''THEOREM: The tangent bundle is the unique natural fibre bundle on $C_n$, such that $F(0)=\mathbb{R}^n$, with the action of
$Diff(\mathbb{R}^n,0)$ given by the first derivative $Diff(\mathbb{R}^n,0) \to GL_n (\mathbb{R})$.''
It is clear that the tangent bundle satisfies these properties, and here is a sketch of uniqueness, in other words, that the tangent
bundle is determined by these properties. Let $F_M \to M$ be a natural bundle. I
show that it is determined, up to natural isomorphism, by the action on $F(0)$.
First restrict to $M = V = \mathbb{R}^n$. We know that there is an action of the diffeomorphism group $Diff(V)$ on $F_V$, covering the action
on $V$. Using the translations $T_x (v):= v +x$, we get a trivialization $F_V \times V \times F(0)$. Let $x \in V$, $f \in Diff(V)$. The action
$f:F(x) \to F(f(x))$ is given in this trivialization by the action of $T_{-f(x)} \circ f \circ T_x \in Diff(V;0)$ on $F(0)$,
which is known by assumption.
This argument shows that $F(0)$ plus the action determines $F_V$ completely.
By restriction to open subsets and naturality, the restriction of
$F$ to the full subcategory $O_n$ of manifolds diffeomorphic to some open subset of $V$ is uniquely determined.
Let $M$ be an arbitrary manifold and let $U(i)_{i \in I}$ be the maximal atlas. It can be written
as a diagram $U : J \to O_n$, for some indexing category $J$ (take the intersection of the charts into account). The diagram has a
colimit in $C_n$, namely $M$ (of course not all colimits in $C_n$ exist). Likewise, we get a diagram $j \mapsto F_{U(j)}$, whose colimit is $F$.
Thus a natural bundle is completely determined, up to natural isomorphism, by the action of $Diff (V,=)$ on the fibre $F(0)$.
The existence question for general natural fibre bundles is a bit subtler, for several reasons. There is a theorem by
Palais and Terng asserting that the action on $F(0)$ automatically has finite order (it factor through $j^k (n)$ for some $k$).
Also, you have to take into account the topology of the diffeomorphism group. But for the tangent bundle,
these subtleties do not
arise: it is known what the tangent bundle of an open subset of $\mathbb{R}^n$ has to be and how diffeomorphism act on that.
The colimit procedure then produces the tangent bundle. The higher frame bundles can be constructed in a similar way.
Best Answer
Ryan Budney's comment pretty much killed the question, but anyway...
Let $X_{m,n} = S^{2m}\times S^{2n+1}$, with $m\le n$ (strictly) positive integers.
Lemma: $X=X_{m,n}$ is parallelisable.
Proof: this follows from playing around with vector bundles, the key facts being that $TX = \pi^*(TS^{2m}) \oplus \pi^*(TS^{2n+1})$ and that trivial bundles are natural. More precisely, the second factor of $X$ has Euler number 0, so one can split off a trivial 1-bundle from its tangent bundle, pull it back through the projection and see it as the pull-back of a trivial bundle over the first factor. So $TX = \pi^*(TS^{2m}\oplus \varepsilon) \oplus V$, and the first summand is trivial. Now one can split off a trivial bundle of rank 2 from the first factor and use it to trivialise the second factor.
Lemma: $X$ is not an $H$-space.
Proof: the cohomology ring (say with $\mathbb{F}_2$ coefficients) of an $H$-space is a finite-dimensional commutative Hopf algebra, therefore it's generated in odd dimensions. But, in our case, $H^*(X)$ has $H^{2m}(X)$ as the first nontrivial group.
This gives a nice family of simply connected counterexamples to your question, the smallest of which is $S^2\times S^3$. Notice how, actually, there is no simply connected counterexample in dimension one, two, three (thanks to Perelman) and four (e.g. because the Euler number of a simply connected 4-manifold is strictly positive).