It is always a good idea to look at the local question first. What is the maximal pro-$p$ extension of $\mathbf{Q}_p$ ? There is a vast literature on the subject, starting with Demushkin whose results are exposed in an old Bourbaki talk by Serre on the Structure de certains pro-p-groupes (d'après Demuškin), (http://www.numdam.org/item?id=SB_1962-1964__8__145_0). They turn out to be extremely interesting groups which satisfy a kind "Poincaré duality" and have a striking presentation mysteriously similar to the presentation of the fundamental group of a compact orientable surface. Now I have to go for my yoga class; I hope you can find more on the web by yourself.
Addendum. Another good idea which I record here although I'm sure you don't need to be reminded of it is that in the local situation one should first ask what happens at primes $l\neq p$. What is the maximal pro-$l$ extension of $\mathbf{Q}_p$ ? This is much easier to answer because the only possible ramification is tame. You have the maximal unramified $l$-extension, which is a $\mathbf{Z}_l$-extension, and then a totally ramified extension of group $\mathbf{Z}_l(1)$, the projective limit of the roots of $1$ of $l$-power order. As a pro-$l$ group it admits the presentation
$\langle\tau,\sigma\mid\sigma\tau\sigma^{-1}=\tau^p\rangle$.
All this can be found in Chapter 16 of Hasse's Zahlentherie and in a paper by Albert in the Annals in the late 30s.
Addendum 2. A third good idea is to look at the function field analogues, wherein you replace $\mathbf{Q}$ by a function field $k(X)$ over a finite field $k$ of characterisitc $p$, where $X$ is a smooth projective absolutely irreducible curve over $k$. There would again be two cases, according as you are looking at the maximal pro-$l$ extension for a prime $l\neq p$ or for the prime $l=p$; the former should be much easier. You can simplify the problem by replacing $k$ by an algebraic closure. We are then in the setting of Abhyankar's conjecture (1957) which was very useful to Grothendieck as a first test for his theory of schemes. In any case, the conjecture was settled by Raynaud and Harbatter in the early 90s.
Well, let me try in an elementary way. Pick any field $K$ of characteristic prime to $p$ and suppose it does not contain any $p^n$-th root of unity. Let $a\in K^\times\setminus (K^\times)^p$. Then, the extension $K(\sqrt[p^n]{a},\mu_{p^n})/K$ is normal, not abelian and its Galois group is isomorphic to $\Delta_n\ltimes\mathbb{Z}/p^n$ where $\Delta_n=\mathrm{Gal}(K_n(\zeta_{p^n})/K)$.
To see this, start by observing that it is clearly Galois (it contains all roots $\xi\;\sqrt[p^n]{a}$ of $X^{p^n}-a$, for $\xi$ running in $\mu_{p^n}$), so the point is to study its Galois group. Set $F_n=K(\zeta_{p^n})$ and let $H=\mathrm{Gal}\big(F_n(\sqrt[p^n]{a})/F_n)\big)$ and $G=\mathrm{Gal}\big(F_n(\sqrt[p^n]{a})/K(\sqrt[p^n]{a})\big)$. I claim that
(*) the order of $a$ in the quotient $F_n^\times/(F_n^\times)^{p^n}$ is
exactly $p^n$ (at least if $K/\mathbb{Q}_p$ is finite)
Admitting the claim, $H$ is a cyclic group of order $p^n$ (let $\eta$ be a generator) while $G$ is a cyclic group of order $d>1$ for some $d\mid p^{n-1}(p-1)$ (this is classic, see for instance Lemma 1 in Birch's paper in Algebraic Number Theory by Cassels-Frölich). Define a $\mathbb{Z}_p^\times$-valued character $\omega:G\to\mathbb{Z}_p^\times$ by $g(\zeta)=\xi^{\omega(g)}$ for all $\xi\in \mu_{p^n}$ and $g\in G$: this implies $g(\xi\;\sqrt[p^n]{a})=\xi^{\omega(g)}\;\sqrt[p^n]{a}$. Finally, there is a $j$ such that $\eta(\sqrt[p^n]{a})=\zeta_{p^n}^j\;\sqrt[p^n]{a}$ where $\zeta_{p^n}$ is a fixed generator of $\mu_{p^n}$. You can readily compute that for each $g\in G$,
$$
g\eta(\zeta_{p^n}\sqrt[p^n]{a})=\zeta_{p^n}^{(1+j)\omega(g)}\;\sqrt[p^n]{a}
$$
while
$$
\eta g(\zeta_{p^n}\sqrt[p^n]{a})=\zeta_{p^n}^{\omega(g)+j}\;\sqrt[p^n]{a}
$$
which shows at once that $g$ and $\eta$ do not commute unless $g=1$ and that $H\cap G=\{1\}$, namely the extension $F_n(\sqrt[p^n]{a})/K$ is not abelian with Galois group $G\ltimes H$, isomorphic to $\Delta_n\ltimes \mathbb{Z}/p^n$.
We are left with my claim (*): this is where one needs some Kummer theory. Indeed, consider the inflation-restriction sequence
$$
1\to H^1(\Delta_n,\mu_p)\to H^1(K,\mu_p)\to H^0(\Delta_n,H^1(F_n,\mu_p))\to H^2(\Delta_n,\mu_p)
$$
and observe that $\Delta_n$-cohomology of $\mu_p$ vanishes: this can be seen by computing $H^2=\hat{H}^0$ which is trivial because $\mu_p(K)=\{1\}$, and then using that the Herbrand quotient of a finite module is $1$: finally, identify the $H^1$'s with the quotients by $p$-th powers by Kummer theory to find
$$
H^0(\Delta_n,F_n^\times/(F_n/^\times)^p)\cong K^\times/(K^\times)^p.
$$
In particular, we see that $a$ does not become a $p$-th power in $F_n^\times$ and since $F_n/\mathbb{Q}_p$ is finite we know $F_n^\times/(F_n^\times)^{p^n}$ is isomorphic to finitely many copies of $\mathbb{Z}/p^n$, so not being a $p$-th power coincides with having order $p^n$.
Now, back to your situation, you simply observe that the extension you call $K_u$ is the direct limit of extensions $F_n(\sqrt[p^n]{u})$ so the Galois group $K_u/K$ is the inverse limit of $\Delta\ltimes(\mathbb{Z}/p^n)$ none of which is abelian, so it is a non-trivial semi-direct product
$$
\Delta\ltimes\mathbb{Z}_p
$$
for some finite-index subgroup $\Delta\subseteq\mathbb{Z}_p^\times$, all this provided that $u\notin (K^\times)^p$. This is certainly true if $u$ is a generator of principal units and certainly false if it is a root of unity of order prime to $p$. Of course, if $u$ is a non-trivial principal unit in $(K^\times)^{p^t}\setminus (K^\times)^{p^{t+1}}$, say $u=v^t$ you repeat the above argument with $v$ instead of $u$ (may be, getting some headache due to index-shifting), and similarly if $K$ contains some $p^k$-th root of unity.
As for Khare-Wintenberger's argument, as Kevin observed, they are simply restating the above elementary computation expressing it in terms of cohomology classes but I guess nothing new appears (observe we used Kummer isomorphism in proving (*) and that, is all is needed).
Best Answer
Let $L/K$ be a finite abelian extension of local fields. Although, there is no generic form for the image of the norm map, $N^{L}_K$, in practice one can follow the following procedure to determine its image.
Choose a uniformizer $\pi_L$ in $\mathcal{O}_L.$ Then $L^{\times}$ is equal to the group generated by $\pi_L$ and $\mathcal{O}_L^{\times}.$ It follows that $N^{L}_K L^{\times} = \langle N^{L}_K(\pi_L) \rangle N^{L}_K \mathcal{O}_L^{\times},$ Hence to determine $N^{L}_K L^{\times}$ it is enough to establish the image of $\mathcal{O}_L^{\times}$ under the norm mapping.
The group $N^L_K \mathcal{O}_L^{\times}$ is a subgroup of $\mathcal{O}_K^{\times}$ and by a group cohomology argument it can be shown that
$[\mathcal{O}_K^{\times}:N^{L}_K\mathcal{O}_L^{\times}] = e(L|K) = [L^{\times} : K^{\times}\mathcal{O}_L^{\times}].$
In particular, if $L/K$ is unramified, $\mathcal{O}_K^{\times} =N^{L}_K\mathcal{O}_L^{\times}$ and hence $N^{L}_K L^{\times} = \langle N^{L}_K(\pi_L) \rangle \mathcal{O}_K^{\times}.$
The norm group of a tamely ramified extension is similarly easy to deduce. Write
$$\mathcal{O}_L^{\times} = \langle \zeta_{q_L - 1} \rangle U_L$$
where $q_L$ is the residue field characteristic of $L$ and $\zeta_n$ denotes a primitive n-th root of unity. Denote the residue field of $L$ by $l$ and that of $K$ by $k,$ then
$$N^L_K(\zeta_{q_L -1}) = N^l_k(\zeta_{q_L -1})^{e(L|K)} = \zeta_{q_K -1}^{e(L|K)}.$$
As $U_K$ is a pro-$p$ group and contains the image of $U_L$ under $N^L_K,$ we have in the tamely ramified case that $U_K = N^L_K(U_L)$ else $\mathcal{O}_K^{\times}/N^{L}_K\mathcal{O}_L^{\times}$ would contain an element of $p$-power order contradicting the equality
$$[\mathcal{O}_K^{\times}:N^{L}_K\mathcal{O}_L^{\times}] = e(L|K)$$
and the fact that $L/K$ was assumed tamely ramfied. It follows in the tamely ramified case that $$N^{L}_K L^{\times} = \langle N^{L}_K(\pi_L), \zeta_{q_K -1}^{e(L|K)}\rangle U_K$$
The case of wild ramification is more difficult. But two facts are helpful. First, in the case $K$ is a p-adic field $U_K^{ap^n} \supset 1 +\mathcal{M}_K^{2ne(k|\mathbb{Q}_p) + 1}$ where $a$ and $p$ are relatively prime. Hence, it is enough to determine the image of the norm mapping in the units of $\mathcal{O}_K \mod \mathcal{M}_K^{2ne(k|\mathbb{Q}_p) + 1},$ a finite set.
Another technique is to determine the higher ramification filtration of $Gal(L/K)^{ab}.$ In practice this can be done by examining the derivatives of the irreducible polynomial of $\pi_L.$ These groups map under the inverse of the artin map to the higher unit groups. As the domain of the inverse of the artin map is $K^{\times}/N^{L}_K L^{\times}$ their sizes reveal norm indexes within the higher unit groups. For a good exposition see Serre's book Local Fields.