In general: let $S \in Syl_p(G)$ and $H \leq G$, with $N_G(S) \subseteq H$, then $N_G(H)=H$. Moreover, $[G:H]\equiv 1$ mod $p$.
Proof for the first part it suffices to show that $N_G(H) \subseteq H$: observe that $S \in Syl_p(H)$ and take a $g \in N_G(H)$. Then $S^g \subseteq H$ and hence $S^g=S^h$ for some $h \in H$. That means $gh^{-1}\in N_G(S)$, so $g \in hN_G(S) \subseteq H$.
For the second part we use the fact that in general for a $p$-subgroup $P$ of $G$, it holds that $[G:P]\equiv[N_G(P):P]$ mod $p$ (this can be shown by letting $G$ act by right multiplication on the right cosets of $P$). Further, since $N_G(S) \subseteq H$, $N_H(S)=N_G(S) \cap H = N_G(S)$, so the number of Sylow $p$-subgroups of $G$ and $H$ are equal. But $[G:S]=[G:H][H:N_G(S)][N_G(S):S]$ and taking the equation mod $p$ and using the fact that the number of Sylow $p$-subgroups $\equiv 1$ mod $p$ yields the required result.
To get a subgroup of order $12$, you might notice this is half of $24 = |S_4|$. You may also remember that elements of $S_4$ are either odd or even (and there's half of each). This might lead you to ask whether the set of all odd permutations, or the set of all even permutations, is a subgroup. If one of these is a subgroup, then we win. Of course, if you remember the definition of an even permutation, you'll find that the even elements do form a subgroup. This is $A_4$.
As for subgroups of order $8$, you might first realize that once you've found one subgroup of order $8$, you're done. This is a Sylow 2-subgroup, and so every subgroup of order $8$ is conjugate (thus isomorphic). So you might start thinking about what groups have order $8$ (there's only so many) and try to find one which is a subgroup of $S_4$.
After some thought, you'll probably land on the dihedral group $D_8$, which measures the symmetries of a square. Since a square has $4$ vertices, this feels plausible as something to do with $S_4$.
Since $D_8$ is defined geometrically, let's try to reason about it geometrically and show that it really is a subgroup of $S_4$, and then we'll be done.
Notice every element of $D_8$ defines a permutation of the vertices. For instance, the "rotate clockwise" action is the element $(4, 1, 2, 3)$ and the "flip along a vertical axis" element is $(4, 3)(1, 2)$. So then the permutations which arise as symmetries of this square form a subgroup isomorphic to $D_8$.
As an aside, this shows why these groups are all conjugate! Notice the only difference between the subgroups you get is the initial labeling of the vertices. So by relabeling the vertices (which corresponds to conjugation), you can move from one subgroup to another. You might want to think of this as a kind of "change of basis" for these symmetries.
I hope this helps ^_^
Best Answer
It is true if you assume $p > 2$. Here is a sketch for the proof.
A $p$-Sylow of $S_n$ can be constructed as a wreath product $P = C_p \wr \cdots \wr C_p$ of cyclic groups of order $p$. Here the number $C_p$ involved is $k$, where $p^k$ is the largest power of $p$ with $p^k \leq n$.
Consider a wreath product $X \wr Y \leq S_{ab}$ with $X \leq S_a$, $Y \leq S_b$. If $Y \trianglelefteq Z$, then $X \wr Y \trianglelefteq X \wr Z$.
Thus because $C_p$ is not self-normalizing in $S_p$ for $p > 2$, conclude that a $p$-Sylow of $S_n$ is not self-normalizing for $p > 2$.
In the case where $p = 2$, you can prove that a $2$-Sylow of $S_n$ is self-normalizing. See for example here.
The idea is that you can first show that the normalizer of $P = (C_2 \wr \cdots \wr C_2) \wr C_2$ must normalize the base of the wreath product. Then $N_G(P) = X \wr C_2$ for some $X$ normalizing $(C_2 \wr \cdots \wr C_2)$, so by induction $N_G(P) = P$.