Consider the effect of conjugation on cycles. Given the cycle $c=(a_1~a_2~\cdots~a_k)$ and $\sigma\in S_n$, the permutation $\sigma c\sigma^{-1}$ maps $\sigma(a_i)$ to $\sigma(a_{i+1})$ (cycle the index modulo $k$), and all other arguments are fixed by the permutation. Hence
$$c^\sigma=\sigma(a_1~a_2~\cdots~a_k)\sigma^{-1}=(\sigma (a_1)~\sigma(a_2)~\cdots~\sigma(a_k)).$$
If $c_1,\cdots,c_l$ are disjoint cycles, $\sigma(c_1\cdots c_l)\sigma^{-1}=(\sigma c_1\sigma^{-1})\cdots(\sigma c_l\sigma^{-1})$, hence conjugation preserves not only cycles and cycle lengths, but cycle types as well. Moreover, conjugation acts transitively on permutations of a given cycle type. Suppose $\pi=c_1\cdots c_l$ and $\rho=d_1\cdots d_l$ are two permutations and their disjoint cycle decompositions such that each $c_i$ and $d_i$ have the same length.
We construct a $\sigma$ in pieces: if $c_i=(a_1~a_2~\cdots~a_k)$ and $d_i=(b_1~b_2~\cdots~b_k)$ (note $k$ can vary with $i$), let $\sigma$ send $a_j\mapsto b_j$ for $j=1,2,\cdots,k$. After doing this for each cycle (even the ones of length one, which are normally left out of written representations for efficiency), we will have that $\sigma \pi \sigma^{-1}=\rho$, by having equivalent cycle decompositions. This $\sigma$ is not unique; we could rewrite $c_i$ cycled by $1$ in index as $(a_k~a_1~\cdots~a_{k-1})$ and $\sigma$ would turn out different.
Cycle types are in bijection with integer partitions. For $n=5$, we have
$$\begin{array}{c | c}\lambda\vdash5 & \sigma \\ \hline (5) & (1~2~3~4~5) \\
(4,1) & (1~2~3~4)(5) \\
(3,2) & (1~2~3)(4~5) \\
(3,1,1) & (1~2~3)(4)(5) \\
(2,2,1) & (1~2)(3~4)(5) \\
(2,1,1,1) & (1~2)(3)(4)(5) \\
(1,1,1,1,1) & (1)(2)(3)(4)(5) \end{array}$$
On the left are integer partitions of $5$, and on the right are easily constructed representatives (in cycle notation) of the conjugacy class - cycle type - associated to the partition.
You have a normal subgroup of order $5$. Your calculations are already sufficient to show that the elements of this group other than the identity don't commute with $\tau$, or indeed any of its powers. So $\tau, \tau^2, \tau^3$ are not in the centre. $1=\tau^0=\tau^4$ is of course in the centre.
Suppose we have an element $\rho$ which is in the centre, and therefore does commute with $\tau$. You have shown that $\rho = \sigma^k\tau^m$ so that $$\rho \tau =\sigma^k\tau^{m+1}$$
$$\tau\rho=\tau\sigma^k\tau^m=\sigma^{2k}\tau^{m+1}$$
For these to be equal you need $k=0$ - so the only possible central elements other than $1$ are the powers of $\tau$ which have already been excluded.
Best Answer
$o(\sigma) = 2$, $o(\tau) = 5$ and $\sigma\tau\sigma = \tau^{-1}$. Thus $\langle\sigma,\tau\rangle = \langle\sigma,\tau\mid\sigma^2 = \tau^5 = 1,\sigma\tau\sigma = \tau^{-1}\rangle\cong D_{10}$, the dihedral group of order $10$. Therefore, the group is $\{1,\tau,\tau^2,\tau^3,\tau^4,\sigma,\sigma\tau,\sigma\tau^2,\sigma\tau^3,\sigma\tau^4\}$.