Proof of (*)
Adding the four finite sums,
$$\sum_{k=1}^{n}H_{k}=(n+1)H_{n}-n,$$
$$\sum_{k=1}^{n}\ln{k}=\ln{n!},$$
$$\sum_{k=1}^{n}\gamma=\gamma\,n,$$
$$\sum_{k=1}^{n}\frac{1}{2k}=\frac12H_{n},$$
gives us a representation of the $n$-th partial sum for the infinite series. Writing the infinite series as the limit of partial sums, we get:
$$\begin{align}
S
&=\sum_{k=1}^{\infty}\left(H_{k}-\ln{n}-\gamma-\frac{1}{2k}\right)\\
&=\lim_{n\to\infty}\sum_{k=1}^{n}\left(H_{k}-\ln{n}-\gamma-\frac{1}{2k}\right)\\
&=\lim_{n\to\infty}\left((n+1)H_{n}-n-\ln{n!}-\gamma\,n-\frac12H_{n}\right)\\
&=\lim_{n\to\infty}\left(\left(n+\frac12\right)H_{n}-(1+\gamma)n-\ln{n!}\right).
\end{align}$$
Use Stirling's approximation for the factorial to obtain an asymptotic formula for the log-factorial term in the series:
$$n!\sim\sqrt{2\pi n}\left(\frac{n}{e}\right)^n\\
\implies \ln{n!}\sim\ln{\left(\sqrt{2\pi n}\left(\frac{n}{e}\right)^n\right)}=\left(n+\frac12\right)\ln{n}-n+\frac12\ln{(2\pi)}.$$
Then,
$$\begin{align}
S
&=\lim_{n\to\infty}\left(\left(n+\frac12\right)H_{n}-(1+\gamma)n-\ln{n!}\right)\\
&=\lim_{n\to\infty}\left(\left(n+\frac12\right)H_{n}-(1+\gamma)n-\left(n+\frac12\right)\ln{n}+n-\frac12\ln{(2\pi)}\right)\\
&=\lim_{n\to\infty}\left(\left(n+\frac12\right)H_{n}-\gamma\,n-\left(n+\frac12\right)\ln{n}\right)-\frac12\ln{(2\pi)}\\
&=\lim_{n\to\infty}\left(n\left(H_{n}-\gamma-\ln{n}\right)+\frac12\left(H_{n}-\ln{n}\right)\right)-\frac12\ln{(2\pi)}\\
&=\lim_{n\to\infty}n\left(H_{n}-\gamma-\ln{n}\right)+\frac12\lim_{n\to\infty}\left(H_{n}-\ln{n}\right)-\frac12\ln{(2\pi)}\\
&=\lim_{n\to\infty}n\left(H_{n}-\gamma-\ln{n}\right)+\frac12\gamma-\frac12\ln{(2\pi)}\\
&=\frac12+\frac12\gamma-\frac12\ln{(2\pi)}.~~~\blacksquare
\end{align}$$
Appendix:
Using the asymptotic series for the digamma function given by Eq.16 on this Wolfram Mathworld page,
$$\begin{align}
\lim_{n\to\infty}n\left(H_{n}-\gamma-\ln{n}\right)
&=\lim_{n\to\infty}n\left(\Psi{(n+1)}-\ln{n}\right)\\
&=\lim_{n\to\infty}n\left(\frac{1}{2n}-\sum_{\ell=1}^{\infty}\frac{B_{2\ell}}{2\ell n^{2\ell}}\right)\\
&=\frac12-\lim_{n\to\infty}\sum_{\ell=1}^{\infty}\frac{B_{2\ell}}{2\ell n^{2\ell-1}}\\
&=\frac12.
\end{align}$$
ORIGINAL ANSWER
To put it succinctly.
\begin{align*}
\lim_{N\rightarrow \infty} \sum_{n=1}^{N-1}{\zeta \left( 2\left( N-n \right) \right) \frac{\left( -1 \right) ^n}{\left( 2n-1 \right) !}}
=&\lim_{N\rightarrow \infty} \sum_{n=1}^{N-1}{\sum_{k=1}^{\infty}{\frac{k^{2n}}{k^{2N}}}\frac{\left( -1 \right) ^n}{\left( 2n-1 \right) !}}
\\
=&\lim_{N\rightarrow \infty} \sum_{k=1}^{\infty}{\frac{1}{k^{2N-1}}}\sum_{n=1}^{N-1}{\frac{\left( -1 \right) ^n}{\left( 2n-1 \right) !}}k^{2n-1}
\\
=&\lim_{N\rightarrow \infty} \sum_{k=1}^{\infty}{\frac{1}{k^{2N-1}}}\sum_{n=1}^{\infty}{\frac{\left( -1 \right) ^n}{\left( 2n-1 \right) !}}k^{2n-1}
\\
=&-\lim_{N\rightarrow \infty} \sum_{k=1}^{\infty}{\frac{\sin \left( k \right)}{k^{2N-1}}}=-\sin \left( 1 \right)
\end{align*}
If you found this a bit too handwavy, I could provide the solution in greater detail.
EDIT
I shall provide a more rigorous proof here.
To start with, we rewrite the sum in terms of the definition of riemann zeta function. So we have
$$
\sum_{n=1}^{N-1}{\zeta \left( 2\left( N-n \right) \right) \frac{\left( -1 \right) ^n}{\left( 2n-1 \right) !}}=\sum_{n=1}^{N-1}{\sum_{k=1}^{\infty}{\frac{k^{2n}}{k^{2N}}}\frac{\left( -1 \right) ^n}{\left( 2n-1 \right) !}}
$$
We can interchange the sums if each terms in the finite sum converges, which is obvious in this case. Therefore
$$
\sum_{n=1}^{N-1}{\sum_{k=1}^{\infty}{\frac{k^{2n}}{k^{2N}}}\frac{\left( -1 \right) ^n}{\left( 2n-1 \right) !}}=\sum_{k=1}^{\infty}{\frac{1}{k^{2N-1}}}\sum_{n=1}^{N-1}{\frac{\left( -1 \right) ^n}{\left( 2n-1 \right) !}}k^{2n-1}
$$
Now we have to proof that
$$
\lim_{N\rightarrow \infty} \sum_{k=1}^{\infty}{\frac{1}{k^{2N-1}}}\sum_{n=N}^{\infty}{\frac{\left( -1 \right) ^n}{\left( 2n-1 \right) !}}k^{2n-1}=0
$$
Since we have
$$
\sum_{n=1}^{\infty}{\frac{\left( -1 \right) ^n}{\left( 2n-1 \right) !}}k^{2n-1}=\sin \left( k \right)
$$
Which converges, by something Cauchy (Cannot remember the exact name)
$\forall \varepsilon >0, \exists N\in \mathbb{N}\,\,\mathrm{s}.\mathrm{t}.$
\begin{align*}
&\sum_{n=N}^{\infty}{\frac{\left( -1 \right) ^n}{\left( 2n-1 \right) !}}k^{2n-1}<\frac{\varepsilon}{\zeta \left( 2 \right)}<\frac{\varepsilon}{\zeta \left( 2N-1 \right)}
\\
\Rightarrow &\sum_{k=1}^{\infty}{\frac{1}{k^{2N-1}}}\sum_{n=N}^{\infty}{\frac{\left( -1 \right) ^n}{\left( 2n-1 \right) !}}k^{2n-1}<\sum_{k=1}^{\infty}{\frac{1}{k^{2N-1}}}\frac{\varepsilon}{\zeta \left( 2N-1 \right)}=\varepsilon
\end{align*}
Which proves the claim. Hence
\begin{align*}
&\lim_{N\rightarrow \infty} \sum_{k=1}^{\infty}{\frac{1}{k^{2N-1}}}\sum_{n=1}^{N-1}{\frac{\left( -1 \right) ^n}{\left( 2n-1 \right) !}}k^{2n-1}
\\
=&\lim_{N\rightarrow \infty} \sum_{k=1}^{\infty}{\frac{1}{k^{2N-1}}}\sum_{n=1}^{N-1}{\frac{\left( -1 \right) ^n}{\left( 2n-1 \right) !}}k^{2n-1}+0
\\
=&\lim_{N\rightarrow \infty} \sum_{k=1}^{\infty}{\frac{1}{k^{2N-1}}}\sum_{n=1}^{N-1}{\frac{\left( -1 \right) ^n}{\left( 2n-1 \right) !}}k^{2n-1}+\lim_{N\rightarrow \infty} \sum_{k=1}^{\infty}{\frac{1}{k^{2N-1}}}\sum_{n=N}^{\infty}{\frac{\left( -1 \right) ^n}{\left( 2n-1 \right) !}}k^{2n-1}
\\
=&\lim_{N\rightarrow \infty} \left( \sum_{k=1}^{\infty}{\frac{1}{k^{2N-1}}}\sum_{n=1}^{N-1}{\frac{\left( -1 \right) ^n}{\left( 2n-1 \right) !}}k^{2n-1}+\sum_{k=1}^{\infty}{\frac{1}{k^{2N-1}}}\sum_{n=N}^{\infty}{\frac{\left( -1 \right) ^n}{\left( 2n-1 \right) !}}k^{2n-1} \right)
\\
=&\lim_{N\rightarrow \infty} \sum_{k=1}^{\infty}{\frac{1}{k^{2N-1}}}\sum_{n=1}^{\infty}{\frac{\left( -1 \right) ^n}{\left( 2n-1 \right) !}}k^{2n-1}
\\
=&-\lim_{N\rightarrow \infty} \sum_{k=1}^{\infty}{\frac{\sin \left( k \right)}{k^{2N-1}}}
\end{align*}
Now, notice that
$$
\sum_{k=1}^{\infty}{\left| \frac{\sin \left( k \right)}{k^{2N-1}} \right|}\leqslant \sum_{k=1}^{\infty}{\frac{1}{k^{2N-1}}}=\zeta \left( 2N-1 \right)
$$
Meaning the sum converge absolutely, implying the sum converges. So we only need to prove
$$
\lim_{N\rightarrow \infty} \left| \sum_{k=2}^{\infty}{\frac{\sin \left( k \right)}{k^{2N-1}}} \right|=\lim_{N\rightarrow \infty} \left| \sin \left( 1 \right) -\sum_{k=1}^{\infty}{\frac{\sin \left( k \right)}{k^{2N-1}}} \right|=0
$$
The gist to prove this is along these lines
\begin{align*}
&\forall \varepsilon >0, \exists N\in \mathbb{N} \,\, \mathrm{s}.\mathrm{t}.
\\
&\left| \sum_{k=2}^{\infty}{\frac{\sin \left( k \right)}{k^{2N-1}}} \right|\leqslant \left| \sum_{k=2}^N{\frac{\sin \left( k \right)}{k^{2N-1}}} \right|+\left| \sum_{k=N+1}^{\infty}{\frac{\sin \left( k \right)}{k^{2N-1}}} \right|\leqslant \sum_{k=2}^N{\frac{1}{k^N}}+\sum_{k=N+1}^{\infty}{\frac{1}{k^2}}<\sum_{k=2}^N{\frac{\varepsilon}{2^{k-1}}}+\frac{\varepsilon}{2^N}=\varepsilon
\end{align*}
By utilizing that cauchy stuff (name forgotten) and the basic definition of limits.
Then, we are done.
Best Answer
For an elementary approach, you want to show that the limit is indeed analytic in $s$ (uniform limit of analytic functions) in an open subset of $D=\{s:\Re(s)>0\land s\ne1\}$. For $\Re(s)>1$ this is fairly trivial since $x^{1-s}\to0$. For $0<\Re(s)\le1$, a full asymptotic expansion makes this more obvious, but it suffices to simply bound the error between the given sum and
$$\int_0^x\frac{\mathrm dt}{t^s}=\frac{x^{1-s}}{1-s}\tag{$0<\Re(s)\le1,s\ne1$}$$
using something such as Taylor expansions.
A much more general approach is given by the Euler-Maclaurin summation formula, which states that
$$\sum_{n\le x}\frac1{n^s}=\zeta(s)+\frac1{(1-s)x^{s-1}}+\frac1{2x^s}-\frac s{12x^{s+1}}+\mathcal O(x^{-s-3})$$
For $\Re(s)>1$, every term after $\zeta(s)$ tends to zero, so we get
$$\lim_{x\to\infty}\sum_{n\le x}\frac1{n^s}=\zeta(s)$$
For $\Re(s)>0$, the $x^{-s+1}$ term needn't go to zero, so we get
$$\lim_{x\to\infty}\left[\sum_{n\le x}\frac1{n^s}-\frac1{(1-s)x^{s-1}}\right]=\zeta(s)$$
In general, by moving all terms which don't go to zero to the other side, we may get a converging limit expression for $\zeta(s)$ for $\Re(s)>-N$ for any natural $N$. It is interesting to note that this gives exacts when $s$ is a negative integer since $\sum_{n\le x}n^{-s}$ has a closed form.