Your answers to (a) and (b) are essentially correct; just be more demanding about your notation. For instance, explicitly specifying your limits of integration when using integration by parts, and not writing "$\int f'(s)B(t)$" when you mean "$\int_0^t f'(s)B(s)~\mathrm ds$".
Your primary question is on how to compute (not necessarily from first principles) quadratic variations for Ito processes, is it? One way is to use the related SDE representation of the process. To illustrate, note that by definition (or applying Ito's lemma, if you prefer), the Ito process $(X_t)_{t\geqslant 0}$ defined in (a) satisfies the SDE
$$
\mathrm dX_t = f(t)\mathrm dB_t\,,\quad X_0 = 0.\tag{1}
$$
And, by definition, the SDE satisfied by the quadratic variation related to this process is given by
$$
\mathrm d\langle X \rangle_t = (f(t)\mathrm dB_t)\cdot(f(t)\mathrm dB_t) = f^2(t)~\mathrm dB_t\cdot\mathrm dB_t = f^2(t)\mathrm d\langle B \rangle_t\,.
$$
This means that, as you correctly deduced, the quadratic variation process is given by**
$$
\begin{eqnarray*}
\langle X\rangle_{t} = \int\limits_{0}^{t}\mathrm d\langle X\rangle_{s} = \int\limits_{0}^{t}f^2(s)~\mathrm d\langle B\rangle_s = \int\limits_{0}^{t}f^2(s)~\mathrm ds\,.
\end{eqnarray*}
$$
A similar approach can be applied to the Ito process $(Y_t)_{t\geqslant 0}$ which, by definition, satisfies the SDE
$$
\mathrm dY_t = f^{'}(B_t)\mathrm dB_t + \frac{1}{2} f^{''}(B_t)\mathrm dt,\quad Y_0 = 0\,.
$$
So, like the $(X_t)_{t\geqslant 0}$ process, the SDE for the quadratic variation is given (by definition) as
$$
\begin{eqnarray*}
\mathrm d\langle Y\rangle_{t} &=& \left(f^{'}(B_t)\mathrm dB_t + \frac{1}{2} f^{''}(B_t)\mathrm dt\right)\cdot\left(f^{'}(B_t)\mathrm dB_t + \frac{1}{2} f^{''}(B_t)\mathrm dt\right) \newline
&=&(f^{'}(B_t)\mathrm dB_t)\cdot(f^{'}(B_t)\mathrm dB_t) + \ldots \newline
&=&\left(f^{'}(B_t)\right)^2~\mathrm d B_t\cdot\mathrm d B_t + \ldots \newline
&=&\left(f^{'}(B_t)\right)^2~\mathrm d \langle B\rangle_t,\tag{2} \newline
\end{eqnarray*}
$$
where all of the other products represented by "$\ldots$" result in processes with zero quadratic (co)variation. There is a pattern here worth noting: the quadratic variation of an Ito process is completely defined by the Wiener process term in the SDE representation of the Ito process.
Therefore, the quadratic variation is
$$
\begin{eqnarray*}
\langle Y\rangle_{t} = \int\limits_{0}^{t}\mathrm d\langle Y\rangle_{s} = \int\limits_{0}^{t}\left(f^{'}(B_s)\right)^2~\mathrm d\langle B\rangle_s = \int\limits_{0}^{t}\left(f^{'}(B_s)\right)^2~\mathrm ds\,.\tag{3}
\end{eqnarray*}
$$
Finally, use either $(2)$ or $(3)$ to deduce when $(Y_t)_{t\geqslant 0}$ is Brownian motion.
** ps: And yes, this shows that $\vert f(t)\vert = 1$ (for all $t$) makes $X_t$ Brownian motion, a fact that is obvious from $(1)$.
Best Answer
I think, I have an easy way to proof Formular I. Using the the law of total expectation:
$E <X>_n = \sum_{i=1}^n [E(E(X_i^2|F_{i-1}) - E(X_{i-1}^2)] = \sum_{i=1}^n [E(X_i^2) - E(X_{i-1}^2)] = E(X_n^2) - E(X_0^2)$
Finally, notice that:
$Var(X_n - X_0) = E(X_n - X_0)^2 = E(X_n^2) - 2E(X_n X_0) + E(X_0^2) = E(X_n^2) - 2E(E(X_n X_0 | F_0)) + E(X_0^2) = E(X_n^2) - 2E(X_0 E(X_n|F_0) + E(X_n^2) = E(X_n^2) - E(X_0^2)$
For Brownian Motion, using that $W_n - W_0 \sim N(0, n)$, we have:
$E(<W_{nh}>) = Var(W_{nh} - X_0) = nh \to \infty, n \to \infty$