@Anne has answered my question in a comment. I formalize her ideas as below.
We have $L^\infty(Y)=(L^1(Y))^*$, so
$$
\|\pi_\sharp f\|_{L^\infty(Y)} = \sup \{\langle \pi_\sharp f, g \rangle_{L^1(Y)} ; \|g\|_{L^1(Y)} = 1\}.
$$
We have the duality
$$
\langle \pi_\sharp f, g \rangle_{L^1(Y)} = \langle f, \pi^\sharp g \rangle_{L^1(X)}.
$$
So
$$
\|\pi_\sharp f\|_{L^\infty(Y)} = \sup \{\langle f, \pi^\sharp g \rangle_{L^1(X)} ; \|g\|_{L^1(Y)} = 1\}.
$$
Notice that
$$
\begin{align}
\langle f, \pi^\sharp g \rangle_{L^1(X)} &\le \|f\|_{C(X)} \langle \pmb 1, \pi^\sharp g \rangle_{L^1(X)} \\
&= \|f\|_{C(X)} \int_X \pi^\sharp g \mathrm d \mu \\
&= \|f\|_{C(X)} \int_X g \circ \pi \mathrm d \mu \\
&= \|f\|_{C(X)} \int_Y g \mathrm d \nu \quad (\star)\\
&\le \|f\|_{C(X)} \|g\|_{L^1(Y)}.
\end{align}
$$
Here $\pmb 1$ is the constant map $x \mapsto 1$. Also, $(\star)$ follows from change-of-variables formula. This completes the proof.
Update 1: @Anne has pointed out a mistake in above proof, i.e., the duality holds for $g \in L^2(Y)$ but not necessarily for $g \in L^1(Y)$. However, it seems we can obtain the same inequality with a tweak on the definition of $\pi^\sharp$. We define a map
$$
\pi^\sharp: L_1(Y) \to L_1(X), g \mapsto g \circ \pi.
$$
Then $\pi^\sharp$ is a linear operator. Moreover,
$$
\|\pi^\sharp\| = \sup_{g \in L_1(Y)} \frac{\|g \circ \pi\|_{L_1(X)}}{\|g\|_{L_1(Y)}} = \sup_{g \in L_1(Y)} \frac{\int_X |g| \circ \pi \mathrm d \mu}{\int_Y |g| \mathrm d \nu} = \sup_{g \in L_1(Y)} \frac{\int_Y |g| \mathrm d \nu}{\int_Y |g| \mathrm d \nu} = 1.
$$
It follows that $\pi^\sharp$ is bounded. We denote its adjoint by
$$
\pi_\sharp : L_\infty(X) \to L_\infty(Y).
$$
We have the duality
$$
\int_Y (\pi_\sharp f) g \mathrm d \nu = \int_X f (\pi^\sharp g) \mathrm d \mu \quad \forall f \in L_\infty(X), \forall g \in L_1(Y). \quad (\star)
$$
For $f \in C(X)$, we have $f \in L_\infty(X)$, and
$$
\begin{align}
\|\pi_\sharp f\|_{L_\infty(Y)} &= \sup \left \{ \int_Y (\pi_\sharp f) g \mathrm d \nu \,\middle\vert\, \|g\|_{L_1(Y)} = 1 \right\} \\
&= \sup \left \{ \int_X f(\pi^\sharp g) \mathrm d \mu \,\middle\vert\, \|g\|_{L^1(Y)} = 1 \right\} \quad \text{by } (\star)\\
&\le \|f\|_\infty \sup \left \{ \int_X \pi^\sharp g \mathrm d \mu \,\middle\vert\, \|g\|_{L^1(Y)} = 1 \right\} \\
&= \|f\|_\infty \sup \left \{ \int_X g \circ \pi \mathrm d \mu \,\middle\vert\, \|g\|_{L^1(Y)} = 1 \right\} \\
&= \|f\|_\infty \sup \left \{ \int_Y g \mathrm d \nu \,\middle\vert\, \|g\|_{L^1(Y)} = 1 \right\} \quad (\star\star)\\
&\le \|f\|_\infty \sup \left \{ \int_Y |g| \mathrm d \nu \,\middle\vert\, \|g\|_{L^1(Y)} = 1 \right\} \\
&= \|f\|_\infty.
\end{align}
$$
Here $(\star\star)$ follows from change-of-variables formula. This completes the proof.
Update 2: I show here why Tao's function $\pi_\sharp f\in L_2(Y)$ is equal $\nu$-a.e. to my function $\pi'_\sharp f\in L_\infty(Y)$. We have
$$
\int_X f(\pi^\sharp g) \mathrm d \mu = \int_Y\left(\pi_{\sharp} f\right) g \mathrm d \nu \quad \forall g \in L_2 (Y).
$$
$$
\int_X f(\pi^\sharp g) \mathrm d \mu = \int_Y \left(\pi'_{\sharp} f\right) g \mathrm d \nu \quad \forall g \in L_1 (Y).
$$
Because $\nu$ is finite, we get $L_\infty(Y) \subset L_2(Y) \subset L_1(Y)$. So
- $\pi_{\sharp} f - \pi'_{\sharp} f \in L_2(Y)$.
- $$
\int_Y (\pi_{\sharp} f - \pi'_{\sharp} f) g \mathrm d \nu =0 \quad \forall g \in L_2 (Y).
$$
Pick $g:= \pi_{\sharp} f - \pi'_{\sharp} f$, we get
$$
\int_Y (\pi_{\sharp} f - \pi'_{\sharp} f)^2 \mathrm d \nu =0
$$
It follows that $\pi_{\sharp} f - \pi'_{\sharp} f=0$ $\nu$-a.e.
Because $X$ is compact, $C(X)$ is separable. Let $F$ be a countable dense subset of $C(X)$. Let $\mathcal F := \operatorname{span}_{\mathbb Q} (F)$. For each $f \in F$, we have $\pi_\sharp f$ is an equivalence class of $L_\infty (Y)$, so we fix a representative $\tilde \pi_\sharp f \in \mathcal L_\infty (Y)$. For each $y \in Y$, we define a map $L_y:\mathcal F \to \mathbb C$ by
$$
L_y \bigg (\sum_{i=1}^n \lambda_i f_i \bigg) := \sum_{i=1}^n \lambda_i (\tilde \pi_\sharp f_i) (y) \quad \forall (\lambda_i)_{i=1}^n \subset \mathbb Q, \forall (f_i)_{i=1}^n \subset F.
$$
Let $\pmb \lambda = (\lambda_i)_{i=1}^n \subset \mathbb Q$ and $\pmb f=(f_i)_{i=1}^n \subset F$ such that $f := \sum_{i=1}^n \lambda_i f_i \in F$. Because $\pi_\sharp$ is linear,
$$
\tilde\pi_\sharp f = \tilde\pi_\sharp \bigg (\sum_{i=1}^n \lambda_i f_i \bigg) = \sum_{i=1}^n \lambda_i (\tilde\pi_\sharp f_i) \quad \nu \text{-a.e.}
$$
So there is a $\nu$-null set $N_{(\pmb \lambda, \pmb f)} \in \mathcal Y$ such that
$$
\tilde\pi_\sharp f (y) = \sum_{i=1}^n \lambda_i (\tilde\pi_\sharp f_i) (y) \quad \forall y \in Y \setminus N_{(\pmb \lambda, \pmb f)}.
$$
Because the set of such pairs $(\pmb \lambda, \pmb f)$ is countable, there is a $\nu$-null set $N_1 \in \mathcal Y$ such that for each $y\in Y \setminus N_1$, $L_y$ is well-defined and $\mathbb Q$-linear on $\mathcal F$.
Let $\pmb \lambda = (\lambda_i)_{i=1}^n \subset \mathbb Q$, $\pmb f=(f_i)_{i=1}^n \subset F$, and $f:= \sum_{i=1}^n \lambda_i f_i$. Because $\pi_\sharp$ is $\mathbb Q$-linear and $\|\pi_\sharp f\|_{L_\infty(Y)} \le \|f\|_\infty$, we get
$$
\sum_{i=1}^n \lambda_i (\tilde\pi_\sharp f_i) = \tilde\pi_\sharp (f) \le \|f\|_\infty \quad \nu \text{-a.e.}
$$
So there is a $\nu$-null set $N_{(\pmb \lambda, \pmb f)} \in \mathcal Y$ such that
$$
\sum_{i=1}^n \lambda_i (\tilde\pi_\sharp f_i) (y) \le \|f\|_\infty \quad \forall y \in Y \setminus N_{(\pmb \lambda, \pmb f)}.
$$
Because the set of such pairs $(\pmb \lambda, \pmb f)$ is countable, there is a $\nu$-null set $N_2 \in \mathcal Y$ such that for each $y\in Y \setminus N_2$, $L_y (f) \le \|f\|_\infty$ for all $f \in \mathcal F$.
Let $N := N_1 \cup N_2$. Then $N \in \mathcal Y$ is a $\nu$-null set such that for each $y \in N^c := Y \setminus N$, we have
- $L_y$ is well-defined and $\mathbb Q$-linear.
- $L_y$ is bounded, i.e., $L_y (f) \le \|f\|_\infty$ for all $f \in \mathcal F$.
Let $\mathcal G := \operatorname{span}_{\mathbb R} (F)$. Then $\mathcal F$ is dense in $\mathcal G$. So for each $y \in N^c$, we extend $L_y$ to a $\mathbb R$-linear continuous function on $\mathcal G$. In particular, if $(f_n) \subset \mathcal F$ and $f \in \mathcal G$ such that $f_n \to f$, then
$$
L_y (f) := \lim_{n \to \infty} L_y (f_n).
$$
Best Answer
By change-of-variables formula, the bounded linear operator $$ \pi^\sharp: L_1(Y) \to L_1(X), g \mapsto g \circ \pi $$ is well-defined and $\|\pi^\sharp\| = 1$. We denote its adjoint by $$ \pi_\sharp : L_\infty(X) \to L_\infty(Y). $$
We have the duality $$ \int_Y (\pi_\sharp f) g \mathrm d \nu = \int_X f (\pi^\sharp g) \mathrm d \mu \quad \forall f \in L_\infty(X), \forall g \in L_1(Y). $$
Also, $$ \|\pi_\sharp f\|_{L_\infty(Y)} \le \|\pi^\sharp\| \cdot \|f\|_{L_\infty(X)} \le \|f\|_\infty \quad \forall f \in \mathcal C(X). $$
For each $f \in \mathcal C(X)$, we have $\pi_\sharp f$ is an equivalence class of $L_\infty (Y)$, so we fix a representative $\tilde \pi_\sharp f \in \mathcal L_\infty (Y)$. Let $\pmb 1 \in \mathcal C(X)$ be the constant function.
Because $X$ is compact, $\mathcal C(X)$ is separable. Let $F$ be a countable dense subset of $\mathcal C(X)$ and $\mathcal F := \operatorname{span}_{\mathbb Q} (F)$. For each $y \in Y$, we define a map $$ L: \mathcal F \times Y \to \mathbb C $$ by $$ L(f, \cdot) := \tilde\pi_\sharp f \quad \forall f \in \mathcal F. $$
Because $\mathcal F$ is countable, $\pi_\sharp$ is linear, and $\|\pi_\sharp f\|_{L_\infty(Y)} \le \|f\|_\infty$, there is a $\nu$-null set $N \in \mathcal Y$ such that for each $y\in N^c := Y \setminus N$,
We fix some $\overline y \in N^c$ and re-define $L$ by $$ L(\cdot, y) := L(\cdot, \overline y) \quad \forall y \in N. $$
Then for each $y\in Y$,
Because $\mathcal F$ is dense in $\mathcal C(X)$, there is an extension $$ L: \mathcal C(X) \times Y \to \mathbb C $$ such that for each $y \in Y$,
In particular, if $(f_n) \subset F$ and $f \in \mathcal C(X)$ such that $f_n \to f$ in $\|\cdot\|_\infty$, then $$ L(f, \cdot) = \lim_{n \to \infty} L(f_n, \cdot). $$
In other words, the sequence $\{L(f_n, \cdot) \mid n \in \mathbb N\}$ converges pointwise and everywhere to $L(f, \cdot)$. It follows that $L(f, \cdot)$ is measurable.
By Riesz–Markov–Kakutani theorem, for each $y \in Y$, there is a unique regular Borel measure $\mu_y$ on $X$ such that $$ L(f, y) = \int_X f \mathrm d \mu_y \quad \forall f \in \mathcal C(X). $$
It follows from $L(\pmb 1, y) = 1$ that $\mu_y$ is a probability measure. Also, for all $f \in \mathcal C(X)$ and for all $g \in L_1(Y)$, $$ \int_X f (\pi^\sharp g) \mathrm d \mu = \int_Y (\pi_\sharp f) g \mathrm d \nu = \int_Y (\tilde \pi_\sharp f) g \mathrm d \nu = \int_Y L(f, \cdot) g \mathrm d \nu = \int_Y \left ( \int_X f \mathrm d \mu_y \right ) g (y) \mathrm d \nu (y). $$
Let $\tau$ be the metric topology of $X$, the map $g:Y\to \mathbb C$ be $\nu$-integrable map. and $$ \mathcal H_g := \left \{f:X \to \mathbb C \text{ bounded measurable} \,\middle\vert\, y \mapsto\int_X f \mathrm d \mu_y \text{ is measurable, and } (\star) \text{ holds for } f \right\}. $$
This completes the proof.