If I understand you correctly,
then from the assumptions that $\mu$ and $\lambda$ are $\sigma$-finite
and that $\mu_\psi$ is absolutely continuous with respect to $\lambda$
you want to derive that $\mu_\psi$ is also $\sigma$-finite.
But I believe that the following presents a counter-example.
Let $X = Y = \mathbb{R}$,
let $\mu$ and $\lambda$ both be the Lebesgue measure,
and let $\psi:X\to Y$ be defined by
\begin{equation*}
\psi(x) = x - [x]
\end{equation*}
for every $x\in\mathbb{R}$,
where $[x]$ is $x$ rounded down.
So $\psi$ can be thought of as a periodic "saw-shaped" function
with "teeth" that range from $0$ to $1$.
A set $B$ in $Y$ is disjoint from $[0,1)$
if and only if $\psi^{-1}(B)$ is empty.
So we can restrict attention to subsets $B$ of $[0,1)$.
Let $B$ be such a set.
For any $x$ in $B$ and any integer $n$,
\begin{equation*}
\psi(n + x) = n + x - [n + x] = n + x - n = x,
\end{equation*}
so
\begin{equation*}
\psi^{-1}(B)
= \{n + x:x\in B, n\in\mathbb{Z}\}
=\bigcup_{n\in\mathbb{Z}}\{n + x:x\in B\}
.
\end{equation*}
It follows that
\begin{equation*}
\mu_\psi(B)
=
\mu(\psi^{-1}(B))
=
\sum_{n\in\mathbb{Z}}\mu(\{n + x:x\in B\})
=
\sum_{n\in\mathbb{Z}}\mu(B)
\end{equation*}
where in the last step I used that the Lebesgue measure $\mu$
is translation invariant.
If $\lambda(B)$ equals $0$ then so does $\mu(B)$,
because both are the Lebesgue measure,
and then $\mu_\psi(B) = 0$,
and so $\mu_\psi$ is absolutely continuous with respect to $\lambda$.
But if $\lambda(B)$ does not equal $0$,
then $\mu_\psi(B)$ is infinite.
So $\mu_\psi$ cannot be $\sigma$-finite.
EDIT:
Actually,
for $\mu_\psi$ to have a Radon-Nikodym derivative with respect to $\lambda$,
which I think is what you are actually after,
I don't think you need $\mu_\psi$ to be $\sigma$-finite.
I will just quote the Radon-Nikodym theorem and Exercise 6 from chapter 4 in Donald Cohn's Measure Theory:
"Theorem 4.2.2 (Radon-Nikodym theorem)
Let $(X,\mathcal{A})$ be a measurable space,
and let $\mu$ and $\nu$ be $\sigma$-finite positive measures on $(X,\mathcal{A})$.
If $\nu$ is absolutely continuous with respect to $\mu$,
then there is an $\mathcal{A}$-measurable function $g:X\to[0,\infty)$
such that $\nu(A)=\int_A gd\mu$ holds for each $A$ in $\mathcal{A}$.
The function $g$ is unique up to $\mu$-almost everywhere equality."
"Exercise 6. Show that the assumption that
$\nu$ is $\sigma$-finite can be removed from Theorem 4.2.2
if $g$ is allowed to have values in $[0,\infty]$.
(Hint: Reduce the general case to the case where $\mu$ is finite.
For each positive integer $n$ choose a Hahn decomposition $(P_n,N_n)$
for the signed measure $\nu - n\mu$;
then consider the measures
$A\mapsto \nu(A\cap(\cap_n P_n))$
and
$A\mapsto \nu(A\cap(\cap_n P_n)^c)$.)"
Best Answer
I will assume $\mu$ to also be a non-negative measure (the case for a signed, or a complex measure can be easily adapted). Now by the Radon Nikodym Theorem (for the existence of the Radon Nikodym derivative it suffices to only assume that $\nu$ is $\sigma$-finite, see: https://math.stackexchange.com/a/3342557/485385) we may find $f\geq 0$ measurable such that $$\mu(A)= \int_A f d\nu$$ for all $A$, so with some fancy notation this becomes $\mu = f \odot \nu$. Now define the sequence of measurable functions $\{f_n\}$ by $f_n=\chi_{\{f \in [n, n+1)\}} f$ and consider the measures $\mu_n = f_n \odot \nu$. Can you deduce the rest?