Calculate determinant of a continuant matrix with variable elements

determinantlinear algebramatricestridiagonal-matrices

One of the problems in Muir's Treatise on determinants in exercise set XXX asks to evaluate the determinant of a continuant matrix
$$
f_n(x,y)=\left|\begin{array}{cccc}
x & 1 & 0 & 0 & \ldots & 0 & 0\\
y-1 & x & 2 & 0 & \ldots & 0 & 0\\
0 & y-2 & x & 3 & \ldots & 0 & 0\\
\vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots\\
0 & 0 & 0 & \ldots & y-n+2 & x & n-1\\
0 & 0 & 0 & \ldots & 0 & y-n+1 & x
\end{array}
\right|_n.
$$
No answer or any hints are given.

I was able to transform this determinant to
\begin{equation*}
(-1)^{n-1} \left|
\begin{array}{cccccc}
\ldots & n-y & y-n & n-y & y-n & n+x-1 \\
\ldots & y-n & n-y & y-n & n+x-3 & n-1 \\
\ldots & n-y & y-n & n+x-5 & n-2 & 0 \\
\ldots & y-n & n+x-7 & n-3 & 0 & 0 \\
\ldots & n+x-9 & n-4 & 0 & 0 & 0 \\
& \vdots & \vdots & \vdots & \vdots & \vdots \\
\end{array}
\right|_n.
\end{equation*}
by a series of lengthy calculations. Of course from this I can calculate the determinant when $y=n$, in which case it becomes diagonal. But not sure if this is the right way to proceed if $y\neq n$ and have no clue what to do next. Does anybody have any ideas?

I also established the following recurrence relation (which should not be confused with the standard recurrence for a determinant of a continuant):
$$
f_n(x,y)=(x+n-1)f_{n-1}(x-1,y-1)+(1-n)(y-n)f_{n-2}(x-1,y-1).
$$
Again, one can see that it can be easily solved when $y=n$. But what to do when $y\neq n$ is not clear.

EDIT: It turns out that when treated as a polynomial in $x$, the the system of functions $f_n(x,y)$ form an orthogonal polynomial system. I found that then $f_n(x,y)$ is related to Meixner polynomials (from Chihara's book on orthogonal polynomials) enter image description here

with $f=d=0$ and $g$ and $h$ suitably chosen. But Muir's book was written in 19-th century, well before the Meixner polynomials were discovered by Meixner in 1934. So this problem in Muir's book is really confusing.

Addendum. The problem exactly how it looks in the 1960 edition of the book:
enter image description here

It is equivalent to the formulation I gave above after a simple eqiuivalence transformation and change of notation.

Best Answer

Thanks to a comment by Jean-Claude Arbaut I finally got the answer from the page 559 of Muir's book (after correcting for some typos) $$ f_n(x,y)=\sum _{k=0}^n (-1)^{n-k} \binom{n}{k} \left(\frac{x+y-1}{2}\right)^{(k)} \left(\frac{-x+y-1}{2}\right)^{(n-k)}, $$ where $(a)^{(n)}=a(a-1)\ldots (a-n+1)$ denotes the falling factorial.

Related Question