First of all, I think that by ”determine the constraint”, it actually means determine the forces of constraint for the considered motion. Also, by ”Lagrange equation of the second kind”, it means that you should use the so-called Lagrange’s $\lambda$-method.
Let us consider the schematics below (which differ very little from your initial drawing). I believe it is much easier and perhaps elegant to work with some generalised coordinates, rather than with the usual cartesian ones. After all, one of the beauties of the Lagrangian method lies in this aptitude.
In my version, $r$ denotes the distance between the ground and the particle along the inclined plane/slope. Also, let us presume that for now, we do not know if the particle can or cannot sink into the surface of the wedge. Hence the presence of the $z$ coordinate. These two coordinates are the ones that really describe the particle motion. Then, if the $z$ coordinate can vary, then the particle height above the ground will be $r\sin\theta - z\cos\theta$. Taking this into account, its potential energy would be:
\begin{equation}
V = mg(r\sin\theta - z\cos\theta)
\end{equation}
But, in order to keep the particle on the surface, its $z$ coordinate must be zero. This leads to the holonomic constraint:
\begin{equation}
f(z) = z = 0.
\end{equation}
This constraint being holonomic, it enables a constraint potential $\lambda f(z)$ which can be included into our initial potential energy:
\begin{equation}
V(z) = mg(r\sin\theta - z\cos\theta) + \lambda z.
\end{equation}
The Kinetic Energy of the particle is:
\begin{equation}
T(\dot r, \dot z) = \frac{1}{2}m(\dot r^2 + \dot z^2),
\end{equation}
so the motion Lagrangian would in our case be:
\begin{equation}
L(r,\dot r,z, \dot z) = \frac{1}{2}m(\dot r^2 + \dot z^2) - mg(r\sin\theta - z\cos\theta) - \lambda z.
\end{equation}
This Lagrangian leads to two equations of motion,
\begin{equation}
\frac{d}{dt}\frac{\partial L}{\partial \dot r} - \frac{\partial L}{\partial r}= 0 \Leftrightarrow m\ddot r = mg\sin\theta
\end{equation}
\begin{equation}
\frac{d}{dt}\frac{\partial L}{\partial \dot r} - \frac{\partial L}{\partial r}= 0 \Leftrightarrow m\ddot z + mg\cos\theta = \lambda.
\end{equation}
But, our initial $f(z)$ constraint implies that $\ddot z = 0$. So we end up with the following two equations of motion :
\begin{equation}
\ddot r = g\sin\theta
\end{equation}
\begin{equation}
\lambda = mg\cos\theta
\end{equation}
where in fact, the second equation of motion corresponds to the force of constraint, where $\mathbf{\lambda}$ is the normal force exerted by the wedge surface on the particle; the first equation corresponds to the sliding motion of the particle.
In the end, the lagrange multiplier $\lambda$ is the constraining force of the sliding motion: the one needed to keep the particle on the surface.
Best Answer
Once you've obtained the Lagrangian, I recommend working in polar coordinates rather than Cartesian ones, as is typical when studying pendula because the pendulum length adds a constraint that reduces the phase space dimension. An infinite-cylinder phase space is $2$-dimensional; no surprises there, because the phase coordinates should be the polar angle $\theta$ and its momentum $p_\theta$. The most famous pendulum is in a vertical plane, and its path through phase space is oscillatory in both variables, with the obvious constraint $\theta\in [0,\,2\pi]$ when we work in the usual coordinate system for angles. If you subject the pendulum in your question to a similar analysis, you should see why the phase space is a cylinder (albeit an infinitely long one) instead of, say, a flat plane.