Solved – How to prove such a kernel is positive semi definite? K(x, y) = min(x, y) – xy over [0, 1]

kernel trickmachine learningmathematical-statistics

For such a kernel:

K(x, y) = min(x, y) - xy over [0, 1] X [0, 1]

How can I prove that it's positive semi definite? I know how to prove min(x, y) is PSD but I think -xy is NSD, so can't use the closure property here. Is there a good approach?

Best Answer

To show that $K(x, y)$ is semi-positive definite, it is equivalent to show that for any $(x, y) \in [0, 1]^2$, the Gram matrix \begin{align} M(x,y) = \begin{bmatrix} \min(x, x) - x^2 & \min(x, y) - xy \\ \min(x, y) - xy & \min(y, y) - y^2 \end{bmatrix} = \begin{bmatrix} x - x^2 & \min(x, y) - xy \\ \min(x, y) - xy & y - y^2 \end{bmatrix} \end{align} is a semi-positive definite matrix. Since it is trivial that $x - x^2 \geq 0$ when $x \in [0, 1]$, it is sufficient to verify that the determinant of $M(x, y)$ is non-negative.

For simplicity, denote $\min(x, y)$ by $z$. Straightforward calculation shows that \begin{align} \det(M(x, y)) & = (x - x^2)(y - y^2) - (z - xy)^2 = xy - xy^2 - x^2y -z^2 +2zxy \\ & = \begin{cases} xy - xy^2 + x^2y - x^2 = x(1 - y)(y - x) & \text{ if } x \leq y \\ xy + xy^2 - x^2y - y^2 = y(1 - x)(x - y) & \text{ if } x > y \end{cases} \\ & \geq 0. \end{align}

This completes the proof.