Let us assume that $W\in\mathbb{R}^{d\times k}$ and $W\geq 0$. Let us also assume that $f(W)$ is a real-valued function such that
$$f(W)\leq 0,\quad\forall W\in\mathbb{R}^{d\times k},\quad W\geq 0.$$
Furthermore, the function $f$ can be convex or not. What are the ways to solve the following optimization problem?
$$ \min_{W \in\mathbb{R}^{d\times k},W \geq 0} f(w) $$
P.S. We know that if $f$ is convex, then it has a unique minimum. However, it is possible that the function $f(W)$ is not convex.
Best Answer
I apologize if I'm misunderstanding the question. If you're asking how to do this numerically, then:
(1) Choose a programming language that you prefer
(2) Create mapping routines from $\rm R^{d \times k}$ to $\rm R^m$ where $m = d \times k$ and vice versa. I.e. if $f(W)$ is nonlinear, there's not a huge benefit in keeping the structure of $W$.
(3) Optimize, e.g. using fmin in Python https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin.html
(4) Drop solutions where $W < 0$. Or if you prefer, add a penalty to your $f(W)$ function when elements of $W$ are $< 0$, e.g. $exp(-w^2)$.
(5) If you must maintain $W>0$ for the entire search, choose one of the functions in https://docs.scipy.org/doc/scipy/reference/optimize.html that allows for bounds.