I admittedly have a fairly stupid question about a problem, which I wanted to make sure I understand correctly, since I actually never came across this before. Let $f$ be a function, $f: \mathbb{R^2} \rightarrow \mathbb{R}$. What does $$\inf_{x \in \mathbb{R}} {\sup_{y\in \mathbb{R}}}f(x,y)$$ exactly mean? If i want to evaluate the supremum first, do I look at every $x$ but treat it as a constant value or do I just pick one x and evaluate $f(x,y_1),…,f(x,y_n),…$ and then take the smallest upper bound of that? And what does the infimum over that then look like? I am kind of confused about that notation, appreciate any help. Thank you in advance!
Edit: My original problem is the following. Let $g$ be an estimator for $\theta$. The risk is given by $R(\theta,g)$. I am now supposed to evaluate $\inf_{g} \sup_{\theta \in \Theta}R(\theta,g)$, where $\Theta$ is the parameter set and $g$ is an estimator. This is used in Stochastics and Probability theory.
Best Answer
You can think of it in the following way. First, define the function $g\colon \mathbb R \to \mathbb R$ by $$ g(x) = \sup_{y \in \mathbb R} f(x,y). $$ Then, letting $C = \inf_{x\in\mathbb R} \sup_{y\in\mathbb R} f(x,y)$, we have that $$ C = \inf_{x\in\mathbb R} g(x). $$
In other words, you can find $C$ by first finding the supremum over $y$ of $f(x,y)$ for each $x$. This gives you the function $g$ described above. Then, you can find the infimum of $g$ over $x$.
Note that, in general, $$ \sup_{y\in\mathbb R} \inf_{x\in\mathbb R} f(x,y) \le C. $$ This means you should not do the process I described above in reverse, as you may sometimes have strict inequality above. (See Wikipedia.) The conditions where you have equality is the subject of a number of different theorems. See, for example, Sion's Theorem.