The epsilon delta definition of limits says that if the limit as $x\to a$ of $f(x)$ is L, then for any $\delta>0$, there is an $\epsilon>0$ such that if $0<|x-a|<\delta$, then $|f(x)-L|<\epsilon$.
But the problem is that this definition says very generally that for ANY $\delta$, there is SOME $\epsilon$. So what if I always choose $\epsilon=\infty$? Then it is guaranteed that the distance between $f(x)$ and $L$ is less than $\epsilon$, and, as a bonus, $L$ can literally be anything, which means that the limit can be any value you like. Which is obviously absurd. What am I missing here?
Also, most people say that this definition intuitively tells us that $f(x)$ can be as close to $L$ as you like, because if $\delta$ gets smaller and smaller and approaches zero, then epsilon gets smaller and smaller and approaches zero as well. But this can't be right, as $\epsilon$ is not a function of $\delta$ or something, so you can't say that if one approaches 0, then the other will as well.
Edit: I feel like the problem has to do with the fact that usually when people use this definition to solve limit problems, then they obtain some expression for epsilon as a function of delta (like I write about above), and using this expression, you usually find that as delta goes to zero, then epsilon goes to zero as well. If it was assumed in the definition itself that this should ALWAYS be the case, then the definition would make total sense to me, but it doesn't seem like it does to me. If someone could share some thoughts on this, then I would be very happy.
Best Answer
You seem to have the definition backwards in your first sentence.
$$\forall \epsilon > 0 \; \exists \delta > 0 \; ...$$
In English: for all $\epsilon > 0$ there exists $\delta > 0$ ...
An intuitive way to think about it is a game. If I am claiming the limit then you can challenge me with any accuracy you want, a positive $\epsilon$, and I need to be able to respond with a positive $\delta$ that achieves it. $\epsilon$ and $\delta$ need to be numbers so $\infty$ is implicitly excluded. Anyway, even we allowed $\infty$ with obvious naive rules and you challenged me to get within $\epsilon = \infty$ of my claimed limit then it would be easy for me to achieve. It wouldn't change things.
Limits are an area where you see the symbol $\infty$ frequently and it is easy to get the impression that it is being treated as a number. It isn't, it is just a suggestive notation for a separate definition. The definitions of limits when $x \rightarrow \infty$ is different from $x \rightarrow a$.
Some extra based on comments, note that although I must be able to supply a suitable $\delta$ for any $\epsilon$ that you give me, it does not in any sense have to be the best or optimal one. Suppose that I am claiming that $x^2 \rightarrow 0$ as $x \rightarrow 0$. In a sense, the best $\delta$ is $\sqrt \epsilon$ which only just does the job but I could just reply $1$ if your $\epsilon$ is $> 1$ and give you your own $\delta$ back if it is $< 1$. This would be more than good enough but that is okay.
Some more based on edited question. Again, it is backwards: $\delta$ is a function of $\epsilon$ not the reverse. $\epsilon$ is the desired accuracy and $\delta$ how close you get need to get to achieve that.
Yes, in general, as $\epsilon$ get smaller, so will $\delta$. This seems quite intuitive to me: in my game, as you challenge to get closer to my claimed limit, I need to go closer to the limit point.
It is not always true but the exceptions are not interesting. Consider the function $f(x) = 1$, a constant function. I claim that $f(x) \rightarrow 1$ as $x \rightarrow 0$. Now for whatever $\epsilon$ you give me, I can just reply $1$ or googleplex if that amused me.