I'll assume $f$ is convex. (I should probably also assume $f$ is closed and proper.)
As you said, the Lagrangian is $L(x,y) = f(x) + \langle y, Ax - b \rangle$. The dual function is
\begin{align*}
g(y) &= \inf_x \, L(x,y) \\
&= \inf_x \, f(x) - \langle -A^Ty, x \rangle - \langle y, b \rangle \\
&= - \sup_x \, \langle -A^Ty, x \rangle - f(x) + \langle y, b \rangle \\
&= - f^*(-A^T y) - \langle y, b \rangle.
\end{align*}
Here $f^*$ is the convex conjugate of $f$.
The dual problem, expressed as a minimization problem, is
\begin{equation*}
\text{minimize} \quad f^*(-A^T y) + \langle y, b \rangle.
\end{equation*}
Note that the dual function might not be differentiable! That is a key point. To guarantee that $f^*$ is differentiable, we need to assume that $f$ is strictly convex, which is often not the case. This prevents us (often) from simply solving the dual problem with gradient ascent.
So what can we do if the dual function is not differentiable? How can we minimize a nondifferentiable function? One option is to use the proximal point method. The proximal point method does not require the objective function to be differentiable.
If you work out the details, you will find that the augmented Lagrangian method is what you get when you solve the dual problem with the proximal point method.
Vandenberghe's 236c notes are a good source for this material.
Best Answer
The number of constraints does not matter here, as long as there are fewer constraints than variables. If the objective function is strictly concave and the constraints are strictly convex in the variable $\theta$, then there is an easy uniqueness result. (There is also a more complicated result, where you only look at the behavior of these functions along the admissible set, and which involves what is called the bordered Hessian.)
Let $f(\theta)$ be the function you want to maximize, where $\theta$ will have the components $\theta=(\theta_1,\dots,\theta_n)$, and let the constraints be $g_1(\theta)=b_1$ and $g_2(\theta)=b_2$. Introduce the Lagrangian function $$L=L(\theta,\alpha)=f(\theta)-\alpha_1(g_1(\theta)-b_1)-\alpha_2(g_2(\theta)-b_2)$$ The Lagrangian conditions are the same as $${\partial L \over \partial \theta_i}=0,\qquad i=1,\dots,n \\ {\partial L\over \partial \alpha_j}=0, \qquad j=1,2\qquad{}$$ which means that the optimal solution $(\theta^*,\alpha^*)$ should be a stationary point for $L$.
Cases where the gradient vectors $\nabla g_1(\theta)$ and $\nabla g_2(\theta)$ are linearly dependent at some admissible point have to be checked separately, as you are not sure that Lagrange's method will find the answer then.
If $\nabla g_1(\theta)$ and $\nabla g_2(\theta)$ are linearly independent at all admissible points (this is often the case), then the Lagrange conditions above are necessary at an optimal point, and $\alpha^*$ will be uniquely determined.
Now, if $f(\theta)$ is strictly concave and each $g_j(\theta)$ is strictly convex then $L(\cdot,\alpha^*)$ will be strictly concave in $\theta$, which means that it can have at most one stationary point $\theta^*$. Hence the solution, if it exists, is unique.