Sure. This is essentially the observation that the Dirichlet distribution is a conjugate prior for the multinomial distribution. This means they have the same functional form. The article mentions it, but I'll just emphasize that this follows from the multinomial sampling model. So, getting down to it...
The observation is about the posterior, so let's introduce some data, $x$, which are counts of $K$ distinct items. We observe $N = \sum_{i=1}^K x_i$ samples total. We'll assume $x$ is drawn from an unknown distribution $\pi$ (on which we'll put a $\mathrm{Dir}(\alpha)$ prior on the $K$-simplex).
The posterior probability of $\pi$ given $\alpha$ and data $x$ is
$$p(\pi | x, \alpha) = p(x | \pi) p(\pi|\alpha)$$
The likelihood, $p(x|\pi)$, is the multinomial distribution. Now let's write out the pdf's:
$$p(x|\pi) = \frac{N!}{x_1!\cdots x_k!} \pi_1^{x_1} \cdots \pi_k^{x_k}$$
and
$$p(\pi|\alpha) = \frac{1}{\mathrm{B}(\alpha)} \prod_{i=1}^K \pi_i^{\alpha - 1}$$
where $\mathrm{B}(\alpha) = \frac{\Gamma(\alpha)^K}{\Gamma(K\alpha)}$. Multiplying, we find that,
$$ p(\pi|\alpha,x) = p(x | \pi) p(\pi|\alpha) \propto \prod_{i=1}^K \pi_i^{x_i + \alpha - 1}.$$
In other words, the posterior is also Dirichlet. The question was about the posterior mean. Since the posterior is Dirichlet, we can apply the formula for the mean of a Dirichlet to find that,
$$E[\pi_i | \alpha, x] = \frac{x_i + \alpha}{N + K\alpha}.$$
Hope this helps!
No, a Gamma(0,0) is not equivalent to the Jeffreys prior of the Poisson and Exponential rates (it is not even well defined). By Gammao(0,0) people usually mean a $Gamma(\epsilon,\epsilon)$ with $\epsilon\approx 0$. Its use became popular since the people from WINBUGS claimed that it "resembles" the shape of the Jeffreys prior for the variance parameters in certain hierarchical models. However, it has many detractors.
See also: Exponential Distribution - Rate - Bayesian Prior?
Although the Gamma prior is decreasing, the tails of this and the Jeffreys priors are different. Moreover, the Jeffreys prior of the Exponential is $1/\lambda$, while in the Poisson case is $\lambda^{-1/2}$, then, the claimed resemblance is not theoretically justified. In practice, however, with large or moderate samples, the influence of the prior is usually negligible. For these simple models, you can simply use the exact expression of the Jeffreys priors since they produce proper posteriors and they are easy to sample from using basic MCMC algorithms.
Best Answer
For simplicity let's just consider a single observation of a variable $Y$ such that $$Y|\mu, \sigma^2 \sim N(\mu, \sigma^2),$$
$\mu \sim \mbox{Laplace}(\lambda)$ and the improper prior $f(\sigma) \propto \mathbb{1}_{\sigma>0}$.
Then the joint density of $Y, \mu, \sigma^2$ is proportional to $$ f(Y, \mu, \sigma^2 | \lambda) \propto \frac{1}{\sigma}\exp \left(-\frac{(y-\mu)^2}{\sigma^2} \right) \times 2\lambda e^{-\lambda \vert \mu \vert}. $$
Taking a log and discarding terms that do not involve $\mu$, $$ \log f(Y, \mu, \sigma^2) = -\frac{1}{\sigma^2} \Vert y-\mu\Vert_2^2 -\lambda \vert \mu \vert. \quad (1)$$
Thus the maximum of (1) will be a MAP estimate and is indeed the Lasso problem after we reparametrize $\tilde \lambda = \lambda \sigma^2$.
The extension to regression is clear--replace $\mu$ with $X\beta$ in the Normal likelihood, and set the prior on $\beta$ to be a sequence of independent laplace$(\lambda)$ distributions.