What you seem to be missing is the early history. You can check the paper by Fienberg (2006) When Did Bayesian Inference Become "Bayesian"?. First, he notices that Thomas Bayes was the first one who suggested using a uniform prior:
In current statistical language, Bayes' paper introduces a uniform
prior distribution on the binomial parameter, $\theta$, reasoning by
analogy with a "billiard table" and drawing on the form of the
marginal distribution of the binomial random variable, and not on the
principle of "insufficient reason," as many others have claimed.
Pierre Simon Laplace was the next person to discuss it:
Laplace also articulated, more clearly than Bayes, his argument for
the choice of a uniform prior distribution, arguing that the posterior
distribution of the parameter $\theta$ should be proportional to what
we now call the likelihood of the data, i.e.,
$$ f(\theta\mid x_1,x_2,\dots,x_n) \propto f(x_1,x_2,\dots,x_n\mid\theta) $$
We now understand that this implies that the prior distribution for
$\theta$ is uniform, although in general, of course, the prior may not
exist.
Moreover Carl Friedrich Gauss also referred to using an uninformative prior, as noted by David and Edwards (2001) in their book Annotated Readings in the History of Statistics:
Gauss uses an ad hoc Bayesian-type argument to show that the posterior
density of $h$ is proportional to the likelihood (in modern
terminology):
$$ f(h|x) \propto f(x|h) $$
where he has assumed $h$ to be uniformly distributed over $[0,
\infty)$. Gauss mentions neither Bayes nor Laplace, although the
latter had popularized this approach since Laplace (1774).
and as Fienberg (2006) notices, "inverse probability" (and what follows, using uniform priors) was popular at the turn of the 19th century
[...] Thus, in retrospect, it shouldn't be surprising to see inverse
probability as the method of choice of the great English statisticians
of the turn of the century, such as Edgeworth and Pearson. For
example, Edgeworth (49) gave one of the earliest derivations of what
we now know as Student's $t$-distribution, the posterior distribution
of the mean $\mu$ of a normal distribution given uniform prior
distributions on $\mu$ and $h =\sigma^{-1}$ [...]
The early history of the Bayesian approach is also reviewed by Stigler (1986) in his book The history of statistics: The measurement of uncertainty before 1900.
In your short review you also do not seem to mention Ronald Aylmer Fisher (again quoted after Fienberg, 2006):
Fisher moved away from the inverse methods and towards his own
approach to inference he called the "likelihood," a concept he claimed
was distinct from probability. But Fisher's progression in this regard
was slow. Stigler (164) has pointed out that, in an unpublished
manuscript dating from 1916, Fisher didn't distinguish between
likelihood and inverse probability with a flat prior, even though when
he later made the distinction he claimed to have understood it at this
time.
Jaynes (1986) provided his own short review paper Bayesian Methods: General Background. An Introductory Tutorial that you could check, but it does not focus on uninformative priors. Moreover, as noted by AdamO, you should definitely read The Epic Story of Maximum Likelihood by Stigler (2007).
It is also worth mentioning that there is no such thing as an "uninformative prior", so many authors prefer talking about "vague priors", or "weekly informative priors".
A theoretical review is provided by Kass and Wasserman (1996) in The selection of prior distributions by formal rules, who go into greater detail about choosing priors, with extended discussion of usage of uninformative priors.
When transforming a uniform distribution on $\log(\sigma)$ to a distribution on $\sigma$ you need to take into account the Jacobian of the transformation. This corresponds, as you correctly intuited, to $1/\sigma$.
Writing this a little more clearly, let $X=\log(\sigma)$ and the transformation we're after is $T(X)=\sigma=e^{X}=Y$, which has inverse transformation $T^{-1}(Y)=\log(Y)$. The jacobian is then $|\frac{\partial X}{\partial Y}|=1/Y$.
So since $p(X)\propto 1$, we have that the induced density for $\sigma$ is the $p(Y)=|\frac{\partial X}{\partial Y}|p(\log(Y))\propto1/Y$.
Best Answer
First of all, $p(\sigma^2) \propto 1/\sigma^2$ is the Jeffreys prior (http://en.wikipedia.org/wiki/Jeffreys_prior) for a scale parameter. It also coincides with the reference prior under certain conditions.
1) and 2): Intuitively, it can be understood as the only prior expressing correclty that $\sigma$ is a scale parameter: let $X$ distributed according to $1/\sigma .f(x/\sigma)$ then $Y=c.X$ has the same distribution than $X$ but with a different scale, $p(\sigma^2) \propto 1/\sigma^2$ also fits this property.
3): Jeffreys priors are generally improper. However, to my knowledge improperness is not related to informativeness (but simply to integrability considerations).