I hope this question is suitable; this problem always bugs me. It is an issue of mathematical orthography.
It is good praxis, recommended in various essays on mathematical writing, to capitalize theorem names when recalling them: for instance one may write "thanks to Theorem 2.4" or "using ii) from Lemma 1.2.1" and so on.
Should these names be capitalized when they appear unnumbered? For instance which of the following is correct?
"Using the previous Lemma we deduce…" versus "Using the previous lemma we deduce…"
"The proof of Lemma 1.3 is postponed to next Section." versus "The proof of Lemma 1.3 is postponed to next section."
Best Answer
In English, proper nouns are capitalized. The numbered instances you mention are all usages as proper nouns, but merely refering to a lemma or corollary not by its name is not using a proper noun, and so is uncapitalized.
Thus, for example, one should write about the lemma before Theorem 1.2 having a proof similar to Lemma 5, while the main corollary of Section 2 does not.
Edit. Well, I've become conflicted. The Chicago Manual of Style, which I have always taken as my guide in such matters, asserts in item 7.136 that "the word chapter is lowercased and spelled out in text". And in 7.141 they favor act 3 and scene 5 in words denoting parts of poems and plays. This would seem to speak against Section 2 and possibly against Theorem 1.2. In 7.135 they say that common titles such as foreward, preface, introduction, contents, etc. are lowercased, as in "Allan Nevins wrote the foreward to...". This may also be evidence against Theorem 1.2. But in 7.147 they favor Piano Sonata no. 2, which may be evidence in favor of Theorem 1.2. But they don't treat mathematical writing explicitly, and now I am less sure of what I have always believed, above. I do note that the CSM text itself refers to "fig. 1.2" and "figure 9.3", and not Figure 1.2, which would clearly speak against Theorem 1.2. So I am afraid that I may have to change my mind about this.