The theorem count doesn't update when I move to another section. I.e. I'm currently in section 6, and LyX has already in theorem 30, when theorem 1 is in section 1. How can I make LyX start counting the theorems and lemmas from "1" in each new section?
[Tex/LaTex] How to number theorems by section in LyX
lyxnumberingtheorems
Related Solutions
You need to make a "dummy" theorem and make all the others subordinate to that. For example:
\documentclass{article}
\usepackage{amsthm,amsmath}
% \newtheorem{dummy}{Dummy}[section]
\newcounter{dummy} \numberwithin{dummy}{section}
\newtheorem{thm}[dummy]{Theorem}
\newtheorem{defn}[dummy]{Definition}
\begin{document}
\section{First}
\begin{thm}
\end{thm}
\begin{defn}
\end{defn}
\section{Second}
\begin{thm}
\end{thm}
\begin{defn}
\end{defn}
\end{document}
Either the commented \newtheorem line or the \newcounter line works; the \numberwithin command is defined by amsmath, but if you are using amsthm you are probably using that too. If not, then the \newtheorem line works just with amsthm.
To add cross references to a theorem, first add a label right after the theorem "title" (e.g. Theorem 1) with Insert --> Label or the 
button on the toolbar. This places, well, a label at that point which you can refer to at other places in the document. The label refers to the last "active" counter, so when you place it right at the beginning of the theorem, you know you get the right one.
A cross reference is added via Insert --> Cross reference or the 
button on the toolbar (right next to the insert label-button). You will get a list of all the labels in the document, and a choice of what kind of reference you would like. For example, <reference> prints just the number.
To change the type of reference just right click the reference and choose from the menu that pops up.
\hyperref
One use of this command is to have a clickable link that consists of more that just the number. hyperref provides a starred \ref command that prints the relevant number without turning it into a clickable link. For example:
\hyperref[thm:ATheorem]{Theorem \ref*{thm:ATheorem}}
Summary of sorts
In the screenshot below I've tried to "sum up" some of this (click it to see larger). To point out the obvious, LyX on the left, PDF output on the right.
Best Answer
If you add the Theorems (Numbered by Section) module your theorems will be numbered 1.x for the first section, 2.x for the second, etc.
If you for some reason don't want the section number in the theorem number (which you should, if you're cross referencing theorems), you can add to the preamble (Document --> Settings --> LaTeX preamble) the following.
(Assuming you're using the Theorems (AMS) module, I didn't check if other theorem modules have other names for that counter.)
Note that the LyX view will not have the correct numbers with this last method, but the PDF will be correct.
Reference: Resetting theorem counters: \section and (missing) \subsection