There is a point of confusion in the question:
For instance, I've read that we can define addition in second-order arithmetic by writing
- $x+0 = x$
- $x+S(y) = S(x+y).$
Why does this work in second order arithmetic, but not in first-order?
That does not work in second-order arithmetic. It is an implicit characterization of the addition function, but it is not an explicit definition of the addition function in terms of the successor function.
A genuine definition is a formula $\phi(n,m,p)$ in a language without the addition symbol $+$ such that, for all natural numbers $n,m,p$, we have $n + m = p$ if and only if $\phi(n,m,p)$ holds. A "pseudo-definition" that is able to refer to the object being defined is called an implicit definition, but implicit definability is much weaker than actual definability.
One actual definition of addition of natural numbers in second-order arithmetic is:
$$
n + m = p \Leftrightarrow (\forall f)\left[ \left( f(0) = n \land (\forall k)[f(S(k)) = S(f(k)]\right ) \to f(m) = p\right].
$$
Here $n,m,p$ are natural numbers, $(\forall k)$ quantifies over the natural numbers, and $(\forall f)$ quantifies over all unary functions from the natural numbers to themselves. Notice that, crucially, the right side does not mention $+$. In the particular definition, we could also rewrite it with an existential function quantifier:
$$
n + m = p \Leftrightarrow (\exists f)\left( f(0) = n \land (\forall k)[f(S(k)) = S(f(k))] \land f(m) = p\right.)
$$
Why does this definition not work in first-order logic? Because, in a single-sorted first-order theory of arithmetic, it is not possible to quantify over functions in the way that the definition does.
Now, that does not prove that it is impossible to define addition of natural numbers in terms of successor. It only shows that the definition in second-order arithmetic does not go through unchanged.
One way to see that addition is not definable from successor is sketched in this answer by Alex Kruckman. The key point is that if we look at the first-order theory of the natural numbers with successor and a constant for 0, every formula in this language (with some free variables) is equivalent to a quantifier-free formula in the language (with the same free variables). A proof of that is given by Richard Kaye here. So if addition was definable in that structure, it would be definable by a quantifier-free formula. But by analyzing the form of such a formula we can show that it cannot define addition.
Actually, more is known. Neither addition nor multiplication is definable from successor alone; multiplication is not definable from successor and addition; and addition is not definable from successor and multiplication. The theory of the natural numbers with multiplication and addition is undecidable, but the restriction to just addition is decidable, and the restriction with just multiplication is decidable.
The informal statement of induction is:
For any property $P$ of natural numbers, if $P(0)$ holds, and $P(n)$ implies $P(n+1)$ for all $n$, then $P(n)$ holds for all $n$.
Of course, this raises the question: What exactly do we mean by a "property of natural numbers"?
One natural interpretation is to identify properties of natural numbers with sets of natural numbers. That is, for any property $P$, we can form the set of all natural numbers satisfying that property. And for any set of natural numbers $X$, we can consider the property of being in $X$. For example, the property of being a prime number corresponds to the set $\{n\in \mathbb{N}\mid n\text{ is prime}\}$
Another natural interpretation is to identify properties of natural numbers with formulas in one free variable in some logic (in this discussion, let's just talk about first-order logic in the language of arithemetic). Here the syntax of the logic gives us a language for writing down properties of natural numbers. For example, the property of being a prime number corresponds to the formula $\lnot (x= 1)\land \forall y\, (\exists z\, (y\cdot z = x) \rightarrow (y = 1 \lor y = x))$.
Induction under the interpretation "properties are sets" can be formalized as follows:
$\forall P\subseteq \mathbb{N}: ((0\in P\land \forall n\in \mathbb{N}: (n\in P \rightarrow (n+1)\in P))\rightarrow \forall n\in \mathbb{N}: n\in P)$
This is a sentence of second-order logic, since it involves a quantification $\forall P\subseteq \mathbb{N}$ over subsets of $\mathbb{N}$.
The interpretation "properties are formulas" leads to the following formalization of induction:
$(\varphi(0)\land \forall n\, (\varphi(n)\rightarrow \varphi(n+1)) \rightarrow \forall n\,\varphi(n)$
Here we have an infinite schema of sentences of first-order logic, one for each first-order formula $\varphi(x)$. It's first-order because the quantifiers only range over elements of $\mathbb{N}$, not subsets, and the formulas $\varphi(x)$ are themselves first-order.
It's worth noting that second-order induction is much stronger than first-order induction. Second-order induction applies to all subsets, while first-order induction only applies to those which can be defined by some first-order formula (and since there are are $2^{\aleph_0}$-many subsets of $\mathbb{N}$ and only $\aleph_0$-many first-order formulas, there are many subsets which are not definable).
The second-order Peano axioms (which consist of some basic rules of arithmetic, together with the second-order induction axiom above) suffice to pin down $\mathbb{N}$ up to isomorphism.
The first-order Peano axioms (which consist of some basic rules of arithmetic, together with the first-order induction axiom schema above) cannot hope to pin down $\mathbb{N}$ up to isomorphism (thanks to the Löwenheim-Skolem theorems). That is, there are "non-standard models" of the first-order Peano axioms, which satisfy induction for all first-order definable properties, but not for arbitrary subsets.
Best Answer
Yes, your understanding is basically right.
In more detail:
The term "Peano arithmetic" is used variously by different communities to refer to either first or second order Peano arithmetic (I'll denote the latter by "$\mathsf{PA}_1$" and "$\mathsf{PA}_2$" respectively). Within the modern mathematical logic community, "Peano arithmetic" almost exclusively refers to $\mathsf{PA}_1$; however, in older texts and among philosophers and historians of mathematics, it often refers to $\mathsf{PA}_2$.
The theory $\mathsf{PA}_1$ is usually formulated in the language $\{0,1,+,\times,<\}$ or something similar; roughly speaking, the only crucial point is that we have both addition and multiplication (arithmetic with only addition, for example, is quite a different matter). The axioms of $\mathsf{PA}_1$ consist of:
the discrete positive ordered semiring axioms (often denoted "$\mathsf{P}^-$"); and
the full first-order induction scheme.
Weakening the induction scheme to apply to only certain first-order formulas results in subtheories of $\mathsf{PA}_1$; e.g. $\mathsf{I\Sigma_{17}}$ is the fragment of $\mathsf{PA_1}$ consisting of $\mathsf{P}^-$ together with induciton for $\Sigma_{17}$ formulas.
Meanwhile, $\mathsf{PA}_2$ is usually formulated in the much smaller language $\{0,\mathsf{succ}\}$ with a very sparse set of axioms ... followed by the full second-order induction axiom. Again, there is some flexibility in the details but the power of the underlying logic is such that it ultimately doesn't matter.
(As an aside, if we build a direct analogue of $\mathsf{PA_1}$ with second-order logic replacing first-order logic in the setup of the induction scheme we get something appropriately equivalent to $\mathsf{PA}_2$; see here.)
"Second-order arithmetic" meanwhile refers to the two-sorted, first-order theory $\mathsf{Z_2}$. The two sorts are generally called "numbers" and "sets," and the axiom of extensionality (which is part of $\mathsf{Z_2}$) ensures that we can WLOG restrict attention to models whose sets sort literally consists of subsets of the numbers sort and whose elementhood relation is actual elementhood. The language of $\mathsf{Z_2}$ consists of the language of $\mathsf{PA}_1$ on the numbers sort, and the new binary relation "$\in$" connecting the numbers and sets sorts. Its axioms consist of:
$\mathsf{P}^-$ for the numbers sort,
extensionality for the sets sort, and
the full induction and comprehension schemes.
Whereas the interesting subtheories of $\mathsf{PA}_1$ essentially arise from restricting the induction scheme, $\mathsf{Z_2}$ is "two-dimensional:" we get interseting subtheories by restricting either the induction or comprehension schemes.
For further reading about $\mathsf{PA}_1$ and $\mathsf{Z_2}$, and their interesting subtheories specifically, I strongly recommend Hajek/Pudlak and Simpson respectively. See also my recommendations here and here.