Yes, the converse is true, but only for a somewhat ridiculous reason: "finite" means "in bijection with an element of $\omega$".
However, it is possible to construct (assuming $ZFC$ is consistent) a model of $ZFC$ in which $\omega$ has an element that is intuitively infinite; this model would be ill-founded, because the immediate predecessors of that element would form a decreasing $\in$-chain, but the key is that the set of those immediate predecessors is not in the model. So as far as the model is concerned, the Axiom of Regularity is satisfied.
Let me try to answer 3. first. It is not even clear that we can state the result you have in mind. Indeed how do you define “finite iteration“ without the finite ordinals under hand ?
And if you already have the finite ordinals, then the question becomes trivially “yes“ because to construct a given finite ordinal as a finite iteration, just use that ordinal as an indexation. However this does not reflect (or so I think) the spirit of your question, which seems to be asking about ”honest” finite iteration, like $1,2, 3$ “and so on“. However the whole problem lies in this “and so on”, which we cannot make precise without appealing to finite ordinals, and then we enter a loop.
Here's one way to answer negatively. Assuming ZF(C) is consistent, it has a model $M$ such that externally, the set of finite ordinals contains a subset on which the membership relation has the order type of $\mathbb Q$. In particular this means that the “finite ordinals“ of this subset cannot be attained by an ”honest finite iteration“. But then, you might say that this model $M$ is artifical, and not the “real“ one : I would agree, but how do you know that from inside of $M$ ?
So the answer to 3. is essentially that you will not be able to prove that in any nontrivial meaningful way.
But all hope is not lost. You can still perform induction on those finite ordinals, and prove that they all belong to $I$, where $I$ is any inductive set (which will allow you to conclude for 4. So your reasoning can be saved !
If you know about transfinite induction, then it should be clear what to do : prove by induction on all ordinals $\alpha$ the formula “$\alpha\in I$ or $\alpha$ is not finite“ which should not be too complicated.
If you do not know about transfinite induction, it's not a problem either, you only need to know that the class of ordinals is itself well-ordered. Once you have that, you can simply ponder : if $I$ is an inductive set, what is the least ordinal that doesn't belong to it ? (If there is any ! - but if there isn't well certainly all finite ordinals belong to it; although if you keep learning about ordinals you'll see that this situation doesn't happen)
Best Answer
Let me preface this by saying that I am an analyst, and not an expert in set theory or mathematical foundations. The answer that I am about to give is based on an undergraduate class that I took maybe a decade ago, so if anyone with an actual background in foundational set theory wants to correct my errors, I am more than happy to defer to them.
That being said, my feeling about the construction is as follows: the axioms of $\mathsf{ZFC}$ (or any other axiom system) tell us how objects should behave, but it is not obvious that those axioms actually provide enough structure to model the objects that we are interested in. In particular, it is not immediately obvious that $\textsf{ZFC}$ is sufficient to model our intuition about counting. Hence the set theoretic construction of the natural numbers is meant to give a model within the axiom system. Once we know that such a model exists, we can largely get on with our lives.
Depending on precisely how the axioms are stating, $\textsf{ZFC}$ doesn't give us many sets to work with. Indeed, the only set that is required in order to start building things up is the empty set–existence of the empty set can either be taken as an axiom, or follows from the Axiom of Schema Specification. From the empty set, we want to be able to build up the natural numbers.
To do this, we need a starting place (zero), and a way of getting from one set to another (a successor operation). The usual way to do this is as you described, i.e. we define $$ 0 := \varnothing \qquad\text{and}\qquad s(n) := n \cup \{n\}.$$ The definition of zero is, I think, relatively easily understood. The successor operation is a little more opaque. The essential idea is that, given a natural number $n$, we want some kind of set theoretic construction which gives the "next" natural number $s(n)$. To do this, we have to construct a new set out of whole cloth which is distinct from all of the other sets thus far constructed. The way we choose to do this is to consider a set which contains two elements: the set $n$ itself, and the set $\{n\}$, which is a set containing only one element.
The two sets $$ 0 = \varnothing \qquad\text{and}\qquad 1 = \{\varnothing\} $$ look like a whole lot of nothing, but they are actually quite distinct. The set containing the empty set–$\{\varnothing\}$–is an element of 1, but not an element of 0. Perhaps part of the problem here is the distinction between the ideas of "subset" and "is an element of." The empty set is a subset of any set. Hence $$ \varnothing \subseteq 0. $$ On the other hand, the empty set is not an element of 0. That is, $$ \varnothing \not\in \varnothing, \qquad\text{though}\qquad \varnothing \in \{\varnothing\} = 1. $$ The construction of the natural numbers depends on understanding how this nesting works, and noting that a set with no elements is distinct from the set which contains a set which has no elements. Think of $\varnothing$ as an empty grocery bag, and $\{\varnothing\}$ as a grocery bag containing an empty grocery bag. These are two different objects: the bag with a bag in it isn't empty!
In general two sets $n$ and $n \cup \{n\}$ are distinct: we will always have $n \in s(n)$, but $n\notin n$. This also has the advantage of giving us an ordering that matches our intuition: $$ m \le n \iff m \subseteq n \iff m \in n. $$ For example (abbreviating notation so that we don't have to write a whole bunch of empty sets over and over again), $$ 3 = 2 \cup \{ 2 \} = (1 \cup \{ 1 \} ) \cup \{2\} = ((0 \cup \{0\}) \cup \{1\}) \cup \{2\} = \{ 0, 1, 2 \}. $$ Note that $2 \in 3$, and also that $2 = \{0,1\} \subseteq 3$. The advantage of this scheme is that the cardinality of the set $n$ is equal to the number of elements contained in $n$, where "cardinality" has a rigorous mathematical definition, and "the number of elements contained in" matches our intuitive notion.