This is Exercise III.2.20 of Bourbaki's Set Theory.
(Von Neumann ordinals are actually called "pseudo-ordinals" by Bourbaki, but I simply call them ordinals here)
Let $X$ be a transitive set, and suppose that each $x\in X$ is an ordinal. Then $X$ is an ordinal (Hint: for each $x\in X$, $x\cup\{x\}$ is an ordinal contained in $X$).
This statement is proven in many textbooks. The problem is that none of them uses Bourbaki's definition of ordinal:
For Bourbaki, a set $X$ is an ordinal if every proper transitive subset of $X$ is an element of $X$. [Edit: I'm not sure if this implies e.g. that $X$ is well-ordered by $\in$, since Bourbaki has no Axiom of Foundation.]
A proof that this implies one of the usual definitions (e.g. $X$ is a transitive set whose members are transitive) would be enough, too. [Sorry!]
This should be easy, but I don't know where to start. I'm glad for any help.
Second Edit: The statement I want to show in a hopefully clearer form:
Let $X$ be a transitive set such that any $x\in X$ has the property that any proper transitive subset of $x$ is an element of $x$. Then $X$ has the same property.
Best Answer
Remark: I see in your edit that Bourbaki doesn't have an axiom of foundation. This doesn't affects my proof in any substantial way, since my contradiction is of the form $B\in B$. Thus a slight modification by letting $B=\bigcup\{A\subset X : A \textrm{ transitive and well-founded}\}$ in the first part and analogously letting $B=\bigcup\{A\subset X\setminus\{a\} : A \text{ transitive and well-founded}\}$ in the second part is enough to show that the ordinals as defined by Bourbaki are the standard well-founded ordinals. I also added another way to show this in my edit that again doesn't use the axiom of foundation.
You want to show that if every transitive proper subset of $X$ is an element of $X$ then $X$ is a transitive set whose elements are transitive:
If there exists an element $a\in X$ such that $a\nsubseteq X$ then take the set $$B=\bigcup\{A\subset X : A \text{ transitive and well-founded}\}$$ This set is transitive and well-founded as the union of transitive well-founded sets and we have that $a\notin B$ since otherwise we would have $a\subset B\subset X$. Thus $B$ is a proper subset of $X$ and so $B\in X$. We also have that $B\neq a$ since $B\subset X$ while $a\nsubseteq X$. Now the set $B\cup\{B\}$ is a transitive subset of $X$, which means that $B\cup\{B\}\subset B$ (by the definition of $B$). Thus $B\in B$, which is impossible since $B$ is well founded. We arrived at a contradiction because we assumed that $a\nsubseteq X$, therefore $a\subset X$.
A similar argument shows that $a\in X$ means that $a$ is transitive: In this case take the set $$B=\bigcup\{A\subset X\setminus\{a\} : A \text{ transitive and well-founded}\}$$ If $a$ is not transitive then $B\neq a$ since $a$ is not transitive while $B$ is. We have that $B\in X$ and therefore $B\cup\{B\}\subset X\setminus\{a\}$. Again this gives us $B\in B$ which is a contradiction.
Edit: The converse it easy. Given an ordinal $\alpha$ let $X\subsetneq\alpha$ be transitive. Then since every element of $\alpha$ is an ordinal and thus transitive, we have that $X$ is a transitive set whose elements are transitive sets, i.e. an ordinal. Since it's a proper subset of $\alpha$ it cannot be $\alpha$ nor can it be longer than $\alpha$ since then we would have $\alpha\in X$ which would give us $\alpha\in\alpha$ a contradiction. Thus $X\in\alpha$. Therefore the two definitions are equivalent. This is enough (since you know that a transitive set of ordinals is an ordinal) to show exactly what you want.
Also here's a quicker way to show the first part of my answer namely that the property implies the fact that $X$ is an ordinal: Let $\alpha$ be the greatest ordinal such that $\alpha\subset X$. Observe that this ordinal indeed exists, since if there is an increasing sequence of ordinals such that each of them is a subset of $X$ then their limit will be a subset of $X$ (because a limit ordinal is the union of a sequence that approaches it and the union of subsets of a set is a subset of that set). Now it's easy to see that $X=\alpha$. If $\alpha\subsetneq X$ then $\alpha\in X$ and thus $\alpha\cup\{\alpha\}\subset X$, which is a contradiction by the definition of $\alpha$.
P.S.: Sorry for the delayed edit but I was out of town for the weekend.